JP2013009262A - Image decoding apparatus, image decoding method and image decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decode encoded data encoded by using a low rate encoding method capable of generating a signal having image quality subjectively close to an original signal.SOLUTION: An image decoding apparatus includes: waveform decode processing means for generating and outputting decoded image data; means for restoring and outputting restored image data having the same resolution as encoding object image data; means for generating and outputting histogram information composed of a histogram of a pixel value with respect to the restored image data and a histogram of a transfer coefficient; means for generating and outputting difference information between histogram information with respect to the encoding object image data and histogram information with respect to the restored image data; means for generating and outputting the histogram information with respect to the encoding object image data; and means for generating and outputting the decoded image data. The image decoding apparatus replaces a low frequency component of composite image data with a low frequency component of the restored image data and sets image data obtained by performing inverse Fourier transformation of the frequency component after replacement as a texture composite processing object.

Description

  The present invention relates to an image decoding device, an image decoding method, and an image decoding program.

  AVC / H. In the current encoding method represented by H.264, a prediction signal is generated by intra-frame prediction, inter-frame prediction, and inter-layer prediction, and a residual signal between the prediction signal and the original signal is to be encoded. Such a conventional encoding method aims to accurately reproduce the original signal waveform.

  However, in such a conventional encoding method, the code amount is controlled by quantization in the framework of waveform reproduction, and therefore, image quality deterioration such as block distortion and ringing becomes obvious at a low rate. Further, since the prediction process refers to the decoded signal, the degraded decoded signal is referred to at a low rate, and therefore it is impossible to predict a certain high-frequency component above a certain level.

  For this reason, the prediction performance deteriorates for an image containing many high-frequency components. For example, image quality degradation at such a low rate is remarkable in scenes such as water surfaces where a rigid body movement model is not established in the case of inter-frame prediction, and in the case of intra-frame prediction and inter-layer prediction, it is prominent in the texture area of fine patterns Become.

  On the other hand, a technique called texture synthesis is being studied. This is a technique for synthesizing an image based on a predetermined generation rule using given data (this is called a seed). The texture synthesis is different from the conventional encoding method in that it does not pursue accurate waveform reproduction of the original signal but uses the texture feature quantity in the original signal as a reproduction target. As a texture synthesis method, for example, there is a method of Non-Patent Document 1.

  In this method, the texture synthesis processing uses a histogram of coefficients output from the directional filter bank as a texture feature amount. In general, the data amount of the seed used for texture synthesis can be suppressed to be extremely small compared to the data amount of the original signal. Therefore, application of texture synthesis to image coding is expected.

D. Heeger and J. Bergeny. Pyramid-based texture analysis / synthesis. In Proc ACM SIGGRAPH 95, 1995.

  On the other hand, the similarity between the signal generated by texture synthesis and the original signal is not guaranteed from the viewpoint of waveform reproduction. This is because there is no guarantee that the phase information in the image is retained. For this reason, when it is required to obtain a signal with an image quality that is subjectively close to the original signal on the premise of comparison with the original image, there is a problem that even if it is simply applied, the request cannot be met. .

  The present invention has been made in view of such circumstances, and as an alternative to the conventional waveform reproduction-based encoding method, a signal having an image quality that is subjectively close to the original signal can be generated based on the texture synthesis technique. An object of the present invention is to provide an image decoding apparatus, an image decoding method, and an image decoding program for decoding encoded data encoded using a low-rate encoding method that can be performed.

  The present invention performs a decoding process on the input encoded data by the waveform encoding process to generate and output decoded image data, and performs reverse resolution conversion on the decoded image data. A reverse resolution conversion processing means for restoring and outputting restored image data having the same resolution as the encoding target image data, and a histogram of pixel values and transform coefficients for the restored image data by texture analysis on the restored image data Generating and outputting histogram information composed of histograms of the image, and decoding processing on the encoded data based on the input texture analysis, and performing histogram processing on the encoding target image data and the restored image data Histogram difference that generates and outputs difference information between histogram information Information decoding means, histogram information addition processing means for generating and outputting histogram information for the encoding target image data by adding the difference information and the histogram information for the restored image data, and the encoding target Texture synthesis processing means for generating and outputting decoded image data by inputting the histogram information with respect to the signal and performing texture synthesis processing, and converting the low frequency component of the synthesized image data into the low frequency component of the restored image data And the image data obtained by performing inverse Fourier transform on the frequency component after replacement is the target of the texture synthesis process.

  The present invention relates to image decoding in an image decoding apparatus comprising waveform decoding processing means, inverse resolution conversion processing means, texture analysis processing means, histogram difference information decoding means, histogram information addition processing means, and texture synthesis processing means. A waveform decoding processing step in which the waveform decoding processing means performs a decoding process on the input encoded data by the waveform encoding process to generate and output decoded image data; and the inverse resolution conversion A processing means for performing reverse resolution conversion on the decoded image data, thereby restoring and outputting restored image data having the same resolution as the encoding target image data; and the texture analysis processing means However, a histogram of pixel values for the restored image data is obtained by texture analysis for the restored image data. And a texture analysis processing step for generating and outputting histogram information composed of histograms of transform coefficients, and the histogram difference information decoding means performs decoding processing on the encoded data based on the input texture analysis to generate an encoding target Histogram difference information decoding step for generating and outputting difference information between histogram information for image data and histogram information for the restored image data, and the histogram information addition processing means, the histogram for the difference information and the restored image data A histogram information addition processing step for generating and outputting histogram information for the encoding target image data, and the texture synthesis processing means for inputting the histogram information for the encoding target signal. Tech A texture composition processing step for generating and outputting decoded image data by performing the tea composition processing, and replacing the low-frequency component of the composite image data with the low-frequency component of the restored image data, and the frequency after the replacement Image data obtained by inverse Fourier transform of the components is the target of the texture synthesis process.

  The present invention is an image decoding program for causing a computer on an image decoding apparatus to perform an image decoding process, which performs decoding processing on input encoded data by waveform encoding processing to generate decoded image data. A waveform decoding processing step to output, a reverse resolution conversion processing step for restoring and outputting restored image data having the same resolution as the encoding target image data by performing reverse resolution conversion on the decoded image data, A texture analysis processing step for generating and outputting histogram information including a histogram of pixel values and a histogram of conversion coefficients for the restored image data by texture analysis on the restored image data, and for encoded data based on the input texture analysis Decoding process is performed, and the histogram information for the image data to be encoded is And generating a difference information between the histogram information for the restored image data and outputting the difference information, and adding the difference information and the histogram information for the restored image data. Histogram information addition processing step for generating and outputting histogram information for data, and texture synthesis processing for generating and outputting decoded image data by inputting the histogram information for the encoding target signal and performing texture synthesis processing And substituting the low-frequency component of the composite image data with the low-frequency component of the restored image data, and subjecting the image data obtained by performing inverse Fourier transform to the post-replacement frequency component to the texture synthesis process. It is characterized by that.

  According to the present invention, it is possible to achieve decoding of encoded data with a reduced code amount while maintaining subjective image quality.

It is a block diagram which shows the structure of the encoding apparatus by one Embodiment of this invention. FIG. 2 is a block diagram illustrating configurations of a texture analysis processing unit 105 and a texture analysis processing unit 106 illustrated in FIG. 1. It is a flowchart which shows operation | movement of the encoding apparatus shown in FIG. It is a block diagram which shows the structure of the decoding apparatus by one Embodiment of this invention. It is a block diagram which shows the structure of the texture synthetic | combination process part 209 shown in FIG. It is a flowchart which shows operation | movement of the decoding apparatus shown in FIG. It is explanatory drawing which shows the structural example of the analysis filter of steerable pyramid conversion. It is a block diagram which shows the structure of an image transmission system.

  Hereinafter, an image encoding device and an image decoding device according to an embodiment of the present invention will be described with reference to the drawings. First, the processing operation principle of the image encoding device and the image decoding device according to the present invention will be described. Waveform reproduction using only texture synthesis is limited in its processing structure. This is because phase information in an image is not retained in texture synthesis. In the present invention, the conventional waveform reproduction-based encoding method and texture synthesis are combined, and the weak points of both are complemented by the advantages of each other. For this purpose, the image signal is divided into components that represent the outline of the waveform (structural components) and other components (texture components), and waveform reproduction-based coding is applied to the structural components. Encoding by texture analysis and texture synthesis is applied.

として規定される２Ｎ＋１×２Ｎ＋１タップの２次元フィルタを対象信号に適用する。サブサンプリングは、解像度比ｒに応じて、信号をｒ画素毎に間引く処理である。 Hereinafter, the flow of the encoding process will be described. First, an image signal (X × Y pixels) is input and converted into a reduced image signal of X ′ × Y ′ pixels by resolution conversion processing. Here, X ′ and Y ′ are integers satisfying X ′ ≦ X and Y ′ ≦ Y. As the resolution conversion process, a spatial filter and subsampling are used. Spatial filters used here can be broadly classified into separation filters and non-separation filters. The separable filter is a filter that independently processes a 2N + 1 tap filter having coefficients (a− _N , a− _{N + 1} ,..., A _N−1 , a _N ) in the horizontal and vertical directions. Non-separable filters

A 2N + 1 × 2N + 1 tap two-dimensional filter defined as follows is applied to the target signal. The sub-sampling is a process for thinning out signals for every r pixels in accordance with the resolution ratio r.

  The reduced image signal obtained as the output of the resolution conversion process is a structural component in the present invention. Therefore, a waveform encoding process is performed on the image signal (X ′ × Y ′ pixel) which is a structural component, and an encoded stream is output. As the waveform encoding process, for example, AVC / H. H.264-compliant encoding method is used, but the waveform encoding process is AVC / H. The method is not limited to H.264, and methods such as JPEG, JPEG2000, and MPEG-2 can also be used.

  For the encoded stream obtained as the output of the waveform encoding process, the AVC / H. The decoding process specified by H.264 is performed to obtain a decoded signal. As an encoding method, AVC / H. When a method other than H.264 (JPEG, JPEG2000, MPEG-2, etc.) is used, a corresponding decoding process is performed. A decoded signal (X ′ × Y ′ pixels) obtained as an output of the decoding process is referred to as a low-resolution structural component decoded signal. An inverse resolution conversion process is performed on the low-resolution structural component decoded signal to obtain an image signal (X × Y pixels) having the same resolution as the original image. As a result, the obtained signal is called a structural component decoded signal.

  Next, an encoding process for the difference signal between the original signal and the structural component decoded signal will be described. First, a difference signal between the original signal and the decoded signal (X × Y pixel) of the structural component is calculated. This difference signal becomes a texture component in this method. As the order of encoding, after obtaining a decoded signal of a structural component, the process proceeds to encoding processing for a texture component. The following two types of histograms are generated as information expressed by the texture component. One is a histogram (referred to as a pixel value histogram) of the original signal. The other is Steerablepyramid transformation on the original signal (Reference: EP Simoncelli and W. T Freeman. The steerable pyramid: A exible architecture for multi-scale derivative computation. In IEEE Int'l Conf. On Image Processing, Oct. 199. 10 (Referred to as “conversion coefficient histogram”). A specific algorithm for steerable pyramid conversion will be described later. The process of generating information (the above-mentioned histogram) for expressing the texture component is called a texture analysis process.

  The encoding of the generated histogram is performed as follows. First, the above-described two types of histograms are generated for the original signal, and then the above-described two types of histograms are also generated for the structural component decoded signal. The difference between the histogram of the original signal and the histogram of the structural component decoded signal is calculated, and the difference is used as an encoding target. Since the structural component is a part of information included in the original signal, there is overlapping information between the structural component and the original signal regarding the histogram. Therefore, by making the difference between the two histograms the object of encoding, it is possible to reduce the amount of codes compared to the case where the histogram of the original signal is the object of encoding as it is.

Next, Steerable pyramid conversion will be described. Steerable pyramid conversion is a kind of directional filter bank, and its output is defined by the number of directions and the decomposition level. When steerable pyramid conversion with the number of directions M and decomposition level J _max is performed with the original signal being 2 ^J × 2 ^J pixels, M types of direction components are represented at the jth decomposition level (0 ≦ j ≦ J _max −1). Each directional component has 2 ^J−j × 2 ^J−j conversion coefficients. In the decomposition level _{j = J max} topmost not divided about the direction component ^{has 2 J-Jmax × 2 J-} Jmax number of transform coefficients. In addition to the above, there are 2 ^J × 2 ^J conversion coefficients as high-frequency components.

Further, since the transform coefficient is expressed by a real value, each direction component for each decomposition level is quantized with a predetermined quantization width K, and a histogram for the quantized transform coefficient is used as texture analysis information. As an example, FIG. 7 shows a configuration example of an analysis filter for steerable pyramid conversion when M = 2 and J _max = 1. FIG. 7 is a diagram illustrating a configuration example of an analysis filter for steerable pyramid conversion. This filter includes a high-pass filter 601, a low-pass filter 602, a first direction component pass filter 603, a second direction component pass filter 604, a low-pass filter 605, and a downsample 606. When increasing the decomposition level, the process of the portion surrounded by the broken line is recursively repeated for the sub-sampled signal.

Here, the expression method of the histogram is organized.
When the histogram is expressed as h _B [i] (i = 0,..., B−1), h _B [i] stores the number of samples whose target signal value is i. At this time, B is referred to as the number of bins in the histogram, and h _B [i] represents the frequency value of the i-th bin.

  Also for the histogram for the conversion coefficient of Steerable pyramid conversion, the number of bins of the histogram is limited based on the power of the image after resolution conversion. Specifically, the bin number of the histogram is set to be smaller as the power becomes smaller. This is because the number of bins in the histogram of the transform coefficient is always kept constant considering that the bit depth necessary for the transform coefficient also changes according to the frequency component of the signal before conversion and the frequency characteristics of the filter used for resolution conversion. This is because when the power of the image after resolution conversion is small, there is a possibility that a redundant expression is given to the conversion coefficient.

A histogram with respect to the transformation coefficient of the m-th (m = 0,..., M−1) direction component at the j-th decomposition level (0 ≦ j ≦ J _max −1) in the steerable pyramid transformation is represented by L ^{(j, m)} _Bδ [ i] (i = 0,..., 2 ^{2 (J−j)} −1).

As described above, at the highest decomposition level j = _Jmax , there is no division related to the direction component, and the histogram is limited to a histogram corresponding to m = 0. In addition to the above, since there are 2 ^J × 2 ^J conversion coefficients as high-frequency components, a histogram storing this high-frequency component is represented as H _J [i] (i = 0,..., 2 ^2J − 1).

A histogram for the transformation coefficient of the m-th (m = 0,..., M−1) direction component at the j-th decomposition level (0 ≦ j ≦ J _max −1) is represented by L ^{(j, m)} _Bδ [i] (i = 0,..., 2 ^{2 (J−j)} −1) means that the conversion coefficient expressed by a real value is approximated by B types of discrete values. In this discretization, the quantization width to be used is q _{j, m} , and the m-th (m = 0,..., M−1) direction component conversion at the j-th decomposition level (0 ≦ j ≦ J _max −1). Assuming that the coefficient c _{j, m} [k] (k = 0,..., 2 ^{2 (J−j)} −1), the approximation error E ^{(j, m)} _Bδ associated with this discretization is expressed by the following equation: It can be expressed as

“X” means truncation after the decimal point for the real number x. Moreover, the approximation error with respect to a high frequency component is represented as follows for convenience.

  Note that m = M used here is not used to identify a direction component, but as an index representing a high-frequency component.

The following equation is defined as the sum of approximation errors for the omnidirectional components at all resolution levels.

As the number of bins increases, the approximate error sum E _total (B _δ ) decreases monotonously.
Therefore, when setting the number of bins, a threshold value T for Expression (2) is set, a plurality of values that are candidates for the number of bins are prepared, and the minimum value that satisfies the following expression among the values included in the candidates Is the number of bins.
E _total (B _δ ) ≦ T

  Hereinafter, the flow of the decoding process will be described. In the decoding process, two types of encoded information are input. First, a corresponding decoding process is performed on the encoded stream obtained as an output of the waveform encoding process to obtain a decoded signal. A reverse resolution conversion process is performed on the decoded signal (X ′ × Y ′ pixel) obtained as an output of the decoding process to obtain a structural component decoded signal (X × Y pixel).

  Next, using the texture analysis information as input, the texture information is decoded based on the following texture synthesis method. For texture synthesis, a method using a decoded signal of a structural component is used based on the method of Non-Patent Document 1. In the method of Non-Patent Document 1, a histogram of coefficients obtained as an output of a directional filter bank called Steerable pyramid transformation is used as an input for texture synthesis processing.

  The outline of this method is shown below. The input of the method includes an appropriate initial image (white noise image in the above-mentioned document), a pixel value histogram for a target texture image (hereinafter referred to as a target texture image), and each sub-image for both images. A band histogram is used. Steerable pyramid transform is used for subband decomposition. An image obtained as an output of texture synthesis is called a synthesized image. Note that at the start of the following processing, the composite image is initialized with the initial image.

  Next, the texture synthesis process will be described. First, the pixel value histogram of the composite image and the pixel value histogram of the target texture image are input, and conversion processing is performed on the pixel value histogram that brings the pixel value histogram of the composite image closer to that of the target texture image. In the conversion processing, histogram matching is performed on the pixel value histogram, and a mapping table F [] for associating both pixel value histograms is calculated. The outline of the histogram matching process for the pixel value histogram is as follows.

  The cumulative histogram of the target texture image and the cumulative histogram of the composite image are set as H1 [x] and H2 [x], respectively. In the case of an 8-bit image, x takes a value of 0 ≦ x ≦ 255. For the pixel value x1 of the target texture image, a pixel value x2 of the composite image that satisfies H1 [x1] = H2 [x2] is obtained. In order to store the correspondence between the pixel values x1 and x2, a lookup table called a mapping table is prepared, and the pixel value x2 is stored as the x1 element of the mapping table. Assuming that the mapping table is F [], the correspondence relationship between x1 and x2 is expressed as F [x1] = x2. The above F [x1] is obtained for all possible pixel values as x1.

  Next, the pixel value of the synthesized image is converted using the obtained mapping table F []. For the pixel value x2 of the composite image, x1 satisfying F [x1] = x2 is obtained with reference to the mapping table, and the pixel value x2 of the composite image is corrected to x1. Using the corrected composite image as an input, forward conversion of the directional filter bank for the image is performed, a conversion coefficient is calculated, and a histogram of the conversion coefficient is generated for each subband. Hereinafter, the histogram of the transform coefficient generated for each subband is referred to as a subband histogram.

  Next, with the target texture image as an input, the directional filter bank is forward-converted with respect to the image, a conversion coefficient is calculated, and a histogram of the conversion coefficient is generated for each subband.

  Next, conversion processing is performed by using the generated subband histograms of both images as input, and bringing the subband histograms of the composite image closer to the subband histograms of the same subband of the target texture image. In the conversion processing, histogram matching is performed on the subband histograms, and a mapping table G [] for associating both subband histograms is calculated.

  Hereinafter, the j-th decomposition level subband of the m-th direction component will be described as an example, but the processing is the same for other subbands. The cumulative histogram of the subband coefficients of the target texture image and the cumulative histogram of the conversion coefficients of the composite image are set as H1 ′ [y] and H2 ′ [y], respectively. A subband coefficient value y2 of the composite image that satisfies H1 ′ [y1] = H2 ′ [y2] is obtained with respect to the subband coefficient value y1 of the target texture image. As a result, a mapping table used for histogram matching is defined as G [y1] = y2. The above G [y1] is obtained for all subband coefficient values that can be taken as y1.

  Subsequently, the subband coefficient value of the composite image is corrected using the mapping table G [] obtained by the above-described histogram matching. For the subband coefficient value y2 of the composite image, y1 satisfying G [y1] = y2 is obtained with reference to the mapping table, and the subband coefficient value y2 of the composite image is corrected to y1. The same correction process is performed on the subband coefficients of all subbands. The process of performing the inverse transformation of the directional filter bank using the subband coefficient value of the composite image after the correction processing as an input and outputting the composite image is repeated a certain number of times.

  As a technique for obtaining subband coefficients in texture synthesis, for example, a filter bank other than the above, such as wavelet transform, can be applied. The texture feature prediction mechanism in the present invention is not limited to the above-described directional filter bank, and can be similarly applied to other filter banks. The present invention adds the following improvements to the texture synthesis process described above.

  The feature of the improvement resides in processing that imposes restrictions on the generation process in texture synthesis using the frequency component of the decoded signal of the structural component. First, when a composite image is obtained, Fourier transform is applied to the composite image to calculate a frequency component of the composite image.

  Next, Fourier transform is applied to the decoded signal of the structural component to calculate the frequency component of the decoded signal of the structural component. Of the frequency components of the synthesized image, the low frequency components (bands of 1 / r or less of the entire frequency band) are replaced with the frequency components of the structural component decoded signal. Note that r = X / X ′ = Y / Y ′.

  In the above-described method, there is no mechanism for storing the structure information in the image signal because there is only a restriction due to the histogram. For this reason, there is a problem that the convergence of the iterative process is not guaranteed, or there is a problem that a large number of iterations are required for convergence. On the other hand, by adding the above-described improvement, it becomes possible to add restrictions due to the structure information in the iterative process, and the above-mentioned problems can be solved.

  Next, the configuration of an image encoding device according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an image encoding device according to the embodiment. In FIG. 1, reference numeral 101 denotes a resolution conversion process in which an encoding target image (X × Y pixels) is input, converted into a reduced image signal of X ′ × Y ′ pixels by a resolution conversion process, and the reduced image is output. Part. Reference numeral 102 denotes a waveform encoding processing unit that receives the reduced image signal obtained as the output of the resolution conversion process, performs the encoding process according to a predetermined encoding method, and outputs the encoded data. As an encoding process, for example, H.264 is used. An encoding method based on the H.264 / AVC standard can be applied.

  Reference numeral 103 denotes a waveform decoding processing unit that receives the encoded data generated by the waveform encoding processing unit 102 and performs decoding processing to obtain a decoded signal (low-resolution structural component decoded signal: X ′ × Y ′ pixel). is there. The decoding method is a method corresponding to the encoding method used in the waveform encoding processing unit 102. If an encoding method compliant with the H.264 / AVC standard is used, the decoding method is also H.264. A decoding method compliant with the H.264 / AVC standard is used.

  Reference numeral 104 receives a low-resolution structural component decoded signal (X ′ × Y ′ pixel) as an input, performs reverse resolution conversion processing, and outputs a structural component decoded signal as an image signal (X × Y pixel) having the same resolution as the original image. This is a reverse resolution conversion processing unit. Reference numeral 105 denotes a texture analysis processing unit that performs texture analysis processing on the structural component decoded signal and outputs a pixel value histogram / transform coefficient histogram for the structural component decoded signal.

  Reference numeral 106 denotes a texture analysis processing unit that performs texture analysis processing on the encoding target signal and outputs a pixel value histogram / transform coefficient histogram for the encoding target signal. Reference numeral 107 denotes a histogram difference information generation unit that receives the pixel value histogram of the encoding target image and the structural component decoded signal, generates difference information between the two histograms, and outputs the difference information. Also, the histogram difference information generation unit 107 receives the encoding target image and the pixel value histogram of the structural component decoded signal, and generates and outputs difference information of both histograms. In addition, the conversion coefficient histogram of the encoding target image and the structural component decoded signal is input, and difference information of both histograms is generated and output.

  Reference numeral 108 denotes a histogram difference information encoding unit that inputs the pixel value histogram difference information output from the histogram difference information generation unit 107, encodes the difference information, and outputs the encoded difference information as encoded data. Further, the histogram difference information encoding unit 108 receives the difference information of the transform coefficient histogram, encodes the difference information, and outputs it as encoded data. As an example of the encoding method, a method of arithmetically encoding each frequency of the histogram is applicable. Reference numeral 109 denotes an encoded data multiplexing unit that multiplexes and outputs the encoded data output from the waveform encoding processing unit 102 and the histogram difference information encoding unit 108.

  Next, the configuration of the texture analysis processing unit 105 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a diagram showing a configuration of the texture analysis processing unit 105 shown in FIG. In FIG. 2, reference numeral 301 denotes a pixel value histogram generation processing unit that generates and outputs a pixel value histogram of an input image. Reference numeral 302 denotes a conversion processing unit that applies Steerable pyramid conversion to an input image and outputs a conversion coefficient. Reference numeral 303 denotes a conversion coefficient histogram generation processing unit that receives the conversion coefficient of Steerable pyramid conversion as an input and generates and outputs a conversion coefficient histogram.

  Next, the processing operation of the image encoding device shown in FIG. 1 will be described with reference to FIG. FIG. 3 is a diagram showing a processing operation of the image encoding device shown in FIG. First, the resolution conversion processing unit 101 receives an encoding target image (X × Y pixels), converts it into a reduced image signal of X ′ × Y ′ pixels by resolution conversion processing, and outputs a reduced image (step S1). ). Subsequently, the waveform encoding processing unit 102 receives the reduced image signal obtained as the output of the resolution conversion process, performs the encoding process according to a predetermined encoding method, and outputs the encoded data ( Step S2). As an encoding process, for example, H.264 is used. An encoding method compliant with the H.264 / AVC standard can be applied.

  Next, the waveform decoding processing unit 103 receives the encoded data output from the waveform encoding processing unit 102, performs decoding processing, and outputs a decoded signal (low-resolution structural component decoded signal: X ′ × Y ′ pixel). Generate (step S3). The decoding method is a method corresponding to the encoding method. If an encoding method compliant with the H.264 / AVC standard is used, the decoding method is also H.264. A decoding method conforming to the H.264 / AVC standard may be used.

  Next, the reverse resolution conversion processing unit 104 receives the low resolution structural component decoded signal (X ′ × Y ′ pixel) as input, performs reverse resolution conversion processing, and performs an image signal (X × Y pixel) having the same resolution as the original image. A structural component decoded signal is generated (step S4). The texture analysis processing unit 105 receives the structural component decoded signal (X × Y pixels) as an input, and generates and outputs a pixel value histogram of the structural component decoded signal (step S5). Further, the texture analysis processing unit 105 receives the structural component decoded signal (X × Y pixel) as input, performs steerable pyramid conversion on the structural component decoded signal, and generates a conversion coefficient histogram for the obtained conversion coefficient. And output (step S6).

The conversion coefficient histogram is generated as follows. First, the parameter direction number M and the decomposition level J _max that define Steerableramid conversion are read. Then, steerable pyramid conversion determined by the number of directions M and the decomposition level J _max is performed on the input signal, and a conversion coefficient is output. Next, a histogram is obtained for 2 ^J × 2 ^J conversion coefficients belonging to the high frequency component, and output as H _J [i] (i = 0,..., 2 ^2J −1). Subsequently, the following processing is repeated for j = 0,..., J _max −1. The following processing is repeated for m = 0,. A histogram is obtained for 2 ^{2 (J−j)} transform coefficients belonging to the m-th direction component at the j-th decomposition level, and L ^{(j, m)} _Bδ [i] (i = 0,..., 2 ^{2 (J -J)} Output as -1). Then, a histogram for 2 ^{2 (J−Jmax)} transform coefficients belonging to the highest decomposition level j = J _max is obtained, and L ^{(Jmax, 0)} _Bδ [i] (i = 0,..., 2 ² Output as ^(J-.Jmax) -1). At the highest decomposition level, there is no division for the directional component and it is limited to the histogram corresponding to m = 0.

  Next, the texture analysis processing unit 106 receives the encoding target image (X × Y pixels), generates and outputs a pixel value histogram of the encoding target image (step S7). Further, the texture analysis processing unit 106 inputs an encoding target image (X × Y pixels), performs Steerable pyramid conversion on the encoding target image, and generates a conversion coefficient histogram for the obtained conversion coefficient. And output (step S8). The generation of the conversion coefficient histogram is performed by the same process as in step S6.

  Next, the histogram difference information generation unit 107 inputs the encoding target image and the pixel value histogram of the structural component decoded signal, generates and outputs difference information of both histograms, and the histogram difference information encoding unit 108 The difference information is encoded and output as encoded data (step S9). Also, the histogram difference information generation unit 107 receives the encoding coefficient image and the transform coefficient histogram of the structural component decoded signal, generates and outputs difference information of both histograms, and the histogram difference information encoding unit 108 Information is encoded and output as encoded data (step S10). As an example of the encoding method, a method of arithmetically encoding each frequency of the histogram is applicable. Then, the encoded data multiplexing unit 109 multiplexes the encoded data and outputs it.

  Next, the configuration of an image decoding device according to an embodiment of the present invention will be described with reference to FIG. FIG. 4 is a block diagram showing the configuration of the image decoding apparatus in the embodiment. In FIG. 4, reference numeral 201 denotes an encoded data separation unit that receives encoded data as input and separates into encoded data of low-resolution structural components and encoded data of difference information of the histogram. Reference numeral 202 denotes a waveform decoding processing unit that receives encoded data for a low-resolution structural component and performs decoding processing to obtain a decoded signal (low-resolution structural component decoded signal: X ′ × Y ′ pixels). The decoding method is a method corresponding to the encoding process used to generate the input encoded data. If an encoding method compliant with the H.264 / AVC standard is used, the decoding method is also H.264. A decoding method compliant with the H.264 / AVC standard is used.

  Reference numeral 203 receives a low-resolution structural component decoded signal (X ′ × Y ′ pixel) as input, performs reverse resolution conversion processing, and outputs a structural component decoded signal as an image signal (X × Y pixel) having the same resolution as the original image. A reverse resolution conversion processing unit that generates and outputs. Reference numeral 204 denotes a structural component decoded signal storage unit that stores the structural component decoded signal. Reference numeral 205 denotes a texture analysis processing unit that receives a structural component decoded signal (X × Y pixels), performs a texture analysis process, and outputs a pixel value histogram and a transform coefficient histogram for the structural component decoded signal. The detailed configuration of the texture analysis unit 205 shown in FIG. 4 is the same as the configuration shown in FIG.

  Reference numeral 206 denotes a histogram difference information decoding unit that reads encoded data of difference information of pixel value histograms of an encoding target image and a structural component decoded signal, performs designated decoding processing, and generates the difference information. As an example of the decoding process, if the difference information is arithmetically encoded, the decoding process corresponds to the arithmetic encoding. A reference numeral 207 receives the histogram information (pixel value histogram and transform coefficient histogram) of the structural component decoded signal output from the texture analysis unit 205 and the histogram information output from the histogram difference information decoding unit 206 as input. It is a histogram information addition processing unit that adds both of them for each generation, and generates and outputs a pixel value histogram and a transform coefficient histogram of the encoding target image.

  Reference numeral 208 denotes a histogram information storage unit that stores histogram information. Reference numeral 209 denotes a texture synthesis processing unit that receives a pixel value histogram and a conversion coefficient histogram of an encoding target image, performs texture synthesis processing, and outputs a decoded image.

  Next, the configuration of the texture synthesis processing unit 209 shown in FIG. 4 will be described with reference to FIG. FIG. 5 is a block diagram showing the configuration of the texture synthesis processing unit 209 in FIG. In FIG. 5, reference numeral 401 denotes a composite image initialization unit that outputs a predetermined image signal as an initial value of the composite image. As an example of an image used for initialization, a white noise image generated with a random number can be used. Reference numeral 402 denotes a composite image storage unit that stores an initial value of a composite image output from the composite image initialization unit 401.

  Reference numeral 403 denotes a Fourier transform unit that performs a Fourier transform using the composite image as an input and generates a frequency component of the signal. Reference numeral 404 denotes a Fourier transform unit that receives a structural component decoded signal as input and performs Fourier transform to generate a frequency component of the signal. Reference numeral 405 receives the frequency component of the composite image output from the Fourier transform unit 403 and the frequency component of the structural component decoded signal output from the Fourier transform unit 404 as input, and the low frequency component (all frequency bands) of the frequency components of the composite image. Is a low-frequency component replacement unit that replaces the frequency component of the structural component decoded signal. Note that r = X / X ′ = Y / Y ′.

  Reference numeral 406 denotes an inverse Fourier transform processing unit that receives the frequency component output from the low-frequency component replacement unit 405, performs inverse Fourier transform processing, and outputs the obtained image signal as a corrected composite image. Reference numeral 407 denotes a composite image storage unit that stores the composite image output from the inverse Fourier transform processing unit 406.

  Reference numeral 408 denotes a pixel value histogram generation unit that generates a pixel value histogram of a composite image using the composite image read from the composite image storage unit 407 as an input. Reference numeral 409 reads the pixel value histogram of the composite image output from the pixel value histogram generation unit 408 and the pixel value histogram of the encoding target image stored in the histogram information storage unit 208 and encodes the pixel value histogram of the composite image. It is a histogram information correction processing unit that performs a process of approaching the pixel value histogram of the target image and outputs a pixel value histogram of the processed composite image.

  Reference numeral 410 denotes a histogram information correction processing unit 409 using the pixel value histogram of the composite image before the processing of the histogram information correction processing unit 409, the pixel value histogram of the composite image obtained by the histogram information correction processing unit 409, and the composite image as inputs. This is an image value correction processing unit that corrects the pixel value of the composite image using the pixel value histograms of the composite image before and after the process, and outputs the composite image after correction. Reference numeral 411 denotes a composite image storage unit that stores the corrected composite image output from the image value correction processing unit 410.

  Reference numeral 412 denotes a conversion processing unit that receives a synthesized image read from the synthesized image storage unit 411, performs steerable pyramid conversion, and outputs a conversion coefficient. Reference numeral 413 denotes a conversion coefficient histogram generation unit that generates a conversion coefficient histogram for the conversion coefficient output from the conversion processing unit 412.

  Reference numeral 414 reads the conversion coefficient histogram of the composite image output from the conversion coefficient histogram generation unit 413 and the conversion coefficient histogram of the encoding target image stored in the histogram information storage unit 208 and encodes the conversion coefficient histogram of the composite image. This is a histogram information correction processing unit that performs processing to approximate the conversion coefficient histogram of the target image and outputs the conversion coefficient histogram of the composite image after processing.

  Reference numeral 415 denotes a conversion coefficient histogram of the composite image before the processing of the histogram information correction processing unit 414, a conversion coefficient histogram of the composite image obtained by the histogram information correction processing unit 414, and the composite image as inputs, and a histogram information correction processing unit 414. Using the conversion coefficient histogram of the composite image before and after the processing according to the above, the conversion coefficient value of the composite image is corrected, the corrected conversion coefficient is input, reverse Steerable pyramid conversion is performed on the conversion coefficient, and the composite image is An image value correction processing unit to output. Reference numeral 416 outputs a composite image output from the image value correction processing unit 415 as a decoded image if the end condition is satisfied, and outputs a composite image output from the image value correction processing unit 415 if the end condition is not satisfied. The image is stored in the composite image storage unit 402.

  Next, the processing operation of the decoding device shown in FIGS. 4 and 5 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing operation of the decoding apparatus shown in FIGS. First, the waveform decoding processing unit 202 receives the encoded data for the low-resolution structural component separated by the encoded data separation unit 201, performs a decoding process, and outputs a decoded signal (low-resolution structural component decoded signal: X ′ × Y). 'Pixel' is generated (step S21). The decoding method is a method corresponding to the encoding process used to generate the input encoded data. If an encoding method compliant with the H.264 / AVC standard is used, the decoding method is also H.264. A decoding method compliant with the H.264 / AVC standard is used.

  Next, the reverse resolution conversion processing unit 203 receives the low-resolution structural component decoded signal (X ′ × Y ′ pixel) as an input, performs reverse resolution conversion processing, and performs an image signal (X × Y pixel) having the same resolution as the original image. The structural component decoded signal is generated (step S22). Subsequently, the texture analysis processing unit 205 receives the structural component decoded signal (X × Y pixels) as an input, and generates a pixel value histogram of the structural component decoded signal (step S23).

  On the other hand, the histogram difference information decoding unit 206 reads the encoded data of the difference information of the pixel value histogram of the encoding target image and the structural component decoded signal separated by the encoded data separation unit 201, and performs a specified decoding process. The difference information is generated (step S24). As an example of the decoding process, if the difference information is arithmetically encoded, the decoding process corresponds to the arithmetic encoding.

  Next, the histogram information addition processing unit 207 receives the histogram difference information output from the histogram output from the texture analysis processing unit 205 and the histogram difference information decoding unit 206, and adds both for each histogram component. A pixel value histogram of the encoding target image is generated (step S25). The texture analysis processing unit 205 receives the structural component decoded signal (X × Y pixels) as input, performs steerable pyramid conversion on the structural component decoded signal, and generates a conversion coefficient histogram for the obtained conversion coefficient ( Step S26).

  Next, the histogram difference information decoding unit 206 reads the encoded data of the difference information of the transform coefficient histogram of the encoding target image and the structural component decoded signal, performs the specified decoding process, and generates the difference information (step) S27). As an example of the decoding process, if the difference information is arithmetically encoded, the decoding process corresponds to the arithmetic encoding. Subsequently, the histogram information addition processing unit 207 receives the output transform coefficient histogram and the difference information of the output histogram as inputs, adds both of the components for each histogram, and generates a pixel value histogram of the encoding target image. A transform coefficient histogram of the encoding target image is generated (step S28).

  Next, the composite image initialization unit 401 initializes the composite image with a predetermined image signal (step S29). As an example of an image used for initialization, a white noise image generated with a random number can be used. Subsequently, the Fourier transform processing unit 404 receives the structural component decoded signal as input and performs Fourier transform to generate a frequency component of the signal (step S30).

  Next, the Fourier transform processing unit 403 performs Fourier transform using the composite image as an input, receives the frequency component of the obtained composite image and the frequency component of the obtained structural component decoded signal, and inputs the frequency component of the composite image. Of these, the low-frequency component (band of 1 / r or less of the entire frequency band) is replaced with the frequency component of the decoded signal of the structural component. Note that r = X / X ′ = Y / Y ′. Then, the inverse Fourier transform processing unit 406 performs inverse Fourier transform on the frequency component after replacement to obtain a composite image after correction processing (step S32).

  Next, the pixel value histogram generation unit 407 receives the composite image and generates a pixel value histogram of the composite image (step S33). Subsequently, the histogram information correction processing unit 409 reads the pixel value histogram of the composite image and the pixel value histogram of the encoding target image, performs a process of bringing the pixel value histogram of the composite image close to the pixel value histogram of the encoding target image, A pixel value histogram of the processed composite image is output (step S34).

  Next, the image value correction processing unit 410 receives the pixel value histogram of the composite image before processing by the histogram information correction processing unit 409, the pixel value histogram of the obtained composite image, and the composite image as input, and the histogram information correction processing unit 409. The pixel values of the composite image are corrected using the pixel value histograms of the composite image before and after the process of step S35 to generate a corrected composite image (step S35). Subsequently, the conversion processing unit 412 performs Steerable pyramid conversion using the composite image as an input, and generates a conversion coefficient histogram for the obtained conversion coefficient (step S36).

  Next, the transformation coefficient histogram generation unit 413 reads the transformation coefficient histogram of the synthesized image and the transformation coefficient histogram of the encoding target image, performs a process of bringing the transformation coefficient histogram of the synthesized image closer to the transformation coefficient histogram of the encoding target image, A conversion coefficient histogram of the composite image after processing is output (step S37).

  Next, the histogram information correction processing unit 414 receives as input the conversion coefficient histogram of the composite image before the processing of the conversion coefficient histogram generation unit 413, the conversion coefficient histogram of the composite image obtained by the conversion coefficient histogram generation unit 413, and the composite image. The conversion coefficient value of the composite image is corrected using the conversion coefficient histogram of the composite image before and after the processing by the conversion coefficient histogram generation unit 413 (step S38).

  Next, the image value correction processing unit 415 receives the corrected conversion coefficient as input, performs inverse steerable pyramid conversion on the conversion coefficient, and generates a composite image (step S39). Then, the end determination unit 416 repeats the processes in steps S31 to S39 until the end condition is satisfied. As an example of the end condition, the number of iterations exceeds the given upper limit, or the amount of change between the input composite image and the output composite image is less than the given threshold.

  Next, the configuration of an image transmission system including the image encoding device and the image decoding device shown in FIGS. 1 and 4 will be described with reference to FIG. FIG. 8 is a block diagram showing a configuration of the image transmission system. In FIG. 8, reference numeral 501 denotes a moving image input unit that inputs a moving image captured by a camera or the like. Reference numeral 502 denotes a moving image encoding apparatus configured by the image encoding apparatus shown in FIG. 1, which encodes and transmits a moving image input by the moving image input unit 1. Reference numeral 503 denotes a transmission path for transmitting encoded moving image data transmitted from the moving image encoding device 502. Reference numeral 504 denotes a moving picture decoding apparatus configured by the image decoding apparatus shown in FIG. 4, which receives encoded moving picture data transmitted through the transmission path 503, decodes the encoded moving picture data, and outputs the decoded moving picture data. Reference numeral 505 denotes a moving image output unit that outputs the moving image decoded by the moving image decoding device 504 to a display device or the like.

  Next, the operation of the image transmission system shown in FIG. 8 will be described. The image encoding device 502 inputs moving image data via the moving image input unit 501 and performs encoding by the above-described processing operation for each frame of the moving image. Then, the moving image encoding device 502 transmits the encoded moving image data to the moving image decoding device 504 via the transmission path 503. The moving picture decoding apparatus 504 decodes the encoded moving picture data by the processing operation described above, and displays the moving picture on a display device or the like via the moving picture output unit 205.

  As described above, as a low-rate coding method, an image that can subjectively generate a signal with an image quality close to the original signal based on the texture synthesis technique as a method that replaces the conventional waveform reproduction-based coding method. An encoding process and an image decoding process can be provided. Thereby, it is possible to realize encoding that reduces the code amount while maintaining the subjective image quality.

  Note that a program for realizing the function of the processing unit in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed, thereby executing image coding processing. In addition, image decoding processing may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As a low-rate encoding method, it can be applied to applications where it is indispensable to generate a signal of an image quality subjectively close to the original signal based on the texture synthesis technique as a method that replaces the conventional waveform reproduction-based encoding method.

  DESCRIPTION OF SYMBOLS 101 ... Resolution conversion process part, 102 ... Waveform encoding process part, 103 ... Waveform decoding process part, 104 ... Reverse resolution conversion process part, 105 ... Texture analysis process part, 106 ... Texture analysis processing unit 107... Histogram difference information generation unit 108... Histogram difference information encoding unit 109 109 Encoded data multiplexing unit 201 201 Encoded data separation unit 202 ... Waveform decoding processing unit, 203... Inverse resolution conversion processing unit, 204... Structural component decoded signal storage unit, 205... Texture analysis processing unit, 206. Histogram information addition processing unit, 208 ... Histogram information storage unit, 209 ... Texture synthesis processing unit, 301 ... Pixel value histogram generation processing unit, 302 Conversion processing unit 303: Conversion coefficient histogram generation processing unit 401 ... Composite image initialization unit 402, 411 ... Composite image storage unit 403, 404 ... Fourier transform processing unit 405 .. Low-frequency component replacement unit, 406... Inverse Fourier transform processing unit, 407... Composite image storage unit, 408... Pixel value histogram generation unit, 409. ... image value correction processing section, 412 ... conversion processing section, 413 ... conversion coefficient histogram generation section, 414 ... histogram information correction processing section, 416 ... end determination section

Claims (3)

Waveform decoding processing means for performing decoding processing on the encoded data by the input waveform encoding processing to generate and output decoded image data; and
Reverse resolution conversion processing means for restoring and outputting restored image data having the same resolution as the encoding target image data by performing reverse resolution conversion on the decoded image data;
Texture analysis processing means for generating and outputting histogram information including a histogram of pixel values and a histogram of conversion coefficients for the restored image data by texture analysis on the restored image data;
Histogram difference information decoding means for performing decoding processing on encoded data based on input texture analysis to generate and output difference information between histogram information for encoding target image data and histogram information for the restored image data When,
Histogram information addition processing means for adding the difference information and the histogram information for the restored image data to generate and output histogram information for the encoding target image data;
Texture synthesis processing means for generating and outputting decoded image data by inputting the histogram information for the encoding target signal and performing texture synthesis processing; and
An image characterized in that image data obtained by replacing a low-frequency component of composite image data with a low-frequency component of the restored image data and performing inverse Fourier transform on the replaced frequency component is the target of the texture synthesis processing Decoding device. An image decoding method in an image decoding apparatus comprising a waveform decoding processing means, a reverse resolution conversion processing means, a texture analysis processing means, a histogram difference information decoding means, a histogram information addition processing means, and a texture synthesis processing means. ,
The waveform decoding processing means performs a decoding process on the input encoded data by the waveform encoding process to generate and output decoded image data; and
A reverse resolution conversion processing step in which the reverse resolution conversion processing unit performs reverse resolution conversion on the decoded image data to restore and output restored image data having the same resolution as the encoding target image data;
The texture analysis processing means generates and outputs histogram information including a histogram of pixel values and a histogram of conversion coefficients for the restored image data by texture analysis for the restored image data; and
The histogram difference information decoding means performs decoding processing on the encoded data based on the input texture analysis, and generates difference information between the histogram information for the encoding target image data and the histogram information for the restored image data. And outputting the histogram difference information decoding step,
The histogram information addition processing means adds the difference information and the histogram information for the restored image data to generate and output histogram information for the encoding target image data; and
The texture synthesis processing means has a texture synthesis processing step of generating and outputting decoded image data by inputting the histogram information for the encoding target signal and performing texture synthesis processing,
An image characterized in that image data obtained by replacing a low-frequency component of composite image data with a low-frequency component of the restored image data and performing inverse Fourier transform on the replaced frequency component is the target of the texture synthesis processing Decryption method. An image decoding program for causing a computer on an image decoding apparatus to perform an image decoding process,
A waveform decoding process step of performing decoding processing on the encoded data by the input waveform encoding processing to generate and output decoded image data; and
A reverse resolution conversion processing step for restoring and outputting restored image data having the same resolution as the encoding target image data by performing reverse resolution conversion on the decoded image data;
Texture analysis processing step for generating and outputting histogram information including a histogram of pixel values and a histogram of conversion coefficients for the restored image data by texture analysis on the restored image data;
Histogram difference information decoding step for generating and outputting difference information between the histogram information for the encoding target image data and the histogram information for the restored image data by performing decoding processing on the input encoded data based on the texture analysis When,
A histogram information addition processing step of adding the difference information and the histogram information for the restored image data to generate and output histogram information for the encoding target image data;
A texture synthesis processing step of generating and outputting decoded image data by inputting the histogram information for the encoding target signal and performing texture synthesis processing; and
An image characterized in that image data obtained by replacing a low-frequency component of composite image data with a low-frequency component of the restored image data and performing inverse Fourier transform on the replaced frequency component is the target of the texture synthesis processing Decryption program.

Images