JP2010287917A - Image encoding/decoding device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image encoding device for generating a prediction image with improved reproducibility of an encoding object area including a feature based on spatial pixel value change represented by an edge component and reducing a predictive residual. <P>SOLUTION: The image encoding device includes: an extraction unit 121 for extracting a first edge component image from an original image; a separation unit 130 for separating a reference image into a second edge component image and an edge-removed image; an auxiliary information generation unit 122 for generating auxiliary information for predicting the first edge component image from the second edge component image; a prediction unit 141 for predicting, using the auxiliary information, a third edge component image from the second edge component image; and a prediction image generation unit 142 for generating an prediction image by combining the edge-removed image and the third edge component image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to image encoding and decoding.

  Conventionally, MPEG-2 or H.264 has been used as a moving image encoding method. H.264 / MPEG4 AVC (hereinafter simply referred to as H.264) is known. In these moving image encoding methods, an original image is divided into blocks of a predetermined size, and prediction images are generated by performing inter-frame prediction based on motion estimation and motion compensation in units of blocks. Further, in the above-described moving image coding method, discrete cosine transform (hereinafter simply referred to as DCT), quantization and entropy coding is performed on the prediction residual between the predicted image and the original image, and the encoded data is converted into the encoded data. Generate.

  H. In motion estimation in H.264, the shape of an object is determined from a large number of block sizes (16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, 8 × 8 pixels, 8 × 4 pixels, 4 × 8 pixels, 4 × 4 pixels). Since an appropriate block size is selected according to the complexity of the motion, the prediction efficiency is higher than that of MPEG-2. However, such motion estimation with block size selection is suitable for inter-frame prediction when a rigid body moves in parallel on the screen, but fails when, for example, object deformation occurs between frames. Thus, a prediction residual is generated. In particular, a prediction residual is likely to occur in a portion where the spatial pixel value changes drastically, such as an edge and a texture, and this prediction residual causes image quality deterioration such as mosquito noise that is easily visible in a decoded image.

  Although the prediction residual can be reduced by selecting a smaller block size in motion estimation, if a small block size is selected, the code amount of header information including a motion vector increases. Further, in places where the spatial pixel value changes drastically, such as edges and textures, transform coefficients after orthogonal transform such as DCT are likely to be scattered in high frequency components, and the edge sharpness in the decoded image due to quantization errors of the transform coefficients Deterioration or loss of texture information occurs. Further, the prediction residual included in the locally decoded image is propagated when it is referred to in the subsequent inter-frame prediction.

  In Patent Document 1, an edge component is previously extracted from an original image, and an image from which the edge component has been removed is MPEG-2 or H.264. In addition to encoding by existing block-based encoding such as H.264, an image having only an edge component is separately accurately encoded (so that a prediction residual does not occur). In this way, by encoding accurately the edge component that is visually important alone, encoding distortion around the edge can be reduced as compared with the conventional video encoding method, so that mosquito noise in the decoded image can be reduced. Can be suppressed. Therefore, a subjective improvement in image quality can be expected particularly in a highly compressed reproduced image.

In Patent Document 2, when performing motion compensation, projective transformation is performed in consideration of parallel movement, rotation, expansion and contraction of an object, and the contour of the object extracted from the original image is determined in order to determine the projection transformation parameter. Information is used as auxiliary information. Therefore, even when an object is deformed between frames, a prediction image obtained by adding a geometric deformation to the reference image can be generated based on the contour information, so that a prediction residual can be reduced. Further, since the outline information extracted from the original image is not encoded, but the residual between the outline information and the outline information extracted from the already encoded reference image is encoded, the amount of code can be reduced.
Japanese Patent Laid-Open No. 7-240925 Japanese Patent No. 3233185

  In Patent Document 1, since an image having only an edge component is accurately encoded alone, the amount of generated code is increased as compared with a case where the entire image is encoded together. In addition, since the encoding of the image with only the edge component is performed separately from the encoding with respect to the image excluding the edge component, it is difficult to control in consideration of the balance of the code amount and the image quality of both encoded data. For example, when the generated code amount due to the image of only the edge component increases, the image quality of the image other than the edge component deteriorates in order to maintain the entire code amount, and the image other than the edge component in order to maintain the entire image quality. The amount of generated code increases.

  Further, in Patent Document 2, contour information is used as auxiliary information for determining projective transformation parameters. Although edge shape information is reflected in the contour information, contour blur and quantization due to the movement of the subject are reflected. The deterioration of the edge sharpness caused by the error is not reflected. Therefore, even if the projective transformation parameter is determined using the contour information, the reproducibility of the edge sharpness cannot be improved.

  Therefore, the present invention generates a predicted image with improved reproducibility of a coding target region including a feature based on a spatial pixel value change typified by an edge component and can reduce a prediction residual. An object is to provide a decoding apparatus.

  An image encoding device according to an aspect of the present invention includes an extraction unit that extracts a first edge component image from an original image; a separation unit that separates a reference image into a second edge component image and an edge-removed image; An auxiliary information generation unit for generating auxiliary information for predicting the first edge component image from the second edge component image; and prediction for predicting a third edge component image from the second edge component image using the auxiliary information A prediction image generation unit that generates a prediction image by synthesizing the edge-removed image and the third edge component image; a prediction residual calculation for obtaining a prediction residual between the original image and the prediction image A coding unit for coding the prediction residual and the auxiliary information.

  An image decoding apparatus according to another aspect of the present invention decodes input encoded data to obtain a prediction residual of a target image and auxiliary information for predicting a first edge component image of the target image. A decoding unit; a separation unit that separates a reference image that has already been decoded into a second edge component image of the reference image and an edge-removed image obtained by removing the second edge component from the reference image; A prediction unit that uses the second edge component image to predict the first edge component image; and combines the edge-removed image and the second edge component image to generate a prediction image; and the prediction A decoded image generation unit that generates a decoded image of the target image using a residual and the predicted image.

  According to the present invention, a moving picture coding capable of generating a prediction image with improved reproducibility of a coding target region including a feature based on a spatial pixel value change represented by an edge component and reducing a prediction residual. / Decoding device can be provided.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
As shown in FIG. 1, the image coding apparatus according to the first embodiment of the present invention includes a coding unit 100 and a coding control unit 150. The coding unit 100 includes a subtracter 101, a transform / quantization unit. 102, an inverse transform / inverse quantization unit 103, an entropy encoder 104, an adder 105, a frame memory 106, a motion estimation / motion compensation unit 107, and a prediction unit 110.

  The encoding unit 100 is controlled by the encoding control unit 150, performs moving image encoding processing called hybrid encoding on the input original image signal 10 of the moving image, and outputs encoded data 14. That is, the encoding unit 100 transforms and quantizes the prediction residual signal between the predicted image signal predicted from the already encoded reference image signal and the original image signal 10, and applies the quantized transform coefficient to the entropy. Encode and output encoded data 14. Hereinafter, in the encoding unit 100, the original image signal 10 is divided into blocks of a predetermined size by a block scan converter (not shown), and processing is performed in units of the blocks. However, processing is performed in units of frames or fields. May be performed.

  The subtractor 101 subtracts a second predicted image signal 19 from a prediction unit 110 described later from the original image signal 10 to calculate a prediction residual signal 11 and passes it to the transform / quantization unit 102.

  The transform / quantization unit 102 performs transform such as DCT on the prediction residual signal 11 from the subtractor 101, performs quantization based on the quantization parameter set by the encoding control unit 150, and performs the quantized transform. The coefficient 12 is passed to the inverse transform / inverse quantization unit 103 and the entropy encoder 104. The transform performed by the transform / quantization unit 102 on the prediction residual signal 11 is not limited to DCT, and may be, for example, wavelet transform or independent component analysis, or other orthogonal transform.

  The inverse transform / inverse quantization unit 103 inversely quantizes the quantized transform coefficient 12 based on the quantization parameter, performs inverse transform such as IDCT, and sends the decoded prediction residual signal 13 to the adder 105. hand over. The inverse transform performed on the quantized transform coefficient 12 by the inverse transform / inverse quantization unit 103 is not limited to IDCT, but is the inverse transform of the transform performed on the prediction residual signal 11 by the transform / quantization unit 102. It shall be.

  The entropy encoder 104 converts the quantized transform coefficient 12 from the transform / quantization unit 102, the motion vector 18 from the motion estimation / motion compensation unit 107, which will be described later, and the auxiliary information 20 from the prediction unit 110 into, for example, a Huffman Entropy encoding such as encoding or arithmetic encoding is performed and output as encoded data 14. The entropy encoder 104 also encodes the quantization parameter and prediction mode information in the same manner.

  The adder 105 adds the decoded prediction residual signal 13 from the inverse transform / inverse quantization unit 103 and a second predicted image signal 19 from the prediction unit 110 described later, and adds the frame memory 106 as a local decoded image signal 15. To pass.

  The frame memory 106 temporarily stores the locally decoded image signal 15 from the adder 105 as a reference image signal 16. The reference image signal 16 is referred to by a motion estimation / motion compensation unit 107 described later. Note that a deblocking filter may be provided before the frame memory 106 to remove block distortion from the locally decoded image signal 15.

  The motion estimation / motion compensation unit 107 performs a motion estimation / motion compensation process using the original image signal 10 and the reference image signal 16 stored in the frame memory 106 to generate the first predicted image signal 17. At the same time, a motion vector 18 is generated. The first predicted image signal 17 is input to the prediction unit 110, and the motion vector 18 is input to the entropy encoder 104.

  The motion estimation / compensation processing performed by the motion estimation / motion compensation unit 107 is performed, for example, in units of blocks of a predetermined size, and blocks between the encoding target block of the original image signal 10 and the reference image signal 16. Perform matching. As a result of the block matching, a block of the reference image signal 16 that minimizes the coding cost is output as the first predicted image signal 17, and a motion vector 18 indicating the position of the first predicted image signal 17 in the reference image signal 16 is generated. Is done. The encoding cost uses, for example, the absolute value sum (SAD) of the difference between the original image signal 10 and the first predicted image signal 17.

In addition, since the reference image signal 16 has already been encoded, the prediction may be performed based on a temporally future frame or a plurality of frames, and the encoding cost may be obtained based on the following mathematical formula. .

Here, K is the coding cost, SAD is the sum of absolute values of differences, λ is a constant determined based on the quantization width and the value of the quantization parameter, OH is header information such as a motion vector and an index indicating the frame to be referred to Respectively. Further, only the header information OH may be used as the encoding cost K, or the difference may be Hadamard transformed or approximated. In addition, a cost function may be created using the activity of the original image signal 10.

  The prediction unit 110 obtains the first predicted image signal 17 from the motion estimation / compensation unit 107 and performs second prediction with improved reproducibility of feature components based on a spatial pixel value change in an image such as an edge. An image signal 19 is generated, and the second predicted image signal 19 is passed to the subtracter 101 and the adder 105. The prediction unit 110 also passes auxiliary information 20 for predicting the second predicted image signal 19 from the first predicted image signal 17 to the entropy encoder 104. In the following description, an edge will be described as an example of a feature component. However, a feature component to which the encoder according to the present embodiment can be applied is not limited to an edge, and may be texture, contrast, and noise, for example.

  As illustrated in FIG. 2, the prediction unit 110 includes a feature extraction unit 121, an auxiliary information generation unit 122, a feature separation unit 130, a feature prediction unit 141, and a signal synthesis unit 142.

  The feature separation unit 130 extracts a feature component from the first predicted image signal 17 from the motion estimation / motion compensation unit 107 to generate the first feature signal 21 and also generates the first feature signal 21 from the first predicted image signal 17. A feature removal signal 22 is generated by removing. That is, the feature separation unit 130 separates the first predicted image signal 17 into the first feature signal 21 and the feature removal signal 22. The first feature signal 21 is input to the auxiliary information generation unit 122 and the feature prediction unit 141, and the feature removal signal 22 is input to the signal synthesis unit 142. As illustrated in FIG. 2, an example of the feature separation unit 130 includes a feature extraction unit 131 and a subtracter 132.

  The feature extraction unit 131 performs, for example, edge extraction processing or filtering processing on the first predicted image signal 17 from the motion estimation / motion compensation unit 107 to extract edge components, and generates the first feature signal 21. The data is passed to the subtractor 132, the feature prediction unit 141, and the signal synthesis unit 142. The feature extraction unit 131 may use an edge detection method by a general differential operator in image processing, and may use, for example, a Sobel operator that is a primary differential or a Laplacian operator that is a secondary differential.

  The subtracter 132 subtracts the first feature signal 21 from the feature extraction unit 131 from the first predicted image signal 17 from the motion estimation / motion compensation unit 107 and passes the feature removal signal 22 to the signal synthesis unit 142.

Note that the feature separation unit 130 is not limited to the configuration illustrated in FIG. 2, and may have the configuration illustrated in FIG. 3A or FIG. 3B, for example.
As illustrated in FIG. 3A, one modification of the feature separation unit 130 includes a smoothing filter 133 and a subtracter 134.

  The smoothing filter 133 is a smoothing filter generally used in image processing such as a moving average filter, a weighted average filter, a median filter, or a Gaussian filter. The smoothing filter 133 removes high frequency components from the first predicted image signal 17 from the motion estimation / motion compensation unit 107. In general, when an image is frequency-converted, an edge is mainly included in a high-frequency component, so that a feature removal signal 22 is generated by removing the high-frequency component from the first predicted image signal 17. The smoothing filter 133 passes the feature removal signal 22 to the subtractor 134 and the signal synthesis unit 142.

  The subtractor 134 subtracts the feature removal signal 22 from the smoothing filter 133 from the first predicted image signal 17 from the motion estimation / motion compensation unit 107 to generate the first feature signal 21, and the auxiliary information generation unit 122. To the feature prediction unit 141.

  As illustrated in FIG. 3B, another modification of the feature separation unit 130 includes a band division unit 135. The band division unit 135 performs frequency band division on the first predicted image signal 17 from the motion estimation / motion compensation unit 107 using frequency transformation such as wavelet transformation, discrete cosine transformation, or independent component analysis, for example, to obtain a high frequency component And divided into low frequency components. As described above, since the edge is mainly included in the high-frequency component when the frequency of the image is converted, the band dividing unit 135 uses the high-frequency component of the first predicted image signal 17 as the first feature signal 21 and the low-frequency component as the feature removal signal 22. Respectively. The first feature signal 21 is input to the auxiliary information generation unit 122 and the feature prediction unit 141, and the feature removal signal 22 is input to the signal synthesis unit 142.

  In addition, the feature separation unit 130 may extract a portion having a high correlation with the edge model as an edge component using an edge model in which the edge is expressed by a function. In addition, various methods can be used for extracting feature components in the feature separation unit 130 as long as they can be similarly performed in the decoding apparatus.

  The feature extraction unit 121 extracts a feature component from the original image signal 10 to generate the second feature signal 23 and passes it to the auxiliary information generation unit 122. The feature extraction unit 121 may extract feature components by various methods such as filtering processing and frequency band division processing, similar to the feature separation unit 130 described above.

  The auxiliary information generation unit 122 acquires the first feature signal 21 from the feature separation unit 130 and the second feature signal 23 from the feature extraction unit 121, and predicts the second feature signal 23 from the first feature signal 21. Auxiliary information 20 used as a parameter is generated. The auxiliary information 20 is input to the feature prediction unit 141 and the entropy encoder 104. A detailed description of the auxiliary information 20 will be described later.

  The feature prediction unit 141 uses the auxiliary information 20 from the auxiliary information generation unit 122 to generate the feature prediction signal 24 by predicting the second feature signal 21 from the first feature signal 23 from the feature separation unit 130. The signal is passed to the signal synthesis unit 142.

  The signal synthesis unit 142 synthesizes the feature removal signal 22 from the feature separation unit 130 and the feature prediction signal 24 from the feature prediction unit 141 to generate a second predicted image signal 19, and the subtracter 101 and the adder 105. To pass.

  Note that the number of feature signals extracted from the original image signal 10 or the first predicted image signal 17 and predicted or synthesized is not limited to one, but a plurality of feature signals are extracted and predicted, and all the feature signals are collectively collected. You may synthesize. When a plurality of first feature signals 21 are extracted from the first predicted image signal 17, it is not necessary to generate auxiliary information 20 for all the extracted first feature signals 21. That is, some of the first feature signals 21 may be combined with the feature removal signal 22 as the feature prediction signal 24 without generating the auxiliary information 20.

  The encoding control unit 150 performs overall control of the encoding unit 100 including feedback control of generated code amount, quantization characteristic control, motion estimation accuracy control, and feature prediction accuracy control.

  Next, an operation example of the prediction unit 110 will be described with reference to FIGS. 4 and 5A to 5J. Here, the flow of processing from the generation of the original image signal 10 shown in FIG. 5A and the corresponding first predicted image signal 17 shown in FIG. 5C to the generation of the second predicted image signal 19 shown in FIG. 5J will be described.

  First, the feature separation unit 130 acquires the first predicted image signal 17 of FIG. 5C from the motion estimation / motion compensation unit 107 (step S201). Next, the feature separation unit 130 extracts the first feature signal 21 shown in FIG. 5D from the first predicted image signal 17 of FIG. 5C acquired in step S201 (step S202). Next, the feature separation unit 130 removes the first feature signal 21 of FIG. 5D acquired in step S202 from the first predicted image signal 17 of FIG. 5C acquired in step S201, and the feature removal signal 22 shown in FIG. 5E. Is generated (step S203). Here, the order in which step S202 and step S203 are performed is not limited to the above. For example, in the case of the feature separation unit 130 shown in FIG. 3A, the order in which steps S202 and S203 are performed may be changed, and in the case of the feature separation unit 130 shown in FIG. Also good.

  On the other hand, the feature extraction unit 121 acquires the original image signal 10 of FIG. 5A (step S204). Next, the feature extraction unit 121 extracts the second feature signal 23 shown in FIG. 5B from the original image signal 10 of FIG. 5A (step S205). Here, the order in which steps S201 to S203 and steps S204 and S205 are performed is not limited to the above, and the order may be changed or may be performed simultaneously.

  Next, the auxiliary information generation unit 122 generates auxiliary information 20 from the first feature signal 21 of FIG. 5D extracted in step S202 and the second feature signal 23 of FIG. 5E extracted in step S205 (step S206). Hereinafter, the auxiliary information 20 will be described in detail. In the present embodiment, the edge is expressed by three pieces of information of shape, intensity, and spread, and the auxiliary information 20 is at least one of the three pieces of information between the first feature signal 21 and the second feature signal 23. Indicates the difference between the two.

  First, generation of the auxiliary information 20 regarding the shape will be described. The auxiliary information generation unit 122 detects the edge center line by performing thinning processing on the first feature signal 21 and the second feature signal 23 in order to generate auxiliary information 20 related to the shape. The thinning process performed by the auxiliary information generation unit 122 on the first feature signal 21 and the second feature signal 23 may be a method generally used in image processing. The auxiliary information generation unit 122 performs the thinning process on the first feature signal 21 in FIG. 5D and the second feature signal 23 in FIG. 5B, thereby obtaining the edge center line shown in FIG. 5F. 5F, the edge center line obtained from the first feature signal 21 of FIG. 5D is indicated by a solid line, and the edge center line obtained from the second feature signal 23 of FIG. 5B is indicated by a broken line. The auxiliary information generation unit 122 generates the shape error of both edge center lines as auxiliary information 20 regarding the shape. The shape error can be expressed by using a higher-order parametric curve such as a chain code or a B-spline curve.

  Next, generation of the auxiliary information 20 relating to strength and spread will be described. As shown in FIG. 5G, the auxiliary information generation unit 122 draws a plurality of perpendicular lines starting and ending with two points adjacent to the edge in the vertical direction in order to generate auxiliary information 20 related to strength and spread. An edge intensity distribution on the perpendicular is obtained, and the plurality of intensity distributions are averaged to obtain one intensity distribution. When the auxiliary information generation unit 122 obtains the intensity distribution for the first feature signal 21 of FIG. 5D and the second feature signal 23 of FIG. 5B, the intensity distribution shown in FIG. 5H is obtained. In FIG. 5H, the horizontal axis indicates the relative position on the vertical line, the vertical axis indicates the edge intensity at the position on the vertical line, the intensity distribution obtained from the first feature signal 21 of FIG. 5D is a solid line, Intensity distributions obtained from the second feature signal 23 are respectively represented by broken lines. In FIG. 5H, the edge intensity at each position is represented by the absolute value of the difference between the pixel value at that position and the pixel value at the start point and end point.

The auxiliary information generation unit 122 generates a difference between the two intensity distributions as auxiliary information 20 regarding the intensity and the spread. The auxiliary information generation unit 122 handles the intensity distribution by approximating it using some function. For example, the intensity distribution shown in FIG. 5H can be expressed by the following equation, where x is the position on the vertical line represented by the horizontal axis, and f (x) is the edge intensity at the position.

Here, E represents edge strength, σ represents edge spread (dispersion), and a represents edge center coordinates. The auxiliary information generation unit 122 has an intensity E that minimizes an error from the distribution represented by Equation (2) for each of the intensity distribution obtained from the first feature signal 21 and the intensity distribution obtained from the second feature signal 23. And the spread σ are obtained, and these difference values are generated as auxiliary information 20 relating to the strength and spread. The function for approximating the intensity distribution is not limited to Equation (2), and may be approximated by a parametric curve such as a B-spline curve drawn between the start point and the end point, or may be approximated by other functions. Also good. Further, instead of approximating with a fixed function, a function suitable for approximation may be selected from a plurality of functions.

  Further, the auxiliary information generation unit 122 may perform motion estimation / motion compensation processing using the first feature signal 21 and the second feature signal 23 to generate a motion vector as the auxiliary information 20.

  Next, the feature prediction unit 141 generates the feature prediction signal 24 shown in FIG. 5I using the auxiliary information 20 generated in step S206 and the first feature signal 21 of FIG. 5D extracted in step S202 (step S207).

  Next, the signal synthesis unit 142 synthesizes the feature removal signal 22 of FIG. 5E generated in step S203 and the feature prediction signal 24 of FIG. 5I generated in step S207, and generates the second predicted image signal 19 shown in FIG. 5J. Generate (step S208).

  As described above, in the video encoding device according to the present embodiment, the second feature signal of the original image signal is predicted and featured using the first feature signal and auxiliary information extracted from the first predicted image signal. A prediction residual signal is generated by generating a prediction signal, generating a second prediction image signal by synthesizing the feature prediction signal with the feature removal signal obtained by removing the first feature signal from the first prediction image signal. Are encoded together with auxiliary information. Therefore, according to the video encoding device according to the present embodiment, the reproducibility of the feature signal is improved compared to the case where the prediction residual between the first predicted image signal and the original image signal is encoded, and the prediction is performed. Residual can be reduced.

  Further, in the video encoding device according to the present embodiment, the second feature signal is not encoded as it is, but the second feature signal is used by using the first feature signal extracted from the first predicted image signal and the auxiliary information. Therefore, the amount of codes can be reduced. In addition, the level of reproducibility of the feature signal is determined by the content of the auxiliary information. However, since the amount of code generated by the auxiliary information can be controlled by a quantization parameter set by the encoding control unit, the reproducibility of the feature signal And the balance of the generated code amount can be easily controlled.

  Note that the moving picture encoding apparatus according to the present embodiment can also be realized by using, for example, a general-purpose computer apparatus as basic hardware. That is, the subtractor 101, the transform / quantization unit 102, the inverse transform / inverse quantization unit 103, the entropy encoder 104, the adder 105, the motion estimation / motion compensation unit 107, the prediction unit 110, and the encoding control unit 150 This can be realized by causing a processor mounted on the computer apparatus to execute a program. At this time, the moving picture encoding apparatus according to the present embodiment may be realized by installing the above program in a computer apparatus in advance, or may be stored in a storage medium such as a CD-ROM or via a network. The above program may be distributed, and this program may be installed in a computer device as appropriate. The frame memory 106 is realized by appropriately using a memory built in or externally attached to the computer device, a hard disk or a storage medium such as a CD-R, CD-RW, DVD-RAM, DVD-R, or the like. Can do.

  Further, the moving picture coding apparatus according to the present embodiment is not limited to coding a moving picture, but generates a predicted picture signal from an already coded picture, and a prediction residual between the input picture signal and the predicted picture signal. Any coding scheme that encodes a signal can be applied to still image coding.

  Note that the present embodiment may be modified to integrate the motion estimation / motion compensation unit 107 and the prediction unit 110. In this modification, a reference image is input from the frame memory 106 to the prediction unit 110 in units of frames. The prediction unit 110 separates each feature. The auxiliary information generation unit 122 performs motion estimation / compensation using the separated components. Then, the motion vector 18 is transmitted as the auxiliary information 20.

(Second Embodiment)
As shown in FIG. 6, the decoding apparatus according to the second embodiment of the present invention includes a decoding unit 300 and a decoding control unit 320. The decoding unit 300 includes an entropy decoder 301, an inverse transform / An inverse quantization unit 302, an adder 303, a frame memory 304, a motion compensation unit 305, and a prediction unit 310 are included.

  The decoding unit 300 is controlled by the decoding control unit 320, performs entropy decoding on the input encoded data 30, performs inverse quantization / inverse orthogonal transformation, decodes the prediction residual signal, and decodes the prediction prediction residual. A decoded image signal 35 is generated by adding a predicted image signal predicted using a reference image signal that has already been decoded to the signal.

  The entropy decoder 301 decodes the encoded data 30 according to a predetermined data structure, and restores the quantized transform coefficient 31, auxiliary information 32, motion vector 33, quantization parameter, prediction mode information, and the like. The entropy decoder 301 passes the quantized transform coefficient 31 to the inverse transform / inverse quantization unit 302, passes the auxiliary information 32 to the prediction unit 310, and passes the motion vector 33 to the motion compensation unit 305.

  The inverse transform / inverse quantization unit 302 inversely quantizes the quantized transform coefficient 31 from the entropy decoder 301 according to the restored quantization parameter, performs inverse transform such as IDCT, and the like, thereby performing a decoded prediction residual. The signal 34 is passed to the adder 303. Note that the inverse transform performed by the inverse transform / inverse quantization unit 302 on the quantized transform coefficient 31 is not limited to IDCT, and may be, for example, inverse wavelet transform or other inverse orthogonal transform, but is predicted on the encoding side. It is assumed that this is an inverse transformation of the transformation performed on the residual signal.

  The adder 303 adds a decoded prediction residual signal 34 from the inverse transform / inverse quantization unit 302 and a second predicted image signal 38 from the prediction unit 310 to be described later to generate a decoded image signal 35. The generated decoded image signal 35 is output and passed to the frame memory 304.

  The frame memory 304 temporarily stores the decoded image signal 35 from the adder 303 as the reference image signal 36. The reference image signal 36 is referred to by a motion compensation unit 305 described later. Note that a deblocking filter may be provided in the previous stage of the frame memory 304 to remove block distortion from the decoded image signal 35.

  The motion compensation unit 305 acquires the reference image signal 36 stored in the frame memory 304 and predicts the region of the reference image signal 36 indicated by the motion vector 33 from the entropy decoder 301 as the first predicted image signal 37. To the unit 310.

  The prediction unit 310 acquires the first predicted image signal 37 from the motion compensation unit 305 and the auxiliary information 32 from the entropy encoder 301, and for example, features components based on spatial pixel value changes in the image such as edges. A second predicted image signal 38 with improved reproducibility is generated, and the second predicted image signal 38 is passed to the adder 303. In the following description, an edge will be described as an example of a feature component. However, a feature component to which the encoder according to the present embodiment can be applied is not limited to an edge, and may be texture, contrast, and noise, for example. As illustrated in FIG. 7, the prediction unit 310 includes a feature separation unit 311, a feature prediction unit 312, and a signal synthesis unit 313.

  Similar to the feature separation unit 130, the feature separation unit 311 extracts a feature component from the first predicted image signal 37 from the motion compensation unit 305 to generate a first feature signal 39, and also generates a first feature signal 39 from the first predicted image signal 37. A feature removal signal 40 obtained by removing one feature signal 39 is generated. The first feature signal 39 is input to the feature prediction unit 312 and the feature removal signal 40 is input to the signal synthesis unit 313. The feature separation unit 311 may have the same configuration as the feature separation unit 130 shown in any of FIGS. 2, 3A, or 3B, or may have another configuration.

  The feature prediction unit 312 generates the feature prediction signal 41 from the first feature signal 39 from the feature separation unit 311 using the auxiliary information 32 from the entropy decoder 301 and passes it to the signal synthesis unit 313.

  The signal synthesis unit 313 synthesizes the feature removal signal 40 from the feature separation unit 311 and the feature prediction signal 41 from the feature prediction unit 312 to generate a second predicted image signal 38 and passes it to the adder 303.

The decoding control unit 320 controls the entire decoding unit 300 including control of decoding timing.
Next, the operation of the prediction unit 310 will be described with reference to FIG.
First, the feature separation unit 311 acquires the first predicted image signal 37 from the motion compensation unit 305 (step S401). Next, the feature separation unit 311 extracts the first feature signal 39 from the first predicted image signal 37 acquired in step S401 (step S402). Next, the feature separation unit 311 removes the first feature signal 39 acquired in step S402 from the first predicted image signal 37 acquired in step S401, and generates a feature removal signal 40 (step S403). Here, the order in which step S402 and step S403 are performed is not limited to the above. For example, in the case of the feature separation unit 311 similar to the feature separation unit 130 illustrated in FIG. 3A, the order in which steps S402 and S403 are performed may be switched, or the feature separation unit similar to the feature separation unit 130 illustrated in FIG. 3B. If it is 311, step S402 and step S403 may be performed simultaneously.

  Next, the feature prediction unit 312 acquires the auxiliary information 32 from the entropy decoder 301 (step S404). Next, the feature prediction unit 312 generates the feature prediction signal 41 using the auxiliary information 32 acquired in step S404 and the first feature signal 39 extracted in step S402 (step S405).

  Next, the signal synthesis unit 313 synthesizes the feature removal signal 40 generated in step S403 and the feature prediction signal 41 generated in step S405 to generate the second predicted image signal 38 (step S406).

  As described above, in the video decoding device according to the present embodiment, the second feature of the original image signal is used by using the first feature signal extracted from the first predicted image signal and the auxiliary information restored from the encoded data. A feature prediction signal is generated by predicting a signal, and a feature prediction signal is synthesized with a feature removal signal obtained by removing the first feature signal from the first prediction image signal to generate a second prediction image signal, and restored from the encoded data The decoded residual signal and the second predicted image signal are added to generate a decoded image signal. Therefore, according to the video decoding device according to the present embodiment, compared with the case where the first predicted image signal and the prediction residual signal are added to generate a decoded image signal, the reproducibility of the feature signal is improved, The prediction residual can be reduced.

  Note that the moving picture decoding apparatus according to the present embodiment can also be realized by using, for example, a general-purpose computer apparatus as basic hardware. That is, the entropy decoder 301, the inverse transform / inverse quantization unit 302, the adder 303, the motion compensation unit 305, the prediction unit 310, and the decoding control unit 320 execute a program on a processor mounted on the computer device. This can be realized. At this time, the video decoding device according to the present embodiment may be realized by installing the above program in a computer device in advance, or may be stored in a storage medium such as a CD-ROM or via a network. The above program may be distributed, and this program may be installed in a computer device as appropriate. The frame memory 304 is realized by appropriately using a memory, a hard disk or a storage medium such as a CD-R, a CD-RW, a DVD-RAM, a DVD-R, etc., which is built in or externally attached to the computer device. Can do.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. Further, for example, a configuration in which some components are deleted from all the components shown in each embodiment is also conceivable. Furthermore, you may combine suitably the component described in different embodiment.

  In addition, it goes without saying that the present invention can be similarly implemented even if various modifications are made without departing from the gist of the present invention.

1 is a block diagram showing a moving image encoding apparatus according to a first embodiment. The block diagram which shows the estimation part of FIG. The block diagram which shows the modification of the characteristic separation part of FIG. The block diagram which shows the other modification of the characteristic separation part of FIG. The flowchart which shows operation | movement of the estimation part of FIG. The figure which shows an example of an original image signal. The figure which shows an example of the 2nd feature signal extracted from the original image signal of FIG. 5A. The figure which shows an example of the 1st prediction image signal which performed motion estimation and motion compensation to the original image signal of FIG. 5A. The figure which shows an example of the 1st feature signal extracted from the 1st prediction image signal of FIG. 5C. The figure which shows an example of the feature removal signal which removed the 1st feature signal of FIG. 5D from the 1st prediction image signal of FIG. 5C. The figure which shows the edge centerline of the 1st feature signal of FIG. 5D, and the 2nd feature signal of FIG. 5B. The figure which shows an example of how to obtain | require the intensity distribution of an edge. The graph which shows intensity distribution of the 1st feature signal of Drawing 5D, and the 2nd feature signal of Drawing 5B. The figure which shows an example of the feature prediction signal produced | generated using the 1st feature signal and auxiliary information of FIG. 5D. The figure which shows an example of the 2nd prediction image signal which synthesize | combined the feature removal signal of FIG. 5E, and the feature prediction signal of FIG. 5I. The block diagram which shows the moving image decoding apparatus which concerns on 2nd Embodiment. The block diagram which shows the estimation part of FIG. The flowchart which shows operation | movement of the estimation part of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Original image signal 11 ... Prediction residual signal 12 ... Quantized transform coefficient 13 ... Decoded prediction residual signal 14 ... Encoded data 15 ... Local decoded image signal 16 ... Reference image signal 17 ... First predicted image signal 18 ... Motion vector 19 ... Second predicted image signal 20 ... Auxiliary information 21 ... First feature signal 22 ... Feature removal Signal 23 ... Second feature signal 24 ... Feature prediction signal 30 ... Encoded data 31 ... Quantized transform coefficient 32 ... Auxiliary information 33 ... Motion vector 34 ... Decoding Prediction residual signal 35 ... Decoded image signal 36 ... Reference image signal 37 ... First prediction image signal 38 ... Second prediction image signal 39 ... First feature signal 40 ... Feature removal Signal 41 ... feature prediction signal 100 ... encoding unit DESCRIPTION OF SYMBOLS 101 ... Subtractor 102 ... Transformation / quantization part 103 ... Inverse transformation / inverse quantization part 104 ... Entropy encoder 105 ... Adder 106 ... Frame memory 107 ... Motion estimation / compensation unit 110 ... prediction unit 121 ... feature extraction unit 122 ... auxiliary information generation unit 130 ... feature separation unit 131 ... feature extraction unit 132 ... subtractor 133 ... Smoothing filter 134: Subtractor 135: Band division unit 141 ... Feature prediction unit 142 ... Signal synthesis unit 150 ... Coding control unit 300 ... Decoding unit 301 ... Entropy decoder 302 ... Inverse transform / inverse quantization unit 303 ... Adder 304 ... Frame memory 305 ... Motion compensation unit 310 ... Prediction unit 311 ... Feature separation unit 312 ... ·Characteristic Measuring unit 313 ... signal synthesizer 320 ... decryption controller

Claims (11)

An extraction unit for extracting the first edge component image from the original image;
A separation unit that separates the reference image into a second edge component image and an edge-removed image;
An auxiliary information generating unit that generates auxiliary information for predicting the first edge component image from the second edge component image;
A prediction unit that predicts a third edge component image from the second edge component image using the auxiliary information;
A predicted image generation unit that generates a predicted image by combining the edge-removed image and the third edge component image;
A prediction residual calculation unit for obtaining a prediction residual between the original image and the prediction image;
An encoding unit for encoding the prediction residual and the auxiliary information;
An image encoding apparatus comprising:   The auxiliary information includes information indicating at least one difference among shape, intensity, and spread between a first edge included in the first edge component image and a second edge included in the second edge component image. The image coding apparatus according to claim 1. The separator is
A second extraction unit for extracting the second edge component image from the reference image;
The image coding apparatus according to claim 1, further comprising: a subtractor that subtracts the second edge component image from the reference image to generate the edge removed image. The separator is
A smoothing filter for extracting the edge-removed image from the reference image;
The image coding apparatus according to claim 1, further comprising: a subtracter that subtracts the edge-removed image from the reference image to generate the second edge component.   The separating unit divides the reference image into a high-frequency component and a low-frequency component, outputs the high-frequency component as the second edge component image, and outputs the low-frequency component as the edge-removed image. The image encoding device according to claim 1. A decoding unit that decodes input encoded data to obtain a prediction residual of a target image and auxiliary information for predicting a first edge component image of the target image;
A separation unit that separates a reference image that has already been decoded into a second edge component image of the reference image and an edge-removed image obtained by removing the second edge component from the reference image;
A prediction unit that predicts the first edge component image from the second edge component image using the auxiliary information;
A combining unit that combines the edge-removed image and the second edge component image to generate a predicted image;
A decoded image generating unit that generates a decoded image of the target image using the prediction residual and the predicted image;
An image decoding apparatus comprising:   The auxiliary information includes information indicating at least one difference among shape, intensity, and spread between a first edge included in the first edge component image and a second edge included in the second edge component image. The image decoding apparatus according to claim 6. Extract the first edge component image from the original image,
Separating the reference image into a second edge component image and an edge-removed image;
Generating auxiliary information for predicting the first edge component image from the second edge component image;
Predicting a third edge component image from the second edge component image using the auxiliary information;
Combining the edge-removed image and the third edge component image to generate a predicted image;
Obtaining a prediction residual between the original image and the predicted image;
The image encoding method, wherein the prediction residual and the auxiliary information are encoded.   The auxiliary information includes information indicating at least one difference among shape, intensity, and spread between a first edge included in the first edge component image and a second edge included in the second edge component image. The image encoding method according to claim 8. Decoding the input encoded data, obtaining the prediction residual of the target image and auxiliary information for predicting the first edge component image of the target image;
Separating the already decoded reference image into a second edge component image of the reference image and an edge-removed image obtained by removing the second edge component from the reference image;
Predicting the first edge component image from the second edge component image using the auxiliary information;
Combining the edge-removed image and the second edge component image to generate a predicted image;
A decoded image that generates a decoded image of the target image using the prediction residual and the predicted image is generated.   The auxiliary information includes information indicating at least one difference among shape, intensity, and spread between a first edge included in the first edge component image and a second edge included in the second edge component. The image decoding method according to claim 10.

Images