JP2006108811A - Image encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize spatial scalability easily in compression encoding of a motion picture. <P>SOLUTION: The image encoder comprises a wavelet transform section 3 for transforming an image hierarchically into spatial frequency components, a section 4 for quantizing the spatial frequency components obtained by transformation, a section 6 for dequantizing the quantized spatial frequency components, a wavelet inverse transform section 7 performing inverse transformation of the dequantized spatial frequency components up to a hierarchy set at the inverse transform hierarchy level setting section 8, and a section 12 performing movement prediction using a reference image generated by enlarging the low frequency component of the hierarchy set at the inverse transform hierarchy level setting section 8 at a reference image enlarging section 9. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image coding method that can easily realize spatial scalability in compression coding of moving images.

In moving images, MPEG-4 and H.264 are used. Standardization of compression coding schemes that achieve high compression efficiency such as H.264 has been mainly performed so far. In the future, it is expected that attention will be focused on standardization of compression methods that emphasize scalability while maintaining high compression efficiency. In Patent Document 1, a discrete wavelet transform (DWT: Discrete Wavelet Transform) that can easily achieve both high compression efficiency and scalability is used as a transform coding method, and a moving image using motion predictive coding as a predictive coding method. An encoding method is disclosed.
JP-A-8-182001

  However, in the moving image encoding method disclosed in Patent Document 1, when performing motion prediction encoding, motion prediction is performed on an original image to be encoded, and a motion vector is detected and encoded. For this reason, when decoding this encoded image, it is necessary to decode up to the resolution of the original image. That is, despite the fact that DWT, which is easy to scale, is used as the transform coding method, the motion predictive coding method does not have spatial scalability, so that it is easy to realize spatial scalability as a whole moving image coding method. There was a problem that it was not possible.

  The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide an image encoding method that can easily realize spatial scalability in the compression encoding of moving images.

  One embodiment of the present invention relates to an image encoding method. The method includes a first step of dividing an image into a plurality of spatial frequency bands, and a second step of further dividing a lowest frequency band among the divided spatial frequency bands into a plurality of spatial frequency bands. A reference image is generated from an arbitrary frequency band component obtained by the first step or the second step, and motion prediction is performed on an image input at a time different from the image, and the motion prediction is performed. The result of is encoded.

  According to this aspect, since the reference image used for motion prediction uses the lowest frequency band component obtained in the first step or the second step, when decoding an image encoded by this method, If at least the spatial frequency band component that generated the reference image can be decoded, the subsequent decoding can be continued. Thereby, in order to decode an image with an arbitrary image size from the encoded image, it is possible to stop decoding one image constituting the moving image halfway and start decoding the next image. Therefore, the image encoding device according to this aspect can easily realize spatial scalability.

  In this aspect, the arbitrary frequency band may be the lowest frequency band among the divided spatial frequency bands. In general, when an image is divided into spatial frequency bands, a large amount of information is collected in the lowest frequency band. Therefore, if the lowest frequency band is further divided into a plurality of spatial frequency bands, the image can be compressed with higher efficiency.

  In this aspect, the motion prediction result may be encoded. Thereby, it is possible to acquire the result of motion prediction on the decoding side and perform motion compensation based on the result.

  In this aspect, the second step may be repeated a predetermined number of times. As a result, an image obtained by repeatedly dividing the lowest frequency band into a plurality of spatial frequency bands can be obtained, and even when an image divided into a plurality of spatial frequency bands is encoded by such a method, spatial scalability is improved. It can be easily realized. In this aspect, information indicating in which step the low-frequency band component that generated the reference image is obtained may be added to the encoded data. As a result, it is possible to inform the decoding side of the hierarchical level necessary for generating the reference image set on the encoding side.

  In this aspect, the motion prediction may be performed after enlarging the reference image to the same resolution as the image input at the different time. As a result, accurate motion prediction is possible.

  In this aspect, the motion prediction may be performed after reducing the image input at the different time to the same resolution as the reference image. Thereby, the amount of calculation of a motion estimation part can be reduced, and cost reduction and low power consumption can be achieved.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, etc. are also effective as an aspect of the present invention.

  Spatial scalability can be easily realized in compression encoding of moving images.

(Embodiment 1)
FIG. 1 is a configuration diagram of an image encoding device 100 according to Embodiment 1. The original image OI input to the image encoding device 100 is a moving image composed of a series of frames, and is compressed and encoded for each frame by the image encoding device 100 to output an encoded image CI. . The configuration of the image processing apparatus 100 can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and is realized in software by a program having a decoding function loaded in the memory. Here, the functional blocks realized by the cooperation are depicted. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  When a frame constituting the original image OI is input, the image encoding apparatus 100 compresses and encodes the frame by intraframe encoding or interframe encoding. When performing intra-frame coding, SW1 is connected to the Intra node, and the original image OI is input to the wavelet transform unit 3 and hierarchized as described later. On the other hand, when performing interframe coding, SW1 is connected to the Inter node. At this time, the original image OI is input to the subtractor 2, and a difference (referred to as a prediction error) between the original image OI and a predicted image obtained by the motion prediction unit 12 described later is calculated. This prediction error is input to the wavelet transform unit 3 and hierarchized.

  The wavelet transform unit 3 uses a Doubechies filter, and as shown in FIG. 2, this filter acts as a low-pass filter and a high-pass filter in the x and y directions of the image at the same time. Divide into 4 frequency subbands. These subbands are LL subbands having low frequency components in both x and y directions, and HL subbands having low frequency components in either one of x and y directions and high frequency components in the other direction. And an HH subband having a high frequency component in both the x and y directions. The number of vertical and horizontal pixels of each subband is ½ that of the image before processing, and a subband image having a resolution, that is, a resolution of ¼ is obtained by one filtering.

  The wavelet transform unit 3 performs the filtering process again on the LL subband among the subbands thus obtained, and further divides it into four subbands of LL, HL, LH, and HH. Filtering is performed a predetermined number of times, and the LL subband generated by the last filtering is acquired as an image closest to the DC component in the original image OI. Hereinafter, each subband of the same layer, that is, four subbands obtained by performing the same number of times of filtering, include low frequency components in the order of HH, HL and LH, LL, and lower frequency components as the layer becomes deeper. An image including is generated. In the example of FIG. 2, a hierarchized image is obtained as an image including a low frequency component in order from the lower right to the upper left.

  Here, the hierarchized image obtained when the filtering process is performed n times is referred to as an nth layer level image. In the example of FIG. 2, the image WI1 obtained by performing the filtering process once is an image at the first hierarchy level, the image WI2 obtained by performing the filtering process twice is the image at the second hierarchy level, and 3 times. The image WI3 obtained by performing the filtering process is referred to as an image at the third hierarchical level.

  The quantization unit 4 quantizes the spatial frequency component output from the wavelet transform unit 3. The encoding unit 5 encodes the quantized spatial frequency data using, for example, a bit plane encoding and arithmetic encoder, and rearranges the encoded data in a predetermined order.

  On the other hand, in order to generate a reference image for performing motion prediction encoding, the spatial frequency component quantized by the quantizing unit 4 is dequantized by the inverse quantizing unit 6 and then the wavelet inverse transforming unit 7. Is input. The wavelet inverse transform unit 7 inversely transforms the hierarchized image in a flow reverse to that in FIG. For example, when the image WI3 hierarchized to the third hierarchy level is input to the wavelet inverse transformation unit 7, first, inverse transformation is performed on the four subbands LL, 3LH, 3HH, and 3HH, and the second hierarchy level is obtained. Image WI2 is obtained. Subsequently, inverse transformation is performed on the four subbands LL, 2LH, 2HL, and 2HH of the image WI2 at the second hierarchy level to obtain the image WI1 at the first hierarchy level. Finally, inverse transformation is performed on the four subbands LL, LH, HL, and HH of the image WI1 of the first hierarchy level, and the image of the original image or the prediction error is decoded.

  The wavelet inverse transformation unit 7 performs inverse transformation up to the hierarchical level set by the inverse transformation hierarchical level setting unit 8. For example, when an image layered up to the third hierarchy level is input to the wavelet inverse transformation unit 7 and the inverse transformation hierarchy level setting unit 8 is set to perform inverse transformation up to the first hierarchy level, the wavelet inverse transformation is performed. The conversion unit 7 ends the reverse conversion process when the first layer level image is obtained. In addition, when the inverse transformation hierarchy level setting unit 8 is set to perform the inverse transformation up to the third hierarchy level, the image input to the wavelet inverse transformation unit 7 is hierarchized to the third hierarchy level. The wavelet inverse transform unit 7 ends the process without performing the inverse transform. The wavelet inverse transform unit 7 sends the LL subband components at the hierarchical level obtained at the stage of processing to the reference image enlargement unit 9.

  The hierarchy level set by the inverse conversion hierarchy level setting unit 8 indicates a hierarchy level necessary for generating a reference image. Which layer level LL component is used is determined by the balance between the range of spatial scalability and the code amount. That is, when the spatial scalability range is increased so that it can be easily decoded from a small resolution to a large resolution, the reference image may be generated from the LL component of the deepest hierarchical level. However, in this case, since the accuracy of the reference image is deteriorated, there is a problem that the prediction error increases and the code amount increases. Conversely, in order to reduce the code amount, it is preferable to generate a reference image from hierarchical LL components having a size as close as possible to the original image. However, in this case, the range of spatial scalability becomes narrow.

  Therefore, the inverse transform layer level setting unit 8 considers the balance between the range of spatial scalability and the coding amount, and the resolution of the original image, the depth of the layer level converted by the wavelet transform unit 3, or the image coding device. It is automatically determined according to the processing capacity of 100 or the like. Alternatively, it may be set manually by the user.

The reference image enlargement unit 9 enlarges the input LL subband component so as to have the same resolution as that of the original image OI. When the LL subband component at the nth layer level is input to the reference image enlargement unit 9, the resolution of the LL subband component is 1/2 ⁿ of the resolution of the original image. Therefore, enlargement processing is performed on the LL subband component so that the resolution is 2 ⁿ . The layer level of the input LL subband component is acquired from the inverse transformation layer level setting unit 8.

  If the data enlarged by the reference image enlargement unit 9 is obtained by decoding a prediction error obtained by intra-frame coding, this data is input to the adder 10 and the prediction used in calculating this prediction error. It is added to the image. At this time, SW2 is connected to the Inter node and stores the image obtained by the adder 10 in the frame memory 11 as a new reference image. On the other hand, when the data enlarged by the reference image enlargement unit 9 is obtained by decoding an image obtained by inter-frame coding, SW2 is connected to the Intra node, and this enlarged image is used as a new reference image as a frame. Store in the memory 11.

  When performing intra-frame coding on the frame of the original image OI input to the image coding apparatus 100, the motion prediction unit 12 compares the frame with the reference image stored in the frame memory 11 to perform motion prediction. The prediction image is output to the subtracter 2 and the adder 10. Also, the motion vector information is output to the encoding unit 5. The reference image may be a frame input to the image encoding device 100 one frame or several frames before the frame to be encoded, or may be a frame input one frame or several frames later. Good. Alternatively, a reference image may be generated from these multiple frames.

  As described above, the encoding unit 5 encodes the spatial frequency data quantized by the quantization unit 4, rearranges the encoded data in a predetermined order, and the frame to be encoded is compressed within the frame or between frames. Information indicating which one has been compressed, and, when inter-frame compression is performed, the layer level necessary for generating the reference image set by the inverse transformation layer level setting unit 8, the motion vector obtained by the motion prediction unit 12, and the motion A code data stream is generated by adding a parameter indicating a prediction direction and a coding condition such as a frame used for a reference image. One hierarchical level may be added to a stream (or sequence), one may be added to a set of frames (for example, GOP), or may be added to each frame. Thus, a final encoded image CI (Coded Image) is output.

  Based on such a configuration, the operation of the image coding apparatus 100 shown in FIG. 1 will be described below. When a frame constituting the original image OI is encoded by intra-frame compression, this frame is input to the wavelet transform unit 3 via the Intra node of SW1. Here, the encoding target frame is divided into subbands that are hierarchized into a plurality of stages. As for the data obtained by the wavelet transform unit 3, a coded image CI is generated via the quantization unit 4 and the coding unit 5.

  The data quantized by the quantization unit 4 is inversely quantized by the inverse quantization unit 6 to generate a reference image, and wavelet inverse transformation is performed by the wavelet inverse transformation unit 7. At this time, the wavelet inverse transformation unit 7 performs inverse transformation up to the hierarchical level set by the inverse transformation hierarchical level setting unit 8. Then, in order to use the LL subband component of this hierarchy as a reference image for motion prediction, the LL subband component is enlarged by the reference image enlargement unit 9, and the enlarged LL subband component is set to the Intra node of SW2. And stored in the frame memory 11.

  On the other hand, when a frame constituting the original image OI is encoded by inter-frame compression, this frame is sent to the subtracter 2, and the prediction image generated by the motion prediction unit 12 is subtracted to output a prediction error. . This prediction error is divided into subbands hierarchized into a plurality of stages by the wavelet transform unit 3 via the Inter node of SW1. As for the data obtained by the wavelet transform unit 3, a coded image CI is generated via the quantization unit 4 and the coding unit 5.

  The data quantized by the quantization unit 4 is inversely quantized by the inverse quantization unit 6 to generate a reference image, and wavelet inverse transformation is performed by the wavelet inverse transformation unit 7. At this time, the wavelet inverse transformation unit 7 performs inverse transformation up to the hierarchical level set by the inverse transformation hierarchical level setting unit 8. Then, the LL subband component is expanded by the reference image enlargement unit 9 in order to add the LL subband component of this hierarchy and the prediction image generated by the motion prediction unit 12 by the adder 10 to generate a reference image. Is done. The reference image generated by the adder 10 is stored in the frame memory 11 via the Inter node of SW2.

  The motion prediction unit 12 reads a reference image generated based on the LL subband component at the hierarchical level set by the inverse transformation hierarchical level setting unit 8 from the frame memory 11 and performs motion prediction.

  Next, an apparatus for decoding the encoded image CI obtained by the image encoding apparatus 100 of FIG. 1 will be described. The image decoding apparatus 200 illustrated in FIG. 3 is an example of an apparatus for decoding the image CI encoded by the image encoding apparatus 100. The image decoding apparatus 200 inputs the encoded image CI to the decoding unit 51, and decodes a parameter indicating an encoding condition added to the encoded image CI and quantized spatial frequency data. The parameters indicating the coding conditions include a hierarchical level necessary for generating a reference image, a motion vector, a motion prediction direction, and a frame used for the reference image. The hierarchical level necessary for generating the reference image is input to the wavelet inverse transformation unit 53, the reference image enlargement / reduction unit 56, and the motion compensation unit 59. The motion vector, the motion prediction direction, and the frame used for the reference image are input to the motion compensation unit 59.

  The quantized spatial frequency data is inversely quantized by the inverse quantization unit 52 and then input to the wavelet inverse transform unit 53. The decoded image hierarchy level setting unit 54 sets the hierarchy level of the decoded image in accordance with the state of the image decoding device 100, the number of pixels of the display device that displays the decoded image, and the like. The wavelet inverse transform unit 53 performs inverse transform up to the layer level set by the decoded image layer level setting unit 54. At this time, when the inverse transformation is performed up to an intermediate hierarchical level, the LL component is sent to SW3 or the adder 55 as a result of the inverse transformation. Further, when the inverse transformation is performed up to the resolution of the original image, all the inverse transformation results are sent to SW3 or the adder 55.

  When the inversely transformed frame is intraframe coding, the SW3 is connected to the Intra node, and the result of inverse transformation by the wavelet inverse transformation unit 53 is output as it is as a decoded image DI. In the case of inter-frame coding, SW3 is connected to the Inter node, and an adder 55 adds the result of inverse transformation by the wavelet inverse transformation unit 53 and a prediction image obtained by a motion compensation unit 59 described later. The result is output as a decoded image DI.

  The wavelet inverse transform unit 53 inputs the LL component at the hierarchical level to the reference image enlargement / reduction unit 56 based on the hierarchical level necessary for the reference image input from the decoding unit 51. At this time, if the hierarchical level necessary for the reference image is equal to or smaller than the hierarchical level of the decoded image, the hierarchical level image necessary for the reference image is obtained at the time of decoding. On the other hand, when the hierarchical level required for the reference image is higher than the hierarchical level of the decoded image, the image of the hierarchical level required for the reference image cannot be obtained at the time of decoding. Perform reverse transformation to the required hierarchy level.

  The reference image enlargement / reduction unit 56 compares the hierarchy level of the decoded image with the hierarchy level necessary for the reference image, and enlarges / reduces the LL component input from the wavelet inverse transformation unit 53 to be the same as the resolution of the decoded image. To do. For example, when the hierarchical level required for the reference image is the second hierarchical level WI2 in FIG. 2, the LL component of the second hierarchical level WI2 is input from the wavelet inverse transformation unit 53. When the hierarchical level of the decoded image is the first hierarchical level WI1 in FIG. 2, the LL component input from the wavelet inverse transformation unit 53 is enlarged twice in both vertical and horizontal directions. When the hierarchical level of the decoded image is the original image level OI of FIG. 2, the LL component input from the wavelet inverse transform unit 53 is expanded by 4 times in both the vertical and horizontal directions. On the other hand, when the hierarchical level of the decoded image is the third hierarchical level WI3 in FIG. 2, the LL component input from the wavelet inverse transformation unit 53 is reduced to 1/2 in both vertical and horizontal directions.

  When the frame being decoded is intra-frame coding, SW 4 is connected to the Intra node, and the result of the reference image enlargement / reduction unit 56 is stored in the frame memory 58. When the frame being decoded is interframe coding, SW4 is connected to the Inter node, and the result of the reference image enlargement / reduction unit 56 and the predicted image obtained by the motion compensation unit 59 described later are added to the adder 57. And the result is stored in the frame memory 58.

  The motion compensation unit 59 receives the motion vector, the motion prediction direction, and the frame information used for the reference image from the decoding unit 51, reads out a necessary image from the frame memory 58, and generates a prediction image. The predicted image is sent to the adders 55 and 57.

  Based on such a configuration, the operation of the image coding apparatus 200 shown in FIG. 3 will be described below. When the encoded image CI is input, the decoding unit 51 first determines whether the frame to be decoded has been subjected to intra-frame compression or inter-frame compression. SW4 is switched to the Intra node, and if it is compressed between frames, SW3 and SW4 are switched to the Inter node.

  The decoding unit 51 further decodes the quantized coefficients, is inversely quantized by the inverse quantization unit 52, and is then inversely wavelet transformed by the wavelet inverse transform unit 53. At this time, the wavelet inverse transform unit 53 performs inverse transform to the layer level set by the decoded image layer level setting unit 54, and when SW3 is switched to the Intra node, the LL subband component of that layer is decoded. Output as an image DI. Further, when SW3 is switched to the Inter node, the adder 55 adds the LL subband component of the layer and the prediction image generated by the motion compensation unit 59, and then the added data is the decoded image. Output as DI.

  On the other hand, in order to generate a reference image for motion compensation, the reference image enlargement / reduction unit 56 converts the LL subband components at the hierarchical level necessary for generating the reference image decoded by the decoding unit 51 into the wavelet inverse conversion unit 53. And scaling is performed so as to be the same as the resolution of the decoded image. When SW4 is switched to the Intra node, the enlarged / reduced LL subband component is stored in the frame memory 58 as a reference image. When SW4 is switched to the Inter node, the adder 57 adds the predicted image generated by the motion compensation unit 59 to the scaled LL subband component, and the added data is It is stored in the frame memory 58 as a reference image.

  The motion compensation unit 59 generates a prediction image from the reference image generated based on the motion vector decoded by the decoding unit 51 and the LL subband component of the hierarchical level necessary for generating the reference image, and adds the adder 55 and the adder 57.

  As described above, according to the configuration relating to the image encoding device 100, the image encoding device 100 does not return the layered image to the resolution of the original image, but the intermediate layer level (that is, the inverse conversion layer level setting unit 8). The reference image is generated using the LL component of the set hierarchical level). Therefore, even when the encoded image CI output by the image encoding device 100 is decoded by the image decoding device 200, a reference image can be generated by using an LL component at a mid-level obtained by wavelet inverse transformation. it can. This is because, for example, when the processing capability of the image decoding device 200 is small or the resolution of the display device that displays the decoded image is small, the image decoding device can generate a reference image even if wavelet inverse transformation is stopped halfway. Decoding can be continued if it is inversely converted to a level or higher.

  As described above, the image encoding apparatus 100 can easily realize spatial scalability when decoding the encoded image output thereby.

(Embodiment 2)
FIG. 4 is a configuration diagram of the image encoding device 110 according to the second embodiment. This image encoding device 110 has a configuration in which the reference image enlargement unit 9 of the image encoding device 100 according to Embodiment 1 is removed and an original image reduction unit 14 and a predicted image enlargement unit 15 are added instead. The same components as those in the first embodiment are denoted by the same reference numerals and description thereof is omitted.

The image encoding device 110 performs motion prediction with the resolution of the LL subband component at the hierarchical level set by the inverse transform level setting unit 8. The original image reduction unit 14 reduces the original image to the resolution of the LL subband component at the hierarchical level set by the inverse conversion level setting unit 8. For example, if the layer level set by the conversion level setting unit 8 is the nth layer level, the original image reduction unit 14 reduces the image to 1 / ²ⁿ .

The predicted image enlarging unit 15 performs an enlarging process in order to match the resolution of the predicted image obtained by the motion predicting unit 12 with the resolution of the original image. In other words, the resolution of the prediction image obtained by the motion prediction unit 12 is enlarged based on the set hierarchical level because the motion prediction is the resolution of the hierarchical level set by the inverse transformation hierarchical level setting unit 8. . For example, if the hierarchical level set by the inverse transformation hierarchical level setting unit 8 is the n-th hierarchical level, the predicted image enlarging unit 15 performs an enlargement process so that the resolution of the predicted image becomes 2 ⁿ times. The enlarged predicted image is input to the subtracter 2.

  Based on such a configuration, the operation of the image encoding device 110 shown in FIG. 4 will be described below. When generating a reference image from the quantization result of the quantization unit 4, after inverse quantization by the inverse quantization unit 6, the wavelet inverse transform unit 7 performs inverse processing up to the layer level set by the inverse transform layer level setting unit 8. Converted. If the frame being processed is an intra-frame compressed one, SW2 is connected to the Intra node, and the LL component at the hierarchical level obtained by the wavelet inverse transform unit 7 is stored in the frame memory 11 as it is. Conversely, if the frame being processed is inter-frame compressed, SW2 is connected to the Inter node, and the LL component at the hierarchical level obtained by the wavelet inverse transform unit 7 in the adder 10 and the predicted image are obtained. After the addition, it is stored in the frame memory 11. Therefore, the resolution of the reference image stored in the frame memory 11 is not enlarged to the resolution of the original image, but becomes a resolution equivalent to the LL component at the hierarchical level set by the inverse transformation hierarchical level setting unit 8.

  The original image is reduced by the original image reduction unit 14 to a resolution corresponding to the LL component of the layer level set by the inverse conversion layer level setting unit 8.

  The motion prediction unit 12 compares the reference image stored in the frame memory 11 with the original image reduced by the original image reduction unit 14 and performs motion prediction. That is, motion prediction is performed at the resolution of the hierarchy level set by the inverse conversion hierarchy level setting unit 8. The motion prediction unit 12 sends the motion vector to the encoding unit 5 from the result of the motion prediction, and sends the predicted image generated based on the motion vector to the adder 10 and the predicted image enlargement unit 15.

  The predicted image enlarging unit 15 performs an enlarging process so that the resolution of the predicted image obtained by the motion predicting unit 12 matches the resolution of the original image, and the predicted image subjected to the enlarging process is input to the subtractor 2.

  FIG. 5 is a configuration diagram of the image decoding device 210 for decoding the encoded image CI obtained by the image encoding device 110. The image decoding apparatus 210 has a configuration in which the reference image enlargement / reduction unit 56 is removed from the image decoding apparatus 200 according to Embodiment 1 and a predicted image enlargement / reduction unit 61 is added instead. The same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  The predicted image enlargement / reduction unit 61 compares the hierarchical level required for the reference image with the hierarchical level of the decoded image, and enlarges / reduces the predicted image so that the resolution of the predicted image is the same as the resolution of the decoded image. For example, when the hierarchical level required for the reference image is the second hierarchical level WI2 in FIG. 2 and the hierarchical level of the decoded image is the first hierarchical level WI1 in FIG. 2, the predicted image is doubled both vertically and horizontally. Zoom in. Further, when the hierarchical level of the decoded image is the original image level OI in FIG. 2, the predicted image is enlarged four times in both the vertical and horizontal directions. On the other hand, when the hierarchical level of the decoded image is the third hierarchical level WI3 in FIG. 2, the predicted image is reduced to 1/2 in both vertical and horizontal directions.

  Based on such a configuration, the operation of the image coding apparatus 210 shown in FIG. 5 will be described below. The reference image stored in the frame memory 58 is not enlarged to the resolution of the original image, but is stored with the resolution of the LL component at the hierarchical level necessary for the reference image obtained by the wavelet inverse transform unit 53. Motion compensation is also performed at the resolution of the hierarchical level LL component required for the reference image by the motion compensation unit 59, and a predicted image having the resolution is output to the adder 57 and the predicted image scaling unit 61.

  Then, the predicted image enlargement / reduction unit 61 enlarges / reduces the predicted image so that the resolution of the predicted image is the same as the resolution of the decoded image, and sends the enlarged / reduced image to the adder 55.

  As described above, according to the configuration related to the image encoding device 110 and the image decoding device 210, the reference image is stored in the frame memory 11 or the frame memory 58 without being expanded to the resolution of the original image, and the motion prediction or Motion compensation can be performed. That is, it is possible to reduce the capacity of the frame memory, and it is possible to reduce the amount of calculation in motion prediction / compensation, although the accuracy is reduced. Therefore, in addition to the effects of the first embodiment, the cost and power consumption of the image encoding device and the image decoding device can be reduced. In addition, when the image is decoded, even if the hierarchical level of the decoded image (that is, the resolution of the decoded image) is changed during decoding, the reference image is always stored in the frame memory with the same resolution. Decoding can be continued correctly.

(Embodiment 3)
FIG. 6 is a configuration diagram of the image encoding device 120 according to Embodiment 3. The image encoding device 120 includes a reference image enlarging unit 9 of the image encoding device 100 according to Embodiment 1 between the wavelet inverse transform unit 7 and the adder 10, and between the frame memory 11 and the motion prediction unit 12. The predicted image reduction unit 16 is added to the image encoding apparatus 100. The same components as those in the first embodiment are denoted by the same reference numerals, and different configurations and operations from those in the first embodiment will be described.

  The image encoding device 120 generates a reference image with the resolution of the LL subband component of the hierarchical level set by the inverse transformation hierarchical level setting unit 8 and stores the reference image in the frame memory 11, and then the motion prediction unit Before performing motion prediction at 10, the reference image is enlarged to the same resolution as the original image.

When generating a reference image from a frame subjected to intraframe prediction encoding, the predicted image reduction unit 16 sets the resolution of the predicted image (corresponding to the resolution of the original image OI) in order to add the predicted image and the prediction error. The resolution of the prediction error (corresponding to the resolution of the hierarchy level set by the inverse conversion hierarchy level setting unit 8) is set. That is, when the hierarchical level set by the inverse transformation hierarchical level setting unit 8 is the n-th hierarchical level, the prediction image obtained by the motion prediction unit 12 is reduced by 1/2 ⁿ . The reduced predicted image is input to the adder 10, and a new reference image is generated.

  Based on such a configuration, the operation of the image encoding device 120 shown in FIG. 6 will be described below. When generating a reference image from the quantization result of the quantization unit 4, after inverse quantization by the inverse quantization unit 6, the wavelet inverse transform unit 7 performs inverse processing up to the layer level set by the inverse transform layer level setting unit 8. Converted. If the frame being processed is an intra-frame compressed one, SW2 is connected to the Intra node, and the LL component at the hierarchical level obtained by the wavelet inverse transform unit 7 is stored in the frame memory 11 as it is. Conversely, if the frame being processed is inter-frame compressed, SW2 is connected to the Inter node, and the LL component at the hierarchical level obtained by the wavelet inverse transform unit 7 in the adder 10 and the predicted image are obtained. After the addition, it is stored in the frame memory 11. Therefore, the resolution of the reference image stored in the frame memory 11 is not enlarged to the resolution of the original image, but becomes a resolution equivalent to the LL component at the hierarchical level set by the inverse transformation hierarchical level setting unit 8.

  The reference image enlargement unit 9 performs enlargement processing in the same manner as the reference image enlargement unit 9 of the image encoding device 100 according to Embodiment 1 in order to match the resolution of the reference image with the resolution of the original image. However, in the case of the image encoding device 100, the result obtained by the wavelet inverse transform unit 7 is enlarged, whereas the reference image enlargement unit 9 of the image encoding device 120 according to Embodiment 3 is a frame memory. 11 is different in that the reference image stored in 11 is enlarged.

  The motion prediction unit 12 compares the original image OI and the reference image enlarged by the reference image enlargement unit 9 at the resolution level of the original image OI, and performs motion prediction. The motion vector thus obtained is sent to the encoding unit 5, and the predicted image is sent to the subtracter 2 and the predicted image reducing unit 16.

  As described above, the predicted image reduction unit 16 reduces the predicted image in order to match the resolution of the predicted image with the resolution of the prediction error (corresponding to the resolution of the hierarchical level set by the inverse transformation hierarchical level setting unit 8). Then, the reduced predicted image is input to the adder 10, and a new reference image is generated.

  FIG. 7 is a configuration diagram of an image decoding device 220 for decoding the encoded image CI obtained by the image encoding device 120. In this image decoding device 220, the reference image enlargement / reduction unit 56 of the image decoding device 200 according to Embodiment 1 is placed between the wavelet inverse transform unit 53 and the adder 57, and between the frame memory 58 and the motion compensation unit 59. And a predicted image scaling unit 62 is further added to the image decoding apparatus 200. The same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  The predicted image enlargement / reduction unit 62 performs an enlargement / reduction process on the predicted image having the layer level resolution set by the decoded image layer level setting unit 54 so as to match the layer level resolution required for the reference image. For example, when the hierarchical level required for the reference image is the second hierarchical level WI2 in FIG. 2 and the hierarchical level of the decoded image is the first hierarchical level WI1 in FIG. Reduce to Also, when the hierarchical level of the decoded image is the original image level OI in FIG. 2, the predicted image is reduced to ¼ both vertically and horizontally. On the other hand, when the hierarchical level of the decoded image is the third hierarchical level WI3 in FIG. 2, the predicted image is enlarged twice in both vertical and horizontal directions.

  Based on such a configuration, the operation of the image encoding device 120 shown in FIG. 6 will be described below. The reference image stored in the frame memory 58 is not enlarged to the resolution of the original image, but is stored with the resolution of the LL component at the hierarchical level necessary for the reference image obtained by the wavelet inverse transform unit 53. Before the motion compensation unit 59 performs motion compensation, the reference image enlargement / reduction unit 56 enlarges the reference image to the resolution of the layer level set by the decoded image layer level setting unit 54. The operation of the reference image enlargement / reduction unit 56 is the same as that of the reference image enlargement / reduction unit 56 of the image decoding apparatus 200. Also, the motion compensation unit 59 performs the same processing as the motion compensation unit 59 of the image decoding apparatus 200, outputs the prediction image obtained by the motion compensation unit 59 to the adder 55, and predicts the prediction error obtained by the wavelet inverse transformation unit 53. And the decoded image DI is generated.

  The predicted image is also sent to the predicted image enlargement / reduction unit 62, and as described above, the predicted image is enlarged / reduced to match the resolution of the hierarchical level required for the reference image. The predicted image enlarged / reduced by the predicted image enlargement / reduction unit 62 is sent to the adder 57 to generate a new reference image added with the prediction error obtained by the wavelet inverse transform unit 53.

  As described above, according to the configuration related to the image encoding device 120 and the image decoding device 220, the reference image is stored in the frame memories 11 to 58 at a low resolution without being expanded to the resolution of the original image. It can be performed at the resolution of the image, and motion compensation can be performed according to the resolution of the decoded image. That is, while the capacity of the frame memory can be reduced, the motion prediction / motion compensation can be performed with high accuracy. Therefore, in addition to the effects of the first embodiment, a low-cost and high-accuracy image encoding device and image decoding device can be realized. In addition, when decoding an image, even if the hierarchical level of the decoded image (that is, the resolution of the decoded image) is changed during decoding, the reference image is always stored in the same resolution, so the image is decoded correctly. Can continue.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to the combinations of the respective components and processing processes, and such modifications are also within the scope of the present invention. Such a modification is shown below.

  In the above embodiment, filtering processing may be performed on the generated reference image. This filtering process may be performed before the generated reference image is written in the frame memory, or may be performed after the reference image is read from the frame memory. Since the reference image generated in the above embodiment is an image in which high-frequency components are cut, many high-frequency components are included in the prediction error that is the difference between the predicted image generated from the reference image and the original image. There is a possibility. Therefore, there is a problem that the compression efficiency is lowered. Therefore, by performing filtering processing on the reference image and performing edge enhancement or the like, the high-frequency component of the prediction error can be reduced and the compression efficiency can be increased by supplementing the high-frequency component of the reference image.

  In the above embodiment, the discrete wavelet transform is used as the transform coding method. However, the present invention is not limited to this, and the object of the present invention can be achieved as long as the original image is layered and transformed. For example, there is a QMF (Quadrature Mirror Filter) as a method for hierarchizing original images.

  In the above-described embodiment, an example in which the hierarchical level of an image necessary for generating a reference image is added to the encoded image CI has been described. In other words, the hierarchical level of the image necessary for generating the reference image defines the number of wavelet inverse transformations necessary for generating the reference image. Therefore, the number of wavelet inverse transforms necessary for generating a reference image may be added to the encoded image CI. In the image decoding apparatus, the hierarchical level of the image necessary for generating the reference image is specified based on the number of times, and the hierarchical level image is extracted from the wavelet inverse transform unit 53 to generate the reference image.

  In the above embodiment, the hierarchical level necessary for generating the reference image can be set by the inverse transformation hierarchical level setting unit 8, but this hierarchical level is always the deepest layered by the wavelet transform unit 3. It may be a hierarchical level. In this case, the wavelet inverse transformation unit 7 may not be provided, and the deepest hierarchical LL component obtained by the inverse quantization unit 6 may be used to generate the reference image. Further, it is not necessary to add a hierarchical level necessary for generating a reference image in the encoding unit 5. When decoding this encoded image CI, the deepest LL component obtained by the inverse quantization unit 52 may be used to generate a reference image.

1 is a configuration diagram of an image encoding device 100 according to Embodiment 1. FIG. It is a figure for demonstrating wavelet transformation. 1 is a configuration diagram of an image decoding device 200 according to Embodiment 1. FIG. 6 is a configuration diagram of an image encoding device 110 according to Embodiment 2. FIG. 6 is a configuration diagram of an image decoding device 210 according to Embodiment 2. FIG. 10 is a configuration diagram of an image encoding device 120 according to Embodiment 3. FIG. It is a block diagram of the image decoding apparatus 220 which concerns on Embodiment 3. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 Subtractor 3 Wavelet transformation part 4 Quantization part 5 Encoding part 6 Inverse quantization part 7 Wavelet inverse transformation part 8 Inverse transformation hierarchy level setting part 9 Reference image expansion part 10 Adder 11 Frame memory 12 Motion prediction part 14 Original image Reduction unit 15 Prediction image enlargement unit 16 Prediction image reduction unit 51 Decoding unit 52 Inverse quantization unit 53 Inverse wavelet transform unit 54 Decoded image layer level setting unit 55 Adder 56 Reference image enlargement / reduction unit 57 Adder 58 Frame memory 59 Motion compensation Unit 61 Predictive Image Enlargement / Reduction Unit 62 Predictive Image Enlargement / Reduction Unit 100, 110, 120 Image Encoding Device 200, 210, 220 Image Decoding Device

Claims (7)

A first step of dividing the image into a plurality of spatial frequency bands;
A second step of further dividing an arbitrary frequency band of the divided spatial frequency bands into a plurality of spatial frequency bands,
An image encoding method for generating a reference image from the lowest frequency band component obtained in the first step or the second step, and performing motion prediction on an image input at a time different from the image. 3. The image encoding method according to claim 1, wherein the arbitrary frequency band is a lowest frequency band among the divided spatial frequency bands. The image encoding method according to claim 1, wherein the result of motion prediction is encoded. The image encoding method according to claim 1, wherein the second step is repeatedly performed a predetermined number of times. The image according to any one of claims 1 to 4, wherein information indicating by which step the low-frequency band component that generated the reference image is obtained is added to the encoded data. Encoding method. 6. The image encoding method according to claim 1, wherein the motion prediction is performed after enlarging the reference image to the same resolution as an image input at the different time. The image coding method according to claim 1, wherein the motion prediction is performed after reducing the image input at the different time to the same resolution as the reference image.

Images