JP2009267573A - Moving picture hierarchical encoder, moving picture hierarchical decoder, moving picture hierarchical encoding method, moving picture hierarchical decoding method, moving picture hierarchical encoding program, and hierarchical decoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently perform encoding by reducing the amount of information of a prediction error signal when the prediction error signal of an image signal between encoding layers is encoded, and also by reducing the amount of encoding. <P>SOLUTION: The moving picture hierarchical encoder 101 includes: a space interpolation part 106 for enlarging a base layer local decoding signal from a base layer encoding part 105 to have the resolution of an enhancement layer; a high frequency component removing part 118 for removing, from an input image signal, a high frequency component estimating signal estimated based on the base layer local decoding signal from a high frequency component estimation enlarging part 117; and an enhancement layer encoding part 107 for obtaining a difference between the high frequency component removing signal from the high frequency component removing part 118 and an interpolation signal enlarged to have the resolution of the enhancement layer from the space interpolation part 106, and then, performing encoding by the enhancement layer. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving picture hierarchical coding apparatus, a hierarchical coding method, and a hierarchical coding for coding a moving picture signal using coding layers of two layers, such as two layers, three layers, and four layers having different spatial resolutions. The present invention relates to a moving picture hierarchical decoding apparatus, a hierarchical decoding method, and a hierarchical decoding program for decoding a coded bit stream in which a moving picture signal is hierarchically encoded by a plurality of encoding layers having different spatial resolutions.

Conventionally, many encoding schemes have been proposed for realizing the spatial resolution, temporal resolution, and SNR (signal to noise ratio) scalability in video encoding, and these have been put into practical use in various fields. In particular, with regard to scalability of spatial resolution, the application range is wide, including encoding of still images, and as a hierarchical encoding device and decoding device that realizes spatial resolution scalability of video, for example, base layer and enhancement layer 2 In a layer hierarchical coding apparatus, prediction using a correlation between a signal obtained by spatially interpolating a base layer decoded signal to obtain an enhancement layer spatial resolution and an enhancement layer video signal is performed. In some cases, the prediction error signal is encoded and the encoded bit stream is transmitted to a decoding device, and the decoding device decodes the encoded bit stream (see, for example, Patent Document 1).
JP-A-7-162870

  However, in an image frame prepared as a base layer encoding target, high frequency components such as those included in an image frame handled in the enhancement layer are limited by the frequency band when reducing (decimating) the spatial resolution of the base layer. Removed. Also, high frequency components that are determined to be less important as a result of encoding in the base layer are removed in the course of encoding processing such as quantization.

  For this reason, as in the prior art described in Patent Document 1, such a base layer encoded signal is decoded and further correlated (interpolated) with the signal having the spatial resolution of the enhancement layer. When the enhancement layer moving image signal is encoded, the prediction error signal obtained by performing the prediction process between the base layer signal and the enhancement layer signal is correct for a region containing many low-frequency components. There is a high possibility that prediction processing is performed, but in regions where relatively large high-frequency components are continuous, for example, in regions including edge portions, high-frequency components are removed from signals obtained from the base layer. There is a problem that sufficient prediction processing cannot be performed, the amount of information of the prediction error signal increases, and the amount of codes increases.

  Therefore, the present invention reduces the information amount of the prediction error signal when encoding the prediction error signal of the image signal between the encoding layers, and the moving image can be efficiently encoded by reducing the code amount. Image hierarchical coding apparatus, hierarchical coding method, hierarchical coding program, and moving picture hierarchical decoding apparatus, hierarchical decoding method, and hierarchical decoding capable of decoding an encoded bitstream encoded by such encoding To provide a program.

In order to solve the above problems, a moving image hierarchical encoding device of the present invention is a moving image hierarchical encoding device that generates and encodes at least a low resolution image signal and a high resolution image signal from an input moving image signal, A spatial decimation unit that performs a spatial reduction process on the input moving image signal to obtain a low resolution image signal having a lower resolution than the input moving image signal, and accumulates the input moving image signal as a high resolution image signal. A low-resolution signal holding unit and a low-resolution image signal are encoded to obtain low-resolution encoded data, and a local decoding signal of the low-resolution encoded data is generated by a local decoding process at the time of encoding. A resolution encoding unit, a spatial interpolation unit that spatially expands the local decoded signal obtained by the local decoding process to generate an expanded decoded image signal, and the station The high-frequency component estimation signal is generated from the high-resolution image signal, and the high-frequency component estimation signal is generated by performing an estimation process of the high-frequency component lost by the spatial reduction process in the spatial decimation unit on the decoded signal. A high-frequency component removed image signal generated by subtracting the high-frequency component removed image, and the high-frequency component-removed image signal is encoded using the enlarged decoded image signal generated by the spatial interpolation unit. A high-resolution encoding unit that obtains encoded data, and a multiplexing unit that multiplexes at least the low-resolution encoded data and the high-resolution encoded data.
Here, the high-frequency component estimation unit extracts a high-frequency component having a high correlation with the high-resolution image signal based on the local decoded signal from the low-resolution encoding unit, and an extraction A spatial interpolation unit that spatially expands the high-frequency component generated to the resolution of the high-resolution image signal, and an amplitude limitation that performs amplitude limitation and constant multiplication on the spatially expanded high-frequency component A constant multiplication processing unit and a second high-pass filtering unit that removes a low frequency component from the high frequency component subjected to amplitude limitation and constant multiplication processing and outputs the low frequency component to the high frequency component removal unit may be included. Yes. Further, the high frequency component estimator further includes the amplitude limiting / constant multiplication processor based on the input image signal and the high frequency component from which the low frequency component from the second high-pass filtering unit has been removed. Intensity parameters when performing amplitude limitation and constant multiplication processing are estimated within a specified range, and the estimated intensity parameters are output to the amplitude limitation and constant multiplication processing unit, and the specified range is output to the amplitude limitation / constant multiplication processing unit. The high-frequency component signal generated by all the estimated intensity parameters estimated within the specified range and the amplitude limiting and constant multiplication processing are performed on the high-frequency component based on the estimated intensity parameter An estimation degree determination unit that outputs an optimum estimation intensity parameter that minimizes a level difference from a high frequency component of the input image signal, and the optimum degree from the estimation degree determination unit An entropy encoding unit that entropy-encodes a constant intensity parameter, and the second high-pass filtering unit includes an amplitude limiter and a constant multiplier process based on the optimum estimated intensity parameter. The low-frequency component is removed from the high-frequency component that has been subjected to, and output to the high-frequency component removal unit, the multiplexing unit further multiplexes the optimum estimated intensity parameter that has been entropy encoded, Also good.
The moving image hierarchical encoding method of the present invention is a moving image hierarchical encoding method for generating and encoding at least a low-resolution image signal and a high-resolution image signal from an input moving image signal. Performing a spatial reduction process on the signal to obtain a low-resolution image signal having a resolution lower than that of the input moving image signal, storing the input moving image signal as a high-resolution image signal, and the low-resolution image signal To generate low resolution encoded data, and to generate a local decoded signal of low resolution encoded data by local decoding processing at the time of encoding, and the local obtained by the local decoding processing Spatially expanding the decoded signal to generate an enlarged decoded image signal, and erasing the local decoded signal by a spatial reduction process in the spatial decimation unit; Performing high frequency component estimation processing, generating a high frequency component estimation signal, generating a high frequency component removal image signal obtained by subtracting the high frequency component estimation signal from the high resolution image signal, and the high frequency component removal image A signal is encoded using the expanded decoded image signal generated by the spatial interpolation unit to obtain high-resolution encoded data, and at least multiplexing of the low-resolution encoded data and the high-resolution encoded data And a step of performing the conversion process.
The moving picture hierarchical coding program of the present invention is a moving picture hierarchical coding for causing a computer to execute an operation of generating and coding at least a low resolution image signal and a high resolution image signal from an input moving picture signal. A program for causing a computer to execute each step of the moving picture hierarchical encoding method.
The moving picture hierarchical decoding apparatus according to the present invention includes at least the low-resolution encoded data and the high-resolution encoded data from a multiplexed bit stream in which at least low-resolution encoded data and high-resolution encoded data are multiplexed. And a high-resolution image signal to generate a low-resolution image signal and a high-resolution image signal, and at least the low-resolution encoded data and the high-resolution encoded data from the multiplexed bitstream A demultiplexing unit that separates, and decodes the separated low resolution encoded data to generate the low resolution decoded image signal. The low resolution decoding unit and the low resolution decoded image signal are spatially expanded. A spatial interpolation unit that generates an expanded decoded image signal, and the expanded decoded image signal generated by the spatial interpolation unit. A high-resolution decoding unit that decodes the high-resolution encoded data and generates the high-resolution decoded image signal, and a high-frequency that has disappeared due to spatial reduction processing in the spatial decimation unit with respect to the low-resolution decoded image signal The high-resolution image obtained by restoring the high-frequency component by adding the high-frequency component estimation signal to the high-resolution image signal, and a high-frequency component estimation unit that performs component estimation processing and generates a high-frequency component estimation signal A high-frequency component restoration unit that generates a signal.
Here, the high-frequency component estimation unit extracts a high-frequency component having a high correlation with the high-resolution image signal based on the local decoded signal from the low-resolution encoding unit, and an extraction A spatial interpolation unit that spatially expands the high-frequency component generated to the resolution of the high-resolution image signal, and an amplitude limitation that performs amplitude limitation and constant multiplication on the spatially expanded high-frequency component A constant multiplication processing unit and a second high-pass filtering unit that removes a low frequency component from the high frequency component subjected to amplitude limitation and constant multiplication processing and outputs the low frequency component to the high frequency component removal unit may be included. . Further, the demultiplexing unit is entropy-coded and used when the low-resolution encoded data, the high-resolution encoded data, and the high-resolution encoded data are encoded from the multiplexed bit stream. The high-frequency component estimation unit further includes an entropy decoding unit that entropy-decodes the optimal estimation intensity parameter that has been entropy-coded, and the amplitude limiting / constant multiplication unit May perform amplitude limiting and constant multiplication processing on the high frequency components spatially expanded based on the optimum estimated intensity parameter subjected to entropy decoding.
The moving picture hierarchical decoding method according to the present invention includes at least the low-resolution encoded data and the high-resolution encoded data from a multiplexed bit stream in which at least the low-resolution encoded data and the high-resolution encoded data are multiplexed. The video hierarchical decoding method for generating a low-resolution image signal and a high-resolution image signal by decoding at least the low-resolution encoded data and the high-resolution encoded data from the multiplexed bitstream Separating the low-resolution encoded data, generating the low-resolution decoded image signal, spatially expanding the low-resolution decoded image signal, and generating an enlarged decoded image signal And using the expanded decoded image signal generated by the spatial interpolation unit, the high-resolution encoded data is converted into Generating the high-resolution decoded image signal, and performing estimation processing of the high-frequency component lost by the spatial reduction processing in the spatial decimation unit on the low-resolution decoded image signal to generate the high-frequency component estimation signal And a step of adding the high frequency component estimation signal to the high resolution image signal to generate the high resolution image signal in which the high frequency component is restored.
Also, the moving picture hierarchical decoding program of the present invention provides at least the low resolution encoded data and the high resolution encoded data from a multiplexed bit stream in which at least the low resolution encoded data and the high resolution encoded data are multiplexed. Is a moving picture hierarchy decoding program for causing a computer to execute an operation of generating a low resolution image signal and a high resolution image signal, and executing each step of the moving picture hierarchy decoding method It is.

  In the moving picture hierarchical encoding apparatus, encoding method, and encoding program of the present invention, the frequency higher than that of the local decoding (local decoding) signal at the time of the low resolution encoding process is provided before the high resolution encoding process in the hierarchical encoding. Since high-frequency component removal processing that estimates the components and removes the estimated high-frequency components has been added, the input video signal to which high-resolution encoding processing is performed becomes a signal from which high-frequency components have been removed. If there is, the high frequency component which is difficult to predict does not exist, so that the prediction error can be reduced. Further, even when motion compensation prediction is performed in the high resolution encoding process, there is no high frequency component to be predicted, so that it is possible to prevent an increase in the high frequency component of motion compensation prediction error due to deformation due to motion or resolution change. Both effects improve the encoding efficiency of the high resolution encoding process.

  Also, when estimating the high frequency component, the degradation state of the local decoding (local decoding) signal and the characteristics of the high frequency component of the input image are judged, and the estimated strength is changed using the judgment result to obtain the optimum estimated strength. When the estimated high-frequency component signal is generated with the optimal estimated strength, the encoding efficiency of the high-resolution enhancement layer is further improved, and efficient and higher-quality video signal hierarchical encoding / decoding is realized. It becomes possible to do.

  Also, in the moving picture hierarchical decoding apparatus, decoding method, and decoding program of the present invention, a high frequency component is estimated from a low resolution decoded signal and the estimated high frequency component is added to the high resolution decoded signal. Since the processing has been added, the encoding side estimates the high frequency component from the local decoded signal at the time of the low resolution encoding process and removes the estimated high frequency component before the high resolution encoding process of the hierarchical encoding. Even if high frequency component removal processing is added, a high-resolution decoded signal can be correctly restored, and compatibility is ensured. In addition, when such high frequency component removal processing is not added on the encoding side, a moving image signal having a better characteristic than that on the encoding side, in which the high frequency component is restored, can be reproduced.

  Next, an embodiment of the present invention will be described.

  The moving image hierarchical encoding apparatus according to the present invention encodes an image signal having a resolution lower than that of the input image signal obtained by decomposing the input image signal into layers having different resolutions compared to the conventional encoding process. Is a process that estimates the high-frequency component of the input image signal using the local decoded signal obtained in step 1, and inputs the signal from which the estimated component has been removed in advance to the process of encoding a high-resolution video signal. is there. The hierarchical decoding device of the present invention estimates a high frequency component of an input image signal using an image signal obtained by decoding encoded data of an image signal having a resolution lower than that of the input image signal, A process of adding to the decoded image signal of the encoded data of the removed signal is added.

  Examples of the configuration, method, and program of a moving picture hierarchical encoding apparatus for realizing these will be described below. In the following embodiments, for the sake of simplicity, hierarchical encoding / decoding having two spatial resolution layers is taken as an example. However, this may be realized by a multi-layer having three or more layers. Is possible.

Embodiment 1.
FIG. 1 is a diagram illustrating a configuration example of a moving picture hierarchical coding apparatus and a hierarchical decoding apparatus according to the first embodiment.

  As illustrated in FIG. 1, the moving picture hierarchy coding apparatus 101 and the moving picture hierarchy decoding apparatus 103 according to Embodiment 1 are connected via a communication line 102. Of course, the connection may be made not via the communication line 102 but via a recording medium such as a DVD.

  As shown in FIG. 1, the moving picture hierarchical encoding apparatus 101 according to the present embodiment includes a spatial decimation (spatial reduction) unit 104, a base layer encoding unit 117, a spatial interpolation (spatial expansion) unit 106, and an enhancement. It has a configuration in which a layer encoding unit 107, a multiplexing unit 108, an input image storage unit 116, a high frequency component estimation expansion unit 117, a high frequency component removal unit 118, a high frequency component estimation expansion unit 119, and a high frequency component restoration unit 120 are added. ing.

  In addition, as shown in FIG. 1, a moving picture hierarchy decoding apparatus 103 according to an embodiment of the moving picture hierarchy decoding apparatus of the present invention includes an extractor 109, a base layer decoding part 110, a spatial interpolation part. 111, an enhancement layer decoding unit 112, a high frequency component estimation expansion unit 119, and a high frequency component restoration unit 120.

  Next, the operation of each unit included in the moving picture hierarchy coding apparatus 101 according to the first embodiment will be described with reference to the flowchart of FIG.

  FIG. 2 is a flowchart showing a procedure of spatial scalable coding of an image signal by the moving picture hierarchical coding apparatus 101 shown in FIG.

  When the original image signal is input to the moving image hierarchical encoding apparatus 101, first, the spatial decimation unit 104 performs spatial resolution decimation, that is, spatial reduction processing, and outputs the result to the base layer encoding unit 117 (step S201). .

  The base layer encoding unit 117 encodes the signal with the spatial resolution decimated in the base layer, and generates a bit stream (step S202). The generated base layer encoded bit stream is sent to the multiplexing unit 108, while the base layer local decoded signal obtained in the encoding process is sent to the spatial interpolation unit 106 and the high frequency component estimation expansion unit 117.

  The spatial interpolation unit 106 spatially interpolates the local decode signal of the base layer from the base layer encoding unit 117 to the spatial resolution (step S203). Then, the spatially interpolated signal is sent to the enhancement layer encoding unit 107.

  On the other hand, the high frequency component estimation enlarging unit 117 estimates the high frequency component of the original image signal from the base layer local decode signal from the base layer encoding unit 117, and outputs it to the high frequency component removing unit 118 (step S204). . Details will be described later.

  The high frequency component removal unit 118 subtracts the estimated high frequency component signal output from the high frequency component estimation enlargement unit 117 from the original input image signal output from the input image storage unit 116 to remove the high frequency component. A resolution video signal is generated and output to the enhancement layer encoding unit 107 (step S205).

  The enhancement layer encoding unit 107 uses the high-resolution video signal from the high-frequency component removal unit 118 and the output signal from the spatial interpolation unit 106 to perform prediction using the correlation between spatial resolutions and time, The accompanying prediction error signal is encoded by the enhancement layer, and the encoded bit stream is sent to the multiplexing unit 108 (step S206).

  Then, the multiplexing unit 108 multiplexes the base layer encoded bit stream from the base layer encoding unit 117 and the enhancement layer encoded bit stream from the enhancement layer encoding unit 107 to obtain one multiplexed bit stream. It is generated and transmitted to the moving picture hierarchy decoding apparatus 103 via the communication line 102 (step S207).

  FIG. 3 is a flowchart showing a procedure of the moving picture hierarchy decoding apparatus 103 shown in FIG. 1 that separates and decodes a multiplexed stream that has been spatially scalable encoded and multiplexed by the moving picture hierarchy encoding apparatus 101 shown in FIG. It is.

  In the moving picture hierarchical decoding apparatus 103 shown in FIG. 1, the multiplexed stream that is spatially scalable encoded and multiplexed by the moving picture hierarchical encoding apparatus 101 shown in FIG. 1 is transmitted via a communication line or media 102. When it comes, first, the extractor 109 receives it. The extractor 109 analyzes that the multiplexed bitstream is received, extracts the necessary code data according to the performance of the moving picture hierarchy decoding apparatus 103 and the display, and the like, and the base layer decoding unit 110, the enhancement layer decoding unit The data is divided into data corresponding to each of the data 112 and output (step S301).

  The base layer decoding unit 110 decodes the base layer encoded stream divided by the extractor 109 in the base layer (step S302), and decodes the decoded base layer decoded image signal to the spatial interpolation unit 111, This is output to the frequency component estimation enlargement unit 119. Here, if necessary, the base layer decoding unit 110 outputs to a display or the like.

  The spatial interpolation unit 111 receives the base layer decoded image signal obtained from the base layer decoding unit 110, performs spatial resolution interpolation (step S303), and performs enhancement layer decoding on the spatially interpolated signal. Send to part 112.

  The enhancement layer decoding unit 112 spatially interpolates the encoded bitstream corresponding to the enhancement layer separated by the extractor 109 and the base layer decoded image signal by the spatial interpolation unit 111. The signal is decoded (step S304), and the decoded decoded image signal is output to the high frequency component restoration unit 120.

  On the other hand, the high frequency component estimation enlarging unit 119 receives the base layer decoded image signal obtained from the base layer decoding unit 110 and encodes the image signal decoded in the enhancement layer based on the base layer decoded signal. The same component as the high-frequency component that is sometimes estimated and removed is generated (step S505).

  The high frequency component restoration unit 120 adds the high frequency component estimation signal output from the high frequency component estimation enlargement unit 119 and the decoded enhancement layer decoded image signal to generate a high resolution decoded image signal in which the high frequency component is restored ( In step S306, the generated high-resolution decoded video signal is output to a display or the like.

  FIG. 4 shows a detailed configuration example of the high frequency component estimation enlargement units 117 and 119 according to the first embodiment.

  In FIG. 4, the high frequency component estimation expansion units 117 and 119 include a first high-pass filtering unit 401, a spatial interpolation unit 402, an amplitude limiting / constant multiplication processing unit 403, and a second high-pass filtering unit 404.

  Next, a more detailed configuration and operation of the high-frequency component estimation / enlargement unit 117 of the moving picture hierarchy coding apparatus 101 and the high-frequency component estimation / enlargement unit 119 of the moving picture hierarchy decoding apparatus 103 according to Embodiment 1 will be described. To do.

  The basic idea of the high-frequency component estimation and expansion process is, for example, “An image magnification method that can be arbitrarily scaled with high-frequency component estimation” (Takamasa Takamasa, Taguchi Ryo, Shingaku (A), vol. -A, no.9, pp.1192-1201, Sep.2001.) And "PARAMETERSTIMATIONS FOR SUPER RESOLUTION BASED ON THE LAPLACIAN PYRAMID REPRESENTATION" (Shuai Yuan, Yasumasa Takahashi *, Akira Taguchi). This is an application of the Laplacian pyramid concept in hierarchical coding. This is a method for achieving the estimation of the Laplacian component of the layer whose spatial resolution is one higher only from the signal of the layer of interest by utilizing the strong correlation of the Laplacian component between the layers.

  FIG. 4 shows a block diagram of an embodiment of the high frequency component estimation enlargement units 117 and 119. As shown in the figure, the high frequency component estimation enlarging unit 117 includes at least a first high pass filtering unit 401, a spatial interpolation unit 402, an amplitude limiting / constant multiplication unit 403, and a second high pass filtering unit 404. Have. However, here, the configuration and operation of the high-frequency component estimation / enlargement unit 117 included in the moving picture hierarchy coding apparatus 101 and the high-frequency component estimation / enlargement unit 119 included in the moving picture hierarchy decoding apparatus 103 are the same. Therefore, hereinafter, the high frequency component estimation enlargement unit 117 will be described as a representative.

  Next, FIG. 4 shows operations of the high frequency component estimation / enlargement unit 117 included in the moving picture hierarchy coding apparatus 101 and the high frequency component estimation / enlargement unit 119 included in the moving picture hierarchy decoding apparatus 200 according to the present invention. The high frequency component estimation enlargement unit 117 will be described as a representative with reference to the flowchart of FIG.

  The first high-pass filtering unit 401 includes a high-frequency component included in the enhancement layer image frame signal to be encoded in the next higher layer in the high-frequency component included in the base-layer local decoded (decoded image) signal. A high frequency component having a high correlation and suitable for estimation is extracted (step S501). Here, as information for expressing a high frequency component suitable for estimation, it is desirable to have a function of extracting a Laplacian component of a local decode (decoded image) signal of a base layer to be enlarged.

Here, the Laplacian component of the input signal is extracted as follows, for example. In order to simplify the description, taking a one-dimensional signal model as an example, if the input signal is G ₀ (x) and the Laplacian component extracted from the input signal is L ₀ (x), the Laplacian component L ₀ (x ) Is calculated by the following equations (1) and (2). In the equation (2), ρ is a parameter for adjusting the band of the Gaussian filter, and is set to a value that becomes a half-band filter.

... Formula (1)

... Formula (2)

Here, the first high-pass filtering unit 401 outputs information for expressing a high-frequency component suitable for estimation extracted from the input signal to the spatial interpolation unit 402. Here, the extracted signal of Laplacian component L ₀ (x) is output to spatial interpolation section 402 as information for expressing a high-frequency component suitable for estimation.

  Here, in the above formulas (1) and (2), a high frequency component is extracted using a Gaussian function, but this may be replaced with another method. However, the filters and interpolation functions used here, the filters used in the spatial decimation unit 104, the spatial interpolation units 106 and 111 in FIG. 1, the spatial interpolation unit 402 and the second high-pass filtering unit 404 in FIG. It is desirable that the relationship between the interpolation function and the interpolation function satisfy the pyramid configuration. For example, when the sinc function is used for the spatial decimation unit, the pyramid configuration relationship based on the sinc function can be constructed by using the sinc function also for the spatial interpolation unit 402 and the second high-pass filtering unit 404.

When the spatial interpolation unit 402 acquires information for expressing a high-frequency component suitable for estimation output from the first high-pass filtering unit 401, the spatial interpolation unit 402 converts the signal to the spatial resolution of the enhancement layer ( Interpolation is performed at a scaling factor r of the enhancement layer spatial resolution / base layer spatial resolution (step S502). Here, an interpolator with a scaling factor r of (enhancement layer spatial resolution / base layer spatial resolution) is received so that the signal of the Laplacian component L ₀ (x) is received as an input and the input signal becomes the spatial resolution of the enhancement layer. Perform

  Here, it is desirable to interpolate (spatial resolution of the enhancement layer / spatial resolution of the base layer) to the magnification r as follows, but the interpolation is not necessarily limited to this method.

That is, the interpolation signal (EXPAND) _r L ₀ (x) interpolated to the magnification r of (enhancement layer spatial resolution / base layer spatial resolution) is expressed as follows when the input Laplacian component is L ₀ (x): It is calculated by the equations (3), (4) and (5).

... Formula (3)

... Formula (4)

... Formula (5)
Here, int () indicates an operation for extracting the integer part.

The spatial interpolation unit 402 outputs the interpolated interpolation signal (EXPAND) _r L ₀ (x) thus generated to the amplitude limit / constant multiplication unit 403.

Amplitude limiting and constant multiplication processing unit 403 obtains the interpolation signal output from the spatial interpolation unit _{402 (EXPAND) r L 0 (} x), from the obtained interpolation signal _{(EXPAND) r L 0 (x} ) A low resolution process for estimating an unknown high frequency component is performed (step S503). The low-resolution process for estimating the unknown high-frequency component is an amplitude limit and constant multiplication process as shown in the following equation (6) for the input interpolation signal (EXPAND) _r L ₀ (x). It is realized by doing.

... Formula (6)

That is, when the amplitude of the input signal (EXPAND) _r L ₀ (x) is between the parameters T and −T for limiting the amplitude, the amplitude limiting / constant multiplication unit 403 receives the input signal (EXPAND) _r. A signal obtained by multiplying L ₀ (x) by a parameter α _r for constant multiplication processing is output, and when the amplitude of the input signal (EXPAND) _r L ₀ (x) is equal to or greater than T, a signal of α _r · T level When the amplitude of the input signal (EXPAND) _r L ₀ (x) is −T or less, a signal of the level of α _r · (−T) is output. Here, for the parameter T for limiting the amplitude and the parameter α _r for the constant multiplication process, for example, values as experimentally obtained in Non-Patent Document 1 may be used. The parameter α _r is variable according to the enlargement ratio.

  The signal subjected to the amplitude limiting / constant multiplication processing as described above by the amplitude limiting / constant multiplication processing unit 403 is supplied to the second high-pass filtering unit 404.

  The second high pass filtering unit 404 performs a high resolution process for estimating an unknown high frequency component (step S504). The processing of the high resolution process by the second high-pass filtering unit 404 is based on the signal acquired from the amplitude limiting / constant multiplication processing unit 403, that is, the signal on which the low resolution process for estimating the unknown high frequency component is performed. This is to remove unnecessary low-frequency components and obtain only the high-frequency components originally intended to be obtained. This is realized by performing high-pass filtering on the input signal.

It is desirable that the estimated unknown high-frequency component output from the second high-pass filtering unit 404 after the high-pass filtering by the second high-pass filtering unit 404 is expressed by the following equation (7). The estimated unknown high-frequency component Lγ (hat ^) (x) is expressed as L _γ (upper bar ￣) (x).

... Formula (7)
Given in. Here, W (i) is as shown in Equation (2).

  Then, the second high pass filtering unit 404 outputs the estimated high frequency component signal to the high frequency component removing unit 118 as shown in FIG.

  Note that the high frequency component estimation expansion unit 119 of the hierarchical decoding device 103 outputs the estimated high frequency component signal to the high frequency component restoration unit 120 as shown in FIG.

  Then, as described above, in the moving picture hierarchical encoding device 101, the enhancement layer encoding unit 107 uses the high-resolution video signal from the high-frequency component removing unit 118 and the output signal from the spatial interpolation unit 106, Prediction using spatial resolution and temporal correlation is performed, and a prediction error signal generated in association with the prediction is encoded by an enhancement layer, and the multiplexing unit 108 includes a base layer encoded bitstream from the base layer encoding unit 117, and The enhancement layer encoded bit stream from the enhancement layer encoding unit 107 is multiplexed and transmitted to the moving picture hierarchy decoding apparatus 103 via the communication line 102.

  On the other hand, the moving picture hierarchy decoding apparatus 103 receives the multiplexed bit stream from the moving picture hierarchy decoding apparatus 103 and obtains a high resolution image signal and a low resolution image signal.

  As described above, since the conventional moving image hierarchical encoding device and hierarchical decoding device perform encoding / decoding processing in units of blocks, performing Laplacian component enhancement processing in units of blocks is sufficient for block boundaries. However, according to the first embodiment, as described above, the base layer local decode (decoded image) signal is obtained, and the local decode (decoded image) signal is obtained. After performing spatial interpolation up to the enhancement layer image size, analysis to perform Laplacian component enhancement processing is performed to estimate high frequency components, thus suppressing the signal amplitude of prediction error signals between coding layers can do. As a result, this increases the correlation with the prediction error signal between surrounding encoding layers, and as a result, the entropy can be suppressed and the encoding target can be expressed with a smaller number of bits when encoding. The effect of improving the efficiency is achieved.

  In addition, the correlation with the prediction error signal between surrounding coding layers is increased, and as a result, when the prediction error signal is encoded, the entropy is suppressed, and encoding is performed with a smaller number of bits when performing entropy encoding. The object can be expressed.

  In addition, since the signal amplitude of the prediction error signal between the encoding layers to be encoded is suppressed compared with the prediction error signal between the encoding layers in the conventional hierarchical encoding, the quantization error that occurs during quantization It is also possible to suppress the influence at the time of decoding due to.

  Therefore, according to the moving picture hierarchical encoding apparatus 101 of the first embodiment, the spatial interpolation unit 106 expands the base layer local decode signal from the base layer encoding unit 105 to the enhancement layer resolution, and removes high frequency components. The unit 118 removes the high frequency component estimation signal estimated based on the base layer local decode signal from the high frequency component estimation enlargement unit 117 from the input image signal, and the enhancement layer encoding unit 107 removes the high frequency component from the high frequency component removal unit 118. Since the difference between the signal and the interpolation signal expanded to the enhancement layer resolution from the spatial interpolation unit 106 is taken and encoded in the enhancement layer, inter-layer prediction in the enhancement layer encoding is performed. Can reduce the high frequency component of the error and motion compensation prediction error, it is possible to improve the coding efficiency of the enhancement layer.

  In other words, since the high frequency components that were difficult to predict using the inter-layer prediction of the prior art no longer exist, the prediction error can be reduced and even when motion compensation prediction is performed in the enhancement layer encoding process. Since there is no high frequency component to be predicted, it is possible to prevent an increase in the high frequency component of the motion compensation prediction error due to deformation or resolution change due to motion, and both effects improve the encoding efficiency of the high resolution encoding process. . Even when the high frequency component removal processing is not added on the decoding side, it is possible to reproduce a moving image signal having a characteristic in which the high frequency component is suppressed.

  Also, according to the moving picture hierarchy decoding apparatus 103 of the first embodiment, the high frequency component estimation expansion unit 119, which is the same as the high frequency component estimation expansion unit 117, and the high frequency component estimation expansion unit, contrary to the high frequency component removal unit 118, are provided. Since the high-frequency component restoration unit 120 that restores the high-frequency component based on the high-frequency component estimation signal from 119 is included, the encoded bitstream encoded by the moving picture hierarchical encoding apparatus 101 according to the first embodiment is correctly decoded. Can do. In addition, when such high frequency component removal processing is not added on the encoding side, a moving image signal having a better characteristic than that on the encoding side, in which the high frequency component is restored, can be reproduced.

Embodiment 2. FIG.
Next, configuration examples of the moving picture hierarchy coding apparatus and the moving picture hierarchy decoding apparatus according to Embodiment 2 of the present invention will be described.

  FIG. 6 is a block diagram illustrating a configuration example of the moving picture hierarchy coding apparatus and the moving picture hierarchy decoding apparatus according to the second embodiment.

  In FIG. 6, the moving picture hierarchy encoding apparatus 601 and the moving picture hierarchy decoding apparatus 603 of the second embodiment are the same as the moving picture hierarchy encoding apparatus 101 and the moving picture hierarchy decoding apparatus 103 of the first embodiment shown in FIG. On the other hand, in the moving picture hierarchy coding apparatus 601, the processes of the spatial decimation unit 604, the high frequency component estimation expansion part 617, and the multiplexing part 608 are different. The processing performed by the frequency component estimation enlargement unit 619 is different. Hereinafter, the processing different from that of the first embodiment will be mainly described.

  That is, the spatial decimation unit 604 of the moving picture hierarchical encoding apparatus 601 according to the second embodiment inputs the original image signal, and the input signal is desired, that is, in this case, the enhancement layer as in the first embodiment. 2 and the base layer, spatial decimation is performed to the spatial resolution of the base layer, and an input signal or a high frequency component of the input signal is generated and output to the high frequency component estimation enlargement unit 617. Here, the high frequency component can be calculated from the difference information between the signal obtained by spatial decimation to the desired spatial resolution and the spatially interpolated signal and the original image signal. Other high frequency component generation methods can also be used.

  In addition to the local decode signal output from the base layer encoding unit 605, the high frequency component estimation / expansion unit 617 according to the second embodiment receives the input signal output from the spatial decimation unit 604 or the high frequency component of the input signal. Are input, and the high frequency component of the image signal encoded in the enhancement layer based on the local decode signal is set with the estimated intensity and the optimum high frequency component is estimated. Details will be described later. The high frequency component estimation expansion unit 617 has a function of outputting a high frequency component estimation signal to the high frequency component removal unit 618 and outputting a control parameter indicating the estimated intensity to the multiplexing unit 608.

  Multiplexing section 608 receives as input the respective encoded bit streams output from base layer encoding section 605 and enhancement layer encoding section 607, and also indicates a control parameter indicating the estimated strength output from high frequency component estimation expanding section 617. Are multiplexed and output as a multiplexed bit stream to the outside of the hierarchical encoding unit 601, for example, a communication line or media 602.

  The extractor 609 receives the multiplexed bit stream transmitted from the moving picture layer encoding apparatus 601 according to the second embodiment, and, as information necessary for decoding, along with the encoded bit streams of the base layer and the enhancement layer, A control parameter indicating the estimated intensity is cut out, divided, and output to the base layer decoding unit 610, the enhancement layer decoding unit 612, and the high frequency component estimation expansion unit 619.

  The high frequency component estimation expansion unit 119 receives, as an input, a control parameter indicating the estimated strength output from the extract unit 609 together with the base layer decoded signal output from the base layer decoding unit 110, and receives the base layer decoded signal and the estimated strength. The high-frequency component from which the image signal decoded in the enhancement layer is removed at the time of encoding is estimated based on the control parameter indicating Further, the high frequency component estimation expanding unit 119 outputs the high frequency component estimation signal to the high frequency component restoring unit 120.

  FIG. 7 is a flowchart showing a procedure of spatial scalable coding of the image signal of the hierarchical coding unit 601 according to the second embodiment shown in FIG.

  First, the spatial decimation unit 604 receives the original image signal, performs decimation to the spatial resolution of the base layer, and outputs the base layer signal to the base layer encoding unit 605 (step S701).

  The base layer encoding unit 605 encodes the signal decimated to the spatial resolution of the base layer in the base layer, sends the encoded signal to the multiplexing unit 608 as a base layer encoded bit stream, and transmits the base layer obtained in the encoding process. The local decode signal is sent to the spatial interpolation unit 606 and the high frequency component estimation expansion unit 617 (step S702).

  The spatial interpolation unit 606 spatially interpolates the base layer local decoded signal obtained from the base layer encoding unit 605 to the spatial resolution of the enhancement layer, and sends the spatially interpolated signal to the enhancement layer encoding unit 607. Send (step S703).

  Subsequently, the high frequency component estimation expansion unit 617 obtains the base layer local decode signal obtained from the base layer encoding unit 605 and the input image signal output from the spatial decimation unit 604 or the high frequency component of the input signal. The high frequency component of the original image signal is estimated, a control parameter indicating the estimated intensity and a high frequency component estimation signal are generated, and the control parameter indicating the estimated intensity is sent to the multiplexing unit 608, while the high frequency component estimation signal is It sends to the high frequency component removal part 618 (step S704). Details will be described later.

  The high frequency component removal unit 618 subtracts the estimated high frequency component signal from the high frequency component estimation enlargement unit 617 from the original input image signal from the input image storage unit 616 to obtain a high resolution video signal from which the high frequency component has been removed. It is generated and output to the enhancement layer encoding unit 607 (step S705).

  The enhancement layer encoding unit 607 inputs the high-frequency component removal signal generated by the high-frequency component removal unit 618 and the spatial interpolation signal from the spatial interpolation unit 606, and uses the correlation between spatial resolutions and time. The prediction error signal generated along with the prediction is encoded, and the bit stream generated by the encoding is sent to the multiplexing unit 608 (step S706).

  The multiplexing unit 608 receives the bit stream of each layer from the base layer encoding unit 605 and the enhancement layer encoding unit 607 and the control parameter indicating the estimated strength from the high frequency component estimation expansion unit 617 for multiplexing. In step S707, one multiplexed bit stream is generated and output to a communication line, media, or the like 602.

  Next, the operation on the decoding side will be described.

  FIG. 8 is a flowchart illustrating a procedure for obtaining a decoded video signal by decoding a multiplexed bitstream that has been subjected to spatial scalable coding according to the configuration example of the moving picture hierarchical decoding apparatus 603 illustrated in FIG. 6.

  In the moving picture layer decoding apparatus 603 of Embodiment 2 shown in FIG. 6, the extract unit 609 receives the multiplexed bit stream from the moving picture layer encoding apparatus 601 via the communication line, media, or the like 602.

  When the extract unit 609 receives the multiplexed bit stream from the layer encoding apparatus 601 via the communication line, the media, or the like 602, the extract unit 609 analyzes the received multiplexed bit stream and demultiplexes it. That is, the extractor 609 is a coding layer (in this case, a coding layer of two layers of a base layer and an enhancement layer) that is decoded according to the performance of the moving picture hierarchy decoding device 603 and a display connected thereto. And the base layer encoded bit stream, the enhancement layer encoded bit stream, and control parameters indicating the estimated strength used for enhancement layer decoding are extracted from the multiplexed bit stream, and The decoding unit 610 outputs a base layer encoded stream, the enhancement layer decoding unit 612 outputs an enhancement layer encoded stream, and the high frequency component estimation expansion unit 619 outputs a control parameter indicating the estimated strength (step S801).

  The base layer decoding unit 610 decodes the base layer encoded bit stream separated by the extract unit 609 using the base layer, and the decoded base layer decoded image signal is transmitted to the spatial interpolation unit 611 and the frequency component estimation expansion unit 619. If necessary, it is also output to a display or the like (step S802).

  The spatial interpolation unit 611 spatially interpolates the base layer decoded image signal obtained from the base layer decoding unit 610 to the spatial resolution of the enhancement layer, and sends the spatially interpolated signal to the enhancement layer decoding unit 612. Send (step S803).

  The enhancement layer decoding unit 612 then encodes the encoded bitstream corresponding to the enhancement layer separated by the extract unit 609, and the spatial interpolation unit spatially interpolated to the spatial resolution of the enhancement layer by the spatial interpolation unit 611. The decoded image signal is output to the high frequency component restoration unit 620 (step S804).

  On the other hand, the high frequency component estimation enlarging unit 619 uses the high frequency component based on the base layer decoded image signal decoded by the base layer decoding unit 610 and the control parameter indicating the estimated intensity separated by the extract unit 609. The high frequency component estimated by the component estimation enlarging unit 617 and removed from the input image by the high frequency component removing unit 618 is estimated and output to the high frequency component restoring unit 620 (step S805).

  The high frequency component restoration unit 620 adds the high frequency component estimation signal output from the high frequency component estimation expansion unit 619 and the enhancement layer decoded image signal decoded by the enhancement layer decoding unit 612 to restore the high frequency component. A high resolution decoded image signal is generated and output to a display or the like as a high resolution decoded video signal (step S806).

  Subsequently, a detailed configuration example of the high frequency component estimation enlarging unit 617 of the moving picture hierarchical encoding device 601 of Embodiment 2 will be described.

  FIG. 9 is a block diagram illustrating a detailed configuration example of the high frequency component estimation enlarging unit 617 of the moving picture hierarchy coding apparatus 601 according to the second embodiment. Note that the high frequency component estimation enlarging unit 619 of the video decoding device 603 is configured in the same manner.

  In FIG. 9, the high frequency component estimation expansion unit 617 of the second embodiment includes a first high-pass filtering unit 901, a spatial interpolation unit 902, an amplitude limiting / constant multiplication processing unit 903, a second high-pass filtering unit 904, an estimation A degree determination unit 905 and an entropy encoding unit 906. It should be noted that the description will be centered on the part added to the high-frequency component estimation / expansion section 117 of the first embodiment shown in FIG. 4, and the same operations as those of the first embodiment will be performed for the parts that are not described.

  That is, the amplitude limiting / constant multiplication processing unit 903 according to the second embodiment receives the control parameter indicating the estimated intensity from the estimation degree determining unit 905 and the spatial interpolation signal from the spatial interpolation unit 902. And a function of performing a first step for estimating an unknown high-frequency component. The present embodiment has a structure in which control parameters are given from the outside to the amplitude limiting / constant multiplication processing unit 903 so that trials for optimal parameter calculation are possible.

  The estimation degree determination unit 905 receives the signal output from the second high-pass filtering unit 904 and the input image signal from the spatial decimation unit 604 shown in FIG. Has the function of obtaining the optimum estimated intensity. Here, the signal output from the second high-pass filtering unit 904 is an estimated signal of a high-frequency component when a parameter having a certain estimated intensity is used in the amplitude limiting / constant multiplication processing unit 903. The estimation degree determination unit 905 has a function of quantifying the degree of correlation between this signal and the signal output from the spatial decimation unit 604 and recording it.

  Here, as a method for quantifying the correlation between two signals, for example, when an input image signal is output from the spatial decimation unit 604, entropy of a signal obtained by subtracting a high-frequency component estimation signal from the input image signal is calculated. This is possible. In addition, when the spatial decimation unit 604 is a high frequency component of the input signal, the energy of both the high frequency component and the high frequency component estimation signal can be calculated and quantified by the similarity of the values. It is.

The purpose of the estimation degree determination unit 905 is to obtain parameters α _r and T (or only α _r ) that reduce the amount of information of the image signal input to the enhancement layer encoder unit. It also has a function of sequentially updating parameters within the range and outputting them to the amplitude limit / constant multiplication unit 903. Then, from the sequentially recorded correlation quantification value, the case where the information amount is the smallest is determined, the parameter at that time is output to the entropy coding unit 906, and the high frequency component estimation signal at that time is removed from the high frequency component in FIG. To the unit 618.

  The entropy encoding unit 906 has a function of accepting a parameter output from the estimation degree determination unit 905 as an input. Also, the input parameter is entropy-coded to generate a bitstream and output to the multiplexing unit 608 in FIG.

  FIG. 10 shows a procedure for generating a high-frequency component estimation signal using the configuration example of the high-frequency component estimation expansion unit 617 shown in FIG.

  In FIG. 10, first, the first high-pass filtering unit 901 extracts a high frequency component signal from the local decoded signal from the base layer encoding unit 605 (step S1001).

  The spatial interpolation unit 902 interpolates the high frequency component signal extracted by the first high-pass filtering unit 901 (step S1002).

  The amplitude limiting / constant multiplication processing unit 903 performs amplitude limiting and constant multiplication processing on the signal interpolated by the spatial interpolation unit 902 based on the estimated intensity parameter (step S1003). Here, the estimated intensity parameter associated with the amplitude limit and constant multiplication processing uses the one given from the estimation degree determination unit 905.

  The second high-pass filtering unit 904 extracts a high-frequency component estimated from the signal subjected to the amplitude limiting and constant multiplication processing based on the estimated intensity parameter by the amplitude limiting / constant multiplication processing unit 903, and outputs the high frequency component to the estimation degree determination unit 905. Output (step S1004).

  The estimation degree determination unit 905 records the high-frequency component estimation signal from the second high-pass filtering unit 904, and also the input image signal output from the spatial decimation unit 604 or the high-frequency component signal of the input image signal, and the high-frequency component estimation The correlation with the signal is detected and recorded in association with the estimated intensity parameter (step S1005).

  Then, the estimation degree determination unit 905 performs the procedure from step S1003 to step S1005 using the estimated intensity parameter in order to obtain the optimum estimated intensity parameter by which the high frequency component estimation signal is closest to the high frequency component of the original input image signal. While updating, it is repeated for the estimated intensity parameter within the designated range designated in advance by the user (step S1006).

  Then, when the processing of step S1003 to step S1005 is completed for the estimated intensity parameter within the specified range (step S1006 “YES”), the estimation degree determination unit 905 generates the estimated intensity parameters within all of the specified range. The optimum estimation strength parameter that minimizes the level difference between each high-frequency component estimation signal and the high-frequency component of the original input image signal is selected, and the amplitude limiting / constant multiplication processing unit 903, the entropy encoding unit And output to 906 (step S1007).

  Then, the amplitude limiting / constant multiplication processing unit 903 performs amplitude limiting and constant multiplication processing on the interpolated signal from the spatial interpolation unit 902 based on the optimal estimated intensity parameter, and the second The high-pass filtering unit 904 extracts a high frequency component estimated from the signal subjected to amplitude limitation and constant multiplication based on the optimal estimated intensity parameter, and outputs the high frequency component as a high frequency component estimation signal to the estimation degree determination unit 905 (step S1008). ).

  The entropy encoding unit 906 entropy-encodes the optimum high-frequency component estimation signal from the estimation degree determination unit 905 and outputs it to the multiplexing unit 608 (step S1009). Note that the estimated intensity parameter may be encoded for each block. For example, an average value of parameters in 1 GOP is adopted, a high resolution estimated signal is generated with the parameters uniformly in the GOP, and encoded in 1 GOP. There may be only one parameter. There are no restrictions on the number or timing of parameters to be encoded.

  Next, a detailed configuration example of the high frequency component estimation expansion unit 619 in the hierarchical decoding according to Embodiment 2 of the present invention will be described with reference to FIG.

  In FIG. 11, the high frequency component estimation enlarging unit 619 includes a first high-pass filtering unit 1101, a spatial interpolation unit 1102, an amplitude limiting / constant multiplication unit 1103, a second high-pass filtering unit 1104, and an entropy decoding unit 1105. Consists of.

  Hereinafter, the part added from the high frequency component estimation expansion part by Embodiment 1 shown in FIG. 4 is demonstrated. For parts not described, the same operation as in the first embodiment is performed.

  That is, the entropy decoding unit 1105 has a function of receiving and decoding a bit stream output from the extract unit 609 in FIG. 6 as an input corresponding to a parameter. Also, it has a function of outputting the decoded parameters to the amplitude limiting / constant multiplication processing unit 1103.

  An amplitude limit / constant multiplication unit 1103 receives the estimated intensity parameter decoded by the entropy decoding unit 1105 and the signal output from the spatial interpolation unit 1102 as inputs, and estimates an unknown high frequency component. It has a function of performing the first step. As a result, the high frequency component is estimated based on the control parameter indicating the estimated intensity set at the time of encoding.

  FIG. 12 shows a procedure for generating a high-frequency component estimation signal using the configuration example of the high-frequency component estimation enlarging unit 619 shown in FIG.

  In FIG. 12, first, the entropy decoding unit 1105 of the high frequency component estimation enlarging unit 619 decodes an encoding parameter such as an optimal estimated intensity parameter included in the bit stream obtained from the extract unit 609 (see FIG. 6). And output to the amplitude limiting / constant multiplication processing unit 1103 (step S1201).

  On the other hand, the first high-pass filtering unit 1101 extracts a high-frequency component signal from the base layer decoding signal obtained from the base layer decoding unit 110 (step S1202).

  The spatial interpolation unit 1102 interpolates the high frequency component signal extracted by the first high-pass filtering unit 1101 to the enhancement layer resolution (step S1203).

  Based on the signal interpolated by the spatial interpolation unit 1102 and the optimum estimated intensity parameter decoded by the entropy decoding unit 1105, the amplitude limit / constant multiplication unit 1103 Processing is performed (step S1204).

  The second high-pass filtering unit 1104 performs a high-pass filtering process on the signal subjected to the amplitude limiting constant multiplication processing from the amplitude limiting / constant multiplication processing unit 1103, and returns the high-frequency component estimation signal estimated on the encoding side. Then, the signal is output to the high frequency component removing unit 618 in FIG.

  Therefore, according to the moving picture hierarchical encoding apparatus 601 of the second embodiment, the spatial interpolation unit 606 receives the base layer local decoded signal from the base layer encoding unit 605 as an enhancement layer, as in the first embodiment. The high frequency component removal unit 618 removes the high frequency component estimation signal estimated based on the base layer local decode signal from the high frequency component estimation enlargement unit 617 from the input image signal, and the enhancement layer encoding unit 607 Since the difference between the high frequency component removal signal from the component removal unit 618 and the interpolation signal expanded to the enhancement layer resolution from the spatial interpolation unit 606 is taken and encoded in the enhancement layer, Enhancement Ray Of can reduce the high frequency component of the inter-layer prediction error and the motion compensated prediction error in the encoding, it is possible to improve the coding efficiency of the enhancement layer.

  In particular, in the moving picture hierarchical encoding apparatus 601 of the second embodiment, the estimated high frequency component generation unit 617 receives the input image signal from the spatial decimation unit 604 or its high frequency component, and the high frequency component estimation signal and the original The optimum estimated intensity parameter that minimizes the level difference from the high frequency component of the input image signal is selected, and the width limiting / constant multiplication processing unit 903 applies the interpolated signal from the spatial interpolation unit 902 to the interpolated signal. Since amplitude limiting and constant multiplication processing are performed based on the optimal estimated intensity parameter, a more optimal high frequency component estimation signal can be output to the high frequency component removing unit 618 than in the first embodiment.

  In other words, when estimating the high frequency component, the degradation state of the local decoding (local decoding) signal and the characteristics of the high frequency component of the input image are judged, and the estimated strength is changed using the judgment result to obtain the optimum estimated strength. In the case where the estimation attack and defense generates a component signal with the optimum estimation strength, the encoding efficiency of the higher resolution enhancement layer is further improved, and the hierarchical encoding / decoding of the higher-quality moving image signal is performed more efficiently. It can be realized.

  Also, according to the moving picture hierarchy decoding apparatus 603 of the second embodiment, as in the first embodiment, the high-frequency component estimation expansion unit 619 and the high-frequency component removal unit 618 are the same as the high-frequency component estimation expansion unit 617. On the contrary, since it has a high-frequency component restoration unit 620 that restores a high-frequency component based on a high-frequency component estimation signal from the high-frequency component estimation expansion unit 619, it is encoded by the moving picture hierarchy coding apparatus 101 according to the second embodiment. The encoded bit stream can be correctly decoded. In addition, when such high frequency component removal processing is not added on the encoding side, a moving image signal having a better characteristic than that on the encoding side, in which the high frequency component is restored, can be reproduced.

  In the first and second embodiments, as described above, the moving picture hierarchy coding apparatus and the moving picture hierarchy decoding apparatus according to the present invention have been described in terms of hardware configuration using block diagrams. In the invention, the present invention is not limited to this, and the moving picture hierarchical encoding apparatus and the moving picture hierarchical decoding apparatus according to the first and second embodiments described above are implemented as software by executing a program downloaded by the CPU via a CD or a neckwork. Of course, this may be achieved.

  In the first and second embodiments, the MPEG2 encoding method is used as the moving picture coding configuration of the base layer and the enhancement layer. However, other moving picture coding standards such as MPEG1 and MPEG4-AVC are used as a base. Of course, the present invention can also be applied.

It is a block diagram which shows the moving image hierarchy encoding apparatus and moving image hierarchy decoding apparatus which are Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the hierarchy encoding part which is Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the moving image hierarchy decoding apparatus which is Embodiment 1 of this invention. It is a block diagram of the high frequency component estimation expansion part in Embodiment 1 of this invention. It is a flowchart which shows the operation | movement of the high frequency component estimation in Embodiment 1 of this invention. It is a block diagram which shows the moving image hierarchy encoding apparatus and moving image hierarchy decoding apparatus which are Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the hierarchy encoding part which is Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the moving image hierarchy decoding apparatus which is Embodiment 2 of this invention. It is a block diagram of the high frequency component estimation expansion part in the moving image hierarchy encoding apparatus which is Embodiment 2 of this invention. It is a flowchart which shows the operation | movement of the high frequency component estimation in the moving image hierarchy encoding apparatus which is Embodiment 2 of this invention. It is a block diagram of the high frequency component estimation expansion part in the moving image hierarchy decoding apparatus which is Embodiment 2 of this invention. It is a flowchart which shows the operation | movement of the high frequency component estimation in the moving image hierarchy decoding apparatus which is the high-resolution Example of this invention.

Explanation of symbols

101, 601 Moving picture layer encoding apparatus 102, 602 Communication line or media 103, 603 Moving picture layer decoding apparatus 104, 604 Spatial decimation section 105, 605 Base layer encoding section (low resolution encoding section)
106,606 Spatial interpolation unit 107,607 Enhancement layer encoding unit (high resolution encoding unit)
108,608 Multiplexing unit 109,609 Extraction unit (demultiplexing unit)
110,610 Base layer decoding unit (low resolution decoding unit)
111,611 Spatial interpolation unit 112,612 Enhancement layer decoding unit (high resolution decoding unit)
117, 617 High frequency component estimation expansion unit 118, 618 High frequency component removal unit 119, 619 High frequency component estimation expansion unit 120, 620 High frequency component restoration unit 401, 901 First high pass filtering unit 402, 902 Spatial interpolation unit 403 , 903 Amplitude limiting / constant multiplication processing unit 404, 904 Second high-pass filtering unit 905 Estimating degree judging unit 906 Entropy coding unit

Claims (10)

A moving image hierarchical encoding device that generates and encodes at least a low resolution image signal and a high resolution image signal from an input moving image signal,
A spatial decimation unit that performs spatial reduction processing on the input moving image signal to obtain a low-resolution image signal having a lower resolution than the input moving image signal;
A high-resolution signal holding unit for storing the input moving image signal as a high-resolution image signal;
A low-resolution encoding unit that encodes the low-resolution image signal to obtain low-resolution encoded data, and generates a local decoded signal of the low-resolution encoded data by a local decoding process at the time of the encoding;
A spatial interpolation unit that spatially expands the local decoded signal obtained by the local decoding process and generates an expanded decoded image signal;
A high-frequency component estimator that performs a process of estimating a high-frequency component that has been lost due to spatial reduction processing in the spatial decimation unit for the local decoded signal,
A high-frequency component removing unit that generates a high-frequency component-removed image signal obtained by subtracting the high-frequency component estimation signal from the high-resolution image signal;
A high-resolution encoding unit that encodes the high-frequency component-removed image signal using the expanded decoded image signal generated by the spatial interpolation unit to obtain high-resolution encoded data;
A multiplexing unit that performs multiplexing processing of at least the low-resolution encoded data and the high-resolution encoded data;
A moving picture hierarchical encoding device. The high frequency component estimator is
A first high-pass filtering unit that extracts a high-frequency component highly correlated with the high-resolution image signal based on the local decoded signal from the low-resolution encoding unit;
A spatial interpolation unit that spatially expands the extracted high-frequency component to the resolution of the high-resolution image signal;
An amplitude limiting / constant multiplication processing unit that performs amplitude limiting and constant multiplication processing on the spatially expanded high frequency component;
A second high-pass filtering unit that removes a low-frequency component from the high-frequency component subjected to amplitude limitation and constant multiplication, and outputs the low-frequency component to the high-frequency component removing unit;
The moving picture hierarchical encoding apparatus according to claim 1, comprising: The high-frequency component estimation unit further includes
Intensity when the amplitude limiting / constant multiplication processing unit performs the amplitude limiting and constant multiplication processing based on the input image signal and the high frequency component from which the low frequency component from the second high-pass filtering unit is removed A parameter is estimated within a specified range, and the estimated intensity parameter is output to the amplitude limit and constant multiple processing unit, and the amplitude limit / constant multiple processing unit is output to the amplitude limit and constant multiple processing unit based on the estimated intensity parameter within the specified range. A level difference between the high-frequency component signal generated by all the estimated intensity parameters estimated within the specified range and the high-frequency component of the input image signal while performing amplitude limiting and constant multiplication processing on the frequency component An estimation degree determination unit that outputs an optimum estimated intensity parameter that minimizes
An entropy encoding unit that entropy encodes the optimum estimated strength parameter from the estimation degree determination unit,
The second high pass filtering unit includes:
The amplitude limiting / constant multiplication processing unit removes a low frequency component from the high frequency component subjected to amplitude limiting and constant multiplication processing based on the optimum estimated intensity parameter, and outputs the high frequency component to the high frequency component removing unit,
The multiplexing unit includes:
The moving picture hierarchical encoding apparatus according to claim 2, wherein the optimum estimated intensity parameter that has been entropy encoded is multiplexed. A moving image hierarchical encoding method for generating and encoding at least a low resolution image signal and a high resolution image signal from an input moving image signal,
Performing a spatial reduction process on the input moving image signal to obtain a low resolution image signal having a lower resolution than the input moving image signal;
Accumulating the input moving image signal as a high resolution image signal;
Encoding the low-resolution image signal to obtain low-resolution encoded data, and generating a local decoded signal of the low-resolution encoded data by local decoding processing at the time of encoding;
Spatially expanding the local decoded signal obtained in the local decoding process to generate an expanded decoded image signal;
Performing a process of estimating a high-frequency component lost by spatial reduction processing in the spatial decimation unit for the local decoded signal, and generating a high-frequency component estimation signal;
Generating a high frequency component removed image signal obtained by subtracting the high frequency component estimation signal from the high resolution image signal;
Encoding the high-frequency component-removed image signal using the enlarged decoded image signal generated by the spatial interpolation unit to obtain high-resolution encoded data;
Performing a multiplexing process of at least the low resolution encoded data and the high resolution encoded data;
A moving picture hierarchical encoding method. A moving image hierarchical encoding program for causing a computer to execute an operation of generating and encoding at least a low resolution image signal and a high resolution image signal from an input moving image signal,
Performing a spatial reduction process on the input moving image signal to obtain a low resolution image signal having a lower resolution than the input moving image signal;
Accumulating the input moving image signal as a high resolution image signal;
Encoding the low-resolution image signal to obtain low-resolution encoded data, and generating a local decoded signal of the low-resolution encoded data by local decoding processing at the time of encoding;
Spatially expanding the local decoded signal obtained in the local decoding process to generate an expanded decoded image signal;
Performing a process of estimating a high-frequency component lost by spatial reduction processing in the spatial decimation unit for the local decoded signal, and generating a high-frequency component estimation signal;
Generating a high frequency component removed image signal obtained by subtracting the high frequency component estimation signal from the high resolution image signal;
Encoding the high-frequency component-removed image signal using the enlarged decoded image signal generated by the spatial interpolation unit to obtain high-resolution encoded data;
Performing a multiplexing process of at least the low resolution encoded data and the high resolution encoded data;
Is a moving picture hierarchical encoding program for causing a computer to execute. A low resolution image signal and a high resolution image are decoded by decoding at least the low resolution encoded data and the high resolution encoded data from a multiplexed bit stream in which at least the low resolution encoded data and the high resolution encoded data are multiplexed. A video hierarchical decoding device for generating a signal,
A demultiplexing unit that separates at least the low resolution encoded data and the high resolution encoded data from the multiplexed bitstream;
A low-resolution decoding unit that decodes the separated low-resolution encoded data and generates the low-resolution decoded image signal;
A spatial interpolation unit that spatially expands the low-resolution decoded image signal and generates an enlarged decoded image signal;
A high-resolution decoding unit that decodes the high-resolution encoded data using the expanded decoded image signal generated by the spatial interpolation unit and generates the high-resolution decoded image signal;
A high-frequency component estimator that performs high-frequency component estimation processing on the low-resolution decoded image signal by performing high-frequency component estimation processing that has been lost by spatial reduction processing in the spatial decimation unit;
A high-frequency component restoration unit that generates the high-resolution image signal obtained by restoring the high-frequency component by adding the high-frequency component estimation signal to the high-resolution image signal;
A moving picture hierarchical decoding apparatus comprising: The high frequency component estimator is
A first high-pass filtering unit that extracts a high-frequency component highly correlated with the high-resolution image signal based on the local decoded signal from the low-resolution encoding unit;
A spatial interpolation unit that spatially expands the extracted high-frequency component to the resolution of the high-resolution image signal;
An amplitude limiting / constant multiplication processing unit that performs amplitude limiting and constant multiplication processing on the spatially expanded high frequency component;
A second high-pass filtering unit that removes a low-frequency component from the high-frequency component subjected to amplitude limitation and constant multiplication, and outputs the low-frequency component to the high-frequency component removing unit;
The moving picture hierarchy decoding apparatus according to claim 6, comprising: The demultiplexing unit includes:
From the multiplexed bit stream, the low-resolution encoded data, the high-resolution encoded data, and the optimum estimated strength parameter entropy-coded used when the high-resolution encoded data is encoded are separated. ,
The high-frequency component estimation unit further includes
An entropy decoding unit that entropy-decodes the optimum estimated intensity parameter that has been entropy-encoded,
The amplitude limiting / constant multiplication processing unit performs amplitude limiting and constant multiplication processing on the high frequency component spatially expanded based on the optimum estimated intensity parameter subjected to entropy decoding. Video hierarchy decoding apparatus. A low resolution image signal and a high resolution image are decoded by decoding at least the low resolution encoded data and the high resolution encoded data from a multiplexed bit stream in which at least the low resolution encoded data and the high resolution encoded data are multiplexed. A video hierarchical decoding method for generating a signal,
Separating at least the low resolution encoded data and the high resolution encoded data from the multiplexed bitstream;
Decoding the separated low-resolution encoded data to generate the low-resolution decoded image signal;
Spatially enlarging the low resolution decoded image signal to generate an enlarged decoded image signal;
Decoding the high-resolution encoded data using the expanded decoded image signal generated by the spatial interpolation unit, and generating the high-resolution decoded image signal;
Performing high-frequency component estimation processing lost by spatial reduction processing in the spatial decimation unit on the low-resolution decoded image signal, and generating the high-frequency component estimation signal;
Generating the high-resolution image signal in which the high-frequency component is restored by adding the high-frequency component estimation signal to the high-resolution image signal;
A moving picture hierarchical decoding method comprising: A low resolution image signal and a high resolution image are decoded by decoding at least the low resolution encoded data and the high resolution encoded data from a multiplexed bit stream in which at least the low resolution encoded data and the high resolution encoded data are multiplexed. A video hierarchical decoding program for causing a computer to execute an operation for generating a signal,
Separating at least the low resolution encoded data and the high resolution encoded data from the multiplexed bitstream;
Decoding the separated low-resolution encoded data to generate the low-resolution decoded image signal;
Spatially enlarging the low resolution decoded image signal to generate an enlarged decoded image signal;
Decoding the high-resolution encoded data using the expanded decoded image signal generated by the spatial interpolation unit, and generating the high-resolution decoded image signal;
Performing high-frequency component estimation processing lost by spatial reduction processing in the spatial decimation unit on the low-resolution decoded image signal, and generating the high-frequency component estimation signal;
Generating the high-resolution image signal in which the high-frequency component is restored by adding the high-frequency component estimation signal to the high-resolution image signal;
A moving picture hierarchical decoding program for causing a computer to execute.