KR100897640B1 - Moving image encoder and moving image decoder, and its method and computer-readable recording medium containing program thereof - Google Patents

Abstract

In temporal space division filtering, the video signal 10 having an arbitrary resolution layer is temporally layered and divided into a temporal low frequency signal 11 and a temporal high frequency signal 12. Using the time low pass signal 11 and the time high pass signal 12, the video signal 21 in which the video signal corresponding to the time high pass signal 12 was reconstructed is produced | generated. A reduced image signal 14 subjected to a reduction process to the moving image signal 21 is generated. The temporal high frequency signal 12 is subjected to the high frequency generation process by spatial stratification to generate the temporal high frequency spatial high frequency signal 13. The temporal low pass signal 11, the reduced image signal 14, and the temporal high pass spatial high pass signal 13 are divided into results. The temporal low-pass signal 11 is regarded as a video signal having a low temporal resolution and the reduced image signal 14 once as a video signal having a low spatial resolution, and the video signal is layered in multiple steps by recursively performing temporal spatial division filtering. .

Time-space division filtering, moving picture signal, time low pass signal, time high pass signal, reduced picture signal, moving picture coding apparatus, control program of moving picture coding apparatus, signal coding processing unit

Description

A video encoding apparatus and a video decoding apparatus, and a computer readable recording medium having recorded thereon a method and a program thereof.

The present invention relates to a moving picture coding method, a moving picture decoding method, a moving picture coding apparatus, a moving picture decoding apparatus and a computer program.

Subband encoding is a method of frequency-dividing an image signal and performing encoding processing on signals (subband signals) of respective frequency bands. Unlike block-based orthogonal transform such as discrete cosine transform, subband coding has no characteristic that block distortion does not occur in principle, and hierarchical coding can be easily realized by recursively dividing low-band components. In still images, subband coding using wavelet transform is employed in JPEG2000, which is an international standard coding method.

When subband coding is applied to video coding, it is necessary to consider not only the correlation in the spatial direction of the signal but also the correlation in the time direction. Background Art Conventionally, research has been conducted on subband MC (Motion Compensation) in which the original signal is subbanded and motion compensation is performed for each subband region to remove correlation in time direction. However, the subband MC has a problem that the prediction efficiency in the high frequency subband is poor and the coding performance is low. On the other hand, three-dimensional wavelet coding has been developed in which sub-band coding is performed on each frame after temporal filtering with motion compensation in the spatial domain to remove correlation in time direction.

The description of a typical three-dimensional wavelet coding method is described below (for example, Non-Patent Document 1).

18-21 is a figure explaining the three-dimensional wavelet coding shown by the nonpatent literature 1. FIG. 18 is a block diagram showing the configuration of an encoder in three-dimensional wavelet coding. Hereinafter, the flow of a process of 3D wavelet coding is demonstrated using FIG.

With respect to the input image signal 2000 consisting of consecutive frames of N (N is a power of 2) sheets, the temporal filtering 201 performs wavelet transform in the temporal direction with motion compensation, thereby providing N / 2. Each time low-frequency subband signal 2001, the N / 2 time high-frequency subband signal 2002, and the motion information 2003 are generated. The temporal filtering 201 is recursively performed on the temporal low-band subband signal 2001. The one time low pass subband signal 2004 and the N-1 time high pass subband signal 2002 generated by the multi-step temporal filtering process are subband-divided in the spatial direction, respectively.

The spatial subband dividing unit 202 divides the temporal highband subband signal 2002 into two horizontally and vertical subbands, and includes one temporal highband spatial lowband subband 2005 and three temporal highband spatial highband subbands ( 2006). The temporal high-band spatial low-band subband 2005 is recursively divided by the spatial subband division 202. In this way, after subband division is performed in the spatial direction by a prescribed number of times, the temporal high frequency spatial low frequency subband signal 2005 and the temporal high frequency spatial high frequency subband signal 2006 are quantized by the quantization unit 204.

The temporal low-band subband signal 2004 is also divided into spatial subbands in multiple stages by the spatial subband division 203, so that the temporal low-band spatial low-band subband signal 2007 and the temporal low-band spatial high-band subband signal 2008 are obtained. Quantization is performed by the quantization unit 204. Each quantized signal is entropy encoded by the entropy encoder 205.

In addition, the motion information 2003 generated by the temporal filtering 201 is encoded by the motion information encoder 206. Each encoded signal is multiplexed by the multiplexer 207 and output as a bit stream 2010.

19 is a conceptual diagram illustrating subband division in the high order temporal and spatial directions in three-dimensional wavelet coding. The input signal 2011 is a third time low-frequency subband signal 2015 and a third time high-frequency subband signal 2016, a second time high-frequency subband signal 2014 by three-step time filtering. The temporal high frequency subband signal 2016 is divided into < RTI ID = 0.0 > 2016. < / RTI >

The third temporal low-band subband signal 2015 is a third-order temporal low-band spatial low-band subband signal 2017, and the third temporal low-band spatial high-band subband signal 2018, 2019, 2020), secondary temporal low-pass spatial high-band subband signals 2021, 2022, and 2023, and primary temporal low-pass spatial high-band subband signals 2024, 2025, and 2026.

The temporal high-band subband signal is divided into three stages of spatial subband division, and thus, the third-order temporal high-band spatial low-band subband signal 2027, the third-order temporal high-band spatial high-band subband signal 2028, 2029, and 2030, 2 The temporal high frequency spatial high frequency subband signals 2031, 2032, and 2033 and the primary temporal high frequency spatial high frequency subband signals 2034, 2035, and 2036 are divided.

When reconstructing an image signal having a spatial resolution or frame rate different from the input image signal from the bit stream, the decoder decodes only a part of the encoded data of the plurality of subband signals included in the bit stream.

19, the extraction process of the coded data of a subband is demonstrated.

In order to reconstruct a moving picture having a frame rate of 1/2, the decoder decodes the coded data corresponding to the temporal low-band subband 2015 and the temporal high-band subbands 2016 and 2014. In order to reconstruct a moving picture having a resolution of 1/2, the decoder performs subbands excluding the first temporal low-space high-band subband of the temporal low-band subband signal, that is, the temporal low-space low-band subband signal 2017 and the temporal low-space. Decode the high frequency subband signals 2018-2023.

Further, among the temporal high-band subband signals, subbands other than the first temporal high-space spatial high-band subband, that is, the temporal high-band spatial low-band subband signal 2027 and the temporal high-band spatial high-band subband signals 2028 to 2033 are decoded.

20 is a block diagram showing the configuration of an encoded data extraction device and a moving picture decoding device for extracting encoded data corresponding to a reduced image from a bit stream generated by three-dimensional wavelet encoding.

The encoded data extracting apparatus discards the spatial high-frequency subband signal 2038 lower than that of the bit stream 2010, generates a bit stream 2037 composed of encoded data of an appropriate subband, and sends it to the moving picture decoding apparatus 209. . The moving picture decoding apparatus 209 synthesizes the subband signals included in the bit stream 2037 and outputs the decoded picture signal 2047.

21 is a block diagram showing the configuration of the moving picture decoding apparatus 209. 21, the flow of decoding processing in three-dimensional wavelet coding will be described.

The demultiplexer 210 cuts the coded data of the subbands from the bit stream 2037, and, through the entropy decoder 211 and the dequantizer 212, the temporal high frequency spatial high frequency subband signal 2039, time A high frequency spatial low frequency subband signal 2040, a temporal high frequency spatial high frequency subband signal 2041, and a temporal high frequency spatial low frequency subband signal 2042 are generated.

The spatial subband synthesis 213 recursively subbands the temporal highband spatial highband subband signal 2039 and the temporal highband spatial lowband subband signal 2040 to generate a temporal highband subband signal 2043. .

Spatial subband synthesis 214 recursively subbands the temporal low-band spatial high-band subband signal 2041 and the temporal low-band spatial low-band subband signal 2042 to generate a temporal low-band subband signal 2044. . Here, the number of spatial subband synthesis processing is less than the number of spatial subband division processing performed by the encoder, and the number is determined by the spatial high-band subband signal discarded by the encoded data extraction apparatus 208.

In addition, the motion information decoding unit 215 decodes the code of the motion information output from the demultiplexer 210 to generate the motion information 2045. The vector reduction unit 216 reduces the length of the vector in accordance with the resolution ratio of the input signal at the time of encoding the motion information 2045 and the decoded image signal output by the decoder. This ratio is determined by the number of spatial high-band subband signals discarded by the extractor 208. For example, when the lowest order spatial high-band subband signal is discarded, the vector length is reduced to 1/2.

Thereafter, the time direction inverse filtering 217 performs encoding on the time high band subband signal 2043 and the time low band subband signal 2044 according to the motion information 2046 output by the vector reduction unit 216. The inverse transform of temporal filtering is performed to generate a decoded signal 2047.

Non-Patent Document 1: J. R. Ohm, "Three-dimensional subband coding with motion compensation", IEEE Trans, Image Processing, vol. 3, pp.559-571, Sept. 1999

<Start of invention>

Problems to be Solved by the Invention

In the conventional three-dimensional wavelet coding, there is a problem in that the image quality of a reduced resolution image obtained by applying spatial scalability is inferior to the image quality when encoding a previously reduced image as an input. There are three reasons for this.

The first reason is a mismatch of motion compensation. The time direction filtering 201 of the video encoding apparatus shown in FIG. 18 and the time direction inverse filtering of the video decoding apparatus shown in FIG. 21 include motion compensation prediction processing in units of blocks based on motion information. When the precision of the motion information is few, the pixel value obtained by the prediction process is obtained by interpolation processing from adjacent pixels. When generating a reduced image in the decoder, the interpolation process at the time of motion compensation prediction is performed based on the reduced vector for the spatial low-band subband signal. The interpolation filter at this time is determined irrespective of the interpolation processing performed by the temporal filtering 201 and the low pass filter in the spatial subband division 203 at the time of encoding. The results of subsampling after the interpolation to the input signal during encoding and the interpolation to the subsampled spatial low-band subband signal generally do not coincide. Inconsistency in prediction processing in motion compensation in the encoding device and the decoding device causes distortion in the decoded signal. This distortion accumulates as the time direction filtering becomes multistage.

22 and 23 are specific examples of conceptual diagrams showing a one-dimensional pixel array and filter coefficients multiplied by each pixel in order to explain that the interpolation processing and subsample processing of motion compensation are not acceptable. In the following description, a half wavelet is used for the subsample processing, and a 6-tap filter is used for the interpolation processing.

In Figs. 22 and 23, p0 to p11 and p0 'to p5', which are displays on the horizontal axis, represent pixels, and display values on the vertical axis extending from each coordinate are multiplied. In the upper figure of FIG. 22, the pixel value q4 of the position shifted by 1/2 from the pixel p4, and the filter value used for calculation of the pixel value q5 of the position shifted by 1/2 from the pixel p5 are shown, respectively. If the interpolation filter is B0 to B6,

q4 = B0 * p2 + B1 * p3 + B2 * p4 + B3 * p5 + B4 * p6 + B5 * p7

q5 = B0 * p3 + B1 * p4 + B2 * p5 + B3 * p6 + B4 * p7 + B5 * p8

It becomes The interpolation value at the reduced resolution obtained by the subsample processing from q4 and q5 is as shown in the drawing below in FIG.

(q4 + q5) / 2 = B0 / 2 * p2 + (B0 + B1) / 2 * p3 + (B1 + B2) / 2 * p4 + (B2 + B3) / 2 * p5 + (B3 + B4 / 2 * p6 + (B4 + B5) / 2 * p7 + B5 / 2 * p8

It becomes

In the above figure of FIG. 23, about the filter values used for calculation of the pixel value q2 'of the position shift | deviated by 1/4 from the pixel p2' with respect to the pixel p0'-p5 'obtained by performing subsample process to p0-p11, it shows. have. If the interpolation filter is C0 to C6,

q2 '= C0 * p0' + C1 * p1 '+ C2 * p2' + C3 * p3 '+ C4 * p4' + C5 * p5 '

It becomes here

p2 '= (p4 + p5) / 2

It is assumed that subsampling has been performed as described above.

When p2 'is represented using p0 to p11 as shown in the following figure of FIG. 23,

q2 '= C0 / 2 * p0 + C0 / 2 * p1 + C1 / 2 * p2 + C1 / 2 * p3 + C2 / 2 * p4 + C2 / 2 * p5 + C3 / 2 * p6 + C3 / 2 * p7 + C4 / 2 * p8 + C4 / 2 * p9 + C5 / 2 * p10 + C5 / 2 * p11

It becomes In general, since the interpolation filter is determined independently of the subsample processing, (q4 + q5) / 2 and q2 'do not coincide.

The second reason is the overhead of motion information. In the moving picture decoding apparatus in FIG. 21, the motion information needs to be the same as that generated at the time of encoding. In order to allocate the same motion information even in the reduced resolution, the size of the coding block and the precision of the motion information, which are the units of the motion compensation processing, become more sensitive than necessary. In FIG. 20, when the transmission rate from the encoded data extraction apparatus to the video decoding apparatus is limited, the code amount required for the motion information occupies most of the time, so that the minimum code amount cannot be assigned to the coefficient information.

24 is a conceptual diagram illustrating the overhead of motion information. In FIG. 24, the motion information group obtained by performing motion estimation on the frames B1 and C1 which subsampled the motion information group obtained by performing motion estimation on the frames B0 and C0 is called MV1. In motion estimation, the minimum size of the block to which the motion information is allocated and the precision of the motion information are determined. When the motion information group MV0 is reduced to 1/2 to fit the frames B1 and C1, the minimum size of the block is 1/2 and the precision of the motion information is doubled. The motion information group MV0, which is reduced in comparison with the motion information group MV1, has a large number of motion information and a large amount of code required to represent individual motion information.

As a third reason, it is a problem of scalability. In three-dimensional wavelet coding, the parameters and processing modules used in the encoder are the same in all bit streams at different resolutions and frame rates obtained by the application of scalability. In order to encode with high efficiency, the delay determined by these parameters and the calculation amount determined by the processing module become large. Part of the encoded data generated in this way is to be distributed to, for example, a mobile terminal by applying the scalability, which results in a large delay and a large amount of computation. On the contrary, if a parameter or a processing module is determined in consideration of an application at a low rate, the coding performance at a high rate is greatly reduced.

Accordingly, the present invention has been invented in view of the above-described problems, and its object is to provide a moving picture encoding apparatus having the same image quality as that of a decoded image when a decoded signal in all hierarchies is coded in a single hierarchical layer. And a moving picture decoding apparatus, a method thereof, and a control program thereof.

Means for solving the problem

In a first aspect of the present invention, a video encoding apparatus generates a video signal corresponding to the temporal high frequency component by using a temporal low frequency component and a temporal high frequency component obtained by temporally hierarchizing a video signal. And a time-space division filtering unit for outputting a reduced image signal subjected to the reduction processing.

According to a second aspect of the present invention, there is provided a video encoding apparatus which performs a reduction process on a temporal low pass component and a temporal high pass component obtained by temporal hierarchizing a video signal, and uses the result of the reduction processing to correspond to the temporal high pass component. And a time-space division filtering unit for generating a reduced image signal of a moving image.

A third aspect of the present invention provides a video encoding apparatus including a time-space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the time-space division filtering unit is configured to perform the moving image signal. Corresponding to the temporal high frequency component using the temporal low frequency component obtained by temporally hierarchizing the temporal low frequency component, the temporal high frequency component obtained by temporally After the reconstruction of the moving image signal, the reduced image signal subjected to the reduction processing is generated.

A fourth aspect of the present invention provides a video encoding apparatus including a time-space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the time-space division filtering unit is configured to perform the moving image signal. Temporally hierarchize to obtain a temporal low pass component and a temporal high pass component, and spatially stratify the temporal high pass component to obtain a spatial high pass component, and use the temporal low pass component and the temporal high pass component to obtain the moving picture signal corresponding to the temporal high pass component. After the reconstruction, the reduced image signal subjected to the reduction processing to the reconstruction result is generated.

A fifth aspect of the present invention provides a video encoding apparatus including a time-space division filtering unit for layering a moving image signal, and a signal encoding processing unit encoding the layered signal, wherein the time-space division filtering unit comprises: the video signal The temporal low frequency component obtained by time layering, the temporal high frequency component obtained by spatial layering the temporal high frequency component obtained by temporally layering the video signal, and the temporal low frequency component and the temporal high frequency component are reduced. A reduced image signal of the moving image signal corresponding to the temporal high frequency component is generated using the processing result.

A sixth invention for solving the above problems is a video encoding apparatus comprising a time-space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the time-space division filtering unit time-decodes the moving image signal. Filtering in the direction to generate motion information indicating a motion between the temporal low pass signal and the temporal high pass signal and the image signal, and using the temporal high pass signal and the temporal low pass signal to the temporal high pass signal. A reduced image generating unit for generating a reduced image signal obtained by reducing a corresponding moving image signal; and a high frequency signal generating unit for generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, wherein the temporal low frequency signal or The time-space division filter for the reduced image signal Addition executed by being characterized in that after the layered video signal, said time low-band signal and said reduced decoded image signal and the time high-band space high-band signal and the motion information on the signal coded by the encoding portion.

According to a seventh aspect of the present invention, in the sixth aspect of the present invention, the reduced image generation unit performs a motion compensation on a temporal low-pass signal based on the motion information to generate a predictive image, and the prediction Reduced decoding, which is a spatial low-band component, by performing spatial filtering on a temporal high frequency signal inverse transform unit that generates a video signal corresponding to the temporal high frequency signal from an image and a temporal high frequency signal, and a video signal generated by the temporal high frequency signal inverse transform unit It is characterized by having a low-pass signal generation part which produces | generates an image signal.

In the eighth invention which solves the said subject, in said 7th invention, the said time high-pass signal performs a weighting process compared with the said time low-pass signal, or the weighting process with respect to the part in which the said time high-pass signal exists. And a weighting unit for outputting to the temporal high frequency signal inverse transform unit.

According to a ninth aspect of the present invention, in the sixth aspect of the present invention, the reduced image generating unit performs spatial filtering on the temporal low pass signal and the temporal high pass signal to respectively perform the temporal low pass spatial low pass signal and the temporal high pass spatial low pass signal, respectively. A low frequency signal generator for generating a motion information; a motion information converter for converting motion information according to the resolution conversion ratio of the spatial filtering in the low frequency signal generator; and based on the motion information converted by the motion information converter. A motion compensator for performing motion compensation on the temporal low-pass spatial low-band signal to generate a predictive image, and a temporal high-band signal inverse transform unit for generating a reduced image signal using the predicted image and the temporal high-band spatial low-band signal. It is done.

According to a ninth invention for solving the above-mentioned problems, in the ninth invention, a weighting process is performed on the temporal high-band spatial low-band signal, or a weighting process is performed on a portion where the temporal high-band spatial low-band signal is present. And a weighting unit for outputting to the temporal high frequency signal inverse transform unit.

In the eleventh invention for solving the above-mentioned problems, in the sixth to tenth inventions, the high frequency signal generation unit is frequency component extraction by subband division.

In the twelfth invention for solving the above-mentioned problems, in the sixth to eleventh inventions, the low-band signal generation unit is frequency component extraction by subband division.

According to a thirteenth invention for solving the above-mentioned problems, in the sixth to tenth inventions, the low-band signal generator generates a reduced image signal by a first subsample filter, and the high-band signal generator generates the first subsample filter. And generating a high pass signal by means of a second subsample filter paired with.

According to a sixteenth invention for solving the above-mentioned problems, in the sixth to thirteenth inventions, the temporal direction filtering unit includes a motion estimation unit for generating motion information on a moving image signal, and a motion estimation unit using the motion information. A first motion compensator for performing motion compensation on the included reference signal to generate a prediction signal, a time highband signal generator for generating a temporal high frequency signal using the non-reference signal and the prediction signal included in the video signal; A second motion compensator for performing motion compensation on the temporal high frequency signal using the motion information to generate a motion compensation time high frequency signal corresponding to the prediction signal, and a time from the motion compensation time high frequency signal and the reference signal Characterized in that it has a time low-pass signal generator for generating a low-pass signal.

According to a fifteenth aspect of the present invention, in the sixth to thirteenth aspects, the temporal direction filtering unit includes a motion estimation unit for generating motion information on a moving image signal, and the motion signal using the motion information. A motion compensator for performing motion compensation on the included reference signal to generate a prediction signal, and a differential signal generator for generating a difference signal between the non-reference signal and the prediction signal included in the moving picture signal, It outputs as a time low pass signal by conversion, and outputs the said difference signal as a time high pass signal.

A sixteenth invention for solving the above problems comprises: a time-space frequency divider for frequency-dividing a moving image signal in a time direction and a spatial direction to generate a reduced image signal, a space-time high-frequency signal, and motion information obtained by reducing the moving image signal; And a reduced image signal encoder for encoding the reduced image signal, a spatial high frequency signal encoder for encoding the space-time high-band signal, a motion information encoder for encoding the motion information, the reduced image signal encoder, and the A video encoding apparatus comprising a multiplexing unit for multiplexing encoded data output from a space-time high-band signal encoding unit and motion information encoded data output from the motion information encoding unit to generate a bit stream that is output. Time bass signal And a temporal high frequency spatial high frequency signal corresponding to the spatial high frequency signal of the temporal high frequency signal, a time space division filtering unit which reduces a video signal corresponding to the temporal high frequency signal and divides the video signal into a temporal high frequency reduced image signal, and the temporal low frequency signal. And a spatial division filtering unit for generating a temporal low frequency spatial low frequency signal and a temporal low frequency spatial high frequency signal by frequency division in the spatial direction, and after the temporal spatial division filtering unit recursively processes the temporal low frequency signal as an input, A temporal low pass spatial low pass signal and the reduced decoded image signal are used as the temporal high pass reduced image signal, and the temporal high pass spatial high pass signal and the temporal low pass spatial high pass signal are output as a space time high pass signal.

A seventeenth invention for solving the above problems is a video decoding apparatus including a signal decoding processing unit for decoding coded data of a hierarchically encoded video signal and generating a layered signal, and a time-space synthesis filtering unit for synthesizing the layered signal. And, based on the decoded image signal that is a result of decoding in an arbitrary resolution layer, the temporal spatial synthesis filtering unit generates a spatial low pass component by spatial layering in the temporal high pass component by temporal layering, and then performs spatial direction frequency synthesis and temporal Directional frequency synthesis is performed to generate a decoded image signal of a higher resolution layer.

An eighteenth aspect of the present invention provides a video decoding apparatus including a signal decoding processing unit for decoding encoded data to generate a layered signal, and a time-space synthesis filtering unit for synthesizing the layered image signal. A temporal high frequency spatial low pass signal for generating a temporal high frequency spatial low frequency signal which is a spatial low frequency component of the temporal high frequency signal paired with the temporal low frequency signal from a temporal low frequency signal and a reduced decoded image signal that is a result of synthesis at an arbitrary resolution layer. A reconstruction unit, a spatial high frequency spatial high frequency signal which is a spatial high frequency component of the temporal high frequency signal, and a spatial synthesis filtering unit which synthesizes the temporal high frequency spatial low frequency signal to generate a temporal high frequency signal, the temporal high frequency signal and the temporal low frequency Time direction invert for generating decoded image signal from signal and motion information Having parts of the ring, and from the additional signal decoding said encoded data characterized in that for decoding and said time low-band signal and said reduced decoded image signal and the time high-band high-pass spatial signal and the motion information.

According to a nineteenth aspect of the present invention, in the eighteenth aspect, the temporal high-band spatial low-band signal reconstruction unit performs motion compensation on the temporal low-band signal based on the motion information to generate a predictive image. And a low pass signal generator for generating a spatial low pass prediction signal which is a spatial low pass component of the predicted image, and a time high pass signal generator for generating a temporal high pass component from the spatial low pass prediction signal and the reduced decoded image signal. And outputting the output of the high frequency signal generator as a temporal high frequency spatial low frequency signal.

In a twentieth invention for solving the above-mentioned problems, in the nineteenth invention, an inverse weighting unit is provided to the output of the temporal high frequency signal generation unit to perform a weighting process for compensating weighting performed at the time of encoding. And outputting a negative output as a temporal high-band spatial low-band signal.

According to a twenty-first aspect of the present invention, in the eighteenth aspect of the present invention, the temporal high-band spatial low-band signal reconstruction unit includes: a low-band signal generator for generating a spatial low-band component of the temporal low-band signal and outputting it as a temporal low-band spatial low-band signal; And a motion information converter for converting motion information according to the resolution conversion ratio of the input / output image of the low-band signal generator and motion compensation for the temporal low-band spatial low-band signal based on the motion information converted by the motion information converter. And a motion compensator for generating a predictive image, and a temporal high frequency signal generator for generating temporal high frequency components from the predicted image and the decoded decoded image signal, and outputting the output of the temporal high frequency signal generator as a temporal high frequency spatial low frequency signal. It is characterized by.

In a twenty-second aspect of the present invention, in the twenty-first aspect of the present invention, the output of the temporal high-frequency signal generation unit includes a reverse weighting unit for performing a weighting process for compensating weighting performed at the time of encoding. And outputting a negative output as a temporal high-band spatial low-band signal.

In a twenty-third aspect of the present invention, in the nineteenth to twenty-second aspects of the present invention, the low-band signal generation unit is a low-pass process by subband division.

In a twenty-fourth aspect of the present invention, in the nineteenth to twenty-third aspects of the present invention, the spatial synthesis filtering unit is subband synthesis, which is an inverse transform of the subband division.

In a twenty fifth aspect of the present invention, in the nineteenth to twenty-fourth aspects of the present invention, the low-band signal generation unit generates a reduced image by a subsample filter.

In the twenty-sixth aspect of the present invention, in the nineteenth to twenty-fifth aspects, the high-frequency signal generated by the second sub-sample filter paired with the sub-sample filter, and the sub-sample It is characterized in that the synthesis of the low-pass signal generated by the filter.

According to a twenty-seventh invention for solving the above-mentioned problems, in the eighteenth to twenty-sixth aspects, the temporal inverse filtering unit performs motion compensation on the temporal high frequency signal based on the motion information to generate a motion compensation time high frequency signal. A first motion compensation unit configured to generate a first decoded image signal from the motion compensation time high frequency signal and the time low frequency signal, and a motion based on the motion information with respect to the first decoded image signal A second motion compensator for compensating to generate a predictive signal, and a temporal high frequency signal inverse transform unit for generating a second decoded image signal from the predicted signal and the temporal high frequency signal, wherein the first decoded image signal and the second The decoded image signal is integrated into a decoded image signal to be output.

In the twenty-eighth invention for solving the above-mentioned problems, in the eighteenth to twenty-seventh inventions, the time direction inverse filtering unit adds the time low pass signal by adding the time high pass signal after performing motion compensation on the time low pass signal. It synthesize | combines and outputs as a decoded image signal.

The 29th invention which solves the said subject is a demultiplexing part which demultiplexes a layered bit stream, produces | generates the lowest coded data, the redundant coded data, and the motion information coded data, and decodes the lowest coded data, and reduces a reduced image signal. A reduced-image signal decoder for generating a video signal; a space-time high-band signal decoder for decoding an excess encoded data to generate a space-time high-band signal; a motion information decoder for decoding the motion information encoded data to generate motion information; A video decoding apparatus comprising a time-space frequency synthesizing unit for generating a decoded image signal from an image signal, the space-time high-band signal and the motion information, wherein the time-space frequency synthesizing unit comprises a time low-space spatial low-band signal and the space-time high-band in the reduced image signal. Temporal low-space high during signal A spatial synthesis filter for synthesizing a signal to generate a temporal low pass signal, and reconstructing the temporal high pass signal corresponding to the temporal low pass signal into the temporal low pass signal, the reduced image signal, and the spatiotemporal high pass signal, and then performing the temporal low pass signal And a time-space synthesis filtering section for outputting a decoded image signal by combining with.

A thirty-eighth invention that solves the above-mentioned problems is a video encoding method, which generates a video signal corresponding to the time high-band component using a time low-pass component and a time high-band component obtained by temporally hierarchizing a video signal. A reduction process is performed to generate a reduced picture signal of a moving picture.

According to a thirty-first aspect of the present invention, there is provided a video encoding method, which performs a reduction process on a temporal low pass component and a temporal high pass component obtained by temporal hierarchizing a video signal, and uses the result of the reduction processing to correspond to the temporal high pass component. A reduced picture signal of a moving picture is generated.

A thirty-second aspect of the present invention provides a video encoding method comprising time-space division filtering for stratifying a moving image signal and signal encoding processing for encoding the layered signal, wherein the time-space division filtering is performed for the video signal. Corresponding to the temporal high frequency component using the temporal low frequency component obtained by temporally hierarchizing the temporal low frequency component, the temporal high frequency component obtained by temporal layering the moving picture signal, and the temporal high frequency component using the temporal low frequency component and the temporal high frequency component After the reconstruction of the moving image signal, the reduced image signal subjected to the reduction processing is generated.

A thirty-third invention for solving the above-mentioned problems is a video encoding method comprising time-space division filtering for stratifying a moving image signal and a signal encoding process for encoding the layered signal, wherein the time-space division filtering performs the operation on the video signal. Temporally hierarchize to obtain a temporal low frequency component and a temporal high frequency component, and spatially hierarchize the temporal high frequency component to obtain a spatial high frequency component, and reconstruct the video signal corresponding to the temporal high frequency component using the temporal low frequency component and the temporal high frequency component After that, a reduced image signal subjected to the reduction process is generated.

A thirty-fourth invention for solving the above problems is a video encoding method comprising time-space division filtering for stratifying a moving image signal and signal encoding processing for encoding the layered signal, wherein the time-space division filtering performs the operation on the moving image signal. A reduction process is performed on the temporal low pass component obtained by temporal layering, the temporal high pass component obtained by spatial layering the temporal high pass component obtained by temporal layering the video signal, and the temporal low pass component and the temporal high pass component. The result is characterized in that a reduced picture signal of a moving picture signal corresponding to the temporal high frequency component is generated.

A thirty-fifth aspect of the present invention provides a video encoding method including time-space division filtering that stratifies a moving picture signal and signal encoding processing that encodes the hierarchical signal, wherein the time-space division filtering moves the video signal in the time direction. A time direction filtering step of generating motion information representing motion between the time low pass signal and the time high pass signal and the image signal, and using the time high pass signal and the time low pass signal to correspond to the time high pass signal. And a high frequency signal generating step of generating a reduced image signal obtained by reducing a moving image signal, and a high frequency signal generating step of generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal. The time space division fill for the reduced picture signal After layering the video signal being a ring step it is executed, characterized in that for encoding the time low-band signal and said reduced decoded image signal and the time high-band high-pass spatial signal and the motion information in the signal coding step.

According to a thirty sixth aspect of the present invention, in the thirty fifth aspect, the reduced image generating step includes a motion compensation step of generating a predictive image by performing motion compensation on a temporal low-pass signal based on the motion information; A time-decoding signal inverse transform step of generating a moving picture signal corresponding to the time-high frequency signal from a predicted image and the time-high frequency signal, and a spatially decoded decoded image that is a spatial low-band component by performing spatial filtering on the video signal generated by the temporal high-band signal inverse transform unit. And a low pass signal generating step of generating a signal.

According to a thirty seventh aspect of the present invention, in the thirty-third aspect of the present invention, the temporal high frequency signal is weighted in comparison with the temporal low frequency signal before the temporal high frequency signal inverse transform step, or the temporal high frequency signal A weighting process is performed for the part having.

In the thirty-eighth aspect of the present invention, in the thirty-fifth aspect of the present invention, the reduced image generating step performs spatial filtering on the temporal low pass signal and the temporal high pass signal to perform a temporal low pass spatial low pass signal and a temporal high pass spatial low pass signal. On the basis of the low pass signal generation step to generate each, a motion information conversion step of converting motion information according to the resolution conversion ratio of the spatial filtering in the low pass signal generation step, and the motion information converted by the motion information conversion step. A motion compensation step of performing motion compensation on the temporal low-band spatial low-band signal to generate a predictive image, and a temporal high-band signal inverse transform step of generating a reduced image signal using the predicted image and the temporal high-band spatial low-band signal It is characterized by.

According to a thirty-ninth aspect of the present invention, in the thirty-eighth aspect of the present invention, a weighting process is performed on the temporal high-frequency spatial low-frequency signal before the temporal high-frequency signal inverse transform step, or the temporal high-frequency spatial low-frequency signal A weighting process is performed.

40th invention which solves the said subject is the invention in any one of the said 35th-39th said WHEREIN: The said high frequency signal generation step is characterized by extracting frequency components by subband division.

In a forty-first aspect of the present invention, in the thirty-fifth aspect of the present invention, the low-pass signal generating step is frequency component extraction by subband division.

In a forty-second aspect of the present invention, in the thirty-third aspect of the present invention, the low-band signal generation step generates a reduced image signal by a first subsample filter, and the high-band signal generation step is performed. A high frequency signal is generated by the second subsample filter paired with the first subsample filter.

In the 43rd invention which solves the said subject, in the invention of any one of said 35-42, the said time direction filtering step uses the motion estimation step which produces | generates motion information with respect to a moving image signal, and the said motion information. Motion compensation for a reference signal included in the video signal to generate a prediction signal, and a time for generating a temporal high frequency signal using the non-reference signal and the prediction signal included in the video signal. A second motion compensation step of generating a motion compensation time high frequency signal corresponding to the prediction signal by performing motion compensation on the temporal high frequency signal using the high frequency signal generation step; A time low pass signal generating step of generating a time low pass signal from the reference signal It is characterized by having.

According to a forty-fourth aspect of the present invention, there is provided a motion estimation step of generating motion information for a moving image signal, wherein the time-direction filtering step includes the motion estimation step and the motion information. A motion compensation step of generating a prediction signal by performing motion compensation on the reference signal included in the moving picture signal, and a differential signal generating step of generating a difference signal between the non-reference signal and the prediction signal included in the moving picture signal. And outputs the reference signal as a time low pass signal without conversion, and outputs the difference signal as a time high pass signal.

The forty-fifth aspect of the present invention provides a time and space frequency division step of performing frequency division on a moving image signal in a time direction and a spatial direction to generate a reduced image signal, a space-time high-frequency signal, and motion information obtained by reducing the moving image signal; A reduced image signal encoding step of encoding the reduced image signal, a spatial high frequency signal encoding step when encoding the space-time high-frequency signal, a motion information encoding step of encoding the motion information, the reduced image signal encoding step and the A video encoding method comprising a multiplexing step of multiplexing encoded data generated in a space-time high-frequency signal encoding step and motion information encoded data generated in the motion information encoding step to generate a bit stream that is output. , Time-space division filtering for dividing an image signal into a temporal low-frequency signal, a temporal high-frequency spatial high-frequency signal corresponding to the spatial direction high-frequency of the temporal high-frequency signal, and a video signal corresponding to the temporal high-frequency signal and dividing the video signal into a temporal high-frequency reduced image signal And a spatial division filtering step of frequency dividing the temporal low frequency signal in a spatial direction to generate a temporal low frequency low frequency signal and a temporal low frequency spatial high frequency signal, wherein the temporal spatial division filtering step inputs the temporal low frequency signal After the recursive processing is performed, the temporal low-band spatial low-band signal and the reduced-decoded image signal are regarded as the temporal high-band reduced image signal, and the temporal high-band spatial high-band signal and the temporal low-band spatial high-band signal are used as the spatiotemporal high-band signal. It is characterized by outputting.

A forty-fifth aspect of the present invention provides a video decoding method including a signal decoding processing step of decoding encoded data of a hierarchically encoded video signal, generating a layered signal, and a time-space synthesis filtering step of synthesizing the layered signal. As a method, the temporal spatial synthesis filtering step generates spatial low-pass components by spatial layering in temporal high-band components by temporal hierarchization based on a decoded image signal that is a result of decoding in an arbitrary resolution layer. Combination and time-direction frequency synthesis are performed to generate a decoded image signal of a higher resolution layer.

A forty-ninth aspect of the present invention provides a video decoding method including a signal decoding processing step of decoding coded data to generate a layered signal, and a time-space synthesis filtering step of synthesizing the layered image signal. The filtering step includes a temporal high pass for generating a temporal high pass spatial low pass signal, which is a spatial low pass component of the temporal high pass signal paired with the temporal low pass signal, from the temporal low pass signal and the reduced decoded image signal that is a result of combining at an arbitrary resolution layer. A spatial low pass signal reconstruction step, a spatial high pass spatial high pass signal which is a spatial high pass component of the temporal high pass signal, a spatial synthesis filtering step of synthesizing the temporal high pass spatial low pass signal to generate a temporal high pass signal, and the temporal high pass signal Generating a decoded image signal from the time low-pass signal and motion information And a temporal direction inverse filtering step, wherein the signal decoding processing step decodes the temporal low pass signal, the reduced decoded image signal, the temporal high frequency spatial high pass signal, and the motion information from the encoded data.

In a forty-eighth invention for solving the above-mentioned problems, in the forty-ninth aspect of the present invention, the temporal high-band spatial low-band signal reconstruction step performs motion compensation on the temporal low-band signal based on the motion information to generate a predictive image. A low pass signal generating step of generating a spatial low pass prediction signal which is a spatial low pass component of said predicted image, and a temporal high pass signal generating step of generating a temporal high pass component from said spatial low pass prediction signal and a reduced decoded image signal, And outputting the output of the temporal high pass signal generating step as a temporal high pass spatial low pass signal.

According to a forty-ninth aspect of the present invention, in the forty-eighth aspect of the present invention, a weighting process for compensating weighting performed at the time of encoding is performed on the temporal high-band component generated in the temporal high-band signal generation step, and the weighting is performed. And outputting the received signal as a temporal high-band spatial low-band signal.

In a fifty-fifth aspect of the present invention, in the forty-ninth aspect of the present invention, the temporal high-band spatial low-band signal reconstruction step generates a low-band signal that generates a spatial low-pass component of the temporal low-band signal and outputs it as a temporal low-band spatial low-band signal. A motion information converting step of converting motion information in accordance with a resolution conversion ratio of an input / output image of the low-band signal generation unit and a motion of the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting unit. A motion compensation step of compensating to generate a predictive image, and a time high pass signal generating step of generating a temporal high frequency component from the predicted image and the reduced decoded image signal, and outputting the output of the temporal high frequency signal generating step to a temporal high frequency spatial low range. It outputs as a signal.

In a fifty-ninth aspect of the present invention, in a fifty-fifth aspect of the present invention, a weighting process for compensating weighting performed at the time of encoding is performed to the output of the temporal high-band signal generation step, and this is a temporal high-band spatial low-band signal. It is characterized by outputting as.

A fifty-second aspect of the present invention, wherein the low-pass signal generating step is low-pass processing by subband division according to any one of the forty-ninth to fifty-ninth aspects.

In a fifty-third aspect of the invention, in one of the forty-ninth to fifty-second aspects, the spatial synthesis filtering step is subband synthesis, which is an inverse transform of the subband division.

In a fifty-fourth aspect of the present invention, the low-pass signal generation step generates a reduced image by a subsample filter according to any one of the forty-ninth to fifty-third aspect.

In a fifty-fifth aspect of the present invention, in the fifty-fifth aspect of the present invention, the spatial synthesis filtering step includes a high pass signal generated by a second subsample filter paired with the subsample filter, and the subsample filter. It is characterized in that the synthesis of the generated low-pass signal.

In a fifty sixth aspect of the present invention, there is provided the motion compensation according to any one of the forty-ninth to fifty-seventh aspects, wherein the time direction inverse filtering step performs motion compensation on the temporal high-frequency signal based on the motion information. A first motion compensation step of generating a temporal high frequency signal, a time low frequency signal inverse transform step of generating a first decoded image signal from the motion compensation time high frequency signal and the time low frequency signal, and the motion with respect to the first decoded image signal A first motion decoding step of performing motion compensation based on the information to generate a predictive signal, and a temporal high frequency signal inverse transform step of generating a second decoded image signal from the predictive signal and the temporal high frequency signal, wherein the first decoding A decoded image signal which is outputted by integrating the image signal and the second decoded image signal It shall be.

In the fifty-seventh invention for solving the above-mentioned problems, in the invention of any one of the forty-ninth to fifty-seventh embodiments, the time direction inverse filtering step is performed by adding the time high-band signal after performing motion compensation on the time low-pass signal. The time low pass signal is synthesized and output as a decoded image signal.

A fifty-eighth aspect of the present invention provides a demultiplexing step of demultiplexing a layered bit stream to generate lowest coded data, redundant coded data, and motion information coded data, and decoded minimum signal A reduced image signal decoding step of generating a signal; a space-time high-band signal decoding step of decoding an excess encoded data to generate a space-time high-band signal; a motion information decoding step of decoding the motion information encoded data to generate motion information; A moving picture decoding method comprising a time-space frequency synthesizing step of generating a decoded image signal from an image signal, the space-time high-band signal and the motion information, wherein the time-space frequency synthesizing step comprises: a time low-space spatial low-pass signal in the reduced image signal; Space-Time High-Frequency Signal A spatial synthesis filtering step of synthesizing a temporal low pass spatial high pass signal of the temporal low pass signal, and reconstructing the temporal high pass signal corresponding to the temporal low pass signal into the temporal low pass signal, the reduced image signal, and the spatiotemporal high pass signal And a time-space synthesis filtering step of outputting a decoded image signal by combining with the time low-pass signal.

In a fifty-ninth aspect of the present invention, a control program of a moving picture encoding apparatus is provided. The control program uses a time low pass component and a time high pass component obtained by time hierarchizing a moving picture signal to the time high pass component. It is characterized by generating a corresponding video signal and functioning as a time-space division filtering unit for outputting a reduced video signal subjected to the reduction process as a result of this generation.

In the sixtieth aspect of the present invention, there is provided a control program of a moving picture encoding apparatus, wherein the control program performs a reduction processing on a time low pass component and a time high pass component obtained by time hierarchizing a moving picture signal. And using the result of the reduction processing, as a time-space division filtering unit for generating a reduced image signal of a moving image corresponding to the temporal high frequency component.

A sixty-fifth aspect of the present invention provides a control program of a video encoding apparatus including a time space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the control program is the time space. A segmentation filtering unit uses a temporal low pass component obtained by temporally hierarchizing the video signal, a spatial high pass component obtained by spatial layering the temporal high pass component obtained by temporal hierarchizing the video signal, and the temporal low pass component and the temporal high pass component. And reconstructing the moving image signal corresponding to the temporal high frequency component, and then generating a reduced image signal subjected to a reduction process as a result of the reconstruction.

A sixty-fifth aspect of the present invention provides a control program of a moving picture encoding apparatus including a time space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the control program is the time space. The segmentation filtering unit temporally hierarchies the video signal to obtain a temporal low pass component and a temporal high pass component, and spatially hierarchies the temporal high pass component to obtain a spatial high pass component, and uses the temporal low pass component and the temporal high pass component to temporal high pass component. And reconstructing the moving image signal corresponding to the function, and generating a reduced image signal subjected to the reduction processing to the reconstruction result.

A sixty-fifth aspect of the present invention provides a control program of a video encoding apparatus including a time-space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the control program is the time space. A segmentation filtering unit reduces the temporal low frequency component obtained by temporally hierarchizing the moving image signal, the spatial high frequency component obtained by spatially hierarchizing the temporal high frequency component obtained by temporally hierarchizing the video signal, and the temporal low frequency component and the temporal high frequency component. And using the result of the reduction processing, to generate a reduced image signal of a moving image signal corresponding to the temporal high frequency component.

A sixty-fifth aspect of the present invention provides a control program of a moving picture encoding apparatus including a time space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding the layered signal, wherein the control program is the time space division filtering. And a temporal filtering unit for filtering the moving image signal in a time direction to generate motion information indicating motion between the temporal low frequency signal, the temporal high frequency signal, and the image signal, and using the temporal high frequency signal and the temporal low frequency signal. A reduced image generating unit for generating a reduced image signal obtained by reducing a moving image signal corresponding to the temporal high frequency signal, and a high frequency signal generating unit for generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal; Function the time low-pass signal or the axis And after the temporal spatial division filtering unit is performed on the image signal to stratify the moving image signal, the signal encoding processing unit encodes the temporal low frequency signal, the reduced decoded image signal, the temporal high frequency spatial high frequency signal, and the motion information. do.

In a sixty-ninth aspect of the present invention, in a sixty-second aspect of the present invention, the control program includes: a motion compensation unit configured to perform motion compensation on a temporal low-band signal based on motion information to generate a predictive image; And a temporal high frequency signal inverse transform unit for generating a moving image signal corresponding to the temporal high frequency signal from the predicted image and the temporal high frequency signal, and performing spatial filtering on the video signal generated by the temporal high frequency signal inverse transform unit. And function as a low-band signal generation unit for generating a reduced decoded image signal.

In the sixty-fifth aspect of the present invention, in the sixty-ninth aspect, the control program performs a weighting process on the moving picture coding apparatus, and the temporal high pass signal is compared with the temporal low pass signal, or the temporal high pass A weighting process is performed on a portion having a signal to function as a weighting unit outputted to the temporal high frequency signal inverse transform unit.

In a sixty-ninth aspect of the present invention, in the sixty-eighth aspect of the present invention, the control program performs spatial filtering on the temporal low pass signal and the temporal high pass signal by performing the spatial low frequency signal and the temporal high pass space on the reduced image generation unit. A low pass signal generator for generating a low pass signal, a motion information converter for converting motion information according to the resolution conversion ratio of the spatial filtering in the low pass signal generator, and motion information converted by the motion information converter A motion compensator for performing motion compensation on the temporal low-pass spatial low-band signal to generate a predictive image, and generating a reduced image signal by using the predicted image and the temporal high-band spatial low-band signal. It functions as a part.

In the sixty-eighth aspect of the present invention, in the sixty-ninth aspect, the control program performs a weighting process on the temporal high-band spatial low-band signal, or the portion in which the temporal high-band spatial low-band signal is present. A weighting process is performed on the control unit so as to function as a weighting unit to be outputted to the temporal high frequency signal inverse transform unit.

In a sixty-ninth aspect of the present invention, the high frequency signal generation unit is a frequency component extraction by subband division.

According to a seventyth aspect of the present invention, there is provided a frequency component extraction by subband dividing according to the invention of any one of the sixty-sixth to sixty-ninth aspects.

According to a seventy-first aspect of the present invention, in one of the sixty-eighth aspects, the low-band signal generation unit generates a reduced image signal by a first subsample filter, and the high-band signal generation unit generates the reduced image signal. A high pass signal is generated by a second subsample filter paired with one subsample filter.

In a seventy-eighth aspect of the present invention, the control program includes: a motion estimation unit for generating motion information with respect to a moving picture signal; A first motion compensator for performing motion compensation on a reference signal included in the video signal using motion information to generate a prediction signal, a non-reference signal included in the video signal, and a temporal high frequency signal using the prediction signal A temporal high frequency signal generator for generating a motion compensation signal; a second motion compensator for performing motion compensation on the temporal high frequency signal using the motion information to generate a motion compensation time high frequency signal corresponding to the predicted signal; Time low-pass signal generation from the time high-pass signal and the reference signal It functions as a part.

In a seventy-third aspect of the present invention, the control program includes: a motion estimation unit for generating motion information on a moving image signal; A motion compensation unit configured to perform motion compensation on the reference signal included in the moving picture signal using information, and generate a differential signal generating a difference signal between the non-referenced signal included in the moving picture signal and the prediction signal It functions as a negative, outputs the reference signal as a time low pass signal without conversion, and outputs the difference signal as a time high pass signal.

The 74th invention which solves the said subject is the time-space frequency division part which performs frequency division with respect to a moving picture signal in the time direction and a spatial direction, and produces | generates the reduced picture signal which reduced the said moving picture signal, the space-time high-pass signal, and motion information; And a reduced image signal encoder for encoding the reduced image signal, a space-time high-frequency signal encoder for encoding the space-time high-band signal, a motion information encoder for encoding the motion information, the reduced image signal encoder, and the space-time A control program of a moving picture encoding apparatus comprising a multiplexer for multiplexing encoded data output from a high frequency signal encoder and motion coded data output from the motion information encoder to generate a bit stream that is output. The dividing unit divides the moving image signal into a temporal high frequency reduced image signal by reducing the temporal low frequency signal, the temporal high frequency spatial high frequency signal corresponding to the spatial direction high frequency of the temporal high frequency signal, and the video signal corresponding to the temporal high frequency signal And a time space division filtering unit and a space division filtering unit for generating a time low frequency spatial low frequency signal and a time low frequency spatial high frequency signal by frequency-dividing the time low frequency signal in a spatial direction, wherein the time space division filtering part is the time low frequency signal. After recursively processing with the input as, the temporal high frequency spatial high frequency signal and the temporal low frequency spatial high frequency signal are regarded as the temporal high frequency reduced image signal and the temporal high frequency spatial low frequency signal and the reduced decoded image signal are It outputs as a signal.

A seventy-fifth aspect of the present invention provides a video decoding apparatus including a signal decoding processing unit for decoding encoded data of a hierarchically encoded video signal, generating a layered signal, and a time-space synthesis filtering unit for synthesizing the layered signal. As the control program, the control program generates the spatial low pass component by spatial layering in the temporal high pass component by temporal hierarchization based on the decoded image signal which is a result of decoding in an arbitrary resolution layer. And performing spatial direction frequency synthesis and temporal direction frequency synthesis to generate a decoded image signal of a higher resolution resolution layer.

The 76th invention which solves the said subject is a control program of the moving image decoding apparatus provided with the signal-decoding process part which decodes the encoded data and produces | generates a layered signal, and the time-space synthesis | combination filtering part which synthesize | combines a layered image signal, The said control program Is a temporal high pass spatial low pass signal which is a spatial low pass component of the temporal high pass signal paired with the temporal low pass signal from a temporal low pass signal and a reduced decoded image signal which is a result of synthesis in an arbitrary resolution layer. A temporal high frequency spatial low pass signal reconstruction unit generating a temporal high frequency spatial high frequency signal, which is a spatial high frequency component of the temporal high frequency signal, and a spatial synthesis filtering unit generating the temporal high frequency signal by synthesizing the temporal high frequency spatial low frequency signal, A time high pass signal, the time low pass signal, and motion information Function as a temporal inverse filtering unit for generating a decoded image signal, wherein the signal decoding processor decodes the temporal low-band signal, the reduced decoded image signal, the temporal high-band spatial high-band signal, and the motion information from the encoded data. It is done.

According to a seventy-seventh aspect of the present invention, in the sixty-seventh aspect, the control program performs motion compensation on the temporal low-band signal based on the motion information to perform a predictive image based on the motion information. A motion compensator to generate, a low pass signal generator for generating a spatial low pass prediction signal which is a spatial low pass component of the predicted image, and a temporal high pass signal generation for generating a temporal high pass component from the spatial low pass prediction signal and a reduced decoded image signal It functions as a negative portion, and outputs the output of the temporal high frequency signal generating section as a temporal high frequency spatial low frequency signal.

In a seventy-seventh invention for solving the above-mentioned problems, in the seventy-seventh aspect, the control program performs a weighting process of performing a weighting process for compensating a weighting process performed at the time of encoding to the output of the temporal high-frequency signal generation unit. It functions as a weighting unit, and outputs the output of the inverse weighting unit as a temporal high-pass spatial low-pass signal.

According to a seventy-seventh aspect of the present invention, in the sixty-seventh aspect, the control program generates the spatial high-band spatial low-band signal reconstruction unit by generating a spatial low-band component of the temporal low-band signal and outputs it as a temporal low-band spatial low-band signal. A low-frequency signal generating unit, a motion information converting unit converting motion information according to the resolution conversion ratio of the input / output image of the low-pass signal generating unit, and the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting unit A motion compensator for performing motion compensation on the video signal to generate a predictive image, and a time high frequency signal generator for generating a temporal high frequency component from the predicted image and the decoded decoded image signal, and outputting the output of the temporal high frequency signal generator It outputs as a high frequency space low frequency signal, It is characterized by the above-mentioned.

In the eightyth aspect of the present invention, in the seventy-ninth aspect of the present invention, the control program performs a weighting process for compensating a weighting process performed at the time of encoding the moving picture coding apparatus at the output of the temporal high-frequency signal generating unit. It functions as a reverse weight assignment unit to be performed, and outputs the output of the said reverse weight assignment unit as a temporal high-pass spatial low-pass signal.

In the 81st invention which solves the said subject, the low-pass signal generation part is low-pass process by subband division in any one of the said 76th-80th invention.

In the eighty-eighth aspect of the present invention, the spatial synthesis filtering unit is subband synthesis, which is an inverse transform of the subband division.

In the eighty-eighth aspect of the present invention, the low-pass signal generator generates a reduced image by a subsample filter according to any one of the sixty-eighth aspects.

In the eighty-eighth aspect of the present invention, in the eighty-eighth aspect, the spatial synthesis filtering unit generates the high pass signal generated by the second subsample filter paired with the subsample filter and the subsample filter. It is characterized by the synthesis of the low-pass signal.

In the 85th invention which solves the said subject, in the invention in any one of the 76th-84th said control program, the said time direction inverse filtering part performs motion compensation to the said time high-pass signal based on the said motion information, A first motion compensation unit generating a motion compensation time high frequency signal, a time low frequency signal inverse converting unit generating a first decoded image signal from the motion compensation time high frequency signal and the time low frequency signal, and the first decoded image signal And a second motion compensator for generating a prediction signal by performing motion compensation on the basis of the motion information, and a temporal high frequency signal inverse transform part for generating a second decoded image signal from the prediction signal and the temporal high frequency signal. And a decoded image signal which is outputted by integrating the first decoded image signal and the second decoded image signal. And it characterized in that.

In the 86th invention which solves the said subject, in the invention in any one of said 76-85, after the said time direction inverse filtering part performs motion compensation with respect to the said time low-pass signal, it adds to the time high-pass signal, and A time low pass signal is synthesized and output as a decoded image signal.

The 87th invention which solves the said subject is a demultiplexing part which demultiplexes a layered bit stream, and produces | generates the lowest coded data, the redundant coded data, and the motion information coded data, and decodes the minimum coded data, and reduces a reduced image signal. A reduced-image signal decoder for generating a video signal; a space-time high-band signal decoder for decoding an excess encoded data to generate a space-time high-band signal; a motion information decoder for decoding the motion information encoded data to generate motion information; A control program of a video decoding apparatus comprising a time-space frequency synthesizing unit for generating a decoded image signal from an image signal, the space-time high-band signal and the motion information, wherein the control program includes: the time-space frequency synthesizing unit; Spatial low-frequency signal and remind A spatial synthesis filter for synthesizing the temporal low-pass spatial high-frequency signal of the temporal-high-frequency signal, and generating a temporal low-pass signal as the temporal low-band signal, the reduced image signal, and the spatiotemporal high-band signal After reconstruction, the signal is synthesized with the temporal low-pass signal to function as a time-space synthesis filtering unit for outputting a decoded image signal.

An outline of temporal spatial division filtering in video encoding, which is a feature of the present invention, will be described with reference to FIG. 25.

In temporal space division filtering, first, a moving picture signal 10 having an arbitrary resolution layer is temporally layered and divided into a temporal low pass signal 11 and a temporal high pass signal 12.

Next, using the time low pass signal 11 and the time high pass signal 12, the video signal 21 which reconstructed the video signal corresponding to the time high pass signal 12 is produced | generated. A reduced image signal 14 subjected to a reduction process to the moving image signal 21 is generated.

In addition, the high frequency generation process by spatial layering is performed on the temporal high frequency signal 12 to generate the temporal high frequency spatial high frequency signal 13.

Thereafter, the temporal low pass signal 11, the reduced image signal 14, and the temporal high pass spatial high pass signal 13 are output as a division result. The temporal low pass signal 11 is regarded as a video signal having a low temporal resolution once, the reduced image signal 14 is regarded as a video signal having a low spatial resolution once, and the temporal spatial division filtering is performed recursively to perform the video signal in multiple steps. Layered.

Next, the moving picture coding according to the present invention will be described.

The moving picture coding according to the present invention includes signal decoding processing for decoding coded data of a hierarchically coded moving picture signal to generate a layered signal, and time-space synthesis filtering for combining the layered signal.

An outline of the temporal space synthesis filtering will be described with reference to FIG. 26.

The signals to be synthesized in the temporal space synthesis filtering are the decoded image signal 15, the temporal low pass signal 16, and the temporal high pass spatial high pass signal 18. Here, the decoded image signal 15 corresponds to the above-mentioned reduced image signal 14.

First, the temporal high frequency signal corresponding to the temporal low frequency signal 16 using the decoded image signal 15 which is the low frequency component in arbitrary spatial resolution and the time low frequency signal 16 which is the low frequency component in temporal resolution Reconstruct the temporal high frequency spatial low signal 17 which is the spatial low frequency component of 12).

 The temporal high frequency signal 19 is obtained by performing a hierarchical synthesis process in the spatial direction on the temporal high frequency spatial low frequency signal 17 and the temporal high frequency spatial high frequency signal 18. Thereafter, the hierarchical synthesis processing in the time direction is performed on the temporal high frequency signal 19 and the temporal low frequency signal 16 to generate a decoded image signal 20 of a higher resolution layer.

The decoded image signal 20 is regarded as the temporal low pass signal 16 or the decoded image signal 15, and multi-level hierarchical synthesis is realized by recursively performing time-space synthesis filtering.

Next, the second temporal spatial division filtering in the second video encoding of the present invention will be described.

In the above-described temporal space division filtering, the video signal corresponding to the temporal high frequency component is reconstructed by using the temporal low frequency component and the temporal high frequency component, and then the reduced image signal is generated by performing a reduction process on the reconstruction result. . However, it is not limited to this method, and it is possible to generate a reduced picture signal of a moving picture.

Therefore, the outline of the second temporal spatial division filtering in the video encoding which is a feature of the present invention will be described with reference to FIG.

In temporal space division filtering, first, a moving picture signal 10 having an arbitrary resolution layer is temporally layered and divided into a temporal low pass signal 11 and a temporal high pass signal 12. So far, this is the same as the time-space division filtering described above.

Next, the temporal low pass signal 11 is generated by low pass generating processing, thereby generating the temporal low pass spatial low pass signal 22.

On the other hand, the low frequency generation process of the time high frequency signal 12 produces | generates the temporal high frequency space low frequency signal 23, and the high frequency generation process of the time high frequency signal 12 produces the temporal high frequency space high frequency signal 13 do. Then, the temporal hierarchical low frequency signal 22 and the temporal high frequency spatial low frequency signal 23 are temporally hierarchically synthesized to generate a reduced image signal 14.

Thereafter, the temporal low pass signal 11, the reduced image signal 14, and the temporal high pass spatial high pass signal 13 are output as a division result. The temporal low pass signal 11 is regarded as a video signal having a low temporal resolution once, the reduced image signal 14 is regarded as a video signal having a low spatial resolution once, and reconstructed temporally and spatially to perform video space multi-step motion. Layer the signal. The decoding is the same as the decoding method explained in FIG.

Effect of the Invention

According to the present invention, the time-space high-band signal in the case of hierarchical coding is equivalent to the three-dimensional wavelet coding scheme in the prior art, and the result of time-direction filtering in the reduced image signal is encoded instead of the temporal high-band spatial low-band signal. That is, after encoding the space-time high-band signal with high efficiency as in the prior art, the reduced image signal is encoded independently of the space-time high-band signal. Thereby, the mismatch of motion compensation and the overhead of motion information, which were the subjects of the prior art, are eliminated, and the coding efficiency of the reduced image signal is greatly improved.

According to the present invention, it is possible to realize image quality equivalent to that of a decoded image when the decoded signals in all layers are encoded in a single layer in the encoded data layered.

In addition, according to the present invention, it is possible to independently determine a parameter or processing module that is a limitation in an application such as a delay or an amount of calculation in the encoded data at the input resolution and the encoded data at the reduced resolution. That is, hierarchical coding that can be simultaneously delivered to a plurality of terminals having different delivery conditions can be realized without degrading the coding efficiency.

In the decoding process, the weighting process is performed immediately before reconstructing the temporal high-frequency spatial low-pass signal during time-space synthesis filtering, and the weighting coefficient at this time is weighted when generating the reduced image signal in the coding process. In this case, the distortion superimposed upon the encoding of the reduced image signal is not propagated while being increased even after spatial direction synthesis filtering, and as a result, the deterioration of the decoded image can be reduced.

In addition, if the filtering such as noise reduction is performed as a preprocessing of the coding of the reduced image signal and as a preprocessing of the decoding, the encoding distortion in the decoded reduced image signal may affect the decoded image signal having a larger resolution. It can reduce the impact.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing the structure of a moving picture coding apparatus according to a first embodiment of the present invention.

2 is a block diagram showing the configuration of the time-space frequency division unit 101. FIG.

FIG. 3 is a conceptual diagram showing subband division in the temporal direction and the spatial direction in the temporal and spatial frequency division unit 101. FIG.

4 is a block diagram showing a configuration of a time space two division filtering unit 107. FIG.

5 is a block diagram showing a configuration of a time direction filtering unit 109. FIG.

6 is a block diagram showing the configuration of a reduced image generating unit 110;

FIG. 7 is a block diagram showing the configuration of a second reduced image generation unit 110 in Embodiment 2. FIG.

8 is a block diagram showing the configuration of a reduced image signal encoder 102.

9 is a block diagram showing the configuration of the space-time high-frequency signal encoder 103.

Fig. 10 is a block diagram showing the construction of a moving picture decoding apparatus according to a first embodiment of the present invention.

11 is a block diagram showing the configuration of a reduced image signal decoding unit 151. FIG.

12 is a block diagram showing the configuration of the space-time high-band signal decoding unit 152.

Fig. 13 is a block diagram showing the configuration of the time-space frequency synthesizing unit 154.

14 is a block diagram showing the configuration of the time-space synthesis filtering unit 168. FIG.

FIG. 15 is a diagram illustrating a configuration of a temporal high-frequency spatial low-frequency signal reconstruction unit 170 corresponding to the reduced image generation unit shown in FIG. 6.

16 is a configuration diagram of a temporal high-frequency spatial low-frequency signal reconstruction unit 170 corresponding to the second reduced image generation unit 110 shown in FIG. 7.

17 is a block diagram showing a configuration of time direction inverse filtering.

18 is a diagram for explaining a prior art.

19 is a diagram for explaining a prior art.

20 is a diagram for explaining a conventional technology.

21 is a diagram for explaining a prior art.

FIG. 22 is a conceptual diagram showing a one-dimensional pixel array and filter coefficients multiplied by each pixel in order to explain that the interpolation processing and subsample processing of motion compensation are not acceptable. FIG.

FIG. 23 is a conceptual diagram showing a one-dimensional pixel array and filter coefficients multiplied by each pixel to explain that the interpolation processing and subsample processing of motion compensation are not acceptable. FIG.

24 is a conceptual diagram illustrating the overhead of motion information.

Fig. 25 is a view for explaining an outline of temporal space division filtering in moving picture coding, which is a feature of the present invention;

FIG. 26 is a diagram for explaining an outline of time-space synthesis filtering, which is a feature of the present invention; FIG.

FIG. 27 is a conceptual diagram illustrating how quantization distortion superimposed on each signal affects a decoded image. FIG.

Fig. 28 is a flowchart showing processing of time space dividing filtering.

Fig. 29 is a block diagram showing the structure of a reduced image generation unit in accordance with the third embodiment;

30 is a block diagram showing a configuration of a temporal high-band spatial low-band signal reconstruction unit in the video decoding device in Example 3. FIG.

FIG. 31 is a block diagram showing a configuration of a second reduced image generation unit in which a weighting unit is added to the second reduced image generation unit described in Embodiment 2. FIG.

Fig. 32 is a block diagram showing the construction of a second temporal high-frequency spatial low-frequency signal reconstruction unit corresponding to the video decoding device of the second embodiment.

Fig. 33 is a block diagram showing the structure of a reduced image generation unit in the fourth embodiment.

34 is a block diagram showing the construction of a temporal high-frequency spatial low-frequency signal reconstruction section in accordance with the fourth embodiment;

Fig. 35 is a block diagram showing the construction of a second reduced image generation unit in the fourth embodiment.

36 is a block diagram showing the configuration of a second temporal high-frequency spatial low-frequency signal reconstruction section in accordance with the fourth embodiment;

37 is a flowchart showing a process of time space synthesis filtering.

38 is a block diagram showing a video encoding apparatus according to a fifth embodiment;

Fig. 39 is a block diagram showing the construction of a second reduced image generation unit in the fifth embodiment.

40 is a block diagram showing a configuration of a temporal high-frequency spatial low-frequency signal reconstruction section in accordance with the fifth embodiment;

Fig. 41 is a block diagram showing the construction of a second temporal high-frequency spatial low-frequency signal reconstruction section according to the fifth embodiment.

Fig. 42 is a general block diagram of an information processing system implemented with a video encoding device according to the present invention.

Fig. 43 is a view for explaining an outline of temporal spatial division filtering in video encoding, which is another feature of the present invention.

<Description of the code>

101: time-space frequency divider

102: reduced image signal encoder

103: space-time high-frequency signal encoder

104: motion information encoder

105: multiplexer

150: demultiplexer

151: Reduced image signal decoder

152: space-time high-frequency signal decoder

153: motion information decoding unit

154: time-space frequency synthesis unit

<Best Mode for Carrying Out the Invention>

The structure of the time-space two-part filtering unit for realizing the time-space two-part filtering in the moving picture encoding, which is a feature of the present invention, will be described.

As shown in FIG. 4, the time-space two-segment filtering unit includes a time direction filtering unit 109, a reduced image generating unit 110, and a high frequency signal generating unit 111. Here, the input image signal 1000, the reduced image signal 1001, and the time low-frequency signal 1010 which are input to the time-space two-division filtering unit 107 are collectively referred to as a division target signal 1013. In addition, the input image signal 1000 corresponds to the moving image signal 10 in FIG. 25, and the reduced image signal 1001 corresponds to the reduced image signal 14 in FIG. 25, and the time low-frequency signal 1010. ) Corresponds to the time low-pass signal 11 in FIG. The temporal high frequency spatial high frequency signal 1002 corresponds to the temporal high frequency spatial high frequency signal 13 in FIG. 25.

The processing of the time-space two-division filtering in such a configuration will be described using the flowchart of FIG. 28.

The division target signal 1013 is divided into a time low pass signal 1010 and a time high pass signal 1014 by temporal hierarchization by the time direction filtering unit 109, and simultaneously receives motion information 1003 used for motion compensation. Output it (Step 10O). In addition, the reduced image generation unit 110 generates the reduced image signal 1001 based on the time low pass signal 1010, the time high pass signal 1014, and the motion information 1003 (Step 101).

On the other hand, the high frequency signal generation part 111 performs the high frequency signal generation process based on the temporal high frequency signal 1014, and produces | generates the temporal high frequency spatial high frequency signal 1002 (step 102).

Then, the reduced image signal 1001, the temporal low pass signal 1010, and the temporal high pass spatial high pass signal 1002 are output as the division result (Step 103).

Next, a configuration of a time-space synthesis filtering section for realizing time-space synthesis filtering in moving picture decoding, which is a feature of the present invention, will be described.

As shown in FIG. 14, the temporal spatial synthesis filtering unit includes a temporal high frequency spatial low pass signal reconstruction unit 170, a spatial synthesis filtering unit 171, and a temporal direction inverse filtering unit 172. Here, the reduced image signal 1073 of FIG. 14 corresponds to the reduced image signal 15 in FIG. 26, and the time low pass signal 1072 corresponds to the time low pass signal 16 in FIG. The high frequency signal 1074 corresponds to the temporal high frequency spatial high frequency signal 18 in FIG. 26, and the temporal high frequency spatial low frequency signal 1076 corresponds to the temporal high frequency spatial low frequency signal 17 in FIG. The high frequency signal 1077 corresponds to the time high frequency signal 19 in FIG. 26, and the decoded image signal 1075 corresponds to the decoded image signal 20 in FIG. 26.

The processing of the temporal space synthesis filtering in this configuration will be described using the flowchart of FIG. 37.

The reduced image signal 1073, the temporal low pass signal 1072, and the spatiotemporal high pass signal 1074 are synthesized (Step 20O), and the temporal high pass spatial low pass signal reconstructing unit 170 performs the time low pass signal 1072 with the temporal low pass signal 1072. Based on the reduced image signal 1073 and the motion information 1056, the temporal high frequency spatial low frequency signal 1076 is reconstructed (Step 201).

Subsequently, the spatial synthesis filtering unit 171 synthesizes the temporal high-frequency spatial low-frequency signal 1076 and the space-time high-frequency signal 1074 to generate the temporal high-frequency signal 1077 (Step 202).

Then, the time direction inverse filtering unit 172 controls the time high frequency signal 1077 and the time low frequency signal 1072 based on the motion information 1056 of the time direction filtering unit 109 in FIG. 4. The inverse transform is performed to generate a decoded image signal 1075 (Step 203).

In addition, the temporal high-band spatial low-band signal reconstruction unit 170 is realized in a form corresponding to the reduced image generation unit 110 in FIG. 4.

Next, the structure of the 2nd time-space dividing filtering part in moving picture coding which is the characteristic of this invention is demonstrated.

In the above-described embodiment, the reduced image generating unit 110 reconstructs a moving image signal corresponding to the temporal high frequency component by using the temporal low frequency component and the temporal high frequency component, and then performs a reduction process on the reconstruction result. A reduced picture signal of a moving picture is generated. However, it is not limited to this method, and it is possible to generate a reduced picture signal of a moving picture. Therefore, the structure of the 2nd reduced image generation part 110 different from embodiment mentioned above is demonstrated using FIG. In addition, since the other structure is the same as that of embodiment mentioned above, detailed description is abbreviate | omitted.

The low pass signal generator 123 and the low pass signal generator 124 include a temporal low pass spatial low pass signal 1023 and a temporal high pass spatial low pass signal, which are the low pass signals of the temporal low pass signal 1010 and the temporal high pass signal 1014. 1024).

The motion information converter 125 generates the motion information 1025 obtained by reducing the motion information 1003 in accordance with the ratio of the resolution due to the low pass signal generation. The motion compensator 126 performs a motion compensation process on the temporal low-band spatial low-band signal 1023 based on the motion information 1025 to generate the predictive signal 1026. The temporal high frequency signal inverse transform unit 127 performs inverse transform of the high frequency signal generation filter processing in the temporal high frequency signal generator 114 on the temporal high frequency spatial low frequency signal 1024 and the predictive signal 1026, thereby reducing the image. Generate signal 1001.

Here, the temporal low pass signal 1010 corresponds to the temporal low pass signal 11 in FIG. 43, and the temporal high pass signal 1014 corresponds to the temporal high pass signal 12 in FIG. 43. The signal 1023 corresponds to the temporal low pass spatial low pass signal 22 in FIG. 43, and the temporal high pass spatial low pass signal 1024 corresponds to the temporal high pass spatial low pass signal 23 in FIG. 43. The prediction signal 1026 is a signal used in temporal hierarchical synthesis in FIG. 43.

Next, the temporal high-frequency spatial low-frequency signal reconstructing unit 170 of the encoding apparatus corresponding to the reduced image generating unit 110 described above will be described. In addition, only portions different from the above-described temporal high-band spatial low-frequency signal reconstruction unit 170 will be described.

In FIG. 16, the low pass signal generation unit 176 generates a temporal low pass spatial low pass signal 1082 that is a spatial low pass component of the temporal low pass signal 1072. The motion information converter 177 generates the motion information 1083 obtained by reducing the motion information 1056 in accordance with the ratio of the resolution due to the low pass signal generation. The motion compensation unit 178 performs a motion compensation process on the temporal low-band spatial low-band signal 1082 based on the motion information 1083, and generates a predictive signal 1084. The temporal high frequency signal generator 179 performs the same high frequency signal generation filter processing as that of the temporal high frequency signal generator 114 in FIG. 5 with respect to the reduced image signal 1073 and the predictive signal 1084. A spatial low pass signal 1076 is output.

Hereinafter, specific embodiments of a video encoding apparatus and a video decoding apparatus using time-space two-division filtering and time-space synthesis filtering will be described.

Example 1

1 to 9 will be described using time-space division filtering, the procedure of recursive execution thereof, and the method of implementing the signal encoding process in the video encoding apparatus according to the first embodiment of the present invention.

Fig. 1 is a block diagram showing the structure of a moving picture coding apparatus according to the first embodiment of the present invention. In FIG. 1, a video encoding apparatus includes a time-space frequency division unit 101, a reduced image signal encoding unit 102, a space-time high-frequency signal encoding unit 103, a motion information encoding unit 104, and a multiplexing unit 105. Is done. The flow of processing of the coding apparatus according to the embodiment will be described with reference to FIG. 1.

First, the time-space frequency dividing unit 101 receives the input image signal 1000, reduces the frame of each frame constituting the input image signal 1000, and the input image signal 1000. From the high-frequency signal from which the correlation of the reduced image signal 1001 is removed, the space-time high-frequency signal 1002 from which the correlation in the time direction is removed and the motion information 1003 are generated.

The reduced image signal 1001 is recursively inputted to the time-space frequency division unit 101 or outputted to the reduced image signal encoding unit 102. The number of recursive operations of the time-space frequency dividing unit 101 is determined from the number of layers of spatial scalability based on the present technology. For example, if the number of layers of spatial scalability is 3, the number of recursive operations to the time-space frequency division unit 101 is two. Each time, the reduced image signal 1001, the temporal high frequency spatial high frequency signal 1002, and the motion information 1003 are generated.

2 is a block diagram showing the configuration of the time-space frequency divider 101. The flow of processing by the time-space frequency division unit 101 will be described with reference to FIG. 2.

The reduced image signal 1001 that is the output of the input image signal 1000 or the time-space frequency division unit 101 is collectively referred to as a process target signal 1009. From the process target signal 1009 consisting of consecutive N-frames (N is a power of 2), the time-space two-segment filtering unit 107 is a space-time high-frequency signal 1002 corresponding to N / 2 frames and reduced. An image signal 1001 and a time low-frequency signal 1010 corresponding to N / 2 frames are generated. Of these, the time low-pass signal 1010 is regarded as an input signal to be processed again, and time-space two-part filtering is performed. This repetitive processing is performed until the number of frames constituting the time low-frequency signal 1010 becomes one. Thereafter, the temporal low pass signal 1010 is frequency-divided in the spatial direction by the spatial division filtering unit 108 to generate the temporal low pass spatial low pass signal 1011 and the temporal low pass spatial high pass signal 1012. The spatial low pass signal 1011 is output as a reduced image signal 1001, and the spatial high pass signal 1012 is a space-time high pass signal 1002.

In the low pass generation process performed by the spatial division filtering unit 108, any filter for reducing the resolution is used, in addition to the subband division filter represented by the wavelet transform. In the former case, a subband dividing filter corresponding to the low pass generation process is used for the high pass generation process. In the latter case, the difference obtained by subtracting the downsampled signal from the input signal is used.

3 is a conceptual diagram illustrating subband division in the temporal direction and the spatial direction in the temporal and spatial frequency division unit 101.

The input image signal 2011 is subjected to one time-space two-segment filtering by the time-space two-division filtering unit 107, and includes a time low-pass signal 2041, a space-time high-pass signal 2042 and a reduced image signal 2043. Divided. The temporal low-pass signal 2041 is filtered by the temporal-space 2-division filtering unit 107 for one time-space 2-division filtering, so as to the temporal low-pass signal 2044, the spatio-temporal high-frequency signal 2045, and the reduced image signal 2046. Divided. The temporal low pass signal 2044 is divided into a temporal low pass signal 2047, a spatiotemporal high pass signal 2048, and a reduced image signal 2049 by one time space bipartition filtering 107. The temporal low pass signal 2047 is divided into a spatial low pass signal 2050 and a spatial high pass signal 2051 by spatial division filtering 108. The space-time high-band signals 2042, 2045, and 2048 and the space high-band signals 2051 are space-time high-band signals 1002, which are outputs of the time-space frequency divider 101, and reduced image signals 2043, 2046, and 2049 and spatial low-bands. The signal 2050 is a reduced image signal 1001 that is an output of the time-space frequency division unit 101.

4 is a block diagram showing the configuration of the time space two division filtering unit 107. The flow of processing by the time space two division filtering unit 107 will be described with reference to FIG. 4.

The input image signal 1000, the reduced image signal 1001, and the time low-frequency signal 1010 that are input to the time-space two-division filtering unit 107 are collectively referred to as a division target signal 1013. The time direction filtering unit 109 divides the division target signal 1013 into a time low pass signal 1010 and a time high pass signal 1014, and outputs motion information 1003 used for motion compensation. The reduced image generating unit 110 generates a reduced image signal 1001 from the time low pass signal 1010, the time high pass signal 1014, and the motion information 1003. The high frequency signal generator 111 generates a temporal high frequency spatial high frequency signal 1002 using the temporal high frequency signal 1014 as an input. The high frequency division in the high frequency signal generation unit 111 is equivalent to the high frequency generation processing in the spatial division filtering shown in FIG. 4 is an example of a circuit configuration in which the time space division filtering, which is a feature of the present invention shown in FIG. 25, is realized. The divided video signal 10, the temporal low-frequency signal 11, the temporal high-frequency signal 12, the temporal high-frequency spatial high-frequency signal 13, and the reduced image signal 14 in Fig. 25 are each divided target signals (Fig. 4). 1013, the temporal low pass signal 1010, the temporal high pass signal 1014, the temporal high pass spatial high pass signal 1014, and the reduced image signal 1001.

5 is a block diagram showing the configuration of the time direction filtering unit 109. The flow of the process of time direction filtering is demonstrated using FIG.

The splitting target signal 1013 is divided into a splitting target signal 1015 that is converted into a temporal low pass signal and a splitting target signal 1016 that is converted into a temporal high pass signal in time direction filtering. The motion estimation unit 112 generates motion information 1003 that defines motion compensation of the split target signal 1015 and the split target signal 1016.

The motion compensator 113 performs motion compensation prediction using the split target signal 1015 as a reference signal, and generates a prediction signal 1017 for the split target signal 1016. The temporal high pass signal generator 114 performs a high pass signal generation filter process for each pixel on the division target signal 1016 and the predictive signal 1017 to generate the temporal high pass signal 1014.

The motion compensator 115 performs a motion compensation process on the temporal high frequency signal 1014 based on the motion information 1003, thereby performing a motion compensating temporal high frequency signal 1018 corresponding to each pixel on the target signal 1015. Create

The temporal low pass signal generation unit 116 performs the low pass signal generation filter processing for each pixel on the division target signal 1015 and the motion compensation time high pass signal 1018 to generate the temporal low pass signal 1010. As the high pass signal generation filter process and the low pass signal generation filter process, a wavelet transform or a wavelet transform of 5-3 taps is used. Alternatively, the conventional inter-frame predictive encoding process for outputting the split target signal with no transform is used as the difference generation and the low pass signal generation filter processing.

6 is a block diagram showing the configuration of the reduced image generating unit 110. The flow of processing by the reduced image generating unit 110 will be described with reference to FIG. 6.

The motion compensator 120 performs motion compensation processing equivalent to that of the motion compensator 113 in FIG. 5 with respect to the time low-frequency signal 1010 to generate the predictive signal 1021.

The temporal high frequency signal inverse transform unit 121 generates a moving image signal 1022 from the temporal high frequency signal 1014 and the predictive signal 1021. The processing in the temporal high frequency signal inverse converting section 121 is an inverse transform of the high frequency signal generating filter processing in the temporal high frequency signal generating section 114 in FIG. 5. When the division target signal 1015 and the time low-frequency signal 1010 in FIG. 5 are not the same, the moving image signal 1022 is not the same as the division target signal 1016 in FIG. The low pass signal generation unit 122 down-samples the moving image signal 1022 to generate a reduced image signal 1001.

In the above, the description of the time-space frequency division 101 in FIG. 2 is finished, and the flow of the encoding process according to the present invention will be described again using FIG. 1.

The motion information 1003 generated by the time-space frequency division 101 is encoded by the motion information encoder 104. As described with reference to FIGS. 4 and 5, the motion information 1003 defines motion compensation for an image signal having a different resolution, that is, an input image signal 1000 or a reduced image signal 1001.

When encoding motion information having different resolutions, the motion information encoder 104 uses the correlation between motion information of blocks adjacent to each other in the spatial direction or uses the correlation between motion information at different resolutions. Reduce redundancy

The reduced image signal 1001 and the space-time high-frequency signal 1002 generated by the time-space frequency division 101 are encoded by the texture signal coding units 102 and 103, respectively.

The motion information encoded data 1006 generated by the motion information encoder 104 and the texture signal encoded data 1004 and 1005 generated by the texture signal encoders 102 and 103 are multiplexed by the multiplexer 105. It is output as coded data 1007.

8 is a block diagram showing the configuration of the reduced image signal encoder 102. The flow of processing of the reduced image signal coding unit will be described with reference to FIG. 8. The reduced image signal 1001 is converted into a texture signal 1030 with redundancy in the time direction by the time direction filtering unit 130. As temporal filtering, frequency transform in the temporal direction with motion compensation as shown in Fig. 5, or motion compensation inter-frame predictive coding is used.

The texture signal 1030 is converted by the frequency converter 131 to generate a frequency transform coefficient 1031. As a process in the frequency converter 131, the conversion process by the block unit represented by discrete cosine transform or the subband division process represented by wavelet transform is mentioned. The frequency transform coefficient 1031 is quantized by the quantization unit 132 to generate a quantization transform coefficient 1032. As the processing in the quantization unit 132, in addition to quantization based on a single quantization step, hierarchical quantization that quantizes an error with a frequency transform coefficient which is input after being quantized in an arbitrary quantization step in a finer quantization step, or And bit plane encoding in which each frequency transform coefficient is binarized and sequentially outputted from a high digit value to a low digit value. The quantization transform coefficients 1032 are entropy coded by the entropy coding unit 133 to generate reduced image signal coded data 1004. As the entropy coding, arithmetic coding is used in addition to variable length coding (VLC) to be coded according to a predetermined half-man table. In addition, the time direction filtering 130 may be omitted. The frequency converter 131 and the quantizer 132 may be omitted.

9 is a block diagram showing the configuration of the space-time high-frequency signal encoder 103. The space-time high-frequency signal 1002 is converted by the frequency converter 134 to generate a frequency conversion factor 1033. The frequency transform coefficient 1033 is quantized by the quantization unit 135 to generate a quantization transform coefficient 1034. The quantization transform coefficients 1034 are entropy coded by the entropy coding unit 136 to generate reduced image signal coded data 1005. Note that the frequency converter 134 and the quantizer 135 may be omitted.

In addition, the frequency converter 134, the quantizer 135, and the entropy encoder 136 in FIG. 9 include the frequency converter 131, the quantizer 132, and the entropy encoder in FIG. You may use what is different from 133). In other words, both the reduced picture coding unit 102 and the space-time high-frequency signal coding unit 103 may use a non-scalable coding method in which discrete cosine transform is used for frequency transformation and quantization based on a single quantization step in the quantization unit. none. Alternatively, the reduced image signal encoding unit 102 performs discrete cosine transform for frequency transformation and quantization based on a single quantization step in the quantization unit. In the space-time high-frequency signal encoding unit 102, wavelet transform and quantization are used for frequency transformation. You may use the scalable coding method which performs bit plane coding to a part.

Next, as described above, the encoded video decoding apparatus will be described.

10 to 17, a time-space synthesis filtering, a recursive procedure, and a method of realizing a signal decoding processing in the moving picture decoding apparatus according to the first embodiment of the present invention will be described.

Fig. 10 is a block diagram showing the structure of a moving picture decoding apparatus according to the first embodiment of the present invention. In FIG. 10, the moving picture decoding apparatus includes a demultiplexer 150, a reduced image signal decoder 151, a space-time high-band signal decoder 152, a motion information decoder 153, and a time-space frequency synthesizer 154. Is made of. The flow of processing by the decoding device of Example 1 will be described with reference to FIG. 10.

First, the demultiplexer 150 divides the encoded data 1050 into low frequency signal encoded data 1051, high frequency signal encoded data 1052, and motion information encoded data 1053.

The reduced image signal decoder 151 and the space-time high-band signal decoder 152 decode the low-band signal coded data 1051 and the high-band signal coded data 1052, respectively, to reduce the reduced image signal 1054 and the space-time high-band signal ( 1055).

The motion information decoding unit 153 decodes the motion information encoded data 1053 to obtain motion information 1056.

The time-space frequency synthesizing unit 154 performs inverse filtering and space-frequency frequency synthesis with motion compensation defined by the motion information 1056 on the reduced image signal 1054 and the space-time high-frequency signal 1055. Are combined to generate a decoded image signal 1057. Alternatively, the decoded image signal 1057 is regarded as a reduced image signal, and the corresponding space-time high-frequency signal 1055 and time-space frequency synthesis 154 are recursively generated to generate a higher resolution decoded image signal 1057. do.

11 is a block diagram showing the configuration of the reduced image signal decoding unit 151.

The reduced image signal encoded data 1051 is decoded into quantization transform coefficients 1060 by the entropy decoding unit 160.

The quantization transform coefficient 1060 is inversely quantized by the inverse quantization unit 161, and the frequency inverse transform unit 162 performs frequency inverse transform on the output frequency transform coefficient 1061 to generate the texture signal 1063.

The time direction inverse filtering unit 163 performs inverse transformation of the time direction filtering unit 130 illustrated in FIG. 8 to generate a reduced image signal 1054.

The entropy decoding unit 160, the inverse quantization unit 161, the frequency inverse transform unit 162, and the time direction inverse filtering unit 163 include the entropy coding unit 133, the quantization unit 132, and the frequency in FIG. 8. The converter 131 and the time-direction filtering unit 130 correspond to each other.

When the reduced picture encoder 102 omits any one of the quantizer 132, the frequency converter 131, and the time-direction filter 130, the inverse quantizer 161 and the frequency inverse converter of FIG. 11 are omitted. 162 and the time direction reverse filtering unit 163 are similarly omitted.

12 is a block diagram showing the configuration of the space-time high-band signal decoding unit 152.

The space-time high-frequency signal encoded data 1052 is decoded into quantization transform coefficients 1063 by the entropy decoding unit 164. The quantization transform coefficient 1063 is inversely quantized by the inverse quantization unit 165, and the frequency inverse transform unit 166 performs frequency inverse transform on the output frequency transform coefficient 1064 to generate the space-time high-frequency signal 1055. The entropy decoding unit 164, the inverse quantization unit 165, and the frequency inverse transform unit 166 correspond to the entropy encoding unit 136, the quantization unit 135, and the frequency conversion unit 134 in FIG. 9, respectively. have.

When the reduced image encoder 102 omits either the quantizer 135 or the frequency converter 134, the inverse quantizer 165 and the frequency inverse converter 166 of FIG. 12 are similarly omitted.

13 is a block diagram showing the configuration of the time-space frequency synthesizing unit 154. The flow of the process of time-space frequency synthesis will be described with reference to FIG.

The spatial synthesis filtering unit 167 performs a temporal low-pass space on each of the N-frame reduced image signals 1154 and the space-time high-frequency signal 1055 for one frame, which becomes the lowest frequency region in the time direction, respectively. The low pass signal 1070 and the temporal low pass spatial high pass signal 1071 are regarded as spatial synthesis filtering.

The reduced time signal 1073 and the space time high frequency signal 1174 corresponding to the temporal low frequency signal 1072 and the temporal high frequency signal paired with the temporal low frequency signal 1072 are moved. It is synthesized by the time-space synthesis filtering unit 168 with motion compensation based on the information 1056 to obtain a decoded image signal 1075 for two frames. The decoded image signal is regarded as a temporal low pass signal 1072, and the temporal space synthesis filtering is recursively performed on the paired reduced image signal 1073 and the space-time high pass signal 1074.

The above processing is repeated until successive N decoded image signals 1057 are obtained. The spatial synthesis filtering of the spatial synthesis filtering unit 167 corresponds to the inverse transformation of the spatial division filtering of the spatial division filtering unit 108 in FIG. 2.

14 is a block diagram showing the configuration of the time-space synthesis filtering unit 168. As shown in FIG. The flow of processing of time-space synthesis filtering will be described with reference to FIG.

The temporal high pass spatial low pass signal reconstruction unit 170 reconstructs the temporal high pass spatial low pass signal 1076 using the temporal low pass signal 1072, the reduced image signal 1073, and the motion information 1056.

The spatial synthesis filtering unit 171 synthesizes the temporal high frequency spatial low frequency signal 1076 and the space time high frequency signal 1074 to generate the temporal high frequency signal 1077.

The time direction inverse filtering unit 172 performs inverse transformation of the time direction filtering unit 109 in FIG. 4 on the time high frequency signal 1077 and the time low frequency signal 1072 based on the motion information 1056. The decoded image signal 1075 is generated.

In addition, the time-space synthesis filtering unit 168 shown in FIG. 14 is an example of the circuit structure which implements the time-space synthesis filtering which is the characteristic of this invention shown in FIG. The decoded image signal 15 in FIG. 26, the temporal low pass signal 16, the temporal high pass spatial low pass signal 17, the temporal high pass spatial high pass signal 18, the temporal high pass signal 19, and the decoded image signal 20 Respectively, the reduced image signal 1073, the temporal low pass signal 1072, the temporal high pass spatial low pass signal 1076, the spatiotemporal high pass signal 1074, the temporal high pass signal 1077, and the decoded image signal 1075, respectively. Corresponds.

The temporal high-band spatial low-band signal reconstruction unit 170 is realized in a form corresponding to the reduced image generation unit 110 in FIG. 4.

FIG. 15 shows the configuration of the temporal high-frequency spatial low-frequency signal reconstruction unit 170 corresponding to the reduced image generation unit shown in FIG. 6. Referring to Fig. 15, the flow of processing by the temporal wide area spatial low frequency signal reconstruction unit 170 will be described.

The motion compensation unit 173 performs the motion compensation process on the time low-pass signal 1072 based on the motion information 1056, and generates a prediction signal 1080. The low pass signal generator 174 generates a spatial low pass prediction signal 1081 which is a spatial low pass component of the prediction signal 1080. The temporal high-band signal generation unit 175 converts the reduced image signal 1073 and the spatial low-band prediction signal 1081 from the temporal high-band signal inverse transform unit 121 in FIG. Generate a high-space spatial low-pass signal 1076.

17 is a block diagram showing the configuration of the time direction inverse filtering unit 172. A flow of processing by the time direction inverse filtering unit 172 will be described with reference to FIG. 17.

The motion compensation unit 181 performs a motion compensation process on the temporal high frequency signal 1077 based on the motion information 1056 to generate the motion compensation time high frequency signal 1081.

The temporal low pass signal inverse transform unit 182 performs inverse transformation of the low pass signal generation filter processing for each pixel on the temporal low pass signal 1072 and the motion compensation temporal high pass signal 1181 to generate a decoded image signal 1082.

The motion compensator 183 performs motion compensation prediction using the decoded image signal 1082 as a reference signal, and generates a prediction signal 1083.

The temporal high frequency signal inverse transform unit 184 performs inverse transform of the high frequency signal generation filter processing for each pixel on the temporal high frequency signal 1077 and the predictive signal 1183 to generate a decoded image signal 1084. Arranged decoded image signals 1082 and 1084 in display time order are decoded image signals 1175 to be output.

This concludes the description of the video encoding apparatus and the video decoding apparatus according to the first embodiment of the present invention.

The video decoding apparatus shown in the embodiment decodes the coded data generated by the video encoding apparatus to reconstruct the input image signal. As shown in Fig. 20, after the encoded data extraction apparatus 208 removes the encoded data of the spatiotemporal high-frequency signal of the encoded data, the moving picture decoding apparatus 209 can decode the remaining encoded data. In this case, an image signal having a spatial resolution and a frame rate based on the reduced image signal encoded data and the space-time high-frequency signal encoded data included in the remaining encoded data is decoded. Alternatively, when the remaining coded data does not contain any spatiotemporal high frequency signal coded data, the moving picture decoding apparatus outputs the result of decoding the reduced image coded data.

In addition, in the hierarchical coding having scalability in the spatial direction, the present invention performs spatial division of frequency in the spatial direction after performing temporal filtering on the spatial high-pass components, and generates temporal information on the reduced image for the spatial low-pass components. Direction filtering. Since the reduced image generation is performed in consideration of the temporal filtering in the original resolution, no fundamental distortion like the MC mismatch in the prior art occurs. As described with reference to Figs. 8 and 9, the present invention can be applied to the case where other frequency transform or entropy coding is used in encoding a reduced image signal and a space-time high-band signal.

In addition, in the present invention, as shown in Figs. 2 and 3, the N picture signals are divided into N / 2 time lowpass signals and N / 2 time highpass signals by one time-direction filtering. However, the temporal filtering is performed recursively for the temporal low-pass signal, but it is also applicable to temporal filtering based on other reference relationships. For example, the case where N picture signals are divided into N / 3 time low-frequency signals and 2N / 3 time high-frequency signals by one time-direction filtering.

Example 2

In the first embodiment described above, the reduced image generating unit 110 reconstructs a moving image signal corresponding to the temporal high frequency component by using the temporal low frequency component and the temporal high frequency component, and then performs a reduction process on the reconstruction result. A reduced picture signal of a moving picture is generated. However, it is not limited to this method, and it is possible to generate a reduced picture signal of a moving picture. Therefore, in Example 2, the structure of the 2nd reduced image generation part 110 different from Example 1 is demonstrated. In addition, since the other structure is the same as that of Example 1, detailed description is abbreviate | omitted.

FIG. 7 is a block diagram showing the configuration of the second reduced image generation unit 110 in the second embodiment. The flow of processing by the reduced image generating unit 110 shown in FIG. 7 will be described.

The low pass signal generator 123 and the low pass signal generator 124 include a temporal low pass spatial low pass signal 1023 and a temporal high pass spatial low pass signal, which are the low pass signals of the temporal low pass signal 1010 and the temporal high pass signal 1014. 1024).

The motion information converter 125 generates the motion information 1025 obtained by reducing the motion information 1003 in accordance with the ratio of the resolution due to the low pass signal generation.

The motion compensator 126 performs a motion compensation process on the temporal low-band spatial low-band signal 1023 based on the motion information 1025 to generate the predictive signal 1026.

The temporal high frequency signal inverse transform unit 127 inversely transforms the high frequency signal generation filter process in the temporal high frequency signal generator 114 in FIG. 5 with respect to the temporal high frequency spatial low frequency signal 1024 and the prediction signal 1026. Is performed to generate a reduced image signal 1001.

Next, the temporal high-frequency spatial low-frequency signal reconstructing unit 170 of the encoding apparatus corresponding to the reduced image generating unit 110 described above will be described.

FIG. 16 shows a configuration of the temporal high-frequency spatial low-frequency signal reconstruction unit 170 corresponding to the second reduced image generation unit 110 shown in FIG. 7.

In FIG. 16, the low pass signal generation unit 176 generates a temporal low pass spatial low pass signal 1082 that is a spatial low pass component of the temporal low pass signal 1072.

The motion information converter 177 generates the motion information 1083 obtained by reducing the motion information 1056 in accordance with the ratio of the resolution due to the low pass signal generation.

The motion compensation unit 178 performs a motion compensation process on the temporal low-band spatial low-band signal 1082 based on the motion information 1083, and generates a predictive signal 1084.

The temporal high frequency signal generator 179 performs the same high frequency signal generation filter processing as that of the temporal high frequency signal generator 114 in FIG. 5 with respect to the reduced image signal 1073 and the predictive signal 1084. A spatial low pass signal 1076 is output.

Example 3

Embodiment 3 of the present invention will be described.

In Embodiments 1 and 2 described above, depending on the image, coding distortion based on weighting at the time of generating the reduced image signal may be increased. It demonstrates by giving a specific example below.

A characteristic of the present invention is a temporal high-band spatial low-band signal with a reduced image signal 15 on a reduced resolution and a predictive image signal obtained by motion compensation and reduction processing from the temporal low-band component 16 as shown in FIG. The point is to reconstruct (17).

The reduced image signal 15 is a reconstruction of the reduced image signal 14 subjected to the reduction processing to the moving image signal 21 shown in FIG. 25, and the time low pass component 16 is the time low pass component (FIG. 25). 11) is reconstructed. At the time of decoding, distortion by quantization may occur in these signals.

27 is a conceptual diagram illustrating how quantization distortion superimposed on each signal affects a decoded image.

Distortion amounts 3004, 3005, 3006 based on the quantization step Δ are superimposed on the reduced image signal 3000, the temporal low pass signal 3001, and the space-time high pass signal 3002, respectively, and are reduced in the moving picture decoding apparatus 300. It is assumed that the decoded image 3007, the temporal low pass signal 3008, and the space-time high pass signal 3009 are reconstructed. In the generation of the reduced image signal 3000, the weighting of 1 / α is assumed to be performed immediately after the low pass signal generation processing for normalization processing and the like. In FIG. 27, after performing motion compensation and low-band signal generation on the temporal low-band signal 3008, a weighting process 301 of 1 / α is performed to obtain a predicted image signal 3010 on a reduced resolution. Lose. In order to reconstruct the temporal high-band spatial low-band signal 3011, after performing inverse transform of temporal filtering between the reduced decoded image signal 3007 and the predicted image signal 3010, the weighting process 301 weights α. You need to do By this weighting, the distortion included in the temporal high-frequency spatial low-frequency signal 3011 becomes an amount obtained by multiplying the quantization step Δ by α.

Therefore, in the third embodiment, a moving picture coding apparatus configured to prevent the coded distortion as described above will be described with reference to the drawings.

First, an embodiment of the configuration corresponding to the video encoding apparatus and the video decoding apparatus according to the first embodiment will be described.

The moving picture coding apparatus of the third embodiment has the same structure as that of the first embodiment except that the reduced picture generating unit shown in FIG. 4 is different. Fig. 29 is a block diagram showing the structure of a reduced image generation unit in the third embodiment. The reduced image generation unit shown in FIG. 29 is compared with the reduced image generation unit shown in FIG. 6, and after the time high frequency signal 1014 is weighted by the weighting unit 159, the output 1078 is inversely transformed in time high frequency signal. The point input to the unit 121 is different.

The weighting unit 159 multiplies the signal to be input by the reciprocal of the weighting factor included in the low pass signal generation process. In addition, the weighting unit 159 adjusts the weighting for each pixel or for each block serving as a unit of motion compensation according to the temporal high-frequency signal 1014. When the same weight is applied to all the pixels, deterioration of the reduced image signal 1001 outputted to the pixel having a large power of the temporal high frequency signal 1014 becomes remarkable. For this reason, weighting is reduced only for pixels with a large power of the temporal high-frequency signal 1114. The determination information for weighting reduction needs to match on the encoding side and the decoding side. As a method therefor, it may be based on a predetermined threshold value, encoding the threshold value as additional information, encoding the determination information for each pixel or for each block, and the like.

Subsequently, a video decoding apparatus for decoding a video coded by the above-described video encoding apparatus will be described with reference to FIG. 30.

The moving picture decoding apparatus of the third embodiment has the same configuration as the moving picture decoding apparatus of the first embodiment except that the temporal high frequency spatial low pass signal reconstruction unit 170 of the first embodiment is different. 30 is a block diagram showing the construction of a temporal high-frequency spatial low-frequency signal reconstruction section in the moving picture decoding apparatus according to the third embodiment. The time high pass spatial low pass signal reconstruction unit shown in FIG. 30 is weighted by the weighting unit 190 to the output 1090 of the time high pass signal low pass signal reconstruction unit shown in FIG. 15. After that, the point is output as the time high-frequency signal 1076. In the weighting unit 190, the inverse of the weighting coefficient in the weighting unit 159 in FIG. 29 is weighted.

Next, an embodiment of the configuration corresponding to the video encoding apparatus and the video decoding apparatus according to the second embodiment will be described. FIG. 31 is a block diagram showing a configuration of a second reduced image generation unit in which a weighting unit is added to the second reduced image generation unit in the second embodiment.

The weight reduction unit 159 is added to the second reduced image generation unit shown in FIG. 31 as in FIG. 29 as compared with the reduced image generation unit shown in FIG. 7. The weighting unit 159 performs weighting processing on the temporal high frequency spatial low frequency signal 124 which is the output of the low frequency signal generating unit 124, and inversely transforms the resulting temporal high frequency spatial low frequency signal 1079 into the temporal high frequency signal low frequency signal. Output to the unit 127. The weighting unit 159 performs weighting processing equivalent to the weighting unit 159 in FIG. 29.

32 is a block diagram showing the configuration of a second temporal high-frequency spatial low-frequency signal reconstruction unit corresponding to the video decoding apparatus of the second embodiment. 32 differs in that the weighting unit 190 is added in the same manner as in FIG. 30 as compared with the temporal high-frequency spatial low-band signal reconstruction unit shown in FIG. 16. The output 1091 of the temporal high pass signal generator 179 is weighted by the weighting unit 190 and then output as the temporal high pass signal 1176. In the weighting unit 190, the inverse of the weighting coefficient in the weighting unit 159 in FIG. 31 is weighted.

According to the third embodiment, weighting processing is performed immediately before reconstructing the temporal high-band spatial low-pass signal during time-space synthesis filtering in the decoding process. In the weighting process, in Embodiments 1 and 2, the encoding distortion of the reduced image signal is emphasized on the temporal high-band signal and propagated by weighting when the temporal high-band signal is synthesized by spatial direction synthesis filtering during decoding. We solve problem to become. Thereby, deterioration of a decoded image can be reduced.

Example 4

The fourth embodiment of the present invention will be described.

In the above-described first and second embodiments, the encoding distortion of the reduced image signal may be propagated to a decoded image having a larger resolution, depending on the image. For example, the moving picture coding apparatus having the configuration as shown in FIG. 1 optimizes the code amount allocation between the reduced picture signal coded data 1004 and the space-time high-frequency signal coded data 1005, thereby improving the quality of the decoded picture. You can adjust to some extent.

However, coded distortions, such as block distortion and ringing, generated in the decoded reduced image signal cannot be completely reduced by the space-time high-band signal.

Therefore, the moving picture coding apparatus and the moving picture decoding apparatus according to the fourth embodiment for solving the above-described problems will be described.

First, the video encoding device and the video decoding device of the fourth embodiment corresponding to the video encoding device and the video decoding device of the first embodiment will be described.

The moving picture coding apparatus of the fourth embodiment has the same structure as the moving picture coding apparatus of the first embodiment except that the reduced picture generating unit shown in FIG. 4 is different.

33 is a block diagram showing the configuration of a reduced image generation unit in the fourth embodiment. After the time high frequency signal 1014 has been processed by the filter 200 in comparison with the reduced image generation unit illustrated in FIG. 6, the output 1078 outputs the time high frequency signal inverse transform unit 121. The input is different. The filter 200 smoothes or removes noise with respect to the input signal.

Next, a video decoding apparatus for decoding a video coded by the above-described video encoding apparatus will be described.

The moving picture decoding apparatus of the fourth embodiment has the same configuration as the moving picture decoding apparatus of the first embodiment except that the temporal high frequency spatial low pass signal reconstruction unit shown in FIG. 15 is different.

34 is a block diagram showing the construction of a temporal high-band spatial low-band signal reconstruction unit in the fourth embodiment. The temporal high frequency spatial low pass signal reconstruction unit shown in FIG. 34 is compared with the temporal high frequency spatial low pass signal reconstruction unit shown in FIG. 15, after the output 1090 of the temporal high frequency signal generator is processed by the filter 201, and then the temporal high frequency signal 1076. ) Is different. The filter 201 reduces distortion or noise in a reduced decoded image signal such as de-ringing or block distortion removal.

Next, the video encoding device and the video decoding device of the fourth embodiment corresponding to the video encoding device and the video decoding device of the second embodiment will be described.

35 is a block diagram showing the configuration of a second reduced image generation unit in the fourth embodiment. The reduced image generating unit shown in FIG. 35 is different from the reduced image generating unit shown in FIG. 7 in that a filter 202 is added. The filter 202 performs the same processing as that of the filter 200 in FIG. 33 with respect to the temporal high frequency spatial low frequency signal 124 which is the output of the low frequency signal generator 124, and the resulting temporal high frequency spatial low frequency signal ( 1079 is output to the time high-frequency signal inverse transform unit 127.

Next, a video decoding apparatus for decoding a video coded by the above-described video encoding apparatus will be described.

The moving picture decoding apparatus of the fourth embodiment has the same configuration as the moving picture decoding apparatus of the second embodiment except that the temporal high-pass spatial low-pass signal reconstruction unit shown in FIG. 16 is different.

36 is a block diagram showing the configuration of a second temporal high-frequency spatial low-frequency signal reconstruction section in the fourth embodiment. The temporal high-pass spatial low-pass signal reconstruction unit shown in FIG. 36 differs from the point where the filter 203 is added compared with the temporal high-pass spatial low-pass signal reconstruction unit shown in FIG. The output 1091 of the time high pass signal generator 179 is processed by the filter 203 and then output as the time high pass signal 1076. In the filter 203, the same processing as that of the filter 201 in FIG. 34 is performed.

According to the fourth embodiment, filtering such as noise reduction is performed as the preprocessing of the encoding of the reduced image signal and the preprocessing of the encoding. As a result, it is possible to reduce the influence of the encoding distortion in the decoded reduced image signal on the decoded image signal having a larger resolution.

Example 5

In hierarchical coding of a moving picture signal, it is necessary to concentrate the power on a low-frequency signal when the image signal is frequency-divided in order to encode with high efficiency. When the same frequency division is performed in the reduced picture generation unit in the present invention, the detail of the reduced picture signal is increased, which makes it difficult to encode the reduced picture signal alone. On the contrary, there is a problem that the encoding efficiency of the entire signal is lowered in the low-pass signal generation filtering in which the fineness of the reduced image signal is properly adjusted. In Example 5, the Example which solves this subject is described.

The moving picture coding apparatus according to the fifth embodiment will be described with reference to FIG. The moving picture coding apparatus of the fifth embodiment has the same structure as that of the first embodiment except that the reduced picture generating unit shown in FIG. 4 is different. 38 is a block diagram showing a configuration of a reduced image generation unit according to the fifth embodiment.

The reduced image generation unit shown in FIG. 38 is compared with the reduced image generation unit shown in FIG. 6, and after the time low-pass signal 1010 has been processed by the filter 300, the output 1079 is sent to the motion compensation unit 120. The input point is different. The filter 300 performs smoothing, blurring, etc. on the signal used as an input.

39 is a block diagram showing the configuration of a second reduced image generation unit in the fifth embodiment. The reduced image generating unit shown in FIG. 39 differs from the reduced image generating unit shown in FIG. 7 in that a filter 301 is added in the same manner as in FIG. 38. After the time low pass signal 1010 is processed by the filter 300, its output 1079 is input to the motion compensator 120. The filter 300 performs the same process as the filter 300 in FIG. 38.

Next, the video decoding apparatus for the above-described video encoding apparatus will be described with reference to FIGS. 40 and 41.

The video decoding apparatus of the fifth embodiment has the same structure as the invention of the first embodiment except that the temporal high-pass spatial low-pass signal reconstruction unit shown in FIG. 14 is different. 40 is a block diagram showing the construction of a temporal high-frequency spatial low-frequency signal reconstruction section according to the fifth embodiment. The temporal high pass spatial low pass signal reconstruction unit shown in FIG. 40 is compared to the temporal high pass spatial low pass signal reconstruction unit shown in FIG. 15, and then processed to the motion compensation unit 173 after the temporal low pass signal 1072 is processed by the filter 400. The input point is different. In the filter 400, the same processing as that of the filter 300 in FIG. 38 is performed.

Fig. 41 is a block diagram showing the construction of a second temporal high-frequency spatial low-frequency signal reconstruction section in the fifth embodiment. The time high-pass spatial low pass signal reconstruction unit shown in FIG. 41 differs from the time high-space spatial low pass signal reconstruction unit shown in FIG. 16 in that a filter 400 is added in the same manner as in FIG. 40. The time low pass signal 1072 is processed by the filter 400 and then input to the motion compensator 173. In the filter 400, the same processing as that of the filter 300 in FIG. 39 is performed.

According to such a configuration, smoothing or the like is performed on the temporal low pass signal when generating the reduced image signal. Thereby, coding of the reduced image signal itself can be facilitated without changing the time-space high-band signal. At the time of decoding, an equal filtering is performed on the temporal low pass signal in the temporal high pass spatial low pass signal reconstruction unit. As a result, the precision of the generated temporal high-frequency spatial low-frequency signal does not deteriorate.

Example 6

A sixth embodiment of the present invention will be described.

The moving picture coding apparatus and the moving picture decoding apparatus according to the present invention can be configured by hardware, as can be seen from the above description, but can also be implemented by a computer program.

Fig. 42 is a general block diagram of an information processing system implemented with a moving picture encoding apparatus according to the present invention.

The information processing system shown in FIG. 42 includes a processor 500, a program memory 501, and storage media 502 and 503. The storage mediums 502 and 503 may be separate storage mediums or may be storage areas formed of the same storage medium. As the storage medium, a magnetic storage medium such as a hard disk can be used.

The program memory 501 includes a time-space frequency division unit 101, a reduced image signal encoding unit 102, a space-time high-frequency signal encoding unit 103 in the video encoding apparatus of the first to fifth embodiments described above, A program for causing the processor 500 to perform the processing as the motion information encoder 104 and the multiplexer 105 is stored, and the processor 500 operates by the program, and the result is stored in the storage medium 502 or 503. ).

The program memory 501 further includes a demultiplexer 150, a reduced image signal decoder 151, a space-time high-band signal decoder 152 in the moving picture decoding apparatus of the first to fifth embodiments described above. A program that causes the processor 500 to perform the processing as the motion information decoding unit 153 and the time-space frequency synthesizing unit 154 is stored. The program operates by the program, and the result is stored in the storage medium ( 502 or 503).

As can be seen from the above description, all or part of the hardware can be realized by a computer program.

Claims (87)

delete delete delete delete delete delete A video encoding apparatus comprising a time-space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding an layered signal, The time space division filtering unit, A time direction filtering unit for filtering the moving picture signal in a time direction to generate motion information indicating a motion between the time low pass signal, the time high pass signal, and each image signal constituting the moving picture signal; A size reduction image generating unit for generating a size reduction image signal obtained by reducing the size of a moving image signal corresponding to the time high frequency signal by using the time high frequency signal and the time low frequency signal; A high frequency signal generation unit configured to generate a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, The temporal low-pass signal or the size-reduced image signal is executed by the time-space division filtering unit to layer a moving image signal, and then the time low-pass signal, the size-reduced image signal, the temporal high-frequency spatial high-band signal, and the motion information are signaled. The encoding processor encodes, The size reduction image generation unit, A motion compensator for performing motion compensation on the temporal low-pass signal based on the motion information to generate a predicted image; A temporal high frequency signal inverse transform unit which generates a video signal corresponding to the temporal high frequency signal from the predicted image and the temporal high frequency signal; A low pass signal generation unit for performing spatial filtering on the moving image signal generated by the temporal high pass signal inverse transform unit to generate a reduced-sized decoded image signal that is a spatial low pass component. And a moving picture encoding apparatus. The method of claim 7, wherein The temporal high frequency signal has a weighting process for performing a weighting process in comparison with the temporal low frequency signal, or performing a weighting process on a portion having the temporal high frequency signal, and outputting the weighted part to the temporal high frequency signal inverse transform unit. A moving picture coding apparatus characterized by the above-mentioned. A video encoding apparatus comprising a time-space division filtering unit for layering a moving image signal and a signal encoding processing unit encoding an layered signal, The time space division filtering unit, A time direction filtering unit for filtering the moving picture signal in a time direction to generate motion information indicating a motion between the time low pass signal, the time high pass signal, and each image signal constituting the moving picture signal; A size reduction image generating unit for generating a size reduction image signal obtained by reducing the size of a moving image signal corresponding to the time high frequency signal by using the time high frequency signal and the time low frequency signal; A high frequency signal generation unit configured to generate a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, The temporal low-pass signal or the size-reduced image signal is executed by the time-space division filtering unit to layer a moving image signal, and then the time low-pass signal, the size-reduced image signal, the temporal high-frequency spatial high-band signal, and the motion information are signaled. The encoding processor encodes, The size reduction image generation unit, A low pass signal generation unit for performing spatial filtering on the temporal low pass signal and the temporal high pass signal to generate a temporal low pass spatial low pass signal and a temporal high pass spatial low pass signal, respectively; A motion information converter for converting motion information according to the resolution conversion ratio of the spatial filtering in the low pass signal generator; A motion compensation unit for performing motion compensation on the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting unit, and generating a predicted image; A temporal high frequency signal inverse transform unit for generating a size-reduced image signal by using the predicted image and the temporal high frequency spatial low frequency signal And a moving picture encoding apparatus. The method of claim 9, A weighting process is performed on the temporal high-band spatial low-band signal, or a weighting process is performed on a portion having the temporal high-band spatial low-band signal and outputs to the temporal high-band signal inverse transform unit. Encoding device. The method according to any one of claims 7 to 10, And the low-band signal generator is frequency component extraction by subband division. The method according to any one of claims 7 to 10, And the high frequency signal generator is extracting frequency components by subband division. The method according to any one of claims 7 to 10, The low-pass signal generator generates a reduced-size image signal by a first subsample filter, And the high pass signal generator generates a high pass signal by a second sub sample filter paired with the first sub sample filter. delete delete delete A video decoding apparatus comprising: a signal decoding processing unit for decoding coded data of a hierarchically encoded video signal and generating a layered signal; and a time-space synthesis filtering unit for synthesizing the layered signal, The temporal spatial synthesis filtering unit generates spatial low-pass components by spatial layering in temporal high-frequency components based on temporal hierarchies based on a decoded image signal that is a result of decoding in an arbitrary resolution layer. A video decoding device, characterized in that frequency decoding is performed to generate a decoded video signal of a higher resolution layer of a higher layer. delete delete delete delete delete delete delete A video decoding apparatus comprising a signal decoding processing unit for decoding encoded data to generate a layered signal and a time-space synthesis filtering unit for combining the layered image signal, The time space synthesis filtering unit, A temporal high pass spatial low pass signal for generating a temporal high pass spatial low pass signal, which is a spatial low pass component of the temporal high pass signal paired with the temporal low pass signal, from a temporal low pass signal and a size reduction decoded image signal that is a result of synthesis at an arbitrary resolution layer. Reconstruction unit, A spatial synthesis filter for synthesizing a temporal high frequency signal, which is a spatial high frequency component of the temporal high frequency signal, and the temporal high frequency spatial low frequency signal to generate a temporal high frequency signal; Decoding from the temporal high-band signal, the temporal low-band signal, and motion information, has a temporal inverse filtering unit for generating an image signal, The signal decoding processing unit decodes the temporal low pass signal, the size reduction decoded image signal, the temporal high pass spatial high pass signal, and the motion information from the encoded data, The temporal high frequency spatial low frequency signal reconstruction unit, A motion compensator for performing motion compensation on the temporal low-pass signal based on the motion information to generate a predicted image; A low pass signal generation unit for generating a spatial low pass prediction signal which is a spatial low pass component of the predicted image; A temporal high pass signal generation unit for generating a temporal high pass component from the spatial low pass prediction signal and the size reduction decoded image signal, Outputting the output of the temporal high frequency signal generator as a temporal high frequency spatial low frequency signal, And the low-pass signal generator generates a reduced size image by a subsample filter. The method of claim 25, And the spatial synthesis filtering unit is a combination of a high pass signal generated by a second sub sample filter paired with the sub sample filter and a low pass signal generated by the sub sample filter. delete delete delete delete delete delete delete delete delete A video encoding method comprising time-space division filtering for layering a moving image signal and signal encoding processing for encoding the layered signal, comprising: The time space division filtering, A time-direction filtering step of filtering the moving picture signal in the time direction to generate motion information indicative of movement between the time low-pass signal, the time high-pass signal and each of the image signals constituting the moving picture signal; A size reduction image generation step of generating a size reduction image signal obtained by reducing the size of a moving image signal corresponding to the time high frequency signal by using the time high frequency signal and the time low frequency signal; A high frequency signal generating step of generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, After the temporal space division filtering step is performed on the temporal low-pass signal or the size-reduced image signal to stratify a moving image signal, the temporal low-pass signal, the size-reduced image signal, the temporal high-band spatial high-band signal, and the motion information Encoded in the signal encoding process step, The size reduction image generation step, A motion compensation step of performing motion compensation on the temporal low-frequency signal based on the motion information to generate a predicted image; A temporal high frequency signal inverse transform step of generating a moving picture signal corresponding to the temporal high frequency signal from the predicted image and the temporal high frequency signal; And a low-pass signal generating step of performing spatial filtering on the video signal generated in the temporal high-band signal inverse transform step to generate a reduced-sized decoded image signal that is a spatial low-pass component. The method of claim 36, Before the temporal high-band signal inverse transform step, the temporal high-band signal is weighted in comparison with the temporal low-band signal, or the weighting process is performed on a portion having the temporal high-band signal. . A video encoding method comprising time-space division filtering for layering a moving image signal and signal encoding processing for encoding the layered signal, comprising: The time space division filtering, A time-direction filtering step of filtering the moving picture signal in the time direction to generate motion information indicative of movement between the time low-pass signal, the time high-pass signal and each of the image signals constituting the moving picture signal; A size reduction image generation step of generating a size reduction image signal obtained by reducing the size of a moving image signal corresponding to the time high frequency signal by using the time high frequency signal and the time low frequency signal; A high frequency signal generating step of generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, After the temporal space division filtering step is performed on the temporal low-pass signal or the size-reduced image signal to stratify a moving image signal, the temporal low-pass signal, the size-reduced image signal, the temporal high-band spatial high-band signal, and the motion information Encoded in the signal encoding process step, The size reduction image generation step, A low pass signal generation step of performing spatial filtering on the temporal low pass signal and the temporal high pass signal to generate a temporal low pass spatial low pass signal and a temporal high pass spatial low pass signal, respectively; A motion information conversion step of converting motion information according to the resolution conversion ratio of the spatial filtering in the low pass signal generation step; A motion compensation step of performing motion compensation on the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting step to generate a predicted image; A temporal high frequency signal inverse transform step of generating a size-reduced image signal by using the predicted image and the temporal high frequency spatial low frequency signal A video encoding method comprising: a. The method of claim 38, And before the temporal high frequency signal inverse transform step, weighting processing is performed on the temporal high frequency spatial low frequency signal or weighting processing is performed on a portion having the temporal high frequency spatial low frequency signal. The method according to any one of claims 36 to 39, wherein The high frequency signal generation step is a frequency component extraction by subband division. The method according to any one of claims 36 to 39, wherein The low-pass signal generating step is a frequency component extraction by subband division. The method according to any one of claims 36 to 39, wherein The low-pass signal generating step generates a size-reduced image signal by a first subsample filter, And the high pass signal generating step generates a high pass signal by a second sub sample filter paired with the first sub sample filter. delete delete delete A video decoding method comprising a signal decoding processing step of decoding coded data of a hierarchically encoded video signal, generating a layered signal, and a time-space synthesis filtering step of synthesizing the layered signal, The temporal spatial synthesis filtering step generates spatial low-pass components by spatial layering in temporal high-frequency components based on temporal hierarchies based on the decoded image signal which is a result of decoding in an arbitrary resolution layer. A directional frequency synthesis is performed to generate a decoded video signal of a higher resolution layer of a higher layer. delete A video decoding method comprising a signal decoding processing step of decoding encoded data to generate a layered signal and a time-space synthesis filtering step of synthesizing the layered image signal, The time space synthesis filtering step, A temporal high pass spatial low pass signal for generating a temporal high pass spatial low pass signal, which is a spatial low pass component of the temporal high pass signal paired with the temporal low pass signal, from a temporal low pass signal and a size reduction decoded image signal that is a result of synthesis at an arbitrary resolution layer. Reconstruction step, A spatial synthesis filtering step of synthesizing a temporal high frequency spatial high frequency signal, which is a spatial high frequency component of the temporal high frequency signal, and the temporal high frequency spatial low frequency signal, and generating a temporal high frequency signal; And a temporal inverse filtering step of generating a decoded image signal from the temporal high pass signal, the temporal low pass signal, and motion information, The signal decoding processing step decodes the temporal low pass signal, the size reduction decoded image signal, the temporal high pass spatial high pass signal, and the motion information from the encoded data, The temporal high-band spatial low-band signal reconstruction step, A motion compensation step of performing motion compensation on the temporal low-pass signal based on the motion information to generate a predicted image; A low pass signal generation step of generating a spatial low pass prediction signal which is a spatial low pass component of the predicted image; And a temporal high pass signal generating step of generating a temporal high pass component from the spatial low pass prediction signal and the size reduction decoded image signal, Outputting the output of the temporal high pass signal generating step as a temporal high pass spatial low pass signal, And the low-pass signal generating step generates a size-reduced image by a subsample filter. The method of claim 48, A video decoding method characterized in that a weighting process for compensating weighting made at the time of encoding is performed on the temporal high frequency component generated in the temporal high frequency signal generating step, and the weighted signal is output as a temporal high frequency spatial low frequency signal. . A video decoding method comprising a signal decoding processing step of decoding encoded data to generate a layered signal and a time-space synthesis filtering step of synthesizing the layered image signal, The time space synthesis filtering step, A temporal high pass spatial low pass signal for generating a temporal high pass spatial low pass signal, which is a spatial low pass component of the temporal high pass signal paired with the temporal low pass signal, from a temporal low pass signal and a size reduction decoded image signal that is a result of synthesis at an arbitrary resolution layer. Reconstruction step, A spatial synthesis filtering step of synthesizing a temporal high frequency spatial high frequency signal, which is a spatial high frequency component of the temporal high frequency signal, and the temporal high frequency spatial low frequency signal, and generating a temporal high frequency signal; And a temporal inverse filtering step of generating a decoded image signal from the temporal high pass signal, the temporal low pass signal, and motion information, The signal decoding processing step decodes the temporal low pass signal, the size reduction decoded image signal, the temporal high pass spatial high pass signal, and the motion information from the encoded data, The temporal high-band spatial low-band signal reconstruction step, A low pass signal generating step of generating a spatial low pass component of the temporal low pass signal and outputting it as a temporal low pass spatial low pass signal; A motion information conversion step of converting motion information according to the resolution conversion ratio of the input / output image of the low pass signal generation step; A motion compensation step of performing motion compensation on the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting step to generate a predicted image; And a temporal high frequency signal generating step of generating a temporal high frequency component from the predicted image and the size reduction decoded image signal, Outputting the output of the temporal high pass signal generating step as a temporal high pass spatial low pass signal, And the low-pass signal generating step generates a size-reduced image by a subsample filter. 51. The method of claim 50, And a weighting process for compensating the weighting made at the time of encoding is output to the output of the temporal high pass signal generating step, and outputting this as a temporal high pass spatial low pass signal. delete delete delete The method according to any one of claims 48 to 51, And the spatial synthesis filtering step is a synthesis of a high pass signal generated by a second sub sample filter paired with the sub sample filter and a low pass signal generated by the sub sample filter. delete delete delete delete delete delete delete delete delete A computer-readable recording medium having recorded thereon a control program of a video encoding apparatus, comprising: a time-space division filtering unit for layering a moving image signal; and a signal encoding processing unit encoding a layered signal; The control program may include the time space division filtering unit, A time direction filtering unit for filtering the moving picture signal in a time direction to generate motion information indicating a motion between the time low pass signal, the time high pass signal, and each image signal constituting the moving picture signal; A size reduction image generating unit for generating a size reduction image signal obtained by reducing the size of a moving image signal corresponding to the time high frequency signal by using the time high frequency signal and the time low frequency signal; Function as a high frequency signal generation unit for generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, The temporal low-pass signal or the size-reduced image signal is executed by the time-space division filtering unit to layer a moving image signal, and then the time low-pass signal, the size-reduced image signal, the temporal high-frequency spatial high-band signal, and the motion information are signaled. The encoding processing unit encodes, The control program may further include the size reduction image generating unit, A motion compensator for performing motion compensation on the temporal low-pass signal based on the motion information to generate a predicted image; A temporal high frequency signal inverse transform unit which generates a video signal corresponding to the temporal high frequency signal from the predicted image and the temporal high frequency signal; A low pass signal generation unit for performing spatial filtering on the moving image signal generated by the temporal high pass signal inverse transform unit to generate a reduced-sized decoded image signal that is a spatial low pass component. A computer-readable recording medium having recorded thereon a control program of a moving picture coding apparatus. The method of claim 65, The control program is a video encoding apparatus, The temporal high frequency signal functions as a weighting unit for performing a weighting process in comparison with the temporal low frequency signal, or performing a weighting process on a portion having the temporal high frequency signal, and outputting the weighted high frequency signal to the temporal high frequency signal inverse transform unit. A computer-readable recording medium having recorded thereon a control program of a moving picture encoding apparatus. A computer-readable recording medium having recorded thereon a control program of a video encoding apparatus, comprising: a time-space division filtering unit for layering a moving image signal; and a signal encoding processing unit encoding a layered signal; The control program may include the time space division filtering unit, A time direction filtering unit for filtering the moving picture signal in a time direction to generate motion information indicating a motion between the time low pass signal, the time high pass signal, and each image signal constituting the moving picture signal; A size reduction image generating unit for generating a size reduction image signal obtained by reducing the size of a moving image signal corresponding to the time high frequency signal by using the time high frequency signal and the time low frequency signal; Function as a high frequency signal generation unit for generating a temporal high frequency spatial high frequency signal corresponding to a spatial high frequency component with respect to the temporal high frequency signal, The temporal low-pass signal or the size-reduced image signal is executed by the time-space division filtering unit to layer a moving image signal, and then the time low-pass signal, the size-reduced image signal, the temporal high-frequency spatial high-band signal, and the motion information are signaled. The encoding processor encodes, The control program may further include the size reduction image generating unit, A low pass signal generation unit for performing spatial filtering on the temporal low pass signal and the temporal high pass signal to generate a temporal low pass spatial low pass signal and a temporal high pass spatial low pass signal, respectively; A motion information converter for converting motion information according to the resolution conversion ratio of the spatial filtering in the low pass signal generator; A motion compensation unit for performing motion compensation on the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting unit, and generating a predicted image; A temporal high frequency signal inverse transform unit for generating a size-reduced image signal by using the predicted image and the temporal high frequency spatial low frequency signal A computer-readable recording medium having recorded thereon a control program of a moving picture coding apparatus. The method of claim 67, The control program is a video encoding apparatus, A weighting process is performed on the temporal high-band spatial low-band signal, or a weighting process is performed on a portion having the temporal high-band spatial low-band signal to function as a weighting unit output to the temporal high-band signal inverse transform unit. A computer-readable recording medium having recorded thereon a control program of a moving picture encoding apparatus. The method of any one of claims 65 to 68, And the high-band signal generation section is frequency component extraction by subband dividing. The method of claim 65, And the low-band signal generator is frequency component extraction by subband division. The computer-readable recording medium having recorded thereon a control program of a video encoding apparatus. The method of claim 65, The low-pass signal generator generates a reduced-size image signal by a first subsample filter, And a high pass signal generated by the high pass signal generation section by a second sub sample filter paired with the first sub sample filter. delete delete delete A computer-readable recording medium having recorded thereon a control program of a moving picture decoding apparatus including a signal decoding processing unit for decoding coded data of a hierarchically encoded moving picture signal and generating a layered signal and a time-space synthesis filtering unit for synthesizing the layered signal. as, The control program generates the spatial low frequency component by spatial layering in the temporal high frequency component by temporal layering based on the decoded image signal that is a result of decoding in an arbitrary resolution layer. And a control program of a moving picture decoding apparatus, characterized by performing combining and time-direction frequency combining to generate a decoded video signal of a higher resolution layer of a higher layer. delete delete delete delete delete delete delete A computer-readable recording medium having recorded thereon a control program of a moving picture decoding apparatus including a signal decoding processing unit for decoding encoded data to generate a layered signal and a time-space synthesis filtering unit for synthesizing the layered image signal, The control program is the time-space synthesis filter, A temporal high-frequency spatial low-pass that generates a temporal high-frequency spatial low-frequency signal that is a spatial low-pass component of the temporal high-frequency signal paired with the temporal low-frequency signal from the temporal low-band signal and the size reduction decoded image signal that is a result of synthesis in an arbitrary resolution layer. A signal reconstruction unit, A spatial synthesis filter for synthesizing a temporal high frequency signal, which is a spatial high frequency component of the temporal high frequency signal, and the temporal high frequency spatial low frequency signal to generate a temporal high frequency signal; Function as a temporal inverse filtering unit for generating a decoded image signal from the temporal high pass signal, the temporal low pass signal, and motion information, The signal decoding processing unit decodes the temporal low pass signal, the size reduction decoded image signal, the temporal high pass spatial high pass signal, and the motion information from the encoded data, The control program may also include the temporal high-band spatial low-band signal reconstruction unit, A low frequency signal generator for generating a spatial low frequency component of the temporal low frequency signal and outputting the spatial low frequency component as a temporal low frequency spatial low frequency signal; A motion information converter for converting motion information according to the resolution conversion ratio of the input / output image of the low-band signal generator; A motion compensation unit for performing motion compensation on the temporal low-pass spatial low-band signal based on the motion information converted by the motion information converting unit, and generating a predicted image; Function as a temporal high-band signal generator for generating temporal high-band components from the predicted image and the size-reduced decoded image signal, Outputting the output of the temporal high frequency signal generator as a temporal high frequency spatial low frequency signal, And the low-pass signal generator generates a reduced-size image by a subsample filter. The computer-readable recording medium having recorded thereon a control program of a moving picture decoding apparatus. 84. The method of claim 83, And the spatial synthesis filtering unit is a combination of a high pass signal generated by a second sub sample filter paired with the sub sample filter and a low pass signal generated by the sub sample filter. Recorded computer readable recording medium. delete delete delete

Images