JP2009267492A - Video image hierarchy encoding apparatus, video image hierarchy encoding method, video image hierarchy encoding program, video image hierarchy decoding apparatus, video image hierarchy decoding method, and video image hierarchy decoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To create a prediction video signal which has high correlation with an original video image signal by appropriately carrying out high resolution processing of a basic decoded video image signal which is decoded basic coding data, and achieve a more efficient video image hierarchy encoding. <P>SOLUTION: A video image hierarchy encoding apparatus 100 is provided with: a high pass filtering unit 208 which separates a high frequency component from the basic decoded video image signal, and creates a high frequency separation signal in a pre-stage of prediction processing between hierarchies when spacial interpolation is carried out from a basic layer to an extension layer; and a high frequency extraction filtering unit 210 which extracts the high frequency component corresponding to the original video image signal from the high frequency separation signal, and creates a high frequency equivalent signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video hierarchical encoding device, a video hierarchical encoding method, a video hierarchical encoding program, a video hierarchical decoding device, and a video hierarchical decoding that generate or use multiplexed data that can be restored to a plurality of video signals having different spatial resolutions. The present invention relates to a method and a video hierarchical decoding program.

  Conventionally, in video signal encoding or decoding, many encoding schemes or decoding schemes have been proposed that realize the spatial resolution, temporal resolution, and SNR (Signal to Noise Ratio) scalability. Practical use has been made. In particular, the spatial resolution scalability (spatial scalability) includes a wide range of applications including encoding of still images, and various techniques have been disclosed.

  For example, Patent Document 1 describes a basic configuration of spatial scalability for generating multiplexed data that can be restored to a plurality of video signals having different spatial resolutions and image formats. Here, the spatially interpolated base layer (base layer) is combined with the motion-compensated temporal prediction of the enhancement layer (enhancement layer) through the selection of appropriate weights and used to encode the enhancement layer A video signal is generated. With such a configuration, it is possible to increase bandwidth efficiency and appropriately cope with transmission errors.

On the other hand, in the field of image enlargement methods, there is an inter-layer estimation processing technique of Non-Patent Document 1 that estimates and adds a high-frequency component appropriate for the resolution after enlargement when an image is enlarged. Non-Patent Document 1 describes a technique in which the idea of the Laplacian pyramid in hierarchical coding is applied to an image enlargement method, and makes use of the fact that the correlation of Laplacian components between layers is strong, so that only the signal of the target layer is used as a space. We have estimated the Laplacian component of the hierarchy with one higher resolution.
JP-A-7-162870 "Image magnification method with arbitrary magnification with high frequency component estimation" Takamasa Takamasa, Taguchi Ryo, IEICE Transactions Vol.J84-A No.9 pp.1192-1201 September 2001

  In order to realize the spatial scalability of the video signal in the video hierarchical encoding device, as shown in Patent Document 1, the basic decoded video signal obtained by decoding the basic encoded data in the base layer is interpolated. Then, it is used as a predicted video signal in enhancement layer coding. This is because there is a certain degree of correlation between an original video signal (hereinafter simply referred to as an original video signal) input in enhancement layer coding and a basic video signal, that is, a part of the frequency of the original video signal. This is based on the fact that the basic video signal contains components.

  Here, the higher the correlation between the basic decoded video signal and the original video signal used for enhancement layer encoding, the higher the encoding efficiency. Therefore, in order to realize more efficient coding, the basic decoded video signal is not only simply spatially interpolated but also predicted video signal through inter-layer estimation processing that makes it closer to the original video signal. Must be generated.

  For such spatial expansion (interpolation), it is desirable to apply the inter-layer estimation processing technique of Non-Patent Document 1 described above. However, the inter-layer estimation processing technique of Non-Patent Document 1 is intended for natural image enlargement in the first place, and causes various problems when applied to spatial interpolation as it is. This is due to the fact that the basic decoded video signal obtained by decoding the basic encoded data is a signal that is deteriorated compared to the basic video signal that is the original. For example, when the quantization step width in the encoding of the base layer is large, the quantization error of the decoded basic decoded video signal becomes large and the correlation with the original video signal becomes low.

  Therefore, when the inter-layer estimation processing technique using the correlation between layers in a natural image is simply applied to spatial interpolation, the expected encoding efficiency may not be obtained. In order to increase the coding efficiency, it is necessary to consider the influence of coding deterioration and the characteristics of the inter-layer estimation process when generating a predicted video signal between layers. Considering the influence of such coding degradation and the characteristics of the inter-layer estimation processing, not only the video hierarchical coding device but also the video hierarchical decoding device that restores the original video signal by the same procedure as the video hierarchical coding device. Need to apply.

  In view of such problems, the present invention generates a predicted video signal having a high correlation with the original video signal by appropriately increasing the resolution of the basic decoded video signal obtained by decoding the basic encoded data, and is more efficient. An object of the present invention is to provide a video hierarchy encoding apparatus, a video hierarchy encoding method, and a video hierarchy encoding program capable of realizing video hierarchy encoding.

  In addition, in accordance with such a video hierarchical encoding device, the present invention appropriately generates a predicted video signal highly correlated with the original video signal by appropriately increasing the resolution of the multiplexed data from the video hierarchical encoding device. Another object of the present invention is to provide a video hierarchy decoding device, a video hierarchy decoding method, and a video hierarchy decoding program that can be generated and realize more efficient video hierarchy decoding.

  In order to solve the above-described problem, a representative configuration of the present invention is a video hierarchical encoding device that generates multiplexed data that can be restored to a plurality of video signals having different spatial resolutions. A spatial decimation unit that generates a basic video signal by reducing the size, a basic layer encoding unit that generates basic encoded data by encoding the basic video signal, and generates a basic decoded video signal by decoding the basic encoded data A base layer decoding unit that encodes a prediction error signal derived using the original video signal and the predicted video signal, and a spatial interpolation unit that generates a predicted video signal by expanding the spatial resolution of the basic decoded video signal. And an enhancement layer encoding unit that generates extended encoded data and a multiplexing unit that multiplexes basic encoded data and extended encoded data to generate multiplexed data. The interpolation unit generates a high-frequency separated signal by separating a high-frequency component from the basic decoded video signal, and generates a high-frequency equivalent signal by extracting a high-frequency component corresponding to the original video signal from the high-frequency separated signal The difference between the high-frequency separation signal and the high-frequency equivalent signal, the high-frequency extraction filtering unit, the high-frequency spatial interpolation unit that expands the spatial resolution of the high-frequency equivalent signal through inter-layer estimation processing of the high-frequency component, and generates the high-frequency component high-resolution signal A differential generation unit that generates a differential signal, a differential spatial interpolation unit that generates a differential high-resolution signal by expanding the spatial resolution of the differential signal, and a basic high-resolution by expanding the spatial resolution of the basic decoded video signal Basic interpolation unit that generates signal, high-frequency component high-resolution signal and differential high-resolution signal And having a signal combining unit for generating a prediction picture signal by synthesizing the basic high-resolution signal.

  According to the present invention, a prediction error signal between layers is obtained by adding an inter-layer estimation process accompanied by an estimation of an original video signal to simple spatial interpolation (spatial expansion) in inter-layer prediction of video hierarchical coding. By making it smaller, it becomes possible to realize high-quality video hierarchical coding efficiently. Also, in spatial interpolation, the basic decoded video obtained by decoding the basic encoded data in consideration of the encoding deterioration of the basic decoded video signal (low resolution signal) obtained by decoding the basic encoded data and the characteristics of the inter-layer estimation process. After passing only the high-frequency component, the signal is separated into a high-frequency component corresponding to the original video signal to be estimated and other signals, and the original video signal (high resolution signal) is estimated by processing suitable for each. Therefore, the predicted video signal can be generated more appropriately, and efficient video hierarchical coding can be realized.

The high-frequency extraction filtering unit is an ε-filter that is a non-linear digital filter that removes noise from a video signal having a sudden change, and the multiplexing unit also multiplexes a coefficient sequence a _k and a threshold ε that are ε-filter parameters. It's okay.

With this ε-filter configuration, only the high-frequency component suitable for the inter-layer estimation process is given to the inter-layer estimation process, and the remaining difference signals are expanded by a process that is not specialized for the high-frequency component. In addition, by including the parameters _ak and ε in the final multiplexed data, appropriate video layer decoding using the same ε-filter is also possible in the video layer decoding device.

  Another representative configuration of the present invention is a video hierarchical encoding method for generating multiplexed data that can be restored to a plurality of video signals having different spatial resolutions, and reducing the spatial resolution of an original video signal to generate a basic video Generate signal, encode basic video signal to generate basic encoded data, decode basic encoded data to generate basic decoded video signal, separate high frequency components from basic decoded video signal, and separate high frequency separated signal The high frequency component corresponding to the original video signal is extracted from the high frequency separation signal to generate a high frequency equivalent signal, and the spatial resolution of the high frequency equivalent signal is expanded through the inter-layer estimation process of the high frequency component, and the high frequency component high resolution signal is generated. To generate a differential signal that is the difference between the high-frequency separated signal and the high-frequency equivalent signal, expand the spatial resolution of the differential signal to generate a differential high-resolution signal, and generate the space of the basic decoded video signal A basic high-resolution signal is generated by enlarging the image, and a high-frequency component high-resolution signal, a differential high-resolution signal, and a basic high-resolution signal are combined to generate a predicted video signal, and the original video signal and the predicted video signal are combined. The prediction error signal derived using the method is encoded to generate extended encoded data, and the basic encoded data and the extended encoded data are multiplexed to generate multiplexed data.

  Another typical configuration of the present invention is a video hierarchical encoding program that causes a computer to function as a video hierarchical encoding device that generates multiplexed data that can be restored to a plurality of video signals having different spatial resolutions. A spatial decimation unit that generates a basic video signal by reducing the spatial resolution of the original video signal, a base layer encoding unit that generates basic encoded data by encoding the basic video signal, and decodes the basic encoded data A base layer decoding unit that generates a basic decoded video signal, a high-pass filtering unit that generates a high frequency separated signal by separating high frequency components from the basic decoded video signal, and extracts a high frequency component corresponding to the original video signal from the high frequency separated signal High-frequency extraction filtering unit that generates a high-frequency equivalent signal, and the spatial resolution of the high-frequency equivalent signal is estimated between layers of high-frequency components. A high-frequency spatial interpolation unit that expands through processing to generate a high-frequency component high-resolution signal, a differential generation unit that generates a differential signal that is the difference between the high-frequency separation signal and the high-frequency equivalent signal, and a difference by expanding the spatial resolution of the differential signal Differential spatial interpolation unit that generates a high-resolution signal, basic interpolation unit that generates a basic high-resolution signal by expanding the spatial resolution of the basic decoded video signal, and high-frequency component high-resolution signal, differential high-resolution signal, and basic A spatial interpolation unit having a signal synthesis unit that synthesizes a high-resolution signal and generates a predicted video signal, and encodes a prediction error signal derived by using the original video signal and the predicted video signal, and an extended code An enhancement layer encoding unit for generating encoded data, and multiplexing the basic encoded data and the extended encoded data to A multiplexing unit for forming, characterized in that to function with.

  The component corresponding to the technical idea in the video hierarchy encoding apparatus mentioned above and its description are applicable also to the said video hierarchy encoding method and a video hierarchy encoding program.

  Another representative configuration of the present invention is a video hierarchical decoding apparatus capable of restoring a plurality of video signals having different spatial resolutions from multiplexed data subjected to spatial scalability, and at least basic encoding is performed from the multiplexed data. An extract unit that separates a plurality of encoded data having different spatial resolutions, including a data and extended encoded data, a base layer decoding unit that decodes the basic encoded data to generate a basic decoded video signal, and a basic decoded video A spatial interpolation unit that generates a predicted video signal by expanding the spatial resolution of the signal, an enhancement layer decoding unit that restores an original video signal using a prediction error signal and a predicted video signal obtained by decoding the extended encoded data; The high-pass filter that generates a high-frequency separation signal by separating high-frequency components from the basic decoded video signal A high-frequency extraction filtering unit that extracts a high-frequency component corresponding to the original video signal from the high-frequency separation signal to generate a high-frequency equivalent signal, and expands the spatial resolution of the high-frequency equivalent signal through inter-layer estimation processing of the high-frequency component. A high-frequency spatial interpolation unit that generates a high-frequency component high-resolution signal, a difference generation unit that generates a difference signal that is a difference between the high-frequency separation signal and the high-frequency equivalent signal, and a differential high-resolution by expanding the spatial resolution of the difference signal A differential spatial interpolation unit that generates a signal, a basic interpolation unit that generates a basic high-resolution signal by expanding the spatial resolution of the basic decoded video signal, a high-frequency component high-resolution signal, a differential high-resolution signal, and a basic high-level signal And a signal synthesizer that synthesizes the resolution signal to generate a predicted video signal.

  According to the present invention, inter-layer estimation processing accompanied by estimation of an original video signal is added to simple spatial interpolation (spatial expansion) in inter-layer prediction of video layer decoding, so that a prediction error signal between layers can be further improved. By reducing the size, it is possible to efficiently realize high-quality video hierarchical decoding. Also, in spatial interpolation, the basic decoded video obtained by decoding the basic encoded data in consideration of the encoding deterioration of the basic decoded video signal (low resolution signal) obtained by decoding the basic encoded data and the characteristics of the inter-layer estimation process. After passing only the high-frequency component, the signal is separated into a high-frequency component corresponding to the original video signal to be estimated and other signals, and the original video signal (high resolution signal) is estimated by processing suitable for each. Thus, the original video signal can be restored more appropriately, and efficient video hierarchical decoding can be realized.

The high-frequency extraction filtering unit is an ε-filter that is a non-linear digital filter that removes noise from a video signal having a sudden change, and the multiplexed data includes a coefficient sequence a _k and a threshold ε that are parameters of the ε-filter. It may be included.

With this ε-filter configuration, only the high-frequency component suitable for the inter-layer estimation process is given to the inter-layer estimation process, and the remaining difference signals are expanded by a process that is not specialized for the high-frequency component. Also, by obtaining the parameters _ak and ε from the multiplexed data, it is possible to execute the same processing as the ε-filter processing in the video layer encoding apparatus, and appropriate video layer decoding is possible.

  Another representative configuration of the present invention is a video hierarchical decoding method capable of restoring a plurality of video signals having different spatial resolutions from multiplexed data subjected to spatial scalability, and includes at least basic encoding from the multiplexed data. Separate multiple encoded data with different spatial resolution, including data and extended encoded data, decode basic encoded data to generate basic decoded video signal, separate high frequency components from basic decoded video signal, Generate a separated signal, extract the high frequency component corresponding to the original video signal from the high frequency separated signal to generate a high frequency equivalent signal, expand the spatial resolution of the high frequency equivalent signal through the inter-layer estimation process of the high frequency component, and increase the high frequency component high Generates a resolution signal, generates a differential signal that is the difference between the high-frequency separated signal and the high-frequency equivalent signal, expands the spatial resolution of the differential signal, and performs high-resolution differential Generate a signal, expand the spatial resolution of the basic decoded video signal to generate a basic high resolution signal, and combine the high-frequency component high resolution signal, the differential high resolution signal, and the basic high resolution signal to generate a predicted video signal The original video signal is restored using the prediction error signal obtained by decoding the extended encoded data and the prediction video signal.

Another representative configuration of the present invention includes a computer,
A video hierarchical decoding program for functioning as a video hierarchical decoding device capable of restoring a plurality of video signals having different spatial resolutions from multiplexed data subjected to spatial scalability, wherein the computer is configured to at least basic multiplexed data from the multiplexed data. An extract unit for separating a plurality of encoded data having different spatial resolutions, a base layer decoding unit for decoding basic encoded data to generate a basic decoded video signal, and a basic decoded video signal High-pass filtering unit that generates a high-frequency separation signal by separating high-frequency components from a high-frequency filtering unit that generates a high-frequency equivalent signal by extracting a high-frequency component corresponding to the original video signal from the high-frequency separation signal, spatial resolution of the high-frequency equivalent signal High-frequency component high resolution High-frequency spatial interpolation unit that generates a degree signal, a difference generation unit that generates a difference signal that is a difference between a high-frequency separation signal and a high-frequency equivalent signal, and a difference that generates a differential high-resolution signal by expanding the spatial resolution of the difference signal Spatial interpolation unit, basic interpolation unit that expands the spatial resolution of basic decoded video signal to generate basic high resolution signal, and synthesizes high frequency component high resolution signal, differential high resolution signal and basic high resolution signal A spatial interpolation unit having a signal synthesis unit that generates a predicted video signal, and an enhancement layer decoding unit that restores the original video signal using the prediction error signal obtained by decoding the extended encoded data and the predicted video signal; , And function.

  The component corresponding to the technical idea in the video hierarchy decoding apparatus mentioned above and its description are applicable also to the said video hierarchy decoding method and a video hierarchy decoding program.

  As described above, according to the present invention, the video signal obtained by decoding the basic encoded data is appropriately subjected to high-resolution processing, thereby generating a predicted video signal having a high correlation with the original video signal, and a more efficient video hierarchy. Encoding can be realized. In addition, in response to such a video hierarchical encoding device, by appropriately increasing the resolution of the multiplexed data from the video hierarchical encoding device, a predicted video signal highly correlated with the original video signal is generated, and more It is also possible to realize efficient video hierarchical decoding.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In this embodiment, estimation processing between layers is introduced in order to increase prediction efficiency between layers in video layer coding. With such an inter-layer estimation process, when the spatial resolution of a low-resolution basic decoded video signal is expanded, it can be as close to the original video signal as possible, and inter-layer prediction can be achieved by improving the prediction efficiency in the enhancement layer. The error can be reduced.

  Such an inter-layer estimation process has the effect of increasing the correlation with the original video signal. However, when simply applied to spatial interpolation, when the quantization step width in the encoding of the base layer is large, the decoding of the decoded video signal is performed. The quantization error is large, the noise due to coding degradation increases, and the correlation with the original video signal becomes low. Therefore, in the present embodiment, the influence of coding degradation and the characteristics of the inter-layer estimation process are taken into account when generating the predicted video signal between the hierarchies.

  In the present embodiment, an inter-layer estimation process that mainly excels in edge (contour) estimation is employed. Therefore, before the inter-layer estimation process, filtering for preserving edges and removing only small amplitude signals is performed. Through such filtering, the signal with degraded coding is separated into edges (high-frequency components corresponding to the original video signal), which are the purpose of spatial interpolation estimation, and other noise-containing portions, and inter-layer estimation processing is performed. By acting on only the edge, it is possible to derive a high-resolution signal closer to the original video signal from the low-resolution signal that has deteriorated encoding. Here, in order to suppress side-effect signal degradation in filtering that preserves edges, high-pass filtering is performed in the preceding stage, and only the high-frequency components that have passed through are subjected to filtering that preserves edges.

  In addition, since the small-amplitude difference signal separated by the filtering for storing the edge described above includes texture and the like, it is re-synthesized with the edge portion after the edge portion estimation processing (inter-layer estimation processing). Specific configurations of the video layer encoding device and the video layer decoding device for realizing the above-described inter-layer estimation processing are shown below by dividing the embodiments. Further, in this embodiment, in order to facilitate understanding, two-level hierarchical encoding of a base layer and an enhancement layer is given as an example. However, the present invention is not limited to this, and three levels such as three levels, four levels,. Needless to say, it can be applied to multiple hierarchies. Of course, it is possible to arbitrarily select the resolution of the base layer.

(First embodiment: video hierarchy encoding apparatus 100)
FIG. 1 is a functional block diagram illustrating a schematic configuration of the video hierarchical encoding device 100. The video hierarchical encoding apparatus 100 receives an original video signal as an original, and transmits a bit stream subjected to spatial resolution scalability to the video hierarchical decoding apparatus through a communication line or a recording medium such as a Blu-ray disc or a DVD. The video hierarchy encoding apparatus 100 includes a spatial decimation unit 110, a base layer encoding unit 112, a base layer decoding unit 114, a spatial interpolation unit 116, an enhancement layer encoding unit 118, a multiplexing unit 120, It is comprised including.

  The spatial decimation unit 110 generates a basic video signal by reducing the spatial resolution of the original video signal input to the video hierarchical encoding apparatus 100 to a predetermined resolution defined by the basic layer, and generates a basic layer encoding Transmitted to the unit 112. For resolution reduction (spatial decimation), various existing resolution reduction methods can be applied, but a resolution reduction method corresponding to inter-layer estimation processing by a Laplacian pyramid in the spatial interpolation unit 116 described later is applied. It is desirable to do. In addition, a reduction ratio can be set to cope with a change in resolution of the original video signal and the basic video signal. For example, assuming that the resolution of the base layer is RB and the resolution of the enhancement layer (here, the resolution of the original video signal) is RE, the reduction rate when spatially decimating from the resolution of the original video signal to the resolution of the basic layer is RB / RE. It is good to set to.

  The base layer encoding (encoding) unit 112 encodes the basic video signal reduced by the spatial decimation unit 110 to generate basic encoded data (bit stream), and transmits the basic encoded data (bit stream) to the base layer decoding unit 114 and the multiplexing unit 120. To do. The encoding method in the base layer encoding unit 112 is, for example, MPEG2 or H.264. A closed loop encoder such as H.264 is used. Here, functions such as scalability in the time direction and SNR scalability may be included. When an open-loop encoder is used as well as a closed-loop, the encoder includes a decoding (reconstruction) function.

  Base layer decoding section 114 further decodes the base encoded data encoded by base layer encoding section 112 to generate a base decoded video signal.

  The spatial interpolation unit 116 spatially enlarges the spatial resolution of the basic decoded video signal decoded by the base layer decoding unit 114 to generate a predicted video signal. Such a predicted video signal is an estimated signal of the original video signal, and in this embodiment, a predicted video signal having a high correlation with the original video signal is generated by appropriately increasing the resolution of the basic decoded video signal. A detailed configuration of the spatial interpolation unit 116 that is characteristic in the present embodiment will be described later. The spatial interpolation unit 116 transmits the generated predicted video signal to the enhancement layer encoding unit 118 and transmits the encoding parameters obtained by encoding the parameters used for the spatial interpolation to the multiplexing unit 120. To do.

  The enhancement layer encoding unit 118 uses the original original video signal and the predicted video signal from the spatial interpolation unit 116, derives a prediction error signal using spatial correlation and temporal correlation, and generates the prediction error. The signal is encoded to generate extended encoded data (bit stream). A detailed configuration of the enhancement layer encoding unit 118 will be described later. The enhancement layer encoding unit 118 transmits the generated extension encoded data to the multiplexing unit 120.

  The multiplexing unit 120 receives the basic encoded data from the base layer encoding unit 112, the encoding parameters from the spatial interpolation unit 116, and the extended encoded data from the enhancement layer encoding unit 118, Multiplexed to generate one multiplexed data (bit stream) and output to the communication line 130 or the medium (storage medium) 132.

  FIG. 2 is a flowchart showing a specific process of a video hierarchical encoding method for spatially scaling an original original video signal by the video hierarchical encoding device 100 described above.

  First, the spatial decimation unit 110 generates a basic video signal by reducing the spatial resolution of the original video signal (S150), and the basic layer encoding unit 112 encodes the generated basic video signal, A bit stream as basic encoded data is generated (S152). The generated bitstream is transmitted to the multiplexing unit 120 and also to the base layer decoding unit 114.

  The base layer decoding unit 114 decodes the base encoded data to generate a base decoded video signal (S154), and the spatial interpolation unit 116 determines the spatial resolution of the base decoded video signal through the high-frequency component inter-layer estimation process. The enlarged video signal is generated (S156). The spatial interpolation unit 116 encodes the parameters used for the spatial interpolation and transmits the encoded parameters to the multiplexing unit 120.

  Then, the enhancement layer encoding unit 118 derives a prediction error signal using the original video signal and the prediction video signal generated by the spatial interpolation unit 116 (S158), encodes the prediction error signal, A bit stream as extended encoded data is generated (S160). The extended encoded data generated in this way is multiplexed together with the basic encoded data and the encoding parameters (S162) and output to the communication line 130 and the medium 132.

(Spatial interpolation unit 116, enhancement layer encoding unit 118)
Next, the spatial interpolation unit 116 and the enhancement layer encoding unit 118 that are characteristic in the present embodiment will be described in detail.

  FIG. 3 is a functional block diagram showing configurations of spatial interpolation section 116 and enhancement layer encoding section 118. The spatial interpolation unit 116 includes a first high-pass filtering unit 208, a high-frequency extraction filtering unit 210, a high-frequency spatial interpolation unit 212, a difference generation unit 214, a difference spatial interpolation unit 216, and a basic interpolation. Configuration unit 218, first signal synthesis unit 220, and first entropy encoding unit 222.

…（数式１）

…（数式２）
ここで、ρは、ガウシアンフィルタの帯域を調整するためのパラメータである。第１ハイパスフィルタリング部２０８は、入力信号から抽出したラプラシアン成分の高周波分離信号を高周波抽出フィルタリング部２１０へ出力する。 A first high-pass filtering unit (high-pass filtering unit) 208 receives the basic decoded video signal decoded by the base layer decoding unit 114 as it is, and extracts a Laplacian component as a high-frequency component. In this way, when encoding degradation can be tolerated, the Laplacian component can be used by directly using the basic decoded video signal decoded by the base layer decoding unit 114 without performing a predetermined filtering process for suppressing the encoding deterioration. By generating, high-quality Laplacian components can be obtained while utilizing more frequency components included in the decoded image. Here, for ease of understanding, a one-dimensional signal model (Equations 1 and 2) is taken as an example, the input signal is G ₀ (x), and the Laplacian component extracted from the input signal is L ₀ (x). To do.

... (Formula 1)

... (Formula 2)
Here, ρ is a parameter for adjusting the band of the Gaussian filter. The first high pass filtering unit 208 outputs the high frequency separation signal of the Laplacian component extracted from the input signal to the high frequency extraction filtering unit 210.

  Here, as an example of the first high-pass filtering unit 208, high-frequency components are extracted using a Gaussian function, but this can be replaced with other methods. However, it is desirable that the relationship between the filter and interpolation function used here and the filter and interpolation function used in the spatial decimation unit 110 and the second high-pass filtering unit 236 described later satisfy the pyramid configuration. For example, when the sinc function is used for the spatial decimation unit 110, the sinc function can be used for the first high-pass filtering unit 208 and the second high-pass filtering unit 236 to build a pyramid configuration relationship based on the sinc function.

  The high-frequency extraction filtering unit 210 includes, for example, an ε-filter that is a non-linear digital filter that effectively removes noise from a video signal having a sudden change, and the first high-pass filtering unit 208 separates high-frequency components. A high frequency component corresponding to the original video signal is extracted from the separated signal to generate a high frequency equivalent signal.

…（数式３）

…（数式４）
ここで、ｙ（）はε−フィルタの出力、ｘ（）はε−フィルタに対する入力信号、ａ_ｋはフィルタの重みを決める係数列、εはエッジを決める閾値である。ａ_ｋおよびεは、ε−フィルタの特性を決めるパラメータとなり、任意に設定することができる。 The high-frequency component inter-layer estimation processing by the high-frequency spatial interpolation unit 212 described later is mainly excellent in the ability to estimate an edge (contour) in a high-resolution signal. On the other hand, as described above, the ε-filter is a filter that preserves edges and removes small amplitudes. Therefore, by providing an ε-filter for the basic decoded video signal that has been deteriorated in coding, it is possible to separate the edge portion that is the estimation purpose and the components other than the edge that should not be emphasized, including the deterioration in coding. Thus, the inter-layer estimation process can be applied only to the edge portion. The one-dimensional ε-filter is given by the following mathematical formulas 3 and 4, and performs processing for both the horizontal and vertical directions of the signal.

... (Formula 3)

... (Formula 4)
Here, y () is the output of the ε-filter, x () is an input signal to the ε-filter, a _k is a coefficient sequence that determines the weight of the filter, and ε is a threshold that determines the edge. a _k and ε are parameters that determine the characteristics of the ε-filter and can be set arbitrarily.

The high frequency extraction filtering unit 210 outputs the signal subjected to the ε-filter to the high frequency spatial interpolation unit 212 and the difference generation unit 214, and transmits the parameters _ak and ε to the first entropy encoding unit 222. Thus, the parameters a _k and ε are also included in the final multiplexed data. By including the parameters _ak and ε in the final multiplexed data, appropriate video layer decoding using the same ε-filter becomes possible in the video layer decoding device.

  The high frequency spatial interpolation unit 212 generates a high frequency component high resolution signal by expanding the spatial resolution of the high frequency equivalent signal through inter-layer estimation processing of the high frequency component. Specifically, the high-frequency spatial interpolation unit 212 includes a first interpolation unit 232, an amplitude limit constant multiplication unit 234, and a second high-pass filtering unit 236.

…（数式５）

…（数式６）

…（数式７）
で与えられる。ここでｉｎｔ（）は整数部分を取り出す関数である。第１インターポレーション部２３２は、インターポレーションした信号を振幅制限定数倍処理部２３４へ出力する。ここで倍率γは、例えば、基本レイヤの解像度をＲＢ、拡張レイヤの解像度をＲＥとすると、基本レイヤの解像度から拡張レイヤの解像度に空間インターポレーションする場合には、ＲＥ／ＲＢの値に設定されるとよい。 The first interpolation unit 232 receives the high-frequency equivalent signal output from the high-frequency extraction filtering unit 210 and performs interpolation at a magnification that provides a desired resolution. The desired magnification interpolation is performed as follows. The signal (EXPAND) _γ L ₀ (x) interpolated to the magnification γ is expressed by Equations 5, 6, and 7 assuming that the input Laplacian component is L ⁰ (x).

... (Formula 5)

... (Formula 6)

... (Formula 7)
Given in. Here, int () is a function for extracting an integer part. The first interpolation unit 232 outputs the interpolated signal to the amplitude limit constant multiplication unit 234. Here, for example, when the base layer resolution is RB and the enhancement layer resolution is RE, the magnification γ is set to the RE / RB value when spatial interpolation is performed from the base layer resolution to the enhancement layer resolution. It is good to be done.

…（数式８）
で与えられる。ここで、振幅制限のためのパラメータＴおよび定数倍処理のためのパラメータα_γは、試行を繰り返すことにより実験的に求められる。なお、パラメータα_γは、拡大率に応じて可変であるが、本実施形態では、拡大率だけではなく、基本レイヤの量子化の程度にも推定精度が関わるため、量子化の程度に応じても決定される。例えば、図４で示されるように、パラメータα_γと量子化ステップとの関係を決定するための変換式、もしくは変換テーブルを予め準備しておき、量子化ステップの変動に応じてパラメータα_γの値を制御できるようにすることが望ましい。振幅制限定数倍処理部２３４は、振幅制限および定数倍処理した信号を第２ハイパスフィルタリング部２３６へ出力し、当該振幅制限および定数倍処理に用いたパラメータα_γおよびＴは第１エントロピー符号化部２２２に伝達される。 The amplitude limit constant multiplication processing unit 234 receives the signal from the first interpolation unit 232 and performs the following steps for estimating an unknown high-frequency component. The process for estimating an unknown high-frequency component is realized by performing amplitude limitation and constant multiplication processing on the signal input from the first interpolation unit 232. The signal L _γ (bar ￣) (x) generated by the amplitude limit constant multiplication unit 234 is expressed by Equation 8 assuming that the input signal is (EXPAND) _γ L ₀ (x).

... (Formula 8)
Given in. Here, the parameter alpha _gamma for parameter T and the constant magnification process for amplitude limiting, experimentally obtained by repeating the attempt. Note that the parameter α _γ is variable according to the enlargement ratio, but in this embodiment, the estimation accuracy is related not only to the enlargement ratio but also to the degree of quantization of the base layer. Is also determined. For example, as shown in FIG. 4, a conversion equation or conversion table for determining the relationship between the parameter α _γ and the quantization step is prepared in advance, and the parameter α _γ is changed according to the variation of the quantization step. It is desirable to be able to control the value. The amplitude limit constant multiplication unit 234 outputs the signal subjected to the amplitude restriction and constant multiplication processing to the second high-pass filtering unit 236, and the parameters α _γ and T used for the amplitude restriction and constant multiplication processing are the first entropy coding unit. 222 is transmitted.

…（数式９）
で与えられる。ここで、Ｗ（ｉ）は、数式２に示したものである。ここでも、数式９以外の方法を用いて高周波成分を抽出することができるが、ピラミッド構成を満たすものが望ましい。第２ハイパスフィルタリング部２３６は、推定された高周波成分を第１信号合成部２２０へ出力する。 The second high-pass filtering unit 236 receives the signal from the amplitude limit constant multiplication unit 234 and performs another process for estimating an unknown high-frequency component. Another process for estimating the unknown high-frequency component is to remove the low-frequency band component from the input signal and obtain only the high-frequency component originally desired. This is realized by performing high-pass filtering on the input signal. The high-pass filtered signal, that is, the estimated unknown high-frequency component Lγ (hat ^) (x) is expressed by Equation 9 when the input signal is L _γ (bar ￣) (x).

... (Formula 9)
Given in. Here, W (i) is shown in Formula 2. Again, high-frequency components can be extracted using methods other than Equation 9, but those satisfying the pyramid configuration are desirable. The second high-pass filtering unit 236 outputs the estimated high frequency component to the first signal synthesis unit 220.

  The difference generation unit 214 generates a difference signal that is a difference between the high-frequency separated signal output from the first high-pass filtering unit 208 and the high-frequency equivalent signal extracted by the high-frequency extraction filtering unit 210 to generate a difference space interpolation unit 216. To communicate.

  The differential spatial interpolation unit 216 generates a differential high resolution signal by expanding the spatial resolution of the differential signal from the differential generation unit 214 to the same resolution as the original video signal, and transmits the differential high resolution signal to the first signal synthesis unit 220. Such spatial enlargement can be realized using Equations 5 to 7 described above, but the filter coefficients and interpolation functions used for enlargement are not limited to Equations 5 to 7.

…（数式１０）
で与えられる。ここで、Ｗ_γ（ｉ）は数式６および７で示したものである。また、基本インターポレーション部２１８による空間的拡大は、上述した数式１０に限らずフィルタ係数や補間関数等が相違する他の拡大方法を用いることもできる。また、倍率γは、例えば、基本レイヤの解像度をＲＢ、拡張レイヤの解像度をＲＥとすると、基本レイヤの解像度から拡張レイヤの解像度に空間インターポレーションする場合には、ＲＥ／ＲＢの値に設定されるとよい。基本インターポレーション部２１８は、拡大した基本高解像度信号を第１信号合成部２２０へ出力する。 The basic interpolation unit 218 receives the basic decoded video signal decoded by the base layer decoding unit 114 as it is, and performs interpolation so as to generate a basic high resolution signal so that the signal has a desired resolution. . As described above, the basic high-resolution signal is generated by using the basic decoded video signal decoded by the base layer decoding unit 114 as it is without using the predetermined filtering process for suppressing the encoding deterioration. This is a case where encoding degradation of the basic decoded video signal can be tolerated. In such a case, a high-quality high-resolution signal can be obtained while using more frequency components included in the decoded image. The desired magnification interpolation is performed as follows. Assuming that the signal (EXPAND) _γ G ₀ (x) interpolated to the magnification γ is given by Equation 10

... (Formula 10)
Given in. Here, W _γ (i) is represented by Equations 6 and 7. Further, the spatial enlargement by the basic interpolation unit 218 is not limited to the above-described Expression 10, and other enlargement methods having different filter coefficients, interpolation functions, and the like can also be used. Further, for example, when the base layer resolution is RB and the enhancement layer resolution is RE, the scaling factor γ is set to the value of RE / RB when spatial interpolation is performed from the base layer resolution to the enhancement layer resolution. It is good to be done. The basic interpolation unit 218 outputs the enlarged basic high resolution signal to the first signal synthesis unit 220.

  The first signal synthesis unit (signal synthesis unit) 220 receives the high-frequency component high-resolution signal from the second high-pass filtering unit 236, the differential high-resolution signal from the differential spatial interpolation unit 216, and the basic interpolation unit 218. Are combined with the basic high resolution signal to generate a predicted video signal.

The first entropy encoding unit 222 entropy-encodes the parameters a _k and ε from the high-frequency extraction filtering unit 210 and the parameters α _γ and T from the amplitude limit constant multiplication processing unit 234 to generate a bit stream as an encoding parameter. And output to the multiplexing unit 120.

  In this way, the first signal synthesis unit 220 of the spatial interpolation unit 116 generates a predicted video signal, and the enhancement layer encoding unit 118 uses the original original video signal and the predicted video signal to obtain a spatial resolution. A prediction error signal is derived using the correlation between time and time, and the prediction error signal is encoded to generate extended encoded data (bit stream). The enhancement layer encoding unit 118 includes a first frame memory 250, a second frame memory 252, a motion estimation unit 254, a motion compensation unit 256, an intra prediction unit 258, a prediction signal selection unit 260, and a prediction error signal. A configuration including a generation unit 262, an orthogonal transform quantization unit 264, a second entropy encoding unit 266, an inverse quantization inverse orthogonal transform unit 268, a second signal synthesis unit 270, and a deblocking filter unit 272 Is done.

  The first frame memory 250 stores at least one GOP (Group Of Picture) signal of the original video signal. Then, a frame signal corresponding to the prediction error signal generation unit 262 and the motion estimation unit 254 is output so that the spatial interpolation unit 116 and the enhancement layer encoding unit 118 can be synchronized.

  The second frame memory 252 stores at least one frame of the signal from the deblocking filter unit 272, and transmits a frame signal necessary for motion estimation to the motion estimation unit 254 and a frame signal necessary for motion compensation to the motion compensation unit 256. Output.

  The motion estimation unit 254 receives the frame signals from the first frame memory 250 and the second frame memory 252, Motion estimation represented by H.264 is performed. The motion information obtained by the motion estimation is output to the motion compensation unit 256 and the second entropy coding unit 266.

  The motion compensation unit 256 receives the frame signal from the second frame memory 252 and the motion information from the motion estimation unit 254, and Motion compensation represented by H.264 is performed. The signal obtained by the motion compensation is output to the prediction signal selection unit 260.

  The intra prediction unit 258 receives the signal synthesized by the second signal synthesis unit 270 and receives the H.264 signal. Intra prediction represented by H.264 is performed. A signal obtained by intra prediction is output to the prediction signal selection unit 260.

  The prediction signal selection unit 260 receives signals from the motion compensation unit 256, the intra prediction unit 258, and the spatial interpolation unit 116, and selects one of them, or weights and combines the signals. Determination criteria in signal selection and signal synthesis can be arbitrarily determined. For example, when importance is placed on coding efficiency, signals are selected or synthesized such that the mean square of the prediction error signal is reduced. The prediction signal selection unit 260 outputs the selected or synthesized signal to the prediction error signal generation unit 262 and the second signal synthesis unit 270.

  The prediction error signal generation unit 262 generates a prediction error signal by subtracting the prediction video signal input via the prediction signal selection unit 260 from the original video signal of the first frame memory 250, and outputs the prediction error signal to the orthogonal transform quantization unit 264. Output.

  The orthogonal transform quantization unit 264 receives the prediction error signal output from the prediction error signal generation unit 262, and orthogonally transforms and quantizes the prediction error signal. For the orthogonal transform, discrete cosine transform (DCT), wavelet, or the like is used. H. As in the case of H.264, it is possible to adopt a means that combines orthogonal transformation and quantization. Orthogonal transform quantization section 264 outputs the orthogonal transformed and quantized signal to second entropy coding section 266 and inverse quantization inverse orthogonal transform section 268.

  The second entropy encoding unit 266 performs entropy encoding on the signal from the orthogonal transform quantization unit 264 to generate a bit stream as extended encoded data. The extension encoded data generated in this way is transmitted to the multiplexing unit 120.

  The inverse quantization inverse orthogonal transform unit 268 receives the orthogonally transformed and quantized signal, and performs inverse quantization and inverse orthogonal transform on the signal again to generate an estimation signal. Then, the estimated signal is transmitted to the second signal synthesis unit 270.

  The second signal synthesis unit 270 synthesizes the prediction video signal from the prediction signal selection unit 260 and the estimation signal from the inverse quantization inverse orthogonal transformation unit 268, and the result is an intra prediction unit 258 and a deblocking filter unit 272. Output to.

  The deblocking filter unit 272 receives the signal from the second signal synthesis unit 270 and performs deblocking filter processing on the input signal. The deblocking filter used here is, for example, H.264. The deblocking filter used in H.264 can be used. The deblocking filtered signal is transmitted to the second frame memory 252.

  FIG. 5 is a flowchart showing specific processing of the video hierarchical encoding method performed by the spatial interpolation unit 116 described above.

  First, the first high-pass filtering unit 208 generates a high-frequency separation signal by separating high-frequency components of the basic decoded video signal decoded by the base layer decoding unit 114 (S300), and the high-frequency extraction filtering unit 210 uses an ε-filter. The high frequency separation signal is subjected to ε-filter processing to extract a high frequency equivalent signal (S302). The high-frequency spatial interpolation unit 212 expands the spatial resolution of the high-frequency equivalent signal generated in this way through high-frequency component inter-layer estimation processing to generate a high-frequency component high-resolution signal. Specifically, the first interpolation unit 232 expands the high-frequency equivalent signal extracted by the high-frequency extraction filtering unit 210 (S304), and the amplitude limit and constant multiplication are performed on the signal using the amplitude limit constant multiplication unit 234. Processing is performed (S306), and the second high-pass filtering unit 236 extracts a high-frequency component from the signal subjected to amplitude limitation and constant multiplication, and generates a high-frequency component high-resolution signal (S308).

  On the other hand, the difference generation unit 214 generates a difference signal that is the difference between the high-frequency separation signal and the high-frequency equivalent signal (S310), and the difference space interpolation unit 216 expands the spatial resolution of the difference signal to increase the difference high resolution. A signal is generated (S312). In addition, the spatial resolution of the basic decoded video signal decoded by the basic layer decoding unit 114 by the basic interpolation unit 218 is separately expanded as it is to generate a basic high resolution signal (S314). Here, the processes a, b, and c shown in FIG. 5 may be performed in any order, and may be performed in parallel as long as the processing capability permits.

Then, the first signal synthesis unit 220 receives the high-frequency component high-resolution signal from the second high-pass filtering unit 236, the differential high-resolution signal from the differential spatial interpolation unit 216, and the basic high-frequency from the basic interpolation unit 218. The predicted video signal is generated by combining the resolution signal (S316). Further, if it is necessary to encode the parameters α _γ and T used in the high frequency extraction filtering unit 210 and the amplitude limit constant multiplication unit 234, the first entropy encoding unit 222 encodes them (S318). The reason why it is necessary here is that there is no need for encoding if the parameters are determined in advance between the video layer encoding device 100 and the video layer decoding device.

  FIG. 6 is a flowchart showing specific processing of the video layer encoding method performed by the enhancement layer encoding unit 118 described above.

  First, intra prediction is performed using the intra prediction unit 258, and the intra-predicted signal is transmitted to the prediction signal selection unit 260 (S350). On the other hand, motion estimation and motion compensation (motion compensation prediction) are performed using the motion estimation unit 254 and the motion compensation unit 256, and a motion compensation predicted signal is also transmitted to the prediction signal selection unit 260 (S352). Similarly, the predicted video signal predicted by the spatial interpolation unit 116 is also transmitted to the predicted signal selection unit 260 (S354). Here, the processes a, b, and c shown in FIG. 6 may be performed in any order, and may be performed in parallel as long as the processing capability permits.

  The prediction signal selection unit 260 selects any one of the intra-predicted signal, the motion-compensated prediction signal, and the high-resolution estimation signal, or synthesizes each signal by weighting (S356). The predicted video signal is subtracted from the original video signal from the first frame memory 250 to generate a prediction error signal (S358). Next, orthogonal transformation and quantization are performed on the prediction error signal using the orthogonal transformation quantization unit 264 (S360). The second entropy encoding unit 266 performs entropy encoding on the orthogonally transformed and quantized signal and motion information (S362).

  Here, it is determined whether or not all signals to be encoded have been encoded (S364), and if encoded, the process ends here. If all are not encoded, decoding and deblocking processing is performed to refer to the currently encoded signal when other signals are encoded. Specifically, the orthogonally transformed and quantized signal is inversely quantized and inversely orthogonally transformed by the inversely quantized inversely orthogonal transform unit 268 (S366), and the second signal synthesis unit is applied to the inversely quantized and inversely orthogonally transformed signal. 270 is combined with the predicted video signal to generate a decoded signal, and the decoded signal is transmitted to the intra prediction unit 258 and the deblocking filter unit 272 (S368). Finally, the signal subjected to the deblocking filter process in the deblocking filter unit 272 is stored in the second frame memory 252 (S370), and the process is repeated from the beginning.

  As described above, according to the video hierarchy coding apparatus 100 described above, an inter-layer estimation process involving estimation of an original video signal in simple spatial interpolation (spatial expansion) in inter-layer prediction of conventional video hierarchy coding. In addition, by reducing the prediction error signal between layers, it is possible to efficiently realize higher-quality video layer coding. In addition, in spatial interpolation, a video signal obtained by decoding basic encoded data is taken into account in consideration of encoding degradation of the basic decoded video signal (low resolution signal) obtained by decoding the basic encoded data and characteristics of inter-layer estimation processing. Since the high-frequency component corresponding to the original video signal to be estimated and the other signals are separated and the original video signal (high resolution signal) is estimated by processing suitable for each, the predicted video signal is generated more appropriately. Therefore, efficient video hierarchical coding can be realized.

(Second Embodiment: Video Hierarchy Decoding Device 400)
FIG. 7 is a functional block diagram illustrating a schematic configuration of the video hierarchy decoding apparatus 400. The video hierarchy decoding apparatus 400 extracts necessary information from the multiplexed data generated by the video hierarchy encoding apparatus 100 and outputs a decoded video signal having a spatial resolution suitable for the performance of a display or the like. Video hierarchy decoding apparatus 400 includes an extractor 410, a base layer decoder 412, a spatial interpolation unit 414, and an enhancement layer decoder 416.

  The extractor 410 receives the multiplexed data (bitstream) generated by the video layer encoding device 100 and matches the performance of the video layer decoding device 400 or the performance of a display that receives the video output. Then, data necessary for decoding is cut out from the entire multiplexed data and separated. Here, the basic encoded data is divided into the base layer decoding unit 412, the extended encoded data is divided into the enhancement layer decoding unit 416, and if an encoding parameter is given, the encoding parameter is divided into the spatial interpolation unit 414. To transmit.

  The base layer decoding unit 412 decodes the basic encoded data extracted by the extractor 410 to generate a basic decoded video signal (low resolution video output). Such low-resolution video output is output to the display 430 as necessary. Such decoding includes MPEG-2 and H.264. Technologies such as H.264 can be applied. Also, scalability in the time direction, SNR scalability, and the like may be combined.

  Similar to the spatial interpolation unit 116 in the first embodiment, the spatial interpolation unit 414 spatially expands the spatial resolution of the basic decoded video signal decoded by the base layer decoding unit 412 to generate a predicted video signal. . However, the spatial interpolation unit 116 in the first embodiment outputs the parameters used in the spatial interpolation unit 116, but in the present embodiment, a predicted video signal is generated using the output parameters. Is different.

  Such a predicted video signal is an estimated signal of the original video signal, and also in the present embodiment, a predicted video signal having a high correlation with the original video signal is generated by appropriately increasing the resolution of the basic decoded video signal. A detailed configuration of the spatial interpolation unit 414 characteristic in the present embodiment will be described later. The spatial interpolation unit 414 transmits the generated predicted video signal to the enhancement layer decoding unit 416.

  The enhancement layer decoding unit 416 decodes the extension encoded data cut out by the extract unit 410, and uses the decoded prediction error signal and the predicted video signal predicted by the spatial interpolation unit 414 to generate the original video signal. Restore (high resolution video output). Such high-resolution video output is output to the display 432 as necessary. A detailed configuration of the enhancement layer decoding unit 416 will be described later.

  FIG. 8 is a flowchart showing specific processing of the video hierarchy decoding method for restoring the original original video signal by the video hierarchy decoding apparatus 400 described above.

  First, the extract unit 410 analyzes the multiplexed data generated by the video layer encoding device 100, separates the data into basic encoded data, extended encoded data, and encoding parameters, and each data is a base layer decoding unit. 412, the enhancement layer decoding unit 416 and the spatial interpolation unit 414 (S450).

  The base layer decoding unit 412 decodes the base encoded data to generate a base decoded video signal (S452). If the display supports only low resolution, this basic decoded video signal is displayed. The spatial interpolation unit 414 uses the parameters from the basic decoded video signal and the extractor 410 to expand the spatial resolution of the basic decoded video signal through high-frequency component inter-layer estimation processing to generate a predicted video signal (S454). ).

  Then, enhancement layer decoding section 416 reconstructs the original video signal using the prediction error signal based on the extension encoded data from extract section 410 and the predicted video signal from spatial interpolation section 414 (S456). In this way, a plurality of video outputs (basic decoded video signal and original video signal) having different levels are output.

(Spatial interpolation unit 414, enhancement layer decoding unit 416)
Next, the spatial interpolation unit 414 and the enhancement layer decoding unit 416 that are characteristic in the present embodiment will be described in detail.

  FIG. 9 is a functional block diagram showing configurations of the spatial interpolation unit 414 and the enhancement layer decoding unit 416. The spatial interpolation unit 414 includes a first high-pass filtering unit 208, a high frequency extraction filtering unit 510, a high frequency spatial interpolation unit 512, a difference generation unit 214, a differential space interpolation unit 216, and a basic interpolation. Configuration unit 218, first signal synthesis unit 220, and first entropy decoding unit 522. Also, the enhancement layer decoding unit 416 includes a second frame memory 252, a motion compensation unit 256, an intra prediction unit 258, a prediction signal selection unit 260, a second entropy decoding unit 566, and an inverse quantization inverse orthogonal transform unit. 268, the 2nd signal synthetic | combination part 270, and the deblocking filter part 572 are comprised.

  Here, the first high-pass filtering unit 208, the difference generation unit 214, the difference space interpolation unit 216, the basic interpolation unit 218, the first signal synthesis, which have already been described as the constituent elements in the first embodiment. Unit 220, second frame memory 252, motion compensation unit 256, intra prediction unit 258, prediction signal selection unit 260, inverse quantization inverse orthogonal transform unit 268, and second signal synthesis unit 270 are substantially Since the functions are the same, redundant description is omitted. Here, the high-frequency extraction filtering unit 510, the high-frequency spatial interpolation unit 512, the first entropy decoding unit 522, and the second entropy decoding unit 566, which have different configurations, The deblocking filter unit 572 will be mainly described.

  The high-frequency extraction filtering unit 510 includes, for example, an ε-filter that is a non-linear digital filter that effectively removes noise from a video signal having a sudden change, like the high-frequency extraction filtering unit 210 in the first embodiment. The high frequency component corresponding to the original video signal is extracted from the high frequency separated signal from which the high frequency component has been separated by the high pass filtering unit 208 to generate a high frequency equivalent signal.

In the first embodiment, the parameters a _k and ε that determine the characteristics of the ε-filter are transmitted to the first entropy encoding unit 222. In the present embodiment, the parameters a _k and ε are received and ε− The difference is that it is set in the filter. With such a configuration, filtering can be performed using the same parameters as in video hierarchical encoding, so that the same high-frequency equivalent signal as in video hierarchical encoding can be generated.

The high frequency spatial interpolation unit 512 includes a first interpolation unit 232, an amplitude limit constant multiplication unit 534, and a second high-pass filtering unit 236. The first interpolation unit 232 and the second high-pass filtering unit 236, which have already been described as components in the first embodiment, have substantially the same function. For the amplitude system limited number multiple processor 534 configured to differences had transmitted the parameter alpha _gamma and T to the first entropy encoding unit 222 in the first embodiment, in this embodiment, and its parameters alpha _gamma The only difference is that T is set as its own parameter.

The first entropy decoding unit 522 decodes the parameters a _k , ε, α _γ , T and the like from the encoding parameters, converts the parameters a _k and ε to the high frequency extraction filtering unit 510, and sets the parameters α _γ and T to the high frequency spatial interpolator. Output to the amplitude limit constant multiplication processing unit 534 of the modulation unit 512.

  The second entropy decoding unit 566 decodes the extended encoded data from the extract unit 410 to generate a prediction error signal. The prediction error signal generated in this way is transmitted to the inverse quantization inverse orthogonal transform unit 268, and the motion information is transmitted to the motion compensation unit 256.

  The deblocking filter unit 572 receives the signal from the second signal synthesis unit 270 and performs deblocking filter processing on the input signal. The deblocking filter used here is, for example, H.264. The deblocking filter used in H.264 can be used. The signal subjected to the deblocking filter process is transmitted to the second frame memory 252 and output to the outside as an original video signal.

  FIG. 10 is a flowchart showing specific processing of the video layer decoding method performed by the enhancement layer decoding unit 416 described above.

  First, the second entropy decoding unit 566 decodes the extended encoded data to generate a prediction error signal (S600), and the prediction error signal is subjected to inverse quantization and inverse orthogonal transform in the inverse quantization inverse orthogonal transform unit 268. (S602).

  Subsequently, it is analyzed whether the target block (macroblock) has been selected or synthesized from intra prediction, motion compensated prediction, and prediction based on the predicted video signal (S604), and processing corresponding to that is analyzed. Do. When intra prediction is selected, intra prediction is performed using the intra prediction unit 258 (S606). On the other hand, when motion compensation prediction is selected, motion compensation is performed using the motion compensation unit 256 (S608). If prediction based on the predicted video signal is selected, the spatial interpolation unit 414 is caused to function to generate a predicted video signal (S610). Detailed processing operations for generating the predicted video signal will be described later. Further, when the respective signals are combined, the above-described intra prediction, motion compensation prediction, and prediction based on the predicted video signal are all performed, weighted, and then combined.

  The prediction signal selection unit 260 combines the selected or synthesized prediction video signal with the signal from the inverse quantization inverse orthogonal transform unit 268 to generate a decoded signal, and the decoded signal is converted into the intra prediction unit 258 and the deblocking filter. The data is transmitted to the unit 572 (S612). Finally, the signal subjected to the deblocking filter processing in the deblocking filter unit 272 is output to the outside as an original video signal (S614). Here, it is determined whether or not all of the extended encoded data to be decoded has been decoded (S616), and if it has been decoded, the process ends here. If not encoded, the process is repeated from the beginning.

  FIG. 11 is a flowchart showing specific processing of the video hierarchy decoding method by the spatial interpolation unit 414 described above.

First, the first entropy decoding unit 522 decodes the parameters a _k , ε, α _γ , T from the encoding parameters, if necessary, the parameters a _k and ε to the high frequency extraction filtering unit 510, and the parameters α _γ and T is output to the amplitude limit constant multiplication unit 534 of the high-frequency spatial interpolation unit 512 (S650). The reason why it is necessary here is that it is not necessary to perform decoding if a parameter is determined in advance between the video hierarchy coding apparatus 100 and the video hierarchy decoding apparatus 400.

  Subsequently, the first high-pass filtering unit 208 generates a high-frequency separated signal by separating the high-frequency component of the basic decoded video signal decoded by the base layer decoding unit 412 (S652), and the high-frequency extraction filtering unit 510 performs an ε-filter. The high frequency separation signal is subjected to ε-filter processing to generate a high frequency equivalent signal (S654). The high frequency spatial interpolation unit 512 generates a high frequency component high resolution signal by expanding the spatial resolution of the high frequency equivalent signal generated in this way through high frequency component inter-layer estimation processing. Specifically, the first interpolation unit 232 expands the high frequency equivalent signal extracted by the high frequency extraction filtering unit 510 (S656), and the amplitude limit and constant multiplication are performed on the signal using the amplitude limit constant multiplication unit 534. Processing is performed (S658), and the second high-pass filtering unit 236 extracts a high frequency component from the signal subjected to the amplitude limitation and constant multiplication processing to generate a high frequency component high resolution signal (S660).

  On the other hand, the difference generation unit 214 generates a difference signal that is a difference between the high-frequency separation signal and the high-frequency equivalent signal (S662), and the difference space interpolation unit 216 expands the spatial resolution of the difference signal to increase the difference high resolution. A signal is generated (S664). In addition, the spatial resolution of the basic decoded video signal decoded by the basic layer decoding unit 412 by the basic interpolation unit 218 is separately expanded as it is to generate a basic high resolution signal (S666). Here, the processes a, b, and c shown in FIG. 11 may be performed in any order, and may be performed in parallel as long as the processing capability permits.

  Then, the first signal synthesis unit 220 receives the high-frequency component high-resolution signal from the second high-pass filtering unit 236, the differential high-resolution signal from the differential spatial interpolation unit 216, and the basic high-frequency from the basic interpolation unit 218. The predicted video signal is generated by combining the resolution signal (S668).

  As described above, according to the video hierarchy decoding apparatus 400 described above, inter-layer estimation processing accompanied with estimation of the original video signal is added to simple spatial interpolation (spatial expansion) in inter-layer prediction of conventional video hierarchy decoding. Thus, by reducing the prediction error signal between layers, it is possible to efficiently realize higher-quality video layer decoding. In addition, in spatial interpolation, a video signal obtained by decoding basic encoded data in consideration of characteristics of basic encoded data and characteristics of inter-layer estimation processing is converted into a high-frequency signal corresponding to the original video signal to be estimated. Since the original video signal (high resolution signal) is estimated by separating the component and other signals and processing appropriate for each, it is possible to generate the predicted video signal more appropriately and realize efficient video hierarchy decoding It becomes possible to do.

  FIG. 12 is a functional block diagram showing a typical example of a computer (information processing apparatus) 700 capable of video hierarchy coding by the video hierarchy coding apparatus 100 and video hierarchy decoding by the video hierarchy decoding apparatus 400 in the present embodiment. . The computer 700 includes a central processing unit 710, a temporary storage device 712, an external storage device 714, a communication unit 716, an input unit 718, and an output unit 720.

  A central processing unit (CPU) 710 manages and controls the entire computer 700 using programs and applications in the temporary storage device 712 and the external storage device 714. The temporary storage device 712 includes a ROM, a RAM, an EEPROM, a nonvolatile RAM, and the like, and temporarily stores a program processed by the central processing unit 710, video data, and the like. The external storage device 714 includes a flash memory, an HDD, and the like, and stores programs processed by the central processing unit 710, video data, and the like. The communication unit 716 is connected to various electronic devices and servers via the communication line 130, and can transmit / receive video data to / from them. The input unit 718 can input multiplexed data and various parameters from the medium 132. The output unit 720 can output video output to the display 432 or the like.

  The video hierarchy coding and video hierarchy decoding described above are performed by the central processing unit 710 executing a program. Therefore, at the same time that the video hierarchy encoding device 100 and the video hierarchy decoding device 400 are provided, a program that causes the computer 700 to function as the video hierarchy encoding device 100 and the video hierarchy decoding device 400 is also provided. The program may be read from a recording medium and loaded into a computer, or may be transmitted via the communication line 130 and loaded into the computer.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  Note that each step in the video hierarchical encoding method and video hierarchical decoding method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine. .

  The present invention relates to a video hierarchical encoding device, a video hierarchical encoding method, a video hierarchical encoding program, a video hierarchical decoding device, and a video hierarchical decoding that generate or use multiplexed data that can be restored to a plurality of video signals having different spatial resolutions. This method can be used for a method and a video hierarchical decoding program.

It is the functional block diagram which showed the schematic structure of the image | video hierarchy coding apparatus. It is the flowchart which showed the specific process of the image | video hierarchy encoding method. It is the functional block diagram which showed the structure of the space interpolation part and the enhancement layer encoding part. It is the conceptual diagram which showed the relationship with a quantization parameter. It is the flowchart which showed the specific process of the image | video hierarchy encoding method. It is the flowchart which showed the specific process of the image | video hierarchy encoding method. It is the functional block diagram which showed the schematic structure of the image | video hierarchy decoding apparatus. It is the flowchart which showed the specific process of the image | video hierarchy decoding method. It is the functional block diagram which showed the structure of the spatial interpolation part and the enhancement layer decoding part. It is the flowchart which showed the specific process of the image | video hierarchy decoding method. It is the flowchart which showed the specific process of the image | video hierarchy decoding method. And FIG. 11 is a functional block diagram showing a typical example of a computer capable of video hierarchy coding and video hierarchy decoding.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Video | video hierarchy coding apparatus 110 ... Spatial decimation part 112 ... Base layer encoding part 114, 412 ... Base layer decoding part 116, 414 ... Spatial interpolation part 118 ... Enhancement layer encoding part 120 ... Multiplexing part 208 ... First high-pass filtering unit (high-pass filtering unit)
210, 510 ... high-frequency extraction filtering units 212, 512 ... high-frequency spatial interpolation unit 214 ... difference generation unit 216 ... difference spatial interpolation unit 218 ... basic interpolation unit 220 ... first signal synthesis unit (signal synthesis unit)
400 ... video hierarchy decoding apparatus 410 ... extract unit 416 ... enhancement layer decoding unit

Claims (8)

A video hierarchical encoding device that generates multiplexed data that can be restored to a plurality of video signals having different spatial resolutions,
A spatial decimation unit that generates a basic video signal by reducing the spatial resolution of the original video signal;
A base layer encoding unit that encodes the basic video signal to generate basic encoded data;
A base layer decoding unit that decodes the basic encoded data to generate a basic decoded video signal;
A spatial interpolation unit that generates a predicted video signal by expanding a spatial resolution of the basic decoded video signal;
An enhancement layer encoding unit that encodes a prediction error signal derived using the original video signal and the predicted video signal to generate extended encoded data;
A multiplexing unit that multiplexes the basic encoded data and the extended encoded data to generate the multiplexed data;
With
The spatial interpolation unit is
A high-pass filtering unit for separating a high-frequency component from the basic decoded video signal to generate a high-frequency separated signal;
A high frequency extraction filtering unit that extracts a high frequency component corresponding to the original video signal from the high frequency separation signal to generate a high frequency equivalent signal;
A high-frequency spatial interpolation unit that expands the spatial resolution of the high-frequency equivalent signal through high-frequency component inter-layer estimation processing and generates a high-frequency component high-resolution signal;
A difference generation unit that generates a difference signal that is a difference between the high-frequency separation signal and the high-frequency equivalent signal;
A differential spatial interpolation unit that generates a differential high resolution signal by enlarging the spatial resolution of the differential signal;
A basic interpolation unit for generating a basic high-resolution signal by expanding a spatial resolution of the basic decoded video signal;
A signal synthesizer that synthesizes the high-frequency component high-resolution signal, the differential high-resolution signal, and the basic high-resolution signal to generate the predicted video signal;
A video hierarchical encoding device comprising: The high-frequency extraction filtering unit is an ε-filter that is a non-linear digital filter that performs noise removal on a video signal having a sudden change,
The video hierarchy encoding apparatus according to claim 1, wherein the multiplexing unit also multiplexes a coefficient sequence _ak and a threshold value ε which are parameters of the ε-filter. A video hierarchical encoding method for generating multiplexed data that can be restored to a plurality of video signals having different spatial resolutions,
Generate the basic video signal by reducing the spatial resolution of the original video signal,
Encode the basic video signal to generate basic encoded data,
Decoding the basic encoded data to generate a basic decoded video signal;
Separating a high frequency component from the basic decoded video signal to generate a high frequency separation signal;
Extracting a high frequency component corresponding to the original video signal from the high frequency separation signal to generate a high frequency equivalent signal;
The spatial resolution of the high-frequency equivalent signal is expanded through high-frequency component inter-layer estimation processing to generate a high-frequency component high-resolution signal,
Generating a differential signal that is a difference between the high-frequency separation signal and the high-frequency equivalent signal;
Enlarging the spatial resolution of the differential signal to generate a differential high resolution signal,
Expanding the spatial resolution of the basic decoded video signal to generate a basic high-resolution signal;
Combining the high-frequency component high-resolution signal, the differential high-resolution signal, and the basic high-resolution signal to generate a predicted video signal;
Encoding the prediction error signal derived using the original video signal and the predicted video signal to generate extended encoded data;
A video layer encoding method, wherein the basic encoded data and the extended encoded data are multiplexed to generate the multiplexed data. Computer
A video hierarchical encoding program that functions as a video hierarchical encoding device that generates multiplexed data that can be restored to a plurality of video signals having different spatial resolutions,
The computer,
A spatial decimation unit that generates a basic video signal by reducing the spatial resolution of the original video signal;
A base layer encoding unit that encodes the basic video signal to generate basic encoded data;
A base layer decoding unit that decodes the basic encoded data to generate a basic decoded video signal;
A high-pass filtering unit that generates a high-frequency separation signal by separating high-frequency components from the basic decoded video signal, and a high-frequency extraction filtering unit that generates a high-frequency equivalent signal by extracting high-frequency components corresponding to the original video signal from the high-frequency separation signal A high-frequency spatial interpolation unit that generates a high-frequency component high-resolution signal by expanding the spatial resolution of the high-frequency equivalent signal through inter-layer estimation processing of high-frequency components; a difference that is a difference between the high-frequency separated signal and the high-frequency equivalent signal A differential generation unit that generates a signal, a differential spatial interpolation unit that generates a differential high resolution signal by expanding the spatial resolution of the differential signal, and generates a basic high resolution signal by expanding the spatial resolution of the basic decoded video signal Basic interpolation unit, and high frequency component high resolution signal and differential high resolution A spatial interpolation unit comprising a signal combining unit, for generating the prediction image signal by synthesizing the signal and the basic high-resolution signal,
An enhancement layer encoding unit that encodes a prediction error signal derived using the original video signal and the predicted video signal to generate extended encoded data;
A multiplexing unit that multiplexes the basic encoded data and the extended encoded data to generate the multiplexed data;
A video hierarchical coding program characterized by being made to function. A video hierarchical decoding apparatus capable of restoring a plurality of video signals having different spatial resolutions from multiplexed data subjected to spatial scalability,
An extract unit for separating a plurality of pieces of encoded data having different spatial resolutions including at least basic encoded data and extended encoded data from the multiplexed data;
A base layer decoding unit that decodes the basic encoded data to generate a basic decoded video signal;
A spatial interpolation unit that generates a predicted video signal by expanding a spatial resolution of the basic decoded video signal;
An enhancement layer decoding unit that restores an original video signal using a prediction error signal obtained by decoding the extension encoded data and the prediction video signal;
With
The spatial interpolation unit is
A high-pass filtering unit for separating a high-frequency component from the basic decoded video signal to generate a high-frequency separated signal;
A high frequency extraction filtering unit that extracts a high frequency component corresponding to the original video signal from the high frequency separation signal to generate a high frequency equivalent signal;
A high-frequency spatial interpolation unit that expands the spatial resolution of the high-frequency equivalent signal through high-frequency component inter-layer estimation processing and generates a high-frequency component high-resolution signal;
A difference generation unit that generates a difference signal that is a difference between the high-frequency separation signal and the high-frequency equivalent signal;
A differential spatial interpolation unit that generates a differential high resolution signal by enlarging the spatial resolution of the differential signal;
A basic interpolation unit for generating a basic high-resolution signal by expanding a spatial resolution of the basic decoded video signal;
A signal synthesizer that synthesizes the high-frequency component high-resolution signal, the differential high-resolution signal, and the basic high-resolution signal to generate the predicted video signal;
A video hierarchical decoding device comprising: The high-frequency extraction filtering unit is an ε-filter that is a non-linear digital filter that performs noise removal on a video signal having a sudden change,
6. The video hierarchical decoding apparatus according to claim 5, wherein the multiplexed data includes a coefficient sequence _ak and a threshold value ε, which are parameters of the ε-filter. A video hierarchical decoding method capable of restoring a plurality of video signals having different spatial resolutions from multiplexed data subjected to spatial scalability,
Separating a plurality of encoded data having different spatial resolutions including at least basic encoded data and extended encoded data from the multiplexed data;
Decoding the basic encoded data to generate a basic decoded video signal;
Separating a high frequency component from the basic decoded video signal to generate a high frequency separation signal;
Extracting a high frequency component corresponding to the original video signal from the high frequency separation signal to generate a high frequency equivalent signal;
The spatial resolution of the high-frequency equivalent signal is expanded through high-frequency component inter-layer estimation processing to generate a high-frequency component high-resolution signal,
Generating a differential signal that is a difference between the high-frequency separation signal and the high-frequency equivalent signal;
Enlarging the spatial resolution of the differential signal to generate a differential high resolution signal,
Expanding the spatial resolution of the basic decoded video signal to generate a basic high-resolution signal;
Combining the high-frequency component high-resolution signal, the differential high-resolution signal, and the basic high-resolution signal to generate a predicted video signal;
A video hierarchical decoding method, wherein an original video signal is restored using a prediction error signal obtained by decoding the extended encoded data and the predicted video signal. Computer
A video hierarchical decoding program for functioning as a video hierarchical decoding device capable of restoring a plurality of video signals having different spatial resolutions from multiplexed data subjected to spatial scalability,
The computer,
An extract unit for separating a plurality of encoded data having different spatial resolutions including at least basic multiplexed data and extended multiplexed data from the multiplexed data;
A base layer decoding unit that decodes the basic encoded data to generate a basic decoded video signal;
A high-pass filtering unit that generates a high-frequency separation signal by separating high-frequency components from the basic decoded video signal, and a high-frequency extraction filtering unit that generates a high-frequency equivalent signal by extracting high-frequency components corresponding to the original video signal from the high-frequency separation signal A high-frequency spatial interpolation unit that generates a high-frequency component high-resolution signal by expanding the spatial resolution of the high-frequency equivalent signal through inter-layer estimation processing of high-frequency components; a difference that is a difference between the high-frequency separated signal and the high-frequency equivalent signal A differential generation unit that generates a signal, a differential spatial interpolation unit that generates a differential high resolution signal by expanding the spatial resolution of the differential signal, and generates a basic high resolution signal by expanding the spatial resolution of the basic decoded video signal Basic interpolation unit, and the high-frequency component high-resolution signal and differential high-resolution Signal synthesizing unit which generates the prediction image signal by combining the degree signal and the basic high-resolution signal, and the spatial interpolation unit having,
An enhancement layer decoding unit that restores an original video signal using a prediction error signal obtained by decoding the extension encoded data and the prediction video signal;
And a video hierarchy decoding program characterized by the above.

Images