JP2002044671A - Dynamic-picture decoding apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a dynamic-picture decoding apparatus wherein encoding data is endowed with a hierarchical property, only a selective region is reproduced in various picture qualities from a part of the encoding data and a low- picture-quality image can be reproduced from a part of the encoding data. SOLUTION: The dynamic-picture decoding apparatus is provided with an encoding-data separation part 10 which separates input encoding data, a regional position and shape decoding part 9 which decodes encoding data on the position and the shape of the selective region, a low-order layer decoding part 8 which decoding encoding data on a low-order layer and which creates a low picture quality decoding image regarding the selective region and a high-order layer decoding part 11. When the encoding data on the low-order layer is input, a prediction is performed on the basis of the decoding image in the low-order layer, the encoding data on the high-order layer is decoded, and a low picture quality decoding image is crated regarding a region other than the selective region. The decoding image of the whole screen to which the decoding image in the selective region created in the part 8 or the decoding image in the region other than the selective region created by the part 11 is added is reproduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention belongs to the field of digital moving picture processing, and relates to a moving picture decoding apparatus for decoding coded data obtained by coding moving picture data with high efficiency.

[0002]

2. Description of the Related Art In moving picture coding, a method has been proposed in which the quality of a specific area is better than that of other areas.

[0003] For example, ISO / IEC JTC1 / SC29 / WG11 MPEG95 /
In the method described in 030, an area is selected from a moving image, and in a selected area (hereinafter, referred to as a selected area), the image quality of the selected area is controlled by controlling a quantization width, a time resolution, and the like. It is better than other areas.

FIG. 12 is a block diagram showing another conventional method. The area selection unit 20 in FIG. 12 is a part that selects a specific area in a moving image. For example, in the case of selecting a face region in a moving image such as a videophone call, the document "Real-time face image tracking method" (Preprint of the Institute of Image Electronics Engineers of Japan, 93-04-04, pp.
13-16 (1994)).

[0005] An area position / shape encoding unit 21 shown in FIG. 1 encodes the position and shape of the selected area. If the shape is an arbitrary shape, it is encoded using, for example, a chain code. The encoded position and shape are incorporated into the encoded data by the encoded data integration unit 22 shown in FIG.

An encoding parameter adjusting section 23 shown in FIG. 1 adjusts various parameters used for controlling the image quality and the amount of data in the moving image encoding. This is a part that controls so that the image quality is better encoded than the other areas.

[0007] The parameter encoding unit 24 in the figure is a part for encoding the various parameters. The coded parameters are incorporated into the coded data by the coded data integration unit 22 shown in the figure, and are transmitted or stored.

The moving picture coding unit 25 shown in FIG. 1 is a part for coding the inputted moving picture data using the above-mentioned various parameters, and employs a conventionally used method.
That is, high efficiency coding is realized by combining motion compensation prediction, orthogonal transform, quantization, variable length coding, and the like. The encoded moving image data is incorporated into the encoded data by the encoded data integration unit 22 shown in the figure, and is transmitted or stored.

As described above, encoding is performed so that the quality of a selected area in an image is better than that of other areas.

[0010]

As described above, in the prior art, the image quality is improved by adjusting parameters such as quantization width, spatial resolution, and time resolution, and allocating a large number of bits to a selected area. ing.

However, in the prior art, since the selected area and the other area are incorporated into one piece of encoded data, only a part of the encoded data is decoded to decode the decoded image of the specific area. There is a problem that processing such as obtaining a decoded image of low image quality cannot be performed. In recent years, hierarchies of such encoded data have been actively studied, but a method of selecting a specific area has not been sufficiently considered.

An object of the present invention is to solve such a problem. In addition to making the selected area in the moving picture better in quality than the other areas as a whole, the encoded data has a hierarchy. To provide a moving picture decoding apparatus capable of reproducing only a selected area with various image qualities from a part of encoded data or reproducing a low-quality image from a part of encoded data. is there.

[0013]

SUMMARY OF THE INVENTION According to the present invention, there is provided a coded data of a position and a shape of a specific selected area of each frame, and a lower layer coding in which pixel values of the selected area of each frame are coded with low image quality. Coded data including data, or coded data of the position and shape of a specific selected area of each frame, and coded data of a lower layer obtained by coding pixel values of the selected area of each frame with low image quality, Moving image decoding that decodes a moving image from encoded data including encoded data of an upper layer obtained by predictively encoding pixel values of each frame other than the selected area in the lower layer using the decoded pixel values of the lower layer. An apparatus, comprising: encoded data separating means for separating input encoded data; area position / shape decoding means for decoding encoded data of the position and shape of the selected area; and a code of the lower layer. It decodes the data, and the lower layer decoding means for generating decoded image of low image quality for the selected region,
When the coded data of the upper layer is input, the coded data of the upper layer is decoded by performing prediction from the decoded image of the lower layer, and a low-quality decoded image is created for areas other than the selected area. A decoded image of the selected area created by the lower layer decoding means, or a decoded image of the entire screen to which a decoded image other than the selected area created by the upper layer decoding means is added. It is characterized by reproducing.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the concept of the present invention will be described. FIG. 9 is a diagram for explaining the concept of the encoding method according to the present invention. In the hierarchical coding of the present invention, a lower layer and two upper layers are used.

In the lower layer, only the selected area indicated by diagonal lines is encoded with low image quality. Let t be the time of interest, and let L (t) be the decoded image. In the first upper layer, a region not coded in the lower layer is coded. The decoded image of this layer is written as H1 (t). At this time, predictive encoding is performed using the decoded image L (t) of the lower layer, the decoded image H1 (t-1) of the first upper layer, and the like.

In the second upper layer, only the selected area is predictively encoded with higher image quality than the lower layer. The decoded image of this layer is written as H2 (t). At this time, predictive coding is performed using the decoded image L (t) of the lower layer, the decoded image H2 (t-1) of the second upper layer, and the like.

FIGS. 10 and 11 are diagrams for explaining the concept of the decoding system of the present invention. FIG. 10 shows a three-layer decoding process when only the lower layer of the encoded data is decoded, when the first upper layer is decoded, and when all are decoded. In this case, when decoding up to the lower layer, only the region selected by the encoding device is reproduced with low image quality, and when decoding up to the first upper layer, the entire image is reproduced with low image quality. The selected area is reproduced with high image quality, and the other areas are reproduced with low image quality.

On the other hand, FIG. 11 shows a case where all the encoded data is decoded after decoding the second upper layer instead of the first upper layer. In this case, only the selected area is reproduced with high image quality on the intermediate layer (second upper layer).

As described above, in the decoding apparatus according to the present invention, only a low-quality selected area is reproduced by a simple apparatus corresponding to only the lower layer, and the entire low-quality image is reproduced by an apparatus corresponding to the upper layer. Alternatively, only the high-quality selected area is reproduced. That is, there are two types of different upper layers above the common lower layer, and these can be selected.

Hereinafter, embodiments of the present invention will be described in detail. FIG. 1 is a block diagram showing an encoder according to an embodiment of the present invention.

The area selecting section 5 and the area position / shape coding section 6 of FIG. 1 operate in the same manner as that of the prior art shown in FIG.

The lower layer encoder 4 in FIG. 1 encodes only the selected area selected by the area selector 5 with low image quality.
This is a part that creates encoded data of the lower layer and creates a decoded image obtained by decoding the encoded data. The decoded image is used as a reference image for predictive coding.

A first upper layer encoding section 1 in FIG. 1 encodes the entire image with low image quality, creates encoded data of the first upper layer, and creates a decoded image obtained by decoding the encoded data. It is. The decoded image is used as a reference image for predictive coding.

The second upper layer encoding section 2 in FIG. 2 encodes only the selected area with high image quality, creates encoded data of the second upper layer, and creates a decoded image obtained by decoding the encoded data. Part. The decoded image is used as a reference image for predictive coding.

The coded data integration unit 3 shown in FIG. 3 converts the code of the position and shape of the selected area, the coded data of the lower layer, the coded data of the first upper layer, and the coded data of the second upper layer. This is the part to be integrated.

As a method of encoding an image by the lower layer encoding unit 4, the first upper layer encoding unit 1, and the second upper layer encoding unit 2, there are several methods described below.

FIGS. 5 and 6 illustrate a method of controlling the image quality of the lower layer and the upper layer according to the size of the quantization width.

FIG. 5A shows the coding of the lower layer, and the area indicated by oblique lines is the selected area. In the first frame of the lower layer, the selected area is intra-coded, and in the other frames, the selected area is predictively encoded by motion compensation prediction. As a reference image for motion compensation prediction, a selected region of a frame that has already been encoded and decoded in a lower layer is used. Although only forward prediction is shown in the figure, a combination of backward prediction may be used.

In the lower layer, the quantization width is controlled to be larger than that of the second upper layer, so that only the selected area of the input image is coded with a low SNR (Signal-to-Noise Ratio) image quality. You. Therefore, the lower layer is encoded with a small code amount.

FIG. 5B shows the encoding of the first upper layer. Here, the entire image is encoded. For example, a method of predictively encoding the entire image based on the decoded image of the lower layer and the decoded image of the first upper layer is used. In this case, in the first frame, the entire image is predictively encoded from the decoded image of the lower layer (except for the selected region, motion compensation prediction is not substantially used, and the intra-frame encoding is performed). Then, encoding is performed by combining prediction encoding by motion compensation prediction.

Alternatively, a method may be adopted in which the selected region is not encoded at all and the other region is subjected to predictive encoding only. FIG. 6 shows such a case. Encoding is performed for an area other than the selected area.

FIG. 5C shows the encoding of the second upper layer. Here, only the selected region is encoded with a small quantization width. At this time, the data to be encoded is the difference data between the original image and the predicted image from the lower layer. As described above, only the prediction from the lower layer is shown in FIG. 5C, but the encoding may be performed in combination with the prediction from the decoded frame in the second upper layer.

FIG. 7 illustrates a method of controlling the image quality of the lower layer and the upper layer according to the difference in time resolution in the encoding apparatus of FIG.

FIG. 7A shows the encoding of the lower layer, and the area indicated by oblique lines is the selected area. In the first frame of the lower layer, the selected area is intra-coded, and in the other frames, the selected area is predictively encoded by motion compensation prediction. As a reference image for motion compensation prediction, a selected region of a frame that has already been encoded and decoded in a lower layer is used. Although only forward prediction is shown in FIG. 7A, backward prediction may be used in combination.

The lower layer is controlled so that the frame rate is lower than that of the second upper layer and the temporal resolution is lower. Here, encoding may be performed so that the quantization width is reduced and the SNR in each frame is increased.

FIG. 7B shows the encoding of the first upper layer. Here, the entire image is encoded with a low temporal resolution. As a method of the prediction, FIG.
The same as described above is used.

FIG. 7C shows the encoding of the second upper layer. Here, only the selected region is encoded with a high temporal resolution. At this time, prediction from the lower layer is used for a frame in which the selected region is encoded in the lower layer, and motion compensation prediction from a decoded frame of the upper layer is used for other frames.

When the prediction from the lower layer is used, the encoding of the second upper layer may not be performed at all. In this case, the decoded image of the lower layer is used as it is as the decoded image of the second upper layer.

FIG. 8 illustrates a method of adjusting the image quality of the lower layer and the upper layer according to the difference in spatial resolution.

FIG. 8A shows the encoding of the lower layer. The original image is converted into an image having a low spatial resolution by an operation such as a low-pass filter or a thinning-out operation, and only the selected area indicated by oblique lines is encoded. In the first frame of the lower layer, the selected area is intra-coded, and in the other frames, the selected area is predictively encoded by motion compensation prediction.

FIG. 8B shows the encoding of the first upper layer. The original image is converted to a low spatial resolution image,
The entire image is encoded with a high temporal resolution. As a prediction method, a method similar to that shown in FIG. 5B or FIG. 6 is used.

FIG. 8C shows the encoding of the second upper layer. Here, only the selected area is encoded with a high spatial resolution. At this time, the decoded image of the lower layer is converted to the same spatial resolution as the original image, and the selected region is predicted using the lower layer and the motion compensated prediction from the frame already coded and decoded in the second higher layer. Encoded.

Although the above description has been made on the gradation (SNR) resolution, time resolution, and spatial resolution, these may be combined. For example, depending on the difference between the spatial resolution and the time resolution, the image quality of the lower layer and the upper layer may be adjusted, or depending on the difference between the quantization width and the time resolution,
The image quality of the lower layer and the upper layer can be adjusted.

As described above, the encoding is performed so that the image quality of the selected area is generally better than that of the other areas, and the encoded data has a hierarchical structure of a lower layer and two types of upper layers. be able to.

Next, a decoder according to the embodiment of the present invention will be described.

FIG. 2 is a block diagram showing a first example of the decoder, in which only the lower layer is decoded.

The coded data separation section 7 shown in FIG. 2 is a section for separating and taking out coded data representing the position and shape of a region and coded data of a lower layer from the coded data.

The area position / shape decoding unit 9 shown in FIG. 9 is a part for decoding the position and shape of the selected area.

The lower layer decoding section 8 shown in FIG. 3 is a section for decoding the encoded data of the lower layer for the selected area and creating a decoded image of low image quality only for the selected area.

Therefore, the image output from the decoding device has image information only in the selected area, and only the selected area is displayed as a "window". Alternatively, a spatial resolution converter may be provided in the lower layer decoding unit 8, and the selected area may be enlarged and displayed on the entire screen.

As described above, in this embodiment, since only the data in the lower layer of the selected area is decoded, the decoded image has a low image quality. The size of the software, the amount of processing, and the like can be reduced, and simple decryption processing can be performed.

FIG. 3 is a block diagram showing a second example of the decoder. Area position / shape decoding section 9 and lower layer decoding section 8
Performs the same operation as that of FIG.

The coded data separation unit 10 shown in FIG. 3 separates and extracts the area position / shape data, the coded data of the lower layer of the selected area, and the coded data of the first upper layer from the coded data. It is.

The first upper layer decoding section 11 shown in FIG. 1 is a section for decoding the encoded data of the first upper layer.
Here, the entire image is decoded with low image quality using the region position / shape data, the decoded image of the lower layer, and the decoded image of the first upper layer, and a decoded image of the first upper layer is created.

Here, the first upper layer is used as the upper layer, but a second upper layer may be used instead. In that case, the encoded data separation unit 10
The region position / shape data, the lower layer encoded data, and the second upper layer encoded data are separated and extracted from the encoded data.

Further, the first upper layer decoding unit 11
, The area position / shape data, the decoded image of the lower layer of the selected area and the decoded image of the second upper layer are used, and only the selected area is decoded with high image quality. A decoded image of the upper layer is created. The decoded image is displayed as a “window” on the display, or is enlarged and displayed on the entire screen.

FIG. 4 is a block diagram showing a third example of the decoder. The area position / shape decoding section 9 and the lower layer decoding section 8 operate in the same manner as in FIG.

The coded data separation unit 13 shown in FIG. 4 converts the coded data into data of the area position and shape, coded data of the lower layer, coded data of the first upper layer, and coded data of the second upper layer. This is the part that separates and retrieves data.

The first upper layer decoding unit 11 in the figure decodes the encoded data of the first upper layer, and the second upper layer decoding unit 12 in the same figure decodes the encoded data of the second upper layer. Decrypt the data.

The upper layer synthesizing unit 14 shown in FIG. 9 synthesizes the decoded image of the first upper layer with the decoded image of the second upper layer using the information on the area position and shape. The combining is performed by using the decoded image of the second upper layer in the selected region and the decoded image of the first upper layer in the region other than the selected region.

Therefore, the image output from the decoding device is
It has image information for the entire image, especially for the selected area.
An image with good R, time resolution, spatial resolution, etc. is decoded. In this way, decoding is performed so that the image quality of the area selected by the encoder is better than the other areas.

In the embodiment of the present invention described above, it is assumed that the decoding apparatus receives all coded data. However, in moving picture communication, only data limited from the decoding side to the coding side is transmitted. May require. For example, a request is made to transmit the coded data of the area position / shape, the coded data of the lower layer, and the coded data of the first upper layer. This is performed mainly for the purpose of communicating over a transmission path with a small bandwidth.

According to the present invention, when used in a very narrow transmission path, only lower layer data is communicated, and in a slightly wider band, one of two types of upper layers is selectively communicated. In a wider band, communication for communicating all data can be realized.

[0064]

According to the present invention, if only the lower layer of the encoded data is decoded, only the selected area is reproduced with low image quality. Also, when decoding the upper layer of the encoded data, it is possible to reproduce the decoded image of the entire screen including the decoded image other than the selected area.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an encoder according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a decoder according to the first embodiment of the present invention.

FIG. 3 is a block diagram illustrating a decoder according to a second embodiment of the present invention.

FIG. 4 is a block diagram illustrating a decoder according to a third embodiment of the present invention.

5A is a diagram illustrating an example of a coding method of a lower layer in the encoder of the present invention, and FIG. 5B is a diagram illustrating an example of a coding method of a first upper layer in the encoder. It is a figure which shows, and (c) is a figure which shows an example of the encoding method of the 2nd upper layer in the same encoder.

FIG. 6 is a diagram illustrating another example of the encoding method of the first upper layer in the encoder of the present invention.

FIG. 7A is a diagram showing another example of the encoding method of the lower layer in the encoder of the present invention, and FIG. 7B is a diagram showing the encoding method of the first upper layer in the encoder. It is a figure which shows another example, and (c) is a figure which shows another example of the encoding method of the 2nd upper layer in the encoder.

FIG. 8A is a diagram showing still another example of a lower layer encoding method in the encoder of the present invention, and FIG. 8B is a diagram showing a first upper layer encoding method in the encoder. FIG. 21 is a diagram showing still another example of (a), and (c) is a diagram showing still another example of the encoding method of the second upper layer in the encoder.

FIG. 9 is a diagram showing the concept of the encoding method of the present invention.

FIG. 10 is a diagram illustrating an example of a concept of a decoding method according to the present invention.

FIG. 11 is a diagram showing another example of the concept of the decoding method of the present invention.

FIG. 12 is a block diagram illustrating a conventional method.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 first upper layer coding unit 2 second upper layer coding unit 3 coded data integration unit 4 lower layer coding unit 5 area selector 6 area position / shape coding unit 7 coded data separation unit 8 lower Layer decoding unit 9 Area position / shape decoding unit 10 Encoded data separation unit 11 First upper layer decoding unit 12 Encoded data separation unit 13 Second upper layer decoding unit 14 Upper layer synthesis unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 MA04 MA32 MA33 MA35 PP06 TA39 TA46 TB04 TC12 TC34 TD05 UA05 5J064 AA02 BB03 BB12 BC25 BD01

Claims (1)

[Claims] 1. Encoded data including encoded data of a position and a shape of a specific selected area of each frame, and encoded data of a lower layer obtained by encoding pixel values of the selected area of each frame with low image quality; Alternatively, encoded data of the position and shape of a specific selected area of each frame, encoded data of a lower layer obtained by encoding pixel values of the selected area of each frame with low image quality, and pixels other than the selected area of each frame A moving image decoding apparatus that decodes a moving image from coded data including coded data of an upper layer obtained by performing low-quality prediction coding on a value using the decoded pixel value of the lower layer, and Coded data separating means for separating coded data; area position / shape decoding means for decoding coded data of the position and shape of the selected area; and decoding and selecting the coded data of the lower layer. A lower-layer decoding unit for creating a low-quality decoded image for an area; and when the encoded data of the upper layer is input, by performing prediction from the decoded image of the lower layer, the encoded data of the upper layer is obtained. And an upper layer decoding unit that creates a low-quality decoded image for areas other than the selected area, and a decoded image of the selected area created by the lower layer decoding unit or created by the upper layer decoding unit. A decoded image of the entire screen to which a decoded image other than the selected area is added.