WO1998002000A1

WO1998002000A1 - Image encoder and image decoder

Info

Publication number: WO1998002000A1
Application number: PCT/JP1997/001448
Authority: WO
Inventors: Norio Ito; Shuichi Watanabe; Hiroyuki Katata
Original assignee: Sharp Kabushiki Kaisha
Priority date: 1996-07-04
Filing date: 1997-04-25
Publication date: 1998-01-15
Also published as: JPH1023400A; JP3474707B2

Abstract

An image coder and an image decoder which can eliminate the deterioration of the quality of image at the time of encoding an image by deforming a shape information area extending over the inside and outside of an image frame based on the shape of an image information area during the course of the irreversibly encoding processing of a shape image, or expanding the circumference of the image information area by interpolation, so that all preceding shape information areas are included in the circumference before the processing.

Description

Description Image coding device and image decoding device

The present invention belongs to the field of digital image processing, and relates to an image encoding device that encodes image data with high efficiency, and an image decoding device that decodes encoded data created by the image encoding device. Background art

In image coding, methods for encoding, decoding, and combining different moving image sequences are being studied. For example, in the document “Image Coding Using Hierarchical Representation and Multiple Templates” (IEICE Technical Report IE 94-159, pp 99-106 (1 995)), a moving image sequence as a background and a component as a foreground are described. It describes a method of creating a new sequence by combining a moving image sequence of moving images (for example, a human image or a video of a fish cut out using chromaki technology).

Also, international standardization of moving picture coding method (IS OZ IEC MP EG 4) requires that the coding device and the decoding device having a hierarchical structure as shown in Fig. 8 use input information and shape information indicating the shape of the region of interest. Encoding, decoding, and combining of multiple component images represented by are considered.

Here, the part image is an image obtained by cutting out a person or an object in a moving image as a part. However, a normal moving image itself is also treated as a kind of component image. The first component image encoding unit 801, the second component image encoding unit 802, and the third component image encoding unit 803 include pixel value data of input images 1, 2, and 3, and shape information 1, 2, and 3. Are respectively encoded, and the multiplexing unit 804 multiplexes a plurality of component image encoded data, and transmits or accumulates them.

In the decoding apparatus, the coded data is separated into a plurality of component image coded data by the demultiplexing unit 805, and the first component image decoding unit 806, the second component image decoding unit 807, and the third component image decoding unit 808 In each of the plurality of component image encoded data is decoded, In the component image synthesizing unit 809, the decoded images 1, 2, and 3 and the shape information 1, 2, and 3 are each synthesized and output as a synthesized video output, which is displayed on a display (not shown) or the like. Here, the combination position information is information indicating a position when each component is displayed on the display at the time of combination. This information may be multiplexed in the coded data at the time of encoding, or may be manually or automatically given at the time of decoding. Fig. 9 schematically shows the component images and how they are combined. The part image 1 in FIG. 9 (a) is a normal moving image representing the background, and the part image 2 in FIG. 9 (b) is a moving image obtained by cutting out only a person. Furthermore, part image 3 in Fig. 9 (c) is a moving image cut out of only the car.

If only the component image 1 of the encoded data is decoded, an image with only the same background as in Fig. 9 (a) can be obtained. If the component image 1 and the component image 2 are decoded and combined, Fig. 9 (d) An image like is played. Furthermore, if component image 3 is decoded and the three component images are combined, an image as shown in Fig. 9 (e) is reproduced.

Problems of the conventional technique will be described with reference to the drawings.

As shown in FIG. 5, the component image is represented by a set of an image information area 501 and a shape information area 502 indicating its shape. At this time, it is assumed that the corresponding image information exists inside the shape information area.

However, when encoding a component image at a low bit rate, it is not appropriate to perform lossless encoding so that the original shape can be reproduced depending on the given bit rate. , It is necessary to reduce the code amount.

However, if the shape information area in the image information area 601 in FIG. 6 is deformed as shown in FIG. 6, the data area in the image information area that is beyond the original image information area is assumed. There will be an area 6 03 where does not exist. Therefore, there is no image information corresponding to the region 603, and if it is handled as if it exists, the image quality may be degraded.

An object of the present invention is to solve the problems of the prior art, and to avoid the inconvenience that occurs when the original shape information region extends over the inside and outside of the image information region by the lossless encoding of the shape information as described above. An object of the present invention is to provide a component image encoding method. Disclosure of the invention

The present invention has been made to achieve the above object, and the gist thereof is as follows.

First, a first gist of the present invention is to provide a moving image encoding apparatus that cuts out an arbitrary shape region of interest from an image frame and encodes data in the cut region. Image coding characterized in that, when performing lossless encoding, if the area shape deformed due to high-efficiency compression protrudes from the image information area, the deformed area shape is deformed according to the shape of the image information area. In the device.

Next, a second gist of the present invention is to provide a moving image encoding apparatus that cuts out an arbitrary shape region of interest from an image frame and encodes data in the cut out region. An image coding apparatus characterized in that when irreversible coding is performed, the coding process is controlled so that the area shape deformed for high efficiency compression does not protrude from the image information area.

Further, a third gist of the present invention resides in a moving picture encoding apparatus that cuts out an arbitrary shape region of interest from an image frame and encodes data in the cut region, wherein the shape information representing the region is not encoded. When performing lossless encoding, if the shape of the area deformed due to high-efficiency compression protrudes from the image information area, an appropriate value is set around the image information area so that the input image area completely includes the deformed area shape. An image coding apparatus characterized in that interpolation is performed in advance by using the image coding apparatus.

Furthermore, a fourth gist of the present invention is a moving image decoding apparatus that decodes an image region that is an arbitrary-shaped region. As a result of decoding the shape information representing the region, the region shape deformed for high-efficiency compression is transformed into an image. An image decoding apparatus characterized in that, when the image is outside the information area, the shape of the deformed area is deformed in accordance with the shape of the image information area.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the present invention.

FIG. 3 is a block diagram showing a third embodiment of the present invention. FIG. 4 is a block diagram showing a fourth embodiment of the present invention.

FIG. 5 is a diagram for explaining a conventional technique.

FIG. 6 is a diagram for explaining a problem of the conventional technique.

FIG. 7 is a diagram illustrating an embodiment of the present invention.

FIG. 8 is a block diagram illustrating a conventional technique.

FIG. 9 is a diagram schematically showing a component image and a state of the combination. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing a first embodiment of the present invention. This solves the problem of the prior art by deforming the shape information deformed by non-reversible coding according to the size of the image shape area.

In this embodiment, the coding method of the image signal (texture) by the image coding unit 105 is not described, but it is for coding an arbitrary shape area defined by the shape information area. The input shape information is coded using the coding parameters determined by the shape coding unit 101.

The inside / outside determination unit 103 determines whether or not the shape information area decoded by the shape decoding unit 102 is inside the image information area. If it is inside, the image signal is encoded by the image encoding unit 105 using the decoding area shape.

If it is outside, the decoded shape area is deformed by the shape deforming unit 104 according to the shape of the image frame, and then the image is again encoded by the image coding unit 105. The coded area shape and image information are multiplexed by the multiplexing unit 106 to become coded data of one arbitrarily shaped area.

Here, the shape deformation processing corresponds to, for example, deleting an area 603 where no data exists in the image information area as shown in FIG. As a result, all the image information in the shape information area is obtained from the data in the image information area.

FIG. 2 is a block diagram showing a second embodiment of the present invention. This solves the problem of the prior art by adjusting the parameters so that the shape information area obtained by decoding when performing non-reversible coding always exists inside the image information area. is there.

In this embodiment, after the shape information encoded by the shape encoding unit 201 is decoded by the shape decoding unit 202, it is determined whether or not the result exists in the image information area by the inside / outside determination unit. Judge at 203. If it extends over the outside, the two switches 206 and 207 are turned off, and the coding parameter change unit 205 changes the parameters.

Otherwise, the two switches 206 and 207 are turned on, and the image information is encoded using the shape information area obtained at that time. Therefore, the shape encoding process is repeated until the target encoding parameter is found. The image encoding unit 204 performs encoding using the shape information area thus obtained. As a method of selecting parameters, for example, a part that can greatly deform the shape is represented by a simple shape by reducing the amount of noise, and conversely, a part with a small amount is used to approach the original shape by spending the amount of code.

As in the first embodiment, the coded area shape and the image information are multiplexed by the multiplexing unit 106 to become coded data of one arbitrarily shaped area. FIG. 3 is a block diagram showing a third embodiment of the present invention. This is a method in which an image information area is created in advance using extrapolation so that corresponding image information always exists inside a shape information area deformed by non-reversible coding. In this embodiment, after the shape information encoded by the shape encoding unit 101 in FIG. 3 is decoded by the shape decoding unit 102, the area extension unit 3 is adjusted to the size of the area shape. 03 Additional regions are created by extrapolation.

Here, the additional area refers to an area 703 indicated by a hatched portion in FIG. By encoding this as a new image information area in the image encoding unit 304 in FIG. 3, the image information area always exists inside the shape information area 720 in FIG. The problem of the conventional technology can be solved.

The image information area 602 and the area 603 where no data exists in the image information area are the same as those in FIG.

Further, as in the first embodiment described above, the coded region shape and image information are multiplexed by the multiplexing unit 106 in FIG. You.

FIG. 4 is a block diagram showing a fourth embodiment of the present invention. This is because, when the decoded shape information area is irreversibly encoded on the encoding side and straddles the inside and outside of the image information area, it is deformed according to the size of the image information, thereby causing a problem in the conventional technology. Is to solve.

Also in this embodiment, although details of the image signal decoding method in the image decoding unit 405 in FIG. 4 are not described, this also decodes image information having an arbitrary shape determined by the shape information. Things. The shape information separated by the demultiplexing unit 401 in the figure is decoded by the shape decoding unit 402.

The decoded shape information area is the same as that obtained in the irreversible process at the time of encoding, and the inside / outside determination unit 403 determines whether or not this crosses the image information area. If it is determined that this straddles the outside of the image information area, the shape is deformed by the shape deforming section 404 so as not to straddle the outside. Otherwise, the original component image is decoded and reproduced by the image decoding unit 405 using the decoding result as it is.

Note that the present invention can be similarly applied to still images as well as moving images as image information. Industrial applicability

The following advantages can be obtained by using the image encoding and decoding apparatus of the present invention.

First, image quality degradation due to external data reference due to changes in the area shape due to lossy coding of the area shape information is adjusted to match the image information area when the shape information area extends inside and outside the image information area. Deformation can suppress image quality degradation.

Next, image quality degradation due to data reference outside the image information area due to changes in the area shape due to lossy coding of the area shape information is explained by the loss of shape due to lossy coding when the shape information area extends over the inside and outside of the frame. By controlling the encoding parameters so that the image quality is restricted to the inside of the image information area, it is possible to suppress the deterioration of the image quality.

In addition, loss of image quality due to external data reference caused by changes in the area shape due to lossy encoding of the area shape information is reduced. When the image information area extends outside and inside, the image information area can be interpolated with a certain appropriate value so as to include the shape information area, so that the deterioration of image quality can be suppressed. Furthermore, when the shape information area in which the area shape information is decoded extends over the inside and outside of the image information area, the quality of the decoded image can be prevented from lowering by deforming it in accordance with the image information area.

Claims

The scope of the claims

1. In a video encoding device that cuts out an arbitrary shape area of interest from an image frame and encodes data in the cut-out area, when irreversibly coding the shape information representing the area, high efficiency compression is performed. An image coding apparatus characterized in that when a deformed region shape protrudes from an image information region, the deformed region shape is deformed according to the shape of the image information region.

2. In a video encoding device that cuts out an arbitrary shape region of interest from an image frame and encodes data in the cut region, when irreversibly encoding the shape information representing the region, high efficiency compression is performed. An image encoding device, wherein an encoding process is controlled so that a deformed area shape does not protrude from an image information area.

3. In a video encoding device that cuts out an arbitrary shape area of interest from an image frame and encodes data in the cut-out area, when irreversibly encoding the shape information representing the area, high efficiency compression is performed. When the deformed region shape protrudes from the image information region, interpolation is performed in advance using appropriate values around the image information region so that the input image region completely includes the deformed region shape. Image coding

4. In the moving picture decoding apparatus that decodes an image area that is an arbitrary-shaped area, if the area information deformed for high-efficiency compression protrudes from the image information area as a result of decoding the shape information representing the area, the deformation is performed. An image decoding device characterized in that the shape of an area is transformed according to the shape of the image information area.