JP2002223440A - Shape coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a shape coder that can code shape information with a small amount of codes without causing overlapping or gap among objects in the case of combining decoded objects to configure an image. SOLUTION: This shape coder 10, that divides an image in the unit of a prescribed image area and codes the shape information of image division areas, is provided with a shape noise elimination filter section 20 that eliminates as very small change included in the shape information and a shape coding section 30 that applies reverse coding to the shape information from which the very small change is eliminated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a shape coding apparatus, and more particularly to a shape coding apparatus for coding shape information of an image area forming a part of an image.

[0002]

2. Description of the Related Art For example, the MPEG-4 video coding standard (IS
O / IEC 14496-2: “Information technology-Generic cod
ing of audio-visual objects-Part2: Visual ”), the image is divided into image areas such as foreground and background (hereinafter referred to as“ objects ”) instead of frame or field units. , Is individually encoded in divided object units.

FIG. 6 is a diagram for explaining an example of processing for dividing an image into object units. In FIG. 6, an image 101 is composed of objects representing a person, a car, a tree, and a background. Therefore, this image 101 is composed of a human object 102 and a car object 10.
3, background object 104, tree object 10
It is divided into five.

For example, each of the objects 102 to 105 divided as shown in FIG. 6 is composed of a texture image representing a picture of the object and object shape information representing the shape of the object. The object-based coding method performs texture coding for coding a texture image and shape coding for coding object shape information.

[0005] The object shape information is represented by a binary bit pattern image indicating whether or not a pixel is included in the object, or contour information of the object shape. That is, the shape encoding device that performs shape encoding includes one that encodes a binary bit pattern image and one that encodes contour information.

[0006] Further, the shape encoding apparatus performs reversible encoding in which the object shape information before encoding and the object shape information after decoding are the same, and performs the object encoding with the object shape information before encoding. This includes irreversible encoding that differs from the subsequent object shape information.

More specifically, as a method for encoding a binary bit pattern image, the JBIG standard of the intra-frame encoding method (ISO / IEC 11544: “Progressive Bi-lev
el Compression ”) and MMR (Modified Modified Re
ad) (ITU-T T.6: “Facsimile co
ding schemes and coding control functions for Grou
p 4 facsimile apparatus ”). As a method of encoding the object shape information represented by the binary bit pattern image using the correlation in the time direction, MPE is used.
There is a G-4 shape coding system.

On the other hand, as a method for encoding contour information, an encoding method using a wavelet descriptor of an intra-frame encoding method (George Muller, et al .: “Progressive T
ransmission of Line Drawings Using the Wavelet Tra
nsform ”, IEEE Transaction on Image Processing, Vol.5
No.4, pp.666-672, April 1996). In addition, as a method of encoding the object shape information represented by the contour information using the correlation in the time direction, there are encoding methods (Kazuhiro Asami et al .: “Coding of moving contour image using wavelet descriptor”, PCSJ97, P-5.13, pp.113-114, 1997).

The above-mentioned encoding system performs lossless encoding. Hereinafter, an encoding system that performs irreversible encoding will be described. As an encoding method for performing irreversible encoding, an encoding method for contour information using a spline function by Myron et al. (Myron Flickner, et al .: “Periodic Qua
si-Orthogonal Spline Bases and Applications to Lea
st-Squares Curve Fitting of Digital Images ”, IEEE
Transaction on ImageProcessing, vol.5 No.1, pp.71-8
8, Jun. 1996). This encoding method approximates contour information and performs irreversible encoding.

In the MPEG-4 shape encoding method, there is a mode for performing irreversible encoding. MPEG-4
Is the object shape information of 1/2,
The image data is reduced to 1/4 or the like, and lossless encoding is performed using the reduced object shape information.

[0011]

However, in the case of irreversible shape coding, object shape information differs between before coding and after decoding. In other words, when an image is divided into objects and irreversible shape encoding is performed,
The shape of the object before encoding is different from the shape of the object after decoding. Therefore, when composing the image by combining the decrypted objects,
There was a problem that a gap was created between objects.

In the case of lossless shape coding, object shape information is the same before coding and after decoding, and the shape of the object before coding and the shape of the object after decoding are the same. . Therefore, when an image is formed by combining the decoded objects, there is no problem that the objects overlap and a gap is generated. However, reversible shape coding has a problem that the amount of information after coding is larger than irreversible shape coding.

[0013] Therefore, it has been desired to perform shape coding in which the amount of information after coding is small, and when the decoded objects are combined to form an image, the objects do not overlap with each other and no gap occurs.

[0014] The present invention has been made in view of the above points, and when composing a screen by combining decoded objects, there is no overlap or gap between objects, and encoding can be performed with a small code amount. It is an object to provide a simple shape encoding device.

[0015]

In order to solve the above-mentioned problems, the present invention provides a shape encoding apparatus for dividing an image into predetermined image area units and encoding shape information of the divided image areas. It is characterized by having a shape noise removal filter unit for removing a minute change included in the shape information, and a shape encoding unit for reversibly encoding the shape information from which the minute change has been removed.

In such a shape coding apparatus, before lossless coding of the shape information, fine changes included in the shape information are removed by a shape noise removal filter. That is, since a minute change included in the shape information can be removed before the lossless encoding, the amount of information in the lossless encoding can be reduced. Therefore, with the shape encoding device of the present invention, shape encoding can be performed with a small amount of code, and there is no overlap or gap between decoded objects.

Further, the present invention is characterized in that the shape noise elimination filter section has a spatiotemporal filter section accompanied by a motion correction process.

In such a shape encoding apparatus, a minute change included in the shape information can be removed by using a spatio-temporal filter unit including a motion correction process.

Further, according to the present invention, the shape noise removal filter unit generates a labeled image information in which the same label information is added to the pixels included in the same shape information, It has a spatio-temporal majority filter unit that performs spatio-temporal majority filter processing on image information, and a shape information conversion unit that converts labeled image information that has been subjected to the spatio-temporal majority filter process into shape information.

In such a shape encoding device, fine changes included in shape information can be removed by using a spatio-temporal majority filtering process.

Further, the present invention is characterized in that the motion vector used for the motion correction processing in the shape noise removal filter unit and the motion vector used for the lossless coding in the shape encoding unit are the same.

In such a shape encoding apparatus, the motion vector used for the motion correction processing in the shape noise elimination filter unit is the same as the motion vector used for the lossless encoding in the shape encoding unit, so that the shape noise is reduced. The shape noise removal effect in the removal filter unit can be enhanced.

[0023]

Next, embodiments of the present invention will be described with reference to the drawings. First, in order to facilitate understanding of the present invention, the cause of an increase in the code amount in shape coding will be described.

The cause of an increase in the amount of information in shape coding is the presence of spatially minute irregularities at the outer edge of the object shape in the case of a still image. FIG. 1 is a diagram illustrating an example of unevenness existing at an outer edge of an object shape. In FIG. 1, objects 1 to 3 are divided from an image. The objects 1 to 3 have spatially fine irregularities at the outer edge of the shape. If the spatially minute unevenness existing at the outer edge of the object shape is shape-coded, the amount of information in shape coding increases.

Further, in the case of a moving image, the amount of information increases in the shape encoding due to the presence of spatially minute irregularities existing at the outer edge of the object shape and the presence of objects belonging to the object at short time intervals. The presence of changing pixels. FIG. 2 is a diagram illustrating an example of a change in an object shape over time. FIG. 2A shows an initial object shape, and FIG.
2 (b), and sequentially changes to the object shape of FIG. 2 (c).

For example, objects 4 and 5 in FIG. 2A, objects 6 and 7 in FIG.
The objects 8 and 9 in FIG. 2C have different object shapes. That is, FIGS.
In the object shape of (c), the object shape changes over time. If the change of the object shape due to the lapse of time is repeated within a short time, the amount of information in shape coding increases.

That is, in order to reduce the amount of information in shape coding, it is only necessary to remove spatially or spatiotemporally minute changes in the object shape. For this reason, according to the present invention, as a pre-process of shape coding, a spatial or spatio-temporal minute change in the object shape is filtered. Specifically, a shape noise elimination filter for removing a spatially or spatiotemporally minute change in the object shape is provided at a stage preceding the shape encoding unit for performing the shape encoding.

Hereinafter, the configuration of the shape encoding apparatus according to the present invention will be described with reference to FIG. FIG. 3 shows a configuration diagram of an embodiment of the shape encoding apparatus according to the present invention. 3, the shape encoding device 10 is configured to include a shape noise removal filter unit 20 and a lossless shape encoding unit 30.

In this embodiment, an example in which a three-frame spatiotemporal majority filter is used as an example of a shape noise elimination filter will be described. It is. When a three-frame spatio-temporal majority filter is used, the shape noise removal filter unit 20 includes a labeled imaging unit 21, a frame memory 22, a frame memory 23,
Motion vector detector 24, spatiotemporal majority filter 2
5, It is configured to include the shape information generating unit 26.

First, the object shape information is supplied to the labeled imaging unit 21 of the shape noise elimination filter unit 20. The labeled imaging unit 21 converts the supplied object shape information into a labeled image. A labeled image is an image in which pixels belonging to the same object have the same label and pixels belonging to different objects have different labels. The labeled imaging unit 21 supplies the converted labeled image to the frame memory 22, the motion vector detection unit 24, and the spatiotemporal majority filter unit 25.

The frame memory 22 stores the supplied labeled image and stores the labeled image at a predetermined timing in the frame memory 23, the motion vector detecting section 24,
It is supplied to the spatiotemporal majority filter unit 25. The frame memory 23 stores the supplied labeled image, and supplies the labeled image to the motion vector detecting unit 24 and the spatiotemporal majority filter unit 25 at a predetermined timing.
In other words, the labeled image output from the labeled imaging unit 21 is stored in the frame memory 22 and the frame memory 23.
Are sequentially shifted.

The motion vector detecting section 24 includes a labeled encoding section 21, a frame memory 22, and a frame memory 23.
Supplies three frames of labeled images. That is, the motion vector detecting unit 24 is supplied with labeled images of three frames at different times. The labeled image supplied from the frame memory 23 is the oldest, and the labeled image supplied from the labeled imaging unit 21 is the newest.

The motion vector detection unit 24 performs motion vector detection on the labeled image of three frames, and supplies the detected motion vector to the spatiotemporal majority filter unit 25 and the lossless shape encoding unit 30.

The spatiotemporal majority decision filter unit 25 is supplied with three frames of labeled images from the labeled encoding unit 21, the frame memory 22, and the frame memory 23. For convenience of description, one frame of labeled image supplied from the frame memory 23 is supplied from the previous frame, and one frame of labeled image supplied from the frame memory 22 is supplied from the central frame and labeled image forming unit 21. One frame of the labeled image is called a subsequent frame.

The spatio-temporal majority filter unit 25 sets a window using the motion vector supplied from the motion vector detection unit 24, performs spatio-temporal majority filter processing on each pixel of the central frame, and performs the spatio-temporal majority filter processing. The applied center frame, in other words, the labeled image is supplied to the shape information generating unit 26.

Here, the processing of the spatiotemporal majority filter unit will be described with reference to FIGS. FIG. 4 is a diagram illustrating an example of processing of a spatiotemporal majority filter unit for a certain pixel. For example, windows 41, 42,
Reference numeral 43 is formed on the front frame, the center frame, and the rear frame.

The pixel of interest 45 is a pixel for which a spatio-temporal majority filtering process is performed. The window 42 is a 5 × 5 window centered on the pixel 45 of interest. The motion vector 47 is supplied from the motion vector detection unit 24, and indicates a corresponding position of the target pixel 45 in the previous frame. The pixel 44 is a pixel 4 of interest based on the motion vector 47.
5 is a pixel in the previous frame associated with 5. The window 41 is a 5 × 5 window centered on the pixel 44.

The motion vector 48 is the motion vector detector 2
4 and indicates the corresponding position in the subsequent frame of the target pixel 45. The pixel 46 is a pixel in the subsequent frame associated with the pixel of interest 45 by the motion vector 48. The window 43 is a 5 × 5 window centered on the pixel 46.

The spatiotemporal majority filter unit 25 finds the label with the largest number among the labels assigned to the 75 pixels included in the three windows 41, 42, and 43, and replaces the label of the target pixel 45 with the label with the largest number. Replace with
FIG. 5 is a diagram illustrating an example of a process of replacing the label of the target pixel with the label having the largest number.

Each of the windows 41, 42, and 43 represents 25 pixels. Numerals 1 to 3 described in a matrix in the windows 41, 42, and 43 represent labels, and pixels having the same label belong to the same object. For example, in the case of FIG.
The number of labels “1” included in 2, 43 is 35, the number of labels “2” is 18, and the number of labels “3” is 22.

Therefore, the three windows 41, 4
The label with the largest number among the labels assigned to the 75 pixels included in the pixels 2 and 43 is the label “1”, and the label “2” of the target pixel 45 is rewritten to the label “1”. As a result, when the spatio-temporal majority filtering process is sequentially performed on a frame, minute changes in the object shape that are spatially or spatio-temporally small are removed.

Returning to FIG. 3 again, the spatiotemporal majority filter unit 25 supplies the labeled image subjected to the spatiotemporal majority filter processing to the shape information generating unit 26. The shape information conversion unit 26 converts the supplied labeled image into object shape information, and outputs the object shape information to the lossless shape encoding unit 30.

The lossless shape encoder 30 receives the motion vector from the motion vector detector 24 and the object shape information from the shape information generator 26. The lossless shape encoding unit 30 performs lossless shape encoding of the object shape information using the motion vector, and outputs an encoded bit sequence.

When both the shape encoding for performing the encoding using the correlation in the time direction and the spatio-temporal shape noise removal filter process perform the motion correction process, if both use the same motion vector, The noise removal effect of the typical shape noise removal filter processing increases. Therefore, the motion vector detection unit 24 supplies the motion vector to the spatiotemporal majority filter unit 25 and the lossless shape encoding unit 30.

In this embodiment, an example in which a spatio-temporal majority filter of three frames is used has been described. However, by increasing or decreasing the number of frame memories in the shape noise elimination filter section 20, a spatio-temporal majority decision filter using more frames is used. The filter can be changed to a spatial majority filter in one frame. Note that the spatial majority filter processing in one frame sets an N × N window centering on each pixel, and replaces the label of the center pixel with the label with the largest number in the window. It is also possible to replace the spatio-temporal majority filter unit 25 with a pixel having no other label or a noise removal filter in which a plurality of labels are not assigned to one pixel.

[0046]

As described above, according to the present invention, a small change included in the shape information is removed by the shape noise removing filter before the shape information is reversibly shaped. That is, since a minute change included in the shape information can be removed before shape coding, the amount of information in shape coding can be reduced. In addition, since a noise removal filter is used in which a pixel having no label or a plurality of labels is not assigned to one pixel, there is no overlap or gap between objects. Therefore, with the shape encoding device of the present invention, shape encoding can be performed with a small code amount, and there is no overlap or gap between encoded objects.

Further, according to the present invention, it is possible to remove a minute change included in the shape information by using a spatio-temporal filter unit involving a motion correction process.

Further, according to the present invention, fine changes included in shape information can be removed by using a spatio-temporal majority filter process.

Further, according to the present invention, the motion vector used for the motion correction processing in the shape noise elimination filter unit and the motion vector used for the lossless encoding in the shape encoding unit are made the same, thereby eliminating the shape noise. The shape noise removal effect in the filter unit can be enhanced.

[0050]

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an example of unevenness existing at an outer edge portion of an object shape.

FIG. 2 is a diagram illustrating an example of a change in an object shape over time.

FIG. 3 is a configuration diagram of an embodiment of a shape encoding device according to the present invention.

FIG. 4 is a diagram illustrating an example of processing of a spatiotemporal majority filter unit for a certain pixel;

FIG. 5 is a diagram illustrating an example of a process of replacing a label of a target pixel with a label having the largest number.

FIG. 6 is a diagram illustrating an example of a process of dividing an image into object units.

[Explanation of symbols]

 1-9 Object 10 Shape Encoding Device 20 Shape Noise Elimination Filter Unit 21 Labeled Imaging Unit 22, 23 Frame Memory 24 Motion Vector Detection Unit 25 Spatio-Temporal Majority Decision Filter Unit 26 Shape Information Unit 30 Reversible Shape Encoding Unit 41 43 window 44,46 pixel 45 pixel of interest 47,48 motion vector

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yutaka Kaneko 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Corporation Japan Broadcasting Research Institute (72) Inventor Yoshiaki Kakuro 1-1-10 Kinuta, Setagaya-ku, Tokyo No. 11 Japan Broadcasting Corporation Broadcasting Technology Laboratory F-term (reference) 5B057 CA12 CA16 CB12 CB16 CE02 CE09 CE10 5C059 KK01 MA00 MB16 NN01 PP28 RC16 RC19 RC37 UA17 UA18 UA34

Claims (4)

[Claims] 1. A shape encoding device that divides an image into predetermined image region units and encodes shape information of the divided image regions, wherein a shape noise removal filter unit that removes a minute change included in the shape information. And a shape encoding unit that losslessly encodes the shape information from which the fine change has been removed. 2. The shape encoding apparatus according to claim 1, wherein the shape noise removal filter unit includes a spatiotemporal filter unit that performs a motion correction process. 3. A labeling image forming section for generating labeled image information in which the same label information is added to the pixels included in the same shape information, wherein the shape noise removing filter section includes: 3. The image processing apparatus according to claim 1, further comprising: a spatio-temporal majority filtering unit that performs a spatial majority filtering process; and a shape information generating unit that converts the labeled image information subjected to the spatio-temporal majority filtering process into shape information. The shape encoding device according to claim 1. 4. A motion vector used for a motion correction process in the shape noise removal filter unit and a motion vector used for lossless coding in the shape encoding unit are set to be the same. The shape encoding device according to claim 1.