JP2002051336A - Image-coding apparatus and image-decoding apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image-coding apparatus and image-decoding apparatus, capable of efficiently coding and decoding the area information. SOLUTION: The image-coding apparatus for coding a plurality of clip motion pictures is provided with an area information encoding section for coding the area information of the motion pictures and an information generating section for generating information, showing that the motion pictures are represented by rectangles. The area information coding section codes information showing the sizes of the rectangles as area information of the component motion pictures represented by the rectangles, to integrate the information as the information showing that the motion pictures are represented by the rectangles.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention belongs to the field of digital image processing, and more particularly to an image encoding apparatus for encoding image data with high efficiency and an image decoding apparatus for decoding encoded data created by the image encoding apparatus. It concerns the device.

[0002]

2. Description of the Related Art In image coding, a method of combining different moving image sequences has been studied. Document "Image Coding Using Hierarchical Representation and Multiple Templates" (IEICE Technical Report)
IE94-159, pp99-106 (1995)) combines a moving image sequence as a background and a moving image sequence of component moving images as a foreground (for example, a human image or a video of a fish cut out by chroma key technology) to create a new image. A technique for creating a simple sequence is described.

[0003] Further, a document "Temporal Scalability based on image conte
nt ", ISO / IEC / JTC1 / SC29 / WG11 MPEG95 / 211 (1995)) creates a new sequence by combining a low-frame-rate video sequence with a high-frame-rate component video sequence. The method is described.

In this method, as shown in FIG. 16, predictive coding is performed at a low frame rate in a lower layer,
In the upper layer, predictive coding is performed at a high frame rate only for the selected area (shaded area). However, the frame encoded in the lower layer is not encoded in the upper layer, and the decoded image of the lower layer is copied and used as it is. Further, it is assumed that a part that attracts the viewer's attention, such as a person part, is selected as the selection area.

FIG. 8 shows a block diagram of a conventional method. First, on the encoding side according to the conventional method, the input moving image is frame-thinned by the first frame dropping unit 801 and the second frame dropping unit 802, and the frame rate is set to be lower than the frame rate of the input image. Unit 803 and lower layer coding section 804. Here, the frame rate of the upper layer is equal to or higher than the frame rate of the lower layer.

[0006] The lower layer encoding section 804 encodes the entire input moving image. As the encoding method, for example, a moving image encoding international standardization method such as MPEG or H.261 is used. In the lower layer encoding unit 804, a decoded image of the lower layer is created and used for predictive encoding, and is input to the synthesizing unit 805 at the same time.

FIG. 9 is a block diagram showing a code amount control section in a conventional coding apparatus. In FIG. 9, an encoding unit 902 performs moving image encoding using motion compensation prediction, orthogonal transform, quantization, variable length encoding, and the like.

A quantization width calculation section 901 calculates a quantization width used in the encoding section 902 and generates a generated code amount calculation section 9.
03 calculates the accumulation of the encoded data. In general, when the generated code amount is large, the quantization width is controlled to suppress this, and conversely, when the generated code amount is small, the quantization width is controlled to be small.

In the upper layer coding section 803 of FIG. 8, only the selected area of the input moving picture is coded. Here, the moving picture coding international standardization method such as MPEG or H.261 is used, but only the selected area is coded based on the area information. However, frames encoded in the lower layer are not encoded in the upper layer.

The area information is information indicating a selected area such as a person, and is, for example, a binary image having a value of 1 at the position of the selected area and a value of 0 at other positions. Further, in the upper layer encoding unit 803, only the selected region of the moving image is decoded,
It is input to the synthesis unit 805.

[0011] In the area information encoding unit 806, the area information is encoded using an 8-way quantization code. As shown in FIG. 17, the eight-way quantization code indicates a direction to the next point by a numerical value, and is generally used when expressing a digital figure.

[0012] When the lower layer frame is encoded in the frame to be synthesized, the synthesizing unit 805 outputs a decoded image of the lower layer. When the lower layer frame is not encoded in the frame to be combined, a moving image is output using two encoded lower layer decoded images before and after the combined frame and one upper layer decoded image. .

Here, the frames of the two images of the lower layer are before and after the frame of the upper layer. Also,
The moving image created by the combining unit 805 is input to the upper layer encoding unit 803 and used for predictive encoding. Combiner 8
The image creation method in 05 is as follows.

First, two lower layer interpolation images are created. Let the decoded image of the lower layer at time t be B (x, y,
t) (where x and y are coordinates representing pixel positions in space), and the times of the two lower layers are t1, t2,
Assuming that the time of the upper layer is t3 (where t1 < t3 < t2), the interpolated image I (x, y, t3) at time t3
Is given by I (x, y, t3) = [(t2-t3) B (x, y, t1) + (t3-t1) B (x, y, t2)] / (t2-t1) (1) Is calculated.

Next, the decoded image E of the upper layer is synthesized with the interpolated image I obtained above. For this purpose, the area information M (x,
y, t) to create weight information W (x, y, t) for synthesis,
A composite image S is obtained by the following equation. S (x, y, t) = [1-W (x, y, t)] I (x, y, t) + E (x, y, t) W (x, y, t) (2) domain Information M (x, y, t) is 1 inside the selected area and 0 outside the selected area
The weight information W (x, y, t) can be obtained by applying a low-pass filter to the image a plurality of times.

That is, the weight information W (x, y, t) takes a value of 1 inside the selected area, 0 outside the selected area, and 0 to 1 at the boundary of the selected area. The above is the description of the image creating method in the combining unit 805.

The coded data coded by the lower layer coding section 804, the upper layer coding section 803, and the area information coding section 806 are integrated by a coded data integration section (not shown) and transmitted or stored.

Next, on the decoding side of the conventional method, the coded data is decomposed into coded data of a lower layer, coded data of an upper layer, and coded data of area information by a coded data decomposing unit (not shown). . These encoded data are decoded by a lower layer decoding unit 808, an upper layer decoding unit 807, and a region information decoding unit 809, as shown in FIG.

The combining unit 810 on the decoding side is composed of the same device as the combining unit 805 on the encoding side, uses the lower layer decoded image and the upper layer decoded image, and is the same as that described in the description on the encoding side. The images are combined by the method. The moving image synthesized here is displayed on the display,
It is input to the upper layer decoding unit 807 and used for prediction of the upper layer.

Here, the decoding device for decoding both the lower layer and the upper layer has been described. However, if the decoding device includes only the decoding unit for the lower layer, the upper layer encoding unit 807 and the combining unit 810 are unnecessary. Thus, a part of the encoded data can be reproduced with a small hardware scale.

[0021]

(1) In the prior art, when an output image is obtained from two lower layer decoded images and one upper layer decoded image as in the above equation (1), Since the interpolation of the two lower layers is performed, when the position of the selected area changes with time, a large distortion occurs around the selected area, and there is a problem that the image quality is largely deteriorated.

FIG. 18 illustrates this problem.
In FIG. 18A, images A and C are two decoded images of the lower layer, image B is a decoded image of the upper layer, and the display time order is A, B, C. However, the selected area is indicated by oblique lines. Also, since only the selected area is encoded in the upper layer, the area outside the selected area is indicated by a broken line.

Since the selected area is moving, images A and C
The interpolated image obtained from the above is an image in which the two selected regions overlap, as shown by the halftone dot portion in FIG. Further, when the image B is synthesized using the weight information, the output image is an image in which three selected regions overlap as shown in FIG.

In particular, around the selected area of the upper layer (outside)
Thus, the selected area of the lower layer appears as an afterimage, and the image quality is greatly deteriorated. As for the whole moving image, when only the lower layer is displayed, there is no such distortion, and when a composite image of the upper layer and the lower layer is displayed, the distortion appears. Then, the image quality is extremely deteriorated.

(2) In the conventional technique, an 8-way quantization code (FIG. 17) is used to encode region information.
When applied to a low bit rate or when the shape of an area becomes complicated, the ratio of the data amount of the area information to the entire coded data amount increases, which causes a problem of deteriorating the image quality.

(3) In the prior art, the weight information is obtained by applying a low-pass filter to the area information a plurality of times. However, since the filtering operation is performed a plurality of times, the processing amount increases. is there.

(4) In the prior art, predictive coding is used. However, even when there is a scene change in a lower layer, predictive coding may be used, resulting in large distortion. Since the distortion in the lower layer propagates to the upper layer, there is a problem that the distortion is maintained for a long time.

(5) In the prior art, since the moving picture coding international standardization scheme such as MPEG or H.261 is used in the lower layer, there is not much difference in image quality between the selected area and other areas. . On the other hand, in the upper layer, only the selected area is coded with high image quality, so that the image quality in the selected area changes with time, and this is detected as flicker-like distortion.

The object of the present invention is to solve these problems,
It is an object of the present invention to provide an image encoding device and an image decoding device that reduce the amount of data after encoding while not deteriorating the quality of a decoded image.

[0030]

An image coding apparatus according to the present invention is an image coding apparatus for coding a plurality of component moving images, and includes an area information code for coding region information of the component moving images. Encoding means, and information generating means for generating information indicating that the component video is represented by a rectangle, the area information encoding means, as the area information of the component video represented by the rectangle, The information indicating the size of the rectangle is encoded and integrated with information indicating that the component moving image is represented by a rectangle.

An image decoding apparatus according to the present invention receives encoded data including data obtained by encoding region information of a plurality of component moving images and information indicating that the component moving images are represented by rectangles. An image decoding apparatus for decoding the plurality of component moving images, comprising: area information decoding means for decoding area information of the component moving image, wherein the area information decoding means is configured to decode the component image represented by the rectangle. It is characterized in that information indicating the size of the rectangle is decoded as area information.

[0032]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The first embodiment of the present invention solves the problem that occurs in the synthesizing unit 805 in FIG. That is, the present invention relates to an image synthesizing apparatus that does not generate distortion such as an afterimage around a selected region of an upper layer when an image is synthesized from two lower layer decoded images.
FIG. 1 is a block diagram showing an image synthesizing apparatus according to the first embodiment.

In FIG. 1, a first area extracting unit 101
Are the first area information of the lower layer and the second area information of the lower layer.
The region that is the first region and is not the second region is extracted from the region information. In FIG. 10A, if the first area information is represented by a dotted line (the inside of the dotted line has a value of 0 and the outside of the dotted line has a value of 1), and similarly, the second area information is represented by a broken line. The region extracted by the first region extraction unit 101 is a hatched portion in FIG.

The second area extracting unit 102 calculates the first area information of the lower layer and the second area information of the lower layer from the first area information.
An area that is the second area and is not the first area is extracted. In the case of FIG. 10A, a halftone dot portion is extracted.

The controller 103 controls the switch 104 based on the outputs of the first area extraction unit 101 and the second area extraction unit 102. That is, when the target pixel position is only in the first region, the switch 104 is connected to the second decoded image side, and when the target pixel position is only in the second region, the switch 104 is connected to the first decoded image. Side, and in other cases, the switch 104 is connected to the output from the interpolation image creation unit 105.

The interpolated image creating unit 105 calculates an interpolated image of the first decoded image of the lower layer and the second decoded image of the lower layer according to the above-described equation (1). Here, in equation (1), B (x, y, t1) is the first decoded image,
B (x, y, t2) is the second decoded image, I (x, y, t3) is the interpolated image, and t1, t2, and t3 are the first decoded image, the second decoded image, and the interpolated image, respectively. It's time.

As described above, since an image is created,
For example, in the case of FIG. 10 (a), the second decoded image is used in the shaded area, so the background pixel outside the selected area appears, and in the halftone dot area, the first decoded image is used. Of the first decoded image and the interpolated image of the second decoded image appear in other portions.

Since the decoded image of the upper layer is superimposed on the image created in this way by the weighted averaging unit 106 in FIG. 1, the synthesized image is selected as shown in FIG. There is no afterimage around the shaded area) and an image with little distortion can be obtained. The weighted averaging unit 106 in FIG. 1 combines the above interpolated image and the decoded image of the upper layer by weighted averaging. Since the synthesizing method has been described in the above-mentioned prior art, the description is omitted here.

In the above-described first embodiment, the interpolation image creating unit 105 shown in FIG. 1 is provided.
Of the decoded image B (x, y, t1) and the second decoded image B (x, y, t2), the pixel value of the decoded image temporally closer to t3 which is the time of the upper layer is used. Is also good.

In this case, using the frame number of each image, when t3-t1 <t1-t2, I (x, y, t3) = B (x, y, t1). Is I (x, y, t3) = B (x, y, t2).

Here, t1, t2, and t3 are the times of the first decoded image, the second decoded image, and the decoded image of the upper layer, respectively.

Next, a second embodiment of the present invention will be described.
The present embodiment relates to an image synthesizing apparatus that more accurately synthesizes an image in consideration of motion information of a decoded image of a lower layer in the image synthesizing apparatus of the first embodiment. FIG.
FIG. 3 is a block diagram showing a device for estimating a motion parameter and transforming two decoded images and corresponding two pieces of area information.

In FIG. 2, the motion parameter estimating section 20
In step 1, motion information from the first decoded image to the second decoded image in the lower layer is estimated. For example, a motion vector is obtained for each block, or a motion (parallel movement, rotation, enlargement / reduction, etc.) of the entire image is obtained and used as a motion parameter.

In the transformation unit 202, the first decoded image, the second decoded image
, The first region information and the second region information are transformed based on the estimated motion parameters based on the temporal position of the synthesis target frame. For example, it is assumed that a motion vector (MVx, MVy) from the first decoded image to the second decoded image is obtained as a motion parameter. here,
MVx is the horizontal component of the motion vector, and MVy is the vertical component of the motion vector.

At this time, a motion vector from the first decoded image to the interpolated image is calculated by (t3-t1) / (t2-t1) (MVx, MVy), and the first decoded image is converted to this motion vector. Shift. When rotation, enlargement / reduction, or the like is used as a motion parameter, deformation is involved instead of a simple shift.

In FIG. 2, the transformed data are represented by a, b, c, and d, respectively, where a is the first data in FIG.
, B is the second decoded image in FIG. 1, and c is FIG.
Are input to the image synthesizing apparatus shown in FIG. 1 as the first area information in FIG. 1 and the second area information in FIG.

In the second embodiment, the motion parameters are estimated from two decoded images. However, at the time of predictive encoding, the motion vector of each block of the image is included in the encoded data. Since these are general, these motion vectors may be used.

For example, the average value of the decoded motion vectors is used as the motion vector of the entire image from the first decoded image to the second decoded image, or the frequency distribution of the decoded motion vectors is obtained. A vector having a high frequency may be used as a motion vector of the entire image from the first decoded image to the second decoded image. The above processing is performed independently in the horizontal and vertical directions.

Next, a third embodiment of the present invention will be described.
The third embodiment relates to an area information encoding device that efficiently encodes area information. 3 and 4 are block diagrams of the present embodiment. FIG. 3 shows the encoding side, and FIG. 4 shows the decoding side.

The area information approximating unit 301 in FIG. 3 approximates the area information with a plurality of figures. FIG. 11 shows an example of approximation. In this example, a rectangle is used as a figure, and the area information (hatched portion) of a person is approximated by two rectangles. The rectangle 1 represents the head of the person, and the rectangle 2 represents the chest of the person.

The area approximation information encoding unit 302 encodes the approximated area information. As shown in FIG. 11, when approximated by rectangles, the upper left coordinate value of each rectangle and the size of the rectangle may be encoded with a fixed length.
Alternatively, when approximated by an ellipse, the coordinates of the center point of the ellipse, the length of the major axis, and the length of the minor axis may be encoded with a fixed length. The approximated area information and the encoded data are:
Sent to selection section 304.

The region information encoding unit 303 does not approximate the region information and encodes using an eight-way quantization code, similarly to the region information encoding unit 806 described in the above prior art. The area information and the encoded data are sent to the selection unit 304.

The selection unit 304 is a region approximation information encoding unit 3
02 or the output of the area information encoding unit 303. When the output of the region approximation information encoding unit 302 is selected, the coded data of the region approximation information is sent to a coded data integration unit (not shown) together with 1-bit selection information (for example, 0), and the region approximation information is shown. Not sent to the synthesis unit.

When the output of the area information encoding unit 303 is selected, the encoded data of the area information that is not approximated is set to 1
Along with the bit selection information (for example, 1), the information is sent to an unillustrated coded data integration unit, and the non-approximate region information is sent to the image synthesis unit. The image synthesizing unit has been described in the first embodiment and the second embodiment of the present invention.

As a selection method in the selection unit 304,
For example, a method of selecting the smaller coded data amount, or the output of the region information coding unit 303 is selected when the coded data amount of the region information that is not approximated is within a certain threshold, and when the coded data amount exceeds the threshold, the region approximate information coding is performed. The output of the unit 302 is selected. By making such a selection, the amount of coded data can be reduced while suppressing the coding distortion of the area information.

Next, the decoding side (FIG. 4) of the third embodiment of the present invention will be described. Referring to FIG.
Selects whether the coded data is based on the area approximation information or based on the area information based on the 1-bit selection information included in the coded data.

The area approximation information decoding section 402 decodes the area approximation information. The area information decoding unit 403 decodes area information that is not approximated. The switch 404 is connected to the selection unit 40
1 and selects area approximation information or non-approximate area information as an output to the synthesis unit.

As described above, since the area approximation information and the area information which is not approximated are adaptively selected and encoded / decoded, when the area information is complicated and has a huge data amount, The encoding of the area approximation information is selected, and the area information can be encoded with a small amount of information.

In the above example, the area information that is not approximated is 8
Although the encoding is performed using the direction quantization code, the encoding may be performed efficiently by further combining predictive encoding. The eight-way quantization code has a value of 0 to 7 as shown in FIG. 17, but if a difference is obtained by predictive coding, it becomes -7 to 7.

However, the difference value can be suppressed to -3 to 4 by subtracting 8 when the difference value is less than -4 and subtracting 8 when the difference value is greater than 4. At the time of decoding, the difference value is added to the previous value, and if the result is negative, 8 is added.
If it exceeds 7, the original 8-way quantized value can be obtained by subtracting 8.

An example is shown below. 8-way quantization value 1, 6, 2, 1, 3, ... Difference value 5, -4, -1, -2, ... Conversion value -3, 4, -1, 2, ... Decoding Value 1, 6, 2, 1, 3, ... For example, the difference between the value 6 and the previous value is 5,
Is subtracted to obtain -3. At the time of decoding, -2 is obtained by adding the previous value 1 to the decoded value -3. However, since the value is negative, 8 is added to this to obtain the decoded value 6. Such predictive coding utilizes the property that an 8-way quantization code is cyclic.

In the third embodiment, the coding of the approximated area information is performed independently for each picture. However, since the moving picture generally has a high correlation between frames, the previous coding result is Utilization may be used to increase the coding efficiency.

That is, when encoding of the approximated area information is continuous between frames, only the difference of the approximated area information is encoded. For example, the area is approximated by a rectangle, and the rectangle of the previous frame is the upper left point: (10, 20), size:
(100, 150), and the rectangle of the current frame is the upper left point: (1
3, 18), size: (100, 152), in the current frame, the upper left difference value: (3, 2), the size difference value: (0, 2)
Is encoded.

When the change in the shape of the area is small, all the difference values are concentrated around 0. Therefore, if entropy coding such as Huffman coding is used, the code amount of the area information can be greatly reduced. Furthermore, when the rectangle does not change in many cases, 1-bit information in the current frame may be encoded as rectangular change information.

That is, when the rectangle does not change, only 1-bit information (for example, 0) representing this is coded, and when the rectangle changes, 1-bit information (for example, 1) and the above difference information are coded. I do.

Next, a fourth embodiment of the present invention will be described.
The fourth embodiment relates to a weight information creating device that creates multi-value weight information from region information. FIG. 5 is a block diagram of the present embodiment.

Referring to FIG. 5, a horizontal direction weight creating unit 501 is provided.
Scans the area information in the horizontal direction, finds a portion where the area information is 1, and finds a weighting function corresponding to it. Specifically, the coordinates x0 of the left end point of the area and the horizontal length N of the area
And a horizontal weighting function as shown in FIG. 12A is calculated.

The weight function may be created by combining straight lines, or may be created by combining straight lines and trigonometric functions. As an example of the latter, when the width of the trigonometric function part is W,
If N> 2W, when 0 ≦ x <W sin [(x + 1/2) π / (2W)] * sin [(x + 1/2)
π / (2W)] When W ≦ x <NW 1 When NW ≦ x <N sin [(x-N + 2W + 1/2) π / (2W)] * sin [(x
-N + 2W + 1/2) π / (2W)] If N ≤ 2W, sin2 [(x + 1/2) π / N] * sin [(x + 1/2) π
/ N] can be used. However, the point x0 at the left end of the area is 0
And

The vertical weight creating section 502 scans the area information in the vertical direction to obtain a portion where the area information is 1, and obtains a corresponding vertical weight function. Specifically, the coordinates y0 of the upper end point of the region and the vertical length M of the region are obtained, and a vertical weighting function as shown in FIG. 12B is calculated.

The multiplier 503 is provided for the horizontal weight generator 50.
1 is multiplied by the output of the vertical weight generator 502 for each pixel position to generate weight information. By creating the weight information in this manner, the weight information that matches the shape of the area information can be obtained with a small amount of calculation.

Next, a fifth embodiment of the present invention will be described.
The fifth embodiment relates to a mode switching method for adaptively switching between intra-frame coding and inter-frame predictive coding in predictive coding of a lower layer or an upper layer. FIG. 6 is a block diagram of the present embodiment.

In FIG. 6, an average value calculation unit 601 receives an original image and area information, and calculates an average of pixel values for pixel values inside the area. The average value is input to the differentiator 603 and the storage unit 602. The differentiator 603 calculates a previous average value and an average value calculation unit 601 stored in the storage unit 602.
Calculates the difference from the current average value output from.

The determination section 604 compares the absolute value of the difference calculated by the differentiator 603 with a predetermined threshold and outputs mode switching information. If the absolute value of the difference value is larger than the threshold, it is determined that there is a scene change in the selected area, and mode switching information is generated so that intra-frame encoding is always performed.

As described above, by performing the mode switching while determining the scene change of the selected area, a good coded image can be obtained even when, for example, a person appears from a shadow or the front and back of an object is reversed. Obtainable.

This embodiment can be applied to a method of performing encoding by separating a selected area and other areas in encoding of a lower layer. In that case, the area information is input to the lower layer. Further, the present embodiment can be applied to encoding of only the selected region of the upper layer.

Next, a sixth embodiment of the present invention will be described.
The sixth embodiment relates to data amount control in a case where a selected area and another area are separately encoded in encoding of a lower layer. FIG. 7 is a block diagram of the present embodiment.

In FIG. 7, an encoding unit 703 encodes a selected area and other areas separately. The region information is input to the region determining unit 701, and it is determined whether the region to be coded is within the selected region or outside the selected region. The generated code amount calculation unit 705 calculates the generated code amount in each area based on the determination result.

The target code amount distribution ratio calculating section 704 determines a target code amount distribution ratio for each frame to be allocated to each area. The method for determining the distribution ratio will be described later. The quantization width calculation unit 702 determines the quantization width according to the target code amount, and the method for this determination is the same as the conventional method.

Here, a method of determining the distribution ratio in the target code amount distribution ratio calculating section 704 will be described. First, the target code amount Bi of the frame is calculated using the following equation. Bi
= (Available code amount-used code amount up to the previous frame) / number of remaining frames This target code amount Bi is allocated to the selected area inside and outside the selected area at a certain ratio, but here an appropriate fixed ratio R0 and Using the previous frame complexity ratio Rp, the ratio is determined.

The previous frame complexity ratio Rp is determined by the following equation. Rp = (gen # bitF * avg # qF) / (gen # bitF * avg # qF + gen # bitB * avg
#qB) where gen # bitF: code amount generated in the previous frame selection area gen # bitB: code amount generated outside the previous frame selection area avg # qF: average quantization width in the selection area of the previous frame avg # qB: previous frame Is the average quantization width outside the selected region of.

In order to improve the quality of the selected area, the quantization width is controlled so that the average quantization width in the selected area is smaller than the average quantization width outside the selected area to some extent. It is desirable to follow the change of the image at the time.

In general, the distribution by the fixed ratio R0 is suitable for keeping the relation of the average quantization width between the inside and outside of the selected region almost constant, and the distribution by the previous frame complexity ratio Rp is within the moving image sequence. It is suitable to follow the change of the image at the time.

Therefore, in the present invention, the target code amount distribution ratio Ra
Is the average of the fixed ratio R0 and the previous frame complexity ratio Rp, and control having both advantages is performed. That is, Ra = (R0 + Rp) / 2.

For example, it is assumed that the fixed ratio R0 and the previous frame complexity ratio Rp in the selected area are as shown by the dotted line in FIG. 13 in the entire moving image sequence. At this time, the target code amount distribution ratio Ra is as shown by the solid line in FIG. 13, and it can be seen that the target code amount distribution ratio does not deviate much from the fixed ratio R0 and reflects a change in the image to some extent.

At this time, the fixed ratio outside the selected area is (1-R0),
Assuming that the previous frame complexity ratio is (1-Rp), the target code amount distribution ratio (1-Ra), which is the average of the two, is as shown by the solid line in FIG. The value obtained by adding the target code amount distribution ratio is 1.

In this way, the quantization width can be appropriately controlled. However, since the generated code amount may exceed the target code amount Bi depending on the frame, the bit rate of the entire sequence is reduced to a predetermined bit rate. Things that don't fit can happen. In such a case, the following method can be used.

As described above, the target code amount distribution ratio Ra in the selected area is an average of the fixed ratio R0 and the previous frame complexity ratio Rp. On the other hand, the target code amount distribution ratio outside the selected region is a fixed ratio outside the selected region (1-R0) and a previous frame complexity ratio (1-Rp).
With the minimum value Rm.

In this way, the change of the target code amount distribution ratio Rm outside the selected area is, for example, as shown by a solid line in FIG. At this time, since Ra + Rm ≦ 1, the target code amount of the frame can be reduced. That is, by suppressing the target code amount in the background area, the bit rate of the entire moving image sequence can be kept at a predetermined bit rate.

[0089]

According to the image encoding device and the image decoding device of the present invention, the information indicating the size of the rectangle is encoded as the region information of the component moving image represented by the rectangle, so that the efficiency is improved. Region information can be encoded / decoded.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a second embodiment of the present invention.

FIG. 3 is a block diagram illustrating an encoding side according to a third embodiment of the present invention.

FIG. 4 is a block diagram illustrating a decoding side according to a third embodiment of the present invention.

FIG. 5 is a block diagram illustrating a fourth embodiment of the present invention.

FIG. 6 is a block diagram illustrating a fifth embodiment of the present invention.

FIG. 7 is a block diagram illustrating a sixth embodiment of the present invention.

FIG. 8 is a block diagram illustrating a conventional encoding device and decoding device.

FIG. 9 is a block diagram illustrating a conventional code amount control unit.

FIG. 10 is a diagram illustrating the effect of the first embodiment of the present invention.

FIG. 11 is an explanatory diagram showing an example of approximating region information according to a third embodiment of the present invention.

FIG. 12 is an explanatory diagram illustrating an example of a method for creating weight information according to a fourth embodiment of the present invention.

FIG. 13 is a diagram illustrating a target code amount ratio of a selected area according to a code amount control method according to a sixth embodiment of the present invention.

FIG. 14 is a diagram illustrating a target code amount ratio outside a selected area according to a code amount control method according to a sixth embodiment of the present invention.

FIG. 15 is a diagram illustrating a target code amount ratio outside a selected area according to another code amount control method according to the sixth embodiment of the present invention.

FIG. 16 is a diagram illustrating the concept of conventional video hierarchical coding.

FIG. 17 is a diagram illustrating an 8-way quantization code.

FIG. 18 is a diagram illustrating a problem of conventional video hierarchical coding.

[Explanation of symbols]

 101 first region extraction unit 102 second region extraction unit 103 controller 104 switch 105 interpolation image creation unit 106 weighted average unit 301 region information approximation unit 302 region approximation information encoding unit 303 region information encoding unit 304 selection unit 401 selection Unit 402 region approximation information decoding unit 403 region information decoding unit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Norio Ito 22-22 Nagaikecho, Abeno-ku, Osaka City, Osaka Inside Sharp Corporation (72) Inventor Toshio Nomura 22-22 Nagaikecho, Abeno-ku, Osaka City, Osaka Incorporated F term (reference) 5C059 KK01 LB11 MA04 MA05 MA35 MB02 MB04 MB12 MB21 MC11 NN21 PP04 TA23 TC10 TC14 TC38 TD03 TD11 UA02 UA05 5J064 AA01 BA09 BB01 BB03 BC16 BC21 BC25

Claims (2)

[Claims] 1. An image encoding apparatus for encoding a plurality of component moving images, comprising: area information encoding means for encoding region information of the component moving image; and wherein the component moving image is represented by a rectangle. Information generating means for generating information indicating the size of the rectangle, as the area information of the component moving image represented by the rectangle, encodes information indicating the size of the rectangle, An image encoding apparatus, which integrates information indicating that a moving image is represented by a rectangle. 2. Inputting encoded data including data obtained by encoding region information of a plurality of component moving images and information indicating that the component moving images are represented by rectangles, An image decoding device that decodes region information of the part moving image, the region information decoding unit decodes the region information of the rectangle as the region information of the component image represented by the rectangle. An image decoding device for decoding information indicating a size.