JP3247836B2 - Image encoding device and image decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates belongs to the field of digital image processing, decodes the encoded data generated by the video encoding apparatus and the moving picture coding apparatus for coding image data with high efficiency The present invention relates to a video decoding device.

[0002]

2. Description of the Related Art In image coding, a method of superimposing different moving image sequences has been studied.

[0003] For example, a document "Image coding using hierarchical representation and multiple templates" (IEICE IE94-159, IEICE Tech.
pp99-106 (1995)), a new sequence is created by superimposing a moving image sequence as a background and a moving image sequence of a component moving image as a foreground (for example, a human image or a video of a fish cut out by the chroma key technique). An approach to doing so is described.

[0004] In addition, the document "Time hierarchical coding based on image contents"("TemporalScalability")
based on image content ",
ISO / IEC / JTC1 / SC29 / WG11 M
PEG95 / 211 (1995)) describes a method of creating a new sequence by superimposing a moving image sequence of a component moving image having a high frame rate on a moving image sequence having a low frame rate.

In this method, as shown in FIG. 10, predictive coding is performed at a low frame rate in a lower layer, and predictive coding is performed at a high frame rate only in a selected region (shaded area) in an upper layer. However, the frame encoded in the lower layer is not encoded in the upper layer, and the decoded image of the lower layer is copied and used as it is. Also, it is assumed that a part that attracts the viewer's attention, such as a person part, is selected as the selected area.

FIG. 8 shows a block diagram of a conventional method. First, on the encoding side of the conventional method, the input moving image is frame-thinned by the first frame dropping unit 801 and the second frame dropping unit 802, and the frame rate is set to be equal to or less than the frame rate of the input image. And to the lower layer coding section 804. Here, it is assumed that the frame rate of the upper layer is equal to or higher than the frame rate of the lower layer.

[0007] The lower layer coding section 804 codes the entire input moving image. Examples of the encoding method include MPEG and H.264. H.261 or the like is used. In lower layer coding section 804,
A decoded image of the lower layer is created, used for predictive coding, and input to the superimposing unit 805 at the same time.

[0008] In the upper layer coding section 803 of FIG. 8, only the selected area of the input moving image is coded. Again, MPEG and H.264. Although a moving picture coding international standardization method such as H.261 is used, only a selected region is coded based on region shape information. However, frames encoded in the lower layer are not encoded in the upper layer. The area shape information is information indicating the shape of a selected area such as a person part. For example, a value 1 is set at a position of the selected area, and a value 0 is set at other positions.
It is a value image. In the upper layer coding section 803, only the selected area of the moving image is decoded and input to the superimposing section 805.

The region shape encoding unit 806 sets the region shape to 8
It is encoded using a direction quantization code. As shown in FIG. 11, the 8-way quantization code indicates the direction to the next point by a numerical value, and is generally used when representing a digital figure.

[0010] The superimposing section 805 outputs a decoded image of the lower layer at the frame position where the lower layer frame is encoded. At a frame position where the lower layer frame is not encoded, an image is created using two encoded lower layer decoded images before and after the target frame and one upper layer decoded image at the same time as the target frame. Output.
The image created here is input to the upper layer encoding unit 803 and used for predictive encoding. The image creation method in the superimposing unit 805 is as follows.

First, two lower layer interpolated images are created. The decoded image of the lower layer at time t is B
(X, y, t) (where x and y are coordinates representing pixel positions in space), the times of the two lower layers are t1 and t2, respectively, and the times of the upper layers are t3 (where t is t).
1 <t3 <t2), the interpolated image I (x, y, t3) at time t3 is I (x, y, t3) = [(t2-t3) B (x, y, t1) + (t3 -t1) B (x, y, t2)] / (t2-t1) (1)

Next, the decoded image E of the upper layer is superimposed on the interpolated image I obtained above. For this purpose, the area shape information M
From (x, y, t), weight information W (x, y,
t) is created, and a superimposed image S is obtained by the following equation.

S (x, y, t) = [1-W (x, y, t)] I (x, y, t) + E (x, y, t) W (x, y, t) (2 The region shape information M (x, y, t) is a binary image having a value of 1 inside the selected region and a value of 0 outside the selected region. The weight information W is obtained by applying a low-pass filter to the image a plurality of times. (X,
y, t) can be obtained.

That is, the weight information W (x, y, t) is 1 in the selected area, 0 outside the selected area, and 0 to 0 at the boundary of the selected area.
Takes a value of 1. The above is the description of the image creating method in the superimposing unit 805.

The coded data coded by the lower layer coding section 804, the upper layer coding section 803, and the area shape coding section 806 are integrated by a coded data integration section (not shown) and transmitted or stored.

Next, on the decoding side of the conventional method, the encoded data is decomposed into encoded data of a lower layer, encoded data of an upper layer, and encoded data of an area shape by an encoded data decomposing unit (not shown). These encoded data are decoded by the lower layer decoding unit 808, the upper layer decoding unit 807, and the area shape decoding unit 809 as shown in FIG.
The decoding side superimposing section 810 is composed of the same device as the encoding side superimposing section 805, and uses the lower layer decoded image and the upper layer decoded image to superimpose the image by the same method as described in the encoding side. Is done. The superimposed moving image is displayed on the display and the upper layer decoding unit 8
07 and used for prediction of an upper layer.

Here, the decoding device for decoding both the lower layer and the upper layer has been described. However, if the decoding device includes only the decoding unit for the lower layer, the decoding device for the upper layer 80
7. The superimposition unit 810 is unnecessary, and a part of the encoded data can be reproduced with a small hardware scale.

[0018]

In the prior art, when an output image is obtained from two lower layer decoded images and one upper layer decoded image as shown in equation (1), interpolation of two lower layers is performed. When the position of the selected area changes with time, large distortion occurs around the selected area,
There is a problem that the image quality is greatly deteriorated.

FIG. 12 illustrates this problem.
In FIG. 12A, images A and C are two decoded images of the lower layer, image B is a decoded image of the upper layer, and the display time order is A, B, and C. However, the selected area is indicated by oblique lines. In the upper layer, only the selected area is encoded, so that the area outside the selected area is indicated by a broken line. Since the selected area is moving, the interpolated image obtained from the image A and the image C has two selected areas overlapping each other as shown by a halftone dot portion in FIG.

Further, when the image B is superimposed using the weight information, the output image is an image in which three selected areas overlap as shown in FIG. In particular, the selected area of the lower layer appears as an afterimage around (outside) the selected area of the upper layer, and the image quality is greatly deteriorated. When only the lower layer is displayed as the whole moving image, there is no such distortion,
When the superimposed image of the upper layer and the lower layer is displayed, the above-mentioned distortion appears, so that flicker-like distortion occurs and the image quality is extremely deteriorated.

An object of the present invention is to solve these problems and to provide an encoding apparatus and a decoding apparatus that reduce the amount of data after encoding while not deteriorating the quality of a decoded image.

[0022]

In order to achieve the above object, the present invention provides: (1) lower layer encoded data obtained by encoding pixel data of a moving image sequence at a first frame rate; Pixel data of a specific region of the sequence is encoded at a second frame rate higher than the first frame rate, and upper layer encoded data obtained by encoding the region shape of the specific region of the video sequence is input. ,
A moving image decoding device that outputs a lower layer decoded image or outputs a superimposed image of a lower layer decoded image and an upper layer decoded image, and decodes the lower layer encoded data at the first frame rate. A lower layer decoding unit, a shunt unit that shunts the upper layer coded data into coded data of pixel data and coded data of an area shape, and coded data of the pixel data shunted by the shunt unit. A pixel data decoding unit for decoding at a second frame rate; and a region shape decoding unit for decoding encoded data of the region shape divided by the division unit, wherein the encoded data of the region shape is an upper layer frame. If there is no lower layer frame corresponding to the same frame position as the first frame, the first
And encoded data of a first area shape and a second area shape indicating a specific area in the second lower layer frame, and the first area shape and the second area shape, and the first and second area shapes. Using the lower layer frame , the first
The second lower layer layer is used for an area having only the area shape.
Using the pixel value of the frame, the area of only the second area shape is used.
For the area, the pixel value of the first lower layer frame is
And combining the non-existent lower layer frames. (2) Lower layer encoded data obtained by encoding pixel data of a moving image sequence at a first frame rate, and a second frame rate higher than the first frame rate of pixel data of a specific area of the moving image sequence In addition, only at the frame position where the pixel data of the specific area is to be encoded, the upper layer encoded data obtained by encoding the area shape of the specific area of the moving image sequence is input, and the lower layer decoded image is encoded. A moving image decoding apparatus for outputting or outputting a superimposed image of the lower layer decoded image and the upper layer decoded image, wherein the lower layer decoding unit decodes the lower layer encoded data at the first frame rate. A diverter for diverting the upper layer encoded data into encoded data of pixel data and encoded data of an area shape; A pixel data decoding unit that decodes the encoded data of the pixel data divided by the division unit at the second frame rate, and an area shape decoding unit that decodes the encoded data of the region shape divided by the division unit When there is no lower layer frame corresponding to the same frame position as the upper layer frame in time, the first and second lower layer frames indicating the specific area in the first and second lower layer frames existing before and after the frame position in time 1 area shape and 2nd area
The area shape, the lower layer decoded image obtained we are in the lower layer decoding unit extracts by area division, and the first area shape and the second area shape, said first and second lower layer frame Using only the first region shape.
For the area, the pixel value of the second lower layer frame is
Used, and for the region having only the second region shape,
The non-existent lower layer frame is synthesized using the pixel value of the first lower layer frame . (3) Pixel data of a moving image sequence is converted to a first frame
Lower layer encoded data encoded at a rate,
Pixel data of a specific area of the image sequence is stored in the first file.
Encoding at a second frame rate higher than the frame rate
And a frame for encoding the pixel data of the specific area.
Only at the camera position,
Upper layer encoded data obtained by encoding the area shape of the area
Output the lower layer decoded image as input, or
Superimposed image of lower layer decoded image and upper layer decoded image
A video decoding device that outputs the lower layer code
For decoding encoded data at the first frame rate
A layer decoding unit, and
The data is divided into encoded data and encoded data of the area shape.
A shunt portion that flows, and pixel data that is shunted by the shunt portion.
An image for decoding encoded data at the second frame rate
A raw data decoding unit, and a region shape divided by the division unit.
And a region shape decoding unit for decoding the encoded data, and when there is no lower layer frame corresponding to the same frame position as the upper layer frame in time, the first and second frames existing before and after the frame position are provided. The first area shape and the second area shape indicating the specific area in the second lower layer frame
If the region shape is the region shape obtained by the region shape decoding unit,
Extracted by estimating using affine transformation from
The first and second region shapes, and the first and second region shapes;
2 lower layer frames and the first region shape
Only the area of the second lower layer frame.
Using the prime value, for the region having only the second region shape
Uses the pixel values of the first lower layer frame,
The non-existent lower layer frame is synthesized . (4) The pixel data of the moving image sequence is stored in the first frame
Lower layer encoded data encoded at a rate,
Pixel data of a specific area of the image sequence is stored in the first file.
Encoding at a second frame rate higher than the frame rate
And a frame for encoding the pixel data of the specific area.
Only at the camera position,
Upper layer encoded data obtained by encoding the area shape of the area
Output the lower layer decoded image as input, or
Superimposed image of lower layer decoded image and upper layer decoded image
A video decoding device that outputs the lower layer code
For decoding encoded data at the first frame rate
A layer decoding unit, and
The data is divided into encoded data and encoded data of the area shape.
A shunting part for flowing, and pixel data shunted by the shunting part
An image for decoding encoded data at the second frame rate
A raw data decoding unit, and a region shape divided by the division unit.
And a region shape decoding unit for decoding the encoded data, and when there is no lower layer frame corresponding to the same frame position as the upper layer frame in time, the first and second frames existing before and after the frame position are provided. The first area shape and the second area shape indicating the specific area in the second lower layer frame
The region shape is extracted by estimating from the region shape obtained by the region shape decoding unit using prediction by block matching , and the first region shape and the second region shape are extracted.
And using the first and second lower layer frames,
The second lower order is applied to an area having only the first area shape.
Using the pixel values of the layer frame, the second region shape
Only the region of the first lower layer frame
Using the pixel values, the non-existent lower layer frame
Is synthesized . (5) The moving picture decoding device according to any one of (1) to (4), further including a flag decoding unit that decodes encoded data of a flag indicating whether to synthesize the lower layer frame, and decoding by the flag decoding unit. Based on the flag set, the first
Using the region shape and the second region shape and the first and second lower layer frames to determine whether to synthesize the nonexistent lower layer frame, and performing the lower layer frame synthesis It is characterized by. (6) In the video decoding device according to (1), the upper layer decoding unit further includes a first lower layer frame in a first lower layer frame temporally preceding the upper layer frame.
A first flag decoding unit that decodes encoded data of a first flag indicating whether or not area information indicating an area shape has been encoded; and a second lower layer that exists temporally after the upper layer frame A second flag decoding unit that decodes encoded data of a second flag indicating whether or not area information indicating a second area shape in the frame has been encoded, wherein the first and second flag decoding are performed. First decrypted by
It is determined whether the first region shape and the second region shape are coded based on the flag and the second flag,
If both the first and second region shapes are not encoded, the region shapes used in the previous lower layer combination are used as the region shapes used in the current lower layer combination, respectively, and only the second region shape is encoded. If it is
The second area shape used in the previous lower layer synthesis is used as the first area shape used in the current lower layer synthesis, and the first area shape and the second area shape, and the first and second lower layers are used. The non-existent lower layer frame is synthesized using a frame. (7) a lower layer encoding unit that encodes the pixel data of the moving image sequence at a first frame rate, and a second frame rate that is higher than the first frame rate for the pixel data of a specific area of the moving image sequence And a higher layer coding unit for coding a region shape of a specific region of the moving image sequence, wherein the higher layer coding unit includes the moving image sequence. A pixel data encoding unit that encodes pixel data of a specific area at a second frame rate higher than the first frame rate; and an area shape encoding unit that encodes an area shape of a specific area of the moving image sequence And the encoded data of the pixel data encoded by the pixel data encoding unit and the encoded data of the area shape encoded by the area shape encoding unit. A multiplexing unit that overlaps, and when there is no lower layer frame corresponding to the same frame position as the upper layer frame in time, the area shape coding unit exists before and after the frame position in time. The first region shape and the second region shape indicating a specific region in the first and second lower layer frames to be encoded are encoded. (8) The video encoding device according to (7), further comprising a flag encoding unit that encodes a flag indicating whether or not the lower-layer frame is to be synthesized by the video decoding device. (9) In the video encoding device according to (7), the upper layer encoding unit further includes a region indicating a first region shape in a first lower layer frame temporally preceding the upper layer frame. A first flag encoding unit that encodes a first flag indicating whether or not information has been encoded, and a second region shape in a second lower layer frame that is temporally subsequent to the upper layer frame. A second flag encoding unit that encodes a second flag indicating whether or not the area information has been encoded.

[0023]

FIG. 1, FIG. 14 and FIG. 15 are block diagrams showing a first embodiment. In FIG. 14, the units other than the upper layer encoding unit 1403, the superimposing unit 1405, the upper layer decoding unit 1406, and the superimposing unit 1408 perform the same operations as in FIG.

The switch 101, the pixel data encoder 102, the area shape encoder 103, the pixel data decoder 104, the area information decoder 105, and the multiplexer 106 shown in FIG.
The higher layer coding section 1403 in FIG. 14 is configured. Also, the first delay unit 107, the second delay unit 108 in FIG.
First region extraction unit 109, second region extraction unit 110, controller 111, switch 112, interpolation image creation unit 1
13, the weighted average unit 114, the third delay unit 115, and the fourth
The delay unit 116 constitutes the superimposing unit 1405 or 1408 in FIG. Note that the overlapping unit 1408 has the same function as the overlapping unit 1405.

Hereinafter, the first embodiment will be described in detail with reference to FIG.

The switch 101 switches between a mode in which the pixel value of the upper layer is encoded and a mode in which the pixel value is not encoded by a controller (not shown). That is, when the lower layer frame corresponding to the same frame position as the upper layer is encoded, the switch is controlled to be turned off, and the pixel value of the upper layer is not encoded. On the other hand, if the lower layer frame corresponding to the same frame position as the upper layer has not been encoded, the switch is turned on to encode the pixel value of the upper layer.

The pixel data encoding unit 102 includes a switch 10
When 1 is on, the pixel data of the upper layer is encoded. As an encoding method, MPEG or H.264 is used. International standardization methods such as H.261 are used. However, at the time of encoding, the region shape is input from the region shape decoding unit 105 via a signal line (not shown), and only the pixel data in the region is encoded.

The region shape encoding unit 103 encodes the region shape by the same operation as the region shape encoding unit 806 in FIG. 8 described in [Prior Art].

The pixel data decoding unit 104 decodes the pixel data encoded by the pixel data encoding unit 102. Here also, the region shape is input from the region shape decoding unit 105 via a signal line (not shown) and used for decoding. The decoded pixel data is input to the third delay unit 115, and is fed back to the pixel data encoding unit via a signal line (not shown) to be used for prediction.

Area shape decoding section 105 decodes the area shape data encoded by area shape encoding section 103 and outputs the decoded data to first delay section 107.

As described above, in the upper layer encoding section of the present embodiment, the case where the upper layer is encoded by the switch 101 is controlled. Next, the superposition unit of the present embodiment will be described.

The first delay unit 107 converts the area shape data into a
Delay by frame. The delayed area shape data is input to the weighted averaging unit 114. The second delay unit 108 further delays the region shape data by b frames. The delayed region shape data is input to the first region extraction unit 109 and the second region extraction unit 110. In these circuits, the area shape data that has not been delayed is also input at the same time. Here, the symbols t + a, t, and tb on the signal lines represent the time of each frame. Further, t, a, and b are integers.

The first area extracting unit 109 extracts an area that is the second area and is not the first area from the first area information and the second area information. In the case of FIG. 9A, a halftone dot portion is extracted.

The first area extracting section 110 extracts an area that is the first area and is not the second area from the first area information and the second area information. In the case of FIG. 9A, a hatched portion is extracted.

The controller 111 controls the switch 112 based on the outputs of the first area extraction unit 109 and the second area extraction unit 110. That is, when the target pixel position is only in the first area, the switch 112 is connected to the decoded image side at the frame time (t + a), and when the target pixel position is only in the second area, the switch 112 is connected to the frame time ( The switch 112 is connected to the decoding side of t-b). In other cases, the switch 112 is connected to the output from the interpolation image creation unit 113.

The third delay unit 115 delays the decoded image data of the upper layer by a frame, and inputs the decoded image data at time t to the weighted average unit 114. Fourth delay unit 1
Reference numeral 16 delays the decoded image data of the lower layer by (a + b) frames and inputs the decoded image at time (t−b) to the interpolation image creation unit 113.

The interpolated image creating unit 113 calculates an interpolated image of the decoded image at the lower layer frame time (t−b) and the decoded image at the lower layer time (t + a) by the equation (1) described in [Prior Art]. ). However, in equation (1), B
(X, y, t1) is the first decoded image, B (x, y, t)
2) is a second decoded image, I (x, y, t3) is an interpolated image, t1, t2, and t3 are first decoded images, respectively.
This is the time of the second decoded image and the interpolated image. Therefore, FIG.
T1 = t−b, t2 = t + a, t3
= T.

As described above, the lower layer is synthesized by using the switching of the switch 112. For example, in the case of FIG. 9A, the second decoded image (frame time t + a) is indicated by the hatched portion.
Is used, a background pixel outside the selected area appears,
Since the first decoded image (frame time tb) is used in the halftone dot portion, a background pixel outside the selected region appears, and in the other portions, an interpolated image of the first decoded image and the second decoded image is displayed. Appears. Since the decoded image of the upper layer is superimposed on the lower layer synthesized in this way by the weighted averaging unit 114 of FIG. 1, the superimposed image is located around the selected area (shaded area) as shown in FIG. 9B. An image with no afterimage and little distortion can be obtained. The weighted averaging unit 114 in FIG. 1 superimposes the above-mentioned interpolation image and the decoded image of the upper layer by weighted averaging. The method of superimposition has been described in [Prior Art], and a description thereof will be omitted.

FIG. 15 is a block diagram showing the configuration of upper layer decoding section 1406 in the decoding apparatus shown in FIG. The splitter 1501 splits the upper layer encoded data into encoded data of pixel data and encoded data of a region shape. The switch 1504 turns off at the frame position where the lower layer is coded, and turns on at the frame position where only the upper layer is coded. The pixel data decoding unit 1502 decodes the pixel data of the upper layer and outputs the decoded pixel to the superimposition unit 1408. The region shape decoding unit 1503 decodes the region shape encoded data, and
08. Superimposition section 1408 is superimposition section 1
By the same operation as 405, the upper layer is superimposed on the lower layer by the method of the present invention.

Next, a second embodiment of the present invention will be described. In this embodiment, a mode in which a region shape is not coded by an upper layer coding device is provided to reduce the number of coded bits. In the case where the region shape hardly changes at all over time, the number of bits can be greatly reduced by using the method of the present embodiment.

This embodiment is shown in FIGS. 2, 14 and 16. FIG. 14 is a diagram used for describing the first embodiment, but is also used for describing the second embodiment. However, upper layer coding section 1403, upper layer decoding section 1406, superimposing section 1405 and superimposing section 1 in FIG.
The operation of 408 is different from that of the first embodiment. These different functions will be described below.

FIG. 2 is a block diagram of an upper layer encoding unit and a superimposing unit according to the second embodiment. Switch 2
01, pixel data encoding unit 203, region shape encoding unit 2
04, pixel data decoding unit 205, region shape decoding unit 206
The multiplexing section 207 has the same function as that already described with reference to FIG.

The switches 202 and 208 are controlled by a control unit (not shown) so that they are simultaneously turned on or off. When the lower layer frame corresponding to the same frame position as the upper layer is encoded, the switch is turned on to encode the region shape of the upper layer. on the other hand,
If the lower layer frame corresponding to the same frame position as the upper layer has not been encoded, the switch is controlled to be turned off, and the region shape of the upper layer is not encoded.

The first region shape extraction unit 209 extracts the first region shape based on the data obtained by the decoding device. Similarly, the second region shape extraction unit 210 extracts a second region shape based on data obtained by the decoding device. As the data obtained by the decoding device, the decoded image of the lower layer and the region shape encoding mode are on (switch 20).
2, when the switch 208 is on). Although not explicitly shown in FIG. 2, these data are input to each region shape extraction unit and used for extracting the region shape.

The first area extracting section 211, the second area extracting section 212, the controller 213, the switch 214, the interpolation image creating section 215, and the weighted averaging section 216 perform the same operations as those already described with reference to FIG. Therefore, the description is omitted.
Further, the delay unit 217 has the same function as the fourth delay unit 116 in FIG.

FIG. 16 is a block diagram of an upper layer decoding unit according to the second embodiment. In this figure, 16
01 is a branching unit, 1602 is a pixel data decoding unit, 1603
Denotes an area shape decoding unit, 1604 denotes a switch, and 1605 denotes a switch.

FIG. 15 differs from FIG. 15 in that a switch 1605 is provided in FIG. Switch 160
5 is off when there is no encoded data of the lower layer frame corresponding to the same frame position at the time of upper layer decoding. Also, it is turned on at a frame position where pixel data is encoded in the upper layer.

As described above, in the second embodiment, when the lower layer frame corresponding to the same frame position as the upper layer is not encoded by the upper layer encoding unit, the switches 202 and 208 are turned off. Control,
Since the region shape of the upper layer is not coded, the amount of code can be reduced.

In the first and second embodiments described above, the switch 101 or the switch 201 switches on / off the encoding of pixel data. This switching is performed by determining whether or not a lower layer frame corresponding to the same frame position as the upper layer is encoded by a control unit not shown in FIGS. This determination can be made in the same way by the encoding device and the decoding device. However, at least in the decoding device, it is also possible to perform the switching without using the control unit that makes such a determination. For this purpose, a first flag generator 301 and a first flag encoder 302 as shown in FIG. 3 may be provided in the encoder, and switches in the upper layer encoder 303 may be switched according to the flags. However, FIG. 3 is a block diagram showing a part of the encoding device and the decoding device.

The flag generator 301 determines whether a lower layer frame corresponding to the same frame position as the upper layer has been encoded, and generates a flag. The first flag encoding unit 302 encodes the first flag, and the encoded data is multiplexed with the encoded data by a multiplexing unit (not shown) and transmitted or stored. As a flag encoding method, fixed-length encoding, variable-length encoding, or the like is used.

First flag decoding section 30 in decoding apparatus
4 decodes the first flag from the encoded data and outputs the result to the upper layer decoding unit 305. Upon switching of the switch 1504 or switch 1604 included in the upper layer decoding unit, the without performing the above-mentioned switching decision, the switching is performed in accordance with the first flag decoded.

Next, a description will be given of an area shape extraction unit according to the second embodiment. Here, the second region shape is extracted based on the data obtained by the decoding device, but the decoded data of the lower layer, the decoded region shape of the upper layer, or the like may be used as the data obtained by the decoding device.

FIG. 4 is a block diagram of a superimposing unit for extracting a region shape using decoded data of a lower layer.
A first decoded image of the lower layer delayed by the delay unit 401 is input to the first region shape extraction unit 402, and a decoded image of the lower layer is input to the second region shape extraction unit 403 without delay. Is done. Each region shape extraction unit divides a decoded image input so as to extract a selected region into regions, and extracts a region shape. As a method of region division, an edge detection method using a differential operation, morphological segmentation, or the like is used. The other parts in FIG. 4 operate in the same manner as in FIG.

FIG. 5 is a block diagram in the case where the decoding area shape of the upper layer is used as data obtained by the decoding apparatus. In this figure, 501 is a delay unit, 502 is a first unit.
503 is a second region shape extraction unit, and 503 is a second region shape extraction unit.
04 is a first region extraction unit, 505 is a second region extraction unit,
Reference numeral 506 denotes a controller, 507 denotes a switch, 508 denotes an interpolation image creation unit, and 509 denotes a weighted average unit.

In FIG. 5, when the region shape of the upper layer is coded, the decoded data is supplied to the first region shape extraction unit 5.
02 and the second region shape extraction unit 503. Each region shape extraction unit stores the decoded region shape,
The region shape corresponding to the lower layer frame is extracted. For example, as shown in FIG. 13, a method of extracting the region shape at the position of the lower layer frame from the decoded region shapes 1 and 2 of the upper layer before and after the lower layer frame by affine transformation expressing parallel movement, rotation, scaling, and the like. Can be considered.

For this purpose, first, an affine transformation from the region shape 1 to the region shape 2 is obtained. That is, an affine transformation parameter that approximates the region shape 2 by converting the region shape 1 is obtained. Next, an affine transformation from the region shape 1 to the lower layer frame is obtained by linearly interpolating the transformation coefficients. The region shape on the lower layer frame can be obtained using the affine transformation. Instead of affine transformation, prediction from region shape 1 to region shape 2 may be performed by block matching, and the result may be linearly interpolated to obtain the region shape on the lower layer frame. Alternatively, the region shape 1 or 2 can be used as it is as the region shape on the lower layer frame.

In the second embodiment, the switch 202 in FIG. 2 is turned off in a frame in which the lower layer is not coded, and the switch 202 is turned on in a frame position where pixel data in the upper layer is coded. You may take control. For example, the temporal change of the region shape is examined. If the change is hardly changed, the switch 202 is turned off. Otherwise, the switch 202 is turned on. May be used.

Next, a third embodiment will be described.
In the present embodiment, when there is no encoded data of the lower layer corresponding to the frame position of the upper layer at the time of decoding the upper layer, the lower layer frame described in the first and second embodiments is not combined. The purpose is to provide a mode. For example, when the region shape does not change much over time, the problem described in [Problems to be Solved by the Invention] can be ignored, and there is no need to combine lower layer frames. Even when the region shape changes greatly, a mode in which lower layer frames are not combined can be selected so as not to increase the processing amount in the encoding device and the decoding device. For such a purpose, as shown in FIG. 6, a second flag generator 601 and a second flag encoder 602 are provided in the encoding device, and a second flag decoder 604 is provided in the decoding device. FIG. 6 is a block diagram showing a part of the encoding device and the decoding device.

The second flag generator 601 shown in FIG. 6 generates a flag indicating whether to combine lower layer frames. The superimposing unit 603 switches between combining and not combining lower layer frames according to the second flag. The second flag encoding unit 602 encodes the second flag, and the encoded data is multiplexed with the encoded data by a multiplexing unit (not shown) and transmitted or stored. As a flag encoding method, fixed-length encoding, variable-length encoding, or the like is used.

Second flag decoding unit 60 in the decoding device
4 decodes the second flag from the encoded data,
5 is output. The superposition unit 605 switches whether to combine lower layers according to the decoded second flag.

In the third embodiment, when the lower layer is not synthesized, it is encoded in the lower layer,
One of the lower layer frames before and after the decoding is used instead of the synthesized lower layer frame. FIG. 7 shows a circuit configuration in this case. In this figure, 701
Denotes a switch, 702 denotes a switch, 703 denotes a pixel data encoding unit, 704 denotes an area shape encoding unit, 705 denotes a pixel data decoding unit, 706 denotes an area shape decoding unit, 707 denotes a multiplexing unit, 708 denotes a switch, and 709 denotes a lower order. Layer composition part, 7
Reference numeral 10 denotes a switch, and 711 denotes a weighted average unit.

Next, the operation of the circuit shown in FIG. 7 will be described. First, the decoded image of the lower layer and the lower layer frame synthesized by the lower
It is switched at 0 and input to the weighted average section 711. The lower layer combining unit 709 in FIG. 7 combines lower layer frames according to the method described in the first and second embodiments. The switch 710 is switched to the lower side when the lower layer combining is on and to the upper side when the lower layer combining is off according to the second flag described in FIG.

In the third embodiment, the following method may be used as the coding method of the area shape used for the synthesis of the lower layer. That is, at the frame position of the upper layer where lower layer synthesis is performed, the region shape in the lower layer before and after the frame position is encoded at the current frame position. FIG. 17 shows an upper layer encoding unit using this method, and FIG. 18 shows an upper layer decoding unit.

In FIG. 17, reference numeral 1701 denotes a pixel data encoding unit; 1702, a pixel data decoding unit;
1704, a switch, 1706, a switch, 1707, an area shape encoder, 1708, an area shape decoder, 1709, a third flag generator, and 1710, a fourth flag generator , 1711 is a third flag encoding unit, 1712 is a fourth flag encoding unit, 1713 is a controller, and 1714 is a multiplexing unit.

In FIG. 17, the pixel data encoding unit 1701 and the pixel data decoding unit 1702 operate in the same manner as those described in the description of FIG. In FIG. 17, the region shape is delayed by a frame in the first delay unit 1703 and further delayed by b frame in the second delay unit 1704.

The third flag generator 1709 and the fourth flag generator 1710 generate a third flag and a fourth flag, respectively. The third flag indicates whether or not to encode the region shape at frame time t + a (hereinafter, region shape 2), and the fourth flag indicates the region shape at frame time tb (hereinafter region shape 1 and region shape 1). ) Is encoded or not. The controller 1713 controls the switches 1705 and 1706 by inputting the third flag and the fourth flag.

That is, if the third flag indicates that the region shape is to be coded, the switch 1705 is turned on; otherwise, the switch 1705 is turned off. If the fourth flag indicates that the region shape is to be coded, the switch 1706 is turned on; otherwise, the switch 1706 is turned off. The third flag encoding unit and the fourth flag encoding unit encode the third flag and the fourth flag, respectively. As a flag encoding method, fixed-length encoding, variable-length encoding, or the like is used.

The region shape encoding unit 1707 encodes the region shape at the frame time when the region shape is input, and outputs encoded data. The region shape decoding unit 1708 decodes the encoded data of the region shape and sends the decoded region shape to the superimposing unit. Here, the device as shown in FIG. 1 is used as the superimposing unit, but the first delay unit 107 and the second delay unit 108 in FIG. 1 are not used. The decoded data of the region shape 1 is input to the first region extracting unit 109 and the second region extracting unit 110 of the superimposing unit. Similarly, the decoded data of the region shape 2 is also input to the first region extracting unit 109 and the second region extracting unit 110. It is input to the extraction unit 110. On the other hand, the decoded data of the area shape corresponding to the frame time t is input to the weighted averaging unit 114.

The ON / OFF combination of the switches 1705 and 1706 is controlled in the following three ways. In other words, there are three types: both switches are on, both switches are off, the switch 1705 is on, and the switch 1706 is off. If performed the first time the synthesis of the lower layer both switches are controlled both turned on, the region shape of the frame position of the front and rear, means that area shape 1 and area shape 2 is encoded,
The decoded area shape is decoded by the first area extracting unit 10.
9 and the second region extraction unit 110. When the same region shape is used as the region shape 1 and the region shape 2 at the time of the previous lower layer combination, both switches are controlled to be off. In this case, the region shape 1 and the region shape 2 used at the time of the previous lower layer combination are input from the memory (not shown) to the first region extraction unit 109 and the second region extraction unit 110.

A switch 1705 is used when the area shape 2 used in the previous lower layer synthesis is used as the area shape 1 in the current lower layer synthesis and a new area shape is used as the area shape 2 in the current synthesis. Is turned on and the switch 1706 is turned off. In this case, the area shape 2 used at the time of the previous lower layer synthesis is input as the area shape 1 at the time of the current synthesis from a memory (not shown) to the first area extractor 109 and the second area extractor 110. Also,
The newly encoded / decoded region shape 2 is input to the first region extraction unit 109 and the second region extraction unit 110.

The pixel data at the frame time t + a decoded by the pixel data decoding unit 1702 in FIG. 17 is delayed by a frame in the third delay unit 115 in the superimposing unit in FIG.
It is input to the weighted averaging unit 114. Multiplexer 17 in FIG.
08 multiplexes the encoded data of each of the pixel data, the area shape, the third flag, and the fourth flag, and outputs the multiplexed data as the encoded data of the upper layer.

Next, an upper layer decoding unit for decoding the above-described upper layer encoded data will be described with reference to FIG.
In this drawing, reference numeral 1801 denotes a branching unit, 1802 denotes a pixel data decoding unit, 1803 denotes an area shape decoding unit, 1804 denotes a third flag decoding unit, 1805 denotes a fourth flag decoding unit,
806 is a first delay unit, 1807 is a second delay unit, 18
08 is a switch, 1809 is a switch, and 1810 is a controller.

In FIG. 18, a branching section 1801 is a section for separating upper layer encoded data into encoded data of pixel data, area shape, third flag and fourth flag. Pixel data decoding unit 1802, region shape decoding unit 18
03, a first delay unit 1806, a second delay unit 1807,
The switches 1808 and 1809 have the same functions as those in FIG.

The third flag decoding unit and the fourth flag decoding unit in FIG. 18 decode the third flag and the fourth flag, respectively, and supply them to the controller 1810. The controller 1810 controls the two switches similarly to the controller 1713 of FIG.
At 01, control for extracting encoded data of the area shape is also performed. That is, when the third flag indicates that the region shape 1 has been coded, the region shape 1 data is separated from the upper layer coded data, otherwise the region shape 1 data does not exist. Control is performed so that this is not separated from the upper layer encoded data.

The same control is performed on the flow dividing unit 1801 for the fourth flag. Turn on two switches
The combination of off is the same as in the upper layer coding section of FIG.
There are kinds. The operation in each combination is shown in FIG.
Is the same as that described in the description.

As described above, in the third embodiment, as a method of encoding a region shape used for combining lower layers,
In the upper layer frame position where the lower layer is synthesized,
It is possible to use a method of encoding the region shape in the lower layer before and after the frame position at the current frame position.

In the description of the present invention, the image in which the upper layer frame is superimposed on the lower layer frame is:
In the coding apparatus, as shown in FIG. 14, the data is fed back to the upper layer coding unit and used for predictive coding of the upper layer. In the decoding device, it is used for predictive coding of an upper layer and is displayed on a display or the like. However, the superimposed image may be used only for display.

That is, the encoding apparatus does not have the superimposing unit of the present embodiment, and the decoded image of the upper layer is directly fed back to the upper layer encoding unit and used for predictive encoding. In the decoding device, the decoded image of the upper layer is directly fed back to the upper layer decoding unit, used for prediction and input to the superimposing unit, and the output of the superimposing unit is displayed on a display or the like.

Further, in the description of the present invention, the encoding of the region shape is all described using a method using an 8-way quantization code, but it goes without saying that other shape encoding methods may be used.

[0080]

According to the moving picture encoding apparatus and the moving picture decoding apparatus of the present invention, (1) when synthesizing a lower layer frame that has not been coded, A first region shape and a second region shape of a lower layer existing later in time are encoded in an upper layer, and the first region shape and the second region shape are used , and only the first region shape is determined.
For the area, the pixel value of the second lower layer frame is
Used, and for the region having only the second region shape,
By performing the synthesis using the pixel values of the first lower layer frame, a good image can be obtained without distortion in the superimposed image of the lower layer and the upper layer even when the region shape changes over time. . (2) When synthesizing an uncoded lower layer frame, encoding the first region shape of the lower layer existing temporally earlier and the second region shape of the lower layer existing temporally later Without performing the first region shape and the second region shape, using the lower layer decoding obtained by the lower layer decoding unit.
By extracting an image by region division, a mode in which region shape is not encoded in an upper layer is introduced, and the number of bits can be reduced. (3) Combining uncoded lower layer frames
When performing, the first area of the lower layer existing temporally before
The shape and the second area shape of the lower layer existing later in time
Without Jo coding, the first area shape and the second
The region shape is derived from the region shape obtained by the region shape decoding unit.
By extracting by estimating using the fin transform , the region shape can be accurately obtained without increasing the number of bits. (4) Combining uncoded lower layer frames
When performing, the first area of the lower layer existing temporally before
The shape and the second area shape of the lower layer existing later in time
Without Jo coding, the first area shape and the second
The region shape is extracted from the region shape obtained by the region shape decoding unit.
By extracting by estimating using prediction by lock matching , the region shape can be easily obtained without increasing the number of bits. (5) In the above (1) to (4), by encoding a flag indicating whether or not to synthesize the non-existing lower layer frame, the non-existing lower layer frame temporally precedes or follows. The lower layer frame can be used as the synthesized lower layer frame, and the amount of processing required for the synthesis can be reduced. (6) In the above (1), the first region shape and the second region shape
By encoding the first and second flags indicating whether or not to encode each of the region shapes, if both the first and second region shapes are not encoded, the previous lower layer synthesis is performed. The region shape used at the time of the current lower layer combination is used as the region shape used at the time of the current lower layer composition. If only the second region shape is encoded, the second region shape used at the time of the previous lower layer composition is The first area shape used at the time of lower layer composition can be used. (7) Since the region information is not encoded in the lower layer, many bits are not generated in the lower layer. Therefore, a good lower layer image can be transmitted or stored without causing large distortion even in a transmission line or memory having a relatively low bit rate for transmitting or storing the lower layer.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the present invention.

FIG. 3 is a diagram illustrating a first flag according to the present invention.

FIG. 4 is a block diagram in a case where a region shape is extracted using decoded data of a lower layer in the present invention.

FIG. 5 is a block diagram in a case where a region shape is extracted using a region shape of an upper layer according to the present invention.

FIG. 6 is a diagram illustrating a third embodiment of the present invention.

FIG. 7 is a diagram illustrating another example of the third embodiment of the present invention.

FIG. 8 is a block diagram illustrating a conventional encoding method and decoding method.

FIG. 9 is a diagram illustrating an example of an effect of the present invention.

FIG. 10 is a diagram showing the concept of a conventional method.

FIG. 11 is a diagram illustrating an 8-way quantization code.

FIG. 12 is a diagram illustrating a problem of the conventional method.

FIG. 13 is a diagram illustrating an example of extracting a region shape using a region shape of an upper layer according to the present invention.

FIG. 14 is a block diagram illustrating a first embodiment and a second embodiment of the present invention.

FIG. 15 is a block diagram illustrating a first embodiment of the present invention.

FIG. 16 is a block diagram illustrating a second embodiment of the present invention.

FIG. 17 is a block diagram illustrating an example of an upper layer encoding unit according to the present invention.

FIG. 18 is a block diagram illustrating an example of an upper layer decoding unit according to the present invention.

[Explanation of symbols]

 Reference Signs List 101 switch 102 pixel data encoding unit 103 area shape encoding unit 104 pixel data decoding unit 105 area shape decoding unit 106 multiplexing unit 107 first delay unit 108 second delay unit 109 first region extraction unit 110 second Region extraction unit 111 controller 112 switch 113 interpolation image creation unit 114 weighted average unit 115 third delay unit 116 fourth delay unit

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiroshi Kusao 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Inside Sharp Corporation (72) Inventor Shuichi Watanabe 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Sharp Corporation (56) References JP-A-8-130733 (JP, A) JP-A-7-38899 (JP, A) JP-A-60-206287 (JP, A) JP-A-60-122871 (JP, A) (58) Investigated field (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 INSPEC (DIALOG) JICST file (JOIS) Practical file (PATOLIS) Patent file (PATOLIS)

Claims (9)

(57) [Claims] 1. A lower layer encoded data obtained by encoding pixel data of a moving image sequence at a first frame rate, and a second layer image data of a specific area of the moving image sequence being higher than the first frame rate. An upper layer encoded data obtained by encoding at a frame rate and an area shape of a specific area of the moving image sequence is input, and a lower layer decoded image is output, or the lower layer decoded image and the upper layer decoded A moving image decoding apparatus that outputs a superimposed image on an image, comprising: a lower layer decoding unit that decodes the lower layer encoded data at the first frame rate; A shunting unit that shunts the coded data and the coded data of the area shape, and coded data of the pixel data shunted by the shunting unit. A pixel data decoding unit for decoding at a second frame rate; and a region shape decoding unit for decoding encoded data of the region shape divided by the division unit, wherein the encoded data of the region shape is an upper layer frame. If there is no lower layer frame corresponding to the same frame position in time, the first region shape and the second region shape indicating the specific region in the first and second lower layer frames existing temporally before and after the frame position It contains coded data of area shape, using said first area shape and the second area shape, said first and second lower layer frame, said first area type
The second lower layer frame
Using the pixel value of
A moving image decoding apparatus for synthesizing the nonexistent lower layer frame by using a pixel value of the first lower layer frame . 2. A lower layer encoded data obtained by encoding pixel data of a moving image sequence at a first frame rate, and a second layer image data of a specific area of the moving image sequence being higher than the first frame rate. At the frame rate at which the pixel data of the specific area is coded at the frame rate, the upper layer coded data obtained by coding the area shape of the specific area of the moving image sequence is input, and the lower layer decoding is performed. What is claimed is: 1. A moving image decoding apparatus that outputs an image or outputs a superimposed image of a lower layer decoded image and an upper layer decoded image, the lower layer decoding decoding the lower layer encoded data at the first frame rate. A portion for dividing the upper layer encoded data into encoded data of pixel data and encoded data of an area shape. A stream unit; a pixel data decoding unit that decodes the encoded data of the pixel data split by the split unit at the second frame rate; and an area that decodes the encoded data of the area shape split by the split unit. When there is no lower layer frame corresponding to the same frame position as the upper layer frame temporally, the first and second lower layer frames located before and after the frame position are provided. the first area shape and the second area shape showing the area, the lower layer decoded image obtained we are in the lower layer decoding unit extracts by area division, and the first area shape and the second area shape, wherein Using first and second lower layer frames , the first region shape
Only the area of the second lower layer frame.
Using the prime value, for the region having only the second region shape
Uses the pixel values of the first lower layer frame ,
A moving image decoding apparatus, wherein the non-existent lower layer frame is synthesized. 3. The method according to claim 1, wherein the pixel data of the moving image sequence is
Lower layer encoded data encoded at frame rate
And pixel data of a specific area of the moving image sequence
A second frame rate higher than the first frame rate
And the pixel data of the specific area is encoded.
Only at the frame position where
Layer coding that encodes the shape of the specific area of the
Input data and output a lower layer decoded image, or
Output a superimposed image of the decoded image and the upper layer decoded image
A moving image decoding apparatus, wherein the lower layer encoded data is transmitted to the first frame rate.
A lower layer decoding unit that decodes the encoded data of the pixel data and an encoded data of the pixel data.
And a shunt section that shunts the data and the coded data of the area shape, and coded data of the pixel data shunted by the shunt section.
A pixel data decoding unit for decoding at a second frame rate;
If, decodes the encoded data of the diverted area shape by the diverter
When there is no lower layer frame corresponding to the same frame position as the upper layer frame in time, the first and second lower layer frames existing before and after the frame position are provided. The first region shape and the second region shape indicating the specific region in the region shape are obtained by the region shape obtained by the region shape decoding unit.
Extraction by estimating from shape using affine transformation
And the first and second region shapes and the first and second region shapes.
2 lower layer frames and the first region shape
Only the area of the second lower layer frame.
Using the prime value, for the region having only the second region shape
Uses the pixel values of the first lower layer frame,
Moving image decoding apparatus you and performing the synthesis of the lower layer frame not the present. 4. The method according to claim 1, wherein the pixel data of the moving image sequence is
Lower layer encoded data encoded at frame rate
And pixel data of a specific area of the moving image sequence
A second frame rate higher than the first frame rate
And the pixel data of the specific area is encoded.
Only at the frame position where
Layer coding that encodes the shape of the specific area of the
Input data and output a lower layer decoded image, or
Output a superimposed image of the decoded image and the upper layer decoded image
A moving image decoding apparatus, wherein the lower layer encoded data is transmitted to the first frame rate.
A lower layer decoding unit that decodes the encoded data of the pixel data and an encoded data of the pixel data.
And a shunt section that shunts the data and the coded data of the area shape, and coded data of the pixel data shunted by the shunt section.
A pixel data decoding unit for decoding at a second frame rate;
If, decodes the encoded data of the diverted area shape by the diverter
When there is no lower layer frame corresponding to the same frame position as the upper layer frame in time, the first and second lower layer frames existing before and after the frame position are provided. the first area shape and the second area shape shows a certain area in the area shape from area shape obtained by the decoding unit using the prediction by block matching extracted by estimating the first area shape and the second Region shape, the first and second
2 lower layer frames and the first region shape
Only the area of the second lower layer frame.
Using the prime value, for the region having only the second region shape
Uses the pixel values of the first lower layer frame,
Moving image decoding apparatus you and performing the synthesis of the lower layer frame not the present. 5. The moving picture decoding apparatus according to claim 1, further comprising: a flag decoding unit configured to decode encoded data of a flag indicating whether or not to synthesize a lower layer frame; Based on the flag decoded in the above, it is determined whether to synthesize the nonexistent lower layer frame using the first area shape and the second area shape, and the first and second lower layer frames. A moving image decoding apparatus for determining and synthesizing the lower layer frame. 6. The video decoding device according to claim 1, wherein the upper layer decoding unit further determines a first region shape in a first lower layer frame temporally preceding the upper layer frame. A first flag for decoding the encoded data of the first flag indicating whether or not the area information indicated is encoded;
Coded data of a second flag indicating whether or not region information indicating a second region shape in a second lower layer frame that is temporally subsequent to the upper layer frame is coded. A second flag decoding unit for decoding, based on the first flag and the second flag decoded by the first and second flag decoding units, a first region shape and a second region shape Is encoded or not, and the first
If neither the second region shape nor the second region shape is encoded, the region shapes used in the previous lower layer combination are used as the region shapes used in the current lower layer combination, respectively, and only the second region shape is encoded. The second area shape used in the previous lower layer synthesis is used as the first area shape used in the current lower layer synthesis, and the first area shape and the second area shape, and And a second lower layer frame, wherein the non-existent lower layer frame is synthesized. 7. A lower layer encoding unit that encodes pixel data of a moving image sequence at a first frame rate, and a second layer encoding unit that encodes pixel data of a specific region of the moving image sequence at a higher rate than the first frame rate. A moving image encoding apparatus comprising: an upper layer encoding unit that encodes at a frame rate and encodes a region shape of a specific region of the moving image sequence; A pixel data encoding unit that encodes pixel data of a specific area of the image sequence at a second frame rate higher than the first frame rate; and an area shape code that encodes an area shape of the specific area of the moving image sequence Encoding unit, encoded data of the pixel data encoded by the pixel data encoding unit, and encoding of the region shape encoded by the region shape encoding unit And a multiplexing unit that multiplexes the data with the upper layer frame. If there is no lower layer frame corresponding to the same frame position as the upper layer frame, the area shape coding unit A first region shape and a second region shape indicating a specific region in first and second lower layer frames existing before and after the first and second lower layer frames. 8. The moving picture encoding apparatus according to claim 7, further comprising: a flag encoding unit that encodes a flag indicating whether or not to synthesize the lower layer frame in the moving picture decoding apparatus. A moving picture coding apparatus characterized by the following. 9. The video encoding apparatus according to claim 7, wherein the upper layer encoding unit further includes a first area in a first lower layer frame temporally preceding the upper layer frame. A first flag encoding unit that encodes a first flag that indicates whether or not region information indicating a shape has been encoded; and a second flag in a second lower layer frame that exists temporally after the upper layer frame. A second flag for encoding a second flag indicating whether or not the area information indicating the area shape has been encoded
And a flag encoding unit.