JPH0865679A - Moving image encoding device and moving image decoding device - Google Patents

Abstract

PURPOSE: To improve the timewise resolution of a moving image by perfoming interpolation while dividing an image into rectangular areas of an arbitrary shape, and reproducing the image thinned on the encoding side on the decoding side with satisfactory picture quality when interpolating a frame on the side of the decoding device. CONSTITUTION: A rectangular area detecting/dividing part 104 detects the positions of the lattice-points of rectangular areas on a reference frame stored in a frame memory (2) 102, prepares position information and divides the image into the rectangular areas surrounded by the lattice-points. A motion vector detecting part 105 detecs the lattice-points on a reference frame stored in another frame memory (1) 103 corresponding to the respective lattice-points of the rectangular areas on the reference frame prepared by the rectangular area detecting/dividing part 104 and calculates a moving amount between the correspondent lattice-points. While using the reference frame, interpolated frame and lattice-point position information of the reference frame, an interpolation frame preparing part 107 transforms the shape of the rectangular areas on the reference frame and composes the rectangular areas on the interpolation frame.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding apparatus and a moving picture decoding apparatus, and more particularly to a moving picture coding apparatus and a moving picture decoding apparatus for coding image data in digital image processing with high efficiency. Regarding the device.

[0002]

2. Description of the Related Art Conventionally, a moving image coding system has adopted a method of thinning out frames which are temporally preceding and succeeding at appropriate intervals to reduce the code amount. At this time, in order to reproduce smooth motion on the decoding side, the technique of frame interpolation is important. As a known document regarding this frame interpolation,
For example, "Frame Interpolation Method for Moving Images Using Bidirectional Motion Vectors" (1990 Image Coding Symposium 8-3, pp. 1
81-184).

FIG. 11 is a diagram showing an example of a conventional frame interpolation method. In FIG. 11, four frames are arranged in the time direction, and the interpolation frame is a frame thinned on the encoding side. When interpolating these decimated frames on the decoding side, the motion vector of each square or rectangular lattice (hereinafter referred to as a block) of the reference frame is used. For example, when creating an interpolation frame,
Motion vector M from reference frame to reference frame
Use V _PC .

Assuming that the position of the block B _P on the reference frame is (X, Y) and the motion vector MV _PC = (V _x , V _y ), the position (X _i , Y _i ) of the interpolation block B _{il in} FIG. Can be expressed as X _i = X + aV _x Y _i = Y + aV _y (1) Here, a is two reference frames and the reference frame is a: 1-
It is the ratio internally divided to a. Interpolation block B _il , block B _{C on} the reference frame, block B _P on the reference frame
Letting α, β, and γ be the corresponding pixels in the above, it can be expressed by α = (1-a) γ + αβ (2).

As a known document describing a conventional moving picture image data transmitting / receiving apparatus, for example, Japanese Patent Application Laid-Open No. 5-2 is known.
There is a publication of 68598. In this publication, in order to display a moving image with a small amount of transmitted data,
The image data generating means has a plurality of frame intervals.
Image data of a subsequent frame is generated based on the motion vector of the block between two frames and the image data of the previous frame, and the interpolation means is based on the motion vector and the image data of at least one of the both frames. Image data of the frames between the two frames are interpolated.

[0006]

As described above,
In the conventional example, the pixel at the position corresponding to the block on the reference frame on the interpolation frame is calculated based on the position of the reference frame or the upper block in FIG. Therefore, as shown in FIG. 12, the blocks interpolated on the interpolation frame may overlap each other or a gap may be generated, and there is a problem that a visually excellent interpolation frame cannot be created. This occurs because the shape of the block is not changed when creating the interpolated frame, and it has always been a problem unless the motion vectors of the blocks are all the same.

The present invention has been made in view of the above circumstances, and when the frame is interpolated on the decoding device side, the image is divided into rectangular regions of arbitrary shape and interpolation is performed, and the interpolation is performed on the encoding side. An object of the present invention is to provide a moving picture coding apparatus and a moving picture decoding apparatus that reproduce a drawn picture with good picture quality on the decoding side and improve the temporal resolution of the moving picture.

[0008]

In order to solve the above-mentioned problems, the present invention provides (1) a moving picture decoding apparatus for creating an interpolation frame between first and second decoded frames,
A quadrilateral region detecting / dividing unit that divides the first decoded frame into quadrilateral regions surrounded by lattice points, and a grid point position on the second decoded frame corresponding to each of the grid points is obtained, and A grid point motion vector detection unit that detects a grid point motion vector between two decoded frames, and a grid point motion vector for the interpolation frame based on the grid point motion vector, and a grid point position on the interpolation frame An interpolated frame motion vector calculation unit that obtains, for each quadrilateral region surrounded by lattice points on the interpolated frame, one of the first decoded frame and the second decoded frame,
Alternatively, by including an interpolation frame creation unit that transforms the respective rectangular areas corresponding to the both to synthesize the rectangular areas on the interpolation frame, one of the first decoded frame and the second decoded frame, or Creating the interpolation frame from both, and (2) the difference between the time interval between the first frame and the interpolation frame and the time interval between the second decoded frame and the interpolation frame is previously set. If it is smaller than the predetermined value, the interpolation frame is created using the first and second decoding frames, and in other cases, the decoding frame on the side closer in time to the interpolation frame. Is used to create the interpolation frame, and (3) in (1) or (2) above, using the first and second decoded frames When you create between frames,
A first interpolation frame is created using the first decoded frame, a second interpolation frame is created using the second decoded frame, and the interpolation frame and the first and second decoded frames are To obtain the interpolation frame by performing a weighted average of the first and second interpolation frames according to the time interval of, and (4) in (1), (2) or (3) above, When the grid point motion vector for all or part of the rectangular area is included in the encoded data, for interpolation of the rectangular area including the grid point motion vector, the grid point motion vector detection unit is not used, The grid point motion vector included in the encoded data is used to interpolate a rectangular area in which the coded data does not include the lattice point motion vector. Using the grid point motion vector, or (5) dividing the image into a plurality of quadrilateral regions and transforming the quadrilateral region using the grid point motion vectors of the quadrilateral region to create a predicted image. However, in the moving picture coding device for coding the prediction error between the predicted image and the original image, without transmitting the prediction error in a specific frame,
An interpolation frame determination unit that is defined as an interpolation frame to be synthesized from the reference frame on the decoding side, information indicating that the interpolation frame is an interpolation frame, and the position of the reference frame used for interpolation on the decoding side are predetermined. If not present, an encoding unit for transmitting information indicating the position of the reference frame is provided, and (6) in (5), the encoding side detects a lattice point motion vector for the interpolation frame. And a coding unit for transmitting the grid point motion vector information. Further, (7) in (5) or (6), the interpolation frame is configured. An error amount calculator that calculates an error from the original image for each square area, and a square area where the error exceeds a predetermined threshold value. A coding means for coding the rectangular area on the original image, or a coding means for coding prediction error information between the original image in the rectangular area and the interpolation frame, and , (8) Information for indicating an interpolated frame included in the encoded data for decoding the encoded data created by the moving image encoding device according to claim 5, and a reference used in interpolation on the decoding side. When the position of the frame is not determined in advance, an interpolation frame creating unit for creating an interpolation frame from the reference frame is provided by using information indicating the position of the reference frame, and (9) above (8) In order to decode the coded data created by the moving picture coding apparatus according to claim 6, information on a lattice point motion vector for an interpolation frame included in the coded data is also included. There are, from the reference frame that includes an interpolation frame creation unit for creating an interpolation frame, and further, (10) the (8) or (9)
In order to decode the encoded data created by the moving image encoding apparatus according to claim 7, the prediction error information between the original image and the interpolated image included in the encoded data, or the original image information is used to It is characterized in that it comprises a creating means for creating a rectangular area.

[0009]

According to the present invention, the image is divided into a plurality of quadrilateral regions, and the image is interpolated by using the lattice point motion vector of each quadrilateral region. Therefore, the quadrilateral region is deformed as the lattice points move. It is possible to create an interpolation frame without causing overlapping or gaps of the rectangular areas on the interpolation frame. (1) In the moving image decoding device according to claim 1, since the decoding device detects the motion vector between the frames of the lattice points of the quadrangle region, the quadrangle region overlaps on the interpolation frame regardless of the encoding method. It is possible to create an image with no gaps. (2) In the moving picture decoding device according to claim 2, the decoding device side sets the time interval between the decoded frame and the interpolated frame as follows.
Since the decoded frame used to create the interpolation frame can be switched, it is possible to create a visually excellent interpolation frame.

(3) In the moving picture decoding apparatus according to claim 3, when the decoding apparatus uses two decoded frames to create an interpolated frame, each decoded frame is used to create two interpolated frames. Is created, and the interpolation frame is obtained by weighted averaging the two interpolation frames according to the time interval between the interpolation frame and the two decoded frames, so that a visually good interpolation frame can be created. . (4) In the moving picture decoding device according to claim 4, since the lattice point motion vector created on the encoding side can be used as the lattice point motion vector used on the decoding device side,
A circuit for detecting a lattice point motion vector is not required on the decoding device side. If all or part of the rectangular area of the reference frame does not have a grid point motion vector, the decoding side can independently detect the motion vector of each grid point.
It is possible to create interpolated frames between any of the frames.

(5) In the moving picture coding device and the decoding device according to the fifth and eighth aspects, the interpolation frame can be determined on the coding side, and the reference frame for creating the interpolation frame is the coding side. Since it can be selected with, it is possible to create a visually good image. (6) In the moving picture coding apparatus and the decoding apparatus according to claims 6 and 9, since the interpolation frame is created, since information of the lattice point motion vector of the interpolation frame is included in the coded data for each rectangular area, the decoding side It is possible to create a visually good image as compared with the case where the lattice point motion vector is detected independently. (7) In the moving picture coding device and the decoding device according to the seventh and tenth aspects, when the error with respect to the original image of the interpolation frame in each square region exceeds a certain threshold value, the original image of the interpolation frame is Since the pixel value of the rectangular area or the prediction error between the original image and the interpolation frame is coded, it is possible to create a visually good image.

[0012]

Embodiments will be described below with reference to the drawings. FIG. 1 is a block diagram for explaining an embodiment (claim 1) of a moving picture decoding apparatus according to the present invention, in which 101 is shown.
Is a moving picture decoding unit, 102 is a second frame memory, 10
3 is a first frame memory, 104 is a square area detection /
A division unit, 105 is a lattice word motion vector detection unit, 106 is an interpolation frame lattice point motion vector calculation unit, 107 is an interpolation frame creation unit, 108 is a selector, and 109 is a display unit.

The moving picture decoding unit 101 is composed of, for example, a variable length decoding unit, an inverse quantization unit, an inverse orthogonal transformation unit, a motion compensation unit and the like. The second frame memory 102 and the first frame memory 103 are frame memories that store the decoded frames decoded by the moving image decoding unit 101. The first frame memory 103 and the second frame memory 102
The decoded frames stored in the above are referred to as a reference frame and a reference frame, respectively.

When the interpolation frame is created, the reference frame is divided into a plurality of quadrilateral regions by a modified grid as shown in FIG. As a known document describing this division method, for example, "Very Low Bitra
te Video Coder using Warping Prediction "(1993 Image Coding Symposium 8-7, pp.167-168), and" Adaptive variable block shape KL transform coding of images "(1991 Image Coding Symposium 8-14, pp.235-238).

Each vertex of this modified grid is hereinafter referred to as a grid point, and this grid point is represented by ◯ in FIG. The quadrilateral region detecting / dividing unit 104 detects the positions of the grid points of the quadrilateral region on the reference frame, creates position information, and divides the image into quadrilateral regions surrounded by the grid points as shown in FIG. The motion vector detection unit 105 detects a grid point in another reference frame corresponding to each grid point of the rectangular area of the reference frame created by the rectangular area detection / division unit 104, and calculates a motion amount between the corresponding grid points. To do.

Interpolation frame lattice point motion vector calculation unit 1
06 recalculates the motion vector between the reference frame and the grid point motion vector into the motion vector between the reference frame and the interpolation frame. The interpolation frame creation unit 107 includes a reference frame, an interpolation frame and a reference frame,
Using the grid point position information of, the shape of the rectangular area of the reference frame is transformed to synthesize the rectangular area on the interpolation frame. The selector 108 selects the reference frame and the interpolation frame created by the interpolation frame creation unit 107 in order of earliest display time. The display unit 109 displays the reference frame and the interpolation frame.

Next, the quadrilateral area detection / division unit in FIG. 1 will be described. 3A to 3E are explanatory diagrams showing a method of detecting a rectangular area. As shown in FIG. 3A, the image is divided into rectangular blocks each having a size of N pixels × M pixels. In FIG. 3A, the intra-block variance d _A , for each of the four blocks A, B, C, D surrounding the grid point P,
Calculate d _B , d _C and d _D. Calculate D1 = d _A + d _B + d _C + d _D.

The grid point P is moved vertically and horizontally by one pixel, and D2, D3, D4 and D5 are calculated for the newly formed quadrangular area as shown in FIGS. 3 (b) to 3 (e). . The minimum value is selected from D1 to D5, and the grid point P is moved to the position having the minimum value. The above processes (1) to (3) are performed once for each grid point. The above process is repeated for the entire image until the shape of the quadrangle does not change.

Next, the lattice point motion vector detection unit in FIG. 1 will be described. Lattice point motion vector detection unit 10
At 5, the position of the grid point of the reference frame is detected and the motion vector for the grid point of the reference frame is obtained. This method is called "backward motion estimation", in which the grid on the reference frame is first determined and then the grid on the reference frame is determined. On the other hand, a method called “forward motion estimation” may be used. This first defines a grid (usually a uniform grid) on the reference frame,
Then, the grid points on the reference frame are determined. The present invention can be used in either case, but for simplicity, the "backward motion estimation" will be described as an example.

In order to detect to which position on the reference frame the grid point on the reference frame has moved, an area consisting of the grid point on the reference frame and the pixels in the vicinity is taken, and this area is set on the reference frame. A method of checking whether the area matches the area is used. Specifically, consider an area of M pixels × N pixels centered on a grid point on the reference frame, check the degree of coincidence with an area of the same size on the reference frame, and determine the center of the area with the highest degree of coincidence. It is the destination of the grid point. As the degree of coincidence between areas, the sum of absolute values of the error of pixel values in the area or the sum of weighted absolute values is used.

Next, the interpolated frame lattice point motion vector calculation unit in FIG. 1 will be described. The interpolated frame lattice point motion vector calculation unit 106 calculates the lattice point motion vector from the reference frame to the interpolated frame based on the lattice point motion vector output from the lattice point motion vector detection unit 105. For example, in FIG. 2, the motion vector from the reference frame at a certain grid point to the reference frame is (mV _x , mV _y ), and the interpolation frame temporally includes the reference frame and the reference frame within 1-a: a. If there is a position to _divide the motion vector, the motion vector (mV _hx , mV _hy ) from the reference frame to the interpolation frame can be expressed by the following equation.

MV _hx = a * mV _x (3) mV _hy = a * mV _y (4) Motion vector (mV _hx , mV _hy ) on this interpolation frame
The interpolated frame creating unit 107 creates an interpolated image from the reference frame for each quadrangular area by using, and outputs it as an image of the quadrangular area of the interpolated frame.

Next, the interpolation frame creating section in FIG. 1 will be described. The interpolation frame creation unit 107 calculates a value obtained by interpolating each pixel value included in the rectangular area of the interpolation frame from the reference frame by using the correspondence relationship between the grid points of the reference frame and the interpolation frame as shown in FIG. . That is, the rectangular area is divided into two triangles, the pixel position of the rectangular area on the reference frame is associated with the pixel position of the rectangular area on the interpolation frame by affine transformation, and the interpolation frame is created by the shape conversion of the rectangular area. Although the interpolation frame is predicted from the reference frame using the motion vector from the reference frame to the reference frame here, the interpolation frame may be predicted from the reference frame using the motion vector from the reference frame to the reference frame.

FIGS. 4A to 4C are diagrams for explaining another embodiment (claim 2) of the moving picture decoding apparatus according to the present invention. Create several interpolation frames between two reference frames (in FIG. 4, there are three interpolation frames)
At this time, as shown in FIG. 4A, the time interval between frames is t. Therefore, the time interval between the interpolation frame and the reference frame is t, the time interval between the reference frame and the reference frame is 3t, and the difference between these time intervals is 3t−t = 2t. Similarly, the time interval between the interpolation frame and the reference frame is 2t, and the time interval between the reference frame 2 and the interpolation frame is 2t.
t, the difference between these time intervals is 2t−2t = 0, the time interval between the interpolated frame and the reference frame is 3t, the time interval between the reference frame is t, and the difference between these time intervals is 3t−t = 2t. Is.

The predetermined threshold value TH is 0.
In the case of the interpolated frame, since the time interval difference (0) is smaller than TH (0.5) in the case of the interpolated frame, the interpolated frame is the reference frame, as shown in FIG.
It is created using both. In the case of the interpolated frame, since the difference in the time interval is TH or more, as shown in FIG. 4B, a reference frame that is close in time is used, the interpolated frame is the reference frame, and the interpolated frame is the reference frame. Created using.

FIGS. 5A to 5D are views for explaining still another embodiment (claim 3) of the moving picture decoding apparatus according to the present invention. Here, a method of creating one interpolation frame from two reference frames will be described. Figure 5
As shown in (a), it is assumed that the time interval between the reference frame and the interpolation frame is t, and the time interval between the interpolation frame is 2t. As shown in FIG. 5B, the interpolation frame is created from the reference frame, and as shown in FIG.
The interpolation frame is created from the reference frame. The final interpolated frame is obtained by calculating the weighted sum of the interpolated frame, as shown in FIG. In FIG. 5, the time interval between the reference frame and the interpolation frame is 1: 2.
Therefore, the weight used here is ⅔ for the interpolation frame and ⅓ for the interpolation frame. In this way, the interpolation frame can be represented by the weighted sum of the two reference frames by using the time interval between the reference frame and the interpolation frame.

FIG. 6 is a block diagram for explaining still another embodiment (claim 4) of the moving picture decoding apparatus according to the present invention.
In the figure, 201 is a moving image decoding unit, 202 is a second frame memory, 203 is a first frame memory, 204 is a quadrilateral area detection / division unit, 205 is a lattice point motion vector detection unit, and 206 is an interpolated frame lattice point. Motion vector calculator,
Reference numeral 207 is an interpolation frame creation unit, 208 is a selector, 20
Reference numeral 9 is a display unit, and 210 is a switch.

The present embodiment is premised on the case where the lattice point motion vector information is included in a part of the rectangular area. The difference between FIG. 6 and FIG. 1 is that in FIG. 6, for a quadrangle region including the lattice point motion vector in the encoded data, the moving image decoding unit 201 decodes the lattice point motion vector from the decoded data, For a quadrangle region that does not include a lattice point motion vector, the decoding device side detects the lattice point motion vector. Therefore, a changeover switch 210 for selecting the lattice point motion vector is added.
Therefore, the grid point motion vector to be used can be switched for each quadrangle area between the quadrangle area having the grid point motion vector created on the encoding side and the quadrangle area having no grid point motion vector.

FIG. 7 is a block diagram of a moving picture decoding apparatus showing another embodiment of claim 4, in which 301 is a moving picture decoding unit.
302 is a frame memory, 304 is a quadrilateral area detection / division unit, 306 is an interpolation frame lattice point motion vector calculation unit, 307 is an interpolation frame creation unit, 308 is a selector,
Reference numeral 309 is a display unit. FIG. 7 shows an example of the configuration of the decoding device in the case where all the rectangular regions in the frame have the lattice point motion vector created on the encoding device side. Since the decoded data includes the lattice point motion vector information for all the rectangular areas in the frame, the first frame memory 203 for detecting the lattice point motion vector, the lattice point motion vector detection unit 205, and the switch 210 in FIG. It is unnecessary. Therefore, the circuit scale of the decoding device can be reduced.

FIG. 8 is a block diagram for explaining an embodiment (Claim 5) of the moving picture coding apparatus according to the present invention. In the figure, 401 is an interpolation frame determining unit, 402 is a coding frame memory, 403 is a moving image coding unit, 404 is an interpolation frame memory, 405 is a local decoding / prediction image creating unit, 4
Reference numeral 06 is a second frame memory, 407 is a first frame memory, 408 is a quadrilateral region detecting / dividing unit, 409 is a quadrilateral region dividing unit, 410 is a lattice point motion vector detecting unit,
411 is an interpolation frame lattice point motion vector calculation unit, 41
Reference numeral 2 is a second interpolation frame creation unit, 413 is a first interpolation frame creation unit, 414 is a second error amount calculation unit, 415 is a first error amount calculation unit, 416 is a comparator, 417 is a code creation unit. Is.

The interpolated frame determination unit 401 selects whether the input image is a coded frame or an interpolated frame, and if it is determined to be a coded frame, stores the input image in the coded frame memory 402, If it is determined to be an interpolation frame, the input image is stored in the interpolation frame memory 404. The moving image coding unit 403 includes, for example, an orthogonal transformation unit and a quantization unit. The local decoding / prediction image creation unit 405 includes, for example, an inverse quantization unit, an inverse orthogonal transform unit, and a motion compensation unit. The first frame memory 407 and the second frame memory 406 are the local decoding / prediction image creation unit 40.
5 is a frame memory for storing the decoded frame created in 5.

The first frame memory 407 and the second frame memory
The frames stored in the frame memory 406 will be referred to as a reference frame and a reference frame, respectively. The reference frame is divided into rectangular areas by the rectangular area detection / division unit 408, and the lattice point motion vector detection unit 410 detects the lattice point motion vector with the encoding target frame. Using this, the interpolated frame lattice point motion vector calculation unit 411 calculates a lattice point motion vector with the interpolated frame. The second interpolation frame creation unit 412 creates an interpolation frame when the reference frame is used from these pieces of information, and the second error amount calculation unit 414 calculates the error amount from the original image.

On the other hand, the reference frame is divided into rectangular areas by the rectangular area dividing section 409 using the lattice point motion vector created by the lattice point motion vector detecting section 410.
The first interpolation frame creating unit 413 creates an interpolation frame, and the first error amount calculating unit 415 calculates the error amount between this interpolation frame and the original image stored in the interpolation frame memory 404.

In the comparator 416, the error amount when the reference frame is used for each quadrangle is compared with the error amount when the reference frame is used, and the one having the smaller error amount is selected and which one is selected. Output the information. The prediction error amount includes a sum of squared errors of pixels and a sum of absolute differences. The code creation unit 417 creates output data by combining the information indicating the reference frame, the position information of the interpolation frame, and the coded data from the moving image coding unit 403.

In this way, the reference frame used for interpolation is
By adding the information indicating the reference frame or the reference frame, a good interpolation frame can be created on the decoding side.

FIG. 9 is a block diagram for explaining another embodiment (claim 6) of the moving picture coding apparatus according to the present invention. In the figure, 501 is an interpolation frame determining unit and 502 is a coding frame memory. , 503 is a moving image coding unit, 504 is an interpolation frame memory, 505 is a local decoding / prediction image creating unit, 5
Reference numeral 06 is a frame memory, 507 is a quadrangular area detection / division unit, 508 is a lattice point motion vector detection unit, 509 is a lattice point motion vector detection unit, and 510 is a code creation unit.

The interpolated frame determination unit 501, the coded frame memory 502, the moving image coding unit 503, the interpolated frame memory 504, and the local decoding / prediction image creating unit 50 shown in FIG.
5, frame memory 506, square area detection / division unit 5
07, lattice point motion vector detection unit 508, code creation unit 5
10 is the same as the component of FIG. The interpolated frame lattice point motion vector detection unit 509 is configured similarly to the lattice point motion vector detection unit 508. The feature of FIG. 9 is that the interpolated frame lattice point motion vector detection unit 509
That is, the lattice point motion vector of the original image stored in the interpolation frame memory 504 is directly detected from the reference frame stored in the frame memory 506, and the lattice point motion vector of the interpolation frame is encoded.

Note that the embodiment of FIG. 9 is for the case of one reference frame, but in the case of two reference frames by adding the interpolation frame grid point motion vector detection unit 509 to the configuration of FIG. It is also possible to detect and encode the motion vector of the lattice point of each quadrangular area of the interpolation frame. By using the lattice point motion vector for interpolation as described above, it is possible to create an interpolated frame with better image quality.

FIG. 10 is a block diagram for explaining still another embodiment of the moving picture coding apparatus according to the present invention.
Reference numeral 01 is an interpolation frame determining unit, 602 is an encoding frame memory, 603 is a moving image encoding unit, 604 is an interpolation frame memory, 605 is a local decoding / prediction image creating unit, 606 is a frame memory, and 607 is a quadrilateral area detection / division. Part, 6
Reference numeral 08 is a grid point motion vector detection unit, 609 is an interpolation frame grid point motion vector calculation unit, 610 is an interpolation frame creation unit, 611 is an error amount calculation unit, 612 is a comparator, 613.
Is a code creating unit, and 614 is a switch.

The interpolated frame determination unit 601, the coded frame memory 602, the moving image coding unit 603, the interpolated frame memory 604, and the local decoding / prediction image creating unit 60 shown in FIG.
5, frame memory 606, square area detection / division unit 6
07, lattice point motion vector detection unit 608, code creation unit 6
10 is the same as the component of FIG. The interpolation frame lattice point motion vector calculation unit 609, the interpolation frame creation unit 610, and the error amount calculation unit 611 are the same as the constituent elements in FIG. The comparator 612 compares the error amount between the original image and the predicted image of the interpolated frame calculated by the error amount calculation unit 611 for each square area with a predetermined threshold value. At this time, the switch 614 is used only when the error amount is large.
Is turned on, and the pixel value is intra-frame coded for this rectangular area.

FIG. 10 shows an example of intra-frame coding of the pixel values of the rectangular area when the rectangular area of the interpolation frame is to be coded. It is also possible to encode the error value of the pixel in the rectangular area as the error value between the original image created by the calculation unit 611 and the interpolation frame. Further, by replacing the interpolated frame lattice point motion vector calculation unit 609 with the interpolated frame lattice point motion vector detection unit 509 of FIG. 9, it is possible to encode the lattice point motion vector of the interpolated frame. Further, by using the switch 614 in FIG. 8, even when a plurality of reference frames are provided, it is possible to code the pixel value or the error value of the pixel in the unit of the rectangular area.

As described above, by encoding the pixel value or the error value of the pixel in the quadrangle region for the interpolation, it is possible to create an interpolation frame with a better image quality. In the embodiment of FIG. 1, the encoded data created by the encoding device of FIG. 8 which is the embodiment of claim 5 is stored in the interpolation frame creation unit 107 as a reference frame or a reference frame from the encoded data. By inputting information indicating whether to select, the reference frame is adaptively switched for each rectangular area, thereby enabling decoding.

In the coded data created by the coding apparatus of FIG. 9 which is an embodiment of claim 6, the interpolated frame grid point motion vector included in the coded data is the interpolated frame grid in the embodiment of FIG. Decoding by directly inputting to the interpolation frame creation unit 307 without performing the processing of the point motion vector calculation unit 306 and creating an interpolation frame using the grid point motion vector calculated on the encoding side for each rectangular area. Is possible.

The coded data created by the coding apparatus of FIG. 10 according to the seventh embodiment is the interpolated image created by the interpolated frame creation unit 107 in the embodiment of FIG.
Decoding is possible by adaptively switching the decoded image decoded by the moving image decoding unit 101 for each rectangular area. As described above, the frames thinned on the encoding side can be interpolated on the decoding side to obtain a visually good image. In the above description, the “predetermined threshold value” is a value theoretically or empirically obtained.

[0045]

Since the moving picture coding apparatus and the decoding apparatus of the present invention are configured as described above, there are the following effects. (1) Since the images thinned out during encoding are interpolated and reproduced during decoding, the temporal resolution of display images can be improved. Even if the frame thinning is not performed at the time of encoding, it is possible to increase the number of display frames per time and make the motion smoother by inserting the interpolation image between the decoded images at the time of decoding. is there. (2) The image is divided into a plurality of quadrilateral regions, and the image is interpolated using the motion vector of the lattice points in each quadrilateral region. When the frame is created, the rectangular areas do not overlap each other or a gap is not formed on the interpolated frame, so that the interpolated frame with good image quality can be created. (3) Since the motion vector of the lattice point between the frames of the rectangular area can be detected on the decoding device side, regardless of the encoding method, an interpolated image in which the rectangular areas do not overlap or are not created on the interpolation frame is created. Is possible. (4) Since the decoding frame used for creating the interpolation frame is switched depending on the time interval between the decoding frame and the interpolation frame on the decoding device side, it is possible to create a visually satisfactory interpolation frame. (5) When creating an interpolated frame using two decoded frames on the decoding device side, two interpolated frames are created using each decoded frame, and the interpolated frame and the two decoded frames are created. Since the final interpolation frame is obtained by performing weighted averaging of the above-mentioned two interpolation frames according to the time interval of, it is possible to create a visually excellent interpolation frame. (6) The lattice point motion vector created on the encoding device side can be used as the lattice point motion vector used on the decoding device side. In the coding method using the lattice point motion vector, since the lattice point motion vector is encoded and transmitted,
In this case, a circuit for detecting the lattice point motion vector is not required on the decoding device side, and the circuit scale can be reduced. (7) When all or part of the rectangular region of the reference frame does not have a lattice point motion vector, the decoding side can independently detect the motion vector of each lattice point, so that in any frame Interpolation frames can be created in between, and smooth motion of a moving image can be reproduced. (8) Since the two reference frames used for interpolation can be adaptively switched for each quadrangular area, it is possible to create a visually favorable interpolation frame. (9) Since information of the lattice point motion vector of the interpolation frame can be provided for each rectangular area, it is possible to create a visually good image. (10) If the interpolation frame created from the reference frame on the encoding side has a large error with respect to the corresponding original image, intraframe coding or interframe coding is performed on the corresponding original image even for the interpolation frame. In addition, since the above processing adaptively selects a square area as a unit, it is possible to create a visually good image.

[Brief description of drawings]

FIG. 1 is a configuration diagram for explaining an embodiment (claim 1) of a moving image decoding apparatus according to the present invention.

FIG. 2 is a diagram showing an example of a rectangular area of the present invention.

FIG. 3 is an explanatory diagram showing a method of detecting a rectangular area according to the present invention.

FIG. 4 is an explanatory diagram showing another embodiment (claim 2) of the moving picture decoding apparatus according to the present invention.

FIG. 5 is an explanatory diagram for showing still another embodiment (claim 3) of the moving picture decoding apparatus according to the present invention.

FIG. 6 is a configuration diagram for explaining still another embodiment (claim 4) of the moving image decoding apparatus according to the present invention.

FIG. 7 is a configuration diagram for explaining another embodiment of claim 4;

[Fig. 8] Fig. 8 is a configuration diagram for explaining an embodiment (claim 5) of a moving image encoding device according to the present invention.

FIG. 9 is a configuration diagram for explaining another embodiment (Claim 6) of the moving picture coding device according to the present invention.

FIG. 10 is a configuration diagram for explaining still another embodiment of the moving image encoding device according to the present invention.

FIG. 11 is an explanatory diagram showing a conventional interpolation frame creation method.

FIG. 12 is a diagram for explaining a conventional problem.

[Explanation of symbols]

101, 201, 301 ... Moving image decoding unit, 102, 10
3,202,203,302,406,407,50
6,606 ... Frame memory, 104, 204, 30
4, 408, 507, 607 ... Square area detection / division unit, 105, 205, 410, 508, 608 ... Lattice point motion vector detection unit, 106, 206, 306, 41
1, 609 ... Interpolation frame lattice point motion vector calculation unit,
509 ... Interpolation frame lattice point motion vector detection unit, 41
6, 612 ... Comparator, 414, 415, 611 ... Error amount calculation section, 107, 207, 307, 412, 413, 6
10 ... Interpolation frame creation unit, 108, 208, 308 ...
Selector, 109, 209, 309 ... Display unit, 210,
614 ... Switch, 401, 501, 601 ... Interpolation frame determination unit, 402, 502, 602 ... Encoding frame memory, 403, 503, 603 ... Moving image encoding unit, 4
04, 504, 604 ... Interpolation frame memory, 405
505, 605 ... Local decoding / prediction image creating section, 409 ...
Rectangular area dividing unit, 417, 510, 613 ... Code creating unit.

Claims (10)

[Claims] 1. A moving image decoding apparatus for creating an interpolation frame between first and second decoded frames, wherein the first decoded frame is divided into rectangular areas surrounded by lattice points, and rectangular area detection / division is performed. Section and the grid point position on the second decoded frame corresponding to each grid point, the first,
A grid point motion vector detection unit that detects a grid point motion vector between the second decoded frames, and a grid point motion vector for the interpolation frame based on the grid point motion vector, and a grid point on the interpolation frame. For the interpolated frame motion vector calculation unit for obtaining the position and each quadrangle region surrounded by lattice points on the interpolated frame, one of the first decoded frame and the second decoded frame, or each quadrangle region corresponding to both And an interpolating frame creating section for synthesizing a rectangular area on the interpolating frame by transforming the shape of the above to create the interpolating frame from one or both of the first decoded frame and the second decoded frame. A video decoding device characterized by the above. 2. When the difference between the time interval between the first frame and the interpolation frame and the time interval between the second decoded frame and the interpolation frame is smaller than a predetermined value. Creating the interpolated frame using the first and second decoded frames, and otherwise creating the interpolated frame using the decoded frame on the side temporally close to the interpolated frame. The moving picture decoding device according to claim 1. 3. When the interpolation frame is created using the first and second decoded frames, the first interpolation frame is created using the first decoded frame, and the second decoding frame is created. A second interpolated frame is created using a frame, and the weighted average of the first and second interpolated frames is calculated according to the time interval between the interpolated frame and the first and second decoded frames. The moving picture decoding apparatus according to claim 1, wherein an interpolation frame is obtained. 4. When the grid point motion vector for all or part of the quadrangle region is included in the encoded data, the grid point motion vector is used for interpolation of the quadrangle region including the grid point motion vector. Using a lattice point motion vector included in the encoded data without using a detecting unit, interpolation of a rectangular area in which the encoded data does not include a lattice point motion vector is made by the lattice point motion vector detecting unit. The moving picture decoding device according to claim 1, 2 or 3, wherein a lattice point motion vector is used. 5. An image is divided into a plurality of quadrilateral regions, the quadrilateral region is shape-transformed using motion vectors of lattice points of the quadrilateral region to create a prediction image, and the prediction image and the original image are combined. A moving image encoding apparatus for encoding a prediction error, wherein an interpolation frame determination unit that does not transmit a prediction error in a specific frame and determines an interpolation frame to be synthesized from a reference frame on the decoding side, and the interpolation frame Is an interpolating frame, and if the position of the reference frame used for interpolation on the decoding side is not predetermined, the encoding unit for transmitting the information indicating the position of the reference frame is provided. A featured moving image encoding device. 6. An interpolating frame lattice point motion vector detecting unit for detecting a lattice point motion vector for the interpolating frame on the encoding side, and an encoding unit for transmitting the lattice point motion vector information. The moving picture coding device according to claim 5. 7. An error amount calculation unit for calculating an error from an original image for each rectangular area forming the interpolation frame,
For a rectangular area in which the error exceeds a predetermined threshold value, a coding means for coding the rectangular area on the original image, or prediction error information between the original image in the rectangular area and the interpolation frame 7. The moving picture coding apparatus according to claim 5, further comprising a coding means for coding. 8. In order to decode the coded data created by the moving picture coding apparatus according to claim 5, the information indicating that the frame is an interpolated frame included in the coded data and the decoding side interpolation are used. A moving image decoding apparatus, comprising: an interpolation frame creation unit that creates an interpolation frame from a reference frame by using information indicating the position of the reference frame when the position of the reference frame is not predetermined. 9. In order to decode the coded data created by the moving picture coding apparatus according to claim 6, the information of the lattice point motion vector for the interpolated frame included in the coded data is used to extract from the reference frame. 9. The moving image decoding apparatus according to claim 8, further comprising an interpolation frame creation unit that creates an interpolation frame. 10. In order to decode the encoded data created by the moving image encoding apparatus according to claim 7, the prediction error information between the original image and the interpolated image included in the encoded data, or the original image information is used. The moving picture decoding apparatus according to claim 8 or 9, further comprising: a creating unit that creates the rectangular area by using the creating unit.