JP4380789B2 - Motion compensation method - Google Patents

Description

    The present invention relates to a video signal encoding method, and more particularly to a motion compensation method using motion compensation prediction.

  FIG. 7 is a conceptual diagram showing an example of a conventional video signal encoding method disclosed in, for example, Japanese Patent Application Publication No. 2-914.

  One of the high-efficiency encoding methods for encoding a television signal is an inter-frame encoding method that encodes only the difference between the current frame and the previous frame. In this method, only the difference value between frames is sent, so if an error occurs after encoding, the error part will remain on the screen as an error permanently. Therefore, it is necessary to periodically erase the frame memory by using intra-frame coding regardless of the presence or absence of an error, and this error is called periodic refresh.

Incidentally, inter-frame coding using motion compensation prediction is performed in order to make inter-frame coding more efficient. In this motion compensated prediction, a motion vector that minimizes the difference between the previous frame and the current frame is detected by a motion vector detection circuit, and the motion vector and the difference value are encoded to suppress the generation of a code amount for the motion. To do. Even in the inter-frame coding using the motion compensation prediction, when an error occurs, it remains permanently, and the error moves according to the motion vector, so that the error spreads according to the motion. In order to eliminate this error, periodic refresh is required as in interframe coding.
However, when performing motion compensation prediction, if the direction of the incomplete refresh portion is detected as a motion vector that minimizes the interframe difference amount in the refresh completion portion, the interframe difference is between the refresh incomplete portion. It will be taken. If an error is included in the refresh incomplete portion, the error shifts to the refresh complete portion, causing a problem that periodic refresh is not performed completely.

  FIG. 7 shows a refresh completion region, a refresh incomplete region, a block in the refresh completion region, and a motion compensation prediction range for the block during execution of periodic refresh of the motion compensation interframe coding apparatus. In the Fth screen, when the periodic refresh area is the area 701, the area 702 is a refresh completion area, the area 703 is an incomplete refresh area, and the areas 704 and 705 are blocks in the refresh completion area. In the F-1th screen, a region 706 is a refresh completion region, a region 707 is a refresh incomplete region, a region 708 is a motion compensation prediction range for a region 704, and a region 709 is a motion compensation prediction range for a region 705. At this time, the motion compensated prediction range 709 of the block 705 near the boundary between the refresh area 701 and the refresh completion area 702 includes the refresh incomplete area 707. If normal motion compensation prediction is performed in block 705 and the motion vector indicates the refresh incomplete area 707, the difference value from the information in the refresh incomplete area is used in block 705, and block 705 Despite the completion of the refresh, the refresh is incomplete. That is, refresh is not completely performed.

  FIG. 8 shows a refresh completion region, a refresh incomplete region, a block in the refresh completion region, and a motion compensation prediction range for the block during execution of periodic refresh of the motion compensation interframe coding apparatus with a limited motion compensation prediction range. Show. In the Fth screen, when the periodic refresh area is the area 801, the area 802 is the refresh completion area, the area 803 is the refresh incomplete area, and the areas 804 and 805 are blocks in the refresh completion area. In the F-1th screen, a region 806 is a refresh completion region, a region 807 is a refresh incomplete region, a region 808 is a motion compensation prediction range for the region 804, and a region 809 is a motion compensation prediction range for the region 805. At this time, the motion compensation prediction range 809 of the block 805 near the boundary between the refresh area 801 and the refresh completion area 802 is limited so as not to include the refresh incomplete area 807. Thus, the refresh can be completely performed by limiting the motion compensation prediction range to the refresh completion region.

  In the conventional video signal encoding method, the frame for referring to motion compensation prediction is limited to the immediately preceding frame.

  The present invention has been made to solve the above problems, and an object of the present invention is to improve the accuracy of motion compensation prediction by using a plurality of reference frames for performing motion compensation prediction.

According to the motion compensation method of the present invention, the motion in the current frame is based on the current frame included in the image frame sequence constituting the image information and a plurality of reference frames temporally before the current frame. A motion compensation method for performing motion compensation for each block in a compensation target area, wherein each of the blocks is a plurality of blocks obtained by dividing the target area in the current frame into rectangular shapes. The motion compensation performed is performed according to the motion vector obtained from the motion compensation range in the reference frame, and the reference frame used for motion compensation in each block is at least the current frame among the plurality of reference frames in each block. It is selected from the reference frames that are one or two frames before the frame, and the current frame A block compensated for motion from the previous reference frame and a block compensated for motion from the previous reference frame, wherein the reference frame is a frame temporally after the predetermined frame, and the predetermined frame Is for not using a frame temporally prior to the predetermined frame as the reference frame, and the predetermined frame is an intra-coded frame that is closest in time to the current frame. In the reference frame that is temporally after the predetermined frame and temporally before the current frame, an area that can be used as a motion compensation range of each block is inter-coded. It is a frame.

  According to the present invention, it is possible to obtain a motion compensation method capable of improving the accuracy of motion compensation prediction.

It is a conceptual diagram of the motion compensation prediction in 1st Example of this invention. It is a conceptual diagram of the motion compensation prediction in 2nd Example of this invention. It is a conceptual diagram of the motion compensation prediction in 3rd Example of this invention. It is a conceptual diagram of the motion compensation prediction in 4th Example of this invention. It is a conceptual diagram of the motion compensation prediction in 5th Example of this invention. It is a schematic block diagram of the decoder in the 6th Example of this invention. It is a conceptual diagram of the motion compensation prediction in the conventional video signal encoding / decoding system. It is a conceptual diagram of motion compensation prediction with a limited prediction range in a conventional video signal encoding / decoding method. It is a figure which shows a screen division refresh system. It is a figure which shows the case where a refresh area | region is located in the uppermost part of a screen. It is a figure which shows the case where a refresh area | region is located other than the top part of a screen.

Examples of the present invention will be described below.
The trend toward digitization of video signals in recent years is spreading to a wide range of fields, and the spread has spread to the broadcasting field. The mainstream of current high-efficiency encoding methods for video signals is a method combining motion-compensated prediction and transform encoding method, and several standards using this method are being established.
Since motion compensation prediction is basically time domain predictive coding, it is necessary to set an initial value. Moreover, in order to prevent the propagation of an error that occurs accidentally after encoding, it is necessary to set the initial value at an appropriate period.
This periodic initial value setting operation is generally referred to as periodic refresh. The periodic refresh in the case of a method using motion compensation prediction and transform coding is specifically transform coding in the same screen without motion compensation prediction.

In general, in the case of a system having a plurality of decoders for one encoder as in broadcasting, the refresh cycle is involved in system performance. For example, the influence is exposed when power is turned on or when a channel is changed in television image reception. This is because it is impossible to reproduce a desired image until the refreshed encoded signal arrives.
From this point of view, it is preferable that the refresh cycle is short. However, excessive refresh, that is, frequent transform coding within the same screen, causes deterioration in compression efficiency, so the refresh cycle is actually about 0.3 to 0.5 seconds. Is set.

In general, the refresh method includes a method in which screens are collectively displayed and a method in which screens are divided. The system referred to in the conventional example corresponds to this latter.
In the case of the screen division refresh method, for example, the screen is divided into about 10 areas and refreshed one area at a time for one screen.
As described in the conventional example, in a system that applies motion compensation prediction, it is necessary to limit the motion compensation prediction range at the boundary of division in the screen division refresh method. This is because the prediction from the incomplete refresh portion occurring on the division boundary surface of the screen permanently prevents the acquisition of the desired image. There are various screen division refresh methods depending on the division method. FIG. 9 shows main division methods.

By the way, as described above, decoding is started at random with respect to the refresh cycle. That is, at the start of decoding, the entire screen is an incomplete refresh area. Therefore, at the time of encoding, for each refresh area, the upper and lower left and right boundaries in the example of FIG. 9 (a), the upper and lower boundaries in the example of FIG. 9 (b), and the left and right boundaries in the example of FIG. 9 (c). In this case, it is necessary to limit the motion compensation prediction range.
However, in the example of FIG. 9B, the restriction can be relaxed only at the lower boundary as in the following proof. (The same consideration is possible for FIG. 9 (c).)

Consider an example as shown in FIG. 9B, classified into two types as shown in FIGS. 10 and 11 according to the location of the refresh area at the start of decoding.
As shown in FIG. 10, when decoding is started from the F-th screen where the refresh area 1001 is located at the top of the screen, the refresh completion area 1003 (area 1004 in the F + 1 screen corresponding to the refresh area position of the F-th screen) )), There is no contamination due to the prediction from the incomplete region in the upper boundary, and therefore, the prediction range only needs to be limited only for the blocks belonging to the region 1004 in the lower boundary. The refresh completion area 1007 (including the area 1008) on the F + 2 screen corresponding to the refresh area position on the F + 1 screen also belongs to the area 1008 on the lower boundary because there is no contamination due to prediction from the uncompleted area on the upper boundary. It is sufficient that the prediction range is limited only for the block. That is, when decoding is started from the top of the screen on the initial screen (Fth screen), the periodic refresh is completed in this way only by limiting the lower boundary.
At this time, when the screen is divided into n regions, the refresh is completed on the n screen.

When decoding is started from the F-th screen in which the refresh area is located at a position other than the top of the screen as shown in FIG. 11, strictly speaking, the prediction range must be limited at the upper boundary and the lower boundary. However, even with only the lower boundary prediction range limitation, the periodic refresh is completely executed as follows.
When decoding starts from the Fth screen where the refresh area is located at a position other than the top of the screen as shown in FIG. 11, the refresh completion area in the (F + 1) th screen corresponding to the refresh area position of the Fth screen is at the upper boundary. There may be contamination due to predictions from unfinished areas. This contaminated area is shown as area 1105 in the figure.

However, since the width of the refresh area allocated by dividing the screen is actually larger than the width of the contaminated area, the periodic refresh is completely performed under the following conditions.
When the number of lines on one screen is L lines and the predicted range in the vertical direction is ± y lines, the number of lines that have been refreshed after f screen is
{(L / n) * f}-{y * (f-1)}
It becomes. If this is larger than the L line, the refresh is complete,
{(L / n) × f} − {y × (f−1)} ≧ L
{(L / n) −y} × f + y ≧ L
And
f ≧ (L−y) / {(L / n) −y}
It becomes. Assuming that the smallest integer equal to or larger than f is fint, the refresh is completely performed after the elapse of the fint screen.

By integrating the above two cases, the following conclusion is obtained.
The refresh completion time Tr when the refresh area at the start of decoding is the i-th area is
Tr = min [n + {n− (i−1)}, fint] (unit: screen time)
It becomes.
Note that the refresh completion time Tr can be obtained by the same method even when the refresh method is different.

Example 1.
The first embodiment of the present invention will be described below.
FIG. 1 is a conceptual diagram of motion compensated prediction in the first embodiment of the present invention.

Next, the operation will be described.
In the conventional example, the periodic refresh is executed by limiting the motion compensation range at the lower boundary. However, in the first embodiment, periodic refresh is performed with overlapping refresh areas as shown in FIG.
A description will be given taking the F + 1st screen as an example. An area 107 is a refresh area allocated by dividing the screen. An area 106 is an area added as an overlap area of the refresh area. By making the area 106 an overlapped refresh area, prediction from an incomplete area in the area 106 can be prevented. Thereby, the periodic refresh can be completed completely.

Example 2
The second embodiment of the present invention will be described below.
FIG. 2 is a conceptual diagram of motion compensation prediction in the second embodiment of the present invention.

Next, the operation will be described.
In the conventional example, the reference screen for motion compensation prediction is limited to one screen.
The second embodiment relates to a periodic refresh method when motion compensation prediction is performed from a plurality of reference screens.
FIG. 2 is a conceptual diagram of a periodic refresh method when motion compensation prediction is performed from two reference screens.
In FIG. 2, reference screens in the (F + 2) -th screen are two screens, that is, an F + 1-th screen and an F-th screen. At this time, as shown in FIG. 2, the motion compensation prediction range in the refresh completion region is restricted and the reference screen is restricted so that the motion compensation prediction range in the refresh completion region does not include the refresh incomplete region.

An example of the F + 2 screen will be described. An area 218 is a refresh area allocated by dividing the screen. At this time, the prediction range limited area 214 is prohibited from being predicted from the lower side of the Fth screen, the prediction range limited area 215 is prohibited from being predicted from the lower side and the parallel direction of the Fth screen, and the prediction range limited area 216 is the Fth screen. Prediction from the screen is prohibited, and prediction from the lower side of the Fth screen and (F + 1) th screen is prohibited in the prediction range limited area 217. Thereby, the periodic refresh can be completed completely.
Note that the prediction range and the reference screen may be restricted in any way as long as the refresh incomplete area is not included.

Example 3
The third embodiment of the present invention will be described below.
FIG. 3 is a conceptual diagram of motion compensation prediction in the third embodiment of the present invention.

Next, the operation will be described.
The third embodiment also relates to a periodic refresh method when motion compensation prediction is performed from a plurality of reference screens.
FIG. 3 is a conceptual diagram of a periodic refresh method when motion compensation prediction is performed from two reference screens.

In FIG. 3, reference screens in the (F + 2) -th screen are two screens (F + 1-th screen and F-th screen). At this time, as shown in FIG. 3, the refresh regions are overlapped so that the motion compensation prediction range in the refresh completion region does not include the refresh incomplete region, and the periodic refresh is performed.
An example of the F + 2 screen will be described. An area 312 is a refresh area allocated by dividing the screen. Regions 311 and 310 are regions added as overlap regions of the refresh region. By setting the region 311 and the region 310 as the refresh region, prediction from the incomplete region in the region 311 and the region 310 can be prevented. Thereby, the periodic refresh can be completed completely.

Example 4
The fourth embodiment of the present invention will be described below.
FIG. 4 is a conceptual diagram of motion compensation prediction in the fourth embodiment of the present invention.

Next, the operation will be described.
The fourth embodiment also relates to a periodic refresh method when motion compensation prediction is performed from a plurality of reference screens.
FIG. 4 is a conceptual diagram of a periodic refresh method when motion compensation prediction is performed from two reference screens. At this time, the refresh area is changed every two screens as shown in FIG. 4, and the motion compensation prediction range in the refresh completion area is limited so that the motion compensation prediction area in the refresh completion area does not include the refresh incomplete area.
The F + 2 screen and the F + 3 screen will be described as an example. In the F + 2 screen, an area 411 is a refresh area allocated by dividing the screen. At this time, the prediction range restriction area 410 is prohibited from being predicted from below the F-th screen and the (F + 1) -th screen. In the (F + 3) th screen, an area 415 is a refresh area allocated by dividing the screen. At this time, the prediction range limited area 414 is prohibited from being predicted from below the F + 1th screen. Thereby, the periodic refresh can be completed completely.
Note that the prediction range and the reference screen may be restricted in any way as long as the refresh incomplete area is not included.

Embodiment 5 FIG.
The fifth embodiment of the present invention will be described below.
FIG. 5 is a conceptual diagram of motion compensation prediction in the fifth embodiment of the present invention.

Next, the operation will be described.
The fifth embodiment also relates to a periodic refresh method for motion compensation prediction from a plurality of reference screens.
FIG. 5 is a conceptual diagram of a periodic refresh method when motion compensation prediction is performed from two reference screens. At this time, as shown in FIG. 5, the refresh area is changed every two screens, and the refresh areas are overlapped so that the motion compensation prediction range in the refresh completion area does not include the refresh incomplete area.

  The F + 2 screen and the F + 3 screen will be described as an example. In the F + 2 screen, an area 511 is a refresh area allocated by dividing the screen. An area 510 is an area added as an overlap area of the refresh area. By making the area 510 a refresh area, prediction from an incomplete area in the area 510 can be prevented. In the (F + 3) th screen, an area 515 is a refresh area allocated by dividing the screen. An area 514 is an area added as an overlap area of the refresh area. By making the area 514 a refresh area, prediction from an incomplete area in the area 514 can be prevented. Thereby, the periodic refresh can be completed completely.

Example 6
The sixth embodiment of the present invention will be described below.
FIG. 6 is a schematic block diagram of a decoder in the sixth embodiment of the present invention.
In FIG. 6, an encoded data series 601 input from the input terminal 1 is given to the first input of the image memory 4 and the selection circuit 5 via the decoding circuit 3. The output 603 of the image memory 4 is given to the second input of the selection circuit 5. A freeze signal 604 is supplied from the freeze signal generation circuit 6 to the third input of the selection circuit. The output of the selection circuit 5 is output from the output terminal 2 as an image signal 605.

Next, the operation will be described.
The sixth embodiment relates to processing in the decoder.
As described above, for example, when power is turned on or a channel is changed in receiving a television image, there is no guarantee that a desired image can be reproduced until the refresh is completely performed. Therefore, in the above case, the specific image or the latest decoded image is frozen until the refresh is completely performed.

In FIG. 6, during normal playback, the selection circuit selects and sends the output of the decoding circuit. At this time, the latest image is always stored in the image memory. When the power is turned on or when the channel is changed, the selection circuit selects and transmits the output of the image memory. At this time, the contents stored in the image memory are not updated. The selection circuit is switched by a freeze signal. The freeze signal generation circuit detects an inaccurate image decoding period until refresh is completed when the power is turned on or the channel is changed, and a freeze signal is transmitted. For example, the inaccurate image decoding period is given as follows in the above-described example.
min [n + (n−i), fint]
Also,
n + (n−i)
Or
fint
It is good.
Also,
min [n + (n−i), fint]
Other approximations may be performed as long as the value is larger.
Even when the refresh method is different, the inaccurate image decoding period can be obtained by the same method as described above.

DESCRIPTION OF SYMBOLS 1 Input terminal 2 Output terminal 3 Decoding circuit 4 Image memory 5 Selection circuit 6 Freeze signal generation circuit

Claims (1)

Based on the current frame included in the image frame sequence constituting the image information and a plurality of reference frames temporally before the current frame, each block in the motion compensation target area in the current frame A motion compensation method for performing motion compensation,
Each of the blocks is a block obtained by dividing the target area in the current frame into a rectangular shape,
Motion compensation performed for each block is performed according to a motion vector obtained from a motion compensation range in the reference frame,
The reference frame used for motion compensation in each block is selected from at least one reference frame that is temporally one or two times before the current frame among the plurality of reference frames in each block. Includes a motion compensated block from the previous reference frame and a motion compensated block from the previous reference frame;
The reference frame is a frame temporally after a predetermined frame,
The predetermined frame is for not using the frame that precedes in time than the predetermined frame as the reference frame,
The predetermined frame is an intra-coded frame that is closest in time to the current frame,
The reference frame that is temporally after the predetermined frame and temporally before the current frame is a frame in which an area that can be used as a motion compensation range of each block is interframe-coded. A motion compensation method characterized by the above.

Images