JP4222090B2 - Moving picture time axis interpolation method and moving picture time axis interpolation apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving image time axis interpolation method and a moving image time axis interpolation apparatus, and more particularly to an original moving image signal in order to obtain a moving image signal having an image rate different from the image rate of an incoming moving image signal. The present invention relates to a moving image time axis interpolation method and a moving image time axis interpolation apparatus that form a frame or field of a non-performing time by interpolating from an incoming moving image signal and change an effective frame (field) rate of the incoming moving image signal.
[0002]
[Prior art]
Since a normal NTSC television signal has 60 fields per second in interlace scanning, there is not much problem with the smoothness of motion of moving images. On the other hand, film images of 24 to 30 frames per second such as movies and computer graphics images created by sequential scanning at 30 frames per second have motion unnaturalness (judder and jerseyness).
[0003]
On the other hand, since the television signal of PAL system or SECAM system is 50 fields per second, in order to broadcast and display in the NTSC system, it is necessary to convert it to 60 fields per second. In addition, since the PAL method and the SECAM method have a low field frequency, there is a problem of large screen flicker, and it is desired that the display is converted to 75 fields or 100 fields instead of 50 fields.
[0004]
As described above, when converting to a moving image signal having an image rate different from that of the input moving image signal, it is desired that the motion of the image is natural. Therefore, a moving image rate conversion method (moving image time axis interpolation method) that performs motion compensation that changes the effective image rate with smooth motion by forming frames and fields that do not exist in time as motion compensated interpolation has been proposed. (For example, refer to Patent Document 1 and Patent Document 2).
[0005]
That is, in Patent Document 1, when a motion correction field that has been motion-corrected based on a motion vector is weighted and added with a predetermined field interpolation ratio, the field interpolation ratio is calculated based on the phase relationship of the field. A moving image time axis interpolation method that adaptively switches based on a fixed value and a motion vector value is disclosed.
[0006]
Further, Patent Document 2 searches for a block unit motion vector based on an image signal, generates a pixel unit motion vector based on the block unit motion vector, and interpolates an image signal generated based on the pixel unit motion vector. A method of converting the number of frames of an image signal using an interpolation frame of an image signal generated from a frame or a frame signal of an image signal is disclosed.
[0007]
Conventionally, a moving image interpolation apparatus shown in the block diagram of FIG. 4 is also known. FIG. 4 shows a moving image interpolating device for converting a progressively scanned image signal of 30 frames per second into a progressively scanned image signal of 60 frames per second. The progressively scanned image signal of 30 frames per second inputted from the image input terminal 1 is The signals are supplied to the frame delay unit 2, the motion estimator 51, the motion compensator 6, and the frame buffer 10, respectively. The frame delay unit 2 delays the inputted progressively scanned image signal of 30 frames per second by approximately one frame.
[0008]
The motion estimator 51 receives a 30-frame sequential scanning image signal from the image input terminal 1 and a 30-frame sequential scanning image signal delayed by about one frame by the frame delay unit 2 and performs interpolation. A motion vector (MV) of each part of the interpolated image is obtained from matching between frames before and after the position. Here, each part is a block of 8 × 8 pixels or the like, but it may not be a finer block or a square block.
[0009]
The motion compensator 6 spatially moves the progressively scanned image signal at 30 frames per second from the image input terminal 1 in accordance with the MV from the motion estimator 51. The motion compensator 7 spatially moves the progressively scanned image signal of 30 frames per second, which is delayed by approximately one frame, which is the output of the frame delay unit 2, according to the MV from the motion estimator 51. Here, MV is common to both motion compensators 6 and 7, but the moving direction is reversed between the motion compensator 6 and the motion compensator 7.
[0010]
The input image signal motion-compensated by the motion compensator 6 and the one-frame delayed image signal motion-compensated in the reverse direction by the motion compensator 7 are added by the adder 8 and then divided by 2 to become an interpolated image. . This interpolated image is stored in the frame buffer 11 and read out at a double speed. Similarly, an input progressive scanning image signal from the image input terminal 1 is also stored in the frame buffer 10 and read out at a double speed. In one frame time of the input image signal, the input image signal held in the frame buffer 10 and the interpolated image signal held in the frame buffer 11 are alternately read out to the image output terminal 13 via the switch 12. It is output as a progressively scanned image signal at twice the image rate, ie 60 frames per second.
[0011]
The state of interpolation by this conventional apparatus is shown in FIG. In the figure, one circle is one pixel, and shading indicates a pixel value. The vertical continuation indicates a part of the scanning line or a pixel column in the vertical direction. Also, what is indicated by a solid line is an input progressive scanning image signal of 30 frames per second, and what is indicated by a broken line is an interpolated image signal. The center is the current interpolated pixel, the dark and thin arrows are MVs with high correlation between the previous and next frames, and the thin and thick arrows are motion compensation interpolations by the MVs. FIG. 5 shows that the pixels move slowly with time, but appropriate interpolation can be achieved by motion compensation.
[0012]
[Patent Document 1]
JP 10-23374 A [Patent Document 2]
JP 11-1112939 A
[Problems to be solved by the invention]
However, since the conventional moving image time axis interpolation apparatus and method described above perform interpolation based on the correlation between the images before and after the image to be interpolated (frame or field), the correlation is large when there is a large change between the images before and after. However, there is a problem that there is no good image portion and proper interpolation cannot be performed. Further, since only one type of motion vector has been conventionally obtained, interpolation is not appropriately performed if the only obtained motion vector is inappropriate.
[0014]
The present invention has been made paying attention to the above points, and obtains an interpolated image that is forward and reverse in time and adapts both interpolated images according to the correlation and accuracy between the interpolated images. It is an object of the present invention to provide a moving image time axis interpolation method and a moving image time axis interpolation apparatus in which an appropriate interpolation image is formed even for an image portion having a large time change by adding and obtaining a final interpolation image.
[0015]
[Means for Solving the Problems]
In order to achieve the above object, the moving image temporal axis interpolation method of the present invention generates interpolated images (frames or fields) that are temporally different from the incoming moving image, and creates a new moving image from the incoming moving image and the interpolated image. A moving image temporal axis interpolation method for forming an image , wherein a frame or field of an incoming moving image is temporally estimated with respect to an interpolation image to be generated, and motion estimation is performed on the image using inter-image correlation. A first step of obtaining a backward- interpolated image that has been motion-compensated based on the backward-direction motion vector obtained from the result, a degree of correlation between the images in the motion estimation as reliability information of the backward-interpolated image, and generation motion based temporally frame or field past incoming video image to the interpolation image to be, in the forward direction of the motion vector obtained by a result of the motion estimation using image correlation with respect to the image A forward interpolated image amortization, a second step of obtaining a degree of inter-image correlation at the motion estimation as information of the reliability of the forward interpolation image, interpolation images in the forward direction interpolation image and backward interpolated image similarity If the inter-interpolated image similarity is high according to the third step for determining the degree, the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the inter- interpolation image similarity When the addition ratio is equal and the similarity between interpolated images is low, the interpolation image is obtained by adaptively weighting the forward interpolation image and the backward interpolation image so that the addition ratio of the interpolation image with higher reliability is increased. And a fifth step of forming a new moving image from the incoming moving image and the generated interpolated image.
[0016]
In order to achieve the above object, the moving image time axis interpolation apparatus of the present invention generates temporally different interpolation images (frames or fields) from the incoming moving image, and newly generates the incoming moving image and the interpolation image. A moving-image time axis interpolating apparatus for forming a moving image using a frame or a field of an incoming moving image that is temporally future with respect to the interpolated image to be generated and using an inter-image correlation with respect to the image A backward-interpolated image that is motion-compensated based on a backward-direction motion vector obtained from the motion estimation result, and a backward-interpolating unit that obtains the degree of correlation between the images in the motion estimation as reliability information of the backward-interpolated image ; , based frame or field of the temporally past incoming video image to the interpolation image to be generated, in the forward direction of the motion vector obtained by a result of the motion estimation using image correlation with respect to the image A forward interpolated image motion compensation, and forward interpolation means for obtaining the degree of inter-image correlation at the motion estimation as information of the reliability of the forward interpolation image, interpolation images in the forward direction interpolation image and backward interpolated image If the inter-interpolated image similarity is high according to the correlation detection means for determining the similarity, the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the inter- interpolation image similarity When the addition ratio is equal and the similarity between interpolated images is low, the interpolation image is obtained by adaptively weighting the forward interpolation image and the backward interpolation image so that the addition ratio of the interpolation image with higher reliability is increased. And an adaptive weighting unit that generates a moving image, and a moving image forming unit that forms a new moving image from the incoming moving image and the generated interpolated image.
[0017]
In the moving image temporal axis interpolation method and apparatus of the present invention described above, a backward interpolation image in which a frame or field of a future incoming moving image is temporally compensated for the interpolation image is formed, and the interpolation image is temporally corrected. A forward-interpolated image in which a frame or field of a past incoming moving image is motion-compensated is formed, information on the reliability of the backward-interpolated image and the forward-interpolated image, the backward-interpolated image and the forward-direction interpolated Since the interpolation image signal is obtained by adaptively weighting the backward interpolation image and the forward interpolation image according to the similarity between the interpolation images of the images, the final interpolation image signal is always obtained from the appropriate interpolation image. Can be formed.
The moving image time axis interpolation method and apparatus according to the present invention includes the fourth step or the adaptive addition means, the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the interpolation. Depending on the similarity between images, the higher the similarity between interpolated images, the higher the addition ratio is, so that the forward interpolation image and the backward interpolation image are added together. When the reliability of the interpolation image is higher than the reliability of the forward interpolation image, the forward interpolation image and the backward interpolation image are set so that the addition ratio of the backward interpolation image is larger than the addition ratio of the forward interpolation image. When the similarity between interpolated images is low and the reliability of the forward interpolation image is higher than the reliability of the reverse interpolation image, the addition ratio of the forward interpolation image is the addition ratio of the reverse interpolation image. Reverse to the forward-interpolated image to be larger than It may be configured to generate an interpolated image by performing an adaptive weighted addition for adding the interpolated image.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a moving picture time axis interpolation apparatus according to the present invention. In the figure, the same components as in FIG. In this embodiment, frame delay units 3 and 4, a subtractor 9, a weight determination unit 14, and multipliers 16 and 17 are added as compared with the conventional apparatus shown in FIG. The processing operations of the devices 5 and 15 are significantly different from those of the motion estimator 51.
[0019]
In FIG. 1, a moving image signal that is a progressively scanned image signal coming from an image input terminal 1 is supplied to a frame delay unit 2 and a motion estimator 5, respectively. The frame delay unit 2 delays the input moving image signal by approximately one frame. The output video signal from the frame delay unit 2 is supplied to the frame delay unit 3, the motion estimators 5 and 15, the motion compensator 6, and the frame buffer 10, respectively. The frame delay unit 3 delays the moving image signal input from the frame delay unit 2 by approximately one frame and supplies the delayed signal to the frame delay unit 4, motion estimators 5 and 15, and the motion compensator 7. The frame delay unit 4 delays the moving image signal input from the frame delay unit 3 by about one frame and supplies the delayed image signal to the motion estimator 15.
[0020]
Here, the input moving image signal from the image input terminal 1, the output delayed moving image signal of the frame delay device 2, the output delayed moving image signal of the frame delay device 3, and the output delayed moving image signal of the frame delay device 4 are substantially omitted. Used as 4 frames delayed by 1 frame. The input video signal is the first image, the output delay video signal of the frame delay unit 2 is the second image, the output delay video signal of the frame delay unit 3 is the third image, and the output delay video signal of the frame delay unit 4 is The fourth image.
[0021]
The motion compensator 6 spatially moves the second image according to the backward motion vector (MV _B ) supplied from the motion estimator 5. The motion compensator 7 spatially moves the third image according to the forward motion vector (MV _F ) supplied from the motion estimator 15. The second image that has been motion compensated by the motion compensator 6 becomes a backward interpolation image that is temporally future than the interpolated image, and is multiplied by the coefficient k supplied from the weight determiner 14 by the multiplier 16. Similarly, the third image that has undergone motion compensation by the motion compensator 7 becomes a temporally past forward-interpolated image from the interpolated image, and the coefficient (k−1) supplied from the weighting determiner 14 by the multiplier 17. And multiplied. The output signals of the multiplier 16 and the multiplier 17 are added by the adder 8 to form a final interpolated image signal, which is supplied to the frame buffer 11 and accumulated for one frame.
[0022]
Further, the backward interpolation image and the forward interpolation image are subtracted by the subtracter 9, and the difference as the subtraction result is supplied to the weight determination unit 14. The weight determination unit 14 obtains a correlation result between the interpolation signals of the block by multiplying the incoming difference by an absolute value or square and spatially integrating the difference. Further, the reliability information of interpolation supplied from the motion estimators 5 and 15 is obtained, the coefficient k is determined based on the correlation result and the reliability information, and is supplied to the multipliers 16 and 17. A specific method for determining k will be described later.
[0023]
The interpolated image signal for one frame stored in the frame buffer 11 is read at twice the speed at the time of writing. On the other hand, after the second image output from the frame delay unit 2 is stored in the frame buffer 10 for one frame, it is read out at a speed twice as high as that at the time of writing. The moving image signal (second image) held in the frame buffer 10 and the interpolated image signal held in the frame buffer 11 are alternately read out in one frame time of the incoming moving image signal, and the incoming moving image It is output as a moving image signal having an image rate that is twice the image rate of the image signal.
[0024]
In this embodiment, motion estimators 5 and 15 are provided instead of the conventional motion estimator 51. FIG. 2 shows a block diagram of an embodiment of the motion estimator 5. In FIG. 2, the frame delay units 2 and 3 are common to the frame delay units 2 and 3 of FIG. The incoming moving image signal as the first image is sent to the motion compensator 21, the output moving image signal from the frame delay device 2 as the second image is sent to the motion compensator 22, and the output moving image from the frame delay device 3 as the third image. The image signals are supplied to the motion compensator 23, respectively. The motion compensators 21, 22 and 23 perform motion compensation in accordance with the temporary MV given from the temporary MV setting unit 29.
[0025]
Here, the temporary MV is common to all the motion compensators 21, 22, and 23, but the amount of spatial movement differs depending on the motion compensators 21, 22, and 23. If the second image motion compensator 22 is set to “1” for both the horizontal component and the vertical component from the temporal relationship between the interpolated image and each image, the first image motion compensator 21 is “3”, and the third image The motion compensator 23 for use has a relative relationship of “−1”.
[0026]
The first image motion-compensated by the motion compensator 21 is sent to the subtractor 31, the second image motion-compensated by the motion compensator 22 is sent to the subtractor 31 and the subtracter 32, and the first image that has been motion compensated by the motion compensator 23. The three images are supplied to the subtracter 32, respectively. The subtractor 31 obtains the difference between the first image and the second image and supplies it to the correlation detector 41. The subtractor 32 obtains the difference between the second image and the third image and supplies the difference to the correlation detector 42.
[0027]
The correlation detectors 41 and 42 convert the input difference into an absolute value or a square, and cumulatively add the difference in the block to obtain a correlation result of the block in the temporary MV. The output correlation result of the correlation detector 41 is a correlation result between the first image and the second image, and is supplied to the correlation comparator 25 as the first correlation. On the other hand, the output correlation result of the correlation detector 42 is a correlation result between the second image and the third image, and is supplied to the correlation comparator 25 as the second correlation.
[0028]
The correlation comparator 25 compares the input first correlation with the second correlation, and supplies the evaluation value of each temporary MV to the MV determiner 26. The MV determination unit 26 compares the evaluation value in each temporary MV between the temporary MVs from the temporary MV setting unit 29, determines the temporary MV that gives the highest evaluation value as the final MV, and outputs the MV output terminal. 27, the motion vector is output as a backward motion vector (MV _B ). The temporary MV setting unit 29 generates a temporary MV set as a search range in the block, and supplies the information to the motion compensators 21, 22, and 23 and the MV determination unit 26.
[0029]
Next, a process for determining an evaluation value and a coefficient k from two types of correlation will be described. Since the output correlation result of the correlation detectors 41 and 42 is a block matching value, the larger the value, the worse the correlation (the correlation is low).
[0030]
The evaluation value C is obtained from the first correlation C ₁ output from the correlation detector 41 and the second correlation C ₂ output from the correlation detector 42 under the following conditions. When the value obtained by doubling one of the first and second correlations is equal to or greater than the value of the other correlation, the values C ₁ and C ₂ of the first and second correlations are added. Is an evaluation value C. Also, when the value obtained by doubling one of the first and second correlations is less than the value of the other correlation, only one of the correlation values is used as the evaluation value C. Here, since the total correlation is better when added, it is multiplied by 3 when not added for correction. The value obtained in this way becomes the evaluation value C of the block concerned temporary MV. The above is summarized as follows.
[0031]
C = (C ₁ + C ₂ )... 2C ₁ ≧ C ₂ or 2C ₂ ≧ C ₁
C = 3C ₁ ... 2C ₁ <C ₂
C = 3C ₂ ... 2C ₂ <C ₁
Evaluation value C is in addition used for the determination of MV in MV determiner 26, an evaluation value C for the MV that is finally determined in the block is output from the reliability information output terminal 28 as a trusted information C _B interpolation .
[0032]
The configuration and operation of the motion estimator 15 are basically the same as those of the motion estimator 5, and only the used image is different. Instead of the first image, the second image, and the third image in the motion estimator 5, the second image, the third image, and the fourth image are used in the motion estimator 15. The ratio of the spatial movement amount in motion compensation also changes from the relationship of the time position with the interpolated image. If the movement amount for the third image is “1” for both the horizontal component and the vertical component, the movement amount for the fourth image is “3” and the movement amount for the second image is “−1”. Thus, from the motion estimator 15 reliability information _{C F} is outputted as the forward motion vector (MV _F).
[0033]
Referring back to FIG. 1 again, in the weight determination unit 14, the coefficient k is the backward interpolation reliability information C _B given from the motion estimator 5 and the forward interpolation reliability information C given from the motion estimator 15. and _F, is determined by the following equation from the correlation C _I of the interpolation image. Since each value is a matching value, the greater the value, the lower the reliability (the lower the correlation).
[0034]
k ′ = 0.5 + (C _F −C _B ) C _I / (C _F + C _B ) ²
k = k ′... 1 ≧ k ′ ≧ 0
k = 1 ... k '> 1
k = 0 ... k '<0
In the above equation, the second term higher similarity value is less forward interpolated image and backward interpolated image C _I approaches 0, k approaches 0.5. On the other hand, when the value of C _B is high reliability of the small reverse interpolation than C _F, the second term becomes positive, k is close to 1. If the value of C _F is high reliability smaller than the forward interpolation C _B Conversely, the second term is negative, k approaches 0. k is limited to between 0 and 1.
[0035]
As described above, the weight determination unit 14 in FIG. 1 performs weighted addition. However, as a slightly simpler process, the above-obtained k is reduced to 0, 0.5, and 1, forward interpolation, It is also possible to select from backward interpolation or an average of both.
[0036]
Further, since the embodiment of FIG. 1 is an example of conversion of the image rate from 30 fps to 60 fps, the time of the interpolated image is the center of the input moving image signal, and the reference (first term) in the equation of k is 0.5. It has become. However, for example, in the conversion of the image rate from 50 fps to 60 fps, the interpolated image is not necessarily centered in time, so the first term in the above equation is not 0.5, and is set to 0 or 1 depending on the time distance. Suppose you are approaching. In this case, the relationship between the motion vector movement amounts also varies depending on the time relationship.
[0037]
Next, time axis interpolation in this embodiment will be described. FIG. 3 shows a state of time axis interpolation according to the present embodiment. In FIGS. 3A, 3B, and 3C, one circle is one pixel, and shading indicates a pixel value. The continuous in the vertical direction indicates a part of the scanning line or the pixel column in the vertical direction. The numbers 1 to 4 indicate the first image to the fourth image. Further, pixels indicated by a solid line are input moving image signals of 30 frames per second, and pixels indicated by a dotted line are interpolation signals. The center is the current pixel to be interpolated, the dark and thin arrows are MVs having a high correlation between the previous and subsequent frames, and the thin and thick arrows are motion compensation interpolations by the MVs. Moreover, the arrow shown with a dotted line is not used for MV determination.
[0038]
FIG. 3A shows a state of reverse-interpolated image formation that is an output of the motion compensator 6. FIG. 3B shows a state of forward-direction interpolation image formation that is an output of the motion compensator 7, but a motion vector different from the backward-direction interpolation image formation of FIG. . FIG. 3 (c) shows the state of adaptive addition interpolation in the adder 8 using both forward interpolation and backward interpolation. In the example of FIG. Since the reliability of reverse interpolation is better than the reliability of forward interpolation, k approaches 1 and reverse interpolation is mainly used.
[0039]
As described above, according to the present embodiment, when an interpolated image is formed using a plurality of moving image signals different from each other by one frame from an incoming moving image signal, a past image in time of the interpolated image is converted. Use to form a forward-interpolated image and use a temporally future image of the interpolated image to form a backward-interpolated image, so that even if there is a large image change before and after the interpolated image signal, the forward direction In one of the interpolation and the backward interpolation, a motion vector for motion compensation is appropriately obtained.
[0040]
Further, according to the present embodiment, the degree of addition of the forward direction interpolation image and the backward direction interpolation image is calculated from the similarity between the forward direction interpolation image and the backward direction interpolation image and the reliability information of each interpolation image. By changing and adding, the final interpolated image is always formed from the appropriate interpolated image, so that an appropriate interpolated image can be stably formed without being influenced by the degree or change of the image movement, In particular, when both backward and forward interpolations are appropriate, equal addition is performed, so that the phase relationship of the time axis is linear and the effect of suppressing random noise is great. As a result, an image rate converted image with smooth motion and no image quality deterioration can be obtained.
[0041]
The present invention is not limited to the above-described embodiment. For example, a field delay device that delays approximately one field can be used instead of the frame delay devices 2 to 4.
[0042]
【The invention's effect】
As described above, according to the present invention, a backward interpolation image in which a frame or a field of a future incoming moving image is temporally compensated for the interpolation image is formed, and the interpolation image is temporally past. Forward-interpolated images in which frames or fields of the incoming moving image are motion-compensated are formed, information on the reliability of these backward-interpolated images and forward-interpolated images, and interpolation of the backward-interpolated images and forward-interpolated images According to the similarity between images, the interpolation image signal is obtained by adaptively weighting the backward interpolation image and the forward interpolation image so as to always form the final interpolation image signal from the appropriate interpolation image. Therefore, even if there is a large image change before and after the interpolated image, an appropriate interpolated image signal can be formed stably without being influenced by the degree or change of the image movement. If Since the forward interpolation image and the backward interpolation image are evenly added, the phase relationship of the time axis is linear, the effect of suppressing random noise is large, and as a result, the image rate conversion image with smooth motion and no image quality degradation Is obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration of an embodiment of a motion estimator in FIG. 1;
FIG. 3 is a diagram showing a state of motion compensation interpolation in FIG. 1;
FIG. 4 is a block diagram showing a configuration of an example of a conventional apparatus.
FIG. 5 is a diagram illustrating a state of conventional motion compensation interpolation.
[Explanation of symbols]
1 Image input terminals 2, 3, 4 Frame delay unit 5, 15 Motion estimator 6, 7, 21, 22, 23 Motion compensator 8 Adder 9, 31, 32 Subtractor 10, 11 Frame buffer 12 Switch 13 Image output Terminal 14 Weighted decision device 16, 17 Multiplier 25 Correlation comparator 26 MV decision device 27 MV output terminal 28 Reliability information output terminal 29 Temporary MV setting device 41, 42 Correlation detector

Claims (4)

A moving picture time axis interpolation method for generating a temporally different interpolation image (frame or field) from an incoming moving picture and forming a new moving picture from the incoming moving picture and the interpolated image,
Based on the motion vector in the reverse direction obtained from the result of motion estimation of the frame or field of the incoming moving image that is future in time with respect to the interpolated image to be generated , using inter-image correlation for the image. A first step of obtaining a motion compensated backward interpolation image and a degree of correlation between images in the motion estimation as reliability information of the backward interpolation image ;
Based on a forward motion vector obtained from a result of motion estimation of the frame or field of the incoming moving image that is past in time with respect to the interpolated image to be generated , using inter-image correlation for the image. A second step of obtaining a motion-compensated forward-interpolated image and a degree of correlation between the images in the motion estimation as reliability information of the forward-interpolated image ;
A third step for obtaining a similarity between the interpolated images of the forward interpolation image and the backward interpolation image;
According to the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the similarity between the interpolation images, if the similarity between the interpolation images is high, the addition ratio is evenly distributed. When the similarity between the interpolated images is low, the forward interpolated image and the reverse interpolated image are adaptively weighted and added so that the addition ratio of the interpolated image with higher reliability becomes larger. A fourth step of generating;
A moving image time axis interpolation method comprising: a fifth step of forming a new moving image from the incoming moving image and the generated interpolated image. A moving picture time axis interpolation method for generating a temporally different interpolation image (frame or field) from an incoming moving picture and forming a new moving picture from the incoming moving picture and the interpolated image,
Based on the motion vector in the reverse direction obtained from the result of motion estimation of the frame or field of the incoming moving image that is future in time with respect to the interpolated image to be generated, using inter-image correlation for the image. A first step of obtaining a motion compensated backward interpolation image and a degree of correlation between images in the motion estimation as reliability information of the backward interpolation image;
Based on a forward motion vector obtained from a result of motion estimation of the frame or field of the incoming moving image that is past in time with respect to the interpolated image to be generated, using inter-image correlation for the image. A second step of obtaining a motion-compensated forward-interpolated image and a degree of correlation between the images in the motion estimation as reliability information of the forward-interpolated image;
  A third step for obtaining a similarity between the interpolated images of the forward interpolation image and the backward interpolation image;
According to the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the similarity between the interpolation images, the higher the similarity between the interpolation images, the closer the addition ratio is. The forward interpolation image and the backward interpolation image are added so that the similarity between the interpolation images is low and the reliability of the backward interpolation image is higher than the reliability of the forward interpolation image. Adds the forward interpolation image and the backward interpolation image so that the addition ratio of the backward interpolation image is larger than the addition ratio of the forward interpolation image, and the similarity between the interpolation images is reduced. When the reliability of the forward interpolation image is higher than the reliability of the backward interpolation image, the forward direction is set so that the addition ratio of the forward interpolation image is larger than the addition ratio of the backward interpolation image. Interpolated image and backward interpolation A fourth step of generating the interpolated image by performing an adaptive weighted addition for adding the image,
A fifth step of forming a new moving image from the incoming moving image and the generated interpolated image;
A video time axis interpolation method characterized by comprising: A moving image time axis interpolation device that generates an interpolation image (frame or field) that differs in time from an incoming moving image, and forms a new moving image from the incoming moving image and the interpolation image,
Based on the motion vector in the reverse direction obtained from the result of motion estimation of the frame or field of the incoming moving image that is future in time with respect to the interpolated image to be generated , using inter-image correlation for the image. A backward interpolation means for obtaining a motion compensated backward interpolation image and a degree of correlation between the images in the motion estimation as reliability information of the backward interpolation image ;
Based on a forward motion vector obtained from a result of motion estimation of the frame or field of the incoming moving image that is past in time with respect to the interpolated image to be generated , using inter-image correlation for the image. Forward-interpolated means for obtaining the motion-compensated forward-interpolated image and the degree of correlation between the images in the motion estimation as reliability information of the forward-interpolated image ;
Correlation detection means for obtaining similarity between interpolated images of the forward direction interpolation image and the reverse direction interpolation image;
According to the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the similarity between the interpolation images, if the similarity between the interpolation images is high, the addition ratio is evenly distributed. When the similarity between the interpolated images is low, the forward interpolated image and the reverse interpolated image are adaptively weighted and added so that the addition ratio of the interpolated image with higher reliability becomes larger. Adaptive weighting means to generate,
A moving image time axis interpolating device comprising moving image forming means for forming a new moving image from the incoming moving image and the generated interpolated image. A moving image time axis interpolation device that generates an interpolation image (frame or field) that differs in time from an incoming moving image, and forms a new moving image from the incoming moving image and the interpolation image,
Based on the motion vector in the reverse direction obtained from the result of motion estimation of the frame or field of the incoming moving image that is future in time with respect to the interpolated image to be generated, using inter-image correlation for the image. A backward interpolation means for obtaining a motion compensated backward interpolation image and a degree of correlation between the images in the motion estimation as reliability information of the backward interpolation image;
Based on a forward motion vector obtained from a result of motion estimation of the frame or field of the incoming moving image that is past in time with respect to the interpolated image to be generated, using inter-image correlation for the image. Forward-interpolated means for obtaining the motion-compensated forward-interpolated image and the degree of correlation between the images in the motion estimation as reliability information of the forward-interpolated image;
  Correlation detection means for obtaining similarity between interpolated images of the forward direction interpolation image and the reverse direction interpolation image;
According to the reliability information of the backward interpolation image, the reliability information of the forward interpolation image, and the similarity between the interpolation images, the higher the similarity between the interpolation images, the closer the addition ratio is. The forward interpolation image and the backward interpolation image are added so that the similarity between the interpolation images is low and the reliability of the backward interpolation image is higher than the reliability of the forward interpolation image. Adds the forward interpolation image and the backward interpolation image so that the addition ratio of the backward interpolation image is larger than the addition ratio of the forward interpolation image, and the similarity between the interpolation images is reduced. When the reliability of the forward interpolation image is higher than the reliability of the backward interpolation image, the forward direction is set so that the addition ratio of the forward interpolation image is larger than the addition ratio of the backward interpolation image. Interpolated image and backward interpolation An adaptive weighting means for generating the interpolated image by performing an adaptive weighted addition for adding the image,
Moving image forming means for forming a new moving image from the incoming moving image and the generated interpolated image;
A video time-axis interpolation apparatus characterized by comprising:

Images