JP2006211010A - Filter and picture interpolator, and method for specifying filter coefficient - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a filter and a picture interpolator for forming a new picture with little blur, and also to provide a method for specifying filter coefficients. <P>SOLUTION: The picture interpolator/extrapolator 1 for interpolating an inputted motion picture including continuous pictures comprises: a means 3 for delaying a picture included in the motion picture; a means 5 for multiplying a delayed picture by filter coefficients; a means 7 for outputting the multiplication results while switching; and a means 9 for forming a new picture (interpolation picture) by adding the multiplication results. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a filter and an image interpolation device used when an image is interpolated into a moving image, and a filter coefficient specifying method for specifying a coefficient of the filter.

  In general, a moving image or a three-dimensional tomographic image of a human body or the like obtained by CT scan or MRI is composed of a plurality of still images (hereinafter simply referred to as images). When this image is temporally continuous, it is replaced instantaneously (60 times per second) and displayed as a moving image. When the image is spatially continuous, it is replaced and displayed. Thus, a three-dimensional tomographic image is obtained.

  In this moving image, since a plurality of images are handled together, the same interpolation technique can be applied. This interpolation technique creates a new image between known images. As this applied technology, there are applications such as system conversion and non-interlace conversion in moving images, and there are applications such as polygon mesh generation that generates shape data from images between the three-dimensional tomographic images in three-dimensional tomographic images.

  By the way, in the method conversion and non-interlace conversion, a technique for creating an interpolated image while maintaining the resolution of the image by using a motion vector indicating the motion of the subject between images has been established (for example, (See Patent Documents 1 and 2).

  Here, interpolation and extrapolation will be described with reference to FIG. FIG. 5 shows a change in the image level of a moving image in which images are continuous in a graph in which time is plotted on the horizontal axis and image level is plotted on the vertical axis. In this graph, the moving image shows an image in which the image level changes in a gentle wave shape with the passage of time, and “◯” indicates discrete and continuous images. ”Indicates a new image to be interpolated or extrapolated

In the case of interpolation, for example, a new image is created using an image of “◯” included in the section b and inserted into the section b.
In the case of extrapolation, for example, a new image is created using an image of “◯” included in the section b and inserted into a section other than the section b, that is, the section a or the section c. .
JP-A-62-230180 JP-A-62-230178

  However, it is difficult to detect an appropriate motion vector for all images in units of block images obtained by subdividing the image into a predetermined number of pixels. For this reason, a new image is created using linear interpolation shown in FIG. 6 for an image for which an appropriate motion vector could not be detected.

As shown in FIG. 6, a new image x (2) is linearly placed between the existing image x (1) = 50 (image level) and the existing image x (3) = 60 (image level). When creating by insertion, if the number of images from the existing image x (1) to the new image is m = 3 and the number of images from the existing image x (3) to the new image is n = 2 ,
x (2) = {2 · x (1) + 3 · x (3)} / (m + n)
= 56
It becomes.

  Such linear interpolation is an effective method that can create a new image with a simple calculation. However, since a new image is uniformly obtained regardless of the pixel value of the pixel of the existing image. There is a problem that the created new image tends to be a blurred image.

  Therefore, an object of the present invention is to provide a filter, an image interpolation device, and a filter coefficient specifying method that can solve the above-described problems and create a new image with less blur.

  In order to solve the above-described problem, the filter according to claim 1 is a filter having a plurality of filter coefficients used when interpolating a moving image in which images are continuous, and multiplying the image, and between existing images. If the number of samples from the front and back images of the interpolated image to be interpolated is 3, and the sampling positions are two before, one before and one after the interpolated image, When the filter coefficient related to the inserted image is set to 1 and enumerated from the filter coefficient related to the subsequent image, the filter coefficient is (3/8, 1, 3/4, 0, −1/8) or the previous image When enumerating from such filter coefficients, the filter coefficients are (−1/8, 0, 3/4, 1, 3/8).

According to such a configuration, the filter has filter coefficients in the case where the number of samplings is 3, and the sampling positions are two before, one before, and one after the interpolation image. This filter is configured as, for example, a 3-tap asymmetric digital filter. Further, for example, the filter coefficient is obtained by determining the number of times that the function indicating the change in the image level of the moving image can be differentiated and executing the Taylor expansion. The Taylor expansion is in a certain section. If the function f (x) can be differentiated n times, a is a fixed point (corresponding to the sample position of the existing image) and x is an arbitrary point (corresponding to the sample position of the image to be interpolated) in the interval. When
f (x) = f (a) + (x−a) f ′ (a) / 1! + (X−a) 2f ″ (a) / 2! +... + (X−a) ⁿ⁻¹ f ⁽ⁿ⁻¹⁾ (a) / (n−1)! + (X−a) ⁿ f ⁽ⁿ⁾ (ξ) / n! Equation (1) is established.

  However, ξ = a + θ (x−a), 0 <θ <1, and in this formula (1), if n times differentiation of f (a) and f (a) is given, the value of f (x) is calculated. It means you can do it.

  In addition, an interpolated image creates and inserts the arbitrary images between one image and the other image, when interpolating in a time-axis direction. In addition, when interpolated in the horizontal and vertical directions, the interpolated image has the number of horizontal pixels set to m + α and the number of vertical pixels set to n + β when the number of horizontal pixels in the original image is m and the number of vertical pixels is n. It is. The number of samples is the number of images (or pixels) used for interpolation, and the sampling position is the number of pixels that correspond to the frame number, field number, or time, or the number of pixels in one image. Corresponds to position. For reference, in the case of an extrapolated image, an image within a section limited from one image to the other image is used to create and insert an arbitrary image outside the section.

  The filter according to claim 2 is a filter having a plurality of filter coefficients to be used when interpolating a moving image in which images are continuous, and interpolating between existing images. The number of samples indicating the number of samples from the preceding and following images is set to five, and the sampling positions indicating the sampling positions are two before, one before, one after, two after and three after the interpolated image. When the filter coefficient related to the interpolated image is set to 1 and enumerated from the filter coefficient related to the post-image, the filter coefficient is (−1/96, 0, −1/48, 0, 1 / 2, 1, 29/48, 0, 7/96), or enumeration from the filter coefficients related to the previous image, the filter coefficients are (−7/96, 0, 29/48, 1/2, − 1 / 48,0, -1 / 96) And wherein the Rukoto.

  According to such a configuration, the filter has a filter coefficient when the number of samplings is 5, and the sampling positions are two before, one before, one after, two after, and three after the interpolation image. have. This filter is configured as, for example, a 5-tap asymmetric digital filter.

  An image interpolation apparatus according to claim 3 is an image interpolation apparatus that includes the filter according to claim 1 or 2, and performs interpolation of a moving image in which an input image is continuous. The filter coefficient multiplying unit, the switching unit, and the image generating unit are provided.

  According to this configuration, the image interpolation device delays the input image by the delay unit. By this delay means, an input image so as to be a plurality of sampling positions (for example, two previous, one previous, one after using the filter of claim 1) from a new image to be interpolated Is delayed by a preset delay amount over a plurality of stages. Subsequently, the image interpolation device multiplies the image delayed by the delay unit by the filter coefficient multiplication unit by the filter coefficient set by the filter in accordance with the delayed delay amount. That is, the filter coefficient is multiplied for each delayed image.

  The image interpolation device switches the output of the image multiplied by the filter coefficient multiplication unit by the switching unit according to the interpolation timing, and synthesizes the images whose outputs are switched by the switching unit by the image generation unit. Then, an interpolation image used for interpolation is generated. In other words, the switching means multiplies the filter coefficients multiplied to be combined, and the output timing is adjusted so that a new image (interpolated image) is inserted into the original moving image at an accurate timing. Will be.

  The filter coefficient specifying method according to claim 4 is a filter coefficient specifying method for specifying a plurality of filter coefficients to be multiplied by an image in a filter used for interpolation or extrapolation to a moving image in which images are continuous. The procedure includes a differentiation number determination step, a differential coefficient replacement step, and a filter coefficient identification step.

  According to this procedure, the filter coefficient specifying method determines the number of times that the function indicating the change in the image level of the moving image can be differentiated by a preset processing method in the differentiation number determination step. The preset processing method includes a processing method for generating an interpolated image between frame images or field images, that is, in a time axis direction, and a case of generating an interpolated image in a horizontal and vertical direction (image Is the processing method. In this differentiation number determination step, for example, when generating an interpolated image in the time axis direction, the number of differentiable times is set to three, and when generating an interpolated image in the horizontal and vertical directions, the number of differentiable numbers is set to 2. Set times. The number of differentiable times corresponds to the number of filter taps.

  Subsequently, in the filter coefficient specifying method, in the differential coefficient replacement step, the insertion position for inserting an image to be interpolated or extrapolated is set to x based on the number of differentiation determined in the number of differentiation determination step, and the function can be differentiated. Assuming that the acquisition position for acquiring a simple image is a, the number of terms indicating the differential coefficient is set for Equation 1 below, and the differential coefficient in this term is set before and after the image for which the differential coefficient is obtained. Replace with the difference equation using the function that represents the image.

f (x) = f (a) + (x−a) f ′ (a) / 1! + (X−a) ² f ″ (a) / 2!... + (X−a) ⁿ f ⁿ (a) / n!
... Formula 1

That is, the filter coefficient specifying method sets the number of terms f ′ (a), f ″ (a),..., F ⁿ (a) in Equation 1, and uses these terms as the preceding and succeeding images. For example, f ′ (a) = f (a + 1) −f (a−1) is substituted for a differential equation using the function shown in FIG. It uses the principle.

  In the filter coefficient specifying method, the filter coefficient specifying step adds the coefficients related to the terms of the difference equation replaced in the differential coefficient replacing step for each function indicating an image to obtain a filter coefficient. In other words, when a term obtained by substituting the differential coefficient term by the difference method is set to f (a + 1), for example, Af (a + 1), Bf (a + 1), Cf (a + 1) is obtained, this term f ( The coefficient A + B + C concerning a + 1) is one of the filter coefficients.

  According to the first and second aspects of the invention, an asymmetric digital filter can be realized by setting the filter coefficient asymmetrically with respect to the sampling position of the interpolated image to be inserted. As a result, if an image is interpolated using this filter, it is not a uniform filter coefficient, so a new image (interpolated image) with less blur can be created compared to simple linear interpolation. .

  According to the third aspect of the present invention, the delayed image is multiplied by the filter coefficient using the filter having the filter coefficient in which the sampling position is set asymmetrically with respect to the sampling position of the interpolated image, and is output. By adjusting and adding the timing, a new image (interpolated image) with less blur can be created.

  According to the invention of claim 4, the number of differentiable times is determined, and each term in the mathematical formula based on the Taylor expansion principle is converted into a differential equation using a function indicating an image before and after the image for which the differential coefficient is obtained. By substituting and calculating, the filter coefficient of the filter can be specified. If this filter is used, since it is not a uniform filter coefficient, it is possible to create a new image (interpolated image) with less blur than simple linear interpolation.

Next, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
<Configuration of image interpolation device>
FIG. 1 is a block diagram of an image interpolation apparatus. As shown in FIG. 1, the image interpolation device 1 creates a new image using an image (frame image, field image) included in the moving image as an input moving image, The created new image is inserted (interpolated) into the moving image, and includes a delay unit 3, a filter coefficient multiplication unit 5, a switching unit 7, and an image generation unit 9. The image interpolation device 1 is realized by using an asymmetric digital filter or the like for the filter coefficient multiplication means 5.

The delay means 3 sequentially delays the input moving image by a preset delay amount, and a plurality of delay units (3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ) as delay elements are provided. I have. The delay units (3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ) are connected in series. Note that the number of the delay units (3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ) corresponds to the number of times the input moving image is differentiated according to Equation (1). The delay amount is a sampling rate of the input moving image. FIG. 1 shows an example of generating three times as many pixels, and FIG. 2 shows an example of generating twice as many pixels.

The filter coefficient multiplication means 5 multiplies the image delayed by each of the delay parts (3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ) of the delay means 3 by a filter coefficient. 5 ₁ , 5 ₂ , 5 ₃ ,..., 5 _{n + 1} ). This multiplication section (5 ₁ , 5 ₂ , 5 ₃ ,..., 5 _{n + 1} ) is the number n of delay sections (3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ) of the delay means 3. One more is provided depending on the situation. Furthermore, each multiplication unit (5 ₁ , 5 ₂ , 5 ₃ ,..., 5 _{n + 1} ) has three filter coefficients (three consecutive among α _{0 to} α _{3n + 2} ) in parallel. It is set and configured to output three multiplication results. One of the multiplication results is switched (selected) by the switching means 7 and added by the image generating means 9.

This filter coefficient is not obtained by simple frequency conversion, but is obtained by Taylor expansion, and will be described in detail below.
As is well known, Taylor expansion is the mathematical theorem shown below: “If f (x) is differentiable up to the nth time in a certain interval, a is a fixed point and x is an arbitrary point in that interval. F (x) = f (a) + (x−a) f ′ (a) / 1! + (X−a) ² f ″ (a) / 2! + ... + (x-a) ^n-1 f ^(n-1) (a) / (n-1)! + (X−a) ⁿ f ⁽ⁿ⁾ (ξ) / n! ... Formula (1) is established (the above formula (1) is described again). Where ξ = a + θ (x−a), 0 <θ <1 ”

  This equation (1) means that the value of f (x) can be calculated if f (a), which is the value of the fixed point a, and n-times differentiation of f (a) are given. That is, in the moving image, if the image f (t) at a certain time (t) and the n-th derivative of f (t) are given, the image level f (tx) at the arbitrary time tx can be calculated.

For example, in Equation (1), f (a) at a fixed point a (x = a), f ′ (a) obtained by differentiating f (a) once, and f (a) differentiated twice. Assuming that f ″ (a) is present, f (b) at a certain point b (x = b) is obtained.
f (b) = f (a) + (b−a) f ′ (a) / 1! + (B−a) ² f ″ (a) / 2! +... + (B−a) ⁿ⁻¹ f ⁽ⁿ⁻¹⁾ (a) / (n−1)! + (B−a) ⁿ f ⁽ⁿ⁾ (ξ) / n! ... Equation (2) holds, where ξ = a + θ (b−a) and 0 <θ <1.

Here, it is assumed that a ≠ b, that is, the fixed point a and a certain point b are not the same, but the two points are in the vicinity. Although Equation (2) is a continuous function, the moving image handled here, that is, the digitized image group is given as a discrete value for each predetermined minute time. For this reason, ξ in the proviso of equation (2) should be between a and b. However, in the discretized moving image, the image f (a) exists at the fixed point a (time a), but the image f (ξ) does not exist at the time ξ. In other words, the nearest neighbor point at time a is a certain point b (time b), and the image f (b) at this time b is obtained.
f (b) ≈f (a) + (b−a) f ′ (a) / 1! + (B−a) ² f ″ (a) / 2! +... + (B−a) ⁿ⁻¹ f ⁽ⁿ⁻¹⁾ (a) / (n−1)! + (B−a) ⁿ f ⁽ⁿ⁾ (a) / n!

Further, the image f (a) at time a can be differentiated up to two times. Then, Equation (3) becomes
f (b) ≈f (a) + (b−a) f ′ (a) / 1! + (B−a) ² f ″ (a) / 2!..

  In Equation (4), the image f (a) at time a becomes an existing image, and the image f (b) at time b becomes a new image created. Further, when creating an interpolated image for interpolation or an extrapolated image for extrapolation, the sampling interval for sampling the image included in the moving image is ba. If a and b are adjacent to each other, a = n and b = n + 1 can be set (that is, a <b and the sampling interval is set to 1).

Then, equation (4) becomes
f (n + 1) ≈f (n) + f ′ (n) / 1! + F ″ (n) / 2!..

Further, assuming that a and b are adjacent to each other, a = n + 1 and b = n can be set (that is, a> b and the sampling interval is set to 1).
Then, equation (4) becomes
f (n) ≈f (n + 1) −f ′ (n + 1) / 1! + F ″ (n + 1) / 2!..

  Here, with reference to FIG. 2, a process for calculating a filter coefficient when an image is interpolated into a moving image will be described using Expression (5) or Expression (6). In the change of the image level of the moving image shown in FIG. 2A, an existing image is represented by a circle, and an image that does not exist is represented by a cross. Here, x (6), which is an image that does not exist, is converted from the existing images x (1), x (3), x (5), x (7), and x (9) to Equation (5). ) Or Equation (6).

According to Equation (5)
x (6) ≈x (5) + x ′ (5) / 1! + X ″ (5) / 2!
According to Equation (6)
x (6) ≈x (7) −x ′ (7) / 1! + X ″ (7) / 2! ... Formula (8)

From here, the expression (7) will be developed.
First, x ′ (5) = {x ′ (4) + x ′ (6)} / 2
Then, x ′ (5) = {x ′ (4) + x ′ (6)} / 2, and one differentiation at a certain point uses the difference between the point adjacent to the certain point as the denominator. If the difference between functions at adjacent points is obtained as a numerator,
x '(5) = [{x (5) -x (3)} / (5-3) + {x (7) -x (5)} / (7-5)] / 2 = {x (7 ) -X (3)} / 4 (9).

Further, if the two differentiations at a certain point are obtained by subtracting the one differentiation at a certain point from the one differentiation of a point adjacent to the certain point, x ″ (5) = x ′ ( 6) −x ′ (5)... Equation (10)
If the premise which calculated | required Numerical formula (9) is followed, it will become x '(6) = {x (7) -x (5)} / 2 ... Numerical formula (11).
Substituting this formula (11) and formula (9) into formula (10),
x ″ (5) = {x (7) −x (5)} / 2− {x (7) −x (3)} / 4
= X (7) / 2-x (5) / 2-x (7) / 4 + x (3) / 4
= X (7) / 4-x (5) / 2 + x (3) / 4... (12)

Then, when substituting Equation (9) and Equation (12) into Equation (7),
x (6) ≈x (5) + {x (7) −x (3)} / 4+ {x (7) / 4−x (5) / 2 + x (3) / 4} / 2
x (6) ≈x (5) + x (7) / 4-x (3) / 4 + x (7) / 8-x (5) / 4 + x (3) / 8
x (6) ≈3x (7) / 8 + 3x (5) / 4-x (3) / 8
≒ (3/8) x (7) + (3/4) x (5) + (-1/8) x (3) ... Formula (13).

Generalizing this equation (13), since the nonexistent image is an even number (time is a multiple of 2), the image of x (2n) is
x (2n) ≈ (3/8) x (2n + 1) + (3/4) x (2n-1) + (-1/8) x (2n-3)... (14) where n Is an arbitrary integer.

  When this equation (14) is expressed as a filter coefficient, it becomes a filter coefficient (3/8, 1, 3/4, 0, −1/8). That is, when trying to create x (6) as a new image (interpolated image), the numerical value multiplied by x (7) is 3/8, the numerical value multiplied by x (6) is 1, The numerical value multiplied by x (5) becomes 3/4, the numerical value multiplied by x (4) becomes 0, and the numerical value multiplied by x (3) becomes −1/8.

Since the filter coefficient is obtained as described above, the multiplication unit (5 ₁ , 5 ₂ , 5 ₃ ,..., 5 _{n + 1} ) of the filter coefficient multiplication means 5 can be configured as a 3-tap asymmetric digital filter. .

  If a 5-tap asymmetric digital filter is constructed by performing the same calculation, the filter coefficients are (−1/96, 0, −1/48, 0, 1/2, 1, 29/48, 0, −7/96).

Here, in FIG. 2B, x (6) is a new image (interpolated image) of the image interpolating apparatus 1 (herein referred to as the image interpolating apparatus 1 ') when trying to create it. A configuration when the multiplication units (5 ₁ , 5 ₂ , 5 ₃ , 5 ₄ ) in the filter coefficient multiplication means 5 are limited is shown.
Here, the multiplication unit (5 ₁ , 5 ₂ , 5 ₃ , 5 ₄ ) can be configured as a 2-tap asymmetric digital filter. That is, the multiplying unit 5 _first filter coefficient alpha ₀ is 3/8 and filter alpha ₁ becomes 1, the multiplication unit 5 _second filter coefficient alpha ₂ 3/4 and filter coefficient alpha ₃ is zero, the multiplication unit 5 ₃ filter coefficient alpha ₄ is -1/8 and the filter coefficient alpha ₅ becomes zero, the multiplication unit ₄ of the filter coefficient alpha ₆ 0 and filter coefficients alpha ₇ becomes zero.

The relationship between FIG. 2A and FIG. 2B will be described in an organized manner. (1), (3), (5) shown in FIG. When (7) and (7) are input (exist), the changeover switch 7 is switched to output (6) and (7) shown in FIG. Also, when the input moving image becomes the next image (when the image has advanced by one), the input of each delay unit of the delay means 3 (3), (5) shown in FIG. , (7) and (9) are input (exist), the changeover switch 7 is switched to output (8) and (9) shown in FIG.
Incidentally, a linear interpolation filter provided for a conventional interpolation technique is a symmetric digital filter with a filter coefficient of (1/2, 1, 1/2).

  In addition, when the input moving image is a television signal, the image interpolating device 1 'determines that an image included in the moving image, that is, a frame image or a field image is differentiated once or 2 It is designed on the assumption that it continues until the first derivative.

  Further, when a three-dimensional tomographic image obtained by imaging the inside of a human body by CT scan or MRI is input instead of a television signal, it is assumed that organs in the human body are continuous in the three-dimensional tomographic image. Therefore, the number of differentiations is larger than that of the television signal, and the image interpolation device 1 (see FIG. 1) needs to increase the number of taps of the asymmetric digital filter constituting the filter coefficient multiplication unit 5.

Returning to FIG. 1, the description of the configuration of the image interpolation device 1 will be continued.
The switching unit 7 switches the multiplication result (image multiplied by the filter coefficient) output from the multiplication unit (5 ₁ , 5 ₂ , 5 ₃ ,..., 5 _{n + 1} ) of the filter coefficient multiplication unit 5. Are output to the image generation means 9 and are configured to include a plurality of changeover switches (7 ₁ , 7 ₂ , 7 ₃ ,..., 7 _{n + 1} ) corresponding to the multiplication section.

The timing for switching the changeover switches (7 ₁ , 7 ₂ , 7 ₃ ,..., 7 _{n + 1} ) is the timing for outputting the image (interpolated image) generated by the image generation means 9 (input video) Timing of insertion into an image). This output timing is performed every time the delay amount is increased by one (every one image) after switching all the changeover switches (7 ₁ , 7 ₂ , 7 ₃ ,..., 7 _{n + 1} ). Is called.

The image generation means 9 adds the multiplication results (images multiplied by the filter coefficients) output from the changeover switches (7 ₁ , 7 ₂ , 7 ₃ ,..., 7 _{n + 1} ) of the switching means 7 ( Composite) to generate an interpolated image used for interpolation, and output it to the outside.

  According to this image interpolation device 1, the switching means 7 multiplies the image delayed by the delay means 3 by the filter coefficient multiplication means 5 having the filter coefficient set by executing Taylor expansion, and the switching means 7. Since the output timing is adjusted and added by the image generation means 9, the images before and after the new image to be interpolated are multiplied by filter coefficients that are not asymmetrically uniform. An image (interpolated image) can be created.

  In this image interpolation apparatus 1, the input moving image is a three-dimensional signal (time axis, horizontal axis and vertical axis), but a two-dimensional or higher signal (for example, a three-dimensional tomographic image) is input. It can be easily extended and used.

<Operation of the image interpolation device>
Next, the operation of the image interpolation device 1 will be described with reference to the flowchart shown in FIG. 3 (see FIG. 1 as appropriate).
First, the image interpolation device 1 uses the delay units (3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ) of the delay means 3 to input images (moving images) in a plurality of stages (delay units). [3 ₁ , 3 ₂ , 3 ₃ ,..., 3 _n ] once) (step S1).

Subsequently, the image interpolation device 1 filters the image delayed by the delay unit 3 by the multiplication unit (5 ₁ , 5 ₂ , 5 ₃ ,..., 5 _{n + 1} ) of the filter coefficient multiplication unit 5. The coefficient is multiplied (step S2). Then, the image interpolation device 1 switches the multiplication result multiplied by the filter coefficient multiplication unit 5 by the changeover switch (7 ₁ , 7 ₂ , 7 ₃ ,..., 7 _{n + 1} ) of the switching unit 7. Then, the image generation means 9 synthesizes (adds) a new image and outputs it as an interpolated image to be interpolated (step S3).

  Thereafter, the image interpolation device 1 determines whether or not the generation of the interpolated image to be interpolated has been completed for the input image (moving image) (step S4), and it is not determined that the generation has been completed (step S4). In step S4, No), the process returns to step S1, and when it is determined that the generation has ended (step S4, Yes), the operation is ended.

<Filter coefficient identification method>
Next, a filter coefficient specifying method for specifying the filter coefficient of the filter used in the image interpolation device 1 will be described with reference to FIG. In the filter coefficient specifying method, as described above, the filter coefficient is specified through three stages by using the Taylor expansion formula. These three stages are a differentiation number specifying step, a differential coefficient replacement step, and a filter coefficient specifying step.

  In the differentiation number specifying step, the number of times that the function indicating the change in the image level of the moving image can be differentiated is determined by a preset processing method. There are two processing methods: a method for processing an interpolated image, that is, a time axis direction, and a horizontal and vertical direction. The number of differentiable times varies depending on the direction in which the interpolated image is interpolated. When the interpolating direction is the time axis direction, the number of times of differentiation is about 2 to 3 times, and the interpolating direction is horizontal and vertical. In the case of the direction, it is about once to twice. Note that the number of differentiable times can be arbitrarily set according to the theory of Taylor expansion. However, from the experimental results, the number of times described above may be optimal when used for moving image interpolation. know.

In the differential coefficient replacement step, the number of terms indicating the differential coefficient is set in the above-described equation (1) based on the differential count determined in the differential count determination step. Is replaced with a difference equation using images before and after the obtained image. In Equation (1), the insertion position, which is a sampling position for inserting an image to be interpolated or extrapolated, is x, and a sampling position for acquiring a differentiable function indicating a change in the image level of a moving image. The acquisition position is a.
In this differential coefficient replacement step, the differential coefficient is replaced with a differential equation as shown in Equation (9).

  In the filter coefficient specifying step, the coefficient relating to each term of the difference equation replaced in the differential coefficient replacement step is added for each function indicating an image to obtain a filter coefficient. In this filter coefficient specifying step, the filter coefficient is specified from Expression (9) to Expression (13).

When these operations are described in order in accordance with FIG. 4, first, the filter coefficient specifying method determines the number of differentiation (step S11), and sets the number of terms of the differential coefficient in Expression (1) based on Taylor expansion ( Step S12).
Then, the filter coefficient specifying method replaces the differential coefficient term with a differential equation (step S13), adds the coefficients related to the differential coefficient term, and specifies the filter coefficient (step S14).

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. For example, in the present embodiment, the image interpolation device 1 (image interpolation device 1 ′) has been described. However, image interpolation described in a general-purpose or special computer language so that the processing of each of these components can be executed. It can be regarded as a program. Further, it can be regarded as an image interpolation method in which the processing of each component of the image interpolation device 1 (image interpolation device 1 ′) is regarded as one process. In these cases, the same effect as that of the image interpolation device 1 (image interpolation device 1 ′) can be obtained.

1 is a block diagram of an image interpolation device according to an embodiment of the present invention. It is a figure explaining the process which calculates the filter coefficient at the time of interpolating an image in a moving image, (a) showed the existing image and the image which does not exist in the change of the image level of a moving image. It is a figure and (b) is a block diagram of the image interpolation apparatus at the time of calculating | requiring the image which does not exist in (a). It is the flowchart explaining operation | movement of the image interpolation apparatus shown in FIG. 5 is a flowchart illustrating a filter coefficient specifying method according to the present invention. It is a figure explaining interpolation and extrapolation. It is a figure explaining the conventional interpolation technique.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 1 'Image interpolation apparatus 3 Delay means 5 Filter coefficient multiplication means 7 Switching means 9 Image generation means

Claims (4)

A filter having a plurality of filter coefficients used when interpolating a continuous moving image, and multiplying the image,
When the number of samples from the images before and after the interpolated image to be interpolated between existing images is 3, and the sampling positions are two before, one before and one after the interpolated image Is
When the filter coefficient related to the interpolated image is 1 and enumerated from the filter coefficients related to the post-image, the filter coefficient is (3/8, 1, 3/4, 0, −1/8) or the front When enumerating from filter coefficients related to an image, the filter coefficient is (−1/8, 0, 3/4, 1, 3/8). A filter having a plurality of filter coefficients used when interpolating a continuous moving image and multiplying the image,
The number of samples from the images before and after the interpolated image to be interpolated between existing images is five, and the sampling positions are two before, one before, one after and two after the interpolated image. If you want three
When filter coefficients related to the interpolated image are set to 1 and enumerated from filter coefficients related to the post-image, the filter coefficients are (−1/96, 0, −1/48, 0, 1/2, 1, 29 / 48, 0, −7/96) or the filter coefficients related to the previous image, the filter coefficients are (−7/96, 0, 29/48, 1, 1/2, −1/48, 0, −1/96). An image interpolation device comprising the filter according to claim 1 or 2, and performing interpolation of a moving image in which input images are continuous,
A delay means for delaying the input image;
Filter coefficient multiplication means for multiplying the image delayed by the delay means by the filter coefficient set by the filter according to the amount of delay delayed;
Switching means for switching the output of the image multiplied by the filter coefficient multiplication means according to the interpolation timing;
An image generation means for adding the images whose outputs are switched by the switching means to generate an interpolation image used for the interpolation;
An image interpolation apparatus comprising: A filter coefficient specifying method for specifying a plurality of filter coefficients to be multiplied by an image in a filter used for interpolation or extrapolation to a moving image in which images are continuous,
A differentiating number determining step of determining a number of times that a function indicating an image level change of the moving image can be differentiated by a preset processing method;
Based on the number of differentiations determined in this differentiation number determination step, an insertion position for inserting an image to be interpolated or extrapolated is x, and the image that can be differentiated by the function determined in the number of differentiation number determination step is Assuming that the acquisition position to be acquired is a, the number of terms indicating a differential coefficient is set for Equation 1 below, and the differential coefficient in this term indicates images before and after the image for which the differential coefficient is obtained. A differential coefficient replacement step for replacing with a differential equation using a function;
f (x) = f (a) + (x−a) f ′ (a) / 1! + (X−a) ² f ″ (a) / 2!... + (X−a) ⁿ f ⁿ (a) / n!
... Formula 1
A filter coefficient specifying step for adding the coefficient applied to each term of the differential equation replaced in the differential coefficient replacement step for each function indicating the image, to obtain the filter coefficient;
A filter coefficient specifying method comprising:

Images