JP4412438B2 - Motion coefficient calculation method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a motion coefficient calculation method for calculating a motion coefficient used when converting an interlaced image to a non-interlaced image, and in particular, an improvement for performing discrimination between a still image and a moving image with higher accuracy. About.
[0002]
[Prior art]
The following three methods (1) to (3) are conventionally known as methods for converting an interlaced image into a non-interlaced image by interpolating lines (scan lines) in each field of the interlaced image. It has been. That is,
(1) Interpolation method using adjacent field pixels;
(2) a method of interpolating using adjacent pixels above or below; and
(3) A method of performing interpolation using an average value of values of pixels adjacent above and below.
[0003]
Method (1) is called inter-field interpolation, and interpolation is performed using pixels in adjacent fields as shown in FIG. Methods (2) and (3) are referred to as intra-field interpolation, and interpolation is performed within the same field as shown in FIG. 5 (b). 5 (a) and 5 (b), the vertical axis V represents the vertical direction on the screen (direction perpendicular to the line), and symbols (i−1), (i), and (i + 1) represent time. Represents the identification number assigned to the field along the flow of. Field (i) is a field to be interpolated, field (i-1) corresponds to the previous field, and field (i + 1) corresponds to the next field. To do. In FIGS. 5A and 5B, black circles represent pixels (interpolation pixels) to be interpolated, and white circles represent pixels included in the interlaced image before interpolation.
[0004]
Inter-field interpolation is suitable for still images, while intra-field interpolation is suitable for moving images. That is, each has suitability / unsuitability depending on the movement of the image. For this reason, in order to obtain a high-quality non-interlaced image, a method called “motion adaptive interpolation” is employed in which the motion amount of the image is obtained and the intra-field interpolation and inter-field interpolation are gradually switched. In this method, a motion coefficient k (0 ≦ k ≦ 1) that expresses the amount of motion of an image based on an absolute value of a difference between frames of pixel values (referred to as “interframe difference absolute value”). Calculated. Then, a value obtained by weighted averaging at a ratio of k: (1-k) with respect to the value obtained by intra-field interpolation and the value obtained by inter-field interpolation is adopted as the value of the interpolated pixel. The motion coefficient k is k = 0 for a still image having no image motion, and k = 1 for a moving image having a large motion.
[0005]
[Problems to be solved by the invention]
However, when the conventional motion coefficient calculation method is used, there is a problem that it may be difficult to correctly distinguish between still images and moving images and perform appropriate field interpolation. The still image shown in FIG. 6 and the moving image shown in FIG. 7 are examples of images that are difficult to discriminate from each other. In the following drawings including FIG. 6 and FIG. 7, the symbol V is synonymous with the symbol V in FIGS. 5 (a) and 5 (b), and the symbol H represents the horizontal direction on the screen (direction parallel to the line). Yes.
[0006]
In the still image of FIG. 6, figures having different pixel values are alternately arranged along the vertical direction V and the horizontal direction H on the screen. In the moving image of FIG. 7, strip-like figures with different pixel values extending along the vertical direction V are alternately arranged along the horizontal direction H, and in the horizontal direction by a distance corresponding to the width of the band for each field. It moves along H.
[0007]
The images shown in FIGS. 6 and 7 are as interlaced images as shown in FIG. In FIG. 8 and the following figures, the symbols (i-1), (i), and (i + 1) are synonymous with the same symbols in FIGS. 5 (a) and (b), and the symbol (i-2) is a field. (i-1) represents the previous field. As shown in FIG. 8, in the interlaced image, there is almost no difference between the still image of FIG. 6 and the moving image of FIG.
[0008]
When the interlaced image in FIG. 8 is a moving image, the image movement is usually slightly larger than that in the case of a still image. Therefore, it is possible to determine whether the image is the still image of FIG. 6 or the moving image of FIG. 7 by estimating the motion coefficient higher than the conventional value based on the interframe difference absolute value. However, if the motion coefficient is estimated uniformly high, it is often mistakenly determined as a moving image even though it is a still image, and the accuracy of determination as a whole decreases.
[0009]
The present invention has been made to solve the above-described problems in the prior art, and provides a motion coefficient calculation method that enables discrimination between a still image and a moving image with higher accuracy. With the goal.
[0010]
[Means for Solving the Problems]
A method according to a first aspect of the present invention is a motion coefficient calculation method for calculating a motion coefficient for interpolating a line in an interlaced image and converting it into a non-interlaced image, wherein the interpolation is a pixel on the line to be interpolated. step of calculating the motion coefficients for each pixel, calculating a difference absolute value between frames in (a) the interpolation pixel, in including regions the position of (b) the interpolation pixel, equal to the interpolated pixel A step of calculating an evaluation value expressing a degree of difference in pixel value between a field and an adjacent field; and (c) the motion coefficient as an increase function of both the interframe difference absolute value and the evaluation value. Calculating.
[0011]
In the method of the second invention, in the motion coefficient calculation method of the first invention, the step (b) comprises (b-1) a pixel value at the position of the interpolated pixel in the adjacent field and the same value in the same field. Calculating an intermediate value between pixel values of two pixels before and after the interpolation pixel vertically adjacent to the interpolation pixel, and (b-2) the pixel at the position of the interpolation pixel in the adjacent field and the adjacent field. Calculating a first difference that is an absolute value of a difference from the value; and (b-3) calculating the evaluation value as an increasing function of the first difference.
[0012]
In the method of the third invention, in the motion coefficient calculation method of the second invention, the step (b) comprises: Calculating an intermediate value between the pixel value of the first neighboring pixel and the pixel value of the two pixels before interpolation adjacent to the position of the first neighboring pixel in the neighboring field in the vertical direction; and (b-5) Calculating a second difference that is an absolute value of a difference between the intermediate value calculated in the step (b-4) and the pixel value of the first adjacent pixel, and further comprising the step (b-3) ) Calculates the evaluation value as an increasing function of the second difference together with the first difference.
[0013]
In the method of the fourth invention, in the motion coefficient calculation method of the third invention, the step (b-3) calculates the evaluation value as a weighted average value of the first difference and the second difference.
[0014]
According to a fifth aspect of the present invention, in the motion coefficient calculation method of the third aspect, the step (b) includes: (b-6) before the interpolation that is adjacent to the interpolated pixel in the same field either above or below Calculating an intermediate value between the pixel value of the second adjacent pixel and the pixel value of the two pre-interpolated pixels adjacent to the position of the second adjacent pixel in the adjacent field, (b-7) A step of calculating a third difference that is an absolute value of a difference between the intermediate value calculated in the step (b-6) and the pixel value of the second adjacent pixel. ) Calculates the evaluation value as an increasing function of the third difference together with the first difference and the second difference.
[0015]
According to a sixth aspect of the invention, in the motion coefficient calculation method of the fifth aspect, the step (b-3) calculates the evaluation value as a weighted average value of the first to third differences.
[0016]
According to a seventh aspect of the present invention, in the motion coefficient calculation method of the fifth aspect, the step (b-3) calculates the evaluation value as an intermediate value of the first to third differences.
[0017]
In the method of the eighth invention, in each of the motion coefficient calculation methods of any of the second to seventh inventions, each of all the pixel values that are the basis of all the intermediate values calculated in the step (b) is , Replaced by a weighted average of pixel values at the same position across multiple fields.
[0018]
According to a ninth aspect of the present invention, in the motion coefficient calculation method according to any one of the first to eighth aspects of the invention, in the step (c), the motion coefficient is changed from an absolute difference value between 0 to a first threshold ( > 0), 0 from the first threshold to the second threshold (> first threshold), and 1 from the second threshold onwards, and the evaluation value is within a predetermined range. A higher value is calculated so that both the first and second threshold values decrease and the absolute value of the difference between the first and second threshold values decreases.
[0019]
In the method of the tenth invention, in the motion coefficient calculation method of the ninth invention, in the step (c), the evaluation value is 0 from the first value (> 0) to 0, and from the first value, The coefficient increases from 0 to 1 until the second value (> the first value), and is 1 when the value is greater than or equal to the second value. Both the first and second threshold values decrease as the coefficient increases. And the motion coefficient is calculated so that the absolute value of the difference between the first and second threshold values decreases.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
[1. Overview]
In the example of the interlaced image shown in FIG. 8, the image difference is large between adjacent fields. In such a case, as described above, it is possible to accurately discriminate between a moving image and a still image by estimating the motion coefficient high. Conversely, for interlaced images with small image differences between adjacent fields, it is highly possible that the image is a still image, so it is possible to make a correct discrimination by estimating the motion coefficient low. It becomes.
[0021]
Therefore, in the motion coefficient calculation method according to the embodiment of the present invention, correction according to the magnitude of the image difference between adjacent fields is added to the motion coefficient. That is, an evaluation value, which is a variable expressing the degree of difference in pixel values between adjacent fields, is calculated, and a motion coefficient is calculated as an increase function of both the interframe difference absolute value and the evaluation value. Thereby, it becomes possible to discriminate between still images and moving images with high accuracy for various interlaced images.
[0022]
[2. Calculation of absolute difference between frames]
FIG. 1 is a flowchart illustrating a processing procedure of a motion coefficient calculation method according to the embodiment. 2 to 4 are explanatory views showing the principle of processing along the procedure of FIG. In FIG. 2, parallel bands extending horizontally represent lines (scan lines) existing in each field from before interpolation, black circles represent pixels on the lines before interpolation, and white circles represent interpolated pixels or interpolated pixels. It represents the pixel to be used. A symbol P represents an interpolation pixel that is currently subject to interpolation. In a color image, the Y component (luminance component) is preferably selected as the pixel value to be calculated, but other components can also be used, and each of the three components can be used individually. It is also possible to perform the same calculation. Hereinafter, in order to simplify the description, a pixel and its value (pixel value) are represented by the same symbol.
[0023]
FIG. 1 shows a processing procedure for one interpolation pixel P, and the processing in FIG. 1 is repeated for each interpolation pixel P. When the processing for one interpolation pixel P is started, an interframe difference absolute value k0 for the interpolation pixel P is calculated (step S1). Since the meaning of the interframe difference absolute value k0 and the calculation procedure thereof are well known in the conventional motion coefficient calculation method, detailed description thereof will be omitted.
[0024]
[3. Calculation of differences dA, dA ′, dB]
Next, in the position of the interpolation pixel P or the region including the position, the pixel value between the same field (i) as the interpolation pixel P and the adjacent field (i-1) or (i + 1) is changed. An evaluation value expressing the degree of difference is calculated (steps S2 and S3). For example, in FIG. 2, the values of the pixels a1 and a1 ′ vertically adjacent to the interpolation pixel P in the field (i), the pixel value at the position b1 of the interpolation pixel P in the adjacent field (i−1), the adjacent field ( The value of pixels b1 and b1 ′ adjacent above and below the position of pixel a1 in i−1) and the value of pixel b1 ″ adjacent to the position of pixel a1 ′ in adjacent field (i−1) Is used to calculate pixel values A, A ′, B, B ′, B ″ as follows:
A = a1;
A ′ = a1 ′;
B = b1;
B ′ = b1 ′; and B ″ = b1 ″.
[0025]
Alternatively, with respect to the pixel values B, B ′, B ″, using the pixel values at the positions b2, b2 ′, b2 ″ of the pixels b1, b1 ′, b1 ″ in the field (i + 1), The calculation may be done as follows:
B = b2;
B ′ = b2 ′; and B ″ = b2 ″.
[0026]
Further, using the pixel values at the positions a2 and a2 ′ of the pixels a1 and a1 ′ in the field (i-2), the pixel values A, A ′, B, and B are obtained as average values of the pixel values as follows. ', B''may be calculated:
A = (a1 + a2) / 2;
A ′ = (a1 ′ + a2 ′) / 2;
B = (b1 + b2) / 2;
B ′ = (b1 ′ + b2 ′) / 2; and B ″ = (b1 ″ + b2 ″) / 2;
The average value here can be extended to a weighted average value with some constant as a weight. Furthermore, it is possible to extend to a weighted average value of not only two pixels but also three or more pixels.
[0027]
Next, using the pixel values A, A ′, B, B ′, B ″ determined in this way, the differences dA, dA ′, dB are calculated as follows:
dA = abs {med (B, B ′, A) −A};
dA ′ = abs {med (B, B ″, A ′) − A ′};
dB = abs {med (A, A ′, B) −B};
Here, abs {} represents an absolute value, and med () represents an intermediate value (median) (step S2 above).
[0028]
[4. Calculation of evaluation value X]
Next, the above-described evaluation value is calculated using these differences dA, dA ′, and dB. The evaluation value X is calculated as follows, for example:
X = (dA + dA ′ + 2 × dB) / 4.
[0029]
This value corresponds to a weighted average value of the differences dA, dA ′, dB with weights added at a ratio of 1: 1: 2. The present invention is not limited to this example, and other weighted average values having different weights can be used, and a simple average value can also be used as one of them.
[0030]
In addition, using intermediate values as follows:
X = med (dA, dA ′, dB);
May be calculated.
[0031]
Further, the evaluation value X may be calculated as a weighted average value of the two differences dA and dB or a weighted average value of the two differences dA ′ and dB. The “weighted average value” is a broad concept including a simple average value, and is also used in the same broad sense in this specification.
[0032]
Furthermore, most simply, using only one difference dB:
X = dB;
May be calculated.
[0033]
Regardless of which is adopted, the evaluation value X is calculated as an increasing function of each of all the differences used. The increasing function means a function in which there is a region that increases as the variable increases, and conversely there is no region that decreases. That is, the increase function may include a region that does not depend on a change in the variable.
[0034]
Subsequently, preferably, the evaluation value X is converted into a coefficient h. As shown in FIG. 3, the coefficient h is an increase function of the evaluation value X, is 0 in the section (0 ≦ X <X1), and has a constant slope in the section (X1 ≦ X <X2). And is 1 in the section (X2 ≦ X) (step S3). In the interval (X1 ≦ X <X2), generally, the coefficient h may increase from 0 to 1 together with the evaluation value X with a positive increase rate that is not necessarily constant.
[0035]
[5. Calculation of motion coefficient k ′]
When the processing in step S3 is completed, a motion coefficient k ′ is calculated based on the interframe difference absolute value k0 and the evaluation value X (step S4). The motion coefficient k ′ is generally calculated as an increasing function of both the interframe difference absolute value k0 and the evaluation value X.
[0036]
As shown in FIG. 4, the motion coefficient k ′ changes in a polygonal line as the interframe difference absolute value k0 increases, as in the relationship between the coefficient h and the evaluation value X shown in FIG. That is, in the section where the interframe difference absolute value k0 is 0 to the first threshold (> 0), the motion coefficient k ′ is 0, and in the section from the first threshold to the second threshold (> first threshold), It increases linearly from 0 to 1 with a constant slope, and is 1 above the second threshold. In general, in the section from the first threshold value to the second threshold value, the motion coefficient k ′ may increase with the interframe difference absolute value k0 with a positive increase rate that is not necessarily constant.
[0037]
Preferably, the coefficient h is used as shown in FIG. 4, and the higher the coefficient h, both the first and second threshold values decrease (ie, approach 0), and the absolute difference between the first and second threshold values. The motion coefficient k ′ is calculated so that the value (second threshold value− first threshold value) decreases (that is, the increase rate becomes higher). This means that the motion coefficient k ′ changes depending on the evaluation value X only within a finite range corresponding to the section (X1 ≦ X <X2) in FIG.
[0038]
For example, when the interlaced image of FIG. 8 is the still image of FIG. 6, k0 = K1, and when it is the moving image of FIG. 7, k0 = K2, and both K1 and K2 are close to 0, and K2 is slightly larger than K1. In the interlaced image of FIG. 8, the evaluation value X is sufficiently large, and therefore the coefficient h is 1. Therefore, the motion coefficient k ′ is calculated as k ′ = 0 for the still image of FIG. 6 and k ′ = 1 for the moving image of FIG. That is, both images are correctly identified.
[0039]
Unlike the example of FIG. 8, an interlaced image in which the difference in pixel values between adjacent fields is sufficiently small is highly likely to be a still image. In an interlaced image in which the difference in pixel values between adjacent fields is sufficiently small, the evaluation value X is lowered to a certain level and the coefficient h is 0. At this time, both the image with k0 = K1 and the image with k0 = K2 are correctly determined as still images. In this way, legitimate discrimination is performed over various interlaced images.
[0040]
The motion coefficient k ′ calculated in step S4 is used to calculate the value of the interpolated pixel P in the same manner as the conventional motion coefficient k. That is, a value obtained by weighted averaging the value obtained by intra-field interpolation and the value obtained by inter-field interpolation at a ratio of k ′ :( 1−k ′) is adopted as the value of the interpolation pixel P.
[0041]
【The invention's effect】
In the method of the first invention, by using the evaluation value, the motion coefficient is calculated to be high when the image difference is large between adjacent fields, and conversely, it is calculated to be low when it is small. For this reason, discrimination between a still image and a moving image is performed with high accuracy for various interlaced images.
[0042]
In the method of the second invention, the evaluation value is calculated as an increasing function of the first difference, so that the arithmetic processing is simple.
[0043]
In the method of the third invention, the evaluation value is calculated as an increasing function of both the first difference and the second difference, so that discrimination with higher accuracy is possible.
[0044]
In the method according to the fourth aspect of the invention, the evaluation value is calculated as a weighted average value which is a kind of linear calculation of the first difference and the second difference, so that highly accurate discrimination is possible through simple calculation.
[0045]
In the method of the fifth invention, since the evaluation value is calculated as an increasing function of the first to third differences, it is possible to perform discrimination with higher accuracy.
[0046]
In the method of the sixth aspect of the invention, since the evaluation value is calculated as a weighted average value that is a kind of linear calculation of the first to third differences, it is possible to perform highly accurate discrimination through a simple calculation.
[0047]
In the method of the seventh aspect, since the evaluation value is calculated as an intermediate value of the first to third differences, it is possible to make a highly accurate discrimination through a simple calculation.
[0048]
In the method of the eighth invention, the pixel value that is the basis of the linear calculation is given as a weighted average value of pixel values over a plurality of pixels, that is, a value that has passed through a kind of filter. Is suppressed, and more accurate discrimination becomes possible.
[0049]
In the ninth aspect of the invention, the threshold and the increase rate in the relationship between the motion coefficient and the absolute difference between frames are corrected based on the evaluation value, so that highly accurate discrimination is achieved through simple calculation.
[0050]
In the method of the tenth invention, since the motion coefficient is calculated using the coefficient converted from the evaluation value, discrimination with higher accuracy is achieved.
[Brief description of the drawings]
FIG. 1 is a flowchart illustrating a processing procedure of a method according to an embodiment.
FIG. 2 is an explanatory diagram illustrating a principle of processing according to FIG. 1;
FIG. 3 is an explanatory diagram illustrating a principle of processing according to FIG. 1;
FIG. 4 is an explanatory diagram showing the principle of processing according to FIG. 1;
FIG. 5 is an explanatory diagram showing the principle of a conventional method.
FIG. 6 is an explanatory diagram showing a problem of a conventional method.
FIG. 7 is an explanatory view showing a problem of a conventional method.
FIG. 8 is an explanatory diagram showing a problem of a conventional method.
[Explanation of symbols]
A, A ′, B, B ′, B ″ Pixel value dA Second difference dA ′ Third difference dB First difference k ′ Motion coefficient k0 Absolute difference between frames
(i-2), (i-1), (i), (i + 1) Field X Evaluation value X1 First value X2 Second value

Claims (10)

A motion coefficient calculation method for calculating a motion coefficient for interpolating a line to an interlaced image and converting it to a non-interlaced image,
Calculating the motion coefficient for each interpolated pixel that is a pixel on the line to be interpolated,
(a) calculating an inter-frame difference absolute value in the interpolation pixel;
(b) in including regions the position of the interpolation pixel, a step of calculating an evaluation value representing the degree of difference of pixel values between the interpolation pixel in the same field and its neighboring fields,
(c) calculating the motion coefficient as an increase function of both the interframe difference absolute value and the evaluation value. Step (b)
(b-1) calculating an intermediate value between a pixel value at the position of the interpolation pixel in the adjacent field and a pixel value of two pixels before interpolation that are adjacent to the interpolation pixel vertically in the same field; ,
(b-2) calculating a first difference that is an absolute value of a difference between the intermediate value and the pixel value at the position of the interpolation pixel in the adjacent field;
The motion coefficient calculation method according to claim 1, further comprising: (b-3) calculating the evaluation value as an increase function of the first difference. Step (b)
(b-4) The pixel value of the first adjacent pixel before interpolation adjacent to one of the upper and lower sides of the interpolation pixel in the same field and the position of the first adjacent pixel in the adjacent field are vertically adjacent Calculating an intermediate value between the pixel values of the two pixels before interpolation;
(b-5) further comprising calculating a second difference that is an absolute value of a difference between the intermediate value calculated in the step (b-4) and the pixel value of the first adjacent pixel,
The motion coefficient calculation method according to claim 2, wherein the step (b-3) calculates the evaluation value as an increasing function of the second difference together with the first difference.   The motion coefficient calculation method according to claim 3, wherein the step (b-3) calculates the evaluation value as a weighted average value of the first difference and the second difference. Step (b)
(b-6) Vertically adjacent to the pixel value of the second adjacent pixel before interpolation adjacent to the other one of the upper and lower sides of the interpolation pixel in the same field and the position of the second adjacent pixel in the adjacent field Calculating an intermediate value between the pixel values of the two pixels before interpolation;
(b-7) calculating a third difference that is an absolute value of a difference between the intermediate value calculated in the step (b-6) and the pixel value of the second adjacent pixel;
The motion coefficient calculation method according to claim 3, wherein the step (b-3) calculates the evaluation value as an increasing function of the third difference together with the first difference and the second difference.   6. The motion coefficient calculation method according to claim 5, wherein the step (b-3) calculates the evaluation value as a weighted average value of the first to third differences.   6. The motion coefficient calculation method according to claim 5, wherein the step (b-3) calculates the evaluation value as an intermediate value of the first to third differences.   8. Each of all pixel values from which all intermediate values calculated in step (b) are based is replaced with a weighted average of pixel values at the same position across a plurality of fields. The motion coefficient calculation method according to any one of the above.   In the step (c), the motion coefficient is 0 when the absolute difference value between frames is 0 to the first threshold (> 0), and from the first threshold to the second threshold (> first threshold). It increases from 0 to 1 and is 1 above the second threshold, and as the evaluation value is higher within a predetermined range, both the first and second thresholds decrease, and between the first and second thresholds The motion coefficient calculation method according to claim 1, wherein the absolute value of the difference is calculated so as to decrease.   In the step (c), the evaluation value is 0 from 0 to the first value (> 0), increases from 0 to 1 from the first value to the second value (> first value), Using a coefficient that is 1 above 2 values, the higher the coefficient, the both the first and second thresholds decrease, and the absolute value of the difference between the first and second thresholds decreases. The motion coefficient calculation method according to claim 9, wherein the motion coefficient is calculated.

Images