JP5752167B2 - Image data processing method and image data processing apparatus - Google Patents

Description

  The present invention relates to an image data processing method for converting an interlaced image into a progressive image, and an image data processing apparatus using the method.

  The moving image scanning methods for television and video are roughly classified into an interlace method and a progressive (also called non-interlace) method. In recent years, progressive systems that can obtain high-definition image quality have come to be used in many cases with the increase in capacity of communication channels and transmission channels, but moving images that have been shot or recorded in the past have been used. There are many interlaced types. In order to display such an interlaced moving image on a monitor compatible with the progressive method, it is necessary to convert the interlaced image signal into a progressive image signal. In the following description, this conversion from an interlaced image signal to a progressive image signal is referred to as deinterlacing according to common usage.

  Deinterlacing methods are roughly classified into spatial interpolation methods and temporal interpolation methods. In short, the spatial interpolation method refers to a missing part (specifically, a horizontal scan that is missing) by using only the information in the frame without referring to a frame other than the one frame to be deinterlaced. This is an interpolation method that supplements the information of (line). On the other hand, the temporal interpolation method is an interpolation method that compensates for missing portion information by referring not only to one frame to be deinterlaced but also to the frames before and after that (in some cases, a plurality of frames before and after the frame). In general, the temporal interpolation method has higher restoration performance than the spatial interpolation method, but there may be cases where motion and changes between frames cannot be estimated properly. Often used in combination with the law. Therefore, the performance improvement of the spatial interpolation method greatly leads to the performance improvement of deinterlacing itself.

  Conventionally, various methods have been used or proposed as a spatial interpolation method for deinterlacing. These methods are roughly classified into linear interpolation methods and nonlinear interpolation methods. The linear interpolation method is literally an interpolation method that uses a linear polynomial and linearly interpolates luminance and color differences between adjacent scanning lines to obtain a target pixel value.

  Consider a case in which the pixel value of the pixel x at the coordinate position (i, j) in the interpolation target scanning line on the restored image shown in FIG. 19 is interpolated from the pixel values of surrounding pixels. In the figure, the vertical coordinate position i is a scanning line number assigned in order from the top to the bottom of the image, and the horizontal coordinate position j is the position of each pixel in one scanning line in order from left to right. This is the number of the attached pixel. The scanning line at the vertical coordinate position i is a scanning line to be interpolated (a scanning line in which the pixel value is interpolated by deinterlacing), and the value of the pixel on the scanning line is unknown. On the other hand, pixel values extracted from the observed image Y are arranged on the scanning lines at the vertical coordinate positions i + 1, i-1, i + 3, i-3,. For each pixel located around the pixel of interest (interpolation target pixel) x, as shown in the figure, it is written a to h. It shall mean a pixel value.

In the simplest linear interpolation, the pixel value of the pixel x is an average of the values of the pixels located above and below it. In other words, when expressed in mathematical formulas:
x = (b + e) / 2
It becomes. As a method developed from this, an interpolation method using more pixel values has been proposed. For example, a cubic interpolation method using two upper and lower pixel values can be expressed by the following equation.
x = -0.0625g + 0.5625b + 0.5625e-0.0625h

  These linear interpolation methods provide high restoration performance for simple data processing. However, since interpolation is performed only in the vertical direction, if there is an oblique line or an edge extending in the oblique direction on the image, jaggies are likely to occur in the oblique line or the edge part, and the subjective image quality is not so good. There is a drawback. In addition, in order to avoid the complexity of description, in the following description, even if not particularly mentioned, the oblique line includes an edge portion extending in an oblique direction.

  The non-linear interpolation method mainly overcomes the drawbacks of the above-mentioned linear interpolation method. Various methods such as the Complexity Interpolation method for Deinterlacing method are known (see, for example, Patent Document 1). In short, in these methods, a set of highly correlated pixel values around the pixel x of interest is searched, and the pixel value of the interpolation target pixel x is estimated as an average value or a median value therefrom.

For example, in the ELA method, if | a−f | is smaller than | b−e | and | c−d |, it is determined that there is a diagonal line connecting the pixel a and the pixel f, and the average value of a + f is determined from the pixel x. Adopt as a value. That is, the pixel value can be obtained according to the following equation.
m: = min {| af |, | be |, | cd |}
When | a−f | = m, x = (a + f) / 2
When | b−e | = m, x = (b + e) / 2
When | c−d | = m, x = (c + d) / 2
However, if there are a plurality of items that match m, it is assumed that | b−e | = e. Although there are differences in details, the APM method, MBI method, LCID method and the like are basically the same.

  These nonlinear interpolation methods identify the presence of diagonal lines, and perform processing specialized for the presence of diagonal lines. Naturally, the diagonal lines are smoothly interpolated compared to the linear interpolation method, It can be expected to improve subjective image quality. For this reason, for example, the restoration performance viewed from an objective index such as PSNR (Peak Signal-to-Noise Ratio) indicating the restoration performance of the original image is not widely used in the linear interpolation method, but is widely used in the spatial interpolation method.

  However, even in the conventional nonlinear interpolation method as described above, there are cases where good interpolation cannot always be performed depending on the state of oblique lines appearing in the image. For example, in the ELA method, for an image having a large amount of spatial high-frequency components, such as a thin diagonal line on a completely white background, a pixel that should originally be determined to be a part of the diagonal line is a part of the background. It may be mistakenly recognized as being present, resulting in broken diagonal lines. Although techniques for improving the APM method, the MBI method, and the LCID method have been devised, the improvement effect is not sufficient.

  Further, in the conventional nonlinear interpolation method, the slope (angle) of the oblique line that can be estimated is limited. In general, various methods such as the ELA method interpolate a slanted line with a 45 ° inclination fairly accurately, but jaggies or the like are likely to occur with a slanted line greatly exceeding 45 °. Although a method corresponding to the inclination of the oblique line more finely has been proposed, it is difficult to perform good interpolation with the oblique line close to the horizontal (see Patent Document 2).

  That is, the conventional linear interpolation method is superior to the nonlinear interpolation method in terms of restoration performance as viewed from an objective index, but jaggy is likely to occur in the oblique lines, and the subjective image quality is inferior. On the other hand, even a conventional nonlinear interpolation method specialized for oblique line processing is not necessarily restored appropriately depending on the state of oblique lines, and is inferior to linear interpolation method in objective restoration performance.

JP 2007-184934 A JP 2012-213136 A Japanese Patent No. 3820331 Japanese Patent No. 5142300

H. Kakemizu and three others, "Noise Reduction of JPEG Images by Sampled-" Data Reduction-Data Etch-Infinity Optimal Epsilon Filters data H-infinity Optimal Epsilon Filters), Proc. SICE Annual Conference, 2005, pp. 1080-1085

  The present invention has been made to solve the above-mentioned problems, and the object of the present invention is to satisfactorily restore a hatched portion that cannot be properly interpolated even by using a conventional nonlinear interpolation method. To provide an image data processing method and an image data processing apparatus capable of improving restoration performance indicated by an objective index such as PSNR and obtaining a progressive image signal with good quality from an interlaced image signal. is there.

  As described above, the conventional linear interpolation method has a relatively high objective restoration performance although it is inferior in the restoration of the oblique lines on the interlaced image. In practice, the pixel value (interpolation value) of the pixel to be interpolated on the restored image obtained by deinterlacing the interlaced image obtained from the original image using the cubic linear interpolation method and the pixel value of the corresponding pixel in the original image When the error is examined, the portion showing the large error is concentrated on the edge portion where the oblique line is noticeable, and the error of the other portion is relatively small. Therefore, both the objective restoration performance and the subjective image quality can be improved by appropriately processing the shaded part that cannot be processed properly by the linear interpolation method and processing the other parts by the linear interpolation method. I guess it can be done. On the other hand, the objective restoration performance of the nonlinear interpolation method is inferior to that of the linear interpolation method because the accuracy of the oblique line estimation is not sufficiently high, and even parts that should not be treated as oblique lines are processed as oblique lines. Can be guessed.

  For this reason, the inventor of the present application has introduced a diagonal determination method with higher certainty compared to the conventional nonlinear interpolation method, and performs interpolation so as to reproduce the diagonal line specifically for a part that can be determined to have a diagonal line. We have come up with a technique that performs linear interpolation in the vertical direction for parts that are not hatched or suspected to be hatched but have low reliability, and various simulation experiments improve the performance of deinterlacing. Confirmed that it can.

That is, an image data processing method according to the present invention made to solve the above-described problem is an image data processing method for converting an interlaced image signal of one field into a progressive image signal by pixel interpolation processing,
a) From a pixel row composed of N (N is an integer of 3 or more) adjacent pixels in the scanning line above the interpolation target pixel and from the N adjacent pixels in the scanning line below the interpolation target pixel. Two pixels located in the vertical direction while generating a virtual annular pixel row in which the right end portions and the left end portions of the pixel row are combined, and rotating the annular pixel row one pixel at a time in the right and left directions. The difference between the pixel values is determined for each of the N sets of pixels, and based on the rotation amount and the rotation direction when the sum of the differences is minimized, the presence / absence of a diagonal line including the interpolation target pixel, An oblique line determination step for identifying the stretching direction in a certain case,
b) When it is determined that there is a diagonal line including the interpolation target pixel in the diagonal line determination step, the pixel of the interpolation target pixel is obtained by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the extending direction of the diagonal line. Interpolation value calculation for obtaining a pixel value of the interpolation target pixel by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the vertical direction when it is determined that there is no oblique line including the interpolation target pixel Steps,
It is characterized by having.

An image data processing apparatus according to the present invention, which has been made to solve the above problems, is an apparatus for carrying out the image data processing method according to the present invention. One field of interlaced image signals is subjected to pixel interpolation processing. An image data processing device for converting to a progressive image signal,
a) From a pixel row composed of N (N is an integer of 3 or more) adjacent pixels in the scanning line above the interpolation target pixel and from the N adjacent pixels in the scanning line below the interpolation target pixel. Two pixels located in the vertical direction while generating a virtual annular pixel row in which the right end portions and the left end portions of the pixel row are combined, and rotating the annular pixel row one pixel at a time in the right and left directions. The difference between the pixel values is determined for each of the N sets of pixels, and based on the rotation amount and the rotation direction when the sum of the differences is minimized, the presence / absence of a diagonal line including the interpolation target pixel, Oblique line determination means for identifying the stretching direction in a certain case,
b) When it is determined by the oblique line determining means that there is an oblique line including the interpolation target pixel, the pixel of the interpolation target pixel is obtained by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the extending direction of the oblique line. Interpolation value calculation for obtaining a pixel value of the interpolation target pixel by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the vertical direction when it is determined that there is no oblique line including the interpolation target pixel Means,
It is characterized by having.

  In the image data processing method according to the present invention implemented by the image data processing apparatus according to the present invention, when one interlaced image is given, in the oblique line determination step, one interpolation target pixel in the image is further processed. A pixel column composed of N pixels adjacent to each other in the scanning line and a pixel column composed of N pixels adjacent to each other in the scanning line below the pixel to be interpolated are extracted, and the right end portion of the two pixel columns A virtual annular pixel column is generated by joining the left ends to each other. In general, the value of N is an odd number, and a pixel column composed of N pixels is extracted by extracting (N−1 / 2) pixels from the center of the position of the interpolation target pixel. .

In yet hatched determination step, 2 × N number of the difference between two pixel values of pixels positioned in the vertical direction while rotating by one pixel a ring shaped pixel rows ing from the pixel to the right and left direction N sets of Each pixel group is obtained, and the rotation amount and the rotation direction when the sum of the differences is minimized are examined.

Now, if there is a diagonal line including the interpolation target pixel, when the circular pixel row is rotated by a certain number of pixels in the right direction or the left direction, the pixels constituting the diagonal line are in the vertical direction across the interpolation target pixel. line up. Since the pixel values of the pixels constituting the oblique lines should be substantially the same, the difference between the pixel values of the two pixels positioned in the vertical direction across the interpolation target pixel is small. On the other hand, if the area other than the diagonal line around the interpolation target pixel is flat, that is, the pixel values are almost the same, the pixels constituting the diagonal line are arranged in the vertical direction with the interpolation target pixel interposed therebetween as described above. In some cases, the difference between the pixel values of two pixels positioned in the vertical direction is small even outside the interpolation target pixel. This is the same even if, for example, the two regions located across the oblique line are flat portions of different brightness and color. Therefore, based on the sum of the differences between the pixel values of the upper and lower pixels, it can be recognized that the pixels constituting the diagonal line are arranged in the vertical direction with the interpolation target pixel interposed therebetween, and it can be determined that there is a diagonal line. . Conversely, for example, when the sum of the differences between the pixel values of the upper and lower pixels does not become small, it can be determined that there is no oblique line.

  Further, by obtaining the rotation amount (number of pixels) and the rotation direction of the annular pixel row until the pixels constituting the oblique line are arranged in the vertical direction with the interpolation target pixel interposed therebetween, the oblique line extending direction can be identified. Furthermore, not only when there is a diagonal line in the flat part, but also when the pixel to be interpolated is included in an oblique edge part where two flat parts having different brightness and color are in contact with each other, the oblique edge part Are arranged in the vertical direction with the interpolation target pixel in between, the difference between the pixel values of the two pixels positioned in the vertical direction becomes small. Therefore, in the oblique line determination step, the presence or absence of the oblique edge portion can be determined similarly to the oblique line, and when the oblique edge portion exists, the extending direction can be identified.

  In this way, in the oblique line determination step in the image data processing method according to the present invention, the determination of the presence or absence of the oblique line existing in the flat part or the oblique edge part that divides the two flat parts and the identification of the extending direction are high. Can be done with precision.

  In the image data processing method according to the present invention, in the oblique line determination step, the minimum value of the sum of the differences when the annular pixel row is rotated rightward and the annular pixel row rotated leftward. The presence / absence of a diagonal line including the interpolation target pixel may be determined based on a difference from a minimum value of the sum of the differences. Thus, for example, when it is difficult to determine the extension direction of the oblique line or the oblique edge part when a region other than the oblique line or the oblique edge part has a complicated pattern instead of a flat part, the oblique line or the oblique edge part is used. By determining that there is no part, it is possible to avoid determining that there is a hatched line even though the certainty is low.

In the image data processing method according to the present invention,
In the interpolation value calculating step, in at least one of interpolation of the interpolation target pixel in the extending direction of the oblique line including the interpolation target pixel and interpolation of the interpolation target pixel in the vertical direction, the original image signal is band-passed through a band limiting filter. Analog including a limited analog image signal, a sampler that discretizes the analog image signal, an upsampler that inserts a predetermined number of zero-point data between sample points, a digital filter that performs filtering, and a hold that converts the discrete signal back to a continuous signal / A formula obtained by approximately converting a conditional expression set to design a digital filter to reduce an error with an analog image signal obtained through a digital / analog conversion system into a finite-dimensional discrete-time system. By solving with ^H∞ control based on a predetermined condition or equivalent Preferably, the pixel value of the interpolation target pixel is calculated by a filtering process using a sample value H ^∞ filter having a parameter calculated by solving with a strict calculation.

The inventor of the present application has been in the field of digital audio over a long period of time, particularly in the field of digital audio, and more particularly, a sample value control theory capable of handling a continuous time characteristic, more specifically, a digital / analog conversion technique for handling a digital audio signal with a sample value ^H∞ control. Research has been continued on attempts to introduce the system (see Patent Document 3). This technology does not simply capture the original digital signal sample as a discrete-time signal, but also considers analog characteristics included in the inter-sample response, and designs a digital filter for D / A conversion and sampling rate conversion. This is intended to achieve the best or close sound quality on the audibility as analog audio. Furthermore, in view of the recent progress in image compression technology and the need for image quality improvement accompanying this, the present inventor has conducted research on attempts to apply the sample value ^H∞ optimization technique to image noise removal, resolution conversion, and the like. Is going. For example, in Patent Document 4, image noise that can achieve high image quality that does not impair the goodness of the original image while making the block noise and mosquito noise that are likely to appear when the compression rate is increased by the MPEG method or the like inconspicuous. A removal method is proposed. Non-Patent Document 1 discloses a resolution converter that can convert the resolution of an image at an arbitrary magnification by using interpolation between sample points by a digital filter designed based on a sample value H ^∞ optimization method. Is disclosed.

In the image data processing method according to the above preferred embodiment, when performing linear interpolation in the vertical direction or the oblique extension direction with respect to the interpolation target pixel, the sample value H ^∞ filter is used instead of the conventional simple linear interpolation or cubic interpolation. Interpolation was performed. In the interpolation using the sample value ^H∞ filter, the interpolation value is calculated so that the error between the original image and the restored image is as small as possible, so that the restoration performance can be further improved.

  According to the image data processing method and the image data processing apparatus according to the present invention, the conventional nonlinear interpolation method is not appropriately interpolated, for example, a slanted line or a slanted edge portion where jaggies or breaks occur. Can be restored well. In addition, since a portion that should not be regarded as a slanted line or a slanted edge portion is not processed as a slanted line, the restoration performance indicated by an objective index such as PSNR can be improved. Thereby, a progressive image signal with good quality can be obtained from the interlaced image signal.

1 is a functional block diagram of an image data processing apparatus that implements an image data processing method according to an embodiment of the present invention. The figure which shows the error type | system | group model for designing the digital filter used for the image data processing method which is a present Example. FIG. 3 is a block diagram when the error system model shown in FIG. 2 is converted into a single rate system model. The block diagram which rewrote the single rate type | system | group model shown in FIG. 3 to the general plant format. FIG. 5 is a block diagram when FIG. 4 is converted into a finite-dimensional discrete time system. The figure which shows the example of the analog characteristic estimated from an original image. The figure which shows the example of two types of diagonal lines which are a process target in the image data processing method of a present Example. The figure which shows the local circulation part set with respect to the interpolation object pixel in an up sample image in the image data processing method of a present Example. FIG. 9 is a schematic explanatory diagram of oblique line determination by rotation of a local circulation unit shown in FIG. 8. The figure which shows the pixel pattern which requires attention by diagonal line determination. The flowchart of the deinterlacing in the image data processing method of a present Example. The detailed flowchart of diagonal line determination. Progressive video of akiyo video for evaluating deinterlacing. Interlaced video of akiyo video for evaluating deinterlacing. An image obtained by deinterlacing the interlaced moving image shown in FIG. 14 by conventional linear interpolation. An image obtained by deinterlacing the interlaced moving image shown in FIG. 14 by conventional ELA. An image obtained by deinterlacing the interlaced moving image shown in FIG. 14 by the image data processing method of this embodiment. The figure which shows the average of PSNR with respect to 10 types of animation when deinterlacing by various methods, and PSNR with respect to akiyo and city. The figure which shows arrangement | positioning of the pixel x in the coordinate (i, j) on a decompression | restoration image, and its surrounding pixel.

  Hereinafter, an image data processing method according to an embodiment of the present invention and an image data processing apparatus that executes the method will be described with reference to the accompanying drawings. The image data processing apparatus according to the present embodiment converts the interlaced image signal into a progressive image signal by interpolating the interlaced image signal for each field, that is, for each image, that is, performs the deinterlacing process.

FIG. 1 is a functional block diagram of the image data processing apparatus of this embodiment.
In the image data processing apparatus according to the present embodiment, the deinterlacing unit 1 receives image data constituting one interlaced image, and fills in the pixels on the scanning line interlaced by inter-sample point interpolation as zero point data. A sampler 2, a linear interpolation filter 3 that calculates an interpolation value instead of 0-point data using pixel values of pixels in the vicinity of the inserted 0-point data, and hatched lines for each interpolation target pixel on the upsampled image A hatching determination unit 4 for determining whether or not included in the diagonal edge portion, and identifying the diagonal line or the extending direction of the diagonal edge portion, and a frequency analysis unit for analyzing the frequency characteristics of the input image data 5 and a filter selection unit 6 that selects a filter coefficient of the digital filter according to the frequency analysis result.

The linear interpolation filter 3 is an ^H∞ optimal linear filter that performs filtering in consideration of the characteristics of the original image, and is one important element that executes a two-dimensional linear convolution operation. In addition, the oblique line determination unit 4 recognizes oblique lines and oblique edges on the interlaced image to be processed with high certainty and accurately grasps the extending direction of the oblique lines as will be described later. Is to execute.
First, the linear interpolation filter 3 in the image data processing apparatus of the present embodiment will be described in detail.

As described above, the linear interpolation filter used in the image data processing apparatus according to the present embodiment is a sample value H ^∞ linear filter designed using the sample value control theory, and uses pixel values of a plurality of pixels. The pixel value of the interpolation target pixel is obtained by interpolation processing. Now, in order to design an ^H∞ optimal linear filter that performs correction between sample points and has a transfer characteristic of K (z), consider an error system model as shown in FIG. In FIG. 2, the lower signal path is a signal processing system for rate conversion, and the upper signal path is a delay system that takes into account the time delay due to the signal processing system.

2, the input w is a continuous time signal, W (s) anti-aliasing filter 11 for band-limiting the continuous time signal, S _h is discrete by sampling a continuous-time signal at sample period h (> 0) Sampler 13, ↑ M, which is a time signal, inserts a zero signal between samples of the discrete time signal to convert it into a discrete time signal having a sample period h, and K (z) is the inserted zero signal. Is a digital filter 15 that corrects to an appropriate value, and H _{h / M} is a zero-order hold 16 that operates at a period h / M to convert a discrete-time signal into a continuous-time signal. On the other hand, e ^−mhs is the time delay element 12. The analog characteristic of the anti-aliasing filter 11 is W (s), and the transfer characteristic of the digital filter 15 is K (z). Thus, the error system model shown in FIG. 2 gives the time limit mh due to the signal processing described above to the band limit signal r, and the band limit delayed by the subtractor 18 from the restored signal y that is the output of the 0th-order hold 16. It can be considered that this is a system for extracting an error signal e from the signal. This error signal e is also a continuous time signal.

Here, in order to obtain an optimal filter, the IIR digital filter 15 is configured so that the error signal e is as small as possible. That is, the IIR digital filter 15 is designed under the condition that a stable continuous-time filter (anti-aliasing filter 11) and positive integers m and M are given. Therefore, when a system for converting the continuous time signal w to the error signal e is set as T _ew , a transfer characteristic K (z) satisfying the following expression (1) is obtained for a given γ.
That is, this equation (1) is a conditional equation set to design the IIR type digital filter described above. Here, γ dominates the magnitude of the error, and the smaller the better. In ^H∞ control, a method of minimizing this by repeated calculation is employed.

Since the error system model shown in FIG. 2 includes an upsampler, it is a time-varying system and is difficult to handle as it is. Therefore, discrete time lifting and inverse lifting are introduced to convert a system including an upsampler and a time delay system (multirate system) into a finite dimensional system having a single sample period. Since it is well known in the above-mentioned Patent Documents 3 and 4, Non-Patent Document 1 and the like, detailed description is omitted, but with the introduction of lifting, the multi-rate system model shown in FIG. Equivalent to a system model. Next, in order to make e ^−mhs which is a continuous time dead time element in FIG. 3 into a finite dimension, a conversion is performed so as to delay the input of the system by m steps. Thereby, the desired design problem is converted into a problem of designing a non-causal filter z ^m K ′ (z) instead of the transfer characteristic K (z).

  This is further reduced to a design problem for finite-dimensional discrete-time systems. Details of the method are given by Kargonek, Yamamoto, “Delayed signal reconstruction using sampled-data control”, Proceedings of 35th Conference on It is described in Decision and Control (Proc. Of 35th Conf. On Decision and Control), p.1259-1263, 1996, but here by applying FSFH (First Sample First Hold) method This results in an approximate discrete-time design problem with no constraints.

  The FSFH method is a method for evaluating the performance of the sample value control system, and the continuous time input / output of the sample value system of h period is discretized by a sampler and hold that operates at a period of h / N (N is a natural number). This is a method of approximating a continuous-time signal with a discrete-time signal for a large N. Details of the FSFH method can be found in Yamamoto, Madievski, Anderson, “Computation and convergence of frequency response via Fast sampling for sampled-data control systems (Computation and convergence of frequency response via). fast sampling for sampled-data control systems), Proc. of 36th Conf. on Decision and Control, p.2157-2162, 1997, Are omitted here.

FIG. 4 is a redraw of FIG. 3 into a generalized plant format for design purposes. When the determinant g of the continuous-time system 20 shown in FIG. 4 is lifted and approximated by the FSFH method, it is reduced to the discrete-time system shown in FIG. The time system G is given by the following equation.
Here, each matrix and operator of G are defined as follows.

Using the approximate discrete time system G, the above equation (1) is approximated by the following equation (2), and obtaining the transfer characteristic K (z) satisfying the equation (1) is approximately a finite-dimensional discrete Reduced to a time-related problem.
However,
It is. That is, FIG. 4 is converted into the finite-dimensional discrete time system shown in FIG.
Thus, the desired transfer characteristic K (z) of the IIR type digital filter can be obtained by obtaining equation (2) and solving a very general discrete time ^H∞ control problem.
The basics of the filter design method based on the sample value control theory described above are known ones described in Patent Documents 3 and 4 and Non-Patent Document 1, as well as papers published by the inventor at various academic societies. It is.

Now, returning to FIG. 2, by giving the analog signal characteristic of the image to be restored, that is, the image to be deinterlaced, as the analog characteristic W (s) of continuous time, the signal r given by this characteristic and the output of the signal restoration system are obtained. The filter interpolates between the sample points of the discrete signal so as to minimize the error signal e from the restored signal y in the sense of the H ^∞ norm of the system from w to e. Therefore, the analog characteristic W (s) must be determined prior to correction between sample points, which is generally not known. In addition, the H ^∞ optimization method is used for designing the filter, which is a computationally intensive process, and it is impractical to perform optimum filter design in real time when moving images are processed online. is there. Due to such restrictions, the following is dealt with here.

(1) A plurality of analog characteristics W (s) estimated to be general are prepared, and a linear interpolation filter corresponding to each analog characteristic W (s) is previously calculated offline. A plurality of filters obtained by calculating K _1, ..., a K _p. In the image data processing apparatus of the present embodiment, the parameters relating to the filters K ₁ ,..., K _p may be stored in the filter selection unit 6 in association with the analog characteristics.
(2) When an observation image Y to be deinterlaced is given, a filter K _q having the highest restoration performance is selected from K ₁ ,..., K _p and applied to the observation image Y. In the image data processing apparatus of the present embodiment, such processing for filter selection is executed by the frequency analysis unit 5 and the filter selection unit 6.
By adopting such a method, it is possible to perform deinterlacing even if the analog characteristic W (s) is unknown, and furthermore, since it is not necessary to calculate a filter in real time, the calculation amount associated with the ^H∞ optimization is also increased. It doesn't matter.

Here, it is necessary to select the filter K _q showing the highest restoration performance among the plurality of filters K ₁ ,..., K _p under the condition where only the observation image Y is given. For this purpose, the following processing is executed. As an example, consider the case where three types of analog characteristics W ₁ (jω), W ₂ (jω), and W ₃ (jω) are prepared in advance as W (s). An example of this analog characteristic is shown in FIG.

As shown in FIG. 6, among the three types of analog characteristics, W ₁ (jω) contains a relatively high frequency component, W ₃ (jω) contains a lot of low frequency components, and W ₂ (jω) is It has intermediate characteristics between the two. These characteristics imitate a typical example of the frequency spectrum of the original image obtained from the preliminary experiment, and correspond to estimation of the spectrum of the original signal that is desired to be restored. These analog characteristics are substantially the same in the vicinity of ω = 0, but | W ₁ (jω) | <| W ₂ (jω) | <| W ₃ (jω) | There is a relationship. These are estimates relating to the characteristics of the original image, but such low-frequency spectrum characteristics are reflected to some extent in the observed image, though not completely. Therefore, the gain of the frequency component of the observed image is examined in the frequency band estimated to be | W ₁ (jω) | <| W ₂ (jω) | <| W ₃ (jω) |, and according to the magnitude of the gain. Switch analog characteristics.

Specifically, at the time of deinterlacing, the frequency analysis unit 5 performs a discrete Fourier transform on the observation image, examines a gain in a predetermined low frequency region among the results, and prepares a plurality of previously prepared based on the result The optimum analog characteristic is selected from the analog characteristics W (s). Then, the filter selection unit 6 selects a transfer characteristic K (z) as an optimal interpolation filter for the selected analog characteristic W (s). Thereby, compared with the case where the transfer characteristic K (z) is fixed, more appropriate filtering in consideration of the characteristic of the original image can be executed. As a result, the error between the deinterlaced restored image and the original image can be further reduced, which is effective in improving restoration performance.
In the following description, a filter having a characteristic configuration based on the above-described sample value H ^∞ optimization method is referred to as a YY filter according to common usage.

Next, a characteristic oblique line determination process executed by the oblique line determination unit 4 in the image data processing apparatus of the present embodiment will be described.
Here, the oblique lines in the two modes shown in FIG. 7 are actually regarded as oblique lines. FIG. 7A shows an example in which two flat portions (regions having substantially the same pixel value) on the image are separated by diagonal lines, and FIG. 7B shows a boundary (edge portion) between the two flat portions. This is an example that is considered to be a diagonal line. Other than these two modes, for example, a diagonal line with a complicated pattern around it is difficult to reliably identify, so that it is not considered to be a diagonal line from the beginning. In this way, by identifying the oblique lines by limiting to the oblique lines with a flat portion around the periphery, it is possible to avoid judging that the part that should not be judged as the oblique line is the oblique line, and with high certainty. Diagonal line determination is possible. In order to detect oblique lines, a virtual annular pixel row as shown in FIG. 8 is considered.

FIG. 8 shows a part of an image obtained by upsampling the observed image Y twice in the vertical direction in the process of deinterlacing. Corresponding to the row coordinates i−1 and i + 1 are scanning lines derived from the observed image, and the pixel x at the coordinates (i, j) on the scanning line generated by upsampling is changed to the pixel values on the peripheral scanning lines. Interpolate based on. Although FIG. 8 is a part of an image, a restored image that has been deinterlaced can be obtained by replacing all of the pixels that have an initial value 0 in the upsampled image with an interpolation value. Here, in the scanning line above the interpolation target pixel x, two pixels on both sides of the interpolation target pixel x as the center, a total of 5 pixel columns, and the interpolation target pixel in the scanning line below the interpolation target pixel x An annular pixel composed of 10 pixels is extracted by extracting a pixel row of 5 pixels in total, 2 pixels on both sides of x, and connecting the right end portions and the left end portions as shown in FIG. Generate a column. An annular pixel row composed of 10 pixels whose pixel values are u ₁ ,..., U ₅ , v ₁ ,..., V ₅ is referred to as a “local circulation unit”. The local circulation part can also be expressed as a number vector as shown in equation (3).
C: = {u ₁ u ₂ u ₃ u ₄ u ₅ v ₅ v ₄ v ₃ v ₂ v ₁ } (3)

It should be noted that in the expression by the number vector, the order of v _j is reversed from the order of u _j . As described above, the function rot representing the rotation operation is recursively defined as shown in the equations (4), (5), and (6) for the local circulation unit represented in the form of a number vector.
rot _L (C): = {u ₂ u ₃ u ₄ u ₅ v ₅ v ₄ v ₃ v ₂ v ₁ u ₁ } (4)
rot _R (C): = {v ₁ u ₁ u ₂ u ₃ u ₄ u ₅ v ₅ v ₄ v ₃ v ₂ } (5)
rot (C, n): = C (when n = 0) (6)
: = Rot (rot _L (C), n-1) (when n <0)
: = Rot (rot _R (C), n + 1) (when n> 0)

The functions rot _L and rot _R are operations corresponding to general left rotation shift and right rotation shift as bit operations in a computer, respectively, and rot is n right rotation shifts (rot _R ) if the argument n is negative. If it is positive, it is a function that outputs the result of n left rotation shifts (rot _L ). FIG. 9 illustrates the operation of the functions rot _L and rot _R.

Further, in the local circulation part, a function diff for obtaining a difference between values of two pixels positioned in the vertical direction on the upper scanning line and the lower scanning line across the interpolation target pixel x is expressed by the following equation (7): Define as follows.
diff (C): = Σ | C [n] −C [5 × 2 + 1−n] | (7)
Here, Σ is the sum from n = 1 to 5.
The above definition is for the case where the width (number of pixels) of the local circulation portion is 5 and the length (total number of pixels) is 10 = 5 × 2, but for any local circulation portion width L, A similar operation can be defined by replacing “5” in the formula (7) with L. Also, the value ρ _C defined by the following equation (8) using the function rot shown in equation (6) and the function diff shown in equation (7) is called the optimum rotation amount of the local circulation unit C. And However, S is an integer satisfying 0 <S <L / 2 with respect to the local circulation part width L, and a specific method of determination will be described later.
ρ _C : = argmin diff (rot (C, n)) (8)
Here, the target range of argmin is −S ≦ n ≦ S (unless otherwise specified, the same applies to the following equations). When the optimal rotation amount is ρ _C, it is considered that there is a diagonal line that passes through the interpolation target pixel x and has an inclination of 1 / ρ _C. Hereinafter, this characteristic determination method is referred to as LCM (Local Circulation Matching).
The LCM algorithm will be specifically described with reference to FIG.

As shown in FIG. 9, when the local circulation part is rotated variously in the right direction and the left direction, when the rotation amount is n = + 1, the pixel originally constituting the diagonal line is the object to be interpolated. Located above and below the pixel x. At this time, the upper and lower pixel values sandwiching the interpolation target pixel x match (even if they do not exactly match, the pixel values are substantially equal). In addition, the pixel values of the two pixels arranged above and below other than the top and bottom of the interpolation target pixel x are also close to each other. This is because the two upper and lower pixels are located on the same flat portion as shown in FIG. Therefore, at this time, diff (rot (C, n)) calculated by the equation (7) is minimized. That is, the optimum rotation amount ρ _C is obtained as +1 from the equation (8), and it is determined that the diagonal line with the gradient 1 which is the reciprocal thereof passes through the interpolation target pixel x.

The oblique line determination is based on the following two assumptions.
(1) When there are pixels whose values match on the scanning lines above and below the interpolation target pixel x, it can be estimated that there is a diagonal line connecting the two pixels.
(2) If the region other than the oblique line and the oblique edge portion is flat, it can be estimated that the difference in pixel value between the pixels included in the flat portion is small. The rotation operation in the local circulation portion as described above has an effect of examining an error (difference in pixel value) in the region other than the oblique line and the oblique edge portion. For example, in FIG. 9, when the rotation amount is n = + 1, the difference in pixel value is examined for each set of vertically aligned pixels such as u ₂ , u ₁ , u ₃ , and v ₁ .
The detection of the hatched portion by the conventional non-linear interpolation method focuses only on the hatched line itself, and does not consider whether there is a flat portion around the hatched line. On the other hand, this oblique line detection method using LCM has an effect that the oblique line to be detected is limited to a highly reliable one on the condition that the periphery of the oblique line is flat.

However, in the actual image, the area around the oblique line and the oblique edge part is not always flat, and the pixel value state of the part is various. To avoid uncertain oblique line judgment.
(1) Coping when the direction of the oblique line is unclear FIG. 10A is an example of a situation assumed in this case. In this example, it can be considered that either a black diagonal line from the upper right to the lower left exists in the original image, or a white diagonal line from the upper left to the lower right exists. In this case, both the rotation amounts n = + 1 and n = −1 are the optimal rotation amounts. In such a case, it may actually be determined inaccurately due to the influence of noise, which would be a factor that impairs image quality. Therefore, in a situation where it is difficult to determine either the right rotation or the left rotation, the processing as the oblique line is not intentionally performed. Specifically, the left optimal rotation amount ρ _L when the rotation direction is limited to the left and the right optimal rotation amount ρ _R when the rotation direction is limited to the right are obtained and defined by the following equation (9): The left-right optimal rotation amount difference D _gap is calculated. Then, only if this lateral optimum rotation amount difference D _gap exceeds a predetermined threshold value T _LR, it determines that there is a high one certainty left rotation or right rotation, the optimum rotation amount of one of the [rho _L or [rho _R to adopt as ρ _C.
D _gap : = {| diff (rot (C, ρ _L )) − diff (rot (C, ρ _R )) |} / {diff (rot (C, ρ _L )) + diff (rot (C, ρ _R )) )}… (9)

(2) Countermeasure when the optimum rotation amount ρ _C is close to the width L of the local circulation portion C FIG. 10B is an example of a situation assumed in this case. In this example, the local circulation part whose width | variety contained in the frame shown by P in the figure is 5 is considered. At this time, the optimum rotation amount is ρ _C = −2, and it is determined that there is a black diagonal line from the upper left to the lower right. However, if you look at the outer area that is not included in C, for example, the pixel at coordinates (i-1, j-2) is the edge of the outer area of the frame P, and there is a diagonal line that passes through the pixel x The correct decision is not to. One cause of such misjudgment is that if the search range of the optimum rotation amount extends to both ends of the width of the local circulation part C, the situation around the oblique line, which is a feature of LCM, is reflected in the judgment. This is because the function does not work. Actually, in the example of FIG. 10B, the pixel existing to the left of the coordinates (i−1, j−2) is not included in the local circulation unit, and scanning before rotation with respect to the rotation amount n = −2. The proper operation of comparing between the top and bottom of the line is not performed. In order to reduce this type of misjudgment, the parameter S for limiting the search range for the optimum rotation amount may be set to be smaller than half the width L of the local circulation unit C.

(3) What to do when the optimal amount of rotation can be obtained despite the absence of diagonal lines
The optimum rotation amount according to equations (4) to (8) is defined only as the rotation amount that minimizes the difference between the upper and lower scanning lines, so as long as it is the smallest, no matter how large the corresponding difference is, the optimum rotation amount The amount will be determined. However, this does not mean estimating a highly reliable oblique line. In order to avoid this situation, rot (C, [rho _C) half the pixel value string on included in (in other words _{rot (C, ρ C) [} 1], ..., rot (C, ρ C) [L]) and the lower half pixel value string (in other words, rot (C, ρ _C ) [2L],..., Rot (C, ρ _C ) [L + 1])) If the predetermined threshold value T _corr is not exceeded, it is considered that there is a low degree of certainty that the diagonal line exists, and the diagonal line process is not performed.

(4) Dependency of inclination of oblique line with different desired width of local circulation part The search range of the optimum rotation amount is limited to a range of ± L / 2 at most. Therefore, in order to detect oblique line with inclination close to horizontal, A wider local circulation must be used. However, when a wide local circulation portion is used, a range that is too wide to detect a small inclination is collated, and there is a high possibility that a portion that should be determined as having a diagonal line is missed. FIG. 10C is an example of a situation assumed in this case.

In the example of FIG. 10C, if the width L of the local circulation portion is 5, a diagonal line from the upper right to the lower left is detected for the center interpolation target pixel x. However, if the width L of the local circulation portion is set to 7, this oblique line cannot be detected. In order to avoid such a problem, it is preferable to use a local circulation unit having a different size according to the rotation amount. That is, the local circulation part may be defined as in the following equation (10) (L / 2 in the equation is appropriately rounded to an integer).
C (L): = [X (i-1, j-L / 2), ..., X (i-1, j), ..., X (i-1, j + L / 2), X (i + 1, j + L / 2), ..., X (i + 1, j), ..., X (i + 1, j-L / 2)] ... (10)
For the variable length local circulation section defined in this way, the definition of the optimum rotation amount can also be corrected as shown in the following equation (11).
ρ _C : = argmin diff (rot (C (n + Lw), n)) (11)
However, the parameter Lw here is a value corresponding to L−S when using a local circulation portion that is not variable length. This Lw is called the wing length. In the following description, C ′ means C (n + Lw) and is distinguished from C.

Next, a deinterlacing processing procedure combining the above-described sample value ^H∞ linear filter and oblique line determination / processing by LCM, which is performed by the image data processing apparatus having the configuration shown in FIG. 1, will be described. FIG. 11 is a flowchart showing this processing procedure, and FIG. 12 is a detailed flowchart of the oblique line determination processing therein.

  First, when image data constituting an observation image Y of one field which is an interlaced image is given, the frequency analysis unit 5 performs a discrete Fourier transform on the image data, and based on the result, a predetermined low frequency region is obtained. Check the gain. The filter selection unit 6 determines a linear filter having an appropriate transfer characteristic K (z) based on the analysis result (step S1). As described above, a plurality of linear filters each having a plurality of transfer characteristics K (z) are obtained in advance.

  On the other hand, the upsampler 2 generates an initial restored image X obtained by upsampling the observed image Y twice in the vertical direction (step S2). The initial value of the pixel generated by upsampling in this initial restored image X is, for example, 0.

Subsequently, the oblique line determination unit 4 selects one pixel in the scanning line generated by upsampling in the initial restored image X as the interpolation target pixel x (step S3). Then, for the interpolation target pixel x whose coordinates are (i, j), it is first determined whether or not the pixel x is a hatched portion (or a slanted edge portion) (step S4).
Specifically, the following processing is executed. That is, a local circulation portion having a wing length Lw centered on x (i, j) is set and defined as C (step S11). Then, for 0 ≦ n ≦ S, the leftward optimum rotation amount ρ _L is calculated by the following equation:
ρ _L : = argmin diff (rot (C, n))
For −S ≦ n ≦ 0, the rightward optimum rotation amount ρ _R is calculated by the following equation (step S12).
ρ _R : = argmin diff (rot (C, n))
Then, the left and right optimum rotation amount ρ _L and the right direction optimum rotation amount ρ _R are used, and the left and right optimum rotation amount difference D _gap is calculated by the equation (9) (step S13).

Next, it is determined whether or not the left-right optimal rotation amount difference D _gap is D _gap <T _LR with respect to a predetermined threshold T _LR (step S14). If D _gap <T _LR (Yes in step S14) For example, it is determined that it is impossible to determine whether right rotation or left rotation is appropriate, and it is assumed that there is no oblique line, and ρ _{C is set} to 0 (step S15). On the other hand, when D _gap ≧ T _LR (No in step S14), diff (rot (C, ρ _L )) <diff (rot to determine whether the pixel value difference in the left or right direction is large. It is determined whether or not (C, ρ _R )) (step S16). Then, if Yes in step S16 to the left optimum rotation amount [rho _L the optimum rotation amount [rho _C (step S17), If No rightward optimum rotation amount [rho _R as the optimum rotation amount [rho _C (step S18 ).

Once the optimum rotation amount ρ _C is determined, the upper half pixel row UpperContent and the lower half pixel row LowerContent of the local circulation unit C at a time point after being rotated according to the optimum rotation amount ρ _C are as follows: Define (step S19).
UpperContent: = {rot (C, ρ _C ) [1], ..., rot (C, ρ _C ) [ρ _C + Lw]}
LowerContent: = {rot (C, ρ _C ) [2 (ρ _C + Lw)],..., Rot (C, ρ _C ) [ρ _C + Lw + 1]}
Then, a correlation coefficient corr (UpperContent, LowerContent) between the upper half pixel row UpperContent and the lower half pixel row LowerContent is calculated (step S20), and it is determined whether or not the correlation coefficient is less than the threshold T _corr. (Step S21). If the correlation coefficient is less than the threshold value (No in step S21), it is determined that there is a high possibility that there is actually no oblique line, and the optimum rotation is performed regardless of the optimum rotation amount determined in steps S17 and S18. The quantity ρ _{C is set} to 0 (step S22). With the above processing, the optimum rotation amount ρ _C for a given local circulation portion is determined.

Thereafter, it is determined whether or not the optimum rotation amount ρ _C is 0 (step S23). This corresponds to the determination process in step S5 in FIG. When the optimal rotation amount ρ _C is 0 (Yes in step S23), it is a determination result that there is no highly reliable oblique line, and the linear interpolation filter 3 that has received this is the interpolation target pixel x. The pixel value of the pixel to be interpolated x is calculated by the interpolation processing using the sample value ^H∞ filter determined as described above using the pixel values of the pixels in the vertical direction sandwiching the pixel (step S25 or S7). At this time, as described above, the filter having the characteristic selected by the filter selection unit 6 is used, so that the filter coefficient differs depending on the estimated characteristic of the original image.

On the other hand, when the optimum rotation amount ρ _C ≠ 0 (No in step S23), it is a determination result that there is a highly reliable oblique line or oblique edge portion, and therefore, the oblique line including the interpolation target pixel x or Using the pixel value of the pixel in the extending direction of the oblique edge portion, the pixel value of the interpolation target pixel x is calculated by simple linear interpolation (step S24 or S6). At this time, the optimum rotation amount ρ _C indicates the extending direction of the oblique line or the oblique edge portion. That is, the pixel value of the interpolation target pixel x (i, j) is obtained by {x (i−1, j + ρ _C ) + x (i + 1, j−ρ _C )} / 2.

  Since the pixel value of one interpolation target pixel is obtained by the processing in steps S3 to S7 (or S11 to S25), the interpolation target pixel is selected until the processing for all the interpolation target pixels is completed (Yes in step S8). While changing, the processes in steps S3 to S7 are repeated. When the processes for all the interpolation target pixels are completed, the deinterlacing process for the field is terminated and a progressive image is output.

The parameters to be prepared or set by the user when executing the processing of the flowchart are as follows. These can be obtained experimentally and set in advance.
(1) Multiple analog characteristics W (s) assumed as candidates
(2) Low frequency range referred to identify analog characteristic W (s) (3) Identification threshold of analog characteristic W (s) (4) Wing length Lw of local circulation section
(5) Optimum rotation amount search limit S
(6) determination threshold T _LR of the left and right optimum amount of rotation D _gap
(7) Determination threshold T _corr for the correlation coefficient of the degree of coincidence of pixel values

  In the above flowchart, it is assumed that the discrete Fourier transform is performed for each observation image of one field and the analog characteristic is selected based on the result. However, from the viewpoint of saving calculation amount, the transfer characteristic is large. In the same scene that is considered not to change (that is, across a plurality of consecutive fields), the analog characteristics once determined may be used in common.

  Next, the results of evaluating the effect of the image data processing method implemented by the image data processing apparatus according to the present invention by simulation calculation on a computer will be described. In this evaluation experiment, 10 types of videos (akiyo, bowing, city, container, crew, foreman provided by Xiph.org on the Internet <URL: http://media.xiph.org/video/derf/> , Hall monitor, news, pamphlet, soccer). Each of these images is a progressive moving image having a height n1 = 288 pixels, a width n2 = 388 pixels, 30 frames per second, and a total length of 300 frames. In the experiment, an interlaced video was generated by downsampling the progressive video twice in the vertical direction, and the deinterlacing process was applied to it.

Here, linear interpolation (simple linear interpolation filter), cubic interpolation (cubic interpolation filter), YY (dynamic switching sample value H ^∞ filter without diagonal line determination), ELA, APM, MBI, LCID, LCM (LCM) And a simple linear interpolation filter), LCM + cubic interpolation (combination of LCM and cubic interpolation filter), and LCM + YY (combination of LCM and dynamic switching sample value ^H∞ filter). The tuning parameters of the LCM are as follows: Wing length of local circulation part: Lw = 5, search limit of optimum rotation amount (number of pixels): S = 3, determination threshold of right and left optimum rotation amount: T _LR = 0.4, Determination threshold for the correlation coefficient of the pixel value coincidence degree: T _corr = 0.6.

  FIG. 18 shows the average PSNR for 10 types of moving images and the PSNR for two types of images called akiyo and city when the above-described methods are used. The numerical value in () in the figure is the PSNR ranking. Overall, when the linear filter and the LCM are combined, there is a tendency that the restoration performance improves, and it can be seen that the ability of the linear filter can be exhibited by appropriately selecting the application site of the linear filter. In addition, the dynamic switching YY filter achieves an average performance improvement of 0.49 [dB] for simple linear interpolation and 0.10 [dB] for cubic interpolation. It can be said that the result of considering the analog characteristics of the original image is reflected in the performance improvement. The effect of utilizing such analog characteristics is also reflected in the combination with LCM, and the order of restoration performance between LCM + YY, LCM + cubic interpolation, and LCM is the dynamic switching YY filter and the cubic interpolation filter. , Corresponding to an order between linear interpolation filters. Overall, LCM + YY of the present embodiment shows the highest performance in terms of average performance, with an average of 0.36 [dB] for cubic interpolation, and LCID with the best restoration performance in the conventional method of oblique line processing. However, it can be seen that the restoration performance is improved by 0.75 [dB].

  In addition, for the akiyo in the video for evaluation, progressive video, interlaced video, images deinterlaced by linear interpolation which is one of the conventional interpolation methods, and deinterlace by ELA which is one of the conventional interpolation methods Images and images deinterlaced by the method of this embodiment (LCM + YY) are shown in FIGS. In these images, enlarged images of two different shaded portions are also shown. Comparing the restoring characteristics of the hatched lines in FIG. 17 and FIG. 15, it can be seen that the method of the present embodiment has improved the process of the hatched area where there is an inclination of 45 ° or more and jaggy appears in the conventional method. . Further, comparing FIG. 17 with FIG. 16, it can be seen that in the conventional method, the method according to the present embodiment correctly detects the diagonal line and smoothly connects the diagonal line at the portion where the diagonal line detection fails and the line is interrupted. As described above, it can be confirmed that the image data processing method according to the present embodiment realizes superiority not only in the objective evaluation index but also in the subjective evaluation of the image quality.

  It should be noted that the above embodiment is merely an example of the present invention, and it should be understood that modifications, corrections, and additions may be made as appropriate within the scope of the present invention and included in the scope of claims of the present application.

DESCRIPTION OF SYMBOLS 1 ... Deinterlacing part 2 ... Up sampler 3 ... Linear interpolation filter 4 ... Diagonal line determination part 5 ... Frequency analysis part 6 ... Filter selection part 11 ... Anti-aliasing filter 12 ... Time delay element 13 ... Sampler 14 ... Up sampler 15 ... Digital Filter 16 ... 0th-order hold 18 ... Subtractor 20 ... Continuous time system 30 ... Sample value system

Claims (7)

An image data processing method for converting an interlaced image signal of one field into a progressive image signal by pixel interpolation processing,
a) From a pixel row composed of N (N is an integer of 3 or more) adjacent pixels in the scanning line above the interpolation target pixel and from the N adjacent pixels in the scanning line below the interpolation target pixel. Two pixels located in the vertical direction while generating a virtual annular pixel row in which the right end portions and the left end portions of the pixel row are combined, and rotating the annular pixel row one pixel at a time in the right and left directions. The difference between the pixel values is determined for each of the N sets of pixels, and based on the rotation amount and the rotation direction when the sum of the differences is minimized, the presence / absence of a diagonal line including the interpolation target pixel, An oblique line determination step for identifying the stretching direction in a certain case,
b) When it is determined that there is a diagonal line including the interpolation target pixel in the diagonal line determination step, the pixel of the interpolation target pixel is obtained by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the extending direction of the diagonal line. Interpolation value calculation for obtaining a pixel value of the interpolation target pixel by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the vertical direction when it is determined that there is no oblique line including the interpolation target pixel Steps,
An image data processing method comprising: The image data processing method according to claim 1 ,
In the oblique line determination step, a difference between a minimum value of the sum of the differences when the annular pixel row is rotated in the right direction and a minimum value of the sum of the differences when the annular pixel row is rotated in the left direction. And determining whether or not there is a diagonal line including the interpolation target pixel. The image data processing method according to claim 1 or 2 ,
In the interpolation value calculating step, in at least one of interpolation of the interpolation target pixel in the extending direction of the oblique line including the interpolation target pixel and interpolation of the interpolation target pixel in the vertical direction, the original image signal is band-passed through a band limiting filter. Analog including a limited analog image signal, a sampler that discretizes the analog image signal, an upsampler that inserts a predetermined number of zero-point data between sample points, a digital filter that performs filtering, and a hold that converts the discrete signal back to a continuous signal / A formula obtained by approximately converting a conditional expression set to design a digital filter to reduce an error with an analog image signal obtained through a digital / analog conversion system into a finite-dimensional discrete-time system. By solving with ^H∞ control based on a predetermined condition or equivalent An image data processing method comprising: calculating a pixel value of the interpolation target pixel by a filtering process using a sample value H ^∞ filter having a parameter calculated by solving with a strict calculation. The image data processing method according to claim 3 ,
In the interpolation value calculating step, an image data processing method is characterized in that a gain in a predetermined frequency region is examined for an interlaced image to be processed, and a transfer characteristic of the sample value ^H∞ filter is changed according to the gain. An image data processing device that converts an interlaced image signal of one field into a progressive image signal by pixel interpolation processing,
a) From a pixel row composed of N (N is an integer of 3 or more) adjacent pixels in the scanning line above the interpolation target pixel and from the N adjacent pixels in the scanning line below the interpolation target pixel. Two pixels located in the vertical direction while generating a virtual annular pixel row in which the right end portions and the left end portions of the pixel row are combined, and rotating the annular pixel row one pixel at a time in the right and left directions. The difference between the pixel values is determined for each of the N sets of pixels, and based on the rotation amount and the rotation direction when the sum of the differences is minimized, the presence / absence of a diagonal line including the interpolation target pixel, Oblique line determination means for identifying the stretching direction in a certain case,
b) When it is determined by the oblique line determining means that there is an oblique line including the interpolation target pixel, the pixel of the interpolation target pixel is obtained by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the extending direction of the oblique line. Interpolation value calculation for obtaining a pixel value of the interpolation target pixel by linear interpolation using a plurality of pixels sandwiching the interpolation target pixel in the vertical direction when it is determined that there is no oblique line including the interpolation target pixel Means,
An image data processing apparatus comprising: The image data processing apparatus according to claim 5,
The oblique line determining means is a difference between a minimum value of the sum of the differences when the annular pixel row is rotated in the right direction and a minimum value of the sum of the differences when the annular pixel row is rotated in the left direction. And determining whether or not there is a diagonal line including the interpolation target pixel. The image data processing apparatus according to claim 5 or 6 ,
The interpolation value calculating means passes the original image signal through a band limiting filter in at least one of interpolation of the interpolation target pixel in the oblique line extending direction including the interpolation target pixel and interpolation of the interpolation target pixel in the vertical direction. Analog including a limited analog image signal, a sampler that discretizes the analog image signal, an upsampler that inserts a predetermined number of zero-point data between sample points, a digital filter that performs filtering, and a hold that converts the discrete signal back to a continuous signal / A formula obtained by approximately converting a conditional expression set to design a digital filter to reduce an error with an analog image signal obtained through a digital / analog conversion system into a finite-dimensional discrete-time system. Exactly equivalent to solving by ^H∞ control based on predetermined conditions An image data processing apparatus, wherein the pixel value of the interpolation target pixel is calculated by a filtering process using a sample value H ^∞ filter having a parameter calculated by solving by simple calculation.