JP4190462B2 - Pixel interpolation direction detection device, pixel interpolation device, and video signal processing device - Google Patents

Description

  The present invention relates to a pixel interpolation direction detection device, a pixel interpolation device, and a video signal processing device. More specifically, when an interlaced image is converted into a progressive image, the interpolation direction is determined for a pixel to be subjected to intra-field interpolation. The present invention relates to a pixel interpolation direction detecting device for detecting, a pixel interpolating device including the device for interpolating pixels, and a video signal processing device such as a television receiver equipped with the device and suitable for obtaining a high-quality progressive image.

  In many video signals, interlaced scanning is used as a scanning form. However, if this video signal is displayed on the interlaced scanning image display unit, interlace interference such as line flickering occurs and the image quality deteriorates. This interlace interference is eliminated by IP conversion, that is, by converting interlaced video to progressive video and displaying it in the form of progressive scanning. The IP conversion is also performed for up-conversion of a television system having a different number of scanning lines and for high-resolution images.

  As the scanning line interpolation method when performing IP conversion, intra-frame interpolation is performed for still pixels, and for moving pixels, the frame correlation is broken, and the vertical line with motion is erroneously jagged in intra-frame interpolation. In-field interpolation is performed because of problems such as conversion.

  As a method of intra-field interpolation, a simple method includes a line interpolation method using the upper line as it is as a line to be interpolated, and a line interpolation method using the average value of the upper and lower lines. In the former interpolation method, an image having no correlation in the vertical direction such as a slanted line is accompanied by image deterioration such as jagged edges in the image outline, and the latter interpolation method is blurred in the image.

  As methods for solving these drawbacks, various intra-field interpolation methods have been proposed (see, for example, Patent Documents 1 and 2). The interpolation method described in Patent Document 1 interpolates each pixel of a line to be interpolated using pixel information in a direction having the strongest correlation around the pixel. First, in order to know in which direction the pixel signal extending radially from the interpolated pixel has the strongest correlation, the absolute value of the difference between adjacent pixels in the vertical direction, right diagonal direction, and left diagonal direction is obtained. It is determined that the direction in which the absolute value of the difference is minimum is the direction having the strongest correlation, and an average value of each pixel in the direction is obtained and used as the value of the interpolation pixel.

  The interpolation method described in Patent Document 2 incorporates interpolation by pixel rows and improves the processing speed as compared with the interpolation method by pixel unit that is generally performed. In the interpolation method described in the publication, each pixel of adjacent lines of a two-dimensional image is compared to detect an edge portion, and interpolation is performed using any adjacent pixel other than the edge portion, and the edge portion is adjacent. A neighboring pixel column centered on the pixel of interest on one of the lines is set, and a pixel column having a correlation therewith is selected from adjacent lines, and the position of the selected pixel column and the neighboring pixel column is selected. The number of pixels corresponding to the amount of deviation is obtained, and interpolation is performed using a pixel row obtained by shifting the selected pixel row or the neighboring pixel row in the direction opposite to the positional deviation direction by half of the obtained number of pixels. This interpolation method is a method of smoothing the contour of an image, and has a problem that it cannot be repaired when an image is interrupted between lines, as in a thin line image close to horizontal.

  FIG. 37 is a diagram for explaining an example of a video signal obtained by intra-field interpolation according to the prior art. FIG. 37A shows an example of an image represented by the video signal, and FIG. 37B shows an example of FIG. FIG. 37C is an enlarged view showing an arcuate edge portion of the image of FIG. 37A. FIG. 38 is a diagram showing an example of typical pixel values of the video signal of FIG.

  As shown in FIGS. 37B and 37C, if there is an oblique line or an edge in the oblique direction, the interpolation becomes jagged and blurred. The example of FIG. 37B will be described with reference to FIG. 38A as a typical luminance value arrangement example. When an oblique line as shown in FIG. In (A), the luminance values are arranged as shown in the actual pixel line indicated by *. FIG. 38A shows an example in which an interlaced image is interpolated with the upper and lower pixel average values, but the value of the pixel of interest represented by x is 50 because the upper and lower values are both 50.

Note that the motion information is detected by the magnitude of the difference signal between frames, and the mixing ratio between the intra-frame interpolation suitable for the still image region and the intra-field interpolation suitable for the moving image region is changed according to the detected motion information of the image. A motion adaptive interpolation process has also been proposed. A motion compensation type interpolation process has also been proposed for the purpose of interpolation that substantially matches the motion of the image. In this motion compensation type interpolation processing, the motion of an image is detected as motion vector information, and signal processing for motion compensation is performed on the signals of the preceding and succeeding fields using this motion vector information to generate interpolation scanning line signals. To do.
JP-A-63-187785 JP-A-5-30487

  However, in the intra-field interpolation processing according to the prior art as described above, when a highly correlated pixel is detected, it is performed in units of pixels or even in units of lines, so that an accurate interpolation direction is obtained. Therefore, an erroneous interpolation value is calculated especially for an oblique line (in particular, an oblique line with a shallower angle) or an image portion having an oblique edge. For example, the example of FIG. 38A actually has a slanting line in the 45 degree direction that is descending to the left. Ideally, the pixel of interest x is to be interpolated with 0. Will be interpolated at 50. Further, even if the interpolation is not performed by the upper and lower pixel average values, in each direction indicated by the arrow in FIG. 38B, even if the luminance value difference between the actual pixels is executed, no difference occurs in each direction. It is not determined whether interpolation can be performed on actual pixels. As a result, interpolation is performed with the upper and lower pixel average values and the upper or lower actual pixels.

  Further, in the intra-field interpolation processing according to the conventional technique, two real pixels having a high correlation corresponding to the determination result of the interpolation direction and two real pixels used for creating an interpolation value are used as two common real pixels. When there is an error in the determination of the correlation level, an interpolation error occurs immediately. Especially when there is an edge in an oblique direction such as an oblique line, optimal interpolation cannot be performed, and the edge portion of the oblique pattern is unstable. It becomes an image, and the image is blurred or flickered when observed as a moving image.

  In addition, in motion adaptive interpolation processing, motion information is detected based on the magnitude of the difference signal between frames, so it is not always possible to correspond to the exact motion of the image, and in the moving image area, the removal of interlace interference is incomplete. There is little improvement effect of image quality. In addition, in motion compensation type interpolation processing, accurate motion vector detection is indispensable in order to perform accurate motion compensation signal processing, and thus enormous signal processing is required.

  The present invention has been made in view of the above circumstances, and when interlaced video signals are interpolated in the field, even if diagonal lines or other diagonal edges exist for each interpolation pixel. A pixel interpolation direction detection device for detecting an interpolation direction that enables interpolation with an optimum actual pixel, and the device is provided with the device to interpolate pixels with an actual pixel in the optimum interpolation direction, and even if an oblique direction edge exists Provided are a pixel interpolation device capable of performing optimal interpolation without any change, and a video signal processing device including the device and capable of interpolating pixels to convert an interlaced video signal into a high-quality progressive video signal. That is the purpose.

In order to solve the above-described problems, the present invention is constituted by the following technical means.
The first technical means is a pixel interpolation direction detecting device for detecting the direction of interpolation for pixels to be interpolated in a field when performing IP conversion for converting an interlaced scanning video signal into a progressive scanning video signal. The pixel of interest that is the pixel to be interpolated is a region centered on the pixel of interest and that does not enclose the pixel of interest with a central pixel region that includes at least a plurality of real pixels present in the current field. A pixel area specifying means for specifying a plurality of peripheral pixel areas including at least a plurality of real pixels, which are circularly arranged at substantially equal distances from the target pixel, and specified by the pixel area specifying means A total calculation means for calculating the total pixel information of the pixels included in the central pixel area and the plurality of peripheral pixel areas for each pixel area, and each pixel area calculated by the total calculation means Interpolating direction determining means for detecting the direction with the strongest correlation based on the sum of each pixel and the positional relationship of each pixel area where the sum is made, and determining the detected direction as the interpolating direction. It is what.

  According to a second technical means, in the first technical means, the total calculating means is configured so that each pixel area in the pixel area is determined for each of the central pixel area and the plurality of peripheral pixel areas specified by the pixel area specifying means. Means is provided for weighting the pixel information of the pixels, and the sum of the weighted pixel information is calculated for each pixel region.

  The third technical means further comprises preliminary interpolation means for preliminarily generating an interpolation line by performing predetermined interpolation from an actual pixel existing in the current field in the first or second technical means, wherein the pixel area specifying means includes: The center pixel region and the plurality of peripheral pixel regions include pixels on the interpolation line interpolated by the preliminary interpolation means.

  According to a fourth technical means, in the first or second technical means, the pixel area specifying means includes, as the central pixel area, only a plurality of real pixels existing in the current field centered on the target pixel. A plurality of peripheral pixel regions including only a plurality of actual pixels arranged in a circle at substantially equal distances from the pixel of interest are specified with the pixel region as the plurality of peripheral pixel regions, respectively. It is.

  A fifth technical means includes a pixel feature determining means for determining a characteristic of the target pixel from the characteristics of surrounding real pixels in any one of the first to fourth technical means, and the interpolation direction determining means includes: Based on the determination result of the pixel feature determination means, it is determined that there is no interpolation direction without specifying a pixel area and calculating a total for a target pixel having a predetermined feature It is.

  In a sixth technical means, in any one of the first to fourth technical means, it is determined whether the target pixel corresponds to a characteristic of a complicated picture, a vertical line picture, a flat picture, or a corner picture. Pixel feature determining means for determining from pixel characteristics, wherein the interpolation direction determining means is one of a complex picture, a vertical line picture, a flat picture, and a corner picture based on the judgment result of the pixel feature judging means. It is characterized in that it is determined that there is no interpolation direction without specifying a pixel area and calculating a total for a target pixel which is a part of pixels.

  A seventh technical means is the fifth or sixth technical means, wherein the pixel feature determining means calculates a difference value of pixel information between two real pixels positioned above and below in the same field of the target pixel to be interpolated. Means for calculating a difference value of pixel information between two real pixels located adjacent to each other in the same field of the target pixel, and located several pixels on the upper line of the target pixel. Means for calculating a difference value of pixel information between two real pixels, and means for calculating a difference value of pixel information between two real pixels located several pixels apart on the lower line of the target pixel. The characteristic of the pixel of interest is the difference between the two real pixels located above and below, the difference between the two real pixels located on the left and right, the two real pixels located above and below, and the left and right Several pixels between two real pixels located in the upper and lower lines Is obtained by and judging by 2 the result of the difference between pixels, any one or more of the results of that.

  According to an eighth technical means, in any one of the first to seventh technical means, the interpolation direction determining means includes a sum for the central pixel area calculated by the total calculating means and a sum for each peripheral pixel area. A means for calculating the absolute value of the difference, a means for adding the absolute value of the difference relating to neighboring pixel regions located symmetrically with respect to the target pixel, and a minimum value among the added values is selected. And a means for detecting the direction having the selected minimum value as the direction with the strongest correlation and determining the detected direction as the interpolation direction.

  According to a ninth technical means, in any one of the first to seventh technical means, the interpolation direction determining means calculates a sum of the central pixel area calculated by the total calculating means and a total of each peripheral pixel area. A means for calculating the absolute value of the difference, a means for adding the absolute value of the difference relating to neighboring pixel regions located symmetrically with respect to the target pixel, and a maximum value among the added values is selected. And a means for detecting the normal direction of the direction having the selected maximum value as a direction having the strongest correlation and determining the detected direction as an interpolation direction.

  A tenth technical means is any one of the first to seventh technical means, wherein the interpolation direction determining means is configured to calculate a sum of the central pixel area calculated by the total calculating means and a sum of the peripheral pixel areas. Means for calculating the absolute value of the difference, means for adding the absolute value of the difference relating to neighboring pixel regions located symmetrically with respect to the target pixel, and the minimum value and the maximum value among the added values When the selected direction matches the normal direction of the direction having the minimum value and the direction having the maximum value, the direction is detected as the direction having the strongest correlation, and the detected direction is detected as the interpolation direction. And determining means.

  In an eleventh technical means according to any one of the eighth to tenth technical means, the means for calculating in the interpolation direction determining means includes the sum for the central pixel area calculated by the total calculating means and each peripheral pixel. In addition to calculating the absolute value of the difference with respect to the total with respect to the region, the difference between the peripheral pixel regions positioned symmetrically with respect to the target pixel is calculated to calculate the absolute value thereof, and the interpolation direction determination unit The means for adding is characterized by adding the absolute value of the difference between the calculated peripheral pixel regions in addition to the absolute value of the difference between the peripheral pixel regions positioned symmetrically with respect to the target pixel. Is.

  In a twelfth technical means according to any one of the eighth to tenth technical means, the adding means in the interpolation direction determining means is configured to calculate the difference between the peripheral pixel regions positioned symmetrically with respect to the target pixel. In addition to the absolute value, the absolute value of the difference between the absolute values is further added.

  In a thirteenth technical means according to any one of the eighth to twelfth technical means, the interpolation direction determining means has interpolation direction redetermining means for determining whether or not the determined interpolation direction is valid, and the interpolation The direction re-determination means is valid only when the surrounding pixel area having the minimum value among the absolute values of the differences calculated by the calculating means substantially coincides with the determined interpolation direction. It is characterized by determining.

  In a fourteenth technical means according to any one of the first to seventh technical means, the interpolation direction determining means includes a sum of the central pixel area calculated by the total calculating means and a sum of each peripheral pixel area. Means for calculating the absolute value of the difference, means for selecting a minimum value and a second minimum value next to the minimum value among the absolute values of the calculated difference, and the selected minimum value Means for detecting the direction having the strongest correlation when the direction having the second minimum value coincides with the direction having the second minimum value and determining the detected direction as the interpolation direction. is there.

  In a fifteenth technical means according to any one of the first to fourteenth technical means, the pixel area specifying means selects any of the plurality of peripheral pixel areas and all the pixels adjacent to the periphery of the central pixel area. It is characterized in that it is specified so as to be included in the peripheral pixel region.

  In a sixteenth technical means according to any one of the first to fifteenth technical means, the pixel area specifying means includes, as the plurality of peripheral pixel areas, two peripheral pixel areas positioned symmetrically with respect to the target pixel. As a pair, a plurality of pairs are specified.

  In a seventeenth technical means according to any one of the first to seventh technical means, the pixel area specifying means has a long and narrow width corresponding to a predetermined number of pixels centered on the target pixel as the central pixel area. A pixel area is specified as the plurality of peripheral pixel areas, and a pair of elongated pixel areas parallel to the central pixel area is specified as one set, and the specified set of pixel areas is centered on the target pixel. A plurality of sets of rotated pixel regions are also specified.

  According to an eighteenth technical means, in the seventeenth technical means, the interpolation direction determining means includes a sum for the central pixel area calculated by the total calculator and a sum for each peripheral pixel area in the same set as the central pixel area. Means for calculating the absolute value of the difference for each set, means for adding the absolute values of the differences calculated in each set, means for selecting the minimum value among the added values, Means for detecting the arrangement direction of the set having the selected minimum value as a direction having the strongest correlation, and determining the detected direction as an interpolation direction.

  According to a nineteenth technical means, in the seventeenth technical means, the interpolation direction determining means is a sum for the central pixel area calculated by the total calculating means and a sum for each peripheral pixel area in the same set as the central pixel area. Means for calculating the absolute value of the difference for each set, means for adding the absolute values of the differences calculated in each set, means for selecting the maximum value among the added values, Means for detecting the normal direction of the arrangement direction of the set having the selected maximum value as the direction having the strongest correlation, and determining the detected direction as the interpolation direction.

  A twentieth technical means is the seventeenth technical means, wherein the interpolation direction determining means is a sum for the central pixel area calculated by the total calculator and a sum for each peripheral pixel area in the same set as the central pixel area. Means for calculating the absolute value of the difference for each set, means for adding the absolute values of the differences calculated in each set, and selecting the minimum value and the maximum value among the added values When the arrangement direction of the set having the minimum value matches the normal direction of the arrangement direction of the set having the maximum value, the direction is detected as the direction having the strongest correlation, and the detected direction And determining means as an interpolation direction.

  In a twenty-first technical means according to any one of the first to twentieth technical means, the pixel area specifying means defines any or all of the plurality of peripheral pixel areas as a part of the central pixel area. It is characterized by specifying so that it does not overlap with a pixel.

  According to a twenty-second technical means, in the technical means according to any one of the first to twentieth technical means, the pixel region specifying means includes any or all of the plurality of peripheral pixel regions as a part of the central pixel region. It is characterized by specifying so that it may overlap with a pixel.

  According to a twenty-third technical means, in any one of the first to twenty-second technical means, the pixel area specifying means specifies the plurality of peripheral pixel areas as pixel areas each having the same number of pixels as the central pixel area. It is characterized by that.

  The twenty-fourth technical means further comprises interpolation direction correction means for correcting the interpolation direction of each pixel of interest determined by the interpolation direction determination means in any one of the first to twenty-third technical means, and the interpolation direction correction The means corrects the interpolation direction of the target pixel based on the information on the interpolation direction and information on the interpolation direction of one or more other target pixels located in the vicinity of the target pixel. It is a feature.

  A twenty-fifth technical means includes: a pixel interpolation direction detecting device according to any one of the first to twenty-fourth technical means; and for each target pixel, the interpolation based on the interpolation direction detected by the pixel interpolation direction detecting device. Interpolation value calculating means for calculating an interpolation value from real pixels in a direction, and interpolation means for interpolating each pixel of interest with the interpolation value calculated by the interpolation value calculation means It is.

  According to a twenty-sixth technical means, in the twenty-fifth technical means, the interpolation value calculating means is specified by the pixel area specifying means on the interpolation direction based on the interpolation direction detected by the pixel interpolation direction detecting device. The interpolation value is calculated from the actual pixel in the pixel area.

  A twenty-seventh technical means includes: a pixel interpolation direction detecting device according to any one of the fifth to seventh technical means; and for each target pixel, the interpolation based on the interpolation direction detected by the pixel interpolation direction detecting device. An interpolation value is calculated from an actual pixel in the direction, and when it is determined that there is no interpolation direction, an interpolation value calculation unit that calculates an interpolation value from a predetermined real pixel of the interpolation source, and the interpolation value calculation unit An interpolating unit that interpolates each pixel of interest with a calculated interpolation value.

  The twenty-eighth technical means further comprises interpolation value determination means for determining whether or not the interpolation value calculated by the interpolation value calculation means in the twenty-fifth to twenty-seventh technical means is valid, For the target pixel determined to be not an appropriate value by the interpolation value determining means, instead of the interpolation value calculated by the interpolation value calculating means, the upper / lower pixel average value of the target pixel or the upper actual pixel value or the lower The feature is that the value of the actual pixel is interpolated as an interpolation value.

  A twenty-ninth technical means is the twenty-eighth technical means, wherein the interpolated value determining means calculates a difference value between the interpolated value calculated by the interpolated value calculating means and the upper and lower pixel average values of the target pixel. And means for determining that the interpolated value is not an appropriate value when the calculated difference value is greater than a predetermined threshold value.

  A thirtieth technical means is the twenty-eighth or twenty-ninth technical means, wherein the interpolated value determining means is configured such that the interpolated value calculated by the interpolated value calculating means includes pixel information of an actual pixel above the target pixel and Means for determining that the interpolated value is not a valid value when it is not between the pixel information of the actual pixels.

  In a thirty-first technical means, in any one of the twenty-eighth to thirty-third technical means, the interpolation value determining means is a difference value between real pixels as an interpolation source with respect to the interpolation value calculated by the interpolation value calculating means. And means for determining that the interpolated value is not an appropriate value when the calculated difference value is larger than a predetermined threshold value.

  A thirty-second technical means is provided with the pixel interpolating device according to any one of the twenty-fifth to thirty-first technical means, interpolates a signal of a pixel determined not to move among interlaced scanning video signals, and performs interpolation. Video signal processing characterized in that interlaced scanning video signal is converted to progressive scanning video signal by interpolating the pixel signal determined to be moving among the video signals of race scanning by the pixel interpolator. Device.

  According to the present invention, when interlaced scanning video signals are interpolated in the field, even if diagonal lines or other diagonal edges exist for each interpolated pixel, it is possible to interpolate at the optimum actual pixel. It is possible to detect the interpolation direction. In addition, it is possible to perform an optimal interpolation that does not rattle diagonal edges such as diagonal lines with the actual pixels in the detected interpolation direction. The interlaced video signal can be converted into a high-quality progressive video signal that can be clearly displayed without blurring or flickering even if there is an oblique edge.

  FIG. 1 is a diagram illustrating a configuration example of a pixel interpolation direction detection device and a pixel interpolation device and a video signal processing device including the pixel interpolation direction detection device according to an embodiment of the present invention. Pixel / moving pixel determination device, 2 is an intra-frame interpolation device, 3 is an intra-field interpolation device, 11 is a pixel feature determination unit, 12 is a pixel interpolation direction detection unit, 13 is a pixel interpolation unit, 14 is a pixel region specifying unit, Is a total calculation means, 16 is an interpolation direction determination means, 17 is an interpolation direction correction means, 18 is an interpolation value calculation means (interpolation value generation means), 19 is an interpolation value determination means, and 20 is an interpolation means.

  The pixel interpolation direction detection device according to the present invention is a device that detects the direction of interpolation for pixels to be interpolated in a field when performing IP conversion for converting an interlaced scanning video signal into a progressive scanning video signal. A description will be given below as the pixel interpolation direction detecting means 12. The present invention relates to an apparatus for improving the image quality of a moving image portion in IP conversion from an interlaced image to a progressive image, and a circuit constituting the apparatus.

  The pixel interpolation apparatus according to the present invention is characterized by including the pixel interpolation direction detection means 12 and is an apparatus for interpolating pixels with real pixels in the optimum interpolation direction. . The intra-field interpolation device 3 mainly includes a process (main process) in the pixel interpolation direction detection unit 12 serving as a center, a process for determining pixel characteristics as a preceding process, and an interpolation process (and interpolation check) as a subsequent process. Processing). In the pixel interpolating device according to the present invention, when performing up-conversion of a television system having a different number of scanning lines or converting from an interlaced scanning video signal to a sequential scanning video signal, line interpolation such as when generating a frame image from a field image is performed. It can be executed accurately.

  Furthermore, the video signal processing apparatus according to the present invention is characterized by including the intra-field interpolation apparatus 3 and is an apparatus for interpolating pixels to convert from an interlace video signal to a progressive video signal. An example including the apparatus 1 and the intra-frame interpolation apparatus 2 is shown in FIG.

  FIG. 2 is a diagram for explaining pixels near a target pixel in one field. In FIG. 2, F is a field (also referred to as a field image), T is a pixel of interest, E is a real pixel, N is a pixel to be interpolated, Lt is an interpolation line where the pixel of interest T is located (hereinafter referred to as a pixel of interest), Lnua Is an interpolation line one above the pixel of interest T, Lnub is an interpolation line two above the pixel of interest T, Lnda is an interpolation line one pixel below the pixel of interest T, and Lndb is an interpolation line two pixels below the pixel of interest T , Leua is an actual pixel line that is one pixel above the target pixel T, Leub is an actual pixel line that is two pixels above the target pixel T, Leda is an actual pixel line that is one pixel below the target pixel T, and Ledb is 2 of the target pixel T. The lower actual pixel line, Lnua, is an interpolation line that is one level above the target pixel T.

  Here, E generally indicates an actual pixel that exists in the input field F, and N generally indicates a pixel that does not exist in the input field F, and generally indicates a pixel to be interpolated (interpolated pixel). Further, the target pixel T is the target interpolation pixel (that is, the target interpolation pixel) among the interpolation pixels N, and this target pixel T is sequentially assigned to all the interpolation pixels N. However, this assignment of the pixel of interest T to the interpolation pixel N may be omitted by the pre-processing described later. The video signal of one field is. . . , Lnub, Lnua, Lnda, Lndb,. . . It can be handled that each real pixel E is located on the following real pixel line, and if it is an odd field, an interpolation line (..., Leua, Lt, Leda, Ledb, ..) Is located, and if it is an even field, an interpolation line is located on each actual pixel line. In FIG. 2, the pixels in the field F are extracted and shown in a 9 pixel × 9 line region, but this is for the purpose of describing the pixel of interest T as a center in consideration of the paper surface.

<Pre-stage processing (pixel feature determination processing)>
First, a description will be given of pre-processing that is incorporated in a pixel interpolation direction detection device in each embodiment described later and functions appropriately. The pre-stage process described here is performed in order to avoid increasing the number of processes by detecting the interpolation direction with the pixel interpolation direction detection device for all interpolation target pixels and to increase the detection accuracy of the interpolation direction. The main purpose is to omit a pixel of interest that is not a constituent pixel such as a diagonal line or a diagonal edge from the processing (and the interpolation processing based thereon) in advance by the pixel interpolation direction detection device before the determination. Accordingly, in FIG. 1, the pixel feature determination unit 11 is illustrated as a component of the intra-field interpolation device 3 so as to be distinguished from the pixel interpolation direction detection unit 12, but the pixel feature determination unit 11 detects the interpolation direction. It can also be said that it is a constituent element of the pixel interpolation direction detection means 12 for performing it more accurately. Of course, the pixel feature determination unit 11 is not an essential component of the pixel interpolation direction detection unit 12, and the present invention may take a form in which the pixel feature determination unit 11 is not provided.

  The pixel feature determination unit 11 is a unit that determines the feature that should be the target pixel from the features of the surrounding real pixels before the pixel interpolation direction detection unit 12 determines the interpolation direction. The image has a pattern with various features in the image, and the pixel feature determination unit 11 is a unit that determines what feature the pixel of interest can be considered as a part of the pattern. The image feature determination unit Some say it. Therefore, for the target pixel having a predetermined feature based on the determination result, the processing in the pixel area specifying unit 14, the total calculating unit 15, and the interpolation direction determining unit 16 need not be substantially executed as will be described later.

  In addition, as a feature to be determined by the pixel feature determination unit 11, any one or more of a complex pattern, a vertical line pattern, a flat pattern, and a corner pattern may be used. For the target pixel T determined to be present, the interpolation direction can be unified and determined in the vertical direction. Of course, it is preferable to make a determination on all the patterns among complex patterns, vertical line patterns, flat patterns, and corner patterns. In any case, the accuracy of interpolation direction detection and the accuracy of interpolation can be improved by extracting and determining image features in advance.

In the following, a suitable determination method for a complicated pattern, a vertical line pattern, a flat pattern, and a corner pattern will be described.
FIG. 3 is a schematic diagram for explaining pixel feature determination means in the intra-field interpolation apparatus of FIG. 1. FIG. 3A shows an example of an actual pixel used for feature determination with respect to the target pixel and its difference portion. FIGS. 3B, 3C, 3D, 3E, and 3F are horizontal line complexity determination, flat pattern determination, and vertical line pattern determination in FIG. It is a figure which shows the real pixel used when performing a corner pattern determination and a complicated pattern determination, and its difference location. In FIG. 3, the same elements as those in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted.

[Horizontal line complexity judgment processing]
In the pixel feature determination unit 11, as illustrated in FIG. 3B, before the execution of the vertical line pattern determination, the complex pattern determination, the corner pattern determination, and the flat pattern determination, the actual pixel E and the arrow direction (or The difference in the opposite direction) is executed to determine the complexity of the horizontal line. Note that the flat pattern determination and the corner pattern determination can be performed in advance since they can be performed without executing the complexity determination described here.

  In the complexity determination, first, a difference value of pixel information between each real pixel adjacent in the horizontal direction in the field, that is, a difference value in the horizontal direction between each real pixel is calculated. Pixel information refers to pixel values such as luminance values. Then, for a target pixel T that is a pixel to be interpolated, a real pixel region in a predetermined range centered on the target pixel T is designated. Here, the calculation of the difference value in the horizontal direction may be executed after designating a predetermined range of actual pixel areas for each pixel of interest T. In the example of FIG. 3B, for one target pixel T, nine actual pixels in the horizontal direction are centered around the horizontal position of the target pixel T in the line one line above and one line below it. The pixel E is designated, and the difference value of eight lines (16 in total) is calculated.

  Then, the horizontal complexity for each real pixel line is determined based on the horizontal difference value in the designated real pixel area among the calculated horizontal difference values of the real pixels. For the target pixel T, there is no problem in determining only the complexity of the one upper line and the one lower line. Here, for each line, the complexity is determined by whether or not the pixel value (luminance value) is uneven in the line direction. As a preferred form of complexity determination, first, the difference value of the pixel information between the respective real pixels E adjacent in the horizontal direction is compared with two threshold values having the same absolute value, and the difference value is determined based on the comparison result. Is encoded as negative, zero, or positive. For example, reference numeral 1 is used when the difference value with the neighbor is larger than the threshold value, reference numeral -1 is assigned when the difference value with the neighbor is smaller than the threshold value, and reference numeral 0 is set for the others. Then, based on the encoded value, the difference value determines whether adjacent codes are the same code or different codes, and as a result of the determination, the number of the same code or different codes is counted for each line. Then, the complexity of the line is selected from a predetermined plurality of levels of complexity for each line based on the number of results counted. For example, when the count value (added value) is equal to or greater than another predetermined threshold (which may be an integer), it is determined that there is no unevenness, and when it is the predetermined threshold-1, it is determined that it is V, and the predetermined threshold-2. If it is, it is determined to be S, and if it is the predetermined threshold value -3, it is determined to be M, and if it is the predetermined threshold value -4, it is determined to be uneven. Thus, the complexity increases as it is determined that V → S → M → concave / convex. V, S, M, etc. are used for vertical line pattern determination and complex pattern determination.

[Flat pattern judgment processing]
To execute the flat pattern determination process, first, a vertical difference is executed. In this vertical direction difference, a difference value of pixel information between each real pixel E adjacent in the vertical direction in the field, that is, a vertical direction difference value between each real pixel E is calculated. Here, the calculation of the difference value in the vertical direction may also be executed after designating a predetermined range of actual pixel regions for each pixel of interest T.

  Based on the calculated vertical difference value of the actual pixel and the vertical difference value in the specified actual pixel area, the pixel of interest T is a part of a flat picture (flat image). It is determined whether or not. Here, in the flat pattern determination process, although it depends on the number of pixels to be determined to be a flat pattern, if it is determined that a pixel over a very long distance is a flat pattern, the determination that it is a flat pattern is the result. Is no longer output. Accordingly, among the vertical direction difference values in the designated actual pixel region (actual pixel region used in the corner pattern determination process described later), the difference values for several pixels from the left and right ends are not used for the determination. Good. Of course, you may make it designate separately the real pixel area used in the below-mentioned square picture determination process, and the real pixel area used in this flat picture determination process.

  In this flat picture determination processing, for example, as shown in FIG. 3C, the target pixel T of each pixel of the actual pixel line Leua one above the target pixel T and the actual pixel line Leda one below the target pixel T is displayed. Five real pixels E centering on the horizontal position are designated, and a vertical difference value between the real pixel E on the real pixel line Leua and the real pixel E on the real pixel line Leda is calculated. Then, when all of the upper and lower difference values are smaller than a predetermined threshold value, the target pixel T is determined to be a pixel included in the flat image.

[Vertical line pattern judgment processing]
In the vertical line pattern determination process, both the horizontal line complexity and the vertical direction difference described above are required. Here, in order to determine the horizontal line complexity, a horizontal direction difference is required, but regardless of the vertical line pattern determination process, both the horizontal direction difference value and the vertical direction difference value are required. The range of the actual pixel area to be specified may be different. If the actual pixel area to be specified for the difference value in the horizontal direction with respect to the target pixel T is a predetermined range X, the range of the difference value in the vertical direction is the predetermined range. The actual pixel area may be specified so that an appropriate number of difference values are specified in the X actual pixel area.

  In the vertical line pattern determination process, the vertical complexity in the designated real pixel area is calculated from the horizontal complexity of the upper and lower lines of the target pixel T and the calculated vertical difference value (up / down difference value) of the actual pixel. Based on the direction difference value, it is determined whether or not the target pixel T is a partial pixel of the vertical line pattern. As a preferred form of vertical line pattern determination, in the determination result of the horizontal complexity, when either the upper line or the lower line of the pixel of interest has a complexity higher than a predetermined complexity, and the vertical direction difference Among the vertical direction difference values calculated and specified in step 1, the difference value of all the real pixels E on the vertical line (vertical line (column)) located within several pixels on both the left and right sides from the target pixel T is 1 for the target pixel T. Whether the difference values between the upper line and the next lower line are all smaller than a predetermined threshold value, or the difference values of the vertical lines (vertical lines (columns)) passing through the target pixel T are all smaller than the predetermined threshold value, Alternatively, all the difference values between the actual pixel line two above the target pixel T and the actual pixel line one above, and the difference values between the actual pixel line one below and the two actual pixel lines below the target pixel T are all. If the target pixel T is smaller than the predetermined threshold, Determines that a part of the pixels of the line pattern.

  In this vertical line pattern determination processing, for example, as shown in FIG. 3D, the actual pixel line Leua one above the target pixel T or the actual pixel line Leda one below the target pixel T has a complexity of V or more, In the vertical direction difference values of all the real pixels E on the vertical line (that is, three vertical lines) located within one pixel on both the left and right sides of the pixel T, the real pixel line Leua that is one pixel below the target pixel T and one pixel below The vertical direction difference values with the actual pixel line Leda are all smaller than a predetermined threshold value, or the difference values of vertical lines (vertical lines (columns)) passing through the target pixel T are all smaller than the predetermined threshold value, or the target pixel T Vertical method difference value between the real pixel line Leub two above and the real pixel line Leua one above, and the vertical direction between the real pixel line Leda one below the target pixel T and the real pixel line Ledb two below All difference values Is smaller than a predetermined threshold value, it is determined that the pixel of interest T is part of the pixels of the vertical line pattern. That is, in this example, the upper line or the lower line of the pixel of interest T has a complexity of V or more, and all the three upper and lower difference values are smaller than a predetermined threshold value, or the upper, middle, and lower 3 If all the upper and lower difference values are smaller than a predetermined threshold value, or if all six upper and lower difference values are smaller than the predetermined threshold value, the pixel of interest T is a pixel representing a vertical line. judge.

[Corner pattern determination processing]
In the corner pattern determination process, a difference between separated real pixels is required. In this separated pixel difference, a difference value of pixel information between the real pixels E that are equidistant from the target pixel T on one line of the target pixel T and one line of the target pixel T in the designated real pixel region. Below, a difference value of pixel information between real pixels E that are equidistant from the target pixel T is calculated. Here, the calculation of the separated pixel difference value may be executed after designating a predetermined range of actual pixel regions for each target pixel.

  In the corner pattern determination process, the target pixel T is determined to be a corner based on the vertical difference value in the designated real pixel area of the calculated vertical difference values of the real pixel E and the separated pixel difference value. It is determined whether or not the pixel is a part of (corner pattern). As a preferred form of the corner pattern determination, among the calculated / designated vertical direction difference values, the vertical direction difference value between the real pixel line two above the target pixel T and the real pixel line one above and the target pixel When the vertical difference values between the actual pixel line one lower than T and the second actual pixel line lower than T are all smaller than a predetermined threshold, and the calculated difference pixel difference value is above the target pixel T When the difference value corresponding to either the line (one upper real pixel line) or the lower line (one lower real pixel line) is larger than a predetermined threshold, the target pixel T is a part of the corner picture. It is determined that it is a pixel.

  In this corner pattern determination process, for example, as shown in FIG. 3E, the vertical direction difference value between the actual pixel line Leub two above the target pixel T and the actual pixel line Leua one above the target pixel T and the target pixel T When the vertical direction difference values between the lower actual pixel line Leda and the lower actual pixel line Ledb are all smaller than a predetermined threshold, and either the upper line Leua or the lower line Leda of the target pixel T When the corresponding separated pixel difference value (the separated difference value between the real pixels separated by 2 pixels from the horizontal position of the pixel of interest T and the real pixels separated by 3 pixels) is larger than a predetermined threshold, the pixel of interest T is a square picture Is determined to be a part of the pixels. In other words, in this example, when the upper and lower ends of the target pixel T are larger than the predetermined threshold value, and all the 14 upper and lower difference values are lower than the predetermined threshold value, It is determined that the pixel represents one corner.

[Complex pattern judgment processing]
In the complex picture determination process, the horizontal complexity of the actual pixel line is used for the determination. Based on the horizontal complexity of the actual pixel line one above and below the target pixel T, it is determined whether or not the target pixel T is a part of a complex picture. As a preferable form of the complex picture determination, in the determination result of the horizontal complexity, either the upper line or the lower line of the target pixel T has a predetermined complexity (complex than the complexity used in the vertical line determination). If it has the above complexity, it is determined that the target pixel T is a part of a complex picture. In this complex picture determination process, for example, as shown in FIG. 3F, when either the upper line Leua or the lower line Leda of the target pixel T has a complexity of M or more, the target pixel T is a complex picture. Is determined to be a part of the pixels.

  By executing the pre-stage processing as described above, various setting values (various setting values such as filter weighting coefficients described later) in interpolation direction detection can be set in consideration of only edges such as diagonal lines. Therefore, not only the number of interpolation target pixels in the pixel interpolation direction detection device is reduced, but also the accuracy of the interpolation direction detection in the pixel interpolation direction detection device is improved.

<This processing (pixel interpolation direction detection processing)>
[Pixel area specification / pixel information total calculation processing (circular filter set)]
FIG. 4 is a diagram for explaining an example of a pixel area specified in the pixel interpolation direction detection apparatus according to an embodiment of the present invention. FIGS. 4A and 4B are different examples of the pixel area. FIG.

In FIG. 4, A is a central image region, B _{0 to} B ₆ (often represented by B), C _{0 to} C ₆ (often represented by C) are peripheral image regions, and Fh is a field in which an interpolation line is inserted as a temporary interpolation value. Image H is an interpolation pixel to be interpolated on the interpolation line. In addition, in FIG. 4, the same code | symbol is attached | subjected to the element similar to FIG. 2, and description is abbreviate | omitted. However, in FIG. 4 (A), E generally indicates an actual pixel existing in the input field F, whereas in FIG. 4 (B), an actual pixel existing in the field Fh is generally indicated. Point to. 4A, N is a pixel that does not exist in the input field F and generally indicates a pixel to be interpolated (interpolated pixel), whereas in FIG. 4B, H is temporary. Generally, the pixel to be interpolated in the field Fh in which the interpolation line is inserted by the interpolation value is indicated. Also, in FIG. 4B, the target pixel T is the target interpolation pixel (that is, the target interpolation pixel) of the interpolation pixels H, and this target pixel T is sequentially assigned to all the interpolation pixels H. is there. However, this assignment of the target pixel T to the interpolation pixel H may be omitted by the above-described pre-processing.

  The pixel interpolation direction detection unit 12 includes at least a pixel region specifying unit 14, a total calculation unit 15, and an interpolation method determination unit 16. The pixel area specifying means 14 is a means for specifying the central image area A and the plurality of peripheral pixel areas B and C with respect to the target pixel T which is a pixel to be interpolated (pixel to be interpolated). The interpolation direction is detected based on the pixel group of the pixel region specified here. In the present specification, the specified pixel region is illustrated as a circular region, but the specified pixel region is not limited to the circular region.

Here, the central pixel region refers to a pixel region including at least a plurality of real pixels E existing in the current field F with the target pixel T as the center, for example, a region A in FIG. Also, as shown in the figure, the plurality of peripheral pixel regions are pixel regions including at least a plurality of real pixels E arranged in a circle at a substantially equal distance from the target pixel T. For example, FIG. Regions B _{0 to} B ₆ and C _{0 to} C ₆ . Each of the plurality of peripheral pixel regions does not include the target pixel T in the region. That is, the peripheral pixel region is a region that does not surround the target pixel T. For later explanation, the peripheral pixel region B is the peripheral pixel region where the pixels on the line above the target pixel line Lt are main, and the peripheral pixel region where the pixels on the lower line are the main peripheral pixel regions. Although C may be illustrated and described as C, the peripheral pixel regions B and C are both peripheral pixel regions of the central pixel region A. Further, the circular arrangement means an arrangement at a position rotated about a certain angle around the target pixel T, and the center of the surrounding pixel area is located at the rotated substantially equidistant position. It should be arranged.

  The substantially equidistant from the target pixel T is based on the actual pixel E and the interpolation pixel H (or interpolation pixel N: no value or 0 value) obtained by sampling the video signal of one field in a lattice shape. It means that the distance is substantially equal. For example, in FIG. 4B, if the distances between the real pixels E adjacent in the horizontal direction and the vertical direction are 1 and 2, respectively, the distance from the target pixel T to the central pixel or the center of the peripheral pixel regions B and C is 2.00 to 3.16, and the distance between them may be treated as a substantially equal distance. Here, the substantially equidistant distance is a distance that is substantially equal in determining the interpolation direction of the target pixel T. The distance from the target pixel T may be defined by the distance from the end of the target pixel T or the central pixel region A on the peripheral pixel region side to the end of the peripheral pixel region on the central pixel region A side.

  These pixel areas A, B, and C are real pixels obtained by sampling one field video signal (input video signal) of interlaced scanning in a grid pattern in consideration of interpolation lines as shown in FIG. Based on E, the signal may be specified by sequentially selecting a signal corresponding to each real pixel E from the input video signal.

  Further, as shown in FIG. 4B, the center pixel area A and the peripheral pixel areas B and C include not only the actual pixel E but also the actual pixel E and the provisional temporarily obtained from them. A form including an interpolation pixel H having an interpolation value can also be adopted. In the case of this embodiment, these pixel areas A, B, and C are formed in a grid pattern in the pixel area specifying unit 14 in consideration of an interpolation line, for example, an interlaced scanning video signal (input video signal) as shown in FIG. Among the interpolated pixels H obtained by calculating the average value of the upper and lower pixels from the input video signal based on the actual pixels E sampled at the same time and the respective real pixels E that are the basis thereof. The signal may be identified by sequentially selecting signals corresponding to the above.

  Therefore, the pixel interpolation direction detection unit 12 according to this embodiment needs to further include a preliminary interpolation unit that performs predetermined interpolation from an actual pixel existing in the current field and generates an interpolation line in advance. However, as the preliminary interpolating means, if the same interpolating means is provided in the pixel interpolating means 13 described later, it may be used. In the pixel area specifying unit 14 in this embodiment, the center pixel area A and the plurality of peripheral pixel areas B and C are specified including the pixels on the interpolation line interpolated by the preliminary interpolation unit. In this case, it is preferable to set the weight of the pixel (interpolation pixel H) on the interpolation line to be lighter than the weight for the actual pixel E by weighting. In addition, the form regarding weighting is mentioned later.

  On the other hand, in the pixel area specifying unit 14 described above with reference to FIG. 4A, a pixel area (centered on the target pixel T) including only a plurality of real pixels E existing in the current field F is set as the central pixel area A. Identify the peripheral pixel regions B and C that include only a plurality of real pixels E arranged in a circle (on the circumference) at a substantially equal distance from the target pixel T as a plurality of peripheral pixel regions B and C. is doing. Of course, in the form of the pixel area specifying unit 14 described with reference to FIG. 4A, not only the pixel areas A, B, and C are specified so as to include the interpolated pixel N that does not include the provisional interpolation value. The pixel regions A, B, and C are specified so as to be included as interpolation pixels N having zero values that do not contribute to the total calculation in the subsequent stage, or the pixel regions A, B, and C include substantially only the actual pixel E. This corresponds to specifying C.

  4A and 4B, the plurality of peripheral pixel regions B and C are arranged with the same number of pixels as the central pixel region A (or on the same grid). It is preferable in terms of the circuit configuration that the pixel region is specified as a (wide) pixel region. However, when performing weighting, which will be described later, it may be specified as a pixel region having the same number of pixels in consideration of the weight.

Further, when the peripheral pixel areas B and C are specified, all the pixels existing around the specified central pixel area A are assigned to any one of the peripheral pixel areas B _{0 to} B ₆ and C _{0 to} C _6. It is preferable to specify the pixel region so that it is included. This form is also applicable to the pixel region specifying means 14 illustrated in any of FIGS. 4A and 4B. In this embodiment, since the periphery of the central pixel region A is surrounded without omission by the plurality of peripheral pixel regions B and C, the interpolation direction calculation contributes not only to the region A but also the pixels in all directions around it. This means that the pixels close to the target pixel T can be used as much as possible when calculating the interpolation direction, and as a result, the accuracy of the interpolation direction detection and interpolation can be improved. In this embodiment, even if it is specified that the peripheral pixel regions B and C are arranged around the central pixel region A, or the central pixel region A and the peripheral pixel regions B and C overlap each other as described later (overlapping pixels). May be specified). Note that this embodiment is not applied to pixels adjacent to both ends of the region A in the later-described embodiments in which the regions A, B, and C are elongated pixel regions.

In addition, it is preferable that the pixel area specifying unit 14 specifies a plurality of pairs as two or more peripheral pixel areas B and C with two peripheral pixel areas positioned symmetrically with respect to the target pixel T as a pair. Here, the direction in which each pair is arranged is specified at a position rotated around the target pixel T. 4A and 4B, a pair of region B ₀ and region C _0, a pair of region B ₁ and region C _1, a pair of region B ₂ and region C ₂ , and a region B ₃ It is illustrated as a pair of region C _{3 and} a pair of region B ₄ and region C ₄ .

As will be described later in the determination of the interpolation direction, the interpolation direction is determined by the direction (or the normal direction) formed by the set of the pair and the central pixel region, in other words, determined by the pair direction (or the normal direction). is doing. Therefore, after specifying the pixel area, it may be determined between the pixel areas whose directions from the target pixel coincide with each other, but it is efficient to specify the pixel area in pairs as described above as described above. . The interpolation direction is determined in this direction using three consecutive regions (for example, regions B ₁ , A, C ₁ ) consisting of a pair of peripheral pixel regions and a central pixel region on a straight line. Therefore, such a filter composed of the pixel region specifying means 14 and the sum calculating means 15 described later is often referred to as a “circular filter set”. Note that each region (e.g., region B ₁ and area A, etc.) is called a circular pixel area, further, the pixel area specifying means 14 for specifying the circular pixel area, such as multiplying the coefficients to real pixel information The total calculation means 15 for calculating the total value is collectively referred to as a “circular filter”, and three consecutive regions (circular pixel regions) on a straight line are referred to as a “circular pixel region set”. However, in these terms, “circular shape” is used, but it is of course not limited to the circular shape, and the area may be specified in any shape such as “circular” or “rectangular”, and the whole Needless to say, the shape is circular but may actually be rattled.

  Further, as shown in FIGS. 4A and 4B, the pixel area specifying unit 14 includes any one or a plurality (all peripheral pixel areas) of the plurality of peripheral pixel areas B and C. ) May be specified so as to overlap the central pixel region A with some pixels. By providing such an overlap, it becomes possible to determine the interpolation direction in a limited range of pixel regions, and as a result, the circuit is compared with a configuration in which the overlap described later in FIGS. 8 and 9 is not provided. In addition to avoiding an increase in scale, interpolation direction can be determined based on more neighboring information. Note that the pixel area to be specified may be, for example, an area of 3 × 3 pixels or 3 × 4 pixels including pixels to be interpolated as illustrated in FIG.

The total calculation means 15 includes pixels (real pixel E / real pixel E and interpolation pixel H / real pixel E and zero-value interpolation pixel) included in the central pixel area and the plurality of peripheral pixel areas specified by the pixel area specifying means 14. N) is a means for calculating the sum of the pixel information for each pixel area, and functions in combination with the pixel area specifying means 14 as a circular filter that removes high-frequency components in each area. A combination of the circular filters related to the regions B ₁ , A, and C ₁ is a “circular filter set”. Here, the pixel information of a pixel refers to a pixel value such as a luminance value.

  5 to 7 are diagrams for explaining examples of weighting coefficients at the time of total calculation in a pixel interpolation direction detection device according to another embodiment of the present invention. 5 and 6 are diagrams illustrating examples of weighting coefficients in each direction in a form in which interpolation pixels are not used for total calculation, and FIG. 7 is a diagram illustrating weighting coefficients in each direction in a form in which interpolation pixels are used for total calculation. It is a figure which shows an example. 5 to 7, the same elements as those in FIGS. 2 and 4 are denoted by the same reference numerals, and the description thereof is omitted.

  In a preferred form, the total calculation means 15 has pixel information of each pixel in the pixel area for each pixel area of the central pixel area A and the plurality of peripheral pixel areas B and C specified by the pixel area specifying means 14. It is assumed that means for weighting is provided. This means corresponds to the circular filter described above, and in this means, it is preferable to multiply the pixel information of each pixel by a weighting coefficient in accordance with the direction of the surrounding pixel regions B and C with respect to the target pixel T. Then, the total calculation unit 15 calculates the sum of the weighted pixel information for each pixel area. Note that the interpolation pixel H may be weighted, or a weighting filter (circular filter) considering the interpolation pixel H is prepared in advance, and the calculation of the interpolation pixel H is not performed. If the sum of the information is calculated, a total value may be calculated in consideration of the provisional interpolation.

  Examples of the interpolation direction to be detected and the weighting coefficient in each direction are 0 to the clockwise direction around the target pixel T with respect to each interpolation direction in which the direction close to the horizontal direction is defined finely and the direction close to the vertical direction is roughly defined. The 11th direction is illustrated in FIGS. Here, the 11th direction is the horizontal direction, the 5th direction is the vertical direction, and the 10th direction, the 9th direction, the 8th direction, the 7th direction, and the 6th direction are the 0th direction, the 1st direction, The second direction, the third direction, and the fourth direction are symmetric with respect to a vertical line passing through the target pixel T. In addition, the 2nd direction and the 8th direction are not taken as interpolation directions to detect because there are few differences between the 1st and 3rd directions and the 7th and 9th directions, respectively. , 8 may be incorporated as an interpolation direction for detecting the direction. Further, the detected interpolation direction may be a direction rotated by a predetermined angle. For example, the direction of about every 15 degrees clockwise around the target pixel T is set as the 0th to 11th directions. Although mentioned above, you may make it not specify the direction which does not contribute much to interpolation direction determination among them as mentioned above.

  Further, when a circular filter in the 11th direction is used, the horizontal direction (straight side line / edge) can be detected. Since it is difficult to detect the horizontal direction (straight line / edge) by the pixel feature determination unit 11 in the previous stage, the horizontal direction (the 11th direction) may be incorporated as an interpolation direction to be detected, but is the same as other oblique directions. If an attempt is made to detect the eleventh direction (in this case, it is conceivable to interpolate with the average of the upper and lower pixels in the end), a smooth line or edge close to the horizontal is determined as the eleventh direction, and the final result is Interpolation may be performed by averaging the upper and lower pixels, and a gentle line may rattle. Therefore, it is preferable not to use the eleventh circular filter to prevent the pixel interpolation direction detection means 12 from performing horizontal detection. Further, when the circular filter in the fifth direction is used, the vertical direction (true vertical line / edge) can be detected. However, in the form in which the pixel feature determination unit 11 in the previous stage detects the vertical line. May not use a circular filter in the fifth direction.

  Here, the directions such as the first to eleventh directions are preferably made to correspond to the positions of the actual pixels to be used for the interpolation value generation described later executed after the interpolation direction is determined. That is, the directions 1 to 11 described here preferably correspond to the direction of the real pixel of the interpolation source that finally creates the interpolation value. When there are a plurality of pairs of real pixels of the interpolation source, It is preferable to correspond to the direction based on the intermediate position. The real pixel of the interpolation source will be described later with reference to FIG. 23. In the examples of FIGS. 5 to 7, the interpolation direction for determining the direction corresponding to the direction from the target pixel T to the real pixel of the interpolation source in FIG. Adopted.

5 and 6, the areas B ₀ , B ₁ , B ₉ , B _{10 in} the _0th , _1st , _9th , _10th directions as the peripheral pixel area B and ₀ , ₁ , ₉ , ₉ as the peripheral pixel area C are shown. The region C ₀ , C ₁ , C ₉ , C ₁₀ in the _tenth direction is composed of 3 pixels in the horizontal direction and 5 pixels in the vertical direction, including the actual pixel E and the interpolation pixel N (the total number of pixels is 3 actual pixels E in the horizontal direction). X shows an example of three real pixels E) in the vertical direction. In the other direction area, the actual pixel E and the interpolated pixel N are combined to be 3 pixels in the horizontal direction × 3 pixels in the vertical direction (the total pixels are 3 actual pixels E in the horizontal direction × 2 actual pixels E in the vertical direction). It is said. In FIG. 5A, as an example of the central pixel area A and the peripheral pixel areas B ₀ and C _{0 in} the _0th direction, each original pixel is weighted so that the center actual pixel becomes a larger weighting coefficient for each area. An example in which coefficients are assigned is shown. Similarly, FIG. 5B is the first direction, FIG. 5C is the third direction, FIG. 5D is the fourth direction, FIG. 5E is the fifth direction, and FIG. 6 (A) corresponds to the 10th direction, FIG. 6 (B) corresponds to the 9th direction, FIG. 6 (C) corresponds to the 7th direction, and FIG. 6 (D) corresponds to the 6th direction. The example of the weighting coefficient to perform is shown.

FIG. 7 also shows that all peripheral pixel areas B and C are 3 pixels in the horizontal direction × 3 pixels in the vertical direction, including the real pixel E and the interpolation pixel H (the total number of pixels is also 3 pixels in the horizontal direction × vertical direction). 3 pixels). In FIG. 7A, as an example of the center pixel region A and the peripheral pixel regions B ₀ and C _{0 in} the _0th direction, each original pixel and interpolation are set so that the center actual pixel becomes a larger weighting coefficient for each region. An example in which a weighting coefficient is assigned to the pixel H is shown. The coefficient obtained by replacing the coefficient shown in FIG. 7A symmetrically with respect to the vertical line passing through the target pixel T is used as a weighting coefficient for the peripheral pixel regions B ₁₀ and C _{10 in} the _tenth direction. Similarly, FIG. 7B shows the first direction, FIG. 7C shows the third direction, FIG. 7D shows the fourth direction, FIG. 7E shows the fifth direction, and FIG. Examples of weighting coefficients corresponding to the peripheral pixel regions in the number direction are shown. The coefficients shown in FIGS. 7B, 7C, and 7D are applied to the vertical line passing through the target pixel T. The coefficients replaced symmetrically are the weighting coefficients for the peripheral pixel areas B ₉ and C _{9 in} the _9th direction, the weighting coefficients for the peripheral pixel areas B ₇ and C _{7 in} the _7th direction, and the peripheral pixel area B _{6 in the 6th} direction, respectively. , used as a weighting coefficient _{C 6.}

  FIG. 8 is a diagram for explaining an example of the pixel area specified in the pixel interpolation direction detecting device according to another embodiment of the present invention, and for explaining an example in which the specified pixel areas do not overlap. FIG. Note that FIG. 8 shows an example of the weighting coefficient in each direction in the form in which the interpolation pixels are used for the total calculation. In this example, the pixel area specified as the pixel area illustrated in FIG. It differs in that it does not overlap. FIG. 9 is a diagram for explaining an example of the weighting coefficient at the time of total calculation in the pixel interpolation direction detection device according to another embodiment of the present invention, and the specified pixel regions do not overlap as in the example of FIG. It is a figure for demonstrating the example to do. FIG. 9 shows an example of the weighting coefficient in each direction in a form in which interpolation pixels are not used for the total calculation. In this example, the pixel area specified as the pixel area illustrated in FIG. It differs in that it does not overlap. In the figure, Leuc is an actual pixel line three above the target pixel T, Ledc is an actual pixel line three lower than the target pixel T, and other elements similar to those in FIGS. 2 and 4 are denoted by the same reference numerals. The description is omitted.

  As another embodiment of the present invention, the pixel area specifying unit 14 specifies any or all of the plurality of peripheral pixel areas B and C so as not to overlap with the central pixel area A and some pixels. It is preferable to do. As in the form in which the peripheral pixel areas B and C illustrated in FIGS. 4A and 4B overlap the central pixel area A, the pixel area to be secured can be narrowed in consideration of the line memory and the processing capability. In the present embodiment described with reference to FIGS. 8 and 9, the central pixel area A and the peripheral pixel areas B and C are arranged so as not to overlap each other. To do. In the present embodiment, it is necessary to secure a wide pixel area, and the advantages are as follows. That is, when the peripheral pixel areas B and C overlap with the central pixel area A, as will be described later, it becomes difficult to make a difference in the calculation results such as | A−B | + | A−C | . This is because A (center pixel area) and B, C (peripheral pixel areas) overlap, so the same value (pixel value) is included and the difference between the same pixels is zero. However, by avoiding overlapping portions as much as possible as in the present embodiment, the above calculation result is different, and the interpolation direction can be easily determined.

FIG. 8A illustrates how to specify a pixel area in which the peripheral pixel area and the central pixel area do not overlap. FIGS. 8B and 8C illustrate weighting coefficients for the pixel areas A, B, and C, respectively. An example is shown. The example of the weighting coefficient in FIG. 8A may be applied to the central pixel area A, and the example of the weighting coefficient in FIG. 8C may be applied to the peripheral pixel areas B and C. . In the specific method of the pixel region illustrated in FIG. 8A, the peripheral pixel regions B ₀ , B ₂ , B ₃ , B ₄ , B ₅ , B ₆ , B ₇ , B ₈ , B ₁₀ , B ₁₁ , The areas are specified so that C ₀ , C ₂ , C ₃ , C ₄ , C ₅ , C ₆ , C ₇ , C ₈ , C ₁₀ , and C ₁₁ do not overlap the central pixel area A. Note that the first direction and the ninth direction are not taken in as the interpolation directions to be detected because there is little difference between the second direction, the third direction, the seventh direction, and the eighth direction. However, the 1st and 9th directions may also be incorporated as interpolation directions to be detected. When the upper and lower pixel average values are inserted in advance in the interpolation line as shown in FIG. 8A, the weighting coefficient exemplified in FIGS. 8B and 8C is multiplied by the pixel value of each pixel area. What is necessary is just to perform the determination of the interpolation direction described later.

  9A to 9F show an example of how to specify a pixel area in each direction in which the central pixel area and the peripheral pixel area do not overlap with each other and the weighting coefficient. FIGS. 9A to 9F are for the directions of No. 0, No. 2, No. 3, No. 4, No. 5, and No. 11, respectively. Here, the 11th direction is the horizontal direction, the 5th direction is the vertical direction, and the peripheral pixel areas specified in the 10th direction, the 8th direction, the 7th direction, and the 6th direction and examples of their weighting coefficients are respectively The 0th direction, the 2nd direction, the 3rd direction, and the 4th direction are symmetric with respect to the vertical line passing through the target pixel T. The 1st direction and 9th direction are not taken as interpolation directions to be detected because the direction difference is small with the 0th direction, the 2nd direction, the 8th direction, and the 10th direction, respectively. However, the 1st and 9th directions may also be incorporated as interpolation directions to be detected. Further, in the example of the weighting coefficient exemplified in FIG. 9, the upper and lower pixel average values are inserted as temporary interpolation values into the interpolation line as shown in FIG. 8A, and the weighting coefficient shown in FIG. It is a coefficient equivalent to the case where the areas A, B and C are multiplied.

In addition, an actual pixel G that is an interpolation source illustrated by a white circle (◯) in FIG. 9 is an actual pixel that is referred to when calculating an interpolation value when the interpolation direction is determined to be an average value of these pixels. Alternatively, it may be calculated by multiplying each of these pixels by a coefficient. With respect to the 0th direction illustrated in FIG. 9A, the third real pixel G on the upper line of the target pixel T and in the horizontal direction from the target pixel T, and the point symmetric with respect to the target pixel T of the real pixel. The actual pixel G is an interpolation source. Note that these real pixels G in FIG. 9A are the central pixels of the peripheral pixel regions B ₀ and C ₀ and the peripheral pixel regions B ₁₀ and C ₁₀ , respectively. Further, for the second direction illustrated in FIG. 9B, a set of real pixels of the pixel T of interest which is 2 left to the top and 1 away from the top and 2 to the right and 1 away from the bottom. G, a set of real pixels G that are one to the left of the pixel of interest T, 1 away from the top, and one of the pixels to the right and 1 from the bottom of the pixel of interest T; A set of real pixels G, one on the right and four on the right and three on the bottom, is the interpolation source. For the eleventh direction (horizontal direction) illustrated in FIG. 9F, all the real pixels adjacent to the target pixel T (a total of six pixels) are interpolation sources. Similarly, when it is determined that the interpolation direction is the (10), (8), 3 (7), 4 (6), or 5th direction, FIGS. 9 (A), (B), (C), ( The actual pixel G at the positions indicated by D) and (E) (or the actual pixel G at the point-symmetrical position) is determined as the interpolation source pixel.

  For the example in which the central pixel area A and the peripheral pixel area B overlap as illustrated in FIG. 4 and the like, as will be described later with reference to FIG. A typical example is to calculate an interpolation value. However, for example in which the central pixel area A and the peripheral pixel area B do not overlap as in the present embodiment, for example, as shown in FIG. The interpolation value may be calculated from the actual pixels on the line or the upper and lower three lines. However, if the distance from the pixel of interest T of the real pixel G to be interpolated is too far as shown in FIG. 9D, the possibility of erroneous interpolation results increases. The interpolation value may be calculated by multiplying the coefficient (the coefficient closer to the target pixel T is larger and the coefficient farther away is smaller). Further, when this interpolation value calculation direction is applied, as shown in FIG. 23, when it is desired to calculate an interpolation value from the actual pixels on the upper and lower lines of the target pixel T, it is applied to the actual pixels on and below the second line or the third line. It is only necessary to set the coefficient to zero. The interpolation value calculation method exemplified here corresponds to another example of “determination processing of actual pixels in the interpolation direction” which will be described later with reference to FIG.

8 and 9, even in the embodiment described above, that is, a form in which a plurality of peripheral pixel areas B and C are specified as pixel areas having the same number of pixels as the central pixel area A, and peripheral pixels. Pixels so that all the pixels existing around the central pixel region A specified when specifying the regions B and C are included in any of the peripheral pixel regions of the regions B _{0 to} B ₆ and C _{0 to} C ₆ It is preferable to apply a form in which a region is specified or a form in which a plurality of peripheral pixel regions B and C are specified as a pair of two peripheral pixel regions positioned symmetrically with respect to the target pixel T.

[Interpolation direction judgment processing]
The interpolation direction determination means 16 detects the direction with the strongest correlation based on the total for each pixel area calculated by the total calculation means 15 and the positional relationship of each pixel area for which the total is made, and determines the detected direction. It is a means to determine as an interpolation direction. In other words, the interpolation direction determining means 16 determines the direction with the strongest correlation among the directions in which the central pixel area and the peripheral pixel areas are arranged based on the total calculated by the total calculating means 15 as the interpolation direction. Judge as. As described above, the interpolation direction determination unit 16 determines an interpolation direction having a higher correlation than the circular filter result.

  First, in the embodiment including the pixel feature determination unit 11, the interpolation direction determination unit 16 specifies a pixel region for a target pixel having a predetermined feature based on the determination result of the pixel feature determination unit 11. It is determined that there is no interpolation direction without calculating the total. When it is determined that there is no interpolation direction, an interpolation value calculation unit 18 described later may calculate an interpolation value from a predetermined real pixel of the interpolation source, and the interpolation unit 20 may perform interpolation. In this way, the interpolation direction is not determined, and the interpolation direction is determined by the method described here only for the target pixel that has a different pattern. It is possible to improve the accuracy of interpolation by extracting and determining image features in advance.

  In particular, the pixel feature determination means 11 is a means for determining from the characteristics of the surrounding real pixels whether the pixel of interest corresponds to a complicated picture, vertical line picture, flat picture, or corner picture feature. Then, the interpolation direction determination unit 16 applies the target pixel which is a partial pixel of any one of a complex pattern, a vertical line pattern, a flat pattern, and a corner pattern based on the determination result of the pixel feature determination unit 11. Therefore, it is determined that there is no interpolation direction without specifying the pixel area and calculating the total. When it is determined that there is no interpolation direction, an interpolation value calculation unit 18 (to be described later) calculates an interpolation value from an actual pixel that is predetermined for each picture, and the interpolation unit 20 performs interpolation. It is good to. The interpolation direction is determined by the method described here only for the pixel of interest having a pattern other than that. It is possible to improve the accuracy of interpolation by extracting and determining image features in advance. Here, the predetermined interpolation source determined in advance for each picture is, as the simplest method of determination, the actual pixels in the vertical direction for all these pictures, and finally interpolated with the upper and lower pixel average values. You may do it. In addition, as actual pixels that are preferable predetermined interpolation sources, real pixels in the upward direction different from the vertical direction, real pixels in the direction of all directions, etc. are prepared, and it is determined that the picture is complicated For the actual pixel in the upper direction, the same pixel value (luminance) as the upper line is set, and when it is determined to be a flat picture, the average value of the surrounding pixels is set as the actual pixel in all directions. When the corner pattern is determined, the upper and lower pixel average values may be used as the actual pixels in the vertical direction.

  Next, the processing in the interpolation direction determination unit 16 will be described in detail with reference to FIGS. 10 to 13, the design is shown by hatching. The same reference numerals are given to the same elements as those in FIGS. 2 and 4, and the description thereof is omitted.

  FIG. 10 is a diagram for explaining interpolation direction determination processing in the pixel interpolation direction detection device according to an embodiment of the present invention, where a black line (color line) pattern facing right upward is present on a white background. It is a figure which shows what applied the circular filter set to.

The interpolation direction determination means 16 in this embodiment is means for calculating the absolute value of the difference between the sum for the central pixel area A calculated by the sum calculation means 15 and the sum for the surrounding pixel areas B and C. Have. That is, the absolute value of the difference between the circular filter result of the region A including the target pixel T and the circular filter results of the plurality of regions B and C arranged in a circle is obtained. Here, the peripheral pixel areas B and C indicate all the areas specified by the pixel area specifying means 14, and in FIG. 10, areas B ₀ , B ₁ , B ₂ , B ₃ , B ₄ (each number 0) Direction, 3rd direction, 5th direction, 7th direction, 10th direction) and areas C ₀ , C ₁ , C ₂ , C ₃ , C ₄ (0th direction, 3rd direction, 5th direction respectively) , 7th direction, and 10th direction). Then, the calculated total difference absolute value will be described by the name of the area. | A-B ₀ |, | A-B ₁ |, | A-B ₂ |, | A-B ₃ |, | A−B ₄ |, | A−C ₀ |, | A−C ₁ |, | A−C ₂ |, | A−C ₃ |, and | A−C ₄ |.

The interpolation direction determination means 16 in the present embodiment further includes means for adding the absolute values of differences relating to peripheral pixel areas located symmetrically with respect to the target pixel T. In the example of FIG. 10, D _n = | A−B _n | + | A−C _n | is obtained for n = 0, 1, 2, 3, and 4, respectively. Then, the interpolation direction determining means 16 detects a direction (selector or the like) that selects the minimum value among the added values D _n , detects the direction having the selected minimum value as the direction with the strongest correlation, and detects the detected direction. And a means for determining as an interpolation direction. In this configuration, the relationship between the three circular filters D _n = | A−B _n | + | A−C _n |, that is, the minimum value direction is set as the diagonal correlation direction by the circular filter set of D _n. To detect.

As shown in FIG. 10, when a set of three circular pixel regions (B ₃ , A, C ₃ ) is on an oblique line, three circular filter results (the luminance of the circular filter coefficient is multiplied by the luminance). The difference between the three circular filters (B ₁ , A, C ₁ or B ₀ , A, C ₀ ) orthogonal to the diagonal line is large. In this case, D ₃ = | A−B ₃ | + | A−C ₃ | becomes a small value, and D ₀ = | A−B ₀ | + | A−C ₀ | and D ₁ = | A−B ₁ | + | A−C ₁ | is a large value. Thus, _if the direction of the small difference D _n = | A−B _n | + | A−C _n | between the three circular filters positioned symmetrically with respect to the interpolation pixel T is a direction having a strong correlation, it is so-called. Yeah.

Here, the interpolation direction determining means 16 detects the normal direction of the direction having the selected maximum value as the direction having the strongest correlation, and selects the maximum value among the added values, and interpolates the detected direction. And a means for determining the direction. In this embodiment, the relationship between the three circular filters D _n = | A−B _n | + | A−C _n |, that is, the normal direction of the maximum value is slanted by the circular filter set of D _n. Detect as correlation direction. Referring to FIG. 10 again, in FIG. 10, a set of three circular pixel regions (B ₃ , A, C ₃ ) is on the diagonal line, and the results of the three circular filters (the coefficients of the circular filter are shown). The difference of the sum of luminance multiplied is small, while the difference between the three circular filters (B ₁ , A, C ₁ or B ₀ , A, C ₀ ) orthogonal to the diagonal line is large. In this case, D ₃ = | A−B ₃ | + | A−C ₃ | becomes a small value, and D ₀ = | A−B ₀ | + | A−C ₀ | and D ₁ = | A−B ₁ | + | A−C ₁ | is a large value. As a result, the normal direction of the direction in which the difference D _n = | A−B _n | + | A−C _n | of the three circular filters positioned symmetrically with respect to the interpolation pixel T is the direction in which the correlation is strong. There is so-called. As described above, even in this form, in accordance with the same idea as the form described above, the difference D _n = | A−B _n | + | between the three circular filters positioned symmetrically about the interpolation pixel T. A−C _n | is obtained, and it can be said that a direction with a large difference has a weak correlation, that is, a direction perpendicular to the direction has a strong correlation. Here, the normal direction may not be strictly a normal line, and refers to the direction of the peripheral pixel region specified at least in the direction near the direction.

The interpolation direction determination means 16 according to another embodiment of the present invention differs in the selection and detection portions of the interpolation direction determination means 16 of each embodiment described above. In this embodiment, the minimum value and the maximum value are selected from the added values, and the direction of the selected minimum value and the direction of the maximum value coincide with each other at the time of detection. In this case, the direction is detected as the direction having the strongest correlation. The direction detected here is determined as the interpolation direction. In this embodiment, the relationship between the three circular filters D _n = | A−B _n | + | A−C _n |, that is, the minimum value direction and the maximum value direction are determined by the circular filter set of D _n. Are orthogonal, the minimum direction is detected as the diagonal correlation direction.

  FIG. 11 is a diagram for explaining interpolation direction determination processing in a pixel interpolation direction detection device according to another embodiment of the present invention, and is colored with an edge that descends to the lower right (an oblique upper left inclined edge) on a white background. It is a figure which shows what applied the circular filter set when the pattern exists.

The interpolation direction determination unit 16 according to another embodiment of the present invention is different from the interpolation direction determination unit 16 of each embodiment described above in the calculation of the total difference absolute value. The means for calculating the total difference absolute value in this embodiment is not only the calculation of the absolute value of the difference between the total for the central pixel area A calculated by the total calculation means 15 and the total for each of the surrounding pixel areas B and C, The total difference between the peripheral pixel regions positioned symmetrically with respect to the target pixel T is calculated and the absolute value thereof is calculated. The addition means in the interpolation direction determination means 16 adds the absolute value of the difference | A−B _n | + | A−C _n | of the difference between the peripheral pixel regions located symmetrically with respect to the target pixel T, Further, the absolute value | C _n −B _n | of the difference between the calculated peripheral pixel regions is added. Therefore, the interpolation direction determination means 16 according to the present embodiment obtains D _n = | A−B _n | + | A−C _n | + | C _n −B _n |, and the minimum value direction has the strongest correlation. The direction is detected, and the direction is determined as the interpolation direction.

Actually, in the diagonal pattern shown in FIG. 10, although the difference | C _n −B _n | is small even _if B _n and C _n are compared, the pattern having an edge as shown in FIG. The difference | C ₀ −B ₀ | between B ₀ and C ₀ on the oblique edge is small, but the difference between B ₂ and C ₂ orthogonal to the edge | C ₂ −B ₂ | or the difference between B ₃ and C ₃ | C ₃ −B ₃ | is large. Therefore, if | C _n −B _n | is obtained and added to the detection using | A−B _n | + | A−C _n |, the detection accuracy in the interpolation direction is further increased.

In another embodiment, the addition means in the interpolation direction determination means 16 adds the absolute value of the difference between the absolute values in addition to the absolute value of the difference between the peripheral pixel regions positioned symmetrically with respect to the target pixel. You may make it add a value. That is, the minimum value direction of D _n = | A−B _n | + | A−C _n | + || A−B _n | − | A−C _n || is obtained, and the minimum value direction has the strongest correlation. The direction is detected, and the direction is determined as the interpolation direction. This embodiment is based on the same concept as the embodiment described with reference to FIG. 11, but | C _n −B _n | is obtained by || A _n −B _n | − | A _n −C _n || it is not newly prepared a circuit _C n -B _n, it is possible to reuse a circuit a n -B _n and _a n -C _n. Note that | C _n −B _n | and || A _n −B _n | − | A _n −C _n || are not equivalent.

  FIG. 12 is a diagram for explaining interpolation direction determination processing in a pixel interpolation direction detection device according to another embodiment of the present invention. 12, elements similar to those in FIGS. 2 and 4 are given the same reference numerals, and descriptions thereof are omitted.

In the interpolation direction determination unit 16 in any one of the above-described embodiments, the interpolation direction determination unit 16 includes an interpolation direction re-determination unit that determines whether the determined interpolation direction is appropriate or not in order to prevent erroneous oblique correlation detection. preferable. The interpolation direction re-determination means includes a peripheral pixel area having a minimum value among the absolute values | A−B _n | and | A−C _n | of the difference calculated by the calculation means in the interpolation direction determination means 16. It is determined that the interpolation direction is valid only when it substantially matches the interpolation direction once determined. Here, as long as the directions are not clearly different, such as being positioned at a right angle to the interpolation direction, they should be regarded as substantially coincident.

First, in the circular filter set, the interpolation direction determination unit 16 in any one of the above-described embodiments such as the minimum value direction of | A−B _n | + | A−C _n | Next, this interpolation direction re-determination means finds the minimum value of | A−B _n |, | A−C _n | (omnidirectional), and if this is on or near the determined interpolation direction, The direction can be regarded as an interpolation direction. On the other hand, if it is positioned at right angles to the interpolation direction once determined, the interpolation direction cannot be said to be a valid interpolation direction, so the upper and lower average pixel values of the target pixel (or the actual pixel above / below the target pixel). The interpolation direction may be set to the vertical direction or the like so that the value is interpolated as an interpolation value. In the example of FIG. 12, | A−B ₀ | + | A−C ₀ | is the minimum value, and the B ₀ AC ₀ direction is once determined as the interpolation direction. If the region C ₃ is where the difference between the circular filter of the center A and the circular filters of the surrounding B _n and C _n is the region C ₃ , the AC ₃ direction is not near the B ₀ AC ₀ direction. There is no correlation in the B ₀ AC ₀ direction.

FIG. 13 is a diagram for explaining an interpolation direction determination process in a pixel interpolation direction detection device according to another embodiment of the present invention. FIG. 13A is an edge (left diagonally upward slope) that falls diagonally to the right on a white background. FIG. 13B is a diagram showing a circular filter set applied when there is a colored pattern having an edge), and FIG. 13B is a case where there is a colored pattern having a trapezoidal edge that extends diagonally downward to the left on a white background It is a figure which shows what applied the circular filter set to. In the example of FIG. 13, it is not necessary to consider the peripheral pixel region in pairs, so that regions B _{0 to} B ₉ (in the directions of ₀ , 3, 5, 7, 10, _{0, 3, 5, 7, 10} respectively). Equivalent).

As another embodiment of the present invention, the oblique correlation direction may be detected by the central circular filter and all the surrounding circular filters. That is, the interpolation direction determination means 16 in the present embodiment takes the difference between the total for the central pixel area A calculated by the total calculation means 15 and the total for each of the surrounding pixel areas B _{0 to} B ₉ , and their absolute value D a means for calculating _n = | A−B _n | (n = 0 to 9). Further, the interpolation direction determination means 16 selects, from the absolute value D _n of the calculated difference, a minimum value and a second minimum value next to the minimum value, a direction having the selected minimum value, When the direction having the two minimum values coincides, that is, the peripheral pixel region B having the selected minimum value, the central pixel region A, and the peripheral pixel region B ′ having the same second selected minimum value are aligned. If the position is located in the direction, the direction is detected as the direction with the strongest correlation, and the detected direction is determined as the interpolation direction. That is, D _n = | A _n −B _n | is calculated, the circular filter (peripheral filter) and the second smallest peripheral filter in the smallest peripheral pixel region of D _n are detected, and the two selected circular shapes When the circular pixel region of the filter is positioned on a straight line, the direction is a direction having a strong correlation. This embodiment cannot be applied to the embodiment described in FIG.

When there is an edge as shown in FIG. 13A, D ₀ = | A−B ₀ | and D ₅ = | A−B ₅ | are both minimum, that is, a circular shape in which the difference from A is minimum. The filters are B ₀ and B ₅ , and their directions from the target pixel T (center pixel area A) coincide (because B ₀ AB ₅ is on a straight line), so the direction is a direction with strong correlation. On the other hand, when there is an edge as shown in FIG. 13B, D ₇ = | A−B ₇ | and D ₈ = | A−B ₈ | are both minimum, but B ₈ AB ₇ is Since it is not in a straight line, it cannot be said to be an oblique correlation direction.

FIG. 14 is a diagram for explaining interpolation direction determination processing in a pixel interpolation direction detection device according to another embodiment of the present invention. FIG. 14A is a colored background with a corner edge at the lower left corner on a white background. FIG. 14 (B) is a diagram showing a circular filter set applied when a pattern is present. FIG. 14B is a graph showing a circular filter set when a colored pattern having a curved line edge that turns to the left is present on a white background. FIG. In the example of FIG. 14 as well, there is no need to consider the surrounding pixel areas in pairs (the positional relationship of the circular pixel areas in the circular filter is not fixed linearly with BAC), so that the area B ₀ is the same as in FIG. To B ₉ (corresponding to directions 0, 3, 5, 7, 10, 0, 3, 5, _{7, and 10} respectively). Further, the embodiment described here cannot be applied to the embodiment described in FIG.

In the present embodiment, the interpolation direction determination means 16 calculates the difference between the total for the central pixel area A calculated by the total calculation means 15 and the total for each of the surrounding pixel areas B _{0 to} B ₉ as in the embodiment described in FIG. The absolute value D _n = | A−B _n | (n = 0 to 9) is calculated, and the absolute value D _n of the calculated difference is the second smallest next to the minimum value and the minimum value. And 2 means for selecting a minimum value.

However, the interpolation direction determination means 16 in this embodiment does not consider the coincidence between the direction having the selected minimum value and the direction having the second minimum value, and the circular pixel region of the circular filter having the minimum value The direction connecting the target pixel T and the direction connecting the target pixel T and the circular pixel region of the circular filter having the second minimum value are determined to be directions having a strong correlation, and are determined as the interpolation directions. That is, the interpolation direction determined in the present embodiment is generally a square shape. Note that the direction connecting the target pixel T and the circular filter refers to one or a plurality of pixels within the range of the circular pixel region in the target pixel T and the circular filter (preferably a pixel close to the center of the range or the range thereof) The pixel closest to the pixel of interest T, more preferably the actual pixel). Therefore, when the target pixel T is interpolated with the pixels in the interpolation direction obtained by this determination, the pixels in the circular pixel region in the circular filter having the minimum value and the circular shape in the circular filter having the second minimum value are used. The interpolated value of the target pixel T is obtained from the pixels in the pixel area. Thus, in the present _{embodiment, D n = | A n -B} n | calculates (here B filter and said) Small circular filter minimizes and second D _n Locate and select, An interpolation value is created from the pixels on the two selected B filters.

When there is a corner edge as shown in FIG. 14A, it is considered that the difference from A is that B ₇ is the smallest and then B ₉ is the smallest, and therefore A (target pixel T) the the AB ₇ direction and AB ₉ and the interpolation direction as a starting point. Therefore, there is no problem even if there is no corner pattern determination in the pixel feature determination means 11. When this interpolation direction is adopted as it is, an interpolation value is generated from the pixels on B ₇ and B ₉ . In addition, when there is a black curve on a white background as shown in FIG. 14B, | A−B ₂ | and | A−B ₈ | are considered to be small, so that AB ₂ starts from A (target pixel T). Direction and AB ₈ are taken as interpolation directions. As if the interpolation direction is employed, and to generate an interpolated value from the pixel on the B ₂ and B _8.

[Other pixel area specification / pixel information total calculation processing (oval filter set)]
FIG. 15 is a diagram for explaining an example of a pixel area specified in a pixel interpolation direction detection apparatus according to another embodiment of the present invention, and FIG. 15A is an example of a direction representing the specified area. FIG. 15B is a diagram showing an example of a pixel region corresponding to one of the directions.

  In each of the above-described embodiments, the central pixel region A is set as one, and a plurality of peripheral pixel regions are specified, so that each region has a directionality, and the central pixel region A alone has no directionality. However, as another embodiment of the present invention, the pixel area specifying unit 14 may specify the central pixel area so that the central pixel area is directed in a plurality of directions. The embodiment described here is an alternative to the detailed modes described in FIGS. 10 to 14 and cannot be applied to these modes.

  In the present embodiment, the pixel area specifying unit 14 uses, as a central pixel area, an elongated pixel area having a width corresponding to a predetermined number of pixels centered on the target pixel T, and a plurality of peripheral pixel areas parallel to the central pixel area ( A pair of elongate pixel areas (parallel to the longitudinal direction) are specified as one set, and a plurality of sets of pixel areas obtained by rotating the specified set of pixel areas around the target pixel T are also specified. Here, the specification of a plurality of peripheral pixel regions with respect to the central pixel region is preferably a pair of peripheral pixel regions that are in contact with each other in parallel and have the same number of pixels (the number of pixels after weighting when weighting as described above). It is preferable to specify. In particular, for the plurality of peripheral pixel regions, a pair of pixel regions sandwiching the central pixel region is sufficient.

Referring to FIG. 15, the center pixel area A ₀ is specified, the peripheral pixel areas B ₀ and C ₀ parallel to the center pixel area A ₀ are specified, and this set is further rotated in directions 1 to 11 in FIG. A plurality of sets of pixel regions, that is, sets of (center pixel region A ₁ , peripheral pixel regions B ₁ , C ₁ ),..., (Center pixel region A ₁₁ , peripheral pixel regions B ₁₁ , C ₁₁ ) Identify the set. FIG. 15B illustrates a plurality of sets of pixel areas in the third direction, that is, a set of a central pixel area A ₃ and peripheral pixel areas B ₃ and C ₃ . The filter in which the pixel region is specified in this embodiment and the sum of the pixel information is calculated is an oval filter arranged in parallel and is often referred to as an “oval filter set”. When the oval filter set is used, since the center pixel region including the target pixel T faces in a plurality of directions, the accuracy of detecting the interpolation direction, and hence the accuracy of interpolation can be improved. Each direction of the oval filter set illustrated in FIG. 15 and the like is the same as the direction illustrated in the circular filter set.

Incidentally, in response to the "elliptical filter set" refers to the individual areas (e.g., areas B ₁ and area A, etc.) and oval pixel region, further, the specific pixel area specifying means of the oblong pixel region 14 and the total calculation means 15 that calculates the total value by multiplying the actual pixel information by a coefficient are collectively referred to as an “oval filter”, and further, three continuous regions (oval pixels) on a straight line. Area) is referred to as an “oval-shaped pixel area set”. However, in these terms, “oval shape” is used, but it includes not only an oval shape but also a track shape. Of course, the shape is not limited to an oval shape. Although it is an oval shape, it may actually be rattled, or the area may be specified in any shape such as “rectangular” (however, one axis is longer than the other axis). Needless to say.

Here, the rotation may be rotation at a predetermined angle. However, as in the above example, the direction close to the horizontal direction is finely defined and the direction close to the vertical direction is roughly defined and detected. It may be possible to detect the interpolation direction that has been difficult. Further, in the case of rotating at every predetermined angle, for example, the direction of about every 15 degrees around the target pixel T is set to the direction of No. 0 to No. 11, etc., but there is a surplus in determining the interpolation direction. A direction that does not contribute may not be specified. In addition, although the set specified first is exemplified as the set of (center pixel area A ₀ , peripheral pixel area B ₀ , C ₀ ), it is needless to mention that this is not the case. In addition, each area by rotation may be specified as an area having the same width, but the oval filter set values must be compared, and the weighting as described above becomes complicated. Accordingly, it is preferable to keep the number of pixels included in the region (the number of pixels after weighting in the case of weighting as described above) constant.

  FIGS. 16 and 17 are diagrams for explaining examples of pixel areas specified in the pixel interpolation direction detection device according to another embodiment of the present invention. The example of the weighting coefficient of the center pixel area in each direction in a form in which only the actual pixels are calculated without the need to insert an interpolation value is also shown. 16 and FIG. 17, the same reference numerals are given to the same elements as those in FIG. 2 and FIG.

  FIGS. 16A to 16D and FIGS. 17A to 17D are oval shapes in the directions of Nos. 1, 3, 4, 5, 9, 9, 7, 11, respectively. Fig. 4 illustrates a filter set. Here, the 11th direction is the horizontal direction, the 5th direction is the vertical direction, and the 10th direction, 9th direction, 8th direction, 7th direction, and 6th direction are the 0th direction, 1st direction, The second direction, the third direction, and the fourth direction are symmetric with respect to the vertical line passing through the target pixel T. Moreover, the difference between the direction of No. 11, No. 1, No. 1, No. 3, No. 7, No. 9, and No. 9, No. 11 is small in the No. 0 direction, No. 2, No. 8, and No. 10, directions, respectively. Therefore, it is not taken in as an interpolation direction to be detected and is not shown in the figure, but as described above, it may be incorporated as an interpolation direction in which the 0th, 2nd, 8th and 10th directions are also detected.

  When an ellipse filter in the 11th direction is used, the horizontal direction (straight side line / edge) can be detected. Since it is difficult to detect the horizontal direction (straight line / edge) by the pixel feature determination unit 11 in the previous stage, the horizontal direction (the 11th direction) may be incorporated as an interpolation direction to be detected, but is the same as other oblique directions. If an attempt is made to detect the eleventh direction (in this case, it is conceivable to interpolate with the average of the upper and lower pixels in the end), a smooth line or edge close to the horizontal is determined as the eleventh direction, and the final result is Interpolation may be performed by averaging the upper and lower pixels, and a gentle line may rattle. Therefore, it is preferable not to use the ellipse-shaped filter in the 11th direction and to prevent the pixel interpolation direction detection means 12 from executing the detection in the horizontal direction. In addition, when the ellipse filter in the 5th direction is used, the vertical direction (true vertical line / edge) can be detected. However, the pixel feature determination unit 11 in the previous stage detects the vertical line. Thus, the ellipse filter in the fifth direction may not be used.

16 and 17 show an example of the positional relationship between the central pixel area A and the peripheral pixel areas B and C in each direction. In this example, the peripheral pixel areas B and C are used as the central pixel area A. Are arranged in a substantially normal direction in the longitudinal direction. 16 and 17, the oval pixel regions A ₁ , B ₁ , C ₁ , A ₉ , B ₉ , and C ₉ in the 1st and 9th directions are shown for each oval pixel region. In the example, the pixel E and the interpolation pixel N are also 37 pixels in total (the total number of pixels is 20 actual pixels E). Similarly, for the 3rd and 7th directions, an example in which the actual pixel E and the interpolation pixel N are also 29 pixels in total (the total number of pixels is 16 actual pixels E) is 4th and 6th. For the direction, an example in which the real pixel E and the interpolation pixel N are also 23 pixels in total (the total number of pixels is 14 of the real pixels E). For the fifth direction, the real pixel E and the interpolation pixel N are interpolated. An example in which the pixel N is also 21 pixels in total (the total number of pixels is 12 actual pixels E). For the 11th direction, the real pixel E and the interpolation pixel N are also 21 pixels in total (summed up) An example in which the number of pixels is 14 actual pixels E) is shown.

  In the central pixel area A in FIG. 16A, an example in which a weighting coefficient is assigned to each original pixel E so that the actual pixel E closer to the first direction of the area and closer to the central pixel T has a larger weighting coefficient. Show. Similarly, FIG. 16B is the third direction, FIG. 16C is the fourth direction, FIG. 16D is the fifth direction, FIG. 17A is the ninth direction, and FIG. FIG. 17C shows an example of the weighting coefficient corresponding to the center pixel region in the number direction, FIG. 17C shows the number 6 direction, and FIG. 17D shows the number 11 direction. 16 and 17 do not illustrate the coefficient values of the oval filter corresponding to the peripheral pixel regions B and C in the oval filter set, but the oval filter corresponding to the center pixel region A is not illustrated. The same coefficient value as in the above may be set appropriately.

  18 and 19 are diagrams for explaining an example of a pixel region specified in a pixel interpolation direction detection device according to another embodiment of the present invention, and the total number of interpolation pixels in which a temporary interpolation value is inserted in advance is calculated. The example of the weighting coefficient of the center pixel area | region in each direction in the form to be used is also shown. 18 and 19, the same elements as those in FIGS. 2 and 4 are denoted by the same reference numerals, and the description thereof is omitted.

  FIG. 18 and FIG. 19 show examples of coefficients that can obtain filter results equivalent to those in FIG. 16 and FIG. 17 by inserting temporary interpolation values in advance and applying coefficients not only to the actual pixel E but also to the interpolation pixel H. . Note that the temporary interpolation value here is the upper and lower pixel average value.

18 (A) to (D) and FIGS. 19 (A) to (D) are oval shapes in the directions of No. 1, No. 3, No. 4, No. 5, No. 9, No. 7, No. 6, and No. 11, respectively. Fig. 4 illustrates a filter set. Here, the 1st to 11th directions are the same as the directions illustrated in FIGS. 16 and 17. 18 and 19 show an example of the positional relationship between the central pixel area A and the peripheral pixel areas B and C in each direction. In this example, as in the examples of FIGS. 16 and 17, The peripheral pixel regions B and C are arranged in a substantially normal direction in the longitudinal direction of the central pixel region A. 18 and 19, the oval pixel regions A ₁ , B ₁ , C ₁ , A ₄ , B ₄ , C ₄ , A ₆ , B ₆ , in the first, fourth, sixth, and ninth directions are shown. An example is shown in which C ₆ , A ₉ , B ₉ , and C ₉ are 23 pixels (23 pixels in total) including the actual pixel E and the interpolation pixel H for each oval pixel region. Yes. In addition, the oval pixel regions A ₃ , B ₃ , C ₃ , A ₇ , B ₇ , and C ₇ in the third and seventh directions are divided into an actual pixel E and an interpolation pixel H with respect to the respective oval pixel regions. In addition, an example in which 25 pixels are combined (the total number of pixels is 25 pixels) is also shown. Further, the area in the 5th and 11th directions is 21 pixels in total including the actual pixel E and the interpolation pixel H (the total number of pixels is also 21 pixels).

  Further, in the central pixel area A in FIG. 18A, a weighting coefficient is applied to each original pixel E and interpolation pixel H so that the actual pixel E closer to the first direction of the area and closer to the central pixel T has a larger weighting coefficient. An example of assignment is shown. Similarly, FIG. 18B is the third direction, FIG. 18C is the fourth direction, FIG. 18D is the fifth direction, FIG. 19A is the ninth direction, and FIG. 19C shows an example of the weighting coefficient corresponding to the center pixel region in the 6th direction and FIG. 19D shows the center pixel region in the 11th direction. 18 and 19 do not illustrate the coefficient values of the oval filter corresponding to the peripheral pixel regions B and C in the oval filter set, but the oval filter corresponding to the center pixel region A is not illustrated. The same coefficient value as in the above may be set appropriately.

  20 and 21 are diagrams for explaining another example of the pixel region specified in the pixel interpolation direction detection device according to another embodiment of the present invention, and the total number of interpolation pixels in which provisional interpolation values have been previously inserted is shown. The other example of the weighting coefficient of the center pixel area | region in each direction in the form used for calculation is shown collectively. 20 and 21, the same elements as those in FIGS. 2 and 4 are denoted by the same reference numerals, and the description thereof is omitted.

  The elliptical pixel region and its weighting coefficient exemplified in FIGS. 20 and 21 are the same as those illustrated in FIGS. 18 and 19 in the central pixel region A, but specific pixel regions B and C are specified. There is a difference in manner, and the overlap between the central pixel area A and the peripheral pixel areas B and C is increased. As shown in FIGS. 18 and 19, it is ideal that the peripheral pixel regions B and C are arranged in a substantially normal direction in the longitudinal direction of the central pixel region A, but a wide pixel region is secured as shown in the figure. Therefore, there is a concern that the circuit becomes large in scale, and the purpose of detecting the interpolation direction even in a limited pixel region is the oval shape illustrated in FIGS. 20 and 21. This is an arrangement example of filters A, B, and C. As shown in this example, the peripheral pixel regions B and C are arranged so as not to surround the center pixel T even for the oval filter. 20A and 20B supplementarily show a straight line in the direction indicated by the central pixel area A and a straight line perpendicular thereto, and the longitudinal direction, the central pixel area A, the peripheral pixel area B, Although the arrangement direction of C is not strictly orthogonal, a correlated directionality can be sufficiently detected even with such an arrangement.

[Other interpolation direction determination processing (supporting oval filter set)]
Here, the process of detecting the diagonal correlation direction from the relationship of the oval pixel region set in the oval filter set will be described.

  FIG. 22 is a diagram for explaining the interpolation direction determination processing in the pixel interpolation direction detection device according to another embodiment of the present invention, in which there is a black line (color line) pattern that faces diagonally right upward on a white background. It is a figure which shows what applied the elliptical filter set in the case. 22 (A), (B), (C), (D), and (E) are respectively applied with the 1st direction filter, the 3rd direction filter, the 5th direction filter, the 9th direction filter, and the 7th direction filter. Is shown. In FIG. 22, the pattern is hatched, and the same reference numerals are given to the same elements as those in FIGS. 2 and 4 and the description thereof is omitted.

  The determination of the interpolation direction for the oval filter set also detects the direction with the strongest correlation from the sum of the pixel information of the central pixel region and the sum of the pixel information of the peripheral pixel region, similar to that for the circular filter set, The direction is determined as the interpolation direction. Here, a preferred embodiment of the determination for the elliptical filter set will be described.

In the present embodiment, the interpolation direction determination unit 16 includes a total (total pixel information) for the central pixel area calculated by the total calculation unit 15 and a total (total pixel information) for each peripheral pixel area in the same set as the central pixel area. ) With respect to each set. 22, | A ₁ −B ₁ |, | A ₁ −C ₁ |, | A ₃ −B ₃ |, | A ₃ −C ₃ |, | A ₄ −B ₄ |, | A ₄ − _{C 4 |, | A 5 -B} 5 |, | A 5 -C 5 |, | A 6 -B 6 |, | A 6 -C 6 |, | A 7 -B 7 |, | A 7 -C 7 |, | A ₉ -B ₉ |, | A ₉ -C ₉ |, | A ₁₁ -B ₁₁ |, | A ₁₁ -C ₁₁ | are calculated, that is, | A _n -B _n |, | A _n −C _n | will be calculated for each set of n = 1, 3, 4, 5, 6, 7, 9, 11. However, in FIG. 22, the illustrations for n = 4, 6, 11 are omitted.

And the interpolation direction determination means 16 in this embodiment shall have a means to add the absolute value of the difference calculated in each set further. In this addition means, similarly as illustrated in FIG. 22, D _n = | A _n −B _n | + | A _n −C _n | is n = 1, 3, 4, 5, 6, 7, 9, 11 respectively. Will be asked for.

As shown in FIG. 22E, when three oval filters arranged in parallel on an oblique line (in the direction of strong correlation) are placed (in parallel), D _n = | A _n −B _n | + | A _n −C _n | (here, D ₇ = | A ₇ −B ₇ | + | A ₇ −C ₇ |) becomes large. From this, D _n = | A _n −B _n | + | A _n −C _n | is calculated for each direction, and the longitudinal direction of the oval pixel region A when the maximum is obtained is the direction of the oblique line. It can be said that there is.

Accordingly, the interpolation direction determining means 16 detects the normal direction of the arrangement direction of the set having the selected maximum value as the direction having the strongest correlation, and selects the maximum value from the added values. And determining means as an interpolation direction. The normal direction of the alignment direction of the set having the maximum value, the regions A _n, B _n, a normal direction of a direction along with C _n, in other words, the longitudinal central pixel regions A _n in the set having the maximum value Direction. In this form, the normal direction of the set arrangement direction of the maximum value of D _n = | A _n −B _n | + | A _n −C _n | is detected as the oblique correlation direction. In other words, D _n is a longitudinal regions A _n of the set showing the maximum value is detected as a diagonal correlation direction. Here, the normal direction may not be strictly a normal line, and refers to the direction of a pixel region that is specified at least in a direction near the direction.

Further, as shown in FIG. 22 (B), oval filter set that parallel aligned elliptical filter three is, when perpendicular to the diagonal line, A _n and B _n, oval filter result of the C _n There is no difference, and D _n (D ₃ here) becomes small. Thus, the set arrangement direction of the minimum value of D _n = | A _n −B _n | + | A _n −C _n | may be detected as a so-called diagonal correlation direction. Therefore, the interpolation direction determination means 16 detects the arrangement direction of the set having the selected minimum value as the direction having the strongest correlation, and selects the detected direction as the interpolation direction. And a means for determining. Note that the arrangement direction of the set having the minimum value, the area A _n, is a direction along with B _n, C _n, in other words, in the short direction (the longitudinal direction of the center pixel regions A _n in the set having a minimum value Orthogonal direction). In the this embodiment, in other words, the D _n the longitudinal direction of the normal direction of a region A _n sets of the minimum value (short direction) is detected as a diagonal correlation direction.

  Further, as another embodiment of the present invention, when the embodiment described based on FIG. 22 (E) and the embodiment described based on FIG. 22 (B) are used together, and those conditions overlap, By detecting the diagonal correlation direction, it becomes possible to detect the interpolation direction more accurately.

The interpolation direction determination means 16 in this embodiment differs in the selection and detection portions of the interpolation direction determination means 16 of each form described above with respect to the elliptical filter set. In the selection of the present embodiment, the minimum value and the maximum value are selected from the added values. Upon detection, the arrangement direction of the set having the selected minimum value and the normal line of the arrangement direction of the set having the maximum value are selected. If the direction matches, the direction is detected as the direction with the strongest correlation. The direction detected here is determined as the interpolation direction. In this form, the oval filter set D _n = | A _n −B _n | + | A _n −C _n | determines the arrangement direction in which the value is the minimum value and the arrangement direction in which the value is the maximum value. When the normal direction matches, the direction is detected as an oblique correlation direction.

[Interpolation direction correction processing]
The interpolation direction for each pixel of interest determined by any of the interpolation direction determination means 16 as described above may be determined as the interpolation direction as it is, but in order to determine the interpolation direction more accurately, the determined interpolation direction Is preferably corrected. This correction is executed by the interpolation direction correction means 17.

  The interpolation direction correcting unit 17 is a unit that corrects the interpolation direction of each pixel of interest determined by the interpolation direction determining unit 16, and this unit corrects the interpolation direction of the pixel of interest (referred to as the pixel of attention Ta). Correction is performed based on the information on the interpolation direction of the target pixel Ta and the information on the interpolation direction of one or more other target pixels located in the vicinity of the target pixel Ta. Here, the other one or a plurality of target pixels located in the vicinity of the target pixel Ta means, for example, two target pixels adjacent to the target pixel Ta in the horizontal direction. A more preferable example of the interpolation direction correcting means 17 will be described later in the embodiment. For example, the direction is numbered as information on the interpolation direction (for example, the first direction, the second direction,. It is preferable to obtain the corrected interpolation direction by averaging the values of the numbers.

<Post-processing (interpolation processing / interpolation check processing)>
[Interpolation processing]
Next, interpolation processing will be mainly described as subsequent processing. The pixel interpolation apparatus described here corresponds to the intra-field interpolation apparatus 3 in FIG. 1, but at least the pixel interpolation direction detection apparatus (pixel interpolation direction detection means 12) according to each embodiment described above with reference to FIGS. 1 to 22. Any one of them is provided, and preferably pixel feature determination means 11 is also provided. Since this pixel interpolation device (intra-field interpolation device 3) generates an interpolation signal, it is also called an interpolation signal generation device, and includes any pixel interpolation means 13 described below.

  The pixel interpolation unit 13 includes at least an interpolation value generation unit (also referred to as an interpolation value calculation unit) 18 and an interpolation unit 20. In the interpolation value calculation means 18, for each pixel of interest, based on the interpolation direction detected or determined by the pixel interpolation direction detection means 12 or the correction after the determination (regardless of the circular filter set or the oval filter set). No), an interpolation value is calculated from actual pixels in the interpolation direction. There are various methods for determining actual pixels in the interpolation direction, examples of which will be described later. In addition, the creation of the interpolation value may be obtained by uniformly calculating the average value of the pixel information of the interpolation source, but particularly when creating the interpolation value using pixel information of three or more actual pixels, You may devise suitably, such as performing weighting. The interpolation unit 20 interpolates each pixel of interest with the interpolation value calculated by the interpolation value calculation unit 18. The interpolation means 20 actually generates a signal after interpolation (interpolation signal) based on the interpolation value of each pixel of interest.

[Determination of actual pixels in the interpolation direction]
FIG. 23 is a schematic diagram for explaining the interpolation processing in the intra-field interpolation apparatus of FIG. 1, and FIGS. 23 (A) to (F), (G) to (K), and (L) each have an interpolation direction. It is a figure for demonstrating an example of the real pixel used for interpolation when it determines with it being the direction of 0th-5th, 10th-6th, and 11th. In FIG. 23, G is an actual pixel of the interpolation source (actual pixel used for interpolation), and other elements equivalent to those in FIG. 2 are denoted by the same reference numerals, and description thereof is omitted.

  The actual pixel in the interpolation direction is determined by predetermining the position of the actual pixel with respect to the target pixel T corresponding to each interpolation direction as the simplest method. For example, as shown in FIG. 23A, when it is determined that the interpolation direction is the 0th direction, the pixel T of interest is 3 left on the left and 3 up on the right, and 3 on the right and 1 on the bottom. One set of real pixels G with the one is determined as the interpolation source pixel. Similarly, as shown in FIG. 23B, when it is determined that the interpolation direction is the first direction, the pixel T of interest is 2 to the left and 2 to the left, and 2 to the right and 1 to the bottom. One set of real pixels G with the pixel is determined as an interpolation source pixel. Further, as shown in FIG. 23C, when it is determined that the interpolation direction is the second direction, the pixel T of interest is separated by 2 to the left and 1 to the left, and to the right by 2 and to the bottom by 1 A set of real pixels G and a set of real pixels G, one left and one up, and one right and one down are determined as interpolation source pixels. . Similarly, when it is determined that the interpolation direction is the third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, or eleventh direction, FIGS. 23 (D), (E), (F), (K), respectively. , (J), (I), (H), (G), and (L), the actual pixel G at the position indicated is determined as an interpolation source pixel. If it is determined that the direction is the 11th direction, there is basically no actual pixel in the 11th direction of the target pixel T. Therefore, as illustrated in FIG. 23L, for example, the upper line Leua of the target pixel T The three actual pixels G in the lower line Leda may be determined as the interpolation source pixels. However, as described above with respect to the interpolation direction to be detected, it is preferable to adopt a form in which detection is not performed for the eleventh direction.

  Further, the interpolation value calculation means 18 is based on the interpolation direction detected by the pixel interpolation direction detection means 12, and the pixel area specified in the interpolation direction (peripheral pixel area (especially in the case of an elliptical filter set). The interpolation value may be calculated from the actual pixel in the center pixel region in many cases)). This form is particularly effective in the case of the interpolation direction determination described with reference to FIG.

[Interpolation check processing]
FIG. 24 is a schematic diagram for explaining an interpolation value check process in the intra-field interpolation apparatus of FIG. In FIG. 24, the same elements as those in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted.

  It is preferable that the pixel interpolation unit 13 further includes an interpolation value determination unit 19 that determines whether or not the interpolation value calculated by the interpolation value calculation unit 18 is appropriate, thereby improving the interpolation accuracy. The interpolation value determination means 19 is also called an interpolation value check means because it checks the interpolation value calculated by the interpolation value calculation means 18.

  The interpolation value determination means 19 includes means for calculating a difference value between the interpolation value calculated by the interpolation value calculation means 18 and the upper and lower pixel average values of the target pixel T. In the example of FIG. 24, the interpolation value calculated by the interpolation value calculation means 18 is an interpolation value (for example, an average value) from a pair of real pixels E at the tip of an oblique arrow drawn from the target pixel T. Point to. Further, the interpolation value determination means 19 has means for determining that the interpolation value is not an appropriate value (correct value) when the calculated difference value is larger than a predetermined threshold value. Then, the interpolation unit 20 interpolates the upper and lower pixel average values of the target pixel instead of the interpolation value calculated by the interpolation value calculation unit 18 with respect to the target pixel determined to be invalid by the interpolation value determination unit 19. Interpolate as a value (replace). Here, instead of the upper and lower pixel average values, the value of the actual pixel above or below the target pixel may be interpolated as an interpolation value.

  Further, the interpolation value determination unit 19 determines that the interpolation value calculated by the interpolation value calculation unit 18 is not between the pixel information of the actual pixel above the target pixel T and the pixel information of the actual pixel below. Means for determining that the interpolation value is not a valid value may be included. Further, the interpolation value determination means 19 calculates the difference value between the interpolation source actual pixels (for example, the actual pixel G in FIG. 23) with respect to the interpolation value calculated by the interpolation value calculation means 18, and the calculation is performed there. Means for determining that the interpolation value is not a valid value when the difference value is greater than a predetermined threshold value. By combining these three forms, a more accurate check can be performed.

<Video signal processing>
Next, the video signal processing apparatus exemplified in FIG. 1 as a whole will be briefly described. This video signal processing apparatus determines and distinguishes a pixel determined to be non-moving (hereinafter referred to as “still pixel”) and a pixel determined to be moving (hereinafter referred to as “moving pixel”). The apparatus includes a prime determination apparatus 1, an intra-frame interpolation apparatus 2 that mainly performs interpolation between a pair of fields, and an intra-frame interpolation apparatus 3 as described above.

  This video signal processing apparatus applies general IP conversion with motion detection to the characteristic part of the present invention, and an interlaced scanning video signal is generated between a still pixel and a moving pixel in the still pixel / moving pixel determination apparatus 1. The interlaced video signal is inter-frame interpolated by the intra-frame interpolating device 2 and the inter-scanned video signal is converted into the moving pixel signal by the intra-field interpolating device. The interlace scanning video signal is converted into a progressive scanning video signal by interpolating at step 3. In this way, inter-field interpolation (= intra-frame interpolation) is performed on non-moving parts such as subtitles and backgrounds, and intra-field interpolation which is a characteristic part of the present invention is performed on moving parts. In addition, the motion detection basically needs to be able to determine a still pixel / moving pixel for each pixel, such as determining whether it is moving or not based on the magnitude of the difference value from the previous field. Note that the still pixel / moving pixel determination device 1 is configured to detect a portion corresponding to a still pixel even in a moving image signal to the intra-frame interpolation device 2 and a moving pixel portion to the intra-field interpolation device 3 based on motion detection. In this case, the output video signal may be generated by being combined when output to an output device or the like.

<Example of intra-field interpolation device>
25 is a block diagram showing a circuit configuration example of the intra-field interpolation apparatus in FIG. 1, in which 31 is a block area synthesis circuit, 32 is a direction determining circuit, 33 and 34 are delay units (D), and 35 is a direction. LPF (directional low-pass filter), 36 is an interpolation value generation circuit, 37 is an interpolation value selection circuit (SEL), 38 is a miss determination circuit, 39 is a horizontal pixel subtractor, 40 is a horizontal complexity determination circuit, and 41 is vertical. A directional pixel subtractor, 42 is a complex determination circuit, 43 is a vertical line determination circuit, 44 is a flat determination circuit, 45 is an angle determination circuit, 46 and 47 are or gates, and 48 is an interpolation value selection circuit (SEL).

  The intra-field interpolation apparatus illustrated in FIG. 25 includes a block area synthesis circuit 31 as an example of the pixel area specifying unit 14 and the total calculation unit 15, a direction determination circuit 32 as an example of the interpolation direction determination unit 16, and an interpolation direction correction unit 17. As an example, the delay units 33 and 34 and the direction LPF 35, the interpolation value calculation circuit (interpolation value generation means) 18 as an example, the interpolation value creation circuit 36, the interpolation value selection circuit 37, and the error as an example of the interpolation value determination means 19 Determination circuit 38, horizontal pixel subtractor 39 as an example of horizontal difference means, horizontal complexity determination circuit 40 as an example of horizontal complexity determination means, and vertical pixel subtractor as an example of vertical difference means 41, complexity determination circuit 42 as an example of complex determination means, vertical line determination circuit 43 as an example of vertical line determination means, and example of flat determination means Flat determination circuit 44, corner determination circuit 45 as an example of a corner determination means, or gate 46 for outputting a signal in the case of any of a complicated pattern, vertical line pattern, flat pattern, or corner pattern, An or gate 47 that outputs a signal when any of the patterns is determined to be erroneous, and an interpolation value selection circuit 48 that finally selects an interpolation value, as an example of a part of the interpolation means 20, are provided.

  FIG. 26 is a flowchart for explaining an example of intra-field interpolation processing in the intra-field interpolation apparatus of FIG. 27 to 36 are schematic diagrams for explaining an example of intra-field interpolation processing in the intra-field interpolation apparatus of FIG. 27 shows an example of a horizontal line complexity determination process, FIG. 28 shows an example of a flat pattern determination process, FIG. 29 shows an example of a vertical line pattern determination process, FIG. 30 shows an example of a corner picture determination process, and FIG. FIG. 32 shows an example of an interpolation direction determination process using a circular filter set, FIG. 33 shows an example of a weighting coefficient of a circular filter, a circular filter result and a circular filter set result, and FIG. 34 shows a direction LPF. For example, FIG. 35 is a diagram for explaining an example of an image determined to be a mistake in the interpolation value check processing example, and FIG. 36 is a diagram for explaining an example of a video signal obtained by this intra-field interpolation. In FIGS. 27 to 32, 34 and 35, the numerical values shown at the pixel positions represent the luminance values of the actual pixels as 0 (black) to 100 (white). Of course, the luminance value corresponds to the processing unit such as a value of 0 to 255 if it is 8 bits.

  This intra-field interpolation processing is also called a pixel unit IP conversion algorithm because it performs IP conversion on a pixel basis. Hereinafter, an example of the intra-field interpolation processing according to the present invention will be described in accordance with the flow of FIG. 26 and with reference to FIGS. 25 and 27 to 36.

  The horizontal pixel subtractor 39 receives the actual pixels in the field of 4 lines × 9 pixels centered on the pixel of interest, and calculates the difference value of the actual pixels in the horizontal direction (step S1). Here, among the actual pixels in the field of 4 lines × 9 pixels, the horizontal difference value (horizontal difference value) between the actual pixels may be calculated using the actual pixels in the field of 2 lines × 9 pixels centered on the target pixel. Further, in step S1, the vertical pixel subtracter 41 receives the actual pixels in the field of 4 lines × 9 pixels centered on the pixel of interest, and calculates the vertical difference value of the actual pixels. Here, among the actual pixels in the field of 4 lines × 9 pixels, the vertical difference value (vertical difference value) between the actual pixels may be calculated using the actual pixels in the field of 4 lines × 7 pixels centered on the target pixel.

  Next, from the horizontal direction difference value between the actual pixels of the horizontal direction pixel subtractor 39, the horizontal direction complexity determination circuit 40 determines the horizontal direction complexity of the upper line and the lower line (here, 9 actual pixels) of the target pixel. Is determined (step S2). FIGS. 27A to 27F show examples of the complexity of horizontal lines, and FIG. 27G schematically shows the actual pixels adjacent to each other in the luminance value difference as schematically shown by the unevenness of the graph. It is determined that the smaller the interval is, the less uneven (not complicated). These determinations can be obtained by the encoding described with reference to FIG. 3B. An example of the tendency will be described with reference to FIG.

  As shown in FIG. 27A, when there is almost no difference in luminance value between adjacent real pixels, it is determined as “uncomplicated 1”. As shown in FIG. 27B, if there is not much difference in luminance value between adjacent real pixels, but it is gradually changing in the same direction in the horizontal direction, it is determined as “uncomplicated 2”. As shown in FIG. 27C, although there is a difference in luminance value between adjacent real pixels, since the line is viewed with a limited number of real pixels, there is a certain amount of one peak or valley in the line. In some cases, it is determined as “complexity V”. As shown in FIG. 27D, there is a difference in luminance value between adjacent actual pixels, and there is one peak or valley in the line with a limited number of actual pixels, and the other changes in the same direction in the horizontal direction. If there is a part that is present, it is determined as “complexity S”. As shown in FIG. 27E, when there is a difference in luminance value between adjacent real pixels and there are two peaks or valleys in the line, it is determined as “complexity M”. As shown in FIG. 27F, when there is a difference in luminance value between adjacent real pixels and the line is uneven, it is determined as “uneven”.

  Next, it is determined whether the periphery of the target pixel (target interpolation point) is complicated, whether it is a flat pattern, a vertical line pattern, or a corner pattern (step S3). In this determination, it is determined whether the pixel of interest should be a complex picture, a flat picture, a vertical line picture, a corner picture, or any other pattern.

  In the flat determination circuit 44, it is determined from the vertical direction difference value of the actual pixel of the vertical direction pixel subtractor 41 whether the periphery of the target interpolation point is a flat image. Here, for the pixel of interest indicated by x in FIG. 28, vertical difference values for five pixels in the horizontal direction including the pixel are used, and as a result, if each difference value is equal to or less than a predetermined threshold, it is determined to be a flat picture. . In the example of FIG. 28, any of the five difference values is 0, and it is determined that the pattern is a flat pattern.

  In the vertical line determination circuit 43, the interpolation point of interest is determined based on the horizontal complexity of the upper line and the lower line determined in the horizontal complexity determination circuit 40 and the vertical difference value of the actual pixel of the vertical pixel subtractor 41. Is a partial image of a vertical line. Here, as shown in FIG. 29A, a vertical difference of three pixels is taken in the horizontal direction around the target pixel, and as shown in FIG. 29B, three vertical lines on the vertical line of the target pixel are taken. As shown in FIG. 29C, the difference between the vertical line of the target pixel from two lines to one line with respect to the vertical line of three pixels in the horizontal direction around the target pixel and 1 of the target pixel is obtained. The vertical difference from the bottom of the line to the bottom of the second line is taken, and whether or not it is a vertical line pattern is determined based on the result and the horizontal complexity of the upper and lower lines. When all of these vertical difference values are within a predetermined threshold and the horizontal complexity of the upper line or the lower line is V or more, it is determined as a vertical line pattern. In the example of FIG. 29, since this condition is met, it is determined as a vertical line pattern.

  In the corner determination circuit 45, it is determined whether the target interpolation point is a partial image of the corner based on the horizontal difference value of the actual pixel of the horizontal pixel subtractor 39 and the vertical difference value of the actual pixel of the vertical pixel subtractor 41. judge. Here, as the horizontal direction difference value, the difference value between the actual pixels separated from the target pixel by three pixels to the left and right with respect to the upper line and the lower line of the target pixel, and the actual pixel spaced by two pixels from the target pixel to the left and right The difference value between them is used. In addition, as a vertical direction difference value, with respect to a vertical line of 7 pixels in the horizontal direction centering on the target pixel, the vertical difference from the second line of the target pixel to the first line and the lower line of the target pixel from the first line to the second line Take the vertical difference to. When these horizontal direction difference values are larger than a predetermined threshold value and all the 14 vertical difference values (upper and lower difference values) of the upper and lower stages are smaller than the predetermined threshold value, it is determined that the pattern is a corner picture. In the example of FIG. 30, the target pixel corresponds to a corner picture.

  The complexity determination circuit 42 determines whether or not the periphery of the interpolation point of interest is complicated based on the horizontal complexity of the upper line and the lower line determined by the horizontal complexity determination circuit 40 (step S3). Here, if any line is more than complexity M, it is determined as a complex picture. FIG. 31A shows a complex picture determination processing example, and FIG. 31B shows a graph for schematically showing it. From this graph, it can be seen that in the example of FIG. 31, the upper and lower lines have a complexity of M or more, and are determined to be complex patterns.

  Then, the results of the complexity determination circuit 42, the vertical line determination circuit 43, the flat determination circuit 44, and the corner determination circuit 45 are input to the or gate 46, and a pre-processing determination result is obtained.

  On the other hand, the actual pixels in the field of 4 lines × 9 pixels are also input to the block area synthesis circuit 31 and the interpolation value creation circuit 36. The block area synthesis circuit 31 applies a circular filter arranged at equal distances around the interpolation value to calculate a block area (circular pixel area set) synthesis value in each direction (step S5). The circular filters used here are shown in FIGS. 32 (A) to (J), respectively, 0th, 1st, 3rd, 4th, 5th, 10th, 9th, 7th, 6th, 11th. A circular filter set rotated in the clockwise direction. In FIG. 32, * is an actual pixel line, and the second and eighth directions are omitted because there is little difference between the first and third directions and the seventh and ninth directions. FIG. 33A illustrates the weighting coefficients used for each circular filter in FIG. 32. In this example, the coefficient array is weighted at the center of the circular filter, but other trends (for example, all are the same) A coefficient array of (coefficient) may be used.

  In step S5, the block area (circular pixel area set) combined value of each direction of the block area combining circuit 31 is received by the direction determining circuit 32, and | A−B | + | A−C | In step S6, the minimum value among them is detected, and the direction No. is detected. Is output.

  FIG. 33B has a pattern as shown in FIG. 32, and each circular filter A, B, C when the weighting coefficient of FIG. 33A is used in the circular filter set shown in FIG. And the sum of absolute differences (D = | A−B | + | A−C |) of the difference between the peripheral filter B and C and the circular filter (center filter) A at the center. . For the center filter A, A = 1 × 100 + 2 × 50 + 1 × 0 + 2 × 50 + 4 × 50 + 2 × 50 + 1 × 0 + 2 × 50 + 1 × 100 = 800. The value of the sum D of absolute differences is, for example, the direction No. 4 (FIG. 32D), B = 1 × 100 + 2 × 100 + 1 × 100 + 2 × 100 + 4 × 100 + 2 × 75 + 1 × 100 + 2 × 100 + 1 × 50 = 1500, similarly C = 1500, and from A = 800, D = 1400. The smallest value among the calculated values D is 0. No. 7 and the seventh direction is determined as the interpolation direction. Therefore, in FIG. 32, it is determined that there is a black line that falls to 45 degrees diagonally to the left, and it can be seen from the color coding of FIG. 32 that this is the preferred interpolation direction.

  Next, in step S7, the direction determined by the direction determination circuit 32 is set to DIR_l, delayed by the delay unit 33, and the output value of the delay unit 33 is set to DIR_c. Further, DIR_c is delayed by the delay unit 34 to be DIR_r. In step S7, when the preparation is completed, the direction values (DIR_l, DIR_c, DIR_r) for three horizontal pixels are input to the direction LPF 35. In the direction LPF 35, the LPF is applied in the direction.

  Processing in the direction LPF 35 will be described. FIG. 34A shows the interpolation direction before correction by the direction LPF 35 (interpolation direction of the target pixel corresponding to the pixels surrounded by the five cells in the figure), and the interpolation value when the interpolation direction is used for interpolation ( (Brightness value) is illustrated. It can be seen that the luminance value of the pixel in the middle square among the five squares in FIG. 34A is lower than that of the left and right pixels, and it can be seen that this interpolation is not correct. Actually, the interpolation direction of these five squares before correction is 3, 3, 10, 0, 0, which is a direction far apart by the middle square, and the luminance value corrected in this direction is the middle three squares. Thus, 59, 44, and 55 are obtained. This depends on the interval (angular pitch) in the direction of the peripheral pixel region, and in the case of the slanted line as shown in the figure, an error occurs in the interpolation direction determination by the interpolation direction determination means (direction determination circuit 32). It is a result.

  In order to correct this erroneous interpolation direction determination, the direction LPF 35 is all passed through the interpolation direction of each pixel of interest. Note that the calculation here is such that the interpolation direction without error is not corrected. FIG. 34B shows the result of calculating the obtained interpolation direction for each horizontal line by three pixels and averaging (horizontal weighting may be performed as appropriate) in the horizontal direction. When the weight is doubled at the center, the pixel of interest in the second cell from the right of the five cells is (3 × 1 + 3 × 2 + 10 × 1) / 4 = 5, and the pixel of interest in the third cell from the right of the five cells is (3 × 1 + 10) × 2 + 0 × 1) / 4 = 6 The pixel of interest in the fourth cell from the right in the fifth cell is (10 × 1 + 0 × 2 + 0 × 1) / 4 = 3, which is the fifth direction, the sixth direction, and the third direction, respectively. As a result, the interpolation direction is corrected correctly. Here, in order to obtain an accurate correction value, the calculation results 0 to 5 are rounded off to the nearest decimal place, and the calculation results 5 to 10 are rounded down to 0.5 or less and larger than 0.5. When rounding up. When interpolation is performed based on these directions, the luminance values of the second, third, and fourth squares from the right of the five squares become 55, 55, and 53, respectively, and the directionality with the uncomfortable feeling can be corrected to the correct result.

  Next, in step S8, the interpolation value creation circuit 36 calculates interpolation values in all directions from the actual pixels in the two lines above and below the interpolation value. For example, what is necessary is just to obtain | require with the average value of the real pixel G of each direction of FIG. Also, the upper and lower pixel average values are included in the calculated interpolation value (step S4). In step S8, when this preparation is completed, the interpolation value selection circuit 37 selects one interpolation value calculated by the interpolation value generation circuit 36 from the result of the direction LPF 35.

  Next, the output result of the interpolation value selection circuit 37 is input to the error determination circuit 38 and compared with the average value of the upper and lower pixels or the upper and lower actual pixels, or based on the difference value between the actual pixels for which the interpolation values are created, the interpolation value selection circuit 37 Correct / incorrect determination of the determined interpolation value is performed (step S9). Then, the correct / incorrect determination result by the error determination circuit 38 and the pre-process determination result of the or gate 46 are passed through the or gate 47 and whether the calculated value up to the interpolation value selection circuit 37 should be an interpolation value, or the upper and lower pixel average values are determined. A determination signal as to whether or not to make an interpolation value is obtained. The output of the interpolation value selection circuit 37 and the upper and lower pixel average values calculated by the interpolation value creation circuit 36 are input to the interpolation value selection circuit 48, and the optimum interpolation value is selected from the output signal of the or gate 47.

  FIGS. 35A and 35B show examples of typical pixel value arrangements that cause errors in the output result of the interpolation value selection circuit 37. FIG. In the examples of FIGS. 35A and 35B, it is difficult to determine the directionality, and it is difficult to determine by vertical line, flat, corner, and complex pattern determination. In the example of FIG. 35A, the interpolation direction is determined as the third direction, and the interpolation value based on that direction is obtained from the actual pixel G of FIG. 23D, and is (100 + 0) / 2 = 50. If this interpolation value is adopted as it is, it will become conspicuous as a miss interpolation like a black dot. In fact, in the example of FIG. 35A, it is ideal to employ a value between 100 and 90 as the interpolation value. Therefore, if the interpolation value (50) calculated by the error determination circuit 38 is too different from the upper and lower pixel average value (95), the upper and lower pixel average value 95 is set as the interpolation value. In the example of FIG. 35B, an example in which an upper and lower pixel average value is previously entered as an interpolation value is shown, but it is ideal that the target pixel has the upper and lower pixel average value 100. However, in this case as well, since the determination is made in the other direction, the upper and lower pixel average values are set as the interpolation values.

  The output value from the interpolation value selection circuit 48 becomes the final interpolation value. FIG. 36A shows an example of an image represented by the video signal obtained by the intra-field interpolation. Of these, the diagonal line portion is appropriately interpolated as shown in the enlarged view of FIG. 36B. (See the comparison with FIG. 37B). At the same time, it can be seen that the arc-shaped edge portion is appropriately interpolated as shown in the enlarged view of FIG. 36C (see comparison with FIG. 37C).

It is a figure which shows the example of 1 structure of the pixel interpolation apparatus provided with the pixel interpolation direction detection apparatus which concerns on one Embodiment of this invention, and the pixel interpolation direction detection apparatus, and a video signal processing apparatus. It is a figure for demonstrating the pixel vicinity of the attention pixel in 1 field. It is a schematic diagram for demonstrating the pixel feature determination means in the intrafield interpolation apparatus of FIG. It is a figure for demonstrating the example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the example of the weighting coefficient at the time of total calculation in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is an example of the weighting coefficient in each direction in the form which does not use an interpolation pixel for total calculation. FIG. It is a figure for demonstrating the example of the weighting coefficient at the time of total calculation in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is an example of the weighting coefficient in each direction in the form which does not use an interpolation pixel for total calculation. FIG. It is a figure for demonstrating the example of the weighting coefficient at the time of total calculation in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and shows the example of the weighting coefficient in each direction in the form which uses an interpolation pixel for total calculation. FIG. It is a figure for demonstrating the example of the pixel area specified in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is a figure for demonstrating the example which makes the specified pixel area not overlap. The figure for demonstrating the example of the weighting coefficient at the time of total calculation in the pixel-interpolation direction detection apparatus which concerns on other embodiment of this invention, and is a figure for demonstrating the other example which makes the specified pixel area not overlap It is. It is a figure for demonstrating the interpolation direction determination process in the pixel interpolation direction detection apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the interpolation direction determination process in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention. It is a figure for demonstrating the interpolation direction determination process in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention. It is a figure for demonstrating the interpolation direction determination process in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention. It is a figure for demonstrating the example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention. It is a figure for demonstrating the example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention. It is a figure for demonstrating the example of the pixel area specified in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is a figure for demonstrating the example of the form which does not use an interpolation pixel for total calculation. It is a figure for demonstrating the example of the pixel area specified in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is a figure for demonstrating the example of the form which does not use an interpolation pixel for total calculation. It is a figure for demonstrating the example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and demonstrates the example of the form which uses the interpolation pixel which inserted the temporary interpolation value previously for total calculation. FIG. It is a figure for demonstrating the example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and demonstrates the example of the form which uses the interpolation pixel which inserted the temporary interpolation value previously for total calculation. FIG. It is a figure for demonstrating the other example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is another form of using the interpolation pixel which inserted the temporary interpolation value previously for a total calculation It is a figure for demonstrating an example. It is a figure for demonstrating the other example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention, and is another form of using the interpolation pixel which inserted the temporary interpolation value previously for a total calculation It is a figure for demonstrating an example. It is a figure for demonstrating the example of the pixel area pinpointed in the pixel interpolation direction detection apparatus which concerns on other embodiment of this invention. It is a schematic diagram for demonstrating the interpolation process in the intrafield interpolation apparatus of FIG. It is a schematic diagram for demonstrating the interpolation value check process in the intrafield interpolation apparatus of FIG. It is a block diagram which shows the circuit structural example of the intra-field interpolation apparatus in FIG. FIG. 26 is a flowchart for explaining an example of intra-field interpolation processing in the intra-field interpolation apparatus of FIG. 25. It is a figure for demonstrating the complexity determination process example of the horizontal direction line in the intra-field interpolation apparatus of FIG. It is a figure for demonstrating the example of a flat pattern determination process in the intra-field interpolation apparatus of FIG. It is a figure for demonstrating the example of a vertical line picture determination process in the intra-field interpolation apparatus of FIG. It is a figure for demonstrating the example of a corner picture determination process in the intra-field interpolation apparatus of FIG. It is a figure for demonstrating the example of a complicated picture determination process in the inter-field interpolation apparatus of FIG. It is a figure for demonstrating the example of an interpolation direction determination process by the circular filter set in the intrafield interpolation apparatus of FIG. It is a figure for demonstrating the example of the weighting coefficient of a circular filter, the circular filter result, and the circular filter set result in the intrafield interpolation apparatus of FIG. FIG. 26 is a diagram for explaining an example of a direction LPF in the intra-field interpolation apparatus of FIG. 25. FIG. 26 is a diagram for describing an example of an image that is determined to be a mistake in an example of an interpolation value check process in the intra-field interpolation apparatus of FIG. 25. It is a figure for demonstrating the example of the video signal obtained by the intrafield interpolation in the intrafield interpolation apparatus of FIG. It is a figure for demonstrating the example of the video signal obtained by the intrafield interpolation by a prior art. It is a figure which shows the example of the typical pixel value of the video signal of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Still pixel / moving pixel determination apparatus, 2 ... Intra-frame interpolation apparatus, 3 ... In-field interpolation apparatus, 11 ... Pixel feature determination means, 12 ... Pixel interpolation direction detection means, 13 ... Pixel interpolation means, 14 ... Pixel area specification Means, 15 ... Total calculation means, 16 ... Interpolation direction determination means, 17 ... Interpolation direction correction means, 18 ... Interpolation value calculation means, 19 ... Interpolation value determination means, 20 ... Interpolation means, 31 ... Block region synthesis circuit, 32 ... Direction determining circuit 33, 34 ... Delay device, 35 ... Direction LPF, 36 ... Interpolation value creation circuit, 37, 48 ... Interpolation value selection circuit, 38 ... Miss judgment circuit, 39 ... Horizontal pixel subtractor, 40 ... Horizontal direction Complexity determination circuit, 41 ... vertical pixel subtractor, 42 ... complexity determination circuit, 43 ... vertical line determination circuit, 44 ... flat determination circuit, 45 ... angle determination circuit, 46, 47 ... or gate.

Claims (32)

A pixel interpolation direction detection device that detects an interpolation direction of a pixel to be interpolated in a field when performing IP conversion for converting an interlaced scanning video signal into a progressive scanning video signal,
For a target pixel that is a pixel to be interpolated, there are a region centered on the target pixel, a central pixel region including at least a plurality of real pixels existing in the current field, and a region not surrounding the target pixel. Pixel area specifying means for specifying a plurality of peripheral pixel areas including at least a plurality of real pixels arranged in a circle at substantially equal distances from the target pixel;
Total calculation means for calculating the sum of pixel information of pixels included in the central pixel area and the plurality of peripheral pixel areas specified by the pixel area specifying means for each pixel area;
Interpolation that detects the direction with the strongest correlation based on the total for each pixel area calculated by the total calculation means and the positional relationship of each pixel area that has been summed, and determines the detected direction as an interpolation direction Direction determination means;
A pixel interpolation direction detecting device comprising:   The total calculation means includes means for weighting pixel information of each pixel in the pixel area for each pixel area of the central pixel area and the plurality of peripheral pixel areas specified by the pixel area specifying means, and weighting The pixel interpolation direction detection apparatus according to claim 1, wherein a total of subsequent pixel information is calculated for each pixel region.   The apparatus further comprises preliminary interpolation means for preliminarily generating an interpolation line by performing predetermined interpolation from an actual pixel existing in the current field, and the pixel area specifying means includes the preliminary interpolation as the central pixel area and a plurality of peripheral pixel areas. 3. The pixel interpolation direction detection apparatus according to claim 1, further comprising pixels on an interpolation line interpolated by the means.   The pixel area specifying unit includes, as the central pixel area, a pixel area including only a plurality of real pixels existing in the current field centered on the target pixel as a plurality of peripheral pixel areas from the target pixel. 3. The pixel interpolation direction detection device according to claim 1, wherein a plurality of peripheral pixel regions including only a plurality of real pixels arranged in a circle at equal distances are specified.   Pixel feature determination means for determining a feature that should be the target pixel from features of surrounding real pixels, and the interpolation direction determination means is a target pixel having a predetermined feature based on a determination result of the pixel feature determination means 5. The pixel interpolation direction detection device according to claim 1, wherein a pixel region is not specified and a total is not calculated and it is determined that there is no interpolation direction. 6.   Pixel feature determination means for determining whether the pixel of interest corresponds to the characteristics of a complex pattern, a vertical line pattern, a flat pattern, or a corner pattern from the characteristics of surrounding real pixels, and the interpolation direction determination unit includes: Based on the determination result of the pixel feature determination means, a pixel area is not specified for a target pixel which is a partial pixel of any of a complex pattern, a vertical line pattern, a flat pattern, and a corner pattern, and the total is also calculated. 5. The pixel interpolation direction detection device according to claim 1, wherein the pixel interpolation direction detection unit determines that there is no interpolation direction without calculating. 6.   The pixel feature determining means is a means for calculating a difference value of pixel information between two real pixels located above and below in the same field of the target pixel to be interpolated, and a position adjacent to the right and left in the same field of the target pixel. Means for calculating a difference value of pixel information between two actual pixels, means for calculating a difference value of pixel information between two actual pixels located several pixels on the upper line of the target pixel, Means for calculating a difference value of pixel information between two real pixels located several pixels apart on the lower line of the target pixel, and the feature of the target pixel is defined as the two real pixels located above and below As a result of the difference between the two actual pixels located on the left and right, as a result of the difference between the two actual pixels located on the upper and lower sides, between the two actual pixels located on the left and right, and two pixels separated by several pixels on the upper and lower lines One or more of the results of the difference between Pixel interpolation direction detecting apparatus according to claim 5 or 6, characterized in that to determine the result.   The interpolation direction determining means includes means for calculating the absolute value of the difference between the total for the central pixel area calculated by the total calculation means and the total for each of the surrounding pixel areas, and is symmetrical with respect to the target pixel. Means for adding the absolute values of the differences related to the neighboring pixel regions positioned, means for selecting the minimum value among the added values, and detecting the direction having the selected minimum value as the direction with the strongest correlation And a means for determining the detected direction as an interpolation direction. 8. The pixel interpolation direction detection apparatus according to claim 1, further comprising:   The interpolation direction determining means includes means for calculating the absolute value of the difference between the total for the central pixel area calculated by the total calculation means and the total for each of the surrounding pixel areas, and is symmetrical with respect to the target pixel. A means for adding the absolute values of the differences related to the neighboring pixel regions located; a means for selecting the maximum value among the added values; and a normal direction of the direction having the selected maximum value. The pixel interpolation direction detection device according to claim 1, further comprising: a unit that detects a strong direction and determines the detected direction as an interpolation direction.   The interpolation direction determining means includes means for calculating the absolute value of the difference between the total for the central pixel area calculated by the total calculation means and the total for each of the surrounding pixel areas, and is symmetrical with respect to the target pixel. Means for adding the absolute values of the differences related to the neighboring peripheral pixel regions; means for selecting a minimum value and a maximum value among the added values; and a direction and a maximum value having the selected minimum value 8. The apparatus according to claim 1, further comprising: a unit that detects the direction as the direction having the strongest correlation when the normal direction of the direction matches, and determines the detected direction as the interpolation direction. The pixel interpolation direction detecting device according to claim 1.   The means for calculating in the interpolation direction determining means, in addition to calculating the absolute value of the difference between the total for the central pixel area and the total for each peripheral pixel area calculated by the total calculation means, The sum of differences between neighboring pixel areas located symmetrically with respect to a point is calculated and the absolute value thereof is calculated, and the adding means in the interpolation direction determining means relates to neighboring pixel areas located symmetrically with respect to the target pixel. 11. The pixel interpolation direction detection device according to claim 8, wherein in addition to the absolute value of the difference, an absolute value of a difference between the calculated peripheral pixel regions is further added.   The means for adding in the interpolation direction determining means adds the absolute value of the difference between the absolute values in addition to the absolute value of the difference related to the peripheral pixel regions positioned symmetrically with respect to the target pixel. The pixel interpolation direction detection device according to claim 8, wherein:   The interpolation direction determination means includes interpolation direction redetermination means for determining whether or not the determined interpolation direction is valid, and the interpolation direction redetermination means determines the absolute value of the difference calculated by the calculation means. 13. The method according to claim 8, wherein the interpolation direction is determined to be valid only when a peripheral pixel region having a minimum value substantially matches the determined interpolation direction. Pixel interpolation direction detection device.   The interpolation direction determining means includes means for calculating the absolute value of the difference between the total for the central pixel area calculated by the total calculation means and the total for each peripheral pixel area, and the absolute value of the calculated difference. If the direction having the minimum value and the direction having the second minimum value coincide with each other, the means for selecting the minimum value and the second minimum value next to the minimum value are correlated. The pixel interpolation direction detection device according to claim 1, further comprising: a unit that detects the detected direction as the strongest direction and determines the detected direction as an interpolation direction.   15. The pixel area specifying unit specifies the plurality of peripheral pixel areas so as to include all pixels adjacent to the periphery of the central pixel area in any one of the peripheral pixel areas. The pixel interpolation direction detection device according to any one of the above.   16. The pixel area specifying unit, as the plurality of peripheral pixel areas, specifies a plurality of pairs by pairing two peripheral pixel areas positioned symmetrically with respect to the target pixel. 2. The pixel interpolation direction detection device according to item 1.   The pixel area specifying means includes a pair of elongated pixel areas having a width corresponding to a predetermined number of pixels centered on the target pixel as the central pixel area, and parallel to the central pixel area as the plurality of peripheral pixel areas. The long and narrow pixel regions are specified as a set, and a plurality of sets of pixel regions obtained by rotating the specified set of pixel regions around the target pixel are also specified. 8. The pixel interpolation direction detection device according to any one of items 1 to 7.   The interpolation direction determination unit is configured to calculate an absolute value of a difference between the total for the central pixel region calculated by the total calculation unit and the total for each peripheral pixel region in the same set as the central pixel region for each set. A means for calculating, a means for adding absolute values of differences calculated in each set, a means for selecting a minimum value among the added values, and an arrangement direction of the set having the selected minimum value are correlated. The pixel interpolation direction detection device according to claim 17, further comprising: a unit that detects the detected direction as the strongest direction and determines the detected direction as an interpolation direction.   The interpolation direction determination unit is configured to calculate an absolute value of a difference between the total for the central pixel region calculated by the total calculation unit and the total for each peripheral pixel region in the same set as the central pixel region for each set. Means for calculating, means for adding absolute values of differences calculated in each set, means for selecting a maximum value among the added values, and a method for arranging the set having the selected maximum value 18. The pixel interpolation direction detection apparatus according to claim 17, further comprising: a unit that detects a line direction as a direction having the strongest correlation and determines the detected direction as an interpolation direction.   The interpolation direction determination unit is configured to calculate an absolute value of a difference between the total for the central pixel region calculated by the total calculation unit and the total for each peripheral pixel region in the same set as the central pixel region for each set. Means for calculating, means for adding absolute values of differences calculated in each set, means for selecting a minimum value and a maximum value among the added values, and an array of sets having the selected minimum value When the direction and the normal direction of the arrangement direction of the set having the maximum value match, the direction is detected as the direction with the strongest correlation, and the detected direction is determined as the interpolation direction. The pixel interpolation direction detection device according to claim 17.   21. The pixel area specifying unit specifies any or all of the plurality of peripheral pixel areas so as not to overlap with the central pixel area and some pixels. 2. The pixel interpolation direction detection device according to item 1.   The pixel area specifying unit specifies any or all of the plurality of peripheral pixel areas so as to overlap with the central pixel area and a part of the pixels. The pixel interpolation direction detection device according to the item.   23. The pixel interpolation according to claim 1, wherein the pixel area specifying unit specifies the plurality of peripheral pixel areas as pixel areas having the same number of pixels as the central pixel area. Direction detection device.   Interpolation direction correction means for correcting the interpolation direction of each target pixel determined by the interpolation direction determination means, the interpolation direction correction means, when correcting the interpolation direction of the target pixel, The pixel interpolation direction detection according to any one of claims 1 to 23, wherein correction is performed based on information on an interpolation direction of one or more other target pixels located in the vicinity of the target pixel. apparatus.   The pixel interpolation direction detection device according to any one of claims 1 to 24, and for each pixel of interest, based on an interpolation direction detected by the pixel interpolation direction detection device, from an actual pixel in the interpolation direction. A pixel interpolation device comprising: an interpolation value calculation means for calculating an interpolation value; and an interpolation means for interpolating each pixel of interest with the interpolation value calculated by the interpolation value calculation means.   The interpolation value calculating means calculates an interpolation value from an actual pixel in the pixel area specified by the pixel area specifying means on the interpolation direction based on the interpolation direction detected by the pixel interpolation direction detecting device. 26. The pixel interpolating apparatus according to claim 25.   The pixel interpolation direction detection device according to any one of claims 5 to 7, and for each pixel of interest, based on an interpolation direction detected by the pixel interpolation direction detection device, from an actual pixel in the interpolation direction An interpolation value is calculated, and when it is determined that there is no interpolation direction, an interpolation value calculation means for calculating an interpolation value from a predetermined real pixel of the interpolation source, and an interpolation value calculated by the interpolation value calculation means An interpolating means for interpolating each pixel of interest, and a pixel interpolating apparatus comprising:   Interpolation value determination means for determining whether or not the interpolation value calculated by the interpolation value calculation means is valid, the interpolation means for the target pixel determined to be invalid by the interpolation value determination means The interpolation value is not the interpolation value calculated by the interpolation value calculation means, but the upper and lower pixel average values of the target pixel, the upper actual pixel value, or the lower actual pixel value are interpolated as interpolation values. Item 28. The pixel interpolation device according to any one of Items 25 to 27.   The interpolation value determining means is a means for calculating a difference value between the interpolation value calculated by the interpolation value calculating means and an upper and lower pixel average value of the target pixel, and the calculated difference value is larger than a predetermined threshold value. 29. The pixel interpolation apparatus according to claim 28, further comprising means for determining that the interpolation value is not a valid value.   The interpolated value determining means, when the interpolated value calculated by the interpolated value calculating means is not between the pixel information of the real pixel above the pixel of interest and the pixel information of the real pixel below, 30. The pixel interpolating apparatus according to claim 28, further comprising means for determining that the value is not a valid value.   The interpolation value determination means is a means for calculating a difference value between real pixels as an interpolation source with respect to the interpolation value calculated by the interpolation value calculation means, and when the calculated difference value is larger than a predetermined threshold value, 31. The pixel interpolation apparatus according to claim 28, further comprising: a unit that determines that the interpolation value is not a valid value.   32. A pixel interpolating device according to claim 25, comprising: inter-frame interpolation of a signal of a pixel determined to be non-moving in an interlaced scanning video signal; A video signal processing apparatus comprising: interpolating a pixel signal determined to be moving by the pixel interpolation device to convert an interlaced scanning video signal into a progressive scanning video signal.

Images