JP2008271085A - Image scaling apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem wherein technical skills for avoiding out-of-focus (bluntness) state and an indent of an edge is hard to be realized due to an excessive calculation volume, because enlarged size of an image is apt to generate the out-of-focus (bluntness) state and the indent of the edge by a conventional linear interpolation method. <P>SOLUTION: Results of edge detections in the horizontal direction and in the vertical direction are compared. With regard to a pixel decided as having continuity, a smooth enlarged image is obtained as for an oblique line which causes jagginess by an enlarging processing by a linear interpolation conducted, by filtering the continual mutual pixels. By varying the number of a filter tap according to ratios of frequency components constituting the numbers of pixel and the images which are continual, flexibility of selecting filter characteristics is increased and optimum filter characteristics are obtained. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video signal enlargement / scaling process that is performed on a digital display display in order to enlarge and display a digital video signal having a sampling number smaller than the number of display pixels to the number of display pixels.

  Conventionally, as a digital video scaling method or technique, a linear interpolation method, a bicubic method, or the like is generally used as a method for inter-pixel interpolation. In these methods, pixels that have been uniformly processed by the interpolation filter from surrounding pixels are added between the original pixels. Since these are relatively small in processing amount, they are suitable for real-time image processing of moving images. However, there are features that an enlarged image is blurred and oblique lines appear jagged.

Also, image enlargement methods based on DCT coefficients, wavelet coefficients, and Laplacian pyramids that perform inter-pixel interpolation by predicting high-frequency components before sampling of a sampled video signal have been proposed. Since there are many, it can be said that it is not suitable for a real-time image processing apparatus for moving images.
Japanese Patent No. 3815514 JP 2005-135410 A

  In the above prior art, in order to perform real-time processing of a moving image, blur (edge blunting) and jaggedness (step) of the enlarged image after processing become problems.

  Therefore, in the present invention, the continuity of the edge portion of the image is detected, and the jaggedness (step) of the enlarged image is reduced by performing filtering between pixels having continuous high correlation. In addition, by switching the number of taps and coefficients of the filter according to the number of continuous pixels having high correlation, it is possible to reduce blurring of the processed image.

  In order to solve the above problems, the video scaling device of the present invention includes a frequency detection device that separates a specific frequency component having a luminance level difference larger than a threshold value in the horizontal and vertical directions from an image luminance signal before enlargement. A pixel comparison device that detects the level of correlation between the edge components of pixels and the horizontal, vertical, and diagonal directions of surrounding pixels, and a filter device that creates interpolated pixels for image enlargement with respect to digital video luminance signals. By detecting the edge part of the direction and comparing the height of the correlation between the detected pixel and its surrounding pixels, it is possible to create a pixel that is interpolated from the more highly correlated pixels when enlarging the image. .

  Also, the present invention selects an enlargement method suitable for the frequency characteristics of the pre-enlargement image by switching the filter characteristics (the number of filter taps and the filter coefficient) used for the enlargement filter from the total result of the frequency components of one image. be able to.

  In addition, the present invention switches the filter characteristics (the number of filter taps and the filter coefficient) used for the enlargement filter according to the number of continuous pixels of an edge in one image, thereby corresponding to the partial frequency characteristics of the image and having high correlation. It is possible to select only and to create a pixel to be interpolated at the time of image enlargement.

  As described above, according to the video scaling device of the present invention, the jaggedness (step) of the edge portion of the video that becomes conspicuous as the enlargement ratio increases is detected, and the highly correlated pixels are filtered by detecting continuous edge components. By performing the enlargement process, the blur (edge blunting) generated by enlarging the edge part is adaptively changed according to the frequency constituting the image of one screen, If there are many low frequency components, a multi-tap filter can be used to select a filter having characteristics that do not easily cause edge dullness. For this reason, on a screen with many high frequency components, the number of filter taps is reduced, thereby making it difficult for a non-correlated pixel to be filtered.

  Further, instead of adaptively changing the number of filter taps according to the frequency constituting the one screen, by changing the number of filter taps according to the number of pixels having high correlation for each pixel, non-correlated pixels It makes it difficult to filter each other.

  Next, an embodiment of a video scaling apparatus according to the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
Embodiment 1 of the present invention will be described below with reference to FIGS. FIG. 1 is a block diagram of an edge direction detection apparatus and method according to Embodiment 1 of the present invention.

  In FIG. 1, reference numeral 1 denotes a horizontal edge component extraction horizontal filter that extracts edge components from an input luminance signal using a horizontal filter, differentiation circuit, and the like. Reference numeral 2 denotes a vertical filter, differentiation circuit, etc., from an input luminance signal. Vertical edge component extraction vertical filter for extracting vertical edge components, 3 is a horizontal edge patterning device that assigns a pattern number corresponding to the luminance level to a horizontal edge component detected in 1, 4 is a vertical direction detected in 2 This is a vertical edge patterning device that assigns pattern numbers corresponding to luminance levels to the edge components.

  Note that the horizontal edge component extraction horizontal filter 1, vertical edge component extraction vertical filter 2, horizontal edge patterning device 3, and vertical edge patterning device 4 described above may perform processing for each pixel, It is also possible to perform processing for each pixel.

  Further, reference numeral 5 denotes an interpolation pixel that performs interpolation processing by linear interpolation on a pixel to be interpolated based on the pattern number assigned by the horizontal direction edge patterning device 3 and the vertical direction edge patterning device 4. 6 is an edge information interpolating device, and 6 is a pixel other than adjacent pixels in the horizontal direction or the vertical direction based on the pattern number of the interpolation pixel assigned by the edge number expansion (edge information interpolation) to the interpolation pixel performed in 5. This is a peripheral target pixel investigation device that examines the pattern numbers of the peripheral pixels of pixels having a pattern number that may continue in the diagonal direction, and detects the direction having a highly relevant pattern number.

  In addition, the horizontal edge component extraction horizontal filter 1, the vertical edge component extraction vertical filter 2, the horizontal edge patterning device 3, the vertical direction edge patterning device 4, the edge information interpolation device 5, and the surrounding target pixel investigation device described above. In the processing of No. 6, the pattern number is assigned for each horizontal direction and the vertical direction, but the present invention is not limited to this, and the pattern number is assigned for each predetermined vertical direction and horizontal direction. It is also possible.

  Reference numeral 7 denotes a horizontal enlargement / vertical enlargement interpolation position generation apparatus that inputs a horizontal synchronization signal and a vertical synchronization signal and determines an interpolation pixel insertion position for image enlargement based on the synchronization signal, and 8 is an interpolation that enlarges in the horizontal direction. This is a horizontal filter device that creates and inserts an interpolation pixel into a pixel by filtering with a pixel adjacent in the horizontal direction.

  Reference numeral 9 denotes a vertical filter device that creates and inserts an interpolation pixel by filtering with an adjacent pixel in the vertical direction into the interpolation pixel expanded in the vertical direction, and 10 denotes an interpolation pixel by the filter constituent pixel selected in 7. An oblique direction filter device that is created and inserted, and 11 is a selector device that selects pixels output from the horizontal enlargement filter 8, the vertical enlargement filter 9, and the oblique direction enlargement filter 10 and outputs them as interpolation pixels.

  The operation of the video scaling device having such a configuration will be described below with reference to FIGS.

  First, only an arbitrary frequency component is extracted from the luminance signal input to one horizontal edge component extraction horizontal filter device by a horizontal filter. The filter characteristics of this horizontal edge component extraction horizontal filter device can be set to an arbitrary value, but in view of the characteristics of the ten diagonally expanding filters, the filter characteristics are set to pass the processable frequency band.

  The luminance signal that has passed through the filter described above passes through a differentiator using an arbitrary number of pixels in the horizontal direction, and an edge component is extracted. An arbitrary threshold value may be further provided for the edge component, and the case where the difference value between the minimum luminance level of the edge and the maximum luminance level of the edge is equal to or more than the above threshold value may be extracted and defined as the edge component.

  Similarly, only arbitrary frequency components are extracted from the luminance signal input to the vertical filter component extraction device 2 by the vertical filter. The filter characteristics of this vertical edge component extraction vertical filter device can be set to an arbitrary value, but in view of the characteristics of the ten diagonally expanding filters, the characteristics are set to pass the processable frequency band.

  In this case as well, as in the horizontal edge component extraction horizontal filter device, the luminance signal that has passed through the above-described filter passes through a differentiator using an arbitrary number of pixels in the vertical direction, and an edge component is extracted. An arbitrary threshold value may be further provided for the edge component, and the case where the difference value between the minimum luminance level of the edge and the maximum luminance level of the edge is equal to or more than the above threshold value may be extracted and defined as the edge component.

  Next, the output of one horizontal edge component extraction horizontal filter is input to three horizontal edge patterning devices. In the horizontal direction edge patterning device 3, for example, for the input edge component, a luminance component lower than the average luminance level in a certain number of pixels in the peripheral horizontal direction is represented by minus, and a higher luminance component is represented by plus. As shown in FIG. 2, patterning is performed with numerical values such as..., −2, −1, −1, 2, 3,.

  Similarly, the output of 2 vertical edge component extraction vertical filters is input to 4 vertical edge patterning devices. The vertical edge patterning device 4 performs the same operation as 3 in the vertical direction. For example, a luminance component whose luminance is lower than an average luminance level in a certain number of pixels in the vertical direction around the input edge component is represented by minus, and a high luminance component is represented by plus, as shown in FIG. Patterning is performed with numerical values such as -3, -2, -1, 1, 2, 3,.

  Next, the output of 3 horizontal edge patterning devices and the output of 4 vertical edge patterning devices are input to the edge information interpolation device of 5 interpolation pixels. The operation of the five interpolation pixel edge information interpolation apparatus will be described with reference to FIGS. The outputs of the horizontal edge patterning device 3 and the vertical edge patterning device 4 are set to −1, 0, and +1 for simplification of explanation. -1 is a lower luminance level when the average value of two surrounding pixels including the target pixel is 0, and +1 is a higher luminance level.

  With this setting, as shown in FIG. 3, the image of the edge portion having both the luminance levels of black (digital value 0) and white (digital value 255) has a patterning result of [0, 0], [0, -1], [0, +1], [+1, +1], and [-1, -1]. X in the above [X, Y] indicates the luminance level of the vertical component, while Y indicates the luminance level of the horizontal component, the lattice portion indicates black, and on the other hand, it is colorless. The part shows white.

  Further, when enlarging 6/5 pixels in the vertical direction and enlarging 6/5 pixels in the horizontal direction, it is ideal to create a black and white pattern by applying a filter as shown in FIG. Therefore, after determining the horizontal interpolation pixel position corresponding to the horizontal enlargement ratio with the horizontal synchronization signal as a reference in the horizontal enlargement / vertical enlargement interpolation position generating device of 7 and determining the vertical interpolation pixel position corresponding to the vertical enlargement ratio, The following operation is performed on the pixel into which the interpolation pixel is inserted. X in the above [X, Y] indicates the luminance level of the vertical component, while Y indicates the luminance level of the horizontal component, the lattice portion indicates black, and on the other hand, it is colorless. The portion indicates white, and the portion surrounded by a broken line indicates the position where the interpolation pixel is inserted.

  In the edge information interpolating device of 5 interpolation pixels, when the vertical and horizontal directions are enlarged by 6/5 pixels, that is, when 1 pixel is enlarged in the vertical direction and 1 pixel is enlarged in the horizontal direction, the pixel pattern above the interpolation pixels in the vertical direction A value obtained by subtracting the pattern value of the lower pixel from the value and a numerical value obtained by subtracting the pattern value of the right pixel from the pattern value of the left pixel of the interpolation pixel in the horizontal direction is output as the pattern value of the interpolation pixel. Hereinafter, the value is referred to as an interpolation pixel pattern. For example, when the pixel above the interpolation pixel in the vertical direction is [0, -1] and the pixel below the interpolation pixel is [0, 0], [0, -1] is the pattern value of the interpolation pixel. When the pixel above the interpolated pixel in the vertical direction is [+1, +1] and the pixel below the interpolated pixel is [-1, -1], [2, 2] is Output as pattern value. The result of such processing is shown in FIG.

  Here, the value of the interpolated pixel pattern is a numerical value having a different meaning from the output of the patterning device including the horizontal edge patterning device 3 and the vertical edge patterning device 4 indicating the luminance of the input video signal. It has become.

  X in the above [X, Y] indicates the luminance level of the vertical component, while Y indicates the luminance level of the horizontal component, the lattice portion indicates black, and on the other hand, it is colorless. The portion indicates white, and the dotted line portion indicates a pixel in which an interpolation pixel is inserted.

  Next, the output of the 5 interpolated pixel edge information interpolating device is input to 6 surrounding target pixel examining devices. In the peripheral object inspection device 6, for example, for a pixel having an interpolation pixel pattern of −2 or less or +2 or more in the vertical direction or horizontal direction of the interpolation pixel, the continuity of the edge with the peripheral pixel is investigated. Further, with respect to the pixel having an interpolation pixel pattern greater than −2 and less than +2 among the interpolation pixels, for example, “0” is output as a detection signal from the six peripheral object investigation devices, and −2 or less and +2 or more are interpolated. For a pixel having a pixel pattern, “1” is output as a detection signal.

  In the present embodiment, the output of the horizontal direction edge patterning device 3 and the vertical direction edge patterning device 4 are set to −1, 0, and +1 for the sake of simplification, so that the description is simplified. The detection signal of a pixel having an interpolation pixel pattern greater than −2 and less than +2 is set to 0, and the detection signal of a pixel having an interpolation pixel pattern of −2 or less and +2 or more is set to 1. However, it is possible to output two or more kinds of numbers as detection signals.

  Here, the directions to be investigated are adjacent right and left pixels, upper and lower pixels, upper right pixels and lower left pixels, upper left pixels and lower right pixels, and combinations of the same edge pattern values, Alternatively, if there is a combination of pattern number values that are close to each other, a pixel of that combination is selected as being continuous.

  In addition, when there is a pixel (a-1) in FIG. 6A, which is a case where an adjacent pixel is also an interpolation pixel, it is investigated whether there is a combination having the same pattern value in combination with a pixel on the extension. To do. At this time, with respect to the target pixel (pixel surrounded by a dotted line) in FIG. 6A, the lower right before enlargement is (a-2), but in the present invention, the lower right pixel after enlargement (a -1) is the lower right pixel, the pixel on the extension is (a-3) is the lower right pixel, the pattern number is compared with the upper left pixel (a-4), and the same pattern number or a close pattern In the case of a combination of number values, the combination of pixels is selected on the assumption that the edges are continuous.

  In the case of the pixel of interest (pixels surrounded by a dotted line) in FIG. 6B in which the horizontal direction interpolation pixel and the vertical direction interpolation pixel overlap, the pattern numbers of the upper right and lower left pixels, and the upper left and lower right pixels. The combination of the same pattern numbers or the combination of the near pattern numbers is selected. However, if there is no pattern number combination that is the same or similar, or if there is no pattern number combination, “0” is output as the detection signal.

  Next, a luminance signal is input to the horizontal enlargement filter 8 and horizontal interpolation using linear interpolation or the like is performed in accordance with the output of the horizontal enlargement interpolation position generation device 7. Also, 9 receives a luminance signal and performs vertical interpolation using linear interpolation or the like in accordance with the output of the horizontal expansion interpolation position generating apparatus 7.

  Further, in the present invention, in addition to the above-described horizontal and vertical interpolations, a combination of pixels selected in 6 is used for the interpolated pixels whose detection signal of the peripheral target pixel survey 6 is “1”. Diagonal direction interpolation using linear interpolation or the like is performed by ten oblique direction expansion filters.

  The interpolation signal and the luminance signal output from the horizontal expansion filter 8, the vertical expansion filter 9, and the oblique direction expansion filter 10 are selected by the selector 11 and output. At this time, if the detection signal output of 6 is “1”, the selector 11 selects the output of the oblique enlargement filter 10, and other pixels are set as horizontal interpolation pixels by the horizontal enlargement interpolation position generation device 7. For the pixel, the output of the horizontal enlargement filter 8 is selected, and for the pixel that has been vertically interpolated by the vertical enlargement interpolation position generation device 7, the output of the vertical enlargement filter 9 is selected and output.

  In addition, the conventional enlargement filter generally applies a filter having a weight according to the pixel position after enlargement to all pixels, not just the interpolation pixels. In the present invention, all pixels are interpolated. It is also possible to select a pixel having high edge continuity by the edge information interpolating device 5 for interpolated pixels and the peripheral target pixel survey 6 and perform enlargement processing in an oblique direction.

  As a result, by detecting edge portions in the horizontal and vertical directions with respect to the digital video luminance signal and comparing the height of the correlation between the detected pixel and its surrounding pixels, pixels having higher correlation can be obtained. From this, it is possible to create pixels that are interpolated during image enlargement.

(Embodiment 2)
Next, a second embodiment of the invention will be described with reference to FIG. 7 have the same configuration as that used in the first embodiment and perform the same operation, and thus the description thereof is omitted. Here, the second embodiment is different from the first embodiment in that a frequency component integrating device 12 is provided.

  Here, the frequency component integrator 12 divides the frequency of the edge component extracted by the horizontal edge component extraction horizontal filter 1 and the vertical edge component extraction vertical filter 2 by a predetermined threshold, and sets the number of pixels for each frequency band. This is a frequency component integrating device that integrates for one screen.

On the screen where the frequency component integrator 12 has many low-band components, by increasing the number of taps of the 8 horizontal expansion filters, 9 vertical expansion filters, and 11 diagonal expansion filters, a multi-tap filter is obtained. You can expand your options.
On the screen where the frequency component integrator 12 has many high-band components, the number of taps of the 8 horizontal enlargement filter, 9 vertical enlargement filter, and 11 oblique enlargement filter is reduced to reduce the edge continuity. The possibility that enlargement processing is performed with a combination of pixels that are not present is reduced.

  With such a configuration, the present invention switches the filter characteristics (the number of filter taps and the filter coefficient) used for the enlargement filter from the summation result of the frequency components of the entire image, thereby enlarging suitable for the frequency characteristics of the pre-enlargement image. A method can be selected.

(Embodiment 3)
Next, Embodiment 3 of the invention will be described with reference to FIG. 8 have the same configuration as that used in the first embodiment and perform the same operation, and thus the description thereof is omitted. Here, the third embodiment is different from the first embodiment in that a filter configuration pixel selection device 13 is provided.

  Here, in this filter constituent pixel selection device 13, the pattern number of a certain number of pixels on the extension further ahead of the combination targeted by the peripheral target pixel survey 6 becomes the target selected by the peripheral target pixel survey 6. If the pattern number of the combined pixel has the same or close value, the extension pixel is also subject to filtering, while the extension pixel pattern number is selected by the peripheral target pixel survey 6 If the pixel number is different from the pattern number of the combination of pixels, the processing is performed such that only the combination of pixels selected by the peripheral target pixel survey 6 is the target of the filter.

  The filter target pixel selected by the filter configuration pixel selection device 13 is input to the horizontal enlargement filter 8, the vertical enlargement filter 9, and the oblique enlargement filter 10, and is enlarged by a filter having the number of taps corresponding to the number of pixels to be filtered. Processing is performed.

  With such a configuration, the present invention corresponds to the partial frequency characteristics of the image by switching the filter characteristics (number of filter taps and filter coefficients) used for the enlargement filter according to the number of continuous pixels of the edge in one image. It is possible to select only pixels with high correlation and create pixels to be interpolated during image enlargement.

  In the claims, for the sake of simplification of terms, “horizontal edge component extraction filter 1” in the embodiment is referred to as “horizontal edge extraction circuit”, and “vertical edge component extraction filter 2” in the embodiment is referred to as “horizontal edge component extraction filter 2”. “Vertical edge extraction circuit”, “horizontal edge component extraction filter 1 and vertical edge component extraction filter 2” in the embodiment are collectively referred to as “edge extraction circuit”, and “horizontal edge patterning device 3” in the embodiment. Is “horizontal patterning circuit”, “vertical edge patterning device 4” in the embodiment is “vertical patterning circuit”, and “horizontal edge patterning device 3 and vertical edge patterning device 4” in the embodiment is Collectively referred to as “patterning circuit”, and “interpolated pixel edge information interpolating device 5” in the embodiment as “interpolating circuit”. The “surrounding target pixel inspection device 6” in the embodiment is a “detection circuit”, the “horizontal enlargement filter 8” in the embodiment is a “horizontal enlargement circuit”, and the “vertical enlargement filter 9” in the embodiment is “vertical”. “Enlargement circuit”, “horizontal enlargement filter 8 and vertical enlargement filter 9” in the embodiment are collectively referred to as “enlargement circuit”, and “diagonal enlargement filter 10” in the embodiment is referred to as “oblique enlargement circuit”. The “selector 11” in the embodiment is a “selection circuit”, the “frequency component integration device 12” in the embodiment is an “integration circuit”, and the “filter configuration pixel selection device 13” in the embodiment is a “pixel selection circuit”. I decided to express it.

  The present invention relates to a video signal enlargement / scaling process that is performed on a digital display display in order to enlarge and display a digital video signal having a sampling number smaller than the number of display pixels to the number of display pixels.

The figure which shows the structure of the image | video scaling apparatus of this invention The figure which shows the value by which the edge pattern of this invention was carried out The figure which shows the numerical value which patterned the image | video before expansion of this invention The figure which shows the direction of filtering after expansion of this invention The figure which shows the edge interpolation information of the interpolation pixel after expansion of this invention The figure which shows the investigation process in the periphery object pixel investigation apparatus of this invention The figure which shows the structure of the image | video scaling apparatus of this invention The figure which shows the structure of the image | video scaling apparatus of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Horizontal edge component extraction horizontal filter apparatus 2 Vertical edge component extraction vertical filter apparatus 3 Horizontal direction edge patterning apparatus 4 Vertical direction edge patterning apparatus 5 Interpolation pixel edge information interpolation apparatus 6 Peripheral object pixel investigation apparatus 7 Horizontal expansion / vertical expansion Interpolation position generation device 8 Horizontal expansion filter 9 Vertical expansion filter 10 Diagonal direction expansion filter 11 Selector 12 Frequency component integration device 13 Filter configuration pixel selection device

Claims (6)

An edge extraction circuit for detecting a luminance change portion of the image from the luminance level of the input video signal;
A patterning circuit that assigns a first luminance value corresponding to a luminance change amount to at least the luminance change portion extracted by the edge extraction circuit;
An interpolation circuit that enlarges an image by adding pixels to the input video signal and distributes a second luminance value calculated from the first luminance value to the added pixels;
A detection circuit for detecting relevance between vertical and horizontal pixels based on the first and second luminance values;
An enlargement circuit that performs linear interpolation using vertical or horizontal pixels for the pixels added by the interpolation circuit;
An oblique enlargement circuit that performs linear interpolation using pixels in an oblique direction for the pixels added by the interpolation circuit;
An image scaling device characterized in that linear interpolation of pixels added by the interpolation circuit is performed by either the enlargement circuit or the oblique enlargement circuit based on the result of the detection circuit. In the detection circuit, when adjacent pixels in the vertical horizontal direction and the diagonal direction of the target pixel are pixels added by the interpolation circuit, the input video signal which is an adjacent pixel in the vertical horizontal direction and the diagonal direction is next. The image scaling device according to claim 1, wherein the image scaling device is compared with a first luminance value of a pixel. The video scaling device further includes an integration circuit that accumulates for each predetermined frequency of the luminance change portion extracted by the edge extraction circuit for each predetermined frame,
The video scaling device according to claim 1, wherein the number of taps of the enlargement circuit and the oblique enlargement circuit is changed based on a distribution for each predetermined frequency accumulated by the integration circuit. When the sampled digital video signal is enlarged and scaled according to the display system, the horizontal and vertical edge components of the video signal are extracted, and the image pattern is determined based on the edge pattern and edge brightness level difference with the surrounding pixels. A video scaling device that detects continuity and selects a pixel to be used for a scaling filter from the number of continuous pixels and the continuous direction of adjacent pixels. The edges in one screen are counted for each frequency divided by a threshold, and a small tap filter is used for images with many high-frequency parts and a multi-tap filter is used for images with many low-frequency parts. The video scaling apparatus according to claim 4, wherein filtering is performed by selecting each. The video scaling device according to claim 4 or 5, wherein a filter of pixels with higher correlation can be formed by switching the number of taps and coefficients of the filter for each number of pixels in which the detected edge components are continuous. .

Images