JP4827137B2 - Resolution conversion processing method, image processing apparatus, image display apparatus, and program - Google Patents

Description

  The present invention relates to a resolution conversion processing method for interpolating input image data to generate output image data having different resolutions, an image processing device for performing the processing, an image display device, and a program for causing a computer to execute the processing.

  Examples of image display devices that display images include CRTs, LCDs, PDPs, projectors, and the like. These display devices have a unique output resolution. The input image also has a unique resolution. If the resolution of the input image is the same as the resolution of the display device, the image can be displayed without any problem. And pixels that are not used in the display device. In order to prevent these problems from occurring, a process called resolution conversion for equalizing the resolution of the input image and the display device is required. Resolution conversion is also used in image enlargement / reduction processing.

  When resolution conversion is performed, each pixel of the input image is allocated on the specified coordinates, and interpolation pixels of the output image are allocated on the coordinates. The interpolation processing for resolution conversion is processing for determining the pixel value of the interpolation pixel on the coordinates based on the pixel values of the reference pixels existing around the interpolation pixel. In the conventional interpolation method, the interpolation pixel value is determined by various methods.

  Typical known methods include nearest neighbor interpolation using the pixel value of the reference pixel closest to the interpolation pixel as the pixel value of the interpolation pixel, and interpolation for the reference pixels around the interpolation pixel, respectively. Each pixel value is weighted inversely proportional to the distance to the pixel, a linear interpolation method for calculating the pixel value of the interpolated pixel, a function or polynomial that is third-order approximated from the pixel value of the reference pixel, and using that formula There are methods such as a sampling function interpolation method and a spline interpolation method (cubic convolution) for calculating pixel values of interpolation pixels.

On the other hand, there have been various proposals for improving image quality by using a plurality of interpolation methods depending on the target image. Some of these are:
(1) Patent Document 1
Using density difference (maximum density-minimum density) at multiple reference pixels to determine whether text or photo (characters greater than a certain density difference, photo for others). For photographs, linear interpolation, for text Is the method using the nearest neighbor interpolation method, or if it is determined to be a photograph, the calculation result by the linear interpolation method is the target pixel value, and if it is determined to be a character, the interpolation pixel value by the linear interpolation method is the first If the threshold value is greater than or equal to the threshold value, the maximum density value is less than the second threshold value, and the minimum density value is less than the second threshold value. Then, enlargement / reduction processing is performed by a method using the average value of the maximum density value and the minimum density value.

(2) Patent Document 2
Interpolated pixel data is obtained by a weighted average calculation based on the four original pixel data located in the vicinity thereof, and within the region surrounded by the four points based on the distribution mode of at least three of the four points of image data. When it is determined that there is a pattern forming the edge of the diagonal line, and it is determined that there is a pattern forming the edge of the diagonal line within the area surrounded by the four points, the edge of the diagonal line in the area is formed As the interpolated pixel data corresponding to the pattern, any one of the four original pixel data itself is applied regardless of the result of the weighted average calculation.

(3) Patent Document 3
CG (Computer Graphics) and natural image separately in the horizontal and vertical directions, using a difference between the pixel values of the four reference pixels arranged side by side and a predetermined threshold value (when the reference pixel is aligned with C A B D | A−B | −threshold * (| A−C | + | B−D |) / 2 is above or below 0), and the interpolation method is nearest neighbor interpolation, linear interpolation, cubic Switch with convolution.

(4) Patent Document 4
If the coordinate / luminance convolution operation method is a basic resolution conversion method and the state of the four reference pixels is not determined to be text data, the coordinate / intensity convolution operation method and text data are The text data image is made clearer and smooth in the other parts, depending on the method used by the side interpolation method or the calculation method with a larger weighting factor for the coordinate / luminance convolution method. Displays an image.

Japanese Patent No. 2991485 Japanese Patent No. 3327961 JP 2003-319171 A JP 2006-053887 A

  In the determination of characters and photographs based on density differences in Patent Document 1, characters, figures, etc. with small density differences are judged to be photographs, and processing by linear interpolation is performed, resulting in a blurred image with insufficient clarity. End up. An example is shown in FIGS. FIG. 14 shows the result of resolution conversion with the density difference used for determination set to a value that does not affect the photograph portion of the original image in FIG. If the original image in FIG. 7 is determined to have a value that does not affect the photographic part, the halftone character or graphic part is not correctly determined, resulting in a blurred image. Further, FIG. 15 is a graph obtained by performing resolution conversion using a density difference that can correctly determine a character in the original image of FIG. 7 as a value used for the determination. When this density difference is used for the determination, characters and graphic diagrams are correctly determined and clear, but a portion that is not correctly determined occurs in a photograph portion, resulting in degradation of image quality. In this way, when the determination is performed using density differences in a plurality of reference pixels, the determination varies depending on the value of the density difference used for the determination, so that there is a problem in accuracy, and when various images are handled. In many cases, a good image cannot be obtained.

  In Patent Document 2, a range in which at least three reference pixels match is set to the same pixel value, and the main purpose is to sharpen the edge of the diagonal line. However, a value adjacent to the range set to the same pixel value is a weighted average. Since it is obtained by calculation, the sharpness of the boundary portion deteriorates in the portion obtained by the weighted average calculation adjacent to the boundary portion. FIG. 16 is an image obtained by processing the original image of FIG. 7 according to Patent Document 2, and shows an example in which linear interpolation is used for photographs and nearest neighbor interpolation is used for characters and figures. In the case of using linear interpolation and nearest neighbor interpolation properly, characters and figures are clear, but jaggy is generated by using nearest neighbor interpolation, so a high quality image cannot be obtained.

  In the method of performing CG and photo determination separately in the horizontal and vertical directions according to an equation using the difference between the pixel values of four reference pixels arranged in the horizontal or vertical direction and a predetermined threshold value in Patent Document 3, The method of using the difference between the pixel values of the four reference pixels and a predetermined threshold value for the determination has a feature that the determination result is greatly different depending on the setting of the threshold value, and when determining various general images, It is difficult to set an optimum threshold value, and even when it is set, it is difficult to accurately determine various images. FIG. 17 shows an example in which resolution conversion is performed using a threshold value that makes CG clear in the original image of FIG. 7, and FIG. 18 shows a resolution conversion that uses a threshold value that reduces natural image degradation. It is an example. As storage methods used for resolution conversion, the nearest neighbor interpolation method is used when CG is determined, and the cubic convolution method is used otherwise. In FIG. 17, the character and graphic portions are clear, but jaggy, which is a feature of nearest neighbor interpolation, occurs, and it can be seen that image quality is greatly deteriorated due to erroneous determination of CG in the natural image portion. In FIG. 18, misjudgment in the natural image portion is reduced and the influence of image quality deterioration is reduced, but blurring that is a deterioration in sharpness has occurred in the character and Zuken portions. As described above, it is difficult to obtain a high-quality image by the determination method of Patent Document 3 for an image in which two images of CG and a natural image coexist.

  In Patent Document 4, the coordinate / luminance convolution operation method further includes means for determining whether the data is text data or a natural image. If the data is determined to be text data, the image information factor and the interpolation pixel between each reference pixel This method uses a calculation method that increases the weighting of the distance factor, but with such a method, a good image can be obtained, but the calculation becomes complicated and the circuit scale becomes large and the cost is high. turn into. On the other hand, when the nearest neighbor method is used, the sharpness of the boundary portion is improved, but jaggy occurs at an oblique portion.

  As described above, conventionally, in order to make the boundary portion of text data clear, edge enhancement is performed after resolution conversion or a plurality of resolution conversion methods are used depending on the state of the reference pixel. However, in these methods, pseudo contours are generated, unclear portions remain, the circuit becomes large and the cost is high, and a resolution conversion method that achieves both low cost and excellent image quality is impossible.

  The present invention improves the sharpness of the boundary portion of an image expressed with a specific gradation such as a figure or a character while suppressing the circuit scale, while maintaining the gradation of the input image in other images, To provide a resolution conversion processing method for creating a high-quality resolution-converted image, an image processing device for performing the same, an image display device including the image processing device, and a program for causing a computer to execute the resolution conversion processing method It is in.

  When a natural image and a character or graphic are identified at the region level, if the identification is mistaken, the image quality of the entire region is deteriorated. Therefore, the present invention focuses on the fact that it is possible to obtain a higher quality image by partially determining a narrow range and performing optimum processing. The gist of the present invention is to perform interpolation processing first, and then to perform edge enhancement processing on a portion that can be determined as a boundary between characters and figures within a certain narrow range, thereby obtaining a high-quality image.

According to a first aspect of the present invention, there is provided a resolution conversion processing method for generating pixels (hereinafter referred to as interpolation pixels) to be interpolated between 2 × 2 pixels (hereinafter referred to as original pixels) of an input image and outputting images having different resolutions. A first calculation step for generating intermediate processing data of an interpolation pixel in which the gradation of the input image is stored, and each determination reference pixel using at least a 2 × 2 reference pixel adjacent to the interpolation pixel as a determination reference pixel combinations and maximum values of the pixel values, both find the minimum, the steps of generating an intermediate value from the pixel value of the judgment reference pixels, the intermediate if the combination of the pixel values of the reference pixels for the determination three or more The value of the processing data is the pixel value of the interpolation pixel. If the number of combinations is two or less , the value of the intermediate processing data is compared with the intermediate value. If the value of the intermediate processing data is smaller than the intermediate value, the determination reference Pixel It comprises a second calculation step in which the minimum value of the pixel values is an intermediate value if they are equal, and the other pixel value is the maximum value of the pixel values of the reference pixel for determination .

According to a second aspect of the present invention, in the resolution conversion processing method according to the first aspect, the first calculation step uses the distance between each of the 2 × 2 reference pixels around the interpolation pixel and the interpolation pixel as a weighting factor. It is characterized in that intermediate processing display data of interpolation pixels is generated by linear interpolation calculation.

According to a third aspect of the present invention, in the resolution conversion processing method of the first aspect, the first calculation step is an image using a difference between a pixel value of each reference pixel of 2 × 2 around the interpolation pixel and a predetermined value. By performing weighting using an information factor and a distance factor using a distance between the interpolation pixel and each reference pixel, intermediate processing data of the interpolation pixel is generated by a coordinate / luminance convolution operation.

  According to a fourth aspect of the present invention, in the resolution conversion processing method according to the first aspect, in the first calculation step, an interpolation pixel is obtained by a third-order convolution interpolation method using reference pixels equal to or more than the number of reference pixels used for the determination reference pixel. The intermediate processing data is generated.

According to a fifth aspect of the present invention, in the resolution conversion processing method according to any one of the first to fourth aspects, the intermediate value is an intermediate value between a maximum value and a minimum value of the plurality of reference pixels for determination. It is characterized by doing.

The invention according to claim 6 is an image processing apparatus comprising input data storage means, processing means, and output data storage means, wherein the resolution conversion processing method according to any one of claims 1 to 5 is implemented. .

A seventh aspect of the present invention is an image display device comprising at least the image display means and the image processing device according to the sixth aspect.

The invention according to claim 8 is a program for causing a computer to execute each step of the resolution conversion processing method according to any one of claims 1 to 5 .

In the first aspect of the present invention, the text data is determined from the pixel values of the 2 × 2 reference pixels adjacent to the interpolation pixel, and the text data includes the intermediate processing data based on the intermediate processing data generated by the first calculation. By comparing the value with the intermediate value, the pixel value of the interpolated pixel is obtained from the 2 × 2 determination pixel as a result, and the value of the intermediate processing data other than the text data is used. It is possible to obtain a clear image of the boundary portion of the text data while utilizing the above.
Specifically, as the second calculation, when the combination of pixel values of 2 × 2 reference pixels for evaluation is two or less, the intermediate processing data is compared with the intermediate value as text data, and the pixel of the interpolation pixel is determined based on the result. By performing the process of setting the value to the value obtained from the reference pixel for evaluation, the certainty of determination on the text data is increased. Specifically, in the second calculation, if the intermediate processing data is smaller than the intermediate value, the minimum value of the pixel value of the evaluation reference pixel is equal, if it is equal, the intermediate value is set, and otherwise, the maximum value of the pixel value of the evaluation reference pixel is set. By setting the pixel value of the interpolation pixel, a clearer image at the boundary can be obtained.

In the second aspect of the invention, the first calculation is performed using the distance between the 2 × 2 reference pixels around the interpolation pixel and the interpolation pixel as a weighting factor, and the pixel value of the interpolation pixel is generated by linear interpolation calculation. By adopting the method, the circuit scale can be reduced, so that a smooth and clear image of the boundary portion of the text data can be obtained at low cost.

According to a third aspect of the present invention, the first calculation is performed based on an image information factor using a difference between a pixel value of a 2 × 2 reference pixel around the interpolation pixel and a predetermined value, and between the interpolation pixel and each reference pixel. By performing weighting with a distance factor using distance, a method of generating pixel values of interpolated pixels by coordinate / luminance convolution calculation, and a very smooth image that is a feature of the first calculation Thus, it is possible to obtain an image that balances the sharpness of the text data boundary.

  In the invention of claim 4, the first calculation is a cubic convolution interpolation method using reference pixels equal to or more than the number of reference pixels used for the determination reference pixels, so that the natural image becomes very clear, An image in which the sharpness of the text data boundary portion is compatible can be obtained.

In the invention of claim 5 , the intermediate value is set to the intermediate value between the maximum value and the minimum value of the 2 × 2 evaluation reference pixels , thereby making the boundary portion clear while utilizing the characteristics of the first calculation. An image can be obtained.

In the invention of claim 6, further comprising means for implementing a resolution converting method according to any one of claims 1 to 5, but also from the conventional image processing apparatus to obtain a clear image of the text data boundary An image processing apparatus that can be used is realized.

In the invention of claim 7 , by providing the image processing device of claim 6 , it is possible to realize an image display device capable of displaying a clearer image of the text data boundary portion than the conventional image display device.

According to the eighth aspect of the present invention, resolution conversion for displaying a clear image of the text data boundary portion can be performed by reading and operating the computer program on a computer.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is an overall block diagram of an image display apparatus including an image processing apparatus that implements the resolution conversion processing method of the present invention. In FIG. 1, 10 is an image processing apparatus, and 20 is an image display apparatus.

  In the image processing apparatus 10, the input image data is temporarily stored in the input data storage unit 11 in units of pixels, predetermined lines, frames, and the like. At the same time, control information (clock, horizontal synchronization signal, vertical synchronization signal, etc.) of the input image data is sent to the resolution detection unit 14. The resolution detection unit 14 determines the resolution of the input image data by counting the number of clocks in the horizontal synchronization signal period and the number of horizontal synchronization signals in the vertical synchronization signal period, and sends the resolution information to the coordinate table unit 15. In general, the resolution of an input image can be specified in several ways, such as VGA, SVGA, XGA, UXGA, and QXGA. The coordinate table unit 15 stores interpolation data including interpolation pixels (interpolation pixels No.), reference pixels to be used (reference pixels No.), and inter-pixel distances between the interpolation pixels and the reference pixels, corresponding to the respective resolutions. The look-up table (LUT) is held. The coordinate table unit 15 selects an LUT corresponding to the resolution determined by the resolution detection unit 14, reads out the interpolation data, and stores it in the interpolation data storage unit 16. The interpolation processing unit 12 sequentially reads out the interpolation pixel and each reference pixel and the inter-pixel distance between the interpolation pixel and each reference pixel from the interpolation data storage unit 16 and reads out the pixel value of each reference pixel from the input data storage unit 11. The process of generating the pixel value of the interpolation pixel based on the inter-pixel distance between the interpolation pixel and each reference pixel and the pixel value of each reference pixel is repeatedly executed. The pixel values of the generated interpolation pixels are sequentially stored in the output data storage unit 13.

  In the image display device 20, the pixel value of each interpolation pixel stored in the output data storage unit 13 is read out in predetermined units and output to the display unit 21.

  A feature of the present invention resides in the processing of the interpolation processing unit 12. The processing in the interpolation processing unit 12 includes a first calculation for generating intermediate processing data of interpolation pixels in which the gradation of the input image is preserved, and an area expressed by a specific gradation such as a figure or a character. On the other hand, an interpolation pixel value in which the boundary portion is emphasized is generated, and in the other regions, it is roughly divided into a second calculation using the intermediate processing data as the interpolation pixel value.

  As the first calculation, a linear interpolation calculation method, a coordinate / luminance convolution calculation method, a cubic convolution interpolation method, or the like is applied. The second calculation obtains the maximum value, the minimum value, and the intermediate value of the pixel values of a plurality of reference pixels adjacent to the interpolation pixel. If the number of combinations of reference pixel values is equal to or greater than a predetermined number, the intermediate processing data is converted to the interpolation pixel value. If the number is less than the predetermined number, the value of the intermediate processing data is compared with the intermediate value, and based on the result, one of the maximum value, the intermediate value, and the minimum value is set as the interpolated pixel value. Hereinafter, each example will be described in detail.

<Example 1>
In this embodiment, linear interpolation calculation is applied as the first calculation. The linear interpolation calculation is a method of calculating the pixel value of the interpolation pixel using the distance between each of the plurality of reference pixels around the interpolation pixel and the interpolation pixel as a weighting factor. Here, when four reference pixels around the interpolation pixel are used and the combination of the four reference pixel values is two or less, the value of the intermediate processing data calculated by the linear interpolation calculation and the four points The intermediate value (boundary value) between the maximum value and the minimum value in the reference pixel values is compared as the intermediate value (boundary value), and the pixel value of the interpolated pixel is determined as the maximum value and the minimum value among the four reference pixels. In the case of any one or an intermediate value and three or more sets, an example in which the value of the intermediate processing data of the linear interpolation calculation method of the first calculation is used as the pixel value of the interpolation pixel will be described as an example.

  Here, the combination of reference pixel values means that when four reference pixels are used, for example, when the value of each reference pixel is 0, 1, 2, 3, all four values are different, and 0, 1 , 0, 3 can be classified into three types of 0, 1, 3 and so on, and 0, 1, 0, 1 can be classified into two types of values 0, 1, and so on. , 0, 0, 0 are counted as one set because the value is only 0. As described above, the combination of reference pixel values is defined as the number of sets into which the values (reference pixel values) of reference pixels having a predetermined number of points (here, four points) are classified. In this embodiment, when the number of combinations of four reference pixel values is two or less (number of combinations ≦ 2), it is determined that the data is text data such as characters, graphics, or the boundary portion thereof. The same applies to Examples 2 and 3 described later.

FIG. 5 schematically shows the positional relationship between the interpolation pixel and the reference pixel in explaining this embodiment. Here, the pixel value of the interpolation pixel B is b, the pixel value of the reference pixel A00 located in the vicinity of the interpolation pixel B is a00, the pixel value of the reference pixel A10 is a10, the pixel value of the reference pixel A01 is a01, and the reference pixel The pixel value of A11 is a11, the horizontal distance between the reference pixels A00, A01 and the interpolation pixel B is x1, the horizontal distance between the reference pixels A10, A11 and the interpolation pixel B is x2, and the reference pixels A00, A10 are interpolated. The distance in the vertical direction from the pixel B is y1, and the distance in the vertical direction between the reference pixels A01 and A11 and the interpolation pixel B is y2. In the case of linear interpolation calculation, the pixel value b of the interpolation pixel B is obtained as shown in Expression (1).
b = x2 · y2 · a00 + x1 · y2 · a10 + x2 · y1 · a01
+ X1 ・ y1 ・ a11 (1)

  FIG. 2 is a flowchart showing a flow related to the interpolation processing of this embodiment. Hereinafter, the operation of the interpolation processing unit 12 in this embodiment will be described with reference to the flowchart of FIG.

  In the interpolation processing unit 12, the reference data (A00, A10, A01, A11) used for the interpolation calculation for the first interpolation pixel, the pixels of the interpolation pixel and each reference pixel are firstly received from the interpolation data storage unit 16. The inter-distance (x1, x2, y1, y2) is read, and the pixel values (a00, a10, a10, a01, a11) of each reference pixel (A00, A10, A01, A11) are read from the input data storage unit 11 (step) 101-103). Then, based on the inter-pixel distance (x1, x2, y1, y2) between the interpolated pixel and each reference pixel and the pixel values (a00, a10, a01, a11) of each reference pixel, resolution conversion is performed using Equation (1). Intermediate processing data b is calculated (step 104).

Next, as a preparation for determination, the interpolation processing unit 12 compares the pixel values of each reference pixel read from the input data storage unit 11 to determine the number of combinations of reference pixel values (step 105). Further, the maximum value Max and the minimum value min among the four reference pixel values are discriminated (step 106), and an intermediate value c between them is determined.
c = (Max + Min) / 2
Is obtained (step 107).

Next, in the interpolation processing unit 12, as a second calculation, it is determined whether or not the four reference pixel values are 3 sets or more (number of combinations> 2) (step 108). The pixel value (B) is the value of the intermediate processing data b obtained by the first calculation (linear interpolation calculation) (step 109). On the other hand, in the case of two sets or less, the intermediate processing data b and the intermediate value C are compared (step 110).
If b> C then B = Max
If b = C then B = C
If b <C, B = Min
(Steps 111 to 113).

  Thus, the pixel value B of the first interpolation pixel is generated. The interpolation processing unit 12 repeats the same processing until there is no unprocessed interpolation data in the interpolation data storage unit 16 (step 114). Thereby, the pixel value of each interpolation pixel is generated. The pixel value of the interpolated pixel processed by the interpolation processing unit 12 is stored in the output data storage unit 6 and is output to the display unit 21 for each predetermined unit.

  As described above, in this embodiment, when the number of combinations of four reference pixel values is two or less, it is determined as text data such as characters, graphics, etc., or a boundary portion thereof. Text data such as characters and graphics are almost always composed of the same pixel value. In most cases, these are drawn on the same color background. In such a case, the combination of four reference pixel values at the boundary is two. Therefore, when the number of combinations is two or less, the sharpness can be enhanced by determining the text data such as characters and the boundary portion such as graphics. In natural images, there are cases where there are two sets. In this embodiment, the result calculated by the first calculation (linear interpolation calculation) is used as intermediate processing data for determining the interpolation pixel value. Since the characteristics of the calculation can be reflected, and thereby the influence on the continuity of gradation can be kept to a minimum, there is almost no influence on the image quality.

  In this way, when the combination of four reference pixel values is 2 or less, it is determined that the boundary portion of text data such as characters or graphics is processed, and the boundary portion of text data such as characters or graphics is processed. It is possible to cover the majority of images and to minimize misjudgment to natural images.

  When the background is graded or when the background changes, the number of sets may be three or four. Although not illustrated in the present embodiment, in the case of three sets, when there are adjacent pixels having the same pixel value, text data such as characters or graphics if the difference between the pixel values of the other two points is within a predetermined range If there is a pixel with the same pixel value on the diagonal, it is determined as a boundary of text data such as text or a figure, and the range is divided into pixels with the same pixel value at the bottom. It is also possible to do so. Furthermore, the evaluation reference pixels are not limited to four points, and surrounding pixels can also be used for the evaluation and can be determined as boundary portions of text data such as characters or figures based on continuity of the same pixel value. .

  In the method of using the reference pixel value before resolution conversion as in the present embodiment for the determination of text data such as characters or graphics or the boundary portion thereof and a natural image, the accuracy is superior to the determination based on the difference in pixel value, Compared to methods such as processing the image data after resolution conversion (determining with a predetermined threshold value and performing processing, etc.), the reference pixel data used for determination is the original image data. It is self-evident that the accuracy is better than using.

  7 is an original image before resolution conversion, FIG. 8 is an image obtained by simply performing resolution conversion by a linear interpolation method, and FIG. 9 is an example of an image obtained by performing resolution conversion in this embodiment.

  The image of the linear interpolation method of FIG. 8 can obtain a smooth image with a natural image or the like, while text data such as characters and graphics have unclear boundaries. On the other hand, in FIG. 9 which is an image obtained by adding the second calculation to the linear interpolation method of the first calculation of the present embodiment, the boundary portion of text data such as characters and figures which are unclear in FIG. It is clear. In addition, this has almost no effect on the natural image portion. In this way, by adding the processing of the second calculation to the first calculation of the linear interpolation method of the present embodiment, the boundary of text data such as characters and figures becomes clear, and a high-quality image is obtained. There is an effect. Further, in the image by the method of Patent Document 2 in FIG. 16, the boundary part of text data such as characters and the figure is clearer than the linear interpolation method of FIG. 8, but a weighted average operation is used for the text data part. Due to the presence of the portion, blurring occurs, which is inferior in sharpness as compared with FIG. 9 of the present embodiment.

  As described above, in this embodiment, a sharper image can be generated by clarifying the boundary portion by not using the halftone of the linear interpolation method that causes the deterioration of the sharpness. Furthermore, by comparing the result of the linear interpolation method, which is the first calculation, with the intermediate value C, the smoothness of the linear calculation method, which is the first calculation, can be achieved even at the boundary of text data such as characters and figures. It is possible to make the best use of the image, and it is possible to achieve both the sharpness and the stored smoothness of the continuity of gradation, and a high-quality image can be obtained.

<Example 2>
In this embodiment, a coordinate / luminance convolution operation is applied as the first operation. The second calculation is the same as in the first embodiment. The coordinate / luminance convolution operation is performed by an image information factor using a difference between a pixel value of a plurality of reference pixels around the interpolation pixel and a predetermined value, and a distance factor using a distance between the interpolation pixel and each reference pixel. This is a method of calculating pixel values of interpolation pixels by performing weighting. This coordinate / luminance convolution calculation is described in, for example, Japanese Patent Application Laid-Open No. 2006-53887 (Patent Document 4).

  Here, as in the first embodiment, an example using four reference pixels (A00, A10, A01, A11) around the interpolation pixel (B) as shown in FIG. 5 will be described.

  FIG. 3 is a flowchart showing a flow related to the interpolation processing of this embodiment. The operation of the interpolation processing unit 12 in this embodiment will be described below with reference to the flowchart of FIG.

  In the interpolation processing unit 12, the reference data (A00, A10, A01, A11) used for the interpolation calculation for the first interpolation pixel, the pixels of the interpolation pixel and each reference pixel are firstly received from the interpolation data storage unit 16. The inter-distance (x1, x2, y1, y2) is read, and the pixel values (a00, a10, a10, a01, a11) of the respective reference pixels (A00, A10, A01, A11) are read from the input data storage unit 11 (step 101-103). Then, for the coordinate / luminance convolution operation, the maximum pixel value Max and the minimum pixel value Min are obtained from the comparison of the four reference pixel values read from the input data storage unit 11 in advance (step 204). ).

  Next, the interpolation processing unit 12 performs a coordinate / luminance convolution operation as a first calculation to obtain intermediate processing data b (step 205).

First, the value M is calculated from the maximum pixel value Max and the minimum pixel value Min as follows.
M = Max-Min + 1

Next, an average value (AVE) of the pixel values of the four reference pixels is obtained as follows.
AVE = (a00 + a01 + a10 + a11) / 4

Next, using the distance between the interpolation pixel and each reference pixel, the difference between the pixel value of each reference pixel and its average value AVE, and the value M, the values of Z00, Z10, Z01, and Z11 are expressed by the following equation (2). Generate by.
Z00 = ((x2 · y2) · (1- | a00-AVE | / M))
Z10 = ((x1 · y2) · (1- | a10-AVE | / M))
Z01 = ((x2.y1). (1- | a01-AVE | / M))
Z11 = ((x1 · y1) · (1- | a11-AVE | / M)) (2)

Next, each reference pixel is weighted using the pixel value of each reference pixel and the values of Z00, Z10, Z01, and Z11, and intermediate processing data b is generated by the following equation (3).
b = (Z00 · a00 + Z10 · a10 + Z01 · a01 + Z11 · a11)
/ (Z00 + Z10 + Z01 + Z11) (3)

Next, the interpolation processing unit 12 determines the number of combinations of reference pixel values as preparation for determination (step 206). Further, an intermediate value C that is an average value is calculated from the maximum pixel value Max and the minimum pixel value Min in the reference pixel value (step 207). Here, since the maximum value Max and the minimum value Min are already obtained in step 204, they are used.
C = (Max + Min) / 2

Next, the interpolation processing unit 12 determines whether the combination of the four reference pixel values is three or more (number of combinations> 2) as the second calculation (step 208). The pixel value (B) of the interpolated pixel is set to the value of the intermediate processing data b obtained by the first calculation (coordinate / luminance convolution calculation) (step 209). On the other hand, in the case of 2 sets or less, the intermediate processing data b and the intermediate value C are compared (step 210),
If b> C then B = Max
If b = C then B = C
If b <C, B = Min
(Steps 111 to 113).

  Thus, the pixel value B of the first interpolation pixel is generated. The interpolation processing unit 12 repeats the same processing until there is no unprocessed interpolation data in the interpolation data storage unit 16 (step 114). Thereby, the pixel value of each interpolation pixel is sequentially generated.

  The pixel value of the interpolation pixel generated by the interpolation processing unit 12 may be output as it is, but here it is stored in the temporary output data storage unit 13 and output to the display unit 21 for each predetermined unit.

  In this embodiment, the feature of the smoothness of the image by the coordinate / luminance convolution operation method, and the sharpness of the boundary portion while reflecting the smoothness of the image in the boundary portion between the text data such as characters and the figure and the boundary portion are remarkably improved. An improved image can be obtained.

  10 is an image obtained by performing resolution conversion on the original image of FIG. 7 only by the coordinate / luminance convolution operation method, and FIG. 11 is an example of an image obtained by performing resolution conversion in this embodiment.

  The coordinate / luminance convolution calculation method is characterized in that a natural image that is very smooth and has little jaggy is obtained, and this feature also appears well in the image of FIG. However, since there are halftone pixels at the boundary of text data such as characters and graphics, there is room for improvement in order to obtain a clear image. In FIG. 11, which is an image of the present embodiment in which the second calculation is added to the first calculation of this coordinate / luminance convolution calculation method, text data such as characters and figures that have room for improvement in FIG. The boundary is clear. In addition, this has almost no effect on the natural image portion.

  As described above, in this embodiment, the boundary of text data such as characters and graphics can improve the sharpness while maintaining the smoothness of the image, which is a feature of the interpolation method based on the coordinate / luminance convolution operation. .

<Example 3>
In this embodiment, a cubic convolution interpolation method is applied as the first calculation. The cubic convolution interpolation method (cubic convolution) is described, for example, in JP-A-11-25067.

  In the present embodiment, a cubic convolution interpolation method using 16 reference pixels as the first calculation is used for the first calculation, and the second calculation is for determining four points adjacent to the interpolation pixel from among the 16 reference pixels. When the combination of pixel values of the reference pixel for determination has two or less sets as reference pixels, the value of the intermediate processing data calculated by the third-order convolution interpolation method and the four reference pixel for determination An intermediate value between the maximum value and the minimum value in the values is compared as an intermediate value (boundary value), and the pixel value of the interpolated pixel is determined based on the result as either the maximum value or the minimum value among the four reference pixel values for determination. In the case of three or more sets of intermediate values, an example in which the intermediate data value of the first-order cubic convolution interpolation is used as the pixel value of the interpolation pixel will be described.

  FIG. 6 schematically shows the positional relationship between the interpolation pixel and the reference pixel in explaining this embodiment. 16 pixels A11 to A44 are reference pixels, and 4 pixels A22 to A33 are among them. The reference pixel for determination, B is an interpolation pixel. Here, the pixel value of the interpolation pixel B is b, and the pixel values of the reference pixels A11 to A44 are a11 to a44. Further, the horizontal distance between the reference pixels A11, A12, A13, and A14 and the interpolation pixel B is x1, the horizontal distance between the reference pixels A21, A22, A23, and A24 and the interpolation pixel B is x2, and the reference pixel A31, The horizontal distance between A32, A33, A34 and the interpolation pixel B is x3, and the distance in the flat water direction between the reference pixels A41, A42, A43, A44 and the interpolation pixel B is x4. Similarly, the vertical distance between the reference pixels A11, A21, A31, A41 and the interpolation pixel B is y1, the vertical distance between the reference pixels A12, A22, A32, A42 and the interpolation pixel B is y2, and the reference pixel. The vertical distance between A13, A23, A33, A43 and the interpolation pixel B is y3, and the vertical distance between the reference pixels A14, A24, A34, A44 and the interpolation pixel B is y4.

  FIG. 4 is a flowchart showing a flow related to the interpolation processing of this embodiment. Hereinafter, the operation of the interpolation processing unit 12 in this embodiment will be described with reference to the flowchart of FIG.

  In the interpolation processing unit 12, from the interpolation data storage unit 16, first, with respect to the first interpolation pixel, each reference pixel (A14 to A44) used for the interpolation processing, and an inter-pixel distance (x1) between the interpolation pixel and each reference pixel. , X2, x3, x4, y1, y2, y3, y4) and pixel values (a11 to a44) of the reference pixels (A11 to A44) are read from the input data storage unit 11 (steps 301 to 303). Then, as the first calculation, the intermediate processing data b is generated by the cubic convolution interpolation method using the read 16 reference pixel values and the inter-pixel distance (step 304).

  The cubic convolution interpolation method determines an interpolation pixel value by executing an interpolation process by convolution using a sampling function in parallel with each axis. As the sampling function at this time, a sinc function approximated by the third order as shown in the following equation is used.

  The interpolation processing unit 12 first substitutes the distances x1, x2, x3, x4, y1, y2, y3, and y4 between the interpolation pixel and each reference pixel into the variable t of the sampling function, thereby obtaining f (x1), f (x2), f (x3), f (x4), f (y1), f (y2), f (y3), and f (y4) are obtained. The distance between pixels is 1. Next, using the obtained f (x1) to f (x4), f (y1) to f (y4), and pixel values a11 to a44 of each reference pixel,

により中間処理データｂを算出する。

Intermediate processing data b is calculated by

Next, as the preparation for determination, the interpolation processing unit 12 selects four reference pixels (A22, A23, A32, A33) adjacent to the interpolation pixel from among the 16 reference pixels read from the input data storage unit 16. Are used as reference pixels for determination in the second calculation (step 305), and pixel values of the reference pixels for determination (A22, A23, A32, A33) are compared to determine a combination of reference pixels for determination. (Step 306). Further, the maximum pixel value Max and the minimum pixel value Min among the four reference pixel values for determination are obtained (step 307), and an intermediate value C is determined from the maximum pixel value Max and the minimum pixel value Min. Calculate (step 308).
C = (Max−Min) / 2

Next, the interpolation processing unit 12 determines whether the combination of the four reference pixel values for determination is three or more (number of combinations> 2) as the second calculation (step 309). The pixel value (B) of the interpolated pixel is the value of the intermediate processing data b obtained by the first calculation (third-order convolution interpolation calculation) (step 310). On the other hand, in the case of two sets or less, the intermediate processing data b and the intermediate value C are compared (step 311),
If b> C then B = Max
If b = C then B = C
If b <C, B = Min
(Steps 312 to 314).

  Thus, the pixel value B of the first interpolation pixel is generated. The interpolation processing unit 12 repeats the same processing until there is no unprocessed interpolation data in the interpolation data storage unit 13 (step 115). Thereby, the pixel value of each interpolation pixel is sequentially generated.

  The pixel value of the interpolation pixel generated by the interpolation processing unit 12 may be output as it is, but here it is stored in the temporary output data storage unit 13 and output to the display unit 21 for each predetermined unit.

  In this embodiment, it is possible to obtain a clear image of the boundary portion while reflecting the feature of the cubic convolution method (cubic convolution) as the first calculation. In this embodiment, four reference pixels adjacent to the interpolation pixel are used as the evaluation reference pixels, and the sinc function is used as the sampling function of the cubic convolution interpolation method. However, the number of reference pixels used for the evaluation reference pixels The sampling function is not limited to this example.

  FIG. 12 shows an image obtained by performing resolution conversion only by cubic convolution, which is a cubic convolution interpolation method, and FIG. 13 shows an example of an image obtained by performing resolution conversion in this embodiment.

  In the cubic convolution of FIG. 12, natural images are very clear, and an emphasis effect is obtained at the boundary of text data and graphics such as characters and graphics whose pixel values change abruptly. Although it is clearer than the interpolation method, it is slightly blurred. In addition, some noise is also generated, and jaggy is generated in characters and diagonal lines. On the other hand, the image of the present embodiment in FIG. 13 has a clearer boundary portion of text data such as characters and figures than in FIG. 12, and pixel values by enhancement of the boundary portion, which is a feature of cubic convolution. This eliminates the problem of non-uniformity, and text data such as characters and figures is an image with uniform pixel values. Moreover, there is almost no influence on the natural image portion by adding the processing of the second calculation. Furthermore, no noise is generated.

  As described above, in this embodiment, by adding the processing of the second operation to the cubic operation convolution interpolation method of the first operation, the boundary portion of the text data such as characters and the figure is the interpolation method. The sharpness is improved while maintaining the clearness of the characteristic natural image.

  Note that the processing functions of some or all of the components in the apparatus shown in FIG. 1 can be configured by a computer program and the program can be executed using the computer to implement the present invention, or FIG. Needless to say, the processing procedure shown in FIG. 4 can be configured by a computer program and the program can be executed by the computer. In addition, a computer-readable recording medium such as an FD, MO, ROM, memory card, CD, or the like is stored in the computer. In addition, the program can be recorded and stored on a DVD, a removable disk, etc., and the program can be distributed through a network such as the Internet.

1 is a block diagram illustrating a configuration example of an image processing apparatus and an image display apparatus according to an embodiment of the present invention. 5 is a flowchart showing a flow related to interpolation processing according to the first embodiment. 10 is a flowchart showing a flow related to interpolation processing according to the second embodiment. 10 is a flowchart illustrating a flow regarding interpolation processing according to the third embodiment. The figure which showed typically the position and distance of the interpolation pixel and reference pixel which were arranged in the two-dimensional lattice form for Example 1,2. The figure which showed typically the position and distance of the interpolation pixel, reference pixel, and evaluation reference pixel which were arranged in the two-dimensional grid | lattice form for Example 3. FIG. An example of an original image. An example of an image obtained by performing resolution conversion on the original image of FIG. 7 using a linear interpolation method. 7 is an example of an image obtained by performing resolution conversion on the original image in FIG. 7 by the method of the first embodiment. 8 is an example of an image obtained by performing resolution conversion on the original image of FIG. 7 using a coordinate / luminance convolution operation method. An example of an image obtained by performing resolution conversion on the original image in FIG. 7 by the method of the second embodiment. FIG. 8 is an example of an image obtained by performing resolution conversion on the original image in FIG. 7 using cubic convolution. An example of an image obtained by performing resolution conversion on the original image in FIG. 7 by the method of the third embodiment. FIG. 8 is an example of an image obtained by performing resolution conversion on the original image in FIG. 7 with the density difference set to a value that does not affect the natural image portion in FIG. 7 is an example of an image obtained by performing resolution conversion with the original image of FIG. 7 in the interpolation method of Patent Document 1 with density differences set to values that allow characters to be correctly determined in the image of FIG. An example of an image obtained by performing resolution conversion on the original image of FIG. FIG. 18 is an example of an image obtained by performing resolution conversion on the original image of FIG. 17 by using the interpolation method of Patent Document 3 and using the threshold value at which CG becomes clear in the original image of FIG. An example of an image obtained by performing resolution conversion on the original image of FIG. 7 by using the interpolation method of Patent Document 3 and using a threshold that reduces natural image degradation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Input data storage part 12 Interpolation processing part 13 Output data storage part 14 Resolution detection part 15 Coordinate table part 16 Interpolation data storage part 20 Image display apparatus 21 Display part

Claims (8)

In a resolution conversion processing method for generating pixels (hereinafter referred to as interpolation pixels) to be interpolated between 2 × 2 pixels (hereinafter referred to as original pixels) of an input image and outputting images having different resolutions,
A first calculation step of generating intermediate processing data of interpolation pixels in which the gradation of the input image is stored;
Combinations and maximum value of the pixel values of the reference pixels for each determined 2 × 2 reference pixels adjacent to at least the interpolation pixel as reference pixels for determining both find the minimum, an intermediate value from the pixel value of the determined reference pixels Generating step;
If the combination of pixel values of the reference pixel for determination is 3 or more, the value of the intermediate processing data is the pixel value of the interpolated pixel. If the combination is 2 or less , the value of the intermediate processing data is compared with the intermediate value. If the value of the intermediate processing data is smaller than the intermediate value, the minimum value of the pixel value of the reference pixel for determination, the intermediate value if equal, otherwise the maximum value of the pixel value of the reference pixel for determination is the pixel of the interpolation pixel A second calculation step for a value ,
A resolution conversion processing method characterized by comprising: The first calculation step, a distance between the interpolation pixel and the reference pixels in the 2 × 2 in the periphery of the interpolation pixel as heavy with factor, to generate intermediate processing data of the interpolation pixel from the linear interpolation operation The resolution conversion processing method according to claim 1, wherein: The first calculation step, the image information element using the difference between the pixel value and a predetermined value of the reference pixels in the 2 × 2 in the periphery of the interpolation pixel, and the interpolation pixel and between the respective reference pixels 2. The resolution conversion processing method according to claim 1, wherein intermediate processing data of the interpolated pixel is generated by a coordinate / luminance convolution operation by weighting with a distance factor using distance.   The first calculation step generates intermediate processing data of an interpolation pixel by a third-order convolution interpolation method using reference pixels equal to or more than the number of reference pixels used for the determination reference pixel. The resolution conversion processing method described. 5. The resolution conversion process according to claim 1, wherein the intermediate value is an intermediate value between a maximum value and a minimum value of pixel values of the 2 × 2 determination reference pixels. 6. Method. An image processing apparatus comprising: an input data storage unit; a processing unit; and an output data storage unit, wherein the resolution conversion processing method according to claim 1 is carried out. An image display device comprising at least an image display means and the image processing device according to claim 6. The program for making a computer perform each step of the resolution conversion processing method of any one of Claims 1 thru | or 5.

Images