JP2004180171A - Device, method and program for processing image, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and image processing method for fast performing magnifying processing in a high quality manner and with a small processing load. <P>SOLUTION: An image block characteristic amount calculation part 12 calculates a characteristic amount in an area under consideration about an image block with a prescribed size including pixels under consideration segmented by an image block setting part 11. A magnified image block generating part 13 has a first generating part 21 for performing magnifying processing in a high quality manner and a second generating part 22 whose processing load is small as a plurality of magnifying means and uses one between the first and second generating parts to generate a magnified image block corresponding to the area under consideration according to the characteristic amount calculated by the image block characteristic amount calculation part 12. For example, the first generation part 21 is used for a part affecting image quality and the second generation part 22 is used for the other parts to realize fast magnifying processing in a high quality manner. An image block arranging part 14 sequentially arranges the generated magnified image block to generate magnified image data. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing technique for performing an enlargement process of an image represented by multi-gradation including color, and in particular, without causing an image quality defect such as blur or jaggy to an input image as much as possible. The present invention relates to an image processing technique for performing enlargement processing with high image quality and light processing load.
[0002]
[Prior art]
The image enlargement process is one of the basic processes for a system that performs editing, filing, display, printing, and the like of an image. In recent years, with the spread of image data on the Internet homepage and image data mainly for display at a display resolution such as digital video, low-resolution images are frequently printed with a high-resolution printer or the like. Has been done. At the time of printing by this printer, it is desired to obtain a high-quality output result, and the importance of high-quality enlargement processing is increasing.
[0003]
Existing representative methods for enlarging an image represented by multi-gradation including color (hereinafter referred to as a multi-valued image) include a nearest neighbor method, a linear interpolation method, a cubic convolution method, and the like. There is. The nearest neighbor method is a method of using the pixel value of a pixel having the closest distance when the pixel is inversely mapped on the original image as each pixel value after enlargement. This method can perform high-speed processing because the amount of calculation is small. However, since one pixel of the original image is directly enlarged to a rectangular shape, the degree of image quality degradation is small and hardly affected when the difference between the pixel values of adjacent pixels is small. The degree of image quality degradation is large, such as when jaggies at edges and edge portions are conspicuous or when the magnification is large, the image becomes mosaic.
[0004]
The linear interpolation method is a method of assuming that pixel values between pixels linearly change, and linearly interpolating pixel values of four pixels near a point at which an enlarged pixel is inversely mapped to obtain a pixel value. It is. In this method, although the processing is heavier than the nearest neighbor method, the amount of calculation is relatively small, and jaggies and the like hardly occur. On the other hand, there is a drawback that the entire image tends to be blurred around an edge portion that does not satisfy the assumption that it changes linearly.
[0005]
The cubic convolution method defines an interpolation function approximating a sinc function (sin (x) / x) based on the sampling theorem, and 16 pixels (X, Y directions) near a point where the enlarged pixel is inversely mapped. This is a method in which a pixel value after enlargement is obtained by a convolution operation of the above-mentioned approximate interpolation function (4 pixels each). This method has a relatively good image quality as compared with the above two methods, but has the disadvantage that the reference range is large and the amount of calculation is large. In addition, since the high range has a characteristic of being slightly emphasized, there is a disadvantage that a light jaggy occurs at an edge portion and a noise component is emphasized.
[0006]
As an attempt to solve these problems, for example, new systems such as Patent Literature 1, Patent Literature 2, Patent Literature 3, and Patent Literature 4 have been proposed. In the technique described in Patent Document 1, an original image is first binarized, and the direction of an oblique component included in the original image is determined from the binary image to coincide with a two-dimensional pattern (matrix data) prepared in advance. Then, an interpolation process is performed along the obtained oblique direction. The other parts are subjected to linear interpolation processing. However, since the direction of the oblique component is determined after binarizing the original image with a predetermined threshold value, it is effective for an edge having a large density difference, but it is effective for reproducing an edge having a small density difference. There's a problem.
[0007]
In the technique described in Patent Document 2, a pixel value of an arrangement file is determined based on a dot arrangement file (enlarged pattern file) for each integral magnification corresponding to a pattern file on a one-to-one basis and a pixel value at which position of a pixel block. An enlarged image block is generated based on such pixel reference information. Furthermore, when there is no matching pattern, an enlarged image block is simply generated using the target pixel of the current image block. However, in Patent Literature 2, since an enlarged block is uniquely generated only by pattern matching, the image quality of the enlarged image is determined by the number of pattern files and dot arrangement files. In other words, many pattern files are required in advance to improve the image quality. Therefore, it is not realistic to prepare many pattern files in advance.
[0008]
In the technique described in Patent Document 3, as a method of detecting the degree of change in the original image, edge detection is performed by an edge detection filter, and an edge pixel is defined based on the edge detection result. If it is determined that the pixel is an edge pixel, enlargement is performed by the M-cubic method in which the shape of the cubic function of the cubic convolution method is adjusted; otherwise, enlargement is performed by the nearest neighbor method. However, since the edge portion having a large degree of change is performed by the M-cubic method, there is a drawback that the image quality characteristics of the cubic convolution method are followed, and jaggies are generated and noise components are emphasized.
[0009]
In the technique described in Patent Document 4, the properties of an image are measured in advance by a density histogram for each block, and based on the result, a first interpolation (a pattern matching method or a nearest neighbor method (NN method)) and a second interpolation are performed. (Cubic convolution method or M-cubic method) is superimposed. In the pattern matching method, which is one of the first interpolation processes, when a pattern (binary pattern) generated from the current image block matches a predetermined angle pattern, a predetermined block is used using a reference block including the current image block. An interpolated pixel block for the target pixel is generated by a rule. However, also in Patent Document 4, there is a problem in that jaggies occur and noise components are emphasized because of superposition with the M-cubic method.
[0010]
In contrast to such a conventional enlargement method, there is a method described in Japanese Patent Application No. 2002-76896, for example, as an enlargement method with high image quality. This method detects an accurate edge direction and generates an enlarged image from a pixel value corresponding to the edge direction, without causing image quality defects such as blur and jaggy in the input image as much as possible. Enlargement processing can be performed. However, since processing takes time, it has been desired to reduce the processing load while maintaining high image quality and to reduce the processing time by high-speed processing.
[0011]
[Patent Document 1]
JP 2000-228723 A
[Patent Document 2]
JP 2001-188900 A
[Patent Document 3]
JP 2000-188681 A
[Patent Document 4]
JP-A-2002-165089
[0012]
[Problems to be solved by the invention]
The present invention has been made in consideration of the above-described problems of the related art, and has high image quality and a low processing load without causing image quality defects such as blurring and jaggy in an input image as much as possible. It is an object of the present invention to provide an image processing apparatus and an image processing method capable of performing light and high-speed enlargement processing. It is another object of the present invention to provide an image processing program for causing a computer to execute the functions or the image processing method of such an image processing device, and a computer-readable storage medium storing such an image processing program. Things.
[0013]
[Means for Solving the Problems]
According to the present invention, in an image processing apparatus and an image processing method for performing image enlargement processing, a plurality of enlargement methods for enlarging an image are prepared, and a feature amount of an image region of a predetermined size including a pixel of interest is prepared. An enlarged image area corresponding to the image area is generated by switching the plurality of enlargement methods provided in the enlarged image area generating means in accordance with the calculated characteristic amount, and calculating the obtained enlarged image area. An output image is generated by arranging the image by an image arranging means. As described above, since the enlargement method is switched for each image region of a predetermined size to be subjected to the enlargement process according to the feature amount of the image region, for example, an enlargement unit having a different processing load is selected according to the feature amount, and the feature is selected. High-quality enlargement processing is performed for image areas that affect image quality, such as edge parts, depending on the amount, and processing such as generating an enlarged image area using flattening means with a small processing load can be performed for flat areas. However, high-speed enlargement processing can be performed as a whole.
[0014]
As the feature amount calculated by the area feature amount calculating means, it is possible to calculate the degree of unevenness in the image region from each pixel value in the image region, or to calculate the change direction in the image region. For a color image, a feature amount is calculated for each color component in the color space, and data of one color component in the color space is selected based on the calculated one or more feature amounts, and the color is calculated. The feature amount of the image region in the component data can be used as the feature amount of the image region.
[0015]
As the enlarging means provided in the enlarged image area generating means, for example, an enlarging means for generating an enlarged image area using the characteristic amount of the image area, the characteristic amount of the neighboring area of the image area, and the pixel value in the neighboring area is used. The enlargement unit is selected for an image region in which the feature amount calculated by the region feature amount calculation unit satisfies a predetermined condition, for example, an image region that requires high-quality enlargement processing including an edge or the like. Enlargement processing can be performed.
[0016]
The arrangement of the enlarged image areas by the image arranging means is arranged so that they do not overlap each other, and the enlarged image areas are sequentially arranged so as to overlap, and for the overlapping pixels, the sum of the overlapping pixel values is calculated and the number of overlapping pixels is calculated. It can be configured to divide.
[0017]
Further, according to the present invention, in an image processing apparatus and an image processing method for performing an image enlargement process, a feature amount of an image region having a predetermined size including a pixel of interest is calculated by an area feature amount calculating unit, and the calculated feature amount is calculated. An enlarged image area corresponding to the image area that satisfies a predetermined condition is generated by an enlarged image area generation unit, and the input image is enlarged by an enlarged image generation unit using an enlargement method different from the enlargement method of the image area. Is generated, and the enlarged image area is arranged at a corresponding position on the enlarged image by the image arrangement means. In this way, as a whole, the enlargement processing is performed using the enlargement method with a light processing load by the enlarged image generation means, and only the portion important to the image quality such as the edge portion is enlarged by the enlarged image area generation means. Is generated and placed. As a result, the processing load can be reduced as compared with the case where the enlargement process is performed on the entire image using a high-quality enlargement method, and the enlargement process can be performed at high speed while maintaining high image quality.
[0018]
Also in the case of the present invention, as the feature amount calculated by the area feature amount calculating means, it is possible to calculate the degree of unevenness in the image region from each pixel value in the image region or calculate the change direction in the image region. . For a color image, a feature amount is calculated for each color component in the color space, and data of one color component in the color space is selected based on the calculated one or more feature amounts. The feature amount of the image region in the data of one color component can be used as the feature amount of the image region in all color components. The enlargement means provided in the enlarged image area generation means includes, for example, an enlargement means for generating an enlarged image area using the characteristic amount of the image area, the characteristic amount of the neighborhood area of the image area, and the pixel value in the neighborhood area. The enlargement unit is selected for an image region in which the feature amount calculated by the region feature amount calculation unit satisfies a predetermined condition, for example, an image region that requires high-quality enlargement processing including an edge or the like. To perform enlargement processing. The enlarged image generating means for performing the enlarging process of the entire image can be configured to generate an enlarged image for each input image or every several lines in the input image.
[0019]
The arrangement of the enlarged image area by the image arranging means is such that the enlarged image is rewritten, and the enlarged image area is sequentially arranged so as to overlap the enlarged image, and for the overlapping pixels, the sum of the overlapping pixel values is calculated and divided by the number of overlaps. can do.
[0020]
Furthermore, the present invention is an image processing program for causing a computer to execute an image enlargement process, wherein the function of the image processing apparatus according to any one of claims 1 to 20 or the function of the image processing apparatus according to claim 21 or 22 is provided. The image processing method described above is executed by a computer. Thus, the function of the above-described enlargement processing can be executed by the computer. Further, according to the present invention, a computer-readable storage medium storing such an image processing program can be provided.
[0021]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 is a block diagram showing a first embodiment of an image processing apparatus and an image processing method according to the present invention. In the figure, 1 is an image processing unit, 2 is an image data storage unit, 11 is an image block setting unit, 12 is an image block feature amount calculation unit, 13 is an enlarged image block generation unit, 14 is an image block arrangement unit, and 21 is a The first generation unit 22 is a second generation unit.
[0022]
The image data storage unit 2 has a function of temporarily storing the image data until the image processing unit 1 enlarges the image data, or temporarily stores the image data after the resolution conversion or the enlargement processing is output to an output device (not shown). The storage device has a function of storing the information in the storage device. The image data is data described in an image format (for example, BMP, TIFF, PNG, etc.) that can be processed by the image processing apparatus, and is captured by a digital camera or scanner (not shown), created by a personal computer, or the like. This is image data or the like created by an application program that processes editing or the like. The enlarged image data has the same image format.
[0023]
The image processing unit 1 includes an image block setting unit 11, an image block feature amount calculating unit 12, an enlarged image block generating unit 13, and an image block arranging unit 14. In the meantime, reading of image data to be enlarged, writing of enlarged image data generated by the enlargement processing, and the like are performed.
[0024]
The image block setting unit 11 sets predetermined block sizes required for processing in the image block feature amount calculation unit 12 and the enlarged image block generation unit 13, and sets input image data stored in the image data storage unit 2. , Image blocks of respective block sizes including the pixel of interest are sequentially cut out and transmitted to the image block feature amount calculation unit 12 and the enlarged image block generation unit 13.
[0025]
The image block feature amount calculation unit 12 calculates the image feature amount in the attention area in the image block sequentially cut out by the image block setting unit 11 from each pixel value in the image block including the attention area or the periphery of the attention area. I do. As the image feature amount, for example, an edge strength and an edge angle of the attention area can be calculated as described later. However, the present invention is not limited to this. For example, an average value of each pixel value of the attention area is obtained, and a value (for example, a standard deviation or a variance) representing variation of each pixel value of the attention area with respect to the average value is obtained. , Various image feature amounts may be calculated.
[0026]
The enlarged image block generation unit 13 includes a plurality of enlargement units, here, a first generation unit 21 and a second generation unit 22, and performs a plurality of enlargement according to the image feature amount calculated by the image block feature amount calculation unit 12. The means is switched to generate an enlarged image block corresponding to the attention area. For example, when the image block feature amount calculation unit 12 calculates the edge strength as a numerical value, the feature amount is compared with a predetermined value, and either of the first generation unit 21 or the second generation unit 22 is determined according to the comparison result. It is possible to generate an enlarged image block by switching between the two. Here, the enlargement process is performed by the first generator 21 when the feature value is larger than the predetermined value, and by the second generator 22 when the feature value is smaller than the predetermined value. Of course, the feature amount used for switching the enlargement means is not limited to the edge strength, and the above-described standard deviation, variance, or other various feature amounts can be used. Although the enlargement method in the first generation unit 21 and the second generation unit 22 is arbitrary, the first generation unit 21 that operates when the edge strength indicates a large numerical value as in the above-described example is an enlargement that prioritizes image quality. An approach is desirable. Further, in this case, the second generation unit 22 is desirably an enlargement method that can perform processing at a higher speed than the first generation unit 21, whereby the overall processing time can be reduced.
[0027]
The image block arranging unit 14 sequentially arranges the enlarged image blocks generated by the enlarged image block generating unit 13 and outputs enlarged image data whose resolution has been converted or enlarged to the image data storage unit 2. Although the method of arranging the enlarged image blocks will be described later, for example, in addition to the method of sequentially arranging the enlarged image blocks, the enlarged image blocks are sequentially arranged so as to overlap each other and overlapped, and the sum of the overlapping pixel values is divided by the number of overlapped pixels. It can be configured to calculate a value.
[0028]
FIG. 2 is a flowchart illustrating an example of an operation of the image processing apparatus and the image processing method according to the embodiment of the present invention. Here, a description will be given including a specific example. In step S41, the input image data stored in the image data storage unit 2 is processed by the image block setting unit 11, the image block feature amount calculation unit 12, and the enlarged image block generation unit 13 for each block. Set the size. Further, an image block having the set block size is cut out from the input image data. FIG. 3 is an explanatory diagram of an image block to be set and cut out by the image block setting unit 11. The pixel in the center, which is finely hatched upward and to the right, is the region of interest, and the surrounding pixels, which are hatched roughly upward and to the left, are the peripheral region. Here, as an example, the attention area has a size of 2 × 2 pixels, and the peripheral area including the attention area has a size of 4 × 4 pixels. In this case, the set block size is a 6 × 6 pixel size image block shown in FIG. 3A in the case of 2 × enlargement, and an 8 × 8 pixel size shown in FIG. 3B in the case of 4 × enlargement. An image block of the size is cut out.
[0029]
In S42, the image block feature amount calculation unit 12 calculates the edge strength G of the region of interest (the region of 2 × 2 pixels hatched in FIG. 3 with a fine upward-sloping hatch) in the cut-out image block by the following equation ( It is calculated in 1). Note that {a, b, c, d} are each pixel value in the attention area. If the input image data is not a grayscale image but a color image in, for example, an RGB color space, an edge of each image block for each color component in each of the R, G, and B color spaces with respect to the region of interest is calculated using Expression (1) The intensities Gr, Gg, Gb are calculated, an image block of a color component having the largest edge intensity among the Gr, Gg, Gb is selected, and the edge intensity is determined as the edge intensity (common to all color components) of the attention area. And
gx = (a + c−b−d) / 2
gy = (a + b−c−d) / 2
G = gx × gx + gy × gy (1)
The edge strength is not limited to the one calculated by the above equation (1), but may be calculated by the following equation (2).
G = | gx | + | gy | (2)
[0030]
In S43, the edge strength G of the attention area calculated in S42 is compared with a predetermined threshold Th. If the edge strength G is larger than the threshold Th, the region of interest is an edge portion or a portion having a large change in the input image. Therefore, an enlarged image block generation process for preserving the image feature of the region of interest is performed in S44. This is performed in steps S47 to S47. If the edge strength G of the attention area is smaller than the threshold value Th, a process of generating an enlarged image block having a smaller processing load than in the processes of S44 to S47 is performed in S48.
[0031]
In S44, the image block feature quantity calculation unit 12 shows the attention area determined to have the edge strength G greater than the threshold Th and the edge angle の of the reference area in one or a plurality of peripheral areas including the attention area below. Each is calculated by equation (3).
Θ = arctan (gy / gx) (3)
Note that gx and gy are the values calculated in S42, respectively. Then, the edge direction θ of the region of interest is estimated from the plurality of obtained edge angles Θ. For example, it is possible to estimate the edge direction θ by performing an operation such as taking an average value of a plurality of obtained edge angles Θ. Of course, the method of estimating the edge direction θ is arbitrary.
[0032]
In S45, the image block feature amount calculation unit 12 selects the first and second edge patterns using the edge direction θ estimated in S44 and the pixel distribution pattern of the attention area. Here, the first and second edge patterns are prepared in advance for each edge direction and each pixel distribution pattern as shown in FIG. 9 described later. For example, the storage unit or the image data storage unit 2 in the image processing unit 1 (not shown) Can be stored as table information in a part of the table. The details will be described later.
[0033]
In S46, the first generation unit 21 of the enlarged image block generation unit 13 uses the predetermined emphasis pattern (hereinafter, referred to as an emphasis kernel) having a size and an element value according to the enlargement ratio, by the image block setting unit 11, The image data of the region of interest and the peripheral region in the cut-out image block is emphasized. The emphasis kernel and emphasis processing using the emphasis kernel will be described later.
[0034]
In S47, the first generation unit 21 of the enlarged image block generation unit 13 determines the edge direction θ of the attention area estimated by the image block feature amount calculation unit 12, the selected edge pattern, and the attention calculated in S46. An enlarged image block for the region of interest is generated using the emphasized pixel values of the region and the peripheral region.
[0035]
In this manner, when the edge strength G of the attention area is high in S43, an enlarged image block that preserves such an edge portion or a feature that the change is large is generated. As a result, it is possible to generate a high-quality enlarged image block with less blurring and jaggies for an edge portion and the like.
[0036]
On the other hand, in S48, the second generation unit 22 of the enlarged image block generation unit 13 generates an enlarged image block for the attention area for which the edge strength G is determined to be smaller than the threshold Th in the image block feature amount calculation unit 12 in S43. . The enlarged image block generation processing in the second generation unit 22 is different from the enlarged image block generation processing in S44 to S47, and may be an enlargement processing with a small processing load such as nearest neighbor interpolation. In other words, when the edge strength G is small, it is possible to obtain sufficient image quality even in the process of generating an enlarged image block with a smaller processing load without performing the above-described complicated and time-consuming processing. . Therefore, when the edge strength G is small, the second generation unit 22 of the enlarged image block generation unit 13 is selected to generate an enlarged image block with a small processing load, thereby shortening the processing time as a whole. .
[0037]
In S49, the image block arranging unit 14 generates the enlarged image block (the enlarged image block generated in the processing of S44 to S47 and the enlarged image block generated in the processing of S48) for the attention area generated by the enlarged image block generating unit 13. Are sequentially arranged by a predetermined method described later, and the arranged image blocks are stored in the image data storage unit 2 as a part of enlarged image data to be output.
[0038]
In S50, it is determined whether the generation of the enlarged image data to be output with respect to the input image data has been completed. If not completed, the process returns to S41, cuts out the next image block, and repeats the above processing. . If completed, the enlarged image data is output and the enlargement processing ends.
[0039]
The example of the operation of the image processing apparatus and the image processing method according to the embodiment of the present invention has been described above. Next, details of the image block feature amount calculation unit 12, the enlarged image block generation unit 13, and the image block arrangement unit 15, which are main components, will be described.
[0040]
First, the details of the image block feature amount calculation unit 12 will be described. Here, as shown in FIG. 3 described above, the region of interest is a 2 × 2 pixel size block, and the peripheral region including the region of interest is a 4 × 4 pixel size block. FIG. 4 is a block diagram illustrating a configuration example of the image block feature amount calculation unit 12. In the figure, 31 is an edge strength calculation unit, 32 is an edge direction estimation unit, and 33 is an edge pattern selection unit. In the example illustrated in FIG. 4, the image block feature amount calculation unit 12 includes an edge strength calculation unit 31, an edge direction estimation unit 32, and an edge pattern selection unit 33.
[0041]
As described above, the edge strength calculation unit 31 calculates the edge strength G of the attention area in the image block cut out by the image block setting unit 11 using the above equation (1), and calculates the calculated edge strength G by a predetermined value. Is compared with the threshold value Th, and an enlargement processing method for the attention area is selected. When the edge strength G of the attention area is smaller than the threshold value Th, the processing in the edge direction estimation unit 32 and the edge pattern selection unit 33 is not performed, and the enlargement processing in the second generation unit 22 of the enlarged image block generation unit 13 is performed. Is performed.
[0042]
Hereinafter, details of the processing of the edge direction estimating unit 32 and the edge pattern selecting unit 33 will be described. FIG. 5 is a diagram illustrating a specific example of the attention area and the peripheral area and an example of the edge direction of the attention area. FIG. 5A shows an example of the attention area and the peripheral area. Each rectangle is a pixel, and each number indicates a pixel value. The attention area is a pixel {a, b, c, d} = {15, 104, 86, 203} surrounded by a thick line frame. The flow of the edge direction estimation processing in the edge direction estimation unit 32 will be described using the example shown in FIG.
[0043]
FIG. 6 is a flowchart illustrating an example of the edge direction estimation processing in the edge direction estimation unit 32. In S61, the edge angle の of the region of interest surrounded by the thick line frame in FIG. 5A is calculated by the above-described equation (3). For example, in the case of the attention area {a, b, c, d} = {15, 104, 86, 203} as shown in FIG. 5 (A), gx and gy are gx = − 103, gy = -85. Therefore, the edge angle に お け る in the attention area shown in FIG. 5A is Θ = −140.5 °. The direction of the edge angle Θ is indicated by a broken arrow in FIG.
[0044]
Further, an angle direction is set as shown in FIG. 5B, and it is checked which of the angle ranges of 22.5 ° (eight directions) includes the obtained edge angle 含 ま. In this case, the edge angle ０ is 0 ° or ± 180 ° in the direction 0, 22.5 ° or −157.5 ° in the direction 1, 45 ° or −135 ° in the direction 2, 67.5 ° or The direction of -112.5 ° is direction 3, the direction of 90 ° or -90 ° is direction 4, the direction of 112.5 ° or -67.5 ° is direction 5, the direction of 135 ° or -45 ° is direction 6. , 157.5 ° or −22.5 ° as direction 7, and ranges of ± 11.25 ° of these angles are the respective directions. Since the edge angle Θ (= −140.5 °) in the above specific example is included in the range of −135 ° ± 11.25 °, the edge angle is the direction 2.
[0045]
In S61, the angle reference number, which is a variable for counting the reference number of the edge angle used for estimating the edge direction of the attention area in S66 described later, is set to 1.
[0046]
In S62, a reference region used for estimating the edge direction is selected from the peripheral regions (regions outside the thick line frame) shown in FIG. 5A according to the edge angle の of the region of interest calculated in S61. FIG. 7 is an explanatory diagram of an example of a reference region used for estimating an edge direction. 2 × 2 pixels indicated by a thick line frame in the drawing are reference regions. 7A shows a case where the direction of the edge angle obtained in S61 is 0, FIG. 7B shows a case where the direction is 4, and FIG. 7C shows a case where the direction is other than that. Note that the number of reference areas in the case of the directions 0 and 4 shown in FIGS. 7A and 7B is two, and the number of the reference areas in the directions other than the directions 0 and 4 shown in FIG. The number is four. In the specific example shown in FIG. 5, since the direction of the edge angle is direction 2, the four reference regions shown in FIG. 7C are selection candidates. Note that the selection of the reference area is not limited to that shown in FIG. 7. For example, in the case of FIG. 7C, the number of reference areas is set to 8, or the reference area is set according to each direction. May be.
[0047]
In S63, the edge angle Θ is calculated for one of the reference regions selected in S62 in accordance with the equation (3) in the same manner as in S61. In S64, the edge angle of the reference region calculated in S63 is compared with the edge angle of the region of interest calculated in S61. If the difference between the two edge angles is smaller than the preset threshold value Δth, the angle reference number is increased by 1 in S65, and the process proceeds to S66. If the difference between the two edge directions is larger than the preset threshold value Δth, it is determined that the reference area is not suitable for the estimation of the edge direction, and the process proceeds directly to S66. In S66, it is determined whether the calculation of the edge angles of all the selectable reference areas has been completed. If it has been completed, the process proceeds to S67. If there is a reference area that has not been completed yet, the processes from S63 to S65 are repeated.
[0048]
In the specific example shown in FIG. 5, the edge angle {U = -149.4 ° from the upper reference region {86, 203, 171, 211}, and the edge angle from the left reference region {10, 15, 20, 86}. {L = -131.2 °, edge angle {D = -175.2 ° from lower reference region {1, 102, 15, 104}, edge angle {R} from right reference region {104, 215, 203, 219} = −141.0 °. The edge angle of the attention area Θ = −140.5 ° is compared with the edge angle of each reference area, and the number of reference areas whose difference is smaller than the threshold value Θth is counted as the angle reference number.
[0049]
In S67, the sum of the edge angle of the attention area and the edge angle of the reference area determined to be suitable for the edge direction estimation in S64 is calculated, and the average edge angle obtained by dividing the sum of the edge angles by the number of angle references is calculated. The estimated edge direction of the area is used. FIG. 8 is an explanatory diagram of an example of the estimated edge direction in the attention area shown in FIG. For example, in the above-described specific example, assuming that the difference from the edge angle of the attention area for all the reference areas is smaller than the threshold value Δth, the sum of the edge angles obtained from the attention area and the four reference areas is −737.3 °. By dividing by the angle reference number 5, the average edge angle can be obtained as -147.5 °. In this case as well, as in the case of the edge direction of the region of interest described above, for example, if the normalization is performed in any one of eight directions, −147.5 ° becomes the direction 1. This is defined as an estimated edge direction.
[0050]
In the above description, the case where the input image data is a grayscale image is described, but it is needless to say that the present invention is not limited to this. For example, in the case of a color image in the RGB color space, the color space data selected based on the strength of the edge angles Gr, Gg, and Gb in the data of each color component as described above in S32 of the flowchart of FIG. The above process may be performed. By doing so, it is possible to suppress a decrease in image quality such as a color shift at an edge portion of enlarged image data in a color image.
[0051]
In the above description, the edge angle is calculated using Equation (1) in the attention area and the reference area, and then normalized to any one of the eight directions. May be normalized to a larger number of directions such as 12 directions (every 15.0 °) and 16 directions (every 12.25 °).
[0052]
Next, the operation of the edge pattern selection unit 33 will be described in detail. FIG. 9 is an explanatory diagram of an example of the edge pattern table. The edge pattern selection unit 33 converts an edge pattern using, for example, an edge pattern table as shown in FIG. As shown in FIG. 9, in the edge pattern table, one or several first edge patterns corresponding to the pattern size of the attention area are registered for each edge direction (here, eight directions). A pattern having a pattern size larger than the pattern size of the first edge pattern is registered as a second edge pattern corresponding to the pattern.
[0053]
The edge pattern selecting unit 33 selects a first edge pattern corresponding to the edge direction of the region of interest estimated by the edge direction estimating unit 32 using, for example, an edge pattern table as shown in FIG. A second edge pattern corresponding to one edge pattern is determined.
[0054]
Specifically, in the case of the attention area and its surrounding area surrounded by the thick line frame shown in FIG. 5A, the estimated edge direction for the attention area is determined by the edge direction estimation unit 11 for all the reference areas as described above. If the difference from the edge angle of the region is smaller than the threshold value Δth, it is determined that the direction is direction 1. According to the estimated edge direction (direction 1) of the attention area, a pattern candidate corresponding to the edge pattern of the attention area is selected from the edge pattern table shown in FIG. In this case, four patterns from pattern 0 to pattern 3, which are the first edge patterns in the direction 1, are selected, and become the first edge pattern candidates for the attention area shown in FIG.
[0055]
Next, the edge pattern selection unit 33 selects any one of the patterns 0 to 3 as the first edge pattern for the attention area, as described below. FIG. 10 is an explanatory diagram of a specific example of the method of selecting the first edge pattern corresponding to the attention area shown in FIG. In this specific example, since the estimated edge direction is direction 1, four edge pattern candidates in the first edge pattern from the edge pattern table shown in FIG. 9 are determined as the edge patterns for the attention area shown in FIG. chosen. The selected edge pattern candidate is shown in FIG.
[0056]
Each of the edge pattern candidates is converted into a bit pattern as shown in FIG. Here, the white portion is set to 0, and the rest is set to 1 to form a bit pattern. However, the edge pattern table shown in FIG. 9 may be converted into a bit table and registered in advance. In this case, the bit pattern conversion process can be omitted.
[0057]
Next, according to equation (4), the average pixel value in the region of interest is calculated, the average value is subtracted from each pixel value in the region of interest, and the sign is used as the pixel value pattern of the region of interest.
Mean = (a + b + c + d) / 4
a_sign = a-Mean
b_sign = b-Mean
c_sign = c-Mean
d_sign = d-Mean (4)
In the case of the example shown in FIG. 5A, Mean = (15 + 104 + 86 + 203) / 4 = 102, a_sign = −87, b_sign = 2, c_sign = −16, d_sign = 101, and the sign is set to the pixel value pattern. Then, the result is as shown in FIG. Further, the pixel value pattern is converted into a bit pattern as shown in FIG.
[0058]
Next, pattern matching is performed between the bit pattern of each edge pattern candidate shown in FIG. 10B and the bit pattern of the attention area shown in FIG. Select one pattern. In this case, pattern 2 is selected as the first edge pattern for the region of interest (FIG. 10E).
[0059]
Finally, when the first edge pattern is selected, a second edge pattern prepared in advance so as to correspond to the first edge pattern is determined. In this case, a second edge pattern (direction 1, pattern 2) corresponding to pattern 2 in direction 1 of the first edge pattern is selected from the edge pattern table shown in FIG. 9 (FIG. 10F). The second edge pattern is used at the time of generating an enlarged image block for the attention area in the enlarged image block generation unit 2 described later.
[0060]
Note that the first and second edge patterns are not limited to those shown in FIG. 9. For example, an edge pattern different from that shown in FIG. 9 may be used depending on the type of input image data. The number of first and second edge pattern candidates at an angle may be increased or decreased. Of course, there is no need to configure a table, and it is sufficient that the second edge pattern can be derived from the specified first edge pattern.
[0061]
Next, the enlarged image block generation unit 13 will be described in detail. First, details of a process of generating an enlarged image area in the first generation unit 21 of the enlarged image block generation unit 13 will be described. The first generation unit 21 of the enlarged image block generation unit 13 first uses the image enhancement kernel set in the image block setting unit 11 to enhance the image as shown in FIG. The block setting unit 11 emphasizes the contrast of the image data of the region of interest and the peripheral region in the image block extracted by the block.
[0062]
FIG. 11 is an explanatory diagram of a specific example of the enhancement kernel used in the first generation unit 21 of the enlarged image block generation unit 13. FIG. 12 is an explanatory diagram in the case where the pixel P is enhanced by the enhancement kernel shown in FIG. The enhancement kernel (kernel 0) shown in FIG. 11 (A) has a magnification of 2 ×, and the enhancement kernel (kernel 1) shown in FIG. 11 (B) has a magnification of 4 ×, and the enhancement shown in FIG. 11 (C). The kernel (kernel 2) is used when the magnification is eight times. In the figure, the center pixel value and the hatched pixels arranged on the upper, lower, left and right sides of the pixel are used, and the emphasis processing is performed using the weight described below.
[0063]
In the case of performing the enlargement processing, when the processing of 4 times or 8 times is performed, the pixels having the characteristics of the original image are two pixels and pixels located at positions separated by four images. Therefore, as shown in FIG. 11, pixels that are farther away from each other are selected as the magnification is higher, and are used for the emphasis processing. Of course, the same applies to a magnification of 16 times or more. Further, the positions of the pixels to be referred to are not limited to the upper, lower, left, and right as shown in FIG. For example, elements of the kernel (pixels to be referred to) and elements such as including a pixel in an oblique direction, referring to a pixel further away, and changing a kernel to be used depending on the type and size of input image data, and the like. A different distance from the enhancement kernel shown in FIG. 11 may be used.
[0064]
For example, when an enhancement kernel as shown in FIG. 11A is used, and as shown in FIG. 12A, for example, when enhancement processing is performed on a central reference pixel P, pixels a, Reference is made to b, c, and d, and 1.6 is used as the weight for the central target pixel, and 0.15 is used for the peripheral reference pixels. Accordingly, the pixel value P ′ of the pixel P can be calculated according to the following equation (5).
Attention pixel value P ′ = 1.60 × P−0.15 × (a + b + c + d) (5)
[0065]
Note that, for example, even when the enhancement kernel as shown in FIG. 11B is used, similarly, when the enhancement processing of the reference pixel P is performed as shown in FIG. The same as the equation (5), referring to the existing pixels a, b, c, and d, using 1.2 as the weight for the target pixel of interest and 0.05 for the peripheral reference pixels. Then, the pixel value P ′ of the pixel P may be calculated.
[0066]
FIG. 13 is an explanatory diagram of the contrast enhancement processing by the enhancement kernel when the input image data is enlarged by 8 times. As will be described later, since the method of the enlargement process by the first generation unit 21 is performed every two times, in the case of performing the enlargement of eight times, the enlargement is first performed twice, and the image data that has been enlarged twice is used. The image data expanded four times is further expanded by two times to generate image data expanded four times, and the image data expanded four times is further expanded twice, and finally expanded to eight times. Obtained image data. The emphasis processing by the above-mentioned emphasis kernel is performed before each double enlargement.
[0067]
That is, when the input image data shown in FIG. 13A is enlarged twice, the contrast of the input image data is enhanced using the kernel 0 shown in FIG. Thus, the double enlarged data shown in FIG. 13B is generated. Next, when the image data that has been enlarged twice as shown in FIG. 13B is enlarged twice, contrast enhancement is performed on the twice enlarged image data using the kernel 1 shown in FIG. Thereafter, the data is enlarged by a factor of two to generate quadruple enlarged data shown in FIG. Further, contrast enhancement is performed on the quadrupled data shown in FIG. 13C by using the kernel 2 shown in FIG. 11C, and then the data is enlarged by 2 times and enlarged by 8 times shown in FIG. 13D. Generate data. The emphasis kernel shown in FIG. 11 is used in this way, and performs emphasis processing on the data at each enlargement stage.
[0068]
Next, the first generation unit 21 of the enlarged image block generation unit 13 actually generates an enlarged image block. The details of the process of generating the enlarged image block will be described. The first generation unit 21 of the enlarged image block generation unit 13 compares the second edge pattern obtained by the edge pattern selection unit 33 of the image block feature amount calculation unit 12 with the pixel value that has been contrast-enhanced in the above-described image enhancement processing. To generate an enlarged image block for the attention area.
[0069]
FIG. 14 is a flowchart illustrating an example of an enlarged image block generation process in the first generation unit 21 of the enlarged image block generation unit 13. In S71, an enlarged image block of 3 × 3 pixels is first used by using the attention area subjected to contrast enhancement by the image enhancement processing and the second edge pattern selected by the edge pattern selection unit 33 of the image block feature amount calculation unit 12. Generate FIG. 15 is an explanatory diagram of a specific example of a process of generating an enlarged image block of 3 × 3 pixels. FIG. 15A illustrates an example of the attention area and the peripheral area illustrated in FIG. Further, FIG. 15B shows the second edge pattern (the pattern 2 in the direction 1 in FIG. 9) obtained, for example, as described in FIG. 10 corresponding to the attention area. In FIG. 15B, pixels are denoted by p0 to p8. Based on the attention area {a, b, c, d} shown in FIG. 15A, the pixel values of the pixels p0 to p8 are calculated by the following equation.
p0 = a
p1 = (a + c) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (b + d) / 2
p8 = d
By performing such calculations, an enlarged image block of 3 × 3 pixels can be generated as shown in FIG.
[0070]
FIG. 16 is an explanatory diagram of an example of a calculation formula used when generating a second edge pattern and an enlarged image block of 3 × 3 pixels. As described above, when the second edge pattern as shown in FIG. 15B is selected, an enlarged image block of 3 × 3 pixels can be generated by the above equation, but this is used. The calculation formula is determined for each selected second edge pattern. The second edge pattern shown in FIG. 16A is the pattern 0 in the direction 1 shown in FIG.
p0 = a
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p1 = 2 × p4-p7
Thus, an enlarged image block of 3 × 3 pixels can be generated.
[0071]
The second edge pattern shown in FIG. 16B is the pattern 1 in the direction 5 shown in FIG. 9, and in this case,
p0 = a
p1 = (a + c) / 2
p2 = b
p3 = (c + d) / 2
p4 = (a + d) / 2
p5 = (a + b) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
Thus, an enlarged image block of 3 × 3 pixels can be generated.
[0072]
The second edge pattern shown in FIG. 16C is the pattern 0 in the direction 2 shown in FIG. 9, and in this case,
p0 = a
p1 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
Thus, an enlarged image block of 3 × 3 pixels can be generated.
[0073]
The second edge pattern shown in FIG. 16D is the pattern 0 in the direction 0 shown in FIG. 9, and in this case,
p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p4 = (p1 + p7) / 2
Thus, an enlarged image block of 3 × 3 pixels can be generated. In the case of other second edge patterns as well, an enlarged image block of 3 × 3 pixels can be generated by performing calculation in accordance with a calculation formula corresponding to each second edge pattern.
[0074]
Returning to FIG. 14, in S72, the estimated edge direction of the region of interest estimated by the edge direction estimating unit 32 of the image block feature amount calculating unit 12 is determined, and the estimated edge directions are directions 1 to 3 and directions 5 to 7 When the estimated edge direction is the direction 0 and the direction 4 (that is, when the edge direction is vertical or horizontal), the process proceeds to S73.
[0075]
In S73, when the estimated edge direction of the region of interest is direction 1 to direction 3 and direction 5 to direction 7, that is, when the edge direction is not vertical or horizontal, the 3 × 3 pixel enlarged image block generated in S71. To generate a 4 × 4 pixel enlarged image block. Here, an enlarged image block of 3 × 4 pixels is first generated from an enlarged image block of 3 × 3 pixels, a focused region of interest and its peripheral region, and then the generated enlarged image block of 3 × 4 pixels. Then, an enlarged image block of 4 × 4 pixels is generated from the attention area where the contrast is enhanced and the surrounding area.
[0076]
FIG. 17 is an explanatory diagram of a specific example of a process of generating an enlarged image block of 3 × 4 pixels. FIG. 17A shows an example of the attention area and the peripheral area shown in FIG. 5A as an example. Further, p0 to p8 in FIG. 17B show the enlarged image block of 3 × 3 pixels shown in FIG. 15C at the center.
[0077]
First, the reference image (r0 to r5) in the peripheral region in which the contrast is emphasized, which is indicated by a bold frame in FIG. 17A, is used together with the enlarged image block of 3 × 3 pixels shown in the center of FIG. Thus, an enlarged image block (q0 to q11) of 3 × 4 pixels shown in FIG. 17C is generated. Each pixel value q0 to q11 of the 3 × 4 pixel enlarged image block is determined by the following formula.
q0 = 0.2 × r0 + 0.8 × p0
q1 = 0.4 × p0 + 0.6 × p1
q2 = 0.6 × p1 + 0.4 × p2
q3 = 0.8 × p2 + 0.2 × r1
q4 = 0.2 × r2 + 0.8 × p3
q5 = 0.4 × p3 + 0.6 × p4
q6 = 0.6 × p4 + 0.4 × p5
q7 = 0.8 × p5 + 0.2 × r3
q8 = 0.2 × r4 + 0.8 × p6
q9 = 0.4 × p6 + 0.6 × p7
q10 = 0.6 × p7 + 0.4 × p8
q11 = 0.8 × p8 + 0.2 × r5
[0078]
FIG. 18 is an explanatory diagram of a method of selecting a reference pixel based on the estimated edge direction. In the example illustrated in FIG. 17, assuming that the estimated edge direction obtained by the image block feature amount calculation unit 12 from the attention area and the peripheral area illustrated in FIG. The positions are three pixels from the upper left to the lower part and three pixels from the lower right to the upper part as enclosed by a thick line frame in FIG. An example of this reference pixel is shown in FIG. 18A. A pixel at such a position is used as a reference pixel when the estimated edge direction of the attention area is from direction 1 (22.5 °) to direction 3 ( 67.5 °). When the estimated edge direction of the attention area is from the direction 5 (112.5 °) to the direction 7 (157.5 °), as shown by a thick line frame in FIG. And three pixels from the upper right to the lower. Thus, the reference pixel is selected according to the estimated edge direction in the attention area. Of course, the selection of the reference pixel is not limited to the selection from two patterns as shown in FIG. 18, and more reference pixel selection patterns may be prepared according to the estimated edge direction. Also, the reference pixel to be selected may be changed according to the estimated edge direction.
[0079]
FIG. 19 is an explanatory diagram of a specific example of the generation processing of the enlarged image block of 4 × 4 pixels. A 4 × 4 pixel enlarged image block is generated using the 3 × 4 pixel enlarged image block generated as described above and the reference pixels (r0 to r7) in the contrast-enhanced peripheral region. FIG. 19A shows the attention area and the reference area in which the contrast is enhanced, and the generated 3 × 4 pixel enlarged image block is shown in the center of FIG. 19B. Here, as the reference pixels r0 to r7, the lower four pixels and the upper four pixels of the contrast-enhanced reference region are used as shown by the thick line frame in FIG. Then, an enlarged image block (s0 to s15) of 4 × 4 pixels shown in FIG. 19C is generated according to the following calculation formula.
s0 = 0.2 × r0 + 0.8 × q0
s1 = 0.2 × r1 + 0.8 × q1
s2 = 0.2 × r2 + 0.8 × q2
s3 = 0.2 × r3 + 0.8 × q3
s4 = 0.4 × q0 + 0.6 × q4
s5 = 0.4 × q1 + 0.6 × q5
s6 = 0.4 × q2 + 0.6 × q6
s7 = 0.4 × q3 + 0.6 × q7
s8 = 0.6 × q4 + 0.4 × q8
s9 = 0.6 × q5 + 0.4 × q9
s10 = 0.6 × q6 + 0.4 × q10
s11 = 0.6 × q7 + 0.4 × q11
s12 = 0.8 × q8 + 0.2 × r4
s13 = 0.8 × q9 + 0.2 × r5
s14 = 0.8 × q10 + 0.2 × r6
s15 = 0.8 × q11 + 0.2 × r7
[0080]
In the above-described processing of generating an enlarged image block of 4 × 4 pixels in S73 of FIG. 14 described above, the enlargement processing is performed according to the flow of processing of 3 × 3 pixel blocks → 3 × 4 pixel blocks → 4 × 4 pixel blocks. Has been described, the processing flow may be from 3 × 3 pixel block → 4 × 3 pixel block → 4 × 4 pixel block, or a 4 × 4 pixel block may be generated from the 3 × 3 pixel block. . Of course, the selection of the reference pixel in these cases will be changed as appropriate.
[0081]
Returning to FIG. 14, in S74, when the estimated edge direction of the attention area is the direction 0 and the direction 4 (that is, in the vertical direction or the horizontal direction), the enlarged image block of 3 × 3 pixels generated in S71 is 4 × Generate an enlarged image block of 4 pixels. FIG. 20 is a diagram illustrating an example of a process of generating an enlarged image block of 4 × 4 pixels when the estimated edge direction is the direction 0 and the direction 4. When the estimated edge directions of the attention area are the directions 0 and 4, the pixels (A, B, C, D) in the contrast-enhanced peripheral area shown in FIG. The left and right reference pixels r0 to r5 are calculated using the following equations.
r0 = A
r1 = B
r2 = 0.5 × A + 0.5 × C
r3 = 0.5 × B + 0.5 × D
r4 = C
r5 = D
In the center of FIG. 20B, the enlarged image blocks p0 to p8 of 3 × 3 pixels generated in S71 of FIG. 14 are shown.
[0082]
When the enlarged image block of 3 × 3 pixels and the reference pixels r0 to r5 are obtained in this way, the enlarged image blocks q0 to q11 of 3 × 4 pixels are obtained by using these, by the calculation method described in FIG. Generate. This is shown in FIG. Further, the enlarged image blocks q0 to q11 of 3 × 4 pixels generated in this way and the upper and lower four pixels of the contrast-enhanced peripheral region shown in FIG. The enlarged image block (s0 to s15) of 4 × 4 pixels shown in FIG.
[0083]
As described above, by performing the processing of S71 to S74 shown in FIG. 14, an enlarged image of 4 × 4 pixels for the attention area for which the edge strength G is determined to be larger than the threshold Th in the image block feature amount calculation unit 12 Blocks (for example, s0 to s15 shown in FIG. 19C or FIG. 20D) are generated.
[0084]
Next, for the attention area for which the edge strength G is determined to be smaller than the threshold value Th in the image block feature quantity calculation unit 12, the second attention unit 22 of the enlarged image block generation unit 13 outputs the attention area as described above. Instead of the enlargement processing in the first generation unit 21 of the enlargement image block generation unit 13 that saves the image feature of, the processing load of the processing load is reduced and a simpler enlarged image block generation process is performed. For example, as an example, an enlarged image block of 4 × 4 pixels can be generated from a region of interest of 2 × 2 pixels by nearest neighbor interpolation. Of course, the enlargement method used in the second generation unit 22 is arbitrary, and any method may be used as long as the processing load is lighter than that of the first generation unit 21.
[0085]
As described above, the input image data can be enlarged with high image quality by selecting the enlargement processing performed in the enlarged image block generation unit 13 in accordance with the image feature amount calculated by the image block feature amount calculation unit 12. Processing load can also be reduced. Specifically, by using an enlargement method with a smaller processing load than the first generation unit 21 in the second generation unit 22, the processing load of the entire enlargement process is reduced, the processing time is shortened, and the high-speed enlargement process is performed. Can be realized. Here, it is assumed that the enlarged image block generation unit 13 has two enlargement methods, and either one is selectively used. However, a plurality of enlargement methods are provided, and the image block feature amount calculation is performed. It is also possible to adopt a configuration in which any one of a plurality of enlargement methods is selected according to various image feature amounts calculated by the unit 12 to perform enlargement processing.
[0086]
Next, the image block arrangement unit 14 will be described. The image block arranging unit 14 sequentially arranges the enlarged image blocks for the attention area generated by the enlarged image block generating unit 13 by a predetermined method. FIG. 21 is an explanatory diagram of a specific example in which an enlarged image block of 4 × 4 pixels generated by the enlarged image block generation unit 13 is arranged. In the example shown in FIG. 21, the enlarged image block 0 and the enlarged image block 1 sequentially generated are arranged so as to overlap. Each overlapping pixel is arranged so as to take the average of the previous pixel value. Alternatively, each pixel value may be calculated by calculating the sum of overlapping pixels and dividing the sum of the pixel values by the number of overlaps. In this case, the target pixel is selected while being shifted, for example, one pixel at a time, and the enlargement process is performed. Alternatively, they can be arranged side by side without overlapping. In this case, the attention areas may be selected so as not to overlap.
[0087]
Since the enlarged image block of 4 × 4 pixels is generated by either the first generation unit 21 or the second generation unit 22 in the enlarged image block generation unit 13, the image quality may be deteriorated due to the difference in the enlargement method. However, by performing the overlap and averaging as described above, the occurrence of such image quality deterioration can be suppressed.
[0088]
FIG. 22 is a block diagram showing a second embodiment of the image processing device and the image processing method of the present invention. In the figure, the same parts as those in FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. 13 'is an enlarged image block generating unit, 14' is an image block arranging unit, and 15 is an enlarged image generating unit. In the second embodiment, the image processing unit 1 includes an image block setting unit 11, an image block feature amount calculation unit 12, an enlarged image block generation unit 13 ', and an image block arrangement unit 14', together with an enlarged image generation unit. 15 are included. Note that the image block setting unit 11 and the image block feature amount calculation unit 12 have the same functions as those in the above-described first embodiment. Further, the enlarged image block generation unit 13 'and the image block arrangement unit 14' have a configuration corresponding to the enlarged image block generation unit 13 and the image block arrangement unit 14 in the above-described first embodiment. Among these, the enlarged image block generation unit 13 'has the same function as the first generation unit 21 of the enlarged image block generation unit 13 in the above-described first embodiment, and a description thereof will not be repeated.
[0089]
The image block arranging unit 14 ′ assigns an enlarged image block that has been enlarged by the enlarged image block generation unit 13 ′ to the attention area for which the edge strength G is determined to be greater than the threshold Th in the image block feature amount calculation unit 12. The image data is sequentially arranged at corresponding positions on the enlarged image output from the enlarged image generation unit 15 described later, and the image data whose resolution has been converted or enlarged is output to the image data storage unit 2. When sequentially arranging the enlarged image blocks at the corresponding positions on the enlarged image output from the enlarged image generating unit 15, the enlarged image blocks may be arranged so as to be replaced with the respective pixel values of the enlarged image block, or the enlarged image generation may be performed. It may be arranged so as to be superimposed on each pixel value at the corresponding position of the enlarged image output from the unit 15. When the enlarged image blocks are arranged to overlap each other, each pixel value is calculated by calculating the sum of the overlapping pixels and dividing the sum of the pixel values by the number of overlaps.
[0090]
The enlarged image generation unit 15 enlarges the input image data stored in the image data storage unit 2 and outputs the enlarged image to the image block arrangement unit 14 '. The enlargement processing in the enlarged image generation unit 15 is different from the enlargement processing in the enlarged image block generation unit 13 ′, and can perform enlargement processing with a smaller processing load, such as nearest neighbor interpolation enlargement and linear interpolation enlargement processing. Further, the processing can be performed not for each image block but for each input image or each input image line.
[0091]
FIG. 23 is a flowchart illustrating an example of the operation of the image processing apparatus and the image processing method according to the second embodiment of the present invention. Steps S81, S82, and steps S84 to S87 are the same as steps S41, S42, and steps S44 to S47 in the example of the process of the first embodiment shown in FIG.
[0092]
In S83, the image block feature amount calculation unit 12 calculates the feature amount of the attention area calculated in S82. Here, the edge strength G is calculated in the same manner as in the above-described example, and is compared with a predetermined threshold Th. Of course, also in the second embodiment, the feature amount calculated by the image block feature amount calculation unit 12 is not limited to the edge strength, as in the above-described first embodiment.
[0093]
Here, when the edge strength G is larger than the threshold Th, the attention area is an edge portion or a portion having a large change in the input image, and thus the enlargement processing for storing the image feature of such attention area is performed in S84 to S87. , To generate an enlarged image block. As described above, the description of the enlargement processing in S84 to S87 is omitted.
[0094]
If the edge strength G of the attention area is smaller than the threshold value Th, the processing proceeds to S81 without performing the processing of S84 to S87, and the next image block is set. That is, the enlarged image block generation unit 13 'does not generate an enlarged image block for the attention area.
[0095]
On the other hand, in parallel with or before or after such a process, a process of generating an enlarged image by the enlarged image generation unit 15 is performed. That is, in S88, the enlarged image generation unit 15 inputs the input image data stored in the image data storage unit 2 for each image or every several lines, performs the nearest neighbor interpolation enlargement process, and sends it to the image block arrangement unit 14 '. Output. Of course, the enlargement process performed in S88 is not limited to the nearest neighbor interpolation enlargement process, and various methods with a lighter processing load than the enlargement method in the enlarged image block generation unit 13 ', such as the linear interpolation enlargement process, can be performed at a processing speed. The present invention can be applied in consideration of a request and image quality required for output image data.
[0096]
In S89, the image block arrangement unit 14 'sequentially arranges the enlarged image blocks for the attention area generated by the enlarged image block generation unit 13' at corresponding positions on the enlarged image generated by the enlarged image generation unit 15. When the enlarged image blocks are arranged so as to overlap, the above-described averaging process is performed. The magnified image block or the magnified image of each line after all of these processes are output to the image data storage unit 2 and stored.
[0097]
In S90, it is determined whether or not the generation of enlarged image data to be output with respect to the input image data has been completed. If not completed, the process returns to S81, cuts out the next image block, and returns to S81. The processing up to S90 is repeated. If completed, the enlarged image data is output to a predetermined output destination, and the enlargement processing ends.
[0098]
By such processing, the input image data is subjected to high-speed enlargement processing by the enlarged image generation unit 15 by an enlargement method with a light processing load. However, blurring, jaggies, etc. may occur at the edge portion or the like as it is. On the other hand, according to the second embodiment of the present invention, an enlarged image block generation unit 13 'performs an enlargement process of preserving an image feature for an edge portion or a portion having a large change, thereby achieving high image quality. I have an enlarged image. Therefore, it is possible to obtain a high-speed enlarged image with high speed as a whole.
[0099]
FIG. 24 is an explanatory diagram of an example of a computer program and a storage medium storing the computer program when the functions or the image processing method of the image processing apparatus of the present invention are realized by a computer program. In the figure, 101 is a program, 102 is a computer, 111 is a magneto-optical disk, 112 is an optical disk, 113 is a magnetic disk, 114 is a memory, 121 is a magneto-optical disk device, 122 is an optical disk device, and 123 is a magnetic disk device.
[0100]
The functions of the image processing unit 1 described in each of the embodiments of the present invention described above can also be realized by a program 101 that can be executed by a computer. In that case, the program 101 and data used by the program can be stored in a computer-readable storage medium. A storage medium is a type of signal corresponding to a change in energy such as magnetism, light, electricity, etc., caused to a reading device provided in a hardware resource of a computer in accordance with a description content of a program. Thus, the program description can be transmitted to the reading device. For example, there are a magneto-optical disk 111, an optical disk 112 (including a CD and a DVD), a magnetic disk 113, and a memory 114 (including an IC card, a memory card, and the like). Of course, these storage media are not limited to portable types.
[0101]
By storing the program 101 in these storage media and mounting these storage media in, for example, the magneto-optical disk device 121, the optical disk device 122, the magnetic disk device 123, or a memory slot (not shown) of the computer 102, the computer 101 The program 101 can be read to execute the functions of the image processing apparatus of the present invention or the image processing method. Alternatively, a storage medium may be mounted on the computer 102 in advance, the program 101 may be transferred to the computer 102 via a network or the like, and the program 101 may be stored and executed on the storage medium. Note that the image data storage unit 2 can apply a memory in the computer 102, an attached magnetic disk device, or another storage medium. Of course, some of the functions of the present invention may be configured by hardware, or all functions may be configured by hardware.
[0102]
【The invention's effect】
As is apparent from the above description, according to the present invention, the feature amount of the image region having the predetermined size including the target pixel is calculated, and the image region of the predetermined size including the target pixel is calculated according to the calculated characteristic amount. The enlarged image area is generated by switching a plurality of enlargement methods for enlarging the image, and the obtained enlarged image area is arranged by a predetermined method to generate an enlarged output image. By such an enlargement process, for example, for a characteristic portion in the input image, an enlargement method for preserving the characteristic amount is applied, and a flat portion (an image portion such as a sky or a skin) in the input image is applied. By applying an enlargement method with a small processing load to non-characteristic parts, high-quality enlargement processing that suppresses blurring and jaggy image quality defects in characteristic parts is performed, and high-speed enlargement processing with a small processing load is performed. be able to.
[0103]
Further, a feature amount of an image region of a predetermined size including the target pixel is calculated, and an image region of a predetermined size including the specific target pixel is enlarged according to the calculated feature amount to generate an enlarged image region. Further, the input image is enlarged by an enlargement method different from the enlargement image area generation process regardless of the feature amount to generate an enlarged image, and the enlarged image area is arranged at a corresponding position on the enlarged image by a predetermined method. Thus, an enlarged output image can be generated. By such enlargement processing, the enlargement processing for each image area with a heavy processing load can be limited to only characteristic parts, and a high-quality enlarged image can be obtained, and the processing load is small and high-speed enlargement processing is realized. There is an effect that can be.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a first embodiment of an image processing apparatus and an image processing method according to the present invention.
FIG. 2 is a flowchart illustrating an example of an operation in an embodiment of the image processing apparatus and the image processing method of the present invention.
FIG. 3 is an explanatory diagram of an image block to be set and cut out by an image block setting unit 11;
FIG. 4 is a block diagram illustrating a configuration example of an image block feature amount calculation unit 12.
FIG. 5 is a diagram illustrating a specific example of an attention area and a peripheral area and an example of an edge direction of the attention area.
FIG. 6 is a flowchart illustrating an example of an edge direction estimation process in an edge direction estimation unit 32.
FIG. 7 is an explanatory diagram of an example of a reference area used for estimating an edge direction.
8 is an explanatory diagram of an example of an estimated edge direction in a region of interest shown in FIG. 5;
FIG. 9 is an explanatory diagram of an example of an edge pattern table.
10 is an explanatory diagram of a specific example of a method for selecting a first edge pattern corresponding to a region of interest shown in FIG. 5;
11 is an explanatory diagram of a specific example of an enhancement kernel used in a first generation unit 21 of the enlarged image block generation unit 13. FIG.
12 is an explanatory diagram in the case where a pixel P is enhanced by the enhancement kernel shown in FIG. 11;
FIG. 13 is an explanatory diagram of contrast enhancement processing by an enhancement kernel when input image data is enlarged by 8 times.
14 is a flowchart illustrating an example of an enlarged image block generation process in a first generation unit 21 of the enlarged image block generation unit 13. FIG.
FIG. 15 is a diagram illustrating a specific example of a process of generating an enlarged image block of 3 × 3 pixels.
FIG. 16 is an explanatory diagram of an example of a calculation formula used when generating a second edge pattern and an enlarged image block of 3 × 3 pixels.
FIG. 17 is an explanatory diagram of a specific example of a process of generating an enlarged image block of 3 × 4 pixels.
FIG. 18 is an explanatory diagram of a method of selecting a reference pixel based on an estimated edge direction.
FIG. 19 is an explanatory diagram of a specific example of a process of generating an enlarged image block of 4 × 4 pixels.
FIG. 20 is a diagram illustrating an example of a process of generating an enlarged image block of 4 × 4 pixels when the estimated edge direction is direction 0 and direction 4.
21 is an explanatory diagram of a specific example in which an enlarged image block of 4 × 4 pixels generated by an enlarged image block generation unit 13 is arranged. FIG.
FIG. 22 is a block diagram illustrating a second embodiment of the image processing apparatus and the image processing method according to the present invention.
FIG. 23 is a flowchart illustrating an example of an operation of the image processing apparatus and the image processing method according to the second embodiment of the present invention.
FIG. 24 is an explanatory diagram of an example of a computer program and a storage medium storing the computer program when the functions or the image processing method of the image processing apparatus of the present invention are realized by the computer program.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image processing part, 2 ... Image data storage part, 11 ... Image block setting part, 12 ... Image block feature-value calculation part, 13, 13 '... Enlarged image block generation part, 14, 14' ... Image block arrangement part, 15 ... Enlarged image generation unit, 21 ... First generation unit, 22 ... Second generation unit, 31 ... Edge strength calculation unit, 32 ... Edge direction estimation unit, 33 ... Edge pattern selection unit, 101 ... Program, 102 ... Computer, 111: magneto-optical disk, 112: optical disk, 113: magnetic disk, 114: memory, 121: magneto-optical disk device, 122: optical disk device, 123: magnetic disk device.

Claims (23)

An image processing apparatus that performs image enlargement processing includes an area feature amount calculation unit that calculates a feature amount of an image region having a predetermined size including a pixel of interest, and a plurality of enlargement units that perform image enlargement processing. An enlarged image region generating unit that switches the plurality of enlarging units according to the feature amount calculated by the amount calculating unit to generate an enlarged image region corresponding to the image region; and the enlargement generated by the enlarged image region generating unit. An image processing apparatus comprising an image arranging means for arranging an image area in an output image. The image processing apparatus according to claim 1, wherein the area feature amount calculation unit calculates a degree of unevenness in the image area as the feature amount from each pixel value in the image area. The image processing apparatus according to claim 1, wherein the area feature calculating unit calculates a change direction in the image area as the feature from each pixel value in the image area. The area feature value calculation means calculates the feature value for each color component in a color space, and selects one color component data in the color space based on the calculated one or more feature values. The image processing apparatus according to claim 1, wherein a feature amount of the image region in the color component data is set as a feature amount of the image region. The enlarged image area generating means includes, as one of the enlarging means, an image area in which the characteristic amount calculated by the area characteristic amount calculating means satisfies a predetermined condition together with the characteristic amount of the image area. 5. The image processing apparatus according to claim 1, further comprising an enlargement unit configured to generate the enlarged image area using a feature amount of the neighborhood area and a pixel value in the neighborhood area. 6. . The enlarged image area generation unit selects an enlargement unit with a light processing load from a plurality of the enlargement units for an image region in which the feature amount calculated by the area feature amount calculation unit satisfies a predetermined condition. The image processing apparatus according to claim 1, wherein an image area is generated. The image arranging unit sequentially arranges the enlarged image regions sequentially generated so as to overlap the image region in which the feature amount calculated by the area feature amount calculating unit satisfies a predetermined condition. The image processing apparatus according to any one of claims 1 to 6, wherein: The image arranging means calculates a sum of overlapping pixel values for overlapping pixels of the enlarged image area sequentially generated, and divides the sum of the pixel values by the number of overlaps to obtain a pixel value of the enlarged image area. The image processing apparatus according to claim 7, wherein is calculated. The image arranging unit sequentially arranges the enlarged image regions that are sequentially generated so as not to overlap with the image region in which the feature amount calculated by the region feature amount calculating unit satisfies a predetermined condition. The image processing apparatus according to claim 1, wherein: In an image processing apparatus that performs an image enlargement process, an area feature amount calculation unit that calculates a feature amount of an image area having a predetermined size including a target pixel, and the feature amount calculated by the area feature amount calculation unit is a predetermined amount. Enlarged image area generating means for generating an enlarged image area with respect to the image area satisfying the condition of the following, and an enlarged image generating means for generating an enlarged image by enlarging an input image by an enlarging method different from the enlarged image area generating means And an image processing means for arranging the enlarged image area obtained by the enlarged image area generating means at a corresponding position on the enlarged image obtained by the enlarged image generating means. . The image processing apparatus according to claim 10, wherein the area feature amount calculation unit calculates a degree of unevenness in the image area as the feature amount from each pixel value in the image area. The image processing apparatus according to claim 10, wherein the area feature calculating unit calculates a change direction in the image area as the feature from each pixel value in the image area. The area feature value calculation means calculates the feature value for each color component in a color space, and selects one color component data in the color space based on the calculated one or more feature values. The image processing apparatus according to claim 10, wherein a feature amount of the image region in the color component data is set as a feature amount of the image region. The enlarged image region generating unit includes, for an image region in which the characteristic amount calculated by the region characteristic amount calculating unit satisfies a predetermined condition, the characteristic amount of the image region and the characteristic amount of a region near the image region together with the characteristic amount of the image region 14. The image processing apparatus according to claim 10, wherein the enlarged image area is generated using a pixel value in the area. 15. The enlarged image generation unit according to claim 10, wherein the enlarged image generation unit generates the enlarged image by an enlargement method with a lighter processing load than the enlargement method in the enlarged image region generation unit. An image processing apparatus as described in the above. The image processing apparatus according to claim 10, wherein the enlarged image generation unit generates an enlarged image for each input image or every several lines in the input image. 17. The image arranging unit according to claim 10, wherein the enlarged image regions sequentially generated by the enlarged image region generating unit are sequentially arranged so as to overlap each other. Image processing device. The image arranging means calculates a sum of overlapping pixel values for overlapping pixels of the enlarged image area sequentially generated, and divides the sum of the pixel values by the number of overlaps to obtain a pixel value of the enlarged image area. The image processing apparatus according to claim 17, wherein is calculated. The image arranging means, when arranging the enlarged image area generated by the enlarged image area generating means at a corresponding position on the enlarged image obtained by the enlarged image generating means, 17. The image processing apparatus according to claim 10, wherein a pixel value is replaced with each pixel value of the enlarged image area. In an image processing method for performing image enlargement processing, a feature amount of an image region having a predetermined size including a pixel of interest is calculated, and a plurality of enlargement methods are switched according to the calculated feature amount to perform enlargement corresponding to the image region. An image processing method, comprising: generating an image area; and arranging the obtained enlarged image area to generate an output image. In an image processing method for performing an image enlargement process, a feature amount of an image region having a predetermined size including a pixel of interest is calculated, and the calculated feature amount corresponds to the image region satisfying a predetermined condition. Generating an enlarged image by enlarging the input image by an enlargement method different from the enlargement method of the image region, and arranging the enlarged image region at a corresponding position on the enlarged image. Image processing method. 22. An image processing program for causing a computer to execute an image enlargement process, the function of the image processing apparatus according to claim 1 or the image processing method according to claim 20 or 21. An image processing program for causing a computer to execute the following. The function of the image processing apparatus according to any one of claims 1 to 19 or a function of the image processing apparatus according to claim 20 or 21 in a computer-readable storage medium storing a program for causing a computer to execute image enlargement processing. A computer-readable storage medium storing a program for causing a computer to execute the image processing method described above.

Images