JP4126541B2 - Image processing apparatus, image processing method, image processing program, and storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing technique for performing an enlargement process of an image expressed in multiple gradations including color, and in particular without causing image quality defects such as blur and jaggy to the input image as much as possible. The present invention relates to an image processing technique for performing enlargement processing with high image quality and light processing load.
[0002]
[Prior art]
The image enlargement process is one of basic processes for a system that performs image editing, filing, display, printing, and the like. In recent years, with the widespread use of image data mainly for display at display resolutions such as images on the Internet homepage and digital video, these low-resolution images are frequently printed by high-resolution printers. It has been broken. At the time of printing with this printer, it is desired to obtain a high-quality output result, and the importance of the high-quality enlargement process is increasing.
[0003]
Typical existing methods for enlarging images expressed in multi-tones including color (hereinafter referred to as multi-valued images) include nearest neighbor method, linear interpolation method, cubic convolution method, etc. There is. The nearest neighbor method is a method in which the pixel value of the pixel having the closest distance is used as each enlarged pixel value when the pixel is reversely mapped on the original image. Since this method has a small amount of calculation, it can be processed at high speed. However, since one pixel of the original image is directly expanded into a rectangular shape, the degree of image quality degradation is small and has little effect when the pixel value difference between adjacent pixels is small. The image quality is greatly deteriorated, for example, when the jaggy on the edge portion is conspicuous or the image has a mosaic shape when the magnification is large.
[0004]
In the linear interpolation method, it is assumed that pixel values between pixels change linearly, and pixel values of four pixels in the vicinity of a point obtained by inversely mapping the enlarged pixel are linearly interpolated to obtain a pixel value. It is. Although this method is heavier than the nearest neighbor method, the amount of calculation is relatively small, and jaggies are less likely to occur. On the other hand, there is a drawback that the entire image becomes blurred, centering on the edge portion that does not apply to the assumption that it changes linearly.
[0005]
The cubic convolution method defines an interpolation function that approximates a sinc function (sin (x) / x) based on the sampling theorem, and 16 pixels in the vicinity of a point obtained by inversely mapping the enlarged pixel (X and Y directions) This is a method for obtaining an enlarged pixel value by a convolution operation of each of 4 pixels) and the approximate interpolation function. This method has a relatively good image quality as compared with the above two methods, but has a drawback of a large reference range and a large amount of calculation. In addition, since the high frequency band is emphasized, there are disadvantages such as light jaggies occurring at the edge and noise components being emphasized.
[0006]
As an attempt to solve these problems, for example, new methods such as Patent Document 1, Patent Document 2, Patent Document 3, and Patent Document 4 have been proposed. In the technique described in Patent Document 1, first, an original image is binarized, and the direction of an oblique component included in the original image is determined from the binary image to coincide with a two-dimensional pattern (matrix data) prepared in advance. Obtain and perform interpolation processing along the obtained oblique direction. In addition, linear interpolation processing is performed for the other portions. However, since the direction of the diagonal component is determined after binarizing the original image with a predetermined threshold, it is effective for edges with a large density difference, but for reproducing edges with a small density difference. There's a problem.
[0007]
In the technique described in Patent Document 2, a pixel value of an arrangement file is determined by a dot arrangement file (enlarged pattern file) for each integer magnification corresponding to one-to-one in the pattern file and a pixel value at any position of the pixel block. An enlarged image block is generated based on the pixel reference information. Further, when there is no matching pattern, an enlarged image block is simply generated with the target pixel of the current image block. However, in Patent Document 2, since an enlarged block is uniquely generated only by pattern matching, the image quality of the enlarged image is determined by the number of pattern files and dot arrangement files. In other words, a large number of pattern files are necessary in advance to improve image quality. Yes, it is not realistic to prepare many pattern files in advance.
[0008]
In the technique described in Patent Literature 3, edge detection is performed by an edge detection filter as a method for detecting the degree of change in an original image, and edge pixels are defined based on the edge detection result. When it is determined that the pixel is an edge pixel, enlargement is performed by the M-cubic method in which the cubic function shape of the cubic convolution method is adjusted. Otherwise, enlargement is performed by the nearest neighbor method. However, since the edge portion having a large degree of change is performed by the M-cubic method, the image quality characteristics of the cubic convolution method are followed, and there is a disadvantage that jaggy generation and noise components are emphasized.
[0009]
In the technique described in Patent Document 4, the property of an image is measured in advance by a density histogram for each block, and based on the result, a first interpolation (pattern matching method or nearest neighbor method (NN method)) and a second interpolation are performed. Superimpose the results of (cubic convolution method or M-cubic method). In the pattern matching method, which is one of the first interpolation processes, when a pattern (binarized pattern) generated from a current image block matches a predetermined angle pattern, a predetermined block is used using a reference block including the current image block. An interpolation pixel block for the target pixel is generated by the rule. However, Patent Document 4 also has a problem that jaggy and noise components are emphasized because of superimposition with the M-cubic method.
[0010]
For example, Japanese Patent Application No. 2002-76896 discloses a high-quality enlargement technique as compared with such a conventional enlargement technique. This method detects an accurate edge direction and generates an enlarged image from pixel values corresponding to the edge direction, and causes as little image quality defects as blur and jaggy to the input image as much as possible. An enlargement process can be performed. However, since processing takes time, it has been desired to reduce the processing load while maintaining high image quality and to shorten the processing time by high-speed processing.
[0011]
[Patent Document 1]
JP 2000-228723 A
[Patent Document 2]
JP 2001-188900 A
[Patent Document 3]
JP 2000-188681 A
[Patent Document 4]
JP 2002-165089 A
[0012]
[Problems to be solved by the invention]
The present invention has been made in consideration of the above-described problems of the prior art. The input image has high image quality without causing image quality defects such as blurring and jaggy as much as possible, and has a processing load. An object of the present invention is to provide an image processing apparatus and an image processing method that can perform enlargement processing lightly and at high speed. Another object of the present invention is to provide an image processing program for executing a function or an image processing method of such an image processing apparatus by a computer, and a computer-readable storage medium storing such an image processing program. Is.
[0013]
[Means for Solving the Problems]
The present invention provides an image processing apparatus and an image processing method for performing image enlargement processing, wherein a feature amount of an image region having a predetermined size including a target pixel is calculated by a region feature amount calculation unit, and the calculated feature amount is An enlarged image region corresponding to the image region that satisfies the condition is generated by the enlarged image region generating unit, and the input image is further enlarged by the enlarged image generating unit by an enlargement method different from the enlargement method of the image region. The enlarged image area is generated and arranged at a corresponding position on the enlarged image by the image arranging means.
[0017]
In this way, as a whole, the enlarged image generating means performs enlargement processing using an enlargement method with a light processing load, and the enlarged image area generating means performs only a high-quality enlarged image area for an important part of image quality such as an edge portion. Is generated and arranged. As a result, the processing load can be reduced as compared with the case where enlargement processing is performed on the entire image using a high-quality enlargement method, and enlargement processing can be performed at high speed while maintaining high image quality.
[0018]
As the feature amount calculated by the region feature amount calculation means, the degree of unevenness in the image region can be calculated from each pixel value in the image region, or the change direction in the image region can be calculated. For a color image, a feature amount is calculated for each color component in the color space, and data of one color component in the color space is selected based on the calculated one or more feature amounts. The feature amount of the image region in the data of one color component can be set as the feature amount of the image region in all the color components. The enlargement means provided in the enlargement image area generation means includes, for example, enlargement means for generating an enlargement image area using the feature quantity of the image area and the feature quantity of the neighborhood area of the image area and the pixel value in the neighborhood area. The enlargement means is selected for an image area in which the feature quantity calculated by the area feature quantity calculation means satisfies a predetermined condition, for example, an image area that requires high-quality enlargement processing including an edge or the like. The enlargement process can be performed. The enlarged image generating means for performing the enlargement process of the entire image can be configured to generate an enlarged image for each input image or for every several lines in the input image.
[0019]
The arrangement of the enlarged image area by the image arrangement means is such that the enlarged image is rewritten, the enlarged image is sequentially arranged so as to overlap with the enlarged image, and the sum of overlapping pixel values is calculated for the overlapping pixels and divided by the number of overlapping. can do.
[0020]
Furthermore, the present invention is an image processing program for causing a computer to execute an image enlargement process, wherein the function of the image processing apparatus according to any one of claims 1 to 10 or the image processing according to claim 11 is performed. The method is executed by a computer. As a result, the above-described enlargement processing function can be executed by a computer. Furthermore, the present invention can provide a computer-readable storage medium that stores such an image processing program.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block diagram showing a first embodiment of an image processing apparatus and an image processing method according to the present invention. In the figure, 1 is an image processing unit, 2 is an image data storage unit, 11 is an image block setting unit, 12 is an image block feature amount calculation unit, 13 is an enlarged image block generation unit, 14 is an image block arrangement unit, and 21 is a first block. Reference numeral 1 denotes a second generator.
[0022]
The image data storage unit 2 temporarily stores the image data until it is enlarged by the image processing unit 1, or temporarily outputs the enlarged image data subjected to resolution conversion or enlargement processing to an output device (not shown). It has a function to memorize. Note that the image data is data described in an image format (for example, BMP, TIFF, PNG, etc.) that can be processed by the image processing apparatus, and is captured by a digital camera or a scanner (not shown) or created by a personal computer or the like. Image data created by an application program for processing editing and the like. The enlarged image data is also data in the same image format.
[0023]
The image processing unit 1 includes an image block setting unit 11, an image block feature amount calculation unit 12, an enlarged image block generation unit 13, and an image block arrangement unit 14. In the meantime, reading of image data to be subjected to enlargement processing, writing of enlarged image data generated by the enlargement processing, and the like are performed.
[0024]
The image block setting unit 11 sets predetermined block sizes required for processing in the image block feature amount calculation unit 12 and the enlarged image block generation unit 13, and input image data stored in the image data storage unit 2. Are sequentially cut out and transmitted to the image block feature quantity calculation unit 12 and the enlarged image block generation unit 13.
[0025]
The image block feature amount calculation unit 12 calculates the image feature amount in the attention area in the image block sequentially cut out by the image block setting section 11 from each pixel value in the image block including the attention area or the peripheral portion of the attention area. To do. As the image feature amount, for example, as described later, the edge strength and the edge angle of the attention area can be calculated. However, the present invention is not limited to this. For example, an average value of each pixel value of the attention area is obtained, and a value (for example, standard deviation or variance) representing variation of each pixel value of the attention area with respect to the average value is obtained. Various image feature amounts may be calculated.
[0026]
The enlarged image block generation unit 13 includes a plurality of enlargement units, here, a first generation unit 21 and a second generation unit 22, and a plurality of enlargements according to the image feature amount calculated by the image block feature amount calculation unit 12. The means is switched to generate an enlarged image block corresponding to the region of interest. For example, when the image block feature value calculation unit 12 calculates the edge strength as a numerical value, the feature value is compared with a predetermined value, and either the first generation unit 21 or the second generation unit 22 is compared according to the comparison result. The enlarged image block can be generated by switching to the above. Here, it is assumed that the enlargement process is performed by the first generation unit 21 when the feature amount is larger than the predetermined value, and the second generation unit 22 when the feature amount is less than the predetermined value. Of course, the feature quantity used for switching the enlargement means is not limited to the edge strength, and the above-described standard deviation and variance, or other various feature quantities can be used. Although the enlargement method in the first generation unit 21 and the second generation unit 22 is arbitrary, the first generation unit 21 that operates when the edge strength indicates a large numerical value as in the above-described example, the enlargement giving priority to image quality Method is desirable. In this case, the second generation unit 22 is preferably an enlargement method capable of processing at a higher speed than the first generation unit 21, thereby reducing the overall processing time.
[0027]
The image block arrangement unit 14 sequentially arranges the enlarged image blocks generated by the enlarged image block generation unit 13 and outputs the enlarged image data subjected to resolution conversion or enlargement to the image data storage unit 2. The arrangement method of the enlarged image blocks will be described later. For example, in addition to the method of sequentially arranging the enlarged image blocks, the enlarged image blocks are sequentially arranged so as to overlap each other, and the sum of the overlapping pixel values is divided by the number of overlapped pixels. It can also be configured to calculate a value.
[0028]
FIG. 2 is a flowchart showing an example of the operation in the embodiment of the image processing apparatus and the image processing method of the present invention. Here, a specific example will be described. In S41, each block required for the processing of the image block feature quantity calculation unit 12 and the enlarged image block generation unit 13 in the image block setting unit 11 with respect to the input image data stored in the image data storage unit 2 Set the size. Further, the image block having the set block size is cut out from the input image data. FIG. 3 is an explanatory diagram of an image block that is set and cut out by the image block setting unit 11. A pixel that has been subjected to fine right-upward hatching at the center is a region of interest, and a pixel that has been subjected to rough left-upward hatching around it is a peripheral region. Here, as an example, the attention area has a 2 × 2 pixel size, and the peripheral area including the attention area has a 4 × 4 pixel size. In this case, the set block size is a 6 × 6 pixel size block shown in FIG. 3A in the case of double enlargement, and the 8 × 8 pixel shown in FIG. 3B in the case of quadruple enlargement. A size image block is cut out.
[0029]
In S <b> 42, the image block feature quantity calculation unit 12 calculates the edge strength G of the attention area (a 2 × 2 pixel area with fine right-upward hatching in FIG. 3) in the clipped image block using the following formula ( Calculate in 1). Note that {a, b, c, d} are each pixel value in the attention area. If the input image data is not a grayscale image but a color image in the RGB color space, for example, the edge of each image block for each color component in the R, G, B color space with respect to the region of interest using the expression (1) Intensities Gr, Gg, and Gb are calculated, an image block having a color component having the maximum edge strength among Gr, Gg, and Gb is selected, and the edge strength is selected as the edge strength of the region of interest (common to all color components). And
gx = (a + c−b−d) / 2
gy = (a + b−c−d) / 2
G = gx × gx + gy × gy (1)
The edge strength is not limited to that calculated by the above formula (1), but may be calculated by the following formula (2).
G = | gx | + | gy | (2)
[0030]
In S43, the edge strength G of the attention area calculated in S42 is compared with a predetermined threshold Th. Here, when the edge strength G is larger than the threshold Th, the attention area is an edge portion or a portion having a large change in the input image, and therefore, an enlarged image block generation process for storing the image feature of the attention area is performed in S44. To S47. When the edge strength G of the attention area is smaller than the threshold Th, the enlarged image block generation process with a smaller processing load than the processes in S44 to S47 is performed in S48.
[0031]
In S <b> 44, the image block feature amount calculation unit 12 shows the edge angle Θ of the reference area in the attention area determined to have the edge strength G greater than the threshold Th and one or more peripheral areas including the attention area. Calculation is performed using Equation (3).
Θ = arctan (gy / gx) (3)
Note that gx and gy are values calculated in S42. Then, the edge direction θ of the region of interest is estimated from the obtained plurality of edge angles Θ. For example, the edge direction θ can be estimated by performing an operation such as taking an average value of the obtained plurality of edge angles Θ. Of course, the estimation method of the edge direction θ is arbitrary.
[0032]
In S45, the image block feature quantity calculation unit 12 selects the first and second edge patterns using the edge direction θ estimated in S44 and the pixel distribution pattern of the attention area. Here, as shown in FIG. 9 described later, the first and second edge patterns are prepared in advance for each edge direction and each pixel distribution pattern. For example, the storage unit or the image data storage unit 2 in the image processing unit 1 (not shown). It can be stored as table information in a part of the table. Details will be described later.
[0033]
In S <b> 46, the first generation unit 21 of the enlarged image block generation unit 13 uses the predetermined enhancement pattern (hereinafter referred to as enhancement kernel) having a size and an element value corresponding to the enlargement ratio, in the image block setting unit 11. The image data of the attention area and the surrounding area in the cut out image block are emphasized. Note that the enhancement kernel and enhancement processing using the enhancement kernel will be described later.
[0034]
In S47, the first generation unit 21 of the enlarged image block generation unit 13 determines the edge direction θ of the region of interest estimated by the image block feature amount calculation unit 12, the selected edge pattern, and the attention calculated in S46. An enlarged image block for the region of interest is generated using the enhanced pixel values of the region and the peripheral region.
[0035]
In this way, when the edge strength G of the region of interest is high in S43, an enlarged image block is generated that preserves such an edge portion or a feature that the change is large. As a result, it is possible to generate a high-quality enlarged image block with less blur and jaggy for the edge portion and the like.
[0036]
On the other hand, in S <b> 48, the second generation unit 22 of the enlarged image block generation unit 13 generates an enlarged image block for the attention area determined in S <b> 43 that the edge strength G is smaller than the threshold Th in the image block feature amount calculation unit 12. . Unlike the enlarged image block generation process in S44 to S47, the enlarged image block generation process in the second generation unit 22 may be an enlarged process with a small processing load such as nearest neighbor interpolation. That is, when the edge strength G is small, it is possible to obtain sufficient image quality even when the enlarged image block is generated with a smaller processing load without performing the complicated and time-consuming processing described above. . Therefore, when the edge strength G is low, the second generation unit 22 of the enlarged image block generation unit 13 is selected to generate an enlarged image block with a small processing load, thereby reducing the overall processing time. .
[0037]
In S49, the image block arrangement unit 14 enlarges the image block (the enlarged image block generated by the processing of S44 to S47 and the enlarged image generated by the processing of S48) with respect to the region of interest generated by the enlarged image block generating unit 13. Blocks) are sequentially arranged by a predetermined method described later, and the arranged image blocks are stored in the image data storage unit 2 as a part of the enlarged image data to be output.
[0038]
In S50, it is determined whether the generation of the enlarged image data to be output with respect to the input image data is completed. If not, the process returns to S41, the next image block is cut out, and the above process is repeated. . If completed, the enlarged image data is output and the enlargement process ends.
[0039]
Heretofore, an example of the operation in the embodiment of the image processing apparatus and the image processing method of the present invention has been described. Next, details of the image block feature quantity calculation unit 12, the enlarged image block generation unit 13, and the image block arrangement unit 15 which are main parts will be described.
[0040]
First, details of the image block feature quantity calculation unit 12 will be described. Here, as shown in FIG. 3 described above, the attention area is a 2 × 2 pixel size block, and the peripheral area including the attention area is a 4 × 4 pixel size block. FIG. 4 is a block diagram illustrating a configuration example of the image block feature amount calculation unit 12. In the figure, 31 is an edge strength calculation unit, 32 is an edge direction estimation unit, and 33 is an edge pattern selection unit. In the example illustrated in FIG. 4, the image block feature quantity calculation unit 12 includes an edge strength calculation unit 31, an edge direction estimation unit 32, and an edge pattern selection unit 33.
[0041]
As described above, the edge strength calculation unit 31 calculates the edge strength G of the region of interest in the image block cut out by the image block setting unit 11 by the above equation (1), and calculates the calculated edge strength G and a predetermined value. Is compared with the threshold value Th, and a method for enlarging the attention area is selected. When the edge strength G of the attention area is smaller than the threshold value Th, the enlargement process in the second generation unit 22 of the enlarged image block generation unit 13 without performing the processes in the edge direction estimation unit 32 and the edge pattern selection unit 33. Is done.
[0042]
Details of the processing of the edge direction estimation unit 32 and the edge pattern selection unit 33 will be described below. FIG. 5 is an explanatory diagram of a specific example of the attention area and the peripheral area and an example of the edge direction of the attention area. FIG. 5A shows an example of the attention area and the peripheral area. Each rectangle is a pixel, and each number indicates a pixel value. The attention area is a pixel {a, b, c, d} = {15, 104, 86, 203} surrounded by a thick line frame. The flow of edge direction estimation processing in the edge direction estimation unit 32 will be described using the example shown in FIG.
[0043]
FIG. 6 is a flowchart illustrating an example of edge direction estimation processing in the edge direction estimation unit 32. In S61, the edge angle Θ of the region of interest surrounded by the thick line frame in FIG. 5A is calculated by the above-described equation (3). For example, in the case of the attention area {a, b, c, d} = {15, 104, 86, 203} as specifically shown in FIG. 5A, gx and gy are gx = − from Equation (1), respectively. 103, gy = −85. Therefore, the edge angle Θ in the region of interest shown in FIG. 5A is Θ = −140.5 °. The direction of the edge angle Θ is indicated by a broken arrow line in FIG.
[0044]
Further, as shown in FIG. 5B, the angle direction is set, and it is examined whether the obtained edge angle Θ is included in the angle range of 22.5 ° (eight directions). In this case, the edge angle Θ is 0 ° or ± 180 ° in the direction 0, 22.5 ° or -157.5 °, the direction 1, 45 ° or -135 ° in the direction 2, 67.5 ° or Direction of −112.5 ° is direction 3, 90 ° or −90 ° is direction 4, direction of 112.5 ° or −67.5 ° is direction 5, direction of 135 ° or −45 ° is direction 6 The direction of 157.5 ° or −22.5 ° is the direction 7, and the range of ± 11.25 ° of these angles is the respective direction. Since the edge angle Θ (= −140.5 °) in the above specific example is included in the range of −135 ° ± 11.25 °, the edge angle is the direction 2.
[0045]
In S61, the angle reference number which is a variable for counting the reference number of the edge angle used for the edge direction estimation of the attention area in S66 which will be described later is set to 1.
[0046]
In S62, a reference area used for estimation of the edge direction is selected from the peripheral area (area outside the thick line frame) shown in FIG. 5A according to the edge angle Θ of the attention area calculated in S61. FIG. 7 is an explanatory diagram of an example of a reference area used for edge direction estimation. A 2 × 2 pixel indicated by a bold frame in the drawing is a reference area. FIG. 7A shows a case where the direction of the edge angle obtained in S61 is 0, FIG. 7B shows a case where the direction is 4, and FIG. 7C shows a case where the direction is other than that. Note that the number of reference areas in the direction 0 and the direction 4 shown in FIGS. 7A and 7B is 2, and the number of reference areas in the directions other than the directions 0 and 4 shown in FIG. The number is four. In the specific example shown in FIG. 5, since the direction of the edge angle is the direction 2, the four reference regions shown in FIG. 7C are candidates for selection. The selection of the reference area is not limited to that shown in FIG. 7. For example, in the case of FIG. 7C, the number of reference areas is set to 8, or the reference area is set according to each direction. May be.
[0047]
In S63, the edge angle Θ is calculated for one of the reference regions selected in S62 according to the equation (3) as in S61. In S64, the edge angle of the reference area calculated in S63 is compared with the edge angle of the attention area calculated in S61. If the difference between the two edge angles is smaller than the preset threshold value Θth, the angle reference number is incremented by 1 in S65, and the process proceeds to S66. If the difference between the two edge directions is larger than the preset threshold value Θth, it is determined that the reference area is not suitable for the edge direction estimation, and the process proceeds to S66 as it is. In S66, it is determined whether or not the edge angle calculation of all selectable reference areas has been completed. If completed, the process proceeds to S67. If there is a reference area that has not been completed, the process from S63 to S65 is repeated.
[0048]
In the specific example shown in FIG. 5, the edge angle ΘU = -149.4 ° from the upper reference area {86, 203, 171, 211}, and the edge angle from the left reference area {10, 15, 20, 86}. ΘL = −131.2 °, edge angle ΘD = −175.2 ° from lower reference area {1,102,15,104}, edge angle ΘR from right reference area {104,215,203,219} = −141.0 °. The edge angle Θ = −140.5 ° of the attention area is compared with the edge angle of each reference area, and the number of reference areas whose difference is smaller than the threshold Θth is counted as the angle reference number.
[0049]
In S67, the sum of the edge angle of the attention area and the edge angle of the reference area determined to be suitable for edge direction estimation in S64 is calculated, and the average edge angle obtained by dividing the sum of the edge angles by the number of angle references is noted. The estimated edge direction of the region. FIG. 8 is an explanatory diagram of an example of the estimated edge direction in the region of interest shown in FIG. For example, in the above specific example, if the difference between the edge angle of the attention area for all the reference areas is smaller than the threshold value Θth, the sum of the edge angles obtained from the attention area and the four reference areas is −737.3 °. By dividing by the angle reference number 5, the average edge angle can be obtained as -147.5 °. Also in this case, as in the edge direction of the region of interest described above, for example, if normalized to any of the eight directions, -147.5 ° becomes the direction 1. This is the estimated edge direction.
[0050]
In the above description, the case where the input image data is a grayscale image has been described. However, the present invention is not limited to this. For example, in the case of a color image in the RGB color space, the color space data selected by the strength of the edge angles Gr, Gg, and Gb in the data of each color component as described in S32 of the flowchart of FIG. The above processing may be performed. By doing so, it is possible to suppress deterioration in image quality such as color shift at the edge portion of the enlarged image data in the color image.
[0051]
In the above description, the edge angle is calculated to any one of the eight directions after calculating the edge angle using the expression (1) in the attention area and the reference area, but of course, the present invention is not limited to this, and the edge direction with higher accuracy is used. May be normalized in more directions such as 12 directions (every 15.0 °) and 16 directions (every 12.25 °).
[0052]
Next, details of the operation of the edge pattern selection unit 33 will be described. FIG. 9 is an explanatory diagram of an example of an edge pattern table. The edge pattern selection unit 33 performs edge pattern conversion using, for example, an edge pattern table as shown in FIG. As shown in FIG. 9, in the edge pattern table, one or several first edge patterns corresponding to the pattern size of the region of interest are registered for each edge direction (eight directions here). Corresponding to the pattern, a pattern having a pattern size larger than the pattern size of the first edge pattern is registered as the second edge pattern.
[0053]
The edge pattern selection unit 33 selects, for example, the first edge pattern corresponding to the edge direction of the attention area estimated by the edge direction estimation unit 32 using the edge pattern table as shown in FIG. A second edge pattern corresponding to one edge pattern is determined.
[0054]
Specifically, in the case of the attention area and its surrounding area surrounded by the thick frame shown in FIG. 5A, the estimated edge direction for the attention area is noticed for all reference areas by the edge direction estimation unit 11 as described above. When the difference from the edge angle of the region is smaller than the threshold value Θth, the direction 1 is obtained. According to the estimated edge direction (direction 1) of the attention area, a pattern candidate corresponding to the edge pattern of the attention area is selected from the edge pattern table shown in FIG. In this case, four patterns, pattern 0 to pattern 3, which are the first edge patterns in direction 1, are selected and become candidates for the first edge pattern for the region of interest shown in FIG.
[0055]
Next, the edge pattern selection unit 33 selects any one of the patterns 0 to 3 as the first edge pattern for the region of interest as described below. FIG. 10 is an explanatory diagram of a specific example of a method for selecting the first edge pattern corresponding to the region of interest shown in FIG. In this specific example, since the estimated edge direction is the direction 1, four edge pattern candidates in the first edge pattern from the edge pattern table shown in FIG. 9 are obtained as edge patterns for the attention area shown in FIG. chosen. The selected edge pattern candidates are shown in FIG.
[0056]
Each of these edge pattern candidates is formed into a bit pattern as shown in FIG. Here, a bit pattern is formed with 0 for the white portion and 1 for the other portions. However, the edge pattern table shown in FIG. 9 may be registered as a bit table in advance, and in this case, the bit patterning process can be omitted.
[0057]
Next, the average pixel value in the attention area is calculated according to the equation (4), the average value is subtracted from each pixel value in the attention area, and the sign is used as the pixel value pattern of the attention area.
Mean = (a + b + c + d) / 4
a_sign = a-Mean
b_sign = b-Mean
c_sign = c-Mean
d_sign = d-Mean (4)
In the case of the example shown in FIG. 5A, Mean = (15 + 104 + 86 + 203) / 4 = 102, a_sign = −87, b_sign = 2, c_sign = −16, and d_sign = 101. Then, as shown in FIG. Further, the pixel value pattern is converted into a bit pattern as shown in FIG.
[0058]
Next, pattern matching between the bit pattern of each edge pattern candidate shown in FIG. 10 (B) and the bit pattern of the attention area shown in FIG. 10 (D) is performed, and the edge in the first edge pattern for the attention area Select one pattern. In this case, pattern 2 is selected as the first edge pattern for the region of interest (FIG. 10E).
[0059]
Finally, when the first edge pattern is selected, a second edge pattern prepared in advance so as to correspond to the first edge pattern is determined. In this case, the second edge pattern (direction 1, pattern 2) corresponding to pattern 2 in direction 1 in the first edge pattern is selected from the edge pattern table shown in FIG. 9 (FIG. 10F). The second edge pattern is used when an enlarged image block is generated for a region of interest in the enlarged image block generation unit 2 described later.
[0060]
The first and second edge patterns are not limited to those shown in FIG. 9. For example, an edge pattern different from that shown in FIG. 9 may be used depending on the type of input image data. The number of first and second edge pattern candidates in the angle may be increased or decreased. Of course, it is not necessary to configure as a table, as long as the second edge pattern can be derived from the identified first edge pattern.
[0061]
Next, the enlarged image block generation unit 13 will be described in detail. First, details of the enlarged image region generation processing in the first generation unit 21 of the enlarged image block generation unit 13 will be described. In the first generation unit 21 of the enlarged image block generation unit 13, first, an image as shown in FIG. 3 described above is used by using an enhancement kernel of a size and elements corresponding to the enlargement magnification set by the image block setting unit 11. The contrast of the image data of the attention area and the surrounding area in the image block cut out by the block setting unit 11 is enhanced.
[0062]
FIG. 11 is an explanatory diagram of a specific example of the enhancement kernel used in the first generation unit 21 of the enlarged image block generation unit 13. FIG. 12 is an explanatory diagram when the pixel P is emphasized by the enhancement kernel shown in FIG. The enhancement kernel (kernel 0) shown in FIG. 11 (A) is shown in FIG. 11 (C) when the magnification is 2 times, and the enhancement kernel (kernel 1) shown in FIG. 11 (B) is when the magnification is 4 times. The kernel (kernel 2) is used when the magnification is 8 times. In the drawing, the center pixel value and hatched pixels arranged on the top, bottom, left, and right of the pixel are used, and the emphasis processing is performed using the weight described in the lower part.
[0063]
When the enlargement process is performed, if the process is performed 4 times or 8 times, the pixels having the characteristics of the original image are 2 pixels, and the pixels are separated by 4 images. Therefore, as shown in FIG. 11, the higher the magnification, the more distant pixels are selected and used for the enhancement process. Of course, the same applies to a magnification of 16 times or more. Further, the position of the pixel to be referred to is not limited to the vertical and horizontal directions as shown in FIG. For example, kernel elements (referenced pixels) and elements, such as including a pixel in an oblique direction, a configuration to refer to a further distant pixel, or changing a kernel to be used depending on the type and size of input image data, etc. The distance between them may be different from the enhancement kernel shown in FIG.
[0064]
For example, when an enhancement kernel as shown in FIG. 11A is used, as shown in FIG. 12A, for example, when enhancement processing of the central reference pixel P is performed, the pixels a, Reference is made to b, c, and d, and 1.6 is used for the center pixel of interest and 0.15 is used for the surrounding reference pixels as weights. Thus, the pixel value P ′ of the pixel P can be calculated according to the following equation (5).
Pixel value of interest P ′ = 1.60 × P−0.15 × (a + b + c + d) (5)
[0065]
For example, even when an enhancement kernel as shown in FIG. 11B is used, when the enhancement processing of the reference pixel P is similarly performed as shown in FIG. And the existing pixels a, b, c, and d are referred to, and 1.2 is used for the center pixel of interest and 0.05 is used for the surrounding reference pixels, respectively, as in Equation (5). Thus, the pixel value P ′ of the pixel P may be calculated.
[0066]
FIG. 13 is an explanatory diagram of the contrast enhancement processing by the enhancement kernel when the input image data is enlarged 8 times. As will be described later, the enlargement process performed by the first generation unit 21 is performed every 2 times. Therefore, when 8 times enlargement is performed, the enlargement is first performed 2 times, and the image data that has been doubled is processed. Then, the image data is further magnified by 2 times to generate image data that has been magnified by 4 times, and the image data that has been magnified by 4 times is further magnified by 2 times to be finally magnified by 8 times. Get the image data. The above-described enhancement processing by the enhancement kernel is performed before each double enlargement.
[0067]
That is, when the input image data shown in FIG. 13A is enlarged twice, the contrast of the input image data is enhanced using the kernel 0 shown in FIG. Thus, the double enlarged data shown in FIG. 13B is generated. Next, when the image data enlarged twice as shown in FIG. 13B is enlarged twice, contrast enhancement is performed on the twice enlarged image data using the kernel 1 shown in FIG. Thereafter, the data is enlarged by a factor of 2 to generate 4 times enlarged data as shown in FIG. Further, contrast enhancement is performed on the 4-times enlarged data shown in FIG. 13C by using the kernel 2 shown in FIG. 11C, and then enlarged twice to 8-times enlarged as shown in FIG. 13D. Generate data. The enhancement kernel shown in FIG. 11 is used in this manner, and performs enhancement processing on the data at each expansion stage.
[0068]
Next, the first generation unit 21 of the enlarged image block generation unit 13 actually generates an enlarged image block. Details of the enlarged image block generation process will be described. In the first generation unit 21 of the enlarged image block generation unit 13, the second edge pattern obtained by the edge pattern selection unit 33 of the image block feature amount calculation unit 12 and the pixel value subjected to contrast enhancement in the image enhancement process described above are used. Use to generate an enlarged image block for the region of interest.
[0069]
FIG. 14 is a flowchart illustrating an example of the enlarged image block generation process in the first generation unit 21 of the enlarged image block generation unit 13. In S71, using the region of interest whose contrast has been enhanced by the image enhancement processing and the second edge pattern selected by the edge pattern selection unit 33 of the image block feature quantity calculation unit 12, first, an enlarged image block of 3 × 3 pixels. Is generated. FIG. 15 is an explanatory diagram of a specific example of generation processing of an enlarged image block of 3 × 3 pixels. FIG. 15A shows an example of the attention area and the peripheral area shown in FIG. FIG. 15B shows a second edge pattern (pattern 2 in direction 1 in FIG. 9) corresponding to this attention area, for example, obtained as described in FIG. In FIG. 15B, each pixel is set to p0 to p8. Based on the attention area {a, b, c, d} shown in FIG. 15A, the pixel values of the pixels p0 to p8 are calculated by the following formula.
p0 = a
p1 = (a + c) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (b + d) / 2
p8 = d
By performing such calculation, an enlarged image block of 3 × 3 pixels can be generated as shown in FIG.
[0070]
FIG. 16 is an explanatory diagram of an example of a calculation formula used when generating the second edge pattern and the 3 × 3 pixel enlarged image block. As described above, when the second edge pattern as shown in FIG. 15B is selected, an enlarged image block of 3 × 3 pixels can be generated by the above formula. The calculation formula is determined for each selected second edge pattern. The second edge pattern shown in FIG. 16A is the pattern 0 in the direction 1 shown in FIG.
p0 = a
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p1 = 2 × p4-p7
Thus, an enlarged image block of 3 × 3 pixels can be generated.
[0071]
The second edge pattern shown in FIG. 16B is the pattern 1 in the direction 5 shown in FIG.
p0 = a
p1 = (a + c) / 2
p2 = b
p3 = (c + d) / 2
p4 = (a + d) / 2
p5 = (a + b) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
Thus, an enlarged image block of 3 × 3 pixels can be generated.
[0072]
The second edge pattern shown in FIG. 16C is the pattern 0 in the direction 2 shown in FIG.
p0 = a
p1 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
Thus, an enlarged image block of 3 × 3 pixels can be generated.
[0073]
The second edge pattern shown in FIG. 16D is the pattern 0 in the direction 0 shown in FIG.
p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p4 = (p1 + p7) / 2
Thus, an enlarged image block of 3 × 3 pixels can be generated. In the case of other second edge patterns as well, an enlarged image block of 3 × 3 pixels can be generated by performing calculation according to the calculation formula corresponding to each second edge pattern.
[0074]
Returning to FIG. 14, in S <b> 72, the estimated edge direction of the region of interest estimated by the edge direction estimating unit 32 of the image block feature value calculating unit 12 is determined, and the estimated edge directions are directions 1 to 3 and directions 5 to 7. If the estimated edge direction is direction 0 and direction 4 (that is, the edge direction is vertical or horizontal), the process proceeds to S74.
[0075]
In S73, when the estimated edge direction of the attention area is direction 1 to direction 3 and direction 5 to direction 7, that is, when the edge direction is other than the vertical or horizontal direction, the 3 × 3 pixel enlarged image block generated in S71 To generate an enlarged image block of 4 × 4 pixels. Here, a 3 × 4 pixel enlarged image block is first generated from the 3 × 3 pixel enlarged image block, the contrast-enhanced attention region and its peripheral region, and then the generated 3 × 4 pixel enlarged image block is generated. A 4 × 4 pixel enlarged image block is generated from the region of interest and its peripheral region with contrast enhanced.
[0076]
FIG. 17 is an explanatory diagram of a specific example of generation processing of a 3 × 4 pixel enlarged image block. FIG. 17A shows an example of the attention area and the peripheral area shown in FIG. 5A as an example. Further, p0 to p8 in FIG. 17B show the 3 × 3 pixel enlarged image block shown in FIG. 15C in the center.
[0077]
First, in addition to the 3 × 3 pixel enlarged image block shown in the center of FIG. 17B, the reference pixels (r0 to r5) in the peripheral region with contrast enhancement shown by the thick line frame in FIG. 17A are used. Thus, an enlarged image block (q0 to q11) of 3 × 4 pixels shown in FIG. The pixel values q0 to q11 of the 3 × 4 pixel enlarged image block are determined by the following calculation formula.
q0 = 0.2 × r0 + 0.8 × p0
q1 = 0.4 × p0 + 0.6 × p1
q2 = 0.6 × p1 + 0.4 × p2
q3 = 0.8 × p2 + 0.2 × r1
q4 = 0.2 × r2 + 0.8 × p3
q5 = 0.4 × p3 + 0.6 × p4
q6 = 0.6 × p4 + 0.4 × p5
q7 = 0.8 × p5 + 0.2 × r3
q8 = 0.2 × r4 + 0.8 × p6
q9 = 0.4 × p6 + 0.6 × p7
q10 = 0.6 × p7 + 0.4 × p8
q11 = 0.8 × p8 + 0.2 × r5
[0078]
FIG. 18 is an explanatory diagram of a reference pixel selection method based on the estimated edge direction. In the example shown in FIG. 17, assuming that the estimated edge direction obtained by the image block feature quantity calculation unit 12 from the attention area and the peripheral area shown in FIG. The positions are 3 pixels from the upper left to the lower and 3 pixels from the lower right to the upper as surrounded by a thick line frame in FIG. An example of this reference pixel is shown in FIG. 18A. The reason why the pixel at such a position is used as the reference pixel is that the estimated edge direction of the attention area is from direction 1 (22.5 °) to direction 3 ( 67.5 °). When the estimated edge direction of the region of interest is from direction 5 (112.5 °) to direction 7 (157.5 °), as shown in FIG. And 3 pixels from the upper right to the lower. Thus, the reference pixel is selected according to the estimated edge direction in the attention area. Of course, the selection of the reference pixels is not limited to the selection from the two patterns as shown in FIG. 18, and more reference pixel selection patterns may be prepared according to the estimated edge direction. Also, the reference pixel to be selected may be changed according to the estimated edge direction.
[0079]
FIG. 19 is an explanatory diagram of a specific example of generation processing of a 4 × 4 pixel enlarged image block. A 4 × 4 pixel enlarged image block is generated using the 3 × 4 pixel enlarged image block generated as described above and the contrast-enhanced peripheral pixels (r0 to r7). FIG. 19A shows the attention region and the reference region with contrast enhancement, and the generated 3 × 4 pixel enlarged image block is shown in the center of FIG. 19B. Here, as the reference pixels r0 to r7, the lower four pixels and the upper four pixels of the reference region in which the contrast is enhanced are used as shown by the thick line frame in FIG. Then, 4 × 4 pixel enlarged image blocks (s0 to s15) shown in FIG. 19C are generated according to the following calculation formula.
s0 = 0.2 × r0 + 0.8 × q0
s1 = 0.2 × r1 + 0.8 × q1
s2 = 0.2 × r2 + 0.8 × q2
s3 = 0.2 × r3 + 0.8 × q3
s4 = 0.4 × q0 + 0.6 × q4
s5 = 0.4 × q1 + 0.6 × q5
s6 = 0.4 × q2 + 0.6 × q6
s7 = 0.4 × q3 + 0.6 × q7
s8 = 0.6 × q4 + 0.4 × q8
s9 = 0.6 × q5 + 0.4 × q9
s10 = 0.6 × q6 + 0.4 × q10
s11 = 0.6 × q7 + 0.4 × q11
s12 = 0.8 × q8 + 0.2 × r4
s13 = 0.8 × q9 + 0.2 × r5
s14 = 0.8 × q10 + 0.2 × r6
s15 = 0.8 × q11 + 0.2 × r7
[0080]
In the generation processing of the 4 × 4 pixel enlarged image block in S73 of FIG. 14 described above, the enlargement processing is performed according to the processing flow of 3 × 3 pixel block → 3 × 4 pixel block → 4 × 4 pixel block here. However, the processing flow may be 3 × 3 pixel block → 4 × 3 pixel block → 4 × 4 pixel block, or a 4 × 4 pixel block may be generated from the 3 × 3 pixel block. . Of course, the selection of the reference pixel in these cases is changed as appropriate.
[0081]
Returning to FIG. 14, in S74, when the estimated edge direction of the attention area is the direction 0 and the direction 4 (that is, in the case of the vertical direction or the horizontal direction), 4 × from the 3 × 3 pixel enlarged image block generated in S71. A 4-pixel enlarged image block is generated. FIG. 20 is an explanatory diagram of an example of a process of generating an enlarged image block of 4 × 4 pixels when the estimated edge directions are direction 0 and direction 4. When the estimated edge directions of the attention area are direction 0 and direction 4, the pixels (A, B, C, D) in the peripheral area with contrast enhancement shown in FIG. Reference pixels r0 to r5 shown on the left and right are calculated using the following equations.
r0 = A
r1 = B
r2 = 0.5 × A + 0.5 × C
r3 = 0.5 × B + 0.5 × D
r4 = C
r5 = D
In the center of FIG. 20B, the 3 × 3 pixel enlarged image blocks p0 to p8 generated in S71 of FIG. 14 are shown.
[0082]
When the 3 × 3 pixel enlarged image block and the reference pixels r0 to r5 are obtained in this way, the 3 × 4 pixel enlarged image blocks q0 to q11 are obtained by the calculation method described above with reference to FIG. Generate. This is shown in FIG. Furthermore, the 3 × 4 pixel enlarged image blocks q0 to q11 generated in this way and the four pixels above and below the contrast-enhanced peripheral region shown in FIG. 20A are used as reference pixels, and are shown in FIG. The enlarged image blocks (s0 to s15) of 4 × 4 pixels shown in FIG.
[0083]
As described above, by performing the processing from S71 to S74 shown in FIG. 14, the enlarged image of 4 × 4 pixels with respect to the region of interest for which the edge strength G is determined to be larger than the threshold Th in the image block feature quantity calculation unit 12 Blocks (for example, s0 to s15 shown in FIG. 19C or FIG. 20D) are generated.
[0084]
Next, for the attention area for which the edge strength G is determined to be smaller than the threshold Th by the image block feature quantity calculation unit 12, the second generation unit 22 of the enlarged image block generation unit 13 performs the attention area as described above. Instead of the enlargement process in the first generation unit 21 of the enlargement image block generation unit 13 that stores the image feature, a simpler enlargement image block generation process with reduced processing load is performed. For example, as an example, an enlarged image block of 4 × 4 pixels can be generated from the attention area of 2 × 2 pixels by nearest neighbor interpolation. Of course, the enlargement method used in the second generation unit 22 is arbitrary, and any method may be used as long as the method has a lighter processing load than the first generation unit 21.
[0085]
As described above, the input image data can be enlarged with high image quality by selecting the enlargement process performed in the enlarged image block generation unit 13 according to the image feature quantity calculated by the image block feature quantity calculation unit 12, and further, The processing load can be reduced. Specifically, the second generation unit 22 uses an enlargement method with a lighter processing load than the first generation unit 21, thereby reducing the processing load of the entire enlargement process, shortening the processing time, and performing a high-speed enlargement process. Can be realized. Here, it is assumed that the enlarged image block generation unit 13 has two enlargement methods, and one of them is selectively used. However, a plurality of enlargement methods are provided to calculate the image block feature amount. According to various image feature amounts calculated by the unit 12, it is possible to select one of a plurality of enlargement methods and perform enlargement processing.
[0086]
Next, the image block arrangement unit 14 will be described. The image block placement unit 14 sequentially places enlarged image blocks for the attention area generated by the enlarged image block generation unit 13 by a predetermined method. FIG. 21 is an explanatory diagram of a specific example in which the 4 × 4 pixel enlarged image blocks generated by the enlarged image block generation unit 13 are arranged. In the example shown in FIG. 21, the sequentially generated enlarged image block 0 and enlarged image block 1 are arranged to overlap. The overlapping pixels are arranged so as to average each of the previous pixel values. Alternatively, the sum of overlapping pixels may be calculated, and each pixel value may be calculated by dividing the sum of the pixel values by the number of overlaps. In this case, for example, the target pixel is selected while being shifted one pixel at a time, and the enlargement process is performed. Alternatively, they can be arranged without overlapping. In this case, the attention area may be selected so as not to overlap.
[0087]
In addition, since the 4 × 4 pixel enlarged image block is generated by either the first generation unit 21 or the second generation unit 22 in the enlarged image block generation unit 13, there is a concern that the image quality is deteriorated due to the difference in the enlargement method. However, it is possible to suppress the occurrence of such image quality degradation by performing the averaging by overlapping as described above.
[0088]
FIG. 22 is a block diagram showing a second embodiment of the image processing apparatus and the image processing method of the present invention. In the figure, the same parts as those in FIG. Reference numeral 13 'denotes an enlarged image block generation unit, 14' denotes an image block arrangement unit, and 15 denotes an enlarged image generation unit. In the second embodiment, the image processing unit 1 includes an enlarged image generation unit 11, an image block feature amount calculation unit 12, an enlarged image block generation unit 13 ′, and an image block arrangement unit 14 ′. 15 is comprised. Note that the image block setting unit 11 and the image block feature amount calculation unit 12 have the same functions as those in the first embodiment described above. Further, the enlarged image block generating unit 13 ′ and the image block arranging unit 14 ′ have a configuration corresponding to the enlarged image block generating unit 13 and the image block arranging unit 14 in the first embodiment described above. Among these, the enlarged image block generation unit 13 ′ is the same as the function of the first generation unit 21 of the enlarged image block generation unit 13 in the first embodiment described above, and a description thereof is omitted here.
[0089]
The image block arrangement unit 14 ′ applies an enlarged image block that has been subjected to enlargement processing by the enlarged image block generation unit 13 ′ to a region of interest for which the edge strength G is determined to be larger than the threshold Th by the image block feature amount calculation unit 12. The image data are sequentially arranged at corresponding positions on the enlarged image output from the enlarged image generation unit 15 described later, and the resolution-converted or enlarged image data is output to the image data storage unit 2. When the enlarged image block is sequentially arranged at a corresponding position on the enlarged image output from the enlarged image generating unit 15, the enlarged image block may be replaced with each pixel value of the enlarged image block, or the enlarged image generation may be performed. You may arrange | position so that it may overlap with each pixel value of the position corresponding to the enlarged image output from the part 15. FIG. When the enlarged image blocks are arranged so as to overlap each other, the sum of the overlapping pixels is calculated, and each pixel value is calculated by dividing the sum of the pixel values by the number of overlaps.
[0090]
The enlarged image generation unit 15 enlarges the input image data stored in the image data storage unit 2 and outputs the enlarged image to the image block arrangement unit 14 ′. Unlike the enlargement process in the enlargement image block generation unit 13 ′, the enlargement process in the enlargement image generation unit 15 can perform an enlargement process with a smaller processing load, for example, nearest neighbor interpolation enlargement or linear interpolation enlargement process. Furthermore, processing can be performed in units of input images or in units of lines of the number of input images, rather than processing for each image block.
[0091]
FIG. 23 is a flowchart showing an example of the operation in the second embodiment of the image processing apparatus and the image processing method of the present invention. The processes from S81, S82, and S84 to S87 are the same as the processes from S41, S42, and S44 to S47 in the example of the process of the first embodiment shown in FIG.
[0092]
In S83, the image block feature value calculation unit 12 calculates the feature value of the attention area calculated in S82. Here, as in the above example, the edge strength G is calculated and compared with a predetermined threshold Th. Of course, also in the second embodiment, the feature amount calculated by the image block feature amount calculation unit 12 is not limited to the edge strength, as in the first embodiment described above.
[0093]
Here, when the edge strength G is larger than the threshold Th, the region of interest is an edge portion or a portion with a large change in the input image. Therefore, enlargement processing for preserving the image feature of the region of interest is performed in S84 to S87. Generate an enlarged image block. Note that the description of the enlargement process in S84 to S87 is omitted as described above.
[0094]
When the edge strength G of the attention area is smaller than the threshold value Th, the processing of S84 to S87 is not performed, the processing is shifted to S81, and the next image block is set. That is, the enlarged image block is not generated by the enlarged image block generation unit 13 ′ for the attention area.
[0095]
On the other hand, the enlarged image generation unit 15 performs enlarged image generation processing in parallel with or before or after such processing. That is, in S88, the enlarged image generation unit 15 inputs the input image data stored in the image data storage unit 2 for each image or every several lines, performs nearest neighbor enlargement processing, and sends it to the image block arrangement unit 14 ′. Output. Of course, the enlargement process performed in S88 is not limited to the nearest neighbor enlargement process, and various methods having a lighter processing load than the enlargement method in the enlarged image block generation unit 13 ′, such as the linear interpolation enlargement process, can be performed. It can also be applied in consideration of requirements and image quality required for output image data.
[0096]
In S89, the image block arrangement unit 14 ′ sequentially arranges the enlarged image blocks for the attention area generated by the enlarged image block generation unit 13 ′ at corresponding positions on the enlarged image generated by the enlarged image generation unit 15. When the enlarged image blocks are arranged so as to overlap, the averaging process as described above is performed. The enlarged image block or the enlarged image in units of lines after completing all of these processes is output to the image data storage unit 2 and stored.
[0097]
In S90, it is determined whether or not generation of the enlarged image data to be output with respect to the input image data is completed. If not, the process returns to S81, and the next image block is cut out from S81 described above. The process up to S90 is repeated. If completed, the enlarged image data is output to a predetermined output destination, and the enlargement process ends.
[0098]
By such processing, the input image data is subjected to enlargement processing at a high speed by the enlarged image generation unit 15 by an enlargement method with a light processing load. However, if this is the case, blurring or jaggy may occur at the edge portion. On the other hand, in the second embodiment of the present invention, the edge portion or the portion having a large change is partially subjected to an enlargement process for storing image features by the enlarged image block generation unit 13 ′, thereby achieving high image quality. An enlarged image is obtained. Therefore, it is possible to obtain an enlarged image with high speed and high image quality as a whole.
[0099]
FIG. 24 is an explanatory diagram of an example of a computer program and a storage medium storing the computer program when the function of the image processing apparatus or the image processing method of the present invention is realized by the computer program. In the figure, 101 is a program, 102 is a computer, 111 is a magneto-optical disk, 112 is an optical disk, 113 is a magnetic disk, 114 is a memory, 121 is a magneto-optical disk apparatus, 122 is an optical disk apparatus, and 123 is a magnetic disk apparatus.
[0100]
The functions of the image processing unit 1 described in the above embodiments of the present invention can also be realized by a program 101 that can be executed by a computer. In that case, the program 101 and data used by the program can be stored in a computer-readable storage medium. A storage medium is a signal format that causes a state of change in energy such as magnetism, light, electricity, etc. according to the description of a program to a reader provided in the hardware resources of a computer. Thus, the description content of the program can be transmitted to the reading device. For example, a magneto-optical disk 111, an optical disk 112 (including a CD and a DVD), a magnetic disk 113, a memory 114 (including an IC card and a memory card), and the like. Of course, these storage media are not limited to portable types.
[0101]
By storing the program 101 in these storage media and mounting these storage media in, for example, the magneto-optical disk device 121, the optical disk device 122, the magnetic disk device 123, or a memory slot (not shown) of the computer 102, The program 101 can be read to execute the function of the image processing apparatus or the image processing method of the present invention. Alternatively, a storage medium may be attached to the computer 102 in advance, and the program 101 may be transferred to the computer 102 via a network, for example, and the program 101 may be stored and executed on the storage medium. The image data storage unit 2 may be a memory in the computer 102 or an attached magnetic disk device or other storage medium. Of course, some of the functions of the present invention may be configured by hardware, or all may be configured by hardware.
[0102]
【The invention's effect】
As is clear from the above description, according to the present invention, a feature amount of an image area having a predetermined size including the target pixel is calculated, and an image region having a predetermined size including the target pixel is calculated according to the calculated feature amount. The enlarged image area is generated by switching a plurality of enlargement methods for enlarging the image, and the obtained enlarged image area is arranged by a predetermined method to generate an enlarged output image. By such enlargement processing, for example, for a characteristic part in the input image, an enlargement method that preserves the feature amount is applied, and a flat part (image part such as sky or skin) in the input image is applied. For non-characteristic parts, an enlargement method with low processing load is applied to perform high-quality enlargement processing that suppresses blurring of characteristic parts and image quality defects such as jaggy, and at the same time, enlarges processing at low speed and with low processing load be able to.
[0103]
Further, a feature amount of an image region having a predetermined size including the target pixel is calculated, and an enlarged image region is generated by enlarging the image region having the predetermined size including the specific target pixel according to the calculated feature amount. Further, regardless of the feature amount, the input image is enlarged by an enlargement method different from the enlargement image region generation process to generate an enlarged image, and the enlarged image region is arranged at a corresponding position on the enlarged image by a predetermined method. Thus, an enlarged output image can be generated. With such enlargement processing, it is possible to limit enlargement processing for each image area with a heavy processing load to a characteristic part, and to obtain a high-quality enlarged image and realize high-speed enlargement processing with low processing load. There is an effect that can be done.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a first embodiment of an image processing apparatus and an image processing method according to the present invention.
FIG. 2 is a flowchart showing an example of an operation in the embodiment of the image processing apparatus and the image processing method of the present invention.
FIG. 3 is an explanatory diagram of an image block that is set and cut out by an image block setting unit 11;
FIG. 4 is a block diagram illustrating a configuration example of an image block feature quantity calculation unit 12;
FIG. 5 is an explanatory diagram of a specific example of an attention area and a peripheral area and an example of an edge direction of the attention area.
6 is a flowchart illustrating an example of edge direction estimation processing in an edge direction estimation unit 32. FIG.
FIG. 7 is an explanatory diagram of an example of a reference region used for estimating an edge direction.
8 is an explanatory diagram of an example of an estimated edge direction in the region of interest shown in FIG.
FIG. 9 is an explanatory diagram of an example of an edge pattern table.
10 is an explanatory diagram of a specific example of a method for selecting a first edge pattern corresponding to the region of interest shown in FIG.
11 is an explanatory diagram of a specific example of an emphasis kernel used in the first generation unit 21 of the enlarged image block generation unit 13. FIG.
12 is an explanatory diagram in a case where a pixel P is emphasized by the enhancement kernel shown in FIG.
FIG. 13 is an explanatory diagram of contrast enhancement processing by an enhancement kernel when input image data is enlarged by 8 times.
FIG. 14 is a flowchart illustrating an example of an enlarged image block generation process in the first generation unit 21 of the enlarged image block generation unit 13;
FIG. 15 is an explanatory diagram of a specific example of generation processing of an enlarged image block of 3 × 3 pixels.
FIG. 16 is an explanatory diagram illustrating an example of a calculation formula used when generating a second edge pattern and a 3 × 3 pixel enlarged image block;
FIG. 17 is an explanatory diagram of a specific example of generation processing of an enlarged image block of 3 × 4 pixels.
FIG. 18 is an explanatory diagram of a reference pixel selection method based on an estimated edge direction;
FIG. 19 is an explanatory diagram of a specific example of generation processing of an enlarged image block of 4 × 4 pixels.
FIG. 20 is an explanatory diagram illustrating an example of a process of generating an enlarged image block of 4 × 4 pixels when the estimated edge directions are direction 0 and direction 4;
FIG. 21 is an explanatory diagram of a specific example in which an enlarged image block of 4 × 4 pixels generated by the enlarged image block generation unit 13 is arranged.
FIG. 22 is a block diagram illustrating a second embodiment of an image processing apparatus and an image processing method according to the present invention.
FIG. 23 is a flowchart illustrating an example of an operation in the second embodiment of the image processing apparatus and the image processing method of the present invention;
FIG. 24 is an explanatory diagram of an example of a computer program and a storage medium storing the computer program when the function of the image processing apparatus or the image processing method of the present invention is realized by the computer program.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image processing part, 2 ... Image data storage part, 11 ... Image block setting part, 12 ... Image block feature-value calculation part, 13, 13 '... Enlarged image block generation part, 14, 14' ... Image block arrangement | positioning part, DESCRIPTION OF SYMBOLS 15 ... Enlarged image production | generation part, 21 ... 1st production | generation part, 22 ... 2nd production | generation part, 31 ... Edge strength calculation part, 32 ... Edge direction estimation part, 33 ... Edge pattern selection part, 101 ... Program, 102 ... Computer, DESCRIPTION OF SYMBOLS 111 ... Magneto-optical disk, 112 ... Optical disk, 113 ... Magnetic disk, 114 ... Memory, 121 ... Magneto-optical disk apparatus, 122 ... Optical disk apparatus, 123 ... Magnetic disk apparatus.

Claims (13)

  In an image processing apparatus that performs image enlargement processing, a region feature amount calculating unit that calculates a feature amount of an image region having a predetermined size including a target pixel, and the feature amount calculated by the region feature amount calculating unit is a predetermined amount. An enlarged image region generating unit that generates an enlarged image region for the image region that satisfies the condition: and an enlarged image generating unit that generates an enlarged image by enlarging the input image by an enlargement method different from the enlarged image region generating unit And an image arrangement unit that arranges the enlarged image region obtained by the enlarged image region generation unit at a corresponding position on the enlarged image obtained by the enlarged image generation unit. .   The image processing apparatus according to claim 1, wherein the region feature amount calculating unit calculates a degree of unevenness in the image region as the feature amount from each pixel value in the image region.   The image processing apparatus according to claim 1, wherein the region feature amount calculating unit calculates a change direction in the image region as the feature amount from each pixel value in the image region.   The area feature value calculating means calculates the feature value for each color component in the color space, and selects data of one color component in the color space based on the calculated one or more feature values. The image processing apparatus according to claim 1, wherein a feature amount of the image area in the color component data is used as a feature amount of the image area.   The enlarged image area generation means includes the feature quantity of the neighborhood area of the image area together with the feature quantity of the image area with respect to the image area where the feature quantity calculated by the area feature quantity calculation means satisfies a predetermined condition. The image processing apparatus according to claim 1, wherein the enlarged image area is generated using a pixel value in the area.   6. The enlarged image generation unit according to claim 1, wherein the enlarged image generation unit generates the enlarged image by an enlargement method having a lighter processing load than the enlargement method in the enlarged image region generation unit. The image processing apparatus described.   The image processing apparatus according to claim 1, wherein the enlarged image generating unit generates an enlarged image for each input image or for every several lines in the input image.   The said image arrangement | positioning means arrange | positions sequentially the enlarged image area | region produced | generated sequentially by the said enlarged image area | region production | generation means so that each may overlap, The one of Claim 1 thru | or 7 characterized by the above-mentioned. Image processing apparatus.   The image arrangement means calculates the sum of the overlapping pixel values for the overlapping pixels of the enlarged image region that are sequentially generated, and divides the sum of the pixel values by the number of overlaps to obtain the pixel value of the enlarged image region The image processing apparatus according to claim 8, wherein:   The image placement unit is configured to place each of the enlarged image regions generated by the enlarged image region generation unit at a corresponding position on the enlarged image obtained by the enlarged image generation unit. The image processing apparatus according to claim 1, wherein a pixel value is replaced with each pixel value of the enlarged image area.   In an image processing method for performing image enlargement processing, a feature amount of an image region having a predetermined size including a pixel of interest is calculated, and the calculated feature amount corresponds to the image region that satisfies a predetermined condition And generating an enlarged image by enlarging the input image by an enlargement method different from the enlargement method of the image region, and arranging the enlarged image region at a corresponding position on the enlarged image. Image processing method.   An image processing program for causing a computer to execute image enlargement processing, wherein the computer executes the function of the image processing apparatus according to any one of claims 1 to 10 or the image processing method according to claim 11. An image processing program characterized in that   The function of the image processing apparatus according to any one of claims 1 to 10, or the image processing according to claim 11, in a computer-readable storage medium storing a program for causing a computer to execute image enlargement processing. A computer-readable storage medium storing a program for causing a computer to execute the method.

Images