JP2004355654A - Image processing device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To convert a low-quality original image into a high-quality image in a relatively short time. <P>SOLUTION: A local area image is obtained for every pixel of the original image at a block size according to edge intensity. An image similar to the obtained local area image is detected by applying affine transformation to a peripheral image of the local area image. The detected affine transformation image is reduced to a size equal to that of the local area image to obtain a distance (similarity) to the local area image. The affine conversion image most similar to the local area image is substituted for the local area image. A part where the respective affine transformation images are superimposed on one another is processed by averaging them. By reducing the block size of the local area image as the degree of fractal similarity is intensified, a calculation time necessary for similarity determination or the like can be reduced. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and an image processing method capable of interpolating a low-quality image into a high-quality image, for example.

  For example, when converting a low-quality digital image to a high-quality image, converting a low-resolution image to a high-resolution image, or enlarging an image, a new pixel is placed between the original pixels. Image interpolation such as inserting a pixel is performed. As a method for interpolating a digital image, for example, a nearest neighbor interpolation method (also referred to as a nearest neighbor interpolation method, a zero-order hold method, a Nearest-Neighbor method), a linear interpolation method (linear interpolation method, bilinear interpolation method, Bi-Linear method) The third-order convolution interpolation method (also called a three-dimensional convolution method, a Bi-Cubic method, or a Cubic Convolution method) is known.

  Each of the above-described interpolation methods has a basic concept of interpolation using a sinc function based on the sampling theorem, and is theoretically correct only when the original image is composed of frequency components equal to or less than half the Nyquist frequency. However, since the frequency components contained in the actual original image are infinitely large, the above-described interpolation methods cannot restore the high-frequency components contained in the original image.

  Therefore, a frequency conversion method has been proposed as a method for interpolating the high-frequency components lost in the sampling process. As the frequency conversion method, the frequency components band-limited in the frequency domain are projected to the real space, only the restricted range of all the real space components is projected to the frequency space, and the band-limited portion of the total frequency components is Is replaced with a known original frequency component, and the operation of projecting again to the real space is infinitely repeated, and the Gerchberg-Papolis iterative method (GP method) is well known. In general, the computational burden is reduced by using DCT computation for frequency conversion (IM-GPDCT method).

  However, it is necessary to repeat the DCT operation and the inverse DCT operation until an appropriate high-frequency component is obtained, so that the processing time becomes long. In addition, noise may be emphasized, ringing may occur, and image quality may be degraded.

  SUMMARY An advantage of some aspects of the invention is to provide an image processing apparatus and an image processing method that can obtain a high-quality image at higher speed.

  In order to achieve the above object, in the present invention, for example, an acquisition size of an image block is set in accordance with the degree of fractal similarity, and each image block is acquired so that adjacent blocks overlap each other. An image similar to is detected and replaced.

  In the image processing device according to the present invention, the degree of similarity of the image around the pixel is determined for each predetermined pixel of the original image by the similarity determination unit. The block size setting means sets the size of the image block to be acquired from the original image according to the determined degree of similarity. The image block acquisition unit acquires the image blocks of the size set by the block size setting unit from the original image such that adjacent image blocks overlap by a predetermined amount. The similar image detecting means detects, for each of the acquired image blocks, a similar image block similar to each image block from a predetermined search area. The image replacement means replaces each detected similar image block with the corresponding image block. When the image adding means superimposes the replaced similar image blocks, the image adjusting means adjusts the image of the overlapped portion of the added blocks.

  The digital data of the original image is held by an original image holding unit such as a memory, for example. The image block acquisition unit acquires an image block of a predetermined size (n × m pixels, n = m may be used) from the held original image.

  Here, the acquisition size of the image block related to a certain pixel is set based on the degree of similarity of the image around the pixel. For example, the edge strength of an image of a predetermined size (x × y) having the pixel of interest substantially at the center can be used as the degree of similarity. The higher the degree of fractal self-similarity, the higher the edge strength. If the degree of fractal self-similarity is high, the size of the image block can be reduced. As the size of the image block is reduced, the processing time required for determining similarity with a similar image block can be reduced.

  Each image block of a predetermined size determined according to the degree of similarity is obtained such that adjacent image blocks overlap by a predetermined amount. For example, by acquiring, for each pixel of the original image, an image block of a predetermined size substantially at the center of the pixel, adjacent image blocks overlap by a predetermined amount.

  The similar image detecting means detects a similar image block similar to each image block from within a predetermined search area. A part of the image is a reduced form of the surrounding larger image, that is, a fractal similarity can be used to detect a similar image block similar to the image block from the original image. .

  More specifically, for example, the similar image detecting unit performs affine transformation such as parallel movement, enlargement, reduction, and rotation of the image in the search area, thereby obtaining a plurality of candidate images similar to the acquired image block. Blocks can be detected. By calculating the distance (similarity) between each of these candidate image blocks and the original image block, a similar image block most similar to the image block can be obtained.

  The similarity image detecting means can be set so that the search area becomes smaller as the degree of similarity becomes higher. It is preferable that the search area is set to be smaller than the original image at the maximum. The higher the degree of similarity, the higher the possibility that an image similar to an image block exists around the image block. By setting the search area variably according to the degree of similarity, the time for detecting similar image blocks can be reduced. The processing time can be reduced by setting the acquisition size of the image block and the search area for the similar image block according to the degree of similarity.

  Here, a similar image block is detected only for each image block whose degree of similarity exceeds a predetermined reference value, and similar image blocks are not detected for each image block that is equal to or less than the reference value. You can also.

  Each of the detected similar image blocks is replaced by a corresponding image block by the image replacement means. The replaced similar image blocks are overlapped by a predetermined amount and added. Since the adjacent image blocks are acquired so as to overlap each other by a predetermined amount, the blocks to be added also overlap the adjacent blocks by a predetermined amount. Therefore, the image adjusting means adjusts the value of the overlapping portion of the block. For example, the image can be adjusted by adding and averaging the values of the overlapping pixels. However, the present invention is not limited to this. For example, an average may be obtained by weighting.

  Each image block whose degree of similarity is equal to or less than a predetermined reference value can be used as it is. In this case, if roughly classified, three types of overlaps may occur: a place where similar image blocks overlap, a place where similar image blocks and image blocks overlap, and a place where image blocks overlap.

  The similar image detecting means may detect the similar image block with a size larger than the size of the image block, and reduce the size of the similar image block to be equal to the size of the image block.

  For example, assuming that the size of an image block is n × m and the size of a similar image block is (kn) × (km), the similar image block is reduced by a factor of k to obtain a similar image block. The size of the image blocks can be matched.

  Alternatively, the similar image detecting means detects a similar image block having the same size as the image block by thinning out pixels from a predetermined area set in the original image with a size larger than the size of the image block. It may be something.

In the above example, 1 / k ² pixels are extracted from a similar image block having (k · n) × (k · m) pixels, and the remaining pixels are discarded to obtain n × m pixels. And a similar image block having the same size as the image block having

  When the original image is a color image, by specifying one of the color system components related to the brightness of the image and the other color system components among the color system components of the original image, The image processing according to the present invention may be applied only to the designated color system components. In other words, for only the designated color system component, each image block can be replaced with a similar image block to adjust the overlapping portion.

  For example, taking the RGB color system as an example, of the R (red), G (green), and B (blue) color components, the G component has the greatest influence on the brightness of the image, and the R component and the B component Is involved in color. Further, for example, in the YUV color system, the YIQ color system, the YCbCr color system, and the Lab color system, the Y component or the L component is the component that is most involved in the brightness of an image, and the other components (U, V , I, Q, etc.) contribute to the color. Therefore, for example, a table related to brightness is set according to the type of product to which the image processing apparatus is applied (digital camera, printer, scanner, etc.), the characteristics of the original image (whether or not the image is a natural image), and user preferences. It is possible to select which of the color components and the other color components the image processing is applied to.

  The present invention can be understood as a recording medium on which a computer program is recorded. The program can be fixed to various tangible recording media such as a hard disk, a floppy disk, and a memory. The present invention is not limited to this, and a communication medium may be used, such as downloading a predetermined program from a server on a network.

  As described above, according to the image processing apparatus and the image processing method according to the present invention, a low-quality image can be converted into a high-quality image in a relatively short time.

  Hereinafter, an embodiment of the present invention will be described with reference to FIGS.

  FIGS. 1 to 7 are block diagrams showing the configuration of the image processing apparatus according to the first embodiment of the present invention.

  The image processing apparatus 1 performs predetermined image processing on an original image input from the original image holding unit 2, and outputs a processed image to the output image holding unit 13. The original image holding unit 2 and the output image holding unit 13 are configured by a memory or the like. The original image holding unit 2 and the output image holding unit 13 may be incorporated as a configuration of the image processing device 1 or may be configured separately from the image processing device 1. For example, when the original image data is read from the PC card on which the original image is recorded, and the processed image is written back to the empty area of the PC card, the PC card becomes the original image holding unit 2 and the output image holding unit 13. .

  The local area image obtaining section 3 as “image block obtaining means” obtains a local area image of a predetermined size for each pixel of the original image. The local area image corresponds to the “image block”. As will be described later with reference to FIG. 2, since a local region image is obtained for each pixel, adjacent local region images overlap by a predetermined amount. The local area image obtaining section 3 obtains a local area image of a predetermined size according to the edge strength based on the local area image obtaining parameters set by the parameter setting section 4. That is, the parameter setting unit 4 refers to the block size management table 15 based on the edge strength input from the edge strength determination unit 14, and calculates the block size according to the edge strength of the image around the pixel of interest in the local area image acquisition unit. Set to 3. In the block size management table 15, for example, the acquisition size of the local area image is associated with each of the low, medium, and high edge strength stages.

  The affine transformed image acquisition unit 6 detects and acquires an image similar to each local region image acquired by the local region image acquisition unit 3 from the original image. The affine transformation image acquisition unit 6 operates an image existing around the local region image based on the affine transformation parameters set by the affine transformation parameter setting unit 7 to detect an affine transformation image similar to the local region image. It is supposed to. An affine transformed image similar to the local region image corresponds to a “similar image block”.

  Here, based on the determination result from the edge intensity determination unit 14, the affine transformed image acquisition unit 6 determines whether the local area related to the pixel of interest is present only when the edge intensity of the image around the pixel of interest exceeds a predetermined reference value. An affine transformed image similar to the image is detected.

  The affine-transformed image acquisition unit 6 acquires an affine-transformed image having a size larger than the local region image, for example, a size that is twice as long in the vertical and horizontal directions. The obtained affine transformed image is reduced by the reducing unit 8 so as to have the same size as the local region image. The similarity calculating unit 9 calculates the distance between the local region image acquired by the local region image acquiring unit 3 and the affine transformed image acquired by the affine transformed image acquiring unit 6 and reduced by the reducing unit 8, and the similarity between the two is calculated. , To calculate the similarity. In the distance calculation, an eigenvector distance may be calculated, or a square mean may be simply taken. Here, the affine transformed image acquisition unit 6, the reduction unit 8, and the similarity calculation unit 9 correspond to “similar image detection means”.

  Then, the affine transformed image determined to be most similar by the similarity calculating unit 9 is replaced with a corresponding local area image by the replacing unit 10 as “image replacing means”.

  The adding unit 11 as an “image adding unit” adds the replaced affine transformed image and the non-replaced local region image. That is, the entire local region image acquired from the original image is replaced with an affine transformed image similar to the self image and added, or the original local region image is added as it is. Since the local region images are acquired such that the adjacent local region images overlap each other, the replaced affine transformed image and the original local region image are added so that the adjacent images overlap by a predetermined amount, and the adding unit 11 Is added by The averaging processing unit 12 as “image adjusting means” adjusts by averaging the values of the portion where each image overlaps. As an image adjustment method of the overlapping portion, a simple average may be taken, or a weighted average may be taken. The image adjusted in this way is held in the output image holding unit 13. After image adjustment, enlargement processing, reduction processing, rotation processing, color conversion processing, and the like may be performed.

  The target pixel pointer 5 detects the position of the pixel currently being processed (target pixel).

  The edge strength determination unit 14 as “similarity determination means” is for determining the degree of fractal property of the image around the target pixel. In the present embodiment, “edge strength” is used as the degree of fractal property. The edge strength will be described later with reference to FIG. The determined edge strength is input to the parameter setting unit 4.

  FIG. 2 is an explanatory diagram illustrating an outline of an image processing method by the image processing apparatus. As shown in FIG. 2A, a local area image having a predetermined size is set for each pixel of the original image, and an affine transformed image is obtained for each local area image.

  A local area image of, for example, an n × n size is acquired as a predetermined block size around the target image of the current process. The figure illustrates a case where n = 4. The affine transformed image is acquired so as to be larger than the local area image size, specifically, for example, twice as large in the vertical and horizontal directions (2n × 2n). In the embodiments, the local region image and the affine transformed image are described as squares unless otherwise specified, but the present invention is not limited to this. Each image can be set to another polygon such as a rectangle or a parallelogram.

  Here, an affine transformed image similar to the local region image can be searched for from the entire original image. However, the processing time becomes longer as the search area is expanded. Further, depending on the size of the image block (local region image), the characteristics of the original image, and the like, a similar image block (affine transformed image) may be more likely to be found around the image block. In particular, the present invention does not divide the entire original image into a plurality of image blocks that do not overlap each other, and obtain image blocks similar to each image block, but set so that a part of each adjacent image block overlaps. Therefore, the number of image blocks increases, and the number of times of detecting similar image blocks increases. Therefore, if the search area (search range) of the similar image block is extended to the entire original image, the processing time may be longer.

  Therefore, in the present embodiment, a similar image block is searched for in a search area set in a part of the original image, instead of searching the entire original image. The size of the search area can be dynamically changed according to the properties of the original image and the like. When the size of the image block is n × m, the size of the original image is Xmax and Ymax, and the size of the search area is αn × βm, the following relationship is established.

1 <α <Xmax / n (Equation 1)
1 <β <Ymax / m (Equation 2)

  The values of the coefficients α and β that determine the size of the search area can be set arbitrarily within a range that satisfies Equations 1 and 2. However, the present invention is not limited to this, and the entire original image may be searched.

  As shown in FIG. 2B, an affine transformed image (“candidate image block”) acquired at a size twice as large as the local region image is reduced to have the same size as the local region image. Then, the distance between the reduced affine transformed image and the local region image is calculated, and the similarity and the similarity are determined. An affine transformed image having the highest similarity is replaced with the local region image from among the affine transformed images obtained for a certain local region image.

  As shown in FIG. 2C, since the local region images are acquired such that the adjacent images overlap each other, the affine transformed images replaced with the respective local region images also have adjacent images. Duplicate. As shown in the figure, when acquiring a local area image of 4 pixels × 4 pixels by shifting one pixel at a time, focusing only on the x direction, a maximum of four images are overlapped. Similarly, a maximum of four local region images overlap in the y direction. Therefore, the overlapping part is adjusted by taking a simple average of the values of each pixel.

  In the present embodiment, an affine transformed image is acquired only for a local region image whose edge strength (fractal property) exceeds a predetermined reference value, instead of acquiring an affine transformed image for the entire local region image. Therefore, all four images shown in FIG. 2C are not always affine transformed images. The affine transformed images, the original local region images, and the affine transformed image and the local region image may overlap each other.

  FIG. 3 is an explanatory diagram showing an image operation method and parameters when acquiring an affine transformed image. As shown in FIG. 3A, the image can be translated by Sx and Sy. Further, as shown in FIG. 3B, the image can be rotated by the angle θ. Further, as shown in FIGS. 3C and 3D, the image can be enlarged or reduced Ex times in the x direction, or enlarged or reduced Ey times in the y direction.

  Next, the operation of the present embodiment will be described with reference to FIGS. Hereinafter, steps are abbreviated as “S”.

  First, FIG. 4 is a flowchart showing the overall flow of the image processing. In S1, (0, 0) is set to the coordinates (x, y) of the target pixel, and conversion is started from the first pixel of the original image. Next, the edge intensity is calculated for the image around the target pixel (S2). In S2, the edge strength of the image around the target pixel is calculated in order to determine the acquisition size of the local region image.

  As shown in FIG. 5, in the case of an image of 3 pixels × 3 pixels centered on the target pixel Px5, the level difference (| Px1 + Px2 + Px3 | − | Px7 + Px8 + Px9 |) of the columns located above and below the target pixel is taken as an example. The level difference (| Px1 + Px4 + Px7 |-| Px3 + Px6 + Px9 |) between the columns located on the left and right of the pixel is detected, and the edge strength is calculated based on the difference (S2). That is, the strength of the fractal property is determined based on the edge strength around the target pixel.

  Note that the size of the image region for edge strength determination does not always match the size of the local region image. This is because the size of the local area image changes according to the edge strength. Note that the example shown in FIG. 5 is merely an example, and the edge strength may be calculated with a size of 4 pixels × 4 pixels or 5 pixels × 5 pixels.

  Next, ranking is performed according to which stage the calculated edge strength corresponds to (S3, S6, S8). That is, whether the edge strength is equal to or lower than the level th1 as the “predetermined reference value” (S3), whether the edge strength is higher than the level th1, but equal to or lower than the next level th2 (S6), It is determined whether the level is higher than the level th2 but equal to or lower than the maximum level th3 (S8).

  Then, as shown in FIG. 6, it is determined whether or not to acquire the block size of the local area image and the affine transformation image according to the detected edge strength.

(1) In the case of edge strength ≦ th1 When the edge strength is equal to or lower than the level th1 (S3: NO), as shown in FIG. 6A, a local region image is acquired with a preset initial block size ns ( S4), the acquired local region images are added as they are (S5).

(2) In the case of th1 <edge strength ≦ th2 When the edge strength exceeds the level th1 and is equal to or lower than the level th2 (th2> th1) (S6: NO), as shown in FIG. "A local region image is acquired with a first block size n1 for edge strength (S7). In FIG. 6, the initial block size ns and the first block size n1 are both set to values of four pixels, but the two sizes do not need to match. The initial block size ns may be set larger than the first block size n1.

(3) th2 <edge strength ≦ th3
When the edge strength exceeds th2 and is equal to or lower than the level th3 (th3> th2) (S8: NO), as shown in FIG. 6C, the second block size n2 for the "medium" edge strength is used. A local region image is obtained (S9).

(4) th3 <Edge Intensity When the calculated edge intensity exceeds the level th3 (S8: YES), as shown in FIG. 6D, a local area having a third block size n3 for “high” edge intensity is used. An area image is obtained (S10).

  As described above, when the block size according to the edge strength is set, a local region image is acquired with this block size, and fractal interpolation is performed on the acquired local region image (S11). The fractal interpolation will be described later with reference to FIG.

  Then, the pixel of interest is moved to the next pixel (S12), and it is determined whether or not either an affine transformed image or a local region image has been acquired for all pixels of the original image (S13). If the processing of the entire original image has not been completed, the process returns to S2 and the above-described processing is repeated.

  For all pixels of the original image, when a local area image or an affine transformed image similar to the local area image is obtained and added (S13: YES), the overlapping part of each image is averaged, and the output image and (S14). When the image converted in this way is enlarged or the like, conventional interpolation processing such as linear interpolation can be further performed.

  Next, FIG. 7 is a flowchart showing the flow of the fractal interpolation process shown as S11 in FIG.

  First, the maximum value is set to the "minimum distance" for determining the similarity between the local area image and the affine transformed image (S21). Then, based on the block size set previously, a local region image having the pixel of interest substantially at the center is acquired (S22), and the parameters (Sx, Sy, Ex, Ey, θ) for affine transformation are respectively initialized to the initial values ( Is set (S23). The same applies to other steps described later, but the processing order of the steps can be changed as long as the processing is not affected. That is, the processing order of S21 to S23 does not matter.

  When an affine transformed image is acquired based on the set parameters (S24), the acquired affine transformed image is reduced so as to have the same size as the local region image (S25). Then, the distance between the local region image and the affine transformed image of the same size corresponding to the local region image is calculated (S26), and the calculated distance is calculated based on the value set in the similarity determination parameter “minimum distance”. Is smaller or not, that is, whether or not they are more similar (S27). If the latest distance calculation result is smaller than the "minimum distance", the latest affine transformation parameters and the distance values are held (S28). If the latest distance is not smaller than the “minimum distance”, the acquired affine transformed image does not resemble the local region image, and thus does not hold the value of each parameter and distance.

  Then, each parameter is changed by a predetermined amount (S29), and it is determined whether or not each parameter has exceeded the variable range (S30). That is, an affine transformation image is acquired while changing each parameter from the initial value to the upper limit value, and the distance to the local region image is calculated (S24 to S30). Therefore, in S28, the affine transformation parameters of the affine transformation image most similar to the local region image related to the current pixel of interest among the affine transformation images obtained within the variable range of each parameter and the distance thereof are held. .

  When each parameter is changed to the upper limit, an affine transformed image most similar to the local region image is obtained based on each parameter held in S28 (S31). Then, the acquired affine transformed image is reduced to the same size as the local region image (S32), and added to the previously acquired affine transformed image (S33). The addition means storing the digital data of the affine transformed image at a predetermined position in the memory area.

  Thus, of all the local region images acquired from the original image, for a local region image having a degree of fractal similarity higher than the reference value th1, an affine transformed image similar to the local region image is determined by a predetermined value in the original image. Detected from within the search area, replaced, and added. On the other hand, for a local region image whose fractal similarity is equal to or smaller than the reference value th1, the affine transformed image is not obtained, and the original local region image is added as it is.

  The present embodiment configured as above has the following effects.

  First, a local region image is acquired such that adjacent images overlap by a predetermined amount, and each local region image having a fractal similarity higher than a predetermined reference value th1 is similar to each local region image. Since the affine-transformed image is detected and replaced from the original image, a low-quality image can be converted into a high-quality image.

  Second, since the block size of the local area image is set according to the edge strength, the time required for similarity determination and the like can be reduced. That is, the higher the edge strength, the smaller the size of the local area image, and the number of pixels can be reduced to reduce the amount of calculation. In particular, in the present invention in which the local area images are acquired so as to overlap each other by a predetermined amount, a large number of local area images are acquired as compared with a case where the original image is simply divided and replaced with a similar image, but the edge image is acquired. The processing time can be reduced by variably setting the block size of the local region image according to the intensity.

  Third, since the affine transformation image is acquired only when the degree of the fractal property is higher than the predetermined reference value th1, the processing time is reduced as compared with the case where the affine transformation images are acquired for all the local region images. can do. Therefore, in combination with the configuration in which the block size of the local area image is variably set, further high-speed processing can be realized.

  Fourth, since the respective images are overlapped by a predetermined amount and the overlapped portions are processed by averaging or the like, it is possible to prevent a sense of incongruity at the joint between the images as compared with a case where the images are not overlapped. Therefore, for example, when the original image is a natural image, the quality can be improved while maintaining a natural gradation change.

  Fifth, while judging the similarity between the affine transformation image and the local region image, the affine transformation parameters most similar to the local region image are held, and after the search of the search region is completed, the stored affine transformation parameters are used. Since an affine transformed image is obtained, an affine transformed image similar to a local region image can be obtained with a small amount of memory resources.

  Next, a second embodiment of the present invention will be described with reference to FIG. In the following embodiments, the same components as those described above are denoted by the same reference numerals, and description thereof will be omitted. The feature of the present embodiment is that an affine transformed image having the same size as the local region image is obtained by thinning out the pixels and acquiring an affine transformed image.

  As shown in FIG. 8A, an affine transformed image is acquired from a predetermined search area at a size of 2n × 2n, and the acquired affine transformed image is reduced in length and width by 、 to obtain an n × n size. Affine transformed image of the same size as the local region image of Conversely, an affine transformation image having a size twice as large as the block size of the local region image is obtained, and this affine transformation image is reduced in length and width by 、, so that the affine transformation image having the same size as the original local region image is obtained. Images can be obtained.

  On the other hand, as shown in FIG. 8B, although the affine transformed image is acquired in the size of 2n × 2n, not all pixels in the size are acquired, but pixels are acquired every other pixel. By doing so, an affine transformed image having the same size as the local area image can be obtained without performing the reduction processing.

  Next, a third embodiment of the present invention will be described with reference to FIGS. The feature of this embodiment lies in that the block size of the local area image and the search area of the affine transformation image are set according to the edge strength.

  As shown in FIG. 9, the image processing device 1 according to the present embodiment further includes a parameter management table 16. The parameter management table 16 can be expressed as “search area setting means” together with the affine transformation parameters 7. The parameter management table 16 manages an affine transformation parameter group in association with each edge strength. In the parameter management table 16, the upper limit value and the lower limit value of the various parameters Sx, SY, θ, Ex, and Ey described above with reference to FIG. 3 are registered as a group. That is, each parameter group includes Sxmax, Sxmin, SYmax, Symin, θmax, θmin, Exmax, Exmin, Eymax, and Eymin.

  FIG. 10 is a flowchart showing the flow of processing according to the present embodiment. In this image processing, new steps are added after S7, S9, and S10. That is, after the block size of the local area image is set according to the edge strength (S7, S9, S10), parameters for the affine transformation are set according to the edge strength.

  Therefore, as shown in FIG. 11A, when the edge strength of the image around the pixel of interest is equal to or less than th1, a local area image is acquired with the initial block size ns, and the acquired local area image is added as it is. You. If th1 <edge strength ≦ th2, a local area image is acquired with a block size n1, and an affine transformation image similar to the acquired local area image is detected from a relatively large search area. If th2 <edge strength ≦ th3, a local area image is acquired with a block size n2, and an affine transformed image similar to this local area image is detected from a medium search area. Similarly, when th3 <edge strength, a local area image is acquired with a block size n3, and an affine transformed image is detected from a relatively small search area. As shown in FIG. 11, as the edge strength increases, the block size of the acquired local area image and the search area of the affine transformation image decrease.

  According to the present embodiment, not only the block size of the local area image but also the search area of the affine transformed image is variably set according to the edge strength, so that the time for the arithmetic processing can be reduced.

  Next, a fourth embodiment of the present invention will be described with reference to FIG. The feature of the present embodiment is that the block size of the local area image and the value of the affine transformation parameter can be set for each color component based on the characteristics of the original image and the like.

  First, when performing image modification by the image processing apparatus, the user can exclusively select the color blur prevention mode or the sharp mode (S51). In the color fringing prevention mode, the fractal interpolation according to the present invention is applied to the R component and the B component of the original image expressed in the RGB color system in order to suppress color fringing. On the other hand, in the sharp mode, fractal interpolation is applied only to the G component in order to sharpen the outline and the like of the image. For example, in a digital camera or the like in which one R component pixel and one B component pixel and two G component pixels form one unit, the color fringing prevention mode is applied to a natural image. The sharp mode is suitably used for images such as characters and diagrams.

  If the color blur prevention mode is selected, the R plane and the B plane are set as the color planes to be subjected to fractal interpolation (S52). On the other hand, when the sharp mode is selected, the G plane is set as the fractal interpolation target plane (S53).

  Then, similarly to the above, after setting the initial value of the target pixel (S54), the edge strength of the image around the target pixel is calculated (S55, S56). If the calculated edge strength exceeds the predetermined reference value th1 (S56: YES), the fractal interpolation is performed by setting the block size of the local area image and the affine transformation parameter according to the edge strength (S57, S58). . If the edge strength is equal to or smaller than the reference value th1 (S56: NO), the obtained local region image is added as it is (S59, S60). The above processes are repeated until the process is completed for all pixels of the original image (S61, S62).

  When the affine transformation image or the local area image has been acquired for all the pixels of the original image (S62: YES), the respective images are added and averaged (S63). Thus, processing of one plane is completed. Therefore, it is determined whether there is a plane to be processed next (S64). If there is a plane to be processed, the plane is switched to the plane and the above-described processing is performed (S65). When the processing has been completed for all the planes for which the execution of the fractal interpolation has been designated, the image is synthesized with another plane to obtain an output image (S66).

  That is, in the color fringing prevention mode, the fractal interpolation according to the present invention is performed on the image data of the R plane and the B plane, and the image data is synthesized with the G plane to obtain an output image. In this case, the G plane may be combined after performing other interpolation processing such as linear interpolation, or may be combined without performing the interpolation processing. Similarly, in the case of the sharp mode, the fractal interpolation according to the present invention is performed only on the image data of the G plane, and is combined with the R plane and the B plane.

  Thus, image processing can be performed according to the characteristics of the original image, the user's request, and the like.

  It should be noted that those skilled in the art can make various additions, modifications, combinations, and the like within the scope of the present invention described in each of the above embodiments. For example, in each of the above embodiments, the case where a local region image is obtained for each pixel of the original image is exemplified. However, the present invention is not limited to this. For example, local local images are obtained every other pixel, such as every other pixel. Pixels may be selected at predetermined intervals within a range in which the area images overlap by a predetermined amount.

FIG. 1 is a block diagram of an image processing device according to a first embodiment of the present invention. FIG. 4 is an explanatory diagram illustrating an outline of image processing. FIG. 4 is an explanatory diagram illustrating an outline of an affine transformation. 5 is a flowchart illustrating an image processing method. FIG. 4 is an explanatory diagram illustrating a method for calculating edge strength. FIG. 9 is an explanatory diagram showing a state in which the block size of a local area image changes according to the level of edge strength. 5 is a flowchart illustrating a flow of specific processing of fractal interpolation illustrated in FIG. 4. FIG. 9 is an explanatory diagram of an image processing method according to a second embodiment of the present invention. It is a block diagram of an image processing device concerning a 3rd embodiment of the present invention. 5 is a flowchart illustrating an image processing method. FIG. 7 is an explanatory diagram showing a state in which the block size of a local area image and the search area size of an affine transformation image change according to the edge strength. 13 is a flowchart of an image processing method according to a fourth embodiment of the present invention.

Explanation of reference numerals

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Original image holding part 3 Local area image acquisition part 4 Parameter setting part 5 Pixel pointer 6 Affine transformation image acquisition part 7 Affine transformation parameter setting part 8 Reduction part 9 Similarity calculation part 10 Substitution part 11 Addition part 12 Average Processing unit 13 output image holding unit 14 edge strength determination unit 15 block size management table 16 parameter management table

Claims (12)

An original image holding unit for holding an original image;
For each predetermined pixel, similarity determination means for determining the degree of similarity of the image around the pixel,
Block size setting means for setting the size of an image block to be obtained from the original image, according to the degree of similarity determined,
Image block acquisition means for acquiring the image blocks of the size set by the block size setting means, so that each adjacent image block overlaps by a predetermined amount, from the held original image,
For each of the image blocks, similar image detection means for detecting a similar image block similar to each image block from within a predetermined search area,
An image replacement unit that replaces each of the detected similar image blocks with a corresponding one of the image blocks,
Image adding means for superimposing the replaced similar image blocks,
Image adjustment means for adjusting the image of the overlapping portion of the block added by the image addition means,
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the block size setting unit sets the size of the image block to be acquired as the degree of the similarity increases. The similarity image detecting means detects the similarity image block only for an image block of a pixel for which the degree of similarity is determined to be higher than a predetermined reference value by the similarity determination means among the image blocks. And
The image adding unit is configured to overlap a similar image block replaced by the image replacing unit and an image block related to a pixel whose degree of similarity is determined to be equal to or less than the predetermined reference value. The image processing device according to claim 1 or 2. The image processing apparatus according to claim 1, wherein the similarity image detection unit sets the search area to be smaller as the degree of the similarity increases. The image processing apparatus according to claim 1, wherein an intensity of an edge of the image block is used as the degree of similarity. The image processing apparatus according to claim 1, wherein the image adjustment unit averages an image of the overlapping portion. The image processing apparatus according to any one of claims 1 to 6, wherein the image block obtaining unit obtains, for each pixel of the original image, an image block of a predetermined size centered on the pixel. . 2. The similar image detecting means detects the similar image block with a size larger than the size of the image block, and reduces the size of the similar image block so as to be equal to the size of the image block. The image processing apparatus according to claim 7. The similar image detecting means detects a similar image block having the same size as the image block by thinning out pixels from a predetermined area set in the original image with a size larger than the size of the image block. The image processing apparatus according to any one of claims 1 to 7, which performs the processing. An original image holding means for holding a color original image,
Designation means for designating one of the color system components related to the brightness of the image and the other color system components among the respective color system components of the original image,
For each predetermined pixel of the specified color system component, similarity determining means for determining the degree of similarity of the image around the pixel,
Block size setting means for setting the size of an image block to be obtained from the original image, according to the degree of similarity determined,
Image block acquisition means for acquiring the image blocks of the size set by the block size setting means, so that each adjacent image block overlaps by a predetermined amount, from the held original image,
For each of the image blocks, similar image detection means for detecting a similar image block similar to each image block from within a predetermined search area,
An image replacement unit that replaces each of the detected similar image blocks with the corresponding one of the image blocks,
Image adding means for superimposing each of the replaced similar image blocks,
Image adjustment means for adjusting the image of the overlapping portion of the block added by the image addition means,
Combining means for combining an original image other than the color system component specified by the specifying means and the image adjusted by the adjusting means,
An image processing apparatus comprising: For each predetermined pixel of the original image, based on the degree of similarity of the image around the pixel, set the size of the image block to be acquired, respectively,
In such a manner that adjacent image blocks overlap by a predetermined amount, each image block is sequentially acquired from the original image at the set size,
Detecting similar image blocks similar to the respective obtained image blocks from the original image,
Each of the detected similar image blocks is replaced with a corresponding one of the image blocks,
Superimposing each of the replaced similar image blocks,
Adjusting and outputting the image of the overlapping portion of the added block,
An image processing method comprising: In a recording medium recording an image processing program for causing a computer to execute image processing,
For each predetermined pixel of the held original image, a function of determining the degree of similarity of the image around the pixel,
A function of setting a size of an image block to be obtained from the original image, according to the degree of similarity determined;
A function of causing the image blocks of the set size to be obtained from the original image, such that adjacent image blocks overlap by a predetermined amount,
For each of the image blocks, similar image detection means for detecting a similar image block similar to each image block from within a predetermined search area,
An image replacement unit that replaces each of the detected similar image blocks with a corresponding one of the image blocks,
Image adding means for superimposing the replaced similar image blocks,
Image adjustment means for adjusting the image of the overlapping portion of the block added by the image addition means,
An image processing apparatus comprising:

Images