JP3491830B2 - Image processing apparatus and image processing method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method capable of, for example, interpolating a low quality image and converting it into a high quality image.

[0002]

2. Description of the Related Art For example, when converting a low-quality digital image into a high-quality image, converting a low-resolution image into a high-resolution image, or enlarging an image, the original pixel and pixel Image interpolation such as inserting a new pixel between the two is performed. As an interpolation method of a digital image, for example, a nearest neighbor interpolation method (nearest neighbor interpolation method, zero-order hold method,
Nearest-Neighbor method), linear interpolation method (linear interpolation method, bilinear interpolation method, also called Bi-Linear method),
Cubic convolutional interpolation method (three-dimensional convolution method, Bi
-Cubic method, called Cubic Convolution method) is known.

[0003]

The above-mentioned interpolation methods are as follows.
Since the basic concept is interpolation by the sinc function based on the sampling theorem, it is theoretically correct only when the original image is composed of frequency components equal to or lower than half the Nyquist frequency. However, since the frequency components contained in the actual original image are infinitely large, the high frequency components contained in the original image cannot be restored by each of the interpolation methods.

Therefore, a frequency conversion method has been proposed as a method for interpolating the high-frequency component lost in such a sampling process. As the frequency conversion method, the frequency component whose band is limited in the frequency domain is projected to the real space, only the limited range of all the real space components is projected to the frequency space, and the band limited part of all the frequency components is projected. A well-known Gerchberg-Papolis iterative method (GP method) in which the operation of substituting a known original frequency component and projecting it again in real space is infinitely repeated. Generally, the DCT calculation is used for frequency conversion to reduce the calculation load (IM-GPDCT method).

However, since it is necessary to repeat the DCT calculation and the inverse DCT calculation until an appropriate high frequency component is obtained, the processing time becomes long. In addition, noise may be emphasized or ringing may occur, resulting in deterioration of image quality.

The present invention has been made in view of the above problems, and an object thereof is to provide an image processing apparatus and an image processing method capable of obtaining a high quality image at a higher speed.

[0007]

In order to achieve the above object, according to the present invention, each image block is acquired such that adjacent blocks overlap each other, and is set according to the degree of fractal similarity of each image block. An image similar to each of the image blocks is detected and replaced in the search area.

In the image processing apparatus according to the present invention, the image block acquisition means causes the image blocks of a predetermined size to be overlapped with each other by a predetermined amount from the original image held in the original image holding means. Get each. The similarity determination means determines the degree of similarity for each of the acquired image blocks. The similar image detecting means sets a search area for each of the image blocks based on the determined degree of similarity, and detects a similar image block similar to the image blocks from within the search area. The image replacement means replaces each detected similar image block with each corresponding image block. When the image adding means superimposes the replaced similar image blocks, the image adjusting means adjusts the image of the overlapping portion of the added blocks.

The digital data of the original image is held by an original image holding means such as a memory. The image block acquisition means acquires an image block of a predetermined size (n × m pixels, n = m may be used) from the held original image. Here, the image blocks are acquired such that adjacent image blocks overlap each other by a predetermined amount. For example, by acquiring an image block of a predetermined size with the pixel substantially centered for each pixel of the original image, adjacent image blocks are overlapped by a predetermined amount.

The similarity determining means determines the degree of similarity for each image block. When a fractal similar image block is used as the similar image block, it can be expressed as “degree of fractalness”. As the degree of similarity, the edge strength of the image block can be used. This is because an image having a higher fractal self-similarity has a property of higher edge strength. Alternatively, the distance between an image block and an image around the image block (for example, an image of a region obtained by multiplying the size of the image block by four) can be used as the “degree of similarity”.

The similar image detecting means sets a search area for each image block according to the degree of similarity. The similar image detecting means detects a similar image block similar to the image block from within the search area. The similar image detection means is
The search area can be set smaller as the degree of similarity becomes higher. The search area is preferably set to be smaller than the original image even at the maximum.

A part of the image is a reduced image of a larger image in the surroundings, that is, the fractal similarity is used to search for a similar image block similar to the image block in the original image. Each can be detected in. The higher the degree of similarity is, the more likely an image that is similar to an image block is around the image block. By variably setting the search area according to the degree of similarity, it is possible to shorten the time required to detect the similar image block.

Here, the similar image block is detected only for each image block whose degree of similarity exceeds a predetermined reference value, and the similar image block is not detected for each image block below the reference value. Can be

Each of the detected similar image blocks is replaced by the corresponding image block by the image replacing means. The replaced similar image blocks are overlapped by a predetermined amount and added. Since the adjacent image blocks are acquired such that they overlap each other by a predetermined amount, the blocks to be added also overlap each other by a predetermined amount. Therefore, the image adjusting means adjusts the value of the overlapping portion of the blocks. For example, the image can be adjusted by adding the values of overlapping pixels and averaging them. Not limited to this, for example, weighting may be performed and an average may be taken.

Each image block whose degree of similarity is equal to or less than a predetermined reference value can be used as it is. In this case, if roughly classified, three types of overlap may occur: a place where similar image blocks overlap with each other, a place where similar image blocks overlap with each other, and a place where image blocks overlap with each other.

The similar image detecting means may detect the similar image block with a size larger than the size of the image block and reduce the size of the similar image block to be equal to the size of the image block. .

For example, if the size of the image block is n × m,
If the size of the similar image block is (k · n) × (k · m), the size of the similar image block can be reduced to 1 / k so that the sizes of the image block and the similar image block can be matched.

Alternatively, the similar image detecting means thins out pixels from a predetermined area set in the original image with a size larger than the size of the image block, so that the similar image block having the same size as the image block. May be detected.

In the above example, (k · n) × (k · n
By extracting 1 / k2 pixels from the similar image block having m) pixels and discarding the remaining pixels, a similar image block having the same size as the image block having n × m pixels can be obtained.

The similar image detecting means can detect similar image blocks similar to each image block in the original image under predetermined conditions.

The predetermined condition may be a search region setting condition for detecting a similar image block. Then, the similar image detecting means may be configured to detect a similar image block similar to the image block in the search area set smaller than the original image.

The similar image detecting means determines the similarity of each candidate image block while changing the image operation parameter,
You may make it hold | maintain only the parameter used for acquisition of the candidate image block which is most similar to an image block, and detect a similar image block by this held parameter.

The "image manipulation" may include, for example, affine transformation for performing parallel movement, enlargement, reduction, and rotation of an image. The “parameter of image operation” can include, for example, the amount of parallel movement, the enlargement ratio, the reduction ratio, and the rotation angle. The similar image detecting unit acquires a plurality of candidate image blocks in a predetermined search area while changing the image operation parameter. Then, among the plurality of candidate image blocks, the one most similar to the image block is detected as a similar image block.

When each candidate image block is held, the memory area for storing each candidate image block becomes large, but only the parameters necessary to obtain the candidate image block with the minimum distance should be held. Due to
Similar image blocks can be detected with a small memory area.

When the original image is a color image, one of the color system components relating to the brightness of the image and the other color system components among the color system components of the original image should be designated. Therefore, the image processing according to the present invention may be applied only to the designated color system component. That is, it is possible to replace the image blocks with similar image blocks only for the designated color system component to adjust the overlapping portion.

For example, taking the RGB color system as an example, R
Among the color components of (red), G (green), and B (blue), the G component has the largest contribution to the brightness of the image, and the R component and the B component contribute to the tint. Also, for example, YUV color system, YI
In the Q color system, the YCbCr color system and the Lab color system, Y
The component or the L component is the component most related to the brightness of the image, and the other components (U, V, I, Q, etc.) are related to the tint. Therefore, for example, the type of product to which the image processing apparatus is applied (digital camera, printer, scanner, etc.)
Whether this image processing is applied to the color system components related to brightness or other color system components according to the characteristics of the original image (whether it is a natural image or not), user preference, etc. Can be selected.

The present invention can also be understood as a recording medium recording a computer program. The program is, for example, a hard disk or floppy (registered trademark)
It can be fixed to various tangible recording media such as disks and memories. Further, the communication medium can be used, not limited to this, such as downloading a predetermined program from a server on the network.

[0028]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to FIGS.

1. First Embodiment FIGS. 1 to 7 are block diagrams showing the configuration of an image processing apparatus according to a first embodiment of the present invention.

The image processing apparatus 1 is for applying a predetermined image processing to the original image input from the original image holding unit 2 and outputting the processed image to the output image holding unit 13. The original image holding unit 2 and the output image holding unit 13 are composed of a memory and the like. Original image holding unit 2 and output image holding unit 13
May be incorporated as a configuration of the image processing apparatus 1,
The image processing device 1 may be configured separately.
For example, when the original image data is read from the PC card on which the original image is recorded and the processed image is written back to the empty area of the PC card, the PC card serves as the original image holding unit 2 and the output image holding unit 13. .

The local area image acquisition unit 3 as "image block acquisition means" acquires a local area image of a predetermined size for each pixel of the original image. The local area image corresponds to the “image block”. As will be described later with reference to FIG. 2, since local area images are acquired for each pixel, adjacent local area images are overlapped by a predetermined amount. The local area image acquisition unit 3 acquires a local area image based on the local area image acquisition parameters set by the parameter setting unit 4. Examples of this parameter include the size of the local area image. The pixel-of-interest pointer 5 detects the position of the pixel currently being processed (pixel of interest).

The affine-transformed image acquisition unit 6 determines, based on the determination result from the edge intensity determination unit 14, an image similar to each local region image from within the search region corresponding to the edge intensity of each local region image. It is detected and acquired from the image. The affine transformation image acquisition unit 6 operates an image existing around the local region image based on the affine transformation parameter set by the affine transformation parameter setting unit 7, and detects an affine transformation image similar to the local region image. It is supposed to do. The affine-transformed image similar to the local area image corresponds to the “similar image block”.

The parameter management table 15 manages the edge strength and the affine transformation parameter as an example of the “degree of similarity” in association with each other. In the parameter management table 15, for example, affine transformation parameter groups are associated with the respective edge strength levels of “low”, “medium”, and “high”. The affine transformation parameter setting unit 7 determines the parameter management table 1 based on the edge strength input from the edge strength determination unit 14.
5, a predetermined affine transformation parameter group is acquired, and the obtained affine transformation parameter group is set in the affine transformation image acquisition unit 6. As a result, the affine-transformed image acquisition unit 6 acquires the affine-transformed image that most resembles each local area image in the search area corresponding to the edge strength of each local area image. When the edge intensity of the local area image is equal to or less than the predetermined reference value, the affine transformed image acquisition unit 6 does not acquire the affine transformed image for the local area image.

The affine-transformed image acquisition unit 6 acquires an affine-transformed image having a size larger than that of the local area image, for example, twice the size in the vertical and horizontal directions. The acquired affine-transformed image is reduced by the reducing unit 8 to have the same size as the local area image. The similarity calculation unit 9 calculates the distance between the local area image acquired by the local area image acquisition unit 3 and the affine-transformed image acquired by the affine-transformed image acquisition unit 6 and reduced by the reduction unit 8, and the similarity between the two is calculated. , The degree of similarity is calculated. In the distance calculation, the eigenvector distance may be calculated or the root mean square may be simply calculated. Here, the affine-transformed image acquisition unit 6, the reduction unit 8, and the similarity calculation unit 9 correspond to “similar image detection means”.

Then, the affine-transformed image determined to be most similar by the similarity calculation unit 9 is replaced by the corresponding local area image by the replacement unit 10 as "image replacement means".

The adding section 11 as "image adding means"
The replaced affine transformation image and the non-replaced local area image are added. That is, the entire local region image acquired from the original image is replaced with an affine-transformed image similar to self and added, or the original local region image is added as it is. Since the local area images are acquired such that the adjacent local area images overlap with each other, the replaced affine-transformed image and the original local area image are arranged such that the adjacent images overlap each other by a predetermined amount, and the addition unit 11 Is added by. The averaging processing unit 12 as "image adjusting means" adjusts by averaging the values of the portions where the images overlap. As an image adjusting method for overlapping portions, a simple average may be taken, or weighted average may be taken. The image adjusted in this way is held in the output image holding unit 13. Note that enlargement processing, reduction processing, rotation processing, color conversion processing, etc. may be performed after image adjustment.

The edge strength determination unit 14 as "similarity determination means" determines the degree of fractal property of each acquired local area image. For the local area image determined to have a fractal degree exceeding a predetermined reference value, the affine transformation image is acquired by the affine transformation image acquisition unit 6 using the affine transformation parameter corresponding to the edge strength of the local area image. To be detected. The local area image whose degree of fractalness is less than or equal to a predetermined reference value is directly added to the affine-transformed image. That is, the affine-transformed images are not acquired for all the local-region images acquired from the original image, but the affine-transformed images are acquired only for the local-region images having a high degree of fractal property.
Edge strength can be used as the “degree of fractal property”. This will be described later with reference to FIG.

FIG. 2 is an explanatory view showing the outline of the image processing method by the image processing apparatus. As shown in FIG. 2 (a),
A local area image is set for each pixel of the original image, and an affine-transformed image is acquired for each local area image having a high edge strength in each local area image.

A local area image of, for example, n × n size is obtained with the image of interest in the current process as the center. In the figure, the case of n = 4 is illustrated. The affine-transformed image is acquired to have a size larger than the local area image size, specifically, twice the size in the vertical and horizontal directions (2n × 2n). In the embodiments, unless otherwise specified, the local area image and the affine transformation image are described as squares, but the present invention is not limited to this. Each image can be set to another polygon such as a rectangle or a parallelogram. It should be noted that the search area for detecting an affine transformed image also changes depending on the edge strength of the local area image.

Here, the affine-transformed image similar to the local area image can be searched from the entire original image. However, the wider the search area, the longer the processing time. Further, depending on the size of the image block (local area image), the characteristics of the original image, and the like, the similar image block (affine-transformed image) is likely to be found around the image block. In particular, the present invention does not divide the entire original image into a plurality of image blocks that do not overlap each other, and obtains an image block similar to each image block, but sets such that a part of each adjacent image block overlaps. Therefore, the number of image blocks increases, and the number of times that similar image blocks are detected increases. Therefore, if the search area (search range) of the similar image block is expanded to the entire original image, the processing time may be longer.

Therefore, in the present embodiment, instead of searching the entire original image, a similar image block is searched for in a part of the original image, and further in a search area corresponding to the edge strength. There is. The size of the search area can be dynamically changed according to the nature of the original image and the like. The image block size is n × m, the original image size is Xmax, Ymax,
When the size of the search area is αn × βm, the following relationship holds.

1 <α <Xmax / n (Equation 1) 1 <β <Ymax / m (Equation 2) The values of the coefficients α and β for determining the size of the search area are the same as those in Equation 1 above. It can be arbitrarily set within the range of 2.
However, the present invention is not limited to this, and the entire original image may be searched.

As shown in FIG. 2B, an affine-transformed image (“candidate image block”) acquired in a size twice as large as the local region image is reduced to have the same size as the local region image. It Then, the distance between the reduced affine-transformed image and the local region image is calculated, and the similarity and similarity are determined. The affine-transformed image having the highest similarity among the affine-transformed images acquired for a certain local-region image is replaced with the local-region image.

As shown in FIG. 2C, since the local area images are acquired so that the adjacent images overlap each other, the affine-transformed images replaced by the local area images are also adjacent to each other. The images overlap. As shown in the figure, when a local area image of 4 pixels × 4 pixels is obtained by shifting each pixel by one pixel, if attention is paid only to the x direction, a maximum of 4 images will overlap. Similarly, in the y direction, a maximum of four local region images overlap. Therefore, the overlapping portion is adjusted by taking the simple average of the values of each pixel.

In the present embodiment, the affine-transformed images are not acquired for all the local-region images, but the affine-transformed images are acquired only for the local-region images whose edge strength (fractalness) exceeds a predetermined reference value. ing.
Therefore, not all the four images shown in FIG. 2C are affine-transformed images. The affine transformed images may overlap each other, the original local region images may overlap each other, and the affine transformed image and the local region image may overlap each other.

FIG. 3 is an explanatory diagram showing an image operation method and parameters when acquiring an affine-transformed image. Figure 3
As shown in (a), the image can be translated by Sx and Sy. Further, as shown in FIG. 3B, the image can be rotated by the angle θ. Furthermore, FIG.
As shown in (c) and (d), the image can be enlarged or reduced by Ex times in the x direction or Ey times or reduced in the y direction.

In the parameter management table 15, upper limit values and lower limit values of these various parameters Sx, SY, θ, Ex, Ey are registered as a group. That is,
Each parameter group consists of Sxmax, Sxmin, SYmax, Sym
in, θmax, θmin, Exmax, Exmin, Eymax, Eym
Composed of in. The affine-transformed image acquisition unit 6
An affine transformed image is acquired by changing each parameter from the lower limit value min to the upper limit value max.

Next, the operation of the present embodiment will be described with reference to FIGS. Hereinafter, the step is abbreviated as “S”.

First, FIG. 4 is a flowchart showing the overall flow of image processing. In S1, (0, 0) is set to the coordinates (x, y) of the pixel of interest, and conversion is started from the first pixel of the original image. Next, the edge strength of the local area image centered on the pixel of interest is calculated (S2). As shown in FIG. 5, taking a region of 3 pixels × 3 pixels centering on the target pixel Px5 as an example, the level difference (| Px1 + Px2 + Px3 | − | Px7 +) between the columns located above and below the target pixel is taken into consideration.
Px8 + Px9 |) and the level difference (| Px1 + Px4 + Px7 | − | Px3 + P) between the columns located to the left and right of the pixel of interest.
x6 + Px9 |) is detected, and thereby the edge strength is calculated (S2). That is, the strength of the fractal property is determined by the edge strength around the pixel of interest.

The area for edge strength judgment and the size of the local area image do not necessarily have to match.
For example, when acquiring a local area image of 4 pixels × 4 pixels centered on the target pixel, 3 pixels centered on the target pixel ×
It is possible to judge the edge strength in the area of 3 pixels,
It is also possible to determine the edge strength in a region of 5 pixels × 5 pixels or more. In the present embodiment, the edge strength determination area and the size of the local area image are made to match.

Next, ranks are assigned to which stage the calculated edge strength corresponds (S3, S6, S).
8). That is, whether the edge strength is below the level th1 as a "predetermined reference value" (S3), whether the edge strength is above the level th1 but below the next level th2 (S6), the edge strength is It is determined whether or not the value is higher than the level th2 but equal to or lower than the highest level th3 (S8).

Then, as shown in FIG. 6, whether or not the affine-transformed image is to be acquired and the range of the search area when the affine-transformed image is acquired are determined according to the detected edge strength.

(1) When Edge Strength ≦ th1 When the edge strength is equal to or lower than the level th1 (S3: NO), a local area image is acquired (S4), and the acquired local area image is added as it is ( S5). As shown in FIG. 6A, the local area image is used as it is, and the affine transformed image is not detected.

(2) When th1 <edge strength ≦ th2, the edge strength exceeds the level th1 and the level th2
If (th2> th1) or less, (S6: NO),
The first affine transformation parameter group for “low” edge strength is set in the affine transformation image acquisition unit 6 (S7). As a result, as shown in FIG. 6B, an affine-transformed image similar to the local area image is detected within the wide search area.

(3) th2 <edge strength ≦ th3 The edge strength exceeds th2, and the level th3 (th
If 3> th2) or less (S8: NO), the second affine transformation parameter group for “medium” edge strength is set in the affine transformation image acquisition unit 6 (S9). This allows
As shown in FIG. 6C, an affine-transformed image similar to the local area image is detected in the search area having a medium size.

(4) th3 <Edge Strength When the calculated edge strength exceeds the level th3 (S8: YES), the third affine transformation parameter group for “high” edge strength is sent to the affine transformation image acquisition unit 6. Set (S10). As a result, as shown in FIG. 6D, an affine-transformed image similar to the local area image is detected within the small search area.

As described above, when the affine transformation parameter is set according to the edge strength of the local area image, fractal interpolation is performed using this affine transformation parameter (S11). For fractal interpolation,
It will be described later with reference to FIG. 7.

Then, the pixel of interest is moved to the next pixel (S12), and it is determined whether or not an affine transformed image or a local area image has been acquired for all the pixels of the original image (S13). If the processing of the entire original image has not been completed, the process returns to S2 and the above-described processing is repeated.

When the local area image corresponding to each pixel of the original image or the affine transformation image similar to the local area image is acquired and added (S13: YES), the overlapping portions of the respective images are averaged, Output image (S1
4). When enlarging the image converted in this way, a conventional interpolation process such as linear interpolation can be further performed.

Next, FIG. 7 is a flow chart showing the flow of the fractal interpolation processing shown as S11 in FIG.

First, the maximum value is set to the "minimum distance" for determining the similarity between the local area image and the affine transformed image (S21). Then, a local area image having the pixel of interest as its center is obtained (S22), and initial values (lower limit values) are set to the respective parameters (Sx, Sy, Ex, Ey, θ) for affine transformation (S23). . The same applies to the other steps described below, but the processing order of the steps can be changed as long as the processing is not affected. That is, the processing order of S21 to S23 does not matter.

When the affine-transformed image is obtained based on the set parameters (S24), the obtained affine-transformed image is reduced to have the same size as the local area image (S25). Then, the distance between the local region image and the affine-transformed image of the same size corresponding to the local region image is calculated (S26), and the calculated distance is calculated from the value set in the parameter "minimum distance" for similarity determination. Is also small, that is, whether or not they are more similar (S2)
7). If the latest distance calculation result is smaller than the "minimum distance", the latest affine transformation parameter and distance value are held (S28). If the latest distance is not less than the "minimum distance", then the acquired affine transformed image is not similar to the local region image,
It does not hold the value of each parameter and distance.

Then, each parameter is changed by a predetermined amount (S29), and it is determined whether or not the variable range of each parameter is exceeded (S30). That is, while changing each parameter from the initial value to the upper limit value, the affine-transformed image is acquired and the distance to the local region image is calculated (S24).
~ S30). Therefore, in S28, among the affine-transformed images obtained within the variable range of each parameter, the affine-transformed parameter of the affine-transformed image that most resembles the local region image of the current pixel of interest and its distance are held. .

When each parameter is changed to the upper limit, an affine-transformed image most similar to the local area image is acquired based on each parameter held in S28 (S31). Then, the acquired affine-transformed image is reduced to the same size as the local area image (S32) and added to the affine-transformed image acquired previously (S33). Addition is to store the digital data of the affine-transformed image at a predetermined position in the memory area.

As a result, the degree of fractal similarity among all local region images acquired from the original image is the reference value th.
For a local area image higher than 1, an affine-transformed image similar to the local area image is detected from a predetermined search area in the original image, replaced, and added. On the other hand, for the local area image whose degree of fractal similarity is equal to or smaller than the reference value th1, the affine-transformed image is not acquired, and the original local area image is added as it is.

In the present embodiment configured as above,
The following effects are achieved.

First, local area images are acquired by causing adjacent images to overlap each other by a predetermined amount, and for each local area image having a fractal similarity higher than a predetermined reference value th1, the local area images are acquired. Since an affine-transformed image similar to is detected and replaced in the original image, a low-quality image can be converted into a high-quality image.

Secondly, since the search area for affine transformation image detection is set according to the edge strength of the local area image, only the affine transformation image is detected within the search area according to the strength of the fractal similarity. The processing speed can be improved and the image processing time can be shortened. In particular, in the present invention in which the local area images are acquired by overlapping each other by a predetermined amount, a large number of local area images are acquired as compared with the case where the original image is simply divided and replaced with a similar image. The processing time can be shortened by variably setting the search area according to the strength.

Thirdly, since the affine-transformed image is acquired only when the degree of fractal property is higher than the predetermined reference value th1, the processing is performed as compared with the case where the affine-transformed images are acquired for all local region images. The time can be shortened. Therefore, even higher speed processing can be realized in combination with the variable setting configuration of the search area.

Fourth, each image is overlapped by a predetermined amount,
Since the overlapping portions are processed by averaging or the like, it is possible to prevent a sense of discomfort from occurring at the joint between the images as compared with the case where the images are not overlapped. Therefore, for example, when the original image is a natural image, the quality can be improved while maintaining a natural gradation change.

Fifth, while judging the similarity between the affine-transformed image and the local area image, the affine transformation parameters most similar to the local area image are held, and after the search of the search area is completed, the held affine Since the affine-transformed image is obtained by the transformation parameter, it is possible to obtain the affine-transformed image similar to the local area image with a small memory resource.

2. Second Embodiment Next, a second embodiment of the present invention will be described with reference to FIG. In each of the following embodiments, the same components as those described above are designated by the same reference numerals, and the description thereof will be omitted. The feature of this embodiment is that an affine-transformed image having the same size as the local region image is obtained by thinning out pixels to obtain an affine-transformed image.

As shown in FIG. 8A, an affine-transformed image having a size of 2n × 2n is obtained from within a predetermined search region, and the obtained affine-transformed image is reduced to have the same size as the local region image. Affine-transformed image can be obtained.

On the other hand, as shown in FIG.
Although an affine-transformed image is acquired in a size of n × 2n, not all pixels in the size are acquired, but every other pixel is thinned out and acquired, so that a local area image is obtained without performing reduction processing. Affine transformed images of the same size can be obtained.

3. Third Embodiment Next, a third embodiment of the present invention will be described with reference to FIG. The feature of this embodiment is that the value of the affine transformation parameter can be set for each color component based on the characteristics of the original image and the like.

First, the user can exclusively select the color fringing prevention mode or the sharp mode when performing the image modification by the image processing apparatus (S41). In the color fringing prevention mode, RG is used to suppress color fringing.
The fractal interpolation according to the present invention is applied to the R and B components of the original image represented by the B color system. On the other hand, in the sharp mode, fractal interpolation is applied only to the G component in order to sharpen the contour of the image. For example, in a digital camera or the like in which one R component pixel and one B component pixel and two G component pixels form one unit, the color fringing prevention mode is applied to a natural image. The sharp mode is preferably used for images such as characters and diagrams.

When the color fringing prevention mode is selected,
The R plane and the B plane are set to the color plane that is the target of fractal interpolation (S42). On the other hand, when the sharp mode is selected, the G plane is set as the fractal interpolation target plane (S43).

Then, similarly to the above, after setting the initial value of the target pixel (S44), the edge strength of the local area image centered on the target pixel is calculated (S45, S46). When the calculated edge strength exceeds the predetermined reference value th1 (S46: YES), fractal interpolation is performed using the affine transformation parameter according to the edge strength (S47, S4).
8). If the edge strength is less than or equal to the reference value th1, (S46: N
O), the acquired local area image is added as it is (S4)
9, S50). The above processes are repeated until the process is completed for all pixels of the original image (S51, S52).

When affine-transformed images or local area images have been acquired for all pixels of the original image (S52: YES), each image is added and averaged (S53). This completes the processing of one plane. Therefore, it is determined whether or not there is a plane to be processed next (S54), and if there is a plane to be processed, the plane is switched to that plane and the above-described processing is performed (S55). When the processing is completed for all the planes for which the fractal interpolation is designated, the image is combined with other planes to obtain an output image (S5).
6).

That is, in the color fringing prevention mode, the fractal interpolation according to the present invention is performed on the image data of the R plane and the B plane, and the output image is obtained by combining with the G plane. In this case, for the G plane,
The synthesis may be performed after performing other interpolation processing such as linear interpolation, or may be performed without performing the interpolation processing. Similarly, in the sharp mode, the fractal interpolation according to the present invention is performed only on the image data of the G plane, and is combined with the R plane and the B plane.

As a result, image processing can be performed according to the characteristics of the original image and the user's wishes.

Those skilled in the art can make various additions, changes, combinations and the like within the scope of the gist of the present invention described in each of the above embodiments. For example, in each of the above-described embodiments, the case where the local area image is acquired for each pixel of the original image is illustrated, but the present invention is not limited to this, and for example, every other pixel, 2
Pixels may be selected at predetermined intervals within a range in which adjacent local area images overlap by a predetermined amount, such as every pixel.

[0083]

As described above, according to the image processing apparatus and the image processing method of the present invention, it is possible to convert a low quality image into a high quality image in a relatively short time.

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an outline of image processing.

FIG. 3 is an explanatory diagram showing an outline of affine transformation.

FIG. 4 is a flowchart showing an image processing method.

FIG. 5 is an explanatory diagram showing a method of calculating edge strength.

FIG. 6 is an explanatory diagram showing how the size of a search region changes according to the level of edge strength.

7 is a flowchart showing a flow of a specific process of fractal interpolation shown in FIG.

FIG. 8 is an explanatory diagram of an image processing method according to a second embodiment of the present invention.

FIG. 9 is a flowchart of an image processing method according to a third embodiment of the present invention.

[Explanation of symbols]

1 Image processing device 2 Original image storage 3 Local area image acquisition unit 4 Parameter setting section 5 pixel pointer 6 Affine transformation image acquisition unit 7 Affine transformation parameter setting section 8 Reduction section 9 Similarity calculation section 10 Replacement part 11 adder 12 Averaging processor 13 Output image holding unit 14 Edge strength determination unit 15 Parameter management table

Claims (13)

(57) [Claims] 1. An original image holding unit for holding an original image, and an image block for acquiring an image block of a predetermined size from the held original image such that adjacent image blocks overlap each other by a predetermined amount. A block acquisition unit, a similarity determination unit that determines the degree of similarity for each of the acquired image blocks, and a search area is set for each of the image blocks based on the determined degree of similarity. Then, a similar image detecting unit that detects a similar image block that is similar to each of the image blocks from the search area, and an image replacement unit that replaces each of the detected similar image blocks with the corresponding image block. An image adding means for superimposing the replaced similar image blocks, and an overlap of the blocks added by the image adding means An image processing apparatus comprising: an image adjusting unit that adjusts an image of a portion. 2. The similar image detecting means detects the similar image blocks only from among the image blocks, for image blocks whose degree of similarity determined by the similarity determining means exceeds a predetermined reference value. The image adding means includes a similar image block replaced by the image replacing means and an image block determined by the similarity determining means as the degree of similarity being equal to or less than the predetermined reference value. The image processing device according to claim 1, wherein the image processing devices are overlapped. 3. The image processing apparatus according to claim 1, wherein the similar image detecting means sets the search area smaller as the degree of similarity becomes higher. 4. The image processing apparatus according to claim 1, wherein an edge strength of the image block is used as the degree of similarity. 5. The image processing apparatus according to claim 1, wherein the image adjusting unit averages the images of the overlapping portion. 6. The image block acquisition means acquires, for each pixel of the original image, an image block of a predetermined size with the pixel substantially at the center.
The image processing device according to any one of 1. 7. The similar image detecting means detects the similar image block with a size larger than the size of the image block, and reduces the size of the similar image block to be equal to the size of the image block. The image processing apparatus according to any one of claims 1 to 6. 8. The similar image detecting means thins out pixels from a predetermined area set in the original image with a size larger than the size of the image block to obtain a pixel having the same size as the image block. The image processing apparatus according to any one of claims 1 to 6, which detects a similar image block. 9. The image processing apparatus according to claim 1, wherein the search area is set smaller than the original image. 10. The similar image detecting means determines the similarity of a plurality of candidate image blocks in the search area while changing the image operation parameter, and obtains a candidate image block most similar to the image block. The image processing apparatus according to claim 1, wherein only the used parameter is held, and the similar image block is detected by the held parameter. 11. An original image holding means for holding a color original image, a color system component related to image brightness among the color system components of the original image and other color system components. And an image block that acquires an image block of a predetermined size from the original image of the specified color system component, by causing each adjacent image block to overlap by a predetermined amount. Acquiring means, for each of the acquired image blocks, similarity determination means for determining the degree of similarity, and for each of the image blocks, a search area is set based on the determined degree of similarity. , A similar image detecting means for detecting a similar image block similar to each image block from the search area, and the detected similar image block to each of the corresponding image blocks. Image replacing means for replacing each of the replaced similar image blocks, image adding means for superimposing the replaced similar image blocks, image adjusting means for adjusting the image of the overlapping portion of the blocks added by the image adding means, and the designating means An image processing apparatus comprising: a synthesizing unit that synthesizes an original image other than the colorimetric component designated by the above and the image adjusted by the adjusting unit. 12. An image block is sequentially acquired from an original image such that adjacent image blocks overlap each other by a predetermined amount, and the degree of similarity is determined for each of the sequentially acquired image blocks, For each of the image blocks, a search area is set based on the determined degree of similarity, and a similar image block similar to each of the image blocks is detected from within the search area, and each detected similar image is detected. A block is replaced with each corresponding image block, the replaced similar image blocks are overlapped, and an image of an overlapping portion of the added block is adjusted and output. . 13. A recording medium having an image processing program for causing a computer to execute image processing, wherein an image block of a predetermined size is held from an original image held so that adjacent image blocks overlap each other by a predetermined amount. , A function of obtaining each, a function of determining the degree of similarity of each of the acquired image blocks, a function of setting a search area based on the determined degree of similarity of each of the image blocks A function of detecting a similar image block similar to each of the image blocks in the search area, a function of replacing each of the detected similar image blocks with a corresponding one of the image blocks, and the replaced A function of superimposing similar image blocks, and an overlapping part of the added blocks Recording medium of the computer program the computer, characterized by recording in the read and understandable form for realizing a function for adjusting an image, to the computer.