JPH0846778A - Image magnification method - Google Patents

Abstract

PURPOSE:To obtain a magnified image whose sharpness is improved by using a reference image without occurrence of ringing so as to conduct processing to suppress ringing with respect to a rectangular region around a noted picture element. CONSTITUTION:An original image from an original image storage section 10 is read by a magnification processing section 12, in which the image is processed and a magnified image is generated and it is stored in a magnified image storage section 14. In this case, a reference image without occurrence of ringing in a flat portion is prepared with respect to a processing object image and the processing object image is divided into plural regions to obtain a flat region, and the reference image is used to process the a rectangular region around a noted picture element in the flat region thereby suppressing ringing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image enlarging method, and more particularly to an image enlarging method capable of obtaining a clear high-quality enlarged image.

[0002]

2. Description of the Related Art Various high-quality image processing functions are required in the fields of image processing systems, image database systems, high-definition color printing, etc., and one of them is image enlargement. This image enlargement is important not only as one function of the image processing system but also for media with different resolutions (for example, HDTV (high resolution television), NTSC television, electronic still camera, medical image system, printing image system). Etc.) is an extremely important function that can also be used as the resolution conversion necessary for connecting.

As a conventional image enlarging method, an interpolation method of simply interpolating pixels has been adopted. A typical interpolation method is nearest neighbor.
r), bilinear, and cubic convolution are known. These interpolation methods are methods based on the interpolation of the sinc function {sinc (x) = sin (x) / x} based on the sampling theorem, and in order to reduce the computational load, sinc
The interpolation function that approximates the function is convoluted with the sample points of the original image to interpolate between the sample points of the original image and increase the number of pixels.

The nearest neighbor is a method in which a rectangular function is used as the interpolation function and the value of the closest sample is used as the interpolation value. The bilinear uses the triangle function as the interpolation function, and in the case of one-dimensional, the two neighboring values are used. This is a method in which a value linearly interpolated from a point is used as an interpolation value, and cubic convolution uses a cubic function as an interpolation function, and in the case of one-dimensional, values interpolated from four neighboring points are used as interpolation values. Is the way.

The idea of enlargement by each of the above-mentioned interpolation methods is that the ideal original image before the original image to be enlarged is observed (sampled by scanning) has a frequency (low frequency component) of half or less of the Nyquist frequency. Correct if it consists of only. However, in general, an ideal original image has infinitely high frequency components, but a low pass filter (LPF) is used to prevent aliasing (a phenomenon such as moire or beat) from occurring in the sampled observed image. ) Is applied to remove unnecessary high-frequency components, so that the original image to be magnified has already lost the spatial high-frequency components that are involved in the definition of image sharpness and detail at the time of sampling.

As described above, the high frequency component removed at the time of observation (sampling) is unnecessary for the original image that is not enlarged, but is an essential element for creating a high-definition enlarged image.

[0007]

However, since the interpolation method cannot restore the spatial high frequency component lost at the time of sampling, it is originally necessary when the original image is enlarged by the interpolation method. This means that a certain spatial high frequency component is missing.

Therefore, when the original image is enlarged by the conventional interpolation method, the spatial high frequency component is absent, so that the image quality is deteriorated due to blurring, smoothing, or edge rattling, and details are reduced. There was a problem that it resulted in a poorly expressed image.

On the other hand, the inventor of the present invention disclosed in Japanese Patent Laid-Open No. 6-54
172, the information on the pass frequency range is correct, and
We have proposed an image enlargement method that restores spatial high-frequency components lost at the time of sampling, using two constraint conditions that the image spread is limited, but the problem is that ringing still occurs in the flat part. was there.

The present invention has been made to solve the above-mentioned conventional problems, and provides an image enlarging method for improving the image quality of an enlarged image without causing blurring or jaggies and without image quality deterioration due to ringing. The purpose is to

[0011]

SUMMARY OF THE INVENTION The present invention provides an image enlarging method for enlarging an image based on image information contained in a sampled original image. Prepare, divide the image to be processed into a plurality of regions, obtain a flat part region, and for a rectangular region centered on the pixel of interest in the flat part region,
The object is achieved by performing processing using the reference image and suppressing ringing.

The present invention also provides the above image enlarging method,
The object is similarly achieved by creating the reference image by an interpolation method.

The present invention also provides the above image enlarging method,
In the process of repeating the forward transformation and the inverse transformation of the image to be processed by orthogonal transformation, the information of the pass frequency range is correct,
Further, the above-mentioned object is similarly achieved by restoring the spatial high frequency component lost at the time of sampling by using the two constraint conditions that the spread of the image is limited and enlarging the image.

[0014]

According to the present invention, when the image to be processed is enlarged, a reference image which does not cause ringing in the flat portion is prepared by an appropriate method in order to suppress ringing occurring in the flat portion. This is created using an interpolation method such as the bilinear method or the cubic method.

The reference image obtained by this interpolation method is
Due to the low-pass filter (LPF) effect of the interpolation method, the edge portion is dull and there is a problem in image quality. However, in the flat portion of this reference image, there is no ringing, and an image that is smoothly expressed is obtained. The present invention uses the flat portion of the reference image for the purpose of suppressing the occurrence of ringing in the flat portion of the processing target image.

In order to classify the processing target image into local areas such as a flat portion, a texture portion, and an edge portion, and compare and modify the processing target image and the reference image for each rectangular area centered on the pixel of interest. Determine the maximum allowable deviation of. At this time, the maximum allowable deviation is determined so as to severely limit the error with the reference image so that ringing does not occur in the flat portion. Further, the edge portion is loosely limited in error from the reference image. By performing processing to modify within this constant deviation, the energy of ringing appearing in the flat part is added to the energy for making the edge stand out,
Ringing is reduced in the flat part, and edges are raised in the edge part. Therefore, according to the present invention, since the processing is performed for each rectangular area, a sharp image can be obtained and more detailed ringing suppression processing can be performed.

Further, in particular, when the reference image is created by the bilinear method, the calculation is simple and the frequency characteristic of the pass frequency range is suppressed, so that there is an LPF-like property and a smooth image can be obtained.

Further, in the process of repeating the forward transform and the inverse transform by, for example, the orthogonal transform, the iterative calculation is performed by using two constraint conditions that the information of the pass frequency range is correct and the spread of the image is limited. Gerchberg-Papou, which asymptotically restores information by
lis) iterative method (hereinafter abbreviated as GP iterative method) is applied as a basic principle of frequency range expansion and the image to be processed is enlarged, the spatial high frequency component lost at the time of sampling is restored. Then, by estimating and restoring the detail information and edge information of the image, it is possible to further improve the image quality of the enlarged image.

The G.P iterative method will be described below with reference to FIG.

On the left side of FIG. 1, (A), (C), (E),
(G) is in the frequency domain, and (B), (D) on the right side,
(F) and (H) correspond to the real space region, (B) of the figure shows the original signal f (x), and the region is limited to the space | x | ≤ x ₀ (the object is constant. Corresponding to being limited in size). FIG. 1A shows the original signal f (x
) Fourier transform F (u) of F), and this F (u) includes infinitely high frequency components because the original signal f (x) is region-limited.

FIG. 1C shows the section | u | of the above F (u).
This means that only the part G (u) of ≤ u ₀ is observed. That is, the following expression (3) using the window function defined by the following expressions (1) and (2) is established.

[0022] W (u) = 1 (| u | ≦ u 0) ... (1) W (u) = 0 (| u |> u 0) ... (2) G (u) = W (u) F ( u) (3)

The inverse Fourier transform of G (u) is shown in FIG.
It is g (x) of (D). Where G (u) or
The problem is to find F (u) or f (x) from g (x).

The first step of the GP iteration method is as follows. Since G (u) is band-limited to | u | ≦ u ₀ , g (x) will infinitely spread. However, the original signal f (x) is the interval | x | since found to be areas restricted to ≦ x _0, performing the same area limits also for g (x). That is, only the part of the section | x | ≦ x ₀ of g (x) is extracted and set as f ₁ (x). When this f ₁ (x) is expressed by the expression using the window function w (x) in the spatial domain expressed by the following expressions (4) and (5), the following expression (6) is obtained. This is f ₁ (x) shown in FIG.

[0025] w (x) = 1 (| x | ≦ x 0) ... (4) w (x) = 0 (| x |> x 0) ... (5) f 1 (x) = w (x) g (X) (6)

Fourier transform of the above f ₁ (x) results in F ₁ (u) of FIG. 1 (E). Since f ₁ (x) is area limited, F ₁ (u) is infinite. However, for the section | u | ≦ u ₀ , the correct value G (u) =
Since F (u) is already known, | in F ₁ (u)
Replace the part of u | ≦ u ₀ with G (u). The waveform thus formed is G ₁ (u) in FIG. This relationship is expressed by the following equations (7) to (9). The inverse Fourier transform of G ₁ (u) is shown in FIG.
It is g ₁ (x) of (H).

G ₁ (u) = G (u) + (1-W (u)) F ₁ (u) (7) G ₁ (u) = G (u) (| u | ≦ u ₀ ) ... (8) G ₁ (u) = F ₁ (u) (| u |> u ₀ ) (9)

In the above description, (C), (D) to (G), (H) in FIG. 1 is the first step of the GP iteration method.
Then, the interval | x | ≦ x ₀ from g ₁ (x) in FIG.
1) is taken out to obtain f ₂ (x) (not shown) corresponding to f ₁ (x) in FIG. 1 (F), and this f ₂ (x) is Fourier transformed to correspond to FIG. It is possible to completely restore the original signal by repeating the same operation of calculating F2 (u) (not shown) that is performed infinitely many times.

[0029]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 2 is an explanatory view schematically showing the flow of processing executed in the image enlarging method according to the first embodiment of the present invention, and FIG. 3 shows the flow of processing shown in FIG. It is a flowchart.

In this embodiment, a monochrome image is enlarged by applying DCT to the G.P iterative method, and the original image is stored in the original image storage unit 10 using the apparatus having the basic structure shown in FIG. The image is read into the enlargement processing unit 12, the enlargement processing unit 12 performs the processing described in detail below to create an enlarged image, and the enlarged image is stored in the enlarged image storage unit 14.

The original image storage unit 10 may be, for example, a magnetic disk, the enlargement processing unit 12 may be an engineering workstation (EWS), and the enlargement image storage unit 14 may be a magnetic disk. The enlarged image created by the enlargement processing unit 12 may be output to an output device such as a display or a printer.

The series of processes shown in FIGS. 2 and 3 are executed by the enlargement processing section 12. Hereinafter, the process of this processing will be described in detail. In addition, in FIG.
The left side represents the image area and the right side represents the DCT area.

Now, assume that an original image of N × N pixels as shown in FIG. 2A is enlarged m times to form an image of m N × m N pixels. The numbers in parentheses in Fig. 2 are
It corresponds to the step number in the flowchart of FIG.

First, in step 1, the number of times the calculation is repeated by the GP iteration method and the enlargement ratio are set, and then step 2
2 sets thresholds t _a and t _b for classifying the region into each region R _i and a parameter β _i for setting the boundary value B of the deviation to an appropriate value, and in step 3, the above-mentioned FIG. The original image to be magnified shown in is read.

Next, at step 4, the data referred to when the ringing is corrected is created.

A flow chart showing the detailed processing of the reference data creation process is shown in FIG. Hereinafter, this process will be described with reference to FIG.

First, in step 21, an m-fold reference image is created by the bilinear method, and in next step 22, the average value of r × r pixels centered on the pixel of interest (p, q) for each rectangular area. Create an average value map for (bar of g). Here, r may be an odd number of 3 or more, but in this embodiment, r = 5. Also, p and q are 1 to mN, respectively.
Scanned up to.

Next, at step 23, the position of the pixel of interest (p
, Q) is initialized, and in step 24, the position of the pixel of interest (p
, Q) are updated. Next, at step 25, the deviation σ _g (p, q) of each rectangular region of r × r pixels centered on the target pixel (p, q) is calculated. The calculation of this deviation is performed by the following equation (10).

[0040]

[Equation 1] However, g (p, q) is a reference image and is a rectangular area of r × r pixels centered on the image of interest, and the average of g (p, q) is added to g by adding bar (￣). It represents.

[0041] In next step 26, sigma _g (p, q) using a threshold value t _a, the t _b, and area classification by equation (11) below, interest pixel (p, q) R1 (edge portion ), R
Which region of 2 (texture part) and R3 (flat part) it belongs to is determined.

[0042]

[Equation 2]

Next, at step 27, the boundary value B (p, q) of the deviation for each rectangular area is calculated. This calculation is performed by the following equation (12). As shown by the equation (12), the deviation boundary value B represents the maximum change in the density (pixel value) from the average value allowed for one rectangular area, and the average density ( Pixel value) (g ba
r) and the actual image density g being processed (variation) ‖
It is obtained by multiplying g − (bar of g) ‖ by a parameter β _i that adjusts the allowable range of concentration fluctuation.

[0044]

(Equation 3) Here, β _i is a parameter for setting an appropriate boundary value, and a different value is set according to the region R _i .

The norm is that when * is a complex conjugate,
It is defined by the following equation (13), and the integration range is the pixel of interest (p,
It is a rectangular region of r × r pixels centered on q).

[0046]

[Equation 4]

Here, β according to the determination result of the upper region
_{Using i} , the boundary value B (p, q) of the deviation is obtained for each rectangular area by the equation (12), and a map is created.

Finally, in step 28, it is judged whether or not the scanning of (p, q) has been completed, that is, whether the entire image has been processed. If No, the process returns to step 24, and if Yes, the process shown in FIG. The reference data creation process shown in FIG.

Returning to the flowchart of FIG. 3 again, in order to limit the spatial spread of the image in step 5,
A gray level around the original image of N × N pixels
Image with l (lowercase letter) added to l)
It is expanded to the image of n N × n N pixels shown in (B). Here, as the gray level l, any value may be used as long as it is a real number, and when l (el) is set to 0, DC is set.
The load of T calculation can be reduced. Further, n is a real number larger than 1 and may be any value so that n N is an integer.
Further, when n is set so that nmN becomes a power of 2, a high-speed calculation algorithm of DCT can be used.

The image shown in FIG. 2B is converted by the two-dimensional DCT into the frequency component a shown in FIG. 2C (step 6). This frequency component a is known information in the DCT domain and corresponds to a spatial low frequency component. Since this frequency component a is used during the iterative calculation, it is stored in the memory (step 7).

Next, regarding the frequency component a, as shown in FIG.
As shown in (D), the frequency region is expanded to the high frequency band according to the expansion rate (step 8). At this time, the initial value of the high frequency component is set to 0 so that the expanded size becomes nmN × nmN pixels. As shown in FIG. 2D, the frequency-extended DCT sequence is subjected to two-dimensional inverse DCT (IDCT) and returned to the image area (step 9). The image obtained at this time is an image of nmN × nmN pixels, and the α of m N × m N pixels at the center thereof is an enlarged image.

Next, the current number of iterations is updated (step 10), and the x mark extended outside the number m N × m N pixels α of the image center portion of FIG. 2 (E) obtained in step 9 above. Although the level of the area marked with is unknown by the IDCT, it is already known that the level is l (el) in the image of FIG. 2B, so the level is correct. The value is corrected to l (el), and the state shown in FIG. This operation is the spatial region limitation.

Further, the extended area is modified as shown in FIG.
By performing DCT on the above image, the frequency component b shown in FIG. 9 (G) is obtained (step 12). In this frequency component b, it is known that the low frequency side is the frequency component a shown in FIG. 2C as known information on the low frequency side. Therefore, the low frequency component is replaced with the frequency component a and the frequency component a in FIG. ) To the DCT sequence (step 13), frequency components
The DCT sequence consisting of a and b is further IDCT'ed to obtain the image of FIG. 9 (I) (step 14). The central portion β of the image of FIG. 2 (I) thus obtained is a magnified image, and this image has a higher resolution than the magnified image α of FIG. 2 (E).

Next, in step 15, step 4
Use the data obtained in step 1 to suppress ringing. A flowchart showing this detailed processing is shown in FIG.

In step 31 of FIG. 6, the target pixel position (p, q) is initialized, and in the next step 32, the target pixel position (p, q) is updated. Here, the scan ranges of p and q are 1 to mN, respectively, and are updated so that the entire region is scanned.

Next, in step 33, the image to be processed
Calculate the norm by h and the mean value data (bar of g) obtained from the reference image. The integration range for calculating the norm is a r × r pixel rectangular area centered on the pixel of interest. Also, the image to be processed has a region expanded to the outside,
Since the size is different from that of the reference image, in order to align the positions of the target pixels, the position shifted by (dx, dy) pixels with respect to h is focused. dx and dy are given by the following equation (14).

Dx = mN (n −1) / 2 dy = mN (n −1) / 2 (14)

Next, in step 34, the square of the norm calculated in step 33 and the square of B obtained in step 4 are compared by the following equation (15).

[0059]

(Equation 5)

When the equation (15) is satisfied, that is, the average value of the rectangular area h (p + dx, q + dy) of the image being processed (in the corresponding position) (g ba
r) (p, q) variation of pixel value ‖h − (g ba
If r) ‖ ² is less than or equal to the maximum value B (p, q) ² of the allowable fluctuation amount of the pixel value that has already been set (calculated), ringing does not occur, so proceed to step 36, and in other cases Advances to the next step 35.

In step 35, since the occurrence of ringing is recognized by the judgment conditions so far,
The image to be processed is corrected by the following equation (16).

[0062]

(Equation 6)

Here, in the equation (16), the change amount from the average of the pixels of the second term is added to the bar of the density average value g of the first term. If the amount of change in the image is added, ringing will still occur. Therefore, as shown in the equation (16), the ringing is suppressed because the amount obtained by linearly contracting (suppressing) the original change amount is added in the second term.

In step 36, it is determined whether or not all (p, q) have been scanned. If No, step 32
If Yes, the ringing correction process ends.

When the processing here is completed, γ in FIG.
Is an image with less ringing than β.

A schematic diagram of this ringing correction processing is shown in FIG. In FIG. 7, the left side is the image before correction (β in FIG. 2)
The right side is the corrected image (γ in FIG. 2) in which the ringing is corrected (the process in step 15 in FIG. 3) by using the data of the reference image. As shown in FIG. 7, by correcting the ringing, the ringing is reduced in the flat portion and the edge is raised in the edge portion. For this reason,
An even sharper image is obtained.

Next, in step 16 of FIG. 3, it is determined whether the current number of iterations has reached the set number of iterations. As a result of the determination, if No, the process returns to step 10, and if Yes, the process proceeds to step 17, the enlarged image is output, and all the processes are finished.

In the normal GP iterative method, the Fourier transform is used, but in this embodiment, in consideration of the fact that the image to be actually processed is represented by a non-negative integer, the Fourier transform is replaced by the Fourier transform. By using the discrete cosine transform (DCT) for the above, the load can be reduced in terms of storage capacity and calculation amount.

An appropriate number of times may be set as the number of iterations of the above-mentioned GP iterative method, but 10 times or less is usually sufficient.

Further, in the present embodiment, the case where the number of regions to be classified is 3 has been shown, but the classification may be made more finely. When the number of regions to be classified is i, i −1 kinds of thresholds (t ₁ , ..., T
_i-1 ), i type parameter β _i should be set.

Next, the two-dimensional DCT adopted in this embodiment.
And IDCT, which is the inverse transformation thereof, will be described. Discrete function i (x, y), 0 ≦ u, v ≦ N−1 N × N points of 2
The dimension DCT is defined by the following equations (17) and (18).

I (u, v) = DCT {i (x, y)} 0 ≦ u, v ≦ N−1 (17)

[0073]

(Equation 7) Here, 0 ≦ u and v ≦ N−1.

Further, c (u) is the following equation (19), (20)
It is defined by a formula. Note that c (v) is also defined in the same way as c (u) above. These functions c (u) and c (v) are also used in the inverse transformation.

C (u) = 1 / √2 (u = 0) (19) c (u) = 1 (u = 1, 2, ..., N-1) (20)

The inverse transform IDCT of the two-dimensional DCT is defined by the following equations (21) and (22).

I (x, y) = IDCT {I (u, v)} 0 ≦ x, y ≦ N−1 (21)

[0078]

(Equation 8) Here, 0 ≦ x and y ≦ N−1.

In this embodiment, the DCT can be used as defined above, but as described above, when the pixel number nmN shown in FIG. Since it exists, it is preferable to actually use the DCT of this high-speed operation algorithm in the operation.

FIG. 8 is a block diagram showing an outline of a device configuration applied to the second embodiment according to the present invention.

The present embodiment is an example of enlarging a color image. In this case, R (red), G from the original image storage unit 10 are used.
For each of the signals of (green) and B (blue), the enlarging processing unit 12 executes the G.P iterative method including the ringing suppressing process similar to that of the first embodiment to create an enlarged image. It was made possible.

According to this embodiment, the same processing as in the first embodiment can be performed for a color image by regarding each color component as a monochrome image. Therefore, the image quality is much higher than that of the conventional interpolation method. Moreover, it is possible to create an enlarged image with little ringing.

FIG. 9 is a block diagram showing the basic structure of an apparatus applied to the third embodiment according to the present invention.

The present embodiment is an example of enlarging a color image for printing, and the C (cyan) and M from the original image storage unit 10 are used.
The magnifying processing unit 12 executes the G.P iterative method including the ringing suppression processing similar to that of the first embodiment on the original signals of respective colors (magenta), Y (yellow) and K (black). By doing so, an enlarged image is created.

According to the present embodiment, each color component can be processed in the same manner as in the first embodiment, as in the second embodiment, and thus similarly for enlarged printing with extremely high image quality and less ringing. Color images can be created.

The present invention has been specifically described above, but the present invention is not limited to the above-mentioned embodiments, and various modifications can be made without departing from the scope of the invention.

For example, only the case where DCT is used as the orthogonal transformation adopted in the G · P iterative method has been described, but the present invention is not limited to this, and energy matching can be performed between the enlarged image and the original image. Other orthogonal transforms such as Fourier transform can also be adopted as long as the orthogonal transform can be taken.

Further, the case where the color spaces are RGB and CMYK has been described, but the present invention is not limited to a specific color space and can be applied to any color space.

Further, although the case where the image enlarging method is the GP iterative method has been described, the ringing suppressing process can also be applied to the image enlarging method using the digital filter.

[0090]

As described above, according to the present invention, the processing for suppressing the ringing is performed on the rectangular area centered on the target pixel by using the reference image in which the ringing does not occur. It is possible to finely correct the ringing, and it is possible to obtain a magnified image with no sharpness and improved sharpness.

In addition, when the reference image is created by the bilinear method, a smooth image with no ringing in the flat portion can be obtained by a simple calculation, so that the ringing of the image to be processed can be suppressed more efficiently. .

When the spatial high frequency component lost when the original image is sampled is reconstructed, edges without jaggies are reproduced, the reproducibility of the texture and the lack of highlights are high, and the high It is said that it is resistant to expansion
It has an excellent effect that an extremely high-quality magnified image having many properties can be created.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing the principle of the GP iteration method.

FIG. 2 is an explanatory view schematically showing the process steps of the first embodiment according to the present invention.

FIG. 3 is a flowchart showing the flow of processing of the above embodiment.

FIG. 4 is a block diagram showing a basic configuration of an apparatus applied to the above embodiment.

FIG. 5 is a flowchart showing a reference data creation process used in the above embodiment.

FIG. 6 is a flowchart showing detailed processing of ringing correction.

FIG. 7 is an explanatory diagram schematically showing a ringing correction process.

FIG. 8 is a block diagram showing a basic configuration of a device applied to a second embodiment according to the present invention.

FIG. 9 is a block diagram showing the basic configuration of an apparatus applied to the third embodiment according to the present invention.

[Brief description of reference numerals]

 10 ... Original image storage unit 12 ... Enlargement processing unit 14 ... Enlarged image storage unit

Claims (3)

[Claims] 1. An image enlarging method for enlarging an image based on image information included in a sampled original image, wherein a reference image having no ringing in a flat portion is prepared for the image to be processed, and the image to be processed is prepared. Is divided into a plurality of areas to obtain a flat portion area, and a rectangular area centered on a pixel of interest in the flat portion area is processed using the reference image to suppress ringing. Method. 2. The image enlarging method according to claim 1, wherein the reference image is created by an interpolation method. 3. The process target image according to claim 1 or 2, wherein the information of the pass frequency range is correct and the spread of the image is limited in the process of repeating the forward transform and the inverse transform by the orthogonal transform. The image enlarging method is characterized in that the spatial high frequency component lost at the time of sampling is restored by using the two constraint conditions that the image is enlarged, and the image is enlarged.