JPH0654172A - Image enlarging method - Google Patents

Abstract

PURPOSE:To obtain the enlarged image with a high picture quality by using the constraint condition that the information of a passage frequency area is correct and that the spread of an image is limited in the process repeating normal/inverse transforming by orthogonal transformation, thereby restoring a spatial high-frequency component lost at the time of sampling. CONSTITUTION:This method sets the repeated times of calculation and an enlargement rate (m), reads an original image A and adds the periphery of the original image for limiting the spread of the image with the image of a gray degree 1 and (n)-times so as to enlarge the original image to an image B. After storing a frequency component (a) obtained by two-dimensional discrete cosine-transforming(DCT) the image B and enlarging a frequency area in accordance with the enlargement rate (m), the method executes inverse DCT it to enlarge to the portion alpha for the picture elements of (m) times. Furthermore, the method updates a present repeating times, corrects the area of a mark X outside of the picture element portion alpha to the gray degree 1, and furthermore, executes DCT it to obtain a frequency component (b). The method furthermore executed inverse-DCT the DCT sequence of the components (a) and (b) obtained by replacing the component (b) to the already-known component (a) so as to obtain an enlarged image beta. Thus, the enlarged image of a high picture quality is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image enlarging method, and more particularly to an image enlarging method capable of obtaining a clear high-quality enlarged image.

[0002]

2. Description of the Related Art Various high-quality image processing functions are required in the fields of image databases and high-definition color printing, and one of them is image enlargement. This image enlargement is
To connect media with different resolutions (for example, HDTV (high-definition television), NTSC television, electronic still camera, medical image system, printing image system, etc.) in addition to its importance as one function of the image processing system. This is an extremely important function that can also be used as the resolution conversion required for.

As a conventional image enlarging method, an interpolation method of simply interpolating pixels has been adopted. A typical interpolation method is nearest neighbor.
r), bilinear, and cubic convolution are known. These interpolation methods are methods based on the interpolation of the sinc function {sinc (x) = sin (x) / x} based on the sampling theorem, and in order to reduce the computational load, sinc
The interpolation function that approximates the function is convoluted with the sample points of the original image to interpolate between the sample points of the original image and increase the number of pixels.

The nearest neighbor is a method in which a rectangular function is used as the interpolation function and the value of the closest sample is used as the interpolation value. The bilinear uses the triangle function as the interpolation function, and in the case of one-dimensional, the two neighboring values are used. This is a method in which a value linearly interpolated from a point is used as an interpolation value, and cubic convolution uses a three-dimensional function as an interpolation function, and in the case of one-dimensional, values interpolated from four neighboring points are used as interpolation values. Is the way.

The idea of enlargement by each of the above-mentioned interpolation methods is that the ideal original image before the original image to be enlarged is observed (sampled by scanning) has a frequency (low frequency component) of half or less of the Nyquist frequency. Correct if it consists of only. However, in general, an ideal original image has infinitely high frequency components, but a low pass filter (LPF) is used to prevent aliasing (a phenomenon such as moire or beat) from occurring in the sampled observed image. ) Is applied to remove unnecessary high-frequency components, so that the original image to be magnified has already lost the spatial high-frequency components that are involved in the definition of image sharpness and detail at the time of sampling.

As described above, the high frequency component removed at the time of observation (sampling) is unnecessary for the original image, but is an essential element for creating a high-definition enlarged image.

[0007]

However, the above-mentioned interpolation method cannot restore the spatial high-frequency component lost at the time of sampling, so that it is originally necessary when the original image is enlarged by interpolation. The spatial high frequency component is lacking. Therefore, nearest neighbors tend to cause distortion due to high frequency leaks, and the distortions appear as mosaic or jaggies at the edges. In the case of bilinear, the frequency characteristics of the pass band are suppressed, so that LPF-like characteristics are suppressed. The resulting image is smoothed, and in the case of cubic convolution, it may look sharper than the above two methods because of the frequency characteristic that emphasizes the high range.
The noise component will also be emphasized.

Therefore, when the original image is enlarged by the conventional interpolation method, the image quality is deteriorated due to blurring, smoothing, or edge rattling as described above, or an image in which details are not sufficiently expressed is brought about. There is a problem. That is, since the operation of interpolation only increases the amount of data and not the amount of information, an enlarged image by interpolation has the same content as the image before enlargement even if its size is enlarged. It does not mean that it is not possible to identify the part that was not visible.

The present invention has been made in order to solve the above-mentioned conventional problems, and restores the spatial high frequency component lost at the time of sampling, estimates the detail information and the edge information of the image, and restores them. An object of the present invention is to provide an image enlarging method that can improve the image quality of an enlarged image.

[0010]

According to the present invention, in an image enlarging method for enlarging an image based on image information contained in a sampled original image, in the process of repeating the forward transform and the inverse transform by orthogonal transform of the image. , The information on the pass frequency range is correct, and the spread of the image is limited.
The above object is achieved by restoring the spatial high frequency component lost at the time of sampling using one constraint condition.

The present invention also achieves the same object by using a discrete cosine transform as an orthogonal transform in the image enlarging method.

[0012]

First, the principle of the present invention will be described. In the present invention, an operation of restoring an original signal lost as a result of frequency band limitation when sampling the original image is performed. Such an operation is a super-resolution problem or bandwidth extrapolation. Called the problem.

For any observation system that can be physically realized,
High frequency components above a certain frequency cannot be observed. For example, since the size of the entrance aperture is limited in the image pickup system, most of the frequency components that can be propagated by the action of the image pickup system itself acting as an LPF are lost, and the resolution is lowered. Since this resolving power depends on the transfer bandwidth of the image pickup system--which varies depending on the aperture size, lens, etc .-- improving the resolving power is a band in which an original signal before passing through the image pickup system is obtained from an image signal obtained through the image pickup system. It is essentially possible only by extension of (super-resolution problem).

Now, if the super-resolution problem is mathematically formulated for a one-variable function, it becomes as follows. The original signal in the real space domain is f (x), and the frequency component of this original signal f (x) is band-limited to the cutoff frequency u ₀ or less, that is, the signal that actually passed through the imaging system is g (x). When the process of band limitation is represented by A, the following equation (1) is obtained. This process A is substantially LP by passing the original signal through the imaging system.
It is equivalent to applying F.

G (x) = Af (x) (1)

The Fourier transform of both signals f (x) and g (x) is represented by the corresponding capital letter F (u).
), G (u), and the window function W (u) in the frequency domain is defined by the following equations (2) and (3). Applying this window function W (u) is equivalent to applying an ideal LPF. When the above equation (1) is expressed in the frequency domain, the following equation (4) is obtained.

[0017] W (u) = 1 (| u | ≦ u 0) ... (2) W (u) = 0 (| u |> u 0) ... (3) G (u) = W (u) F ( u) (4)

The super-resolution means to obtain the original signal f (x) from the band-limited signal g (x) in the above equation (1) in the real space domain. Considering this in the frequency domain, To obtain F (u) from G (u) in the equation (4).
However, if there is no restriction on the original signal f (x), then the remaining part cannot be known from G (u) which is a part of F (u). Therefore, the object has a limited size with respect to the original signal f (x), and f (x) exists only within a certain area, for example, between −x ₀ to + x ₀ , and It is possible to solve the super-resolution problem by applying the assumption that, in principle, an unlimited resolution can be obtained when a spatial region constraint that is 0 outside is applied.

In the present invention, a method for solving the above super-resolution problem,
That is, as a band extension method, Gerchberg-Papolis (Ger
Chberg-Papoulis) iteration method is adopted. Hereinafter, the iterative method of Gerchberg-Papolis will be described with reference to FIG. In the following description, the Gerchberg-Papolis iterative method may be abbreviated as GP iteration method.

On the left side of FIG. 1, (A), (C), (E),
(G) is in the frequency domain, and (B), (D) on the right side,
(F) and (H) correspond to the real space region, (B) of the figure shows the original signal f (x), and the region is limited to the space | x | ≤ x ₀ (the object is constant. Corresponding to being limited in size). FIG. 1A shows the original signal f (x
) Fourier transform F (u) of F), and this F (u) includes infinitely high frequency components because the original signal f (x) is region-limited.

FIG. 1C shows the section | u | of the above F (u).
This means that only the part G (u) of ≤ u ₀ is observed. That is, the equation (4) using the window function such as the equations (2) and (3) is established. The inverse Fourier transform of G (u) is g (x) in FIG. 1 (D). And to solve the super-resolution problem, the above G (u) or
This is equivalent to obtaining F (u) or f (x) from g (x).

The first step of the GP iteration method is as follows. Since G (u) is band-limited to | u | ≦ u ₀ , g (x) will infinitely spread. However, the original signal f (x) is the interval | x | since found to be areas restricted to ≦ x _0, performing the same area limits also for g (x). That is, only the part of the section | x | ≦ x ₀ of g (x) is extracted and set as f ₁ (x). When this f ₁ (x) is expressed by an expression using the window function w (x) in the spatial domain expressed by the following expressions (5) and (6), the following expression (7) is obtained. This is f ₁ (x) shown in FIG.

[0023] w (x) = 1 (| x | ≦ x 0) ... (5) w (x) = 0 (| x |> x 0) ... (6) f 1 (x) = w (x) g (X) (7)

Fourier transform of the above f ₁ (x) results in F 1 (u) of FIG. 1 (E). Since f ₁ (x) is area limited, F 1 (u) is infinite. However, for the section | u | ≦ u ₀ , the correct value G (u) =
Since F (u) is already known, | in F1 (u)
Replace the part of u | ≦ u ₀ with G (u). The waveform thus formed is G1 (u) in FIG. 1 (G). This relationship is expressed by the following equations (8) to (10). The inverse Fourier transform of G1 (u) is shown in FIG.
It is g ₁ (x) of (H).

[0025] G1 (u) = G (u ) + (1-W (u)) F1 (u) ... (8) G1 (u) = G (u) (| u | ≦ u 0) ... (9) G1 (u) = F1 (u ) (| u |> u 0) ... (10)

In the above description, (C), (D) to (G), (H) in FIG. 1 is the first step of the GP iteration method.
Then, the interval | x | ≦ x ₀ from g ₁ (x) in FIG.
1) is taken out to obtain f ₂ (x) (not shown) corresponding to f ₁ (x) in FIG. 1 (F), and this f ₂ (x) is Fourier transformed to correspond to FIG. It is possible to completely restore the original signal by repeating the same operation of calculating F2 (u) (not shown) that is performed infinitely many times.

In the present invention, as described in detail above, the super-resolution method is used as a basic principle for restoring information lost at the time of sampling (expanding spatial high frequency components), and among them, two constraint conditions ( Applying the Gerchberg-Papolis iterative method, which asymptotically restores information by iterative calculation, as the basic principle of frequency band extension By doing so, the idea that has not been considered at all in the past, that is, the lost information (spatial high frequency component lost at the time of sampling) is restored, and the detail information of the image,
By estimating and restoring the edge information, it is possible to improve the image quality of the enlarged image.

In the present invention, when the discrete cosine transform (DCT) is used as the orthogonal transform adopted in the Gerchberg-Papolis iterative method, the computational load is reduced and the high speed algorithm is used. Can be applied so that
High-speed calculation can be performed.

[0029]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 2 is an explanatory view schematically showing the flow of processing executed in the image enlarging method according to the first embodiment of the present invention, and FIG. 3 shows the flow of processing shown in FIG. It is a flowchart.

In this embodiment, a monochrome image is enlarged by applying DCT to the G.P iterative method, and the original image is stored in the original image storage unit 10 using the apparatus having the basic structure shown in FIG. The image is read into the enlargement processing unit 12, the enlargement processing unit 12 performs the processing described in detail below to create an enlarged image, and the enlarged image is stored in the enlarged image storage unit 14.

The original image storage unit 10 may be, for example, a magnetic disk, the enlargement processing unit 12 may be an engineering workstation (EWS), and the enlargement image storage unit 14 may be a magnetic disk. The enlarged image created by the enlargement processing unit 12 may be output to an output device such as a display or a printer.

The series of processes shown in FIGS. 2 and 3 are executed by the enlargement processing section 12. Hereinafter, the process of this processing will be described in detail. In FIG. 2, the left side represents the image area and the right side represents the DCT area.

Now, assume that an original image of N × N pixels as shown in FIG. 2A is enlarged m times to form an image of m N × m N pixels. The numbers in parentheses in Fig. 2 are
It corresponds to the step number in the flowchart of FIG.

First, in step 1, the number of times the calculation is iterated by the G.P iterative method and the enlargement ratio are set, and then the original image to be enlarged shown in FIG. 2A is read (step 2). Then, to limit the spatial extent of the image, a gray level (gray l
evel) with l (lowercase letter) added,
It is expanded to the image of n N × n N pixels shown in (B) (step 3). Here, as the gray level l, any value may be used as long as it is a real number, and when l is set to 0, the load of the DCT calculation can be reduced. Further, n is a real number larger than 1 and may be any value so that n N is an integer. Furthermore, n should be set so that nmN becomes a power of 2 (power).
When is set, the high-speed calculation algorithm of DCT can be used.

The image shown in FIG. 2B is converted into the frequency component a shown in FIG. 2C by the two-dimensional DCT (step 4). This frequency component a is known information in the DCT domain and corresponds to a spatial low frequency component. Since this frequency component a is used during the iterative calculation, it is stored in the memory (step 5).

Next, regarding the frequency component a, as shown in FIG.
As shown in (D), the frequency region is expanded to a high frequency band according to the expansion rate (step 6). At this time, the initial value of the high frequency component is set to 0 so that the expanded size becomes nmN × nmN pixels. As shown in FIG. 2D, the frequency-extended DCT sequence is subjected to two-dimensional inverse DCT (IDCT) and returned to the image area (step 7). The image obtained at this time is an image of nmN × nmN pixels, and the α of m N × m N pixels at the center thereof is an enlarged image.

Next, the current number of iterations is updated (step 8), and the X mark extended outside the m N × m N pixel α of the image center portion of FIG. 2 (E) obtained in step 7 above. Although the level of the area marked with is unknown by the IDCT, it is already known that the level is l in this area from the image of FIG. 2 (B). It is corrected to l and brought into the state shown in FIG. This operation is the spatial region limitation.

Further, the extended area is modified as shown in FIG.
By performing DCT on the above image, the frequency component b shown in FIG. 7G is obtained (step 10). In this frequency component b, it is known that the low frequency side is the frequency component a shown in FIG. 2C as known information on the low frequency side. Therefore, the low frequency component is replaced with the frequency component a and the frequency component a in FIG. ) To the DCT sequence (step 11), the frequency components
The DCT sequence consisting of a and b is further IDCT'ed to obtain the image of FIG. 9 (I) (step 12). The central portion β of the image of FIG. 2 (I) thus obtained is a magnified image, and this image has a higher resolution than the magnified image α of FIG. 2 (E).

Thereafter, it is determined whether or not the current number of iterations has reached the set number of iterations. If no, the process returns to step 8 to repeat the above steps 8 to 12, and if yes. Output the enlarged image in step 14,
Writing to the storage unit, displaying on the display, etc. are performed, and all operations are completed.

As described above in detail, according to the present embodiment, in the process of repeating the forward transform and the inverse transform of the image by the orthogonal transform, the information of the pass frequency band is correct and the spread of the image is limited. By applying the GP iterative method that asymptotically restores information by iterative calculation using the two constraint conditions as follows,
Since the lost spatial high frequency component can be restored, a high-quality magnified image can be created.

Further, in the usual GP iterative method, Fourier transform is used. However, considering that the image actually processed in this embodiment is represented by a non-negative integer, the Fourier transform is performed. By using the discrete cosine transform (DCT) instead of, it is possible to reduce the load in terms of storage capacity and calculation amount.

An appropriate number of times is set as the number of iterations of the above-mentioned GP iteration method, but 10 times or less is usually sufficient.

Next, the two-dimensional DCT adopted in this embodiment.
And IDCT, which is the inverse transformation thereof, will be described. Discrete function i (x, y), 0 ≦ u, v ≦ N−1 N × N points of 2
The dimension DCT is defined by the following equations (11) and (12).

I (u, v) = DCT {i (x, y)} 0≤u, v≤N-1 (11)

[0046]

[Equation 1]

Here, 0 ≦ u and v ≦ N−1.

Further, c (u) is the following equation (13), (14)
It is defined by a formula. Note that c (v) is also defined in the same way as c (u) above. These functions c (u) and c (v) are also used in the inverse transformation.

C (u) = 1 / √2 (u = 0) (13) c (u) = 1 (u = 1, 2, ..., N-1) (14)

The inverse transform IDCT of the two-dimensional DCT is defined by the following equations (15) and (16).

I (x, y) = IDCT {I (u, v)} 0 ≦ x, y ≦ N−1 (15)

[0052]

[Equation 2]

Here, 0≤x and y≤N-1.

In the present embodiment, the DCT can be used as defined above, but as described above, when the number of pixels nmN shown in FIG. Since it exists, it is preferable to actually use the DCT of this high-speed operation algorithm in the operation.

Table 1 shows the error of the enlarged image when the original image is enlarged 2 times or 4 times by the image enlarging method of this embodiment. In the table, ebi corresponds to the case of using the natural image and res corresponds to the case of using the resolution chart as the original image. The enlargement error (percent error) corresponds to what percentage of the error is with respect to the average value of the original signal.
In addition, for comparison, the results of the above-described conventional interpolation methods of nearest neighbor (NN), bilinear (BL), and cubic convolution (CB) are also shown.

[0056]

[Table 1]

Table 2 shows the degree of error reduction of the enlargement method according to the present embodiment when the smallest error among the conventional methods is 100% at each enlargement ratio.

[0058]

[Table 2]

As is clear from Tables 1 and 2, according to this embodiment, the spatial high frequency component can be restored as compared with the image enlargement by the conventional interpolation method.
It is possible to create an extremely high-precision magnified image. The magnified image actually created (1) improves the sharpness,
(2) Edges without jaggies are reproduced, (3) Texture reproducibility is good, (4) Highlight dropout is good, and (5) High magnification ratio is strong.

FIG. 5 is a block diagram showing an outline of a device configuration applied to the second embodiment according to the present invention.

The present embodiment is an example of enlarging a color image. In this case, R (red), G from the original image storage unit 10 is used.
With respect to each of the signals of (green) and B (blue), the magnified image can be created by executing the same G · P iterative method in the magnifying processing unit 12 as in the case of the first embodiment. It is a thing.

According to the present embodiment, the color image can be treated as the same as the first embodiment by regarding each color component as a monochrome image. It becomes possible to create an enlarged image.

FIG. 6 is a block diagram showing the basic construction of an apparatus applied to the third embodiment according to the present invention.

The present embodiment is an example of enlarging a color image for printing, in which C (cyan), M from the original image storage unit 10 is used.
The magnified image is obtained by executing the same G · P iterative method in the enlarging processing unit 12 on the original signals of respective colors (magenta), Y (yellow), and K (black), as in the first embodiment. Is created.

According to the present embodiment, each color component can be processed in the same manner as in the first embodiment as in the second embodiment, so that an enlarged color image for printing with extremely high image quality can be similarly obtained. Can be created.

The present invention has been specifically described above, but the present invention is not limited to the above-mentioned embodiments, and various modifications can be made without departing from the scope of the invention.

For example, only the case where DCT is used as the orthogonal transformation adopted in the G · P iterative method has been described, but the present invention is not limited to this, and energy matching can be performed between the enlarged image and the original image. Other orthogonal transforms such as Fourier transform can also be adopted as long as the orthogonal transform can be taken.

Further, the case where the color spaces are RGB and CMYK has been described, but the present invention is not limited to a specific color space, and the present invention can be applied to any color space.

[0069]

As described above, according to the present invention, the spatial high frequency component lost when the original image is sampled can be restored, so that an enlarged image of extremely high quality can be created. .

Therefore, when a certain image original is used as a database, it is possible to easily create an enlarged image having the same image quality even if the size of the registered image is different from the size used. Therefore, the function of the image database system can be expanded.

Also, for example, when printing a television image, one side is 4 cm as it is because the image size is different.
Although only a print image of about the size can be obtained, according to the present invention, it is possible to enlarge the image with high accuracy, so that it is possible to easily enlarge to a practical print image size of, for example, 20 cm × 20 cm. . As a result, it becomes possible to integrate a plurality of media such as a television and a printer. Therefore, the present invention can be used as a resolution conversion technique for multimedia integration.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing the basic principle of the present invention.

FIG. 2 is an explanatory view schematically showing the process steps of the first embodiment according to the present invention.

FIG. 3 is a flowchart showing the flow of processing of the above embodiment.

FIG. 4 is a block diagram showing a basic configuration of an apparatus applied to the above embodiment.

FIG. 5 is a block diagram showing a basic configuration of an apparatus applied to a second embodiment according to the present invention.

FIG. 6 is a block diagram showing the basic configuration of an apparatus applied to a third embodiment according to the present invention.

[Short description of reference numeral] 10 ... Original image storage unit 12 ... Enlargement processing unit 14 ... Enlarged image storage unit

Claims (2)

[Claims] 1. An image enlarging method for enlarging an image on the basis of image information contained in a sampled original image, wherein in the process of repeating normal transformation and inverse transformation of the image by orthogonal transformation, information in the passing frequency range is correct. An image enlarging method is characterized in that the spatial high frequency component lost at the time of sampling is restored by using two constraint conditions that the extent of the image is limited. 2. The image enlarging method according to claim 1, wherein a discrete cosine transform is used as the orthogonal transform.