JPH11232446A - Pixel interpolation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the processing speed of a pixel interpolation device which can restore a high frequency component. SOLUTION: An interpolation device (image enlargement device) applying an IM-GPDCT method reads the images (S102) and extracts a block to be processed (S103). It's decided whether the extracted block includes an edge component (S104). If the edge component is included in the block, the image processing is carried out by the IM-GPDCT method that can reproduce a high frequency component (S105 to S116). If no edge component is included in the block, the image processing is carried out by a comparatively simple pixel interpolation method such as a cubic convolution method that reproduces no high frequency component (S117).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pixel interpolating apparatus, and more particularly to a pixel interpolating apparatus using an IM-GPDCT method for interpolating pixels while restoring high-frequency components of an image.

[0002]

2. Description of the Related Art Conventionally, in a case where an image is subjected to pixel density conversion (pixel interpolation) based on image information included in a sampled original image, a process of repeating normal and inverse transforms by orthogonal transform of an image is known. A method of restoring a spatial high-frequency component lost at the time of sampling by using two constraint conditions that information of a pass frequency band is correct and image spread is limited (so-called IM-GPDCT)
Law) is known.

[0003] The principle of this method will be described below. An operation of restoring an original signal lost as a result of limiting the frequency band at the time of sampling the original image is known, and such an operation is generally called a super-resolution problem.

[0004] In any observation system that can be physically realized, it is not possible to observe high frequency components above a certain frequency.

For example, in an imaging system, the size of the entrance aperture is limited, and the imaging system itself has a low-pass filter (LP).
F) Acts like a. As a result, most of the frequency components that can be propagated are lost, and the resolution is reduced.

[0006] The lost resolution can be obtained only by expanding the band (super-resolution problem) of obtaining an original signal before passing through the imaging system from an image signal obtained through the imaging system.

Now, if the super-resolution problem is mathematically formulated for a one-variable function, the following is obtained. An original signal in the real space area is defined as f (x), and a frequency component of the original signal f (x) is band-limited to a cutoff frequency u0 or less. When the process of performing the restriction is represented by A, the following equation (1) is derived.

G (x) = Af (x) (1) Step A corresponds to substantially applying an LPF by passing the original signal through an imaging system.

The functions corresponding to the Fourier transform of the signals f (x) and g (x) are F (u) and G (u), and the window function W (u) in the frequency domain is expressed by the following equation (2). And (3).

W (u) = 1 (| u | ≦ u0) (2) W (u) = 0 (| u |> u0) (3) Applying this window function W (u) Ideal LP
This is equivalent to applying F.

When the above equation (1) is expressed in the frequency domain, it is expressed as the following equation (4).

G (u) = W (u) F (u) (4) The super-resolution problem refers to a signal g (x) band-limited by the above equation (1) in the real space domain from an original signal f (X), and considering this in the frequency domain, G in the above equation (4)
That is, F (u) is obtained from (u).

However, if there is no restriction on the original signal f (x), F (u) cannot be obtained.

Therefore, the object has a limited size with respect to the original signal f (x), and f (x) exists only in a certain area, for example, between -x0 to + x0, and is outside this area. By applying a process in which an unlimited resolving power is obtained in principle when a spatial region is limited to 0, the super-resolution problem can be solved.

In the conventional method, as a method of solving the above-mentioned super-resolution problem, there is a method of Gerchberg-Papolis.
Papoulis) (GP iteration).

FIG. 5 is a diagram for explaining the principle of the GP iteration method. (A), (C),
(E), (G) are in the frequency domain, (B), (D),
(F) and (H) correspond to the real space area. FIG. 5B shows the original signal f (x), and the space | x | ≦ x0
Area is limited. FIG. 5A shows the original signal f.
This is the Fourier transform F (u) of (x), and this F (u) includes infinitely high frequency components because the original signal f (x) is region-limited.

FIG. 5C shows the space | u | of F (u).
This indicates that only G (u), which is a part of ≦ u0, is observed. That is, Expression (4) using a window function such as Expressions (2) and (3) holds.

FIG. 5 shows an inverse Fourier transform of G (u).
G (x) of (D). Solving the super-resolution problem is based on G (u) or g (x)
(U) or f (x).

The operation in the GP iteration method will be described below. First, since G (u) is band-limited to | u | ≦ u0, g (x) extends infinitely.

However, the original signal f (x) has a section | x | ≦ x
Since it is known that the area is limited to 0, g
The same area restriction is performed for (x).

In other words, only the section | x | ≦ x0 of g (x) is extracted and defined as f1 (x). This f1 (x) is converted to a window function w (x) expressed by the following equations (5) and (6).
Expression (7) can be expressed by the expression using. This is the function f1 (x) shown in FIG.

W (x) = 1 (| x | ≦ x0) (5) w (x) = 0 (| x |> x0) (6) f1 (x) = w (x) g (x) (7) If the above f1 (x) is subjected to Fourier transform, it becomes F1 (u) in FIG. 5E. Since f1 (x) is area-limited, F1 (u) extends infinitely. However, space | u |
Since the correct value G (u) = F (u) is already known for ≦ u0, the portion of | u | ≦ u0 of F1 (u) is replaced with G (u).

The waveform thus formed is G1 (u) in FIG. 5 (G). Expressing this relationship as an equation,
Expressions (8) to (10) are obtained. And G1 (x) in FIG. 5 (H) is the result of inverse Fourier transform of G1 (u).

G1 (u) = G (u) + (1−W (u)) F1 (u) (8) G1 (u) = G (u) (| u | ≦ u0) (9) G1 (U) = F1 (u) (| u |> u0) (10) The above-described processing from (C) and (D) to (G) and (H) in FIG. 5 is the first processing of the GP iterative method. Then, only the portion of the section | x | ≦ x0 is extracted from g1 (x) of (H) of FIG. 5 and f2 corresponding to f1 (x) of (F) of FIG.
An operation of Fourier-transforming (x) (not shown) to obtain F2 (u) (not shown) corresponding to (E) in FIG. 5 is repeatedly performed. As a result, the original signal can be completely restored.

In the prior art, the computational load is reduced by replacing the Fourier transform of the GP iteration method with a discrete cosine transform (DCT). This method is called the IM-GPDCT method.

FIG. 6 is a flowchart schematically showing the flow of processing performed in an image enlargement processing (an example of pixel interpolation processing) using the conventional IM-GPDCT method, and FIG. 7 is a flowchart of FIG. FIG. 4 is a diagram for describing the processing performed in FIG.

Here, it is assumed that an original image composed of N × N pixels shown in FIG. 7A is magnified m times to produce an image of (N × m) ² pixels. The numbers in parentheses in FIG. 7 correspond to the step numbers in the flowchart in FIG.

Referring to FIG. 6, in step S1, the number of repetitions in the GP repetition method and a value of an enlargement ratio (resolution conversion ratio) are set. In step S2, the original image to be enlarged shown in FIG. 7A is read. In step S3, an image of interest (here, the image shown in FIG. 7A) is extracted.

In step S4, an image (extended area) extending around the target image of N × N pixels is obtained. Here, in the related art, the data of the image to be extended is fixed to a specific value, and the process of calculating the data of the extended area is not performed. That is, in step S4, predetermined image data “L” is added to the original image as an extended area, and the image is extended to an image of nN × nN pixels shown in FIG. 7B. Here, n is a real number greater than 1 and n
n is set so that mN is a power of two.

In step S5, the image shown in FIG. 7B is converted into the frequency component a shown in FIG. 7C by two-dimensional DCT. This frequency component a is known information in the DCT domain, and corresponds to a spatially low frequency component.

In step S6, the value of a is stored. In step S7, the frequency band of the frequency component a is extended to a high frequency band corresponding to the magnification as shown in FIG. 7D.

At this time, the initial value 0 is set for the extended high frequency band.
Set. The extended frequency range is set to be nmN × nmN pixels.

In step S8, the frequency domain extended as shown in FIG. 7D is subjected to inverse DCT (IDCT) to convert it into an image area. At this time, the image size becomes nmN × nmN, and α of mN × mN pixels at the center thereof becomes an enlarged image.

In step S9, the number of repetitions is updated, and in step S10, FIG.
Although the value of the region marked with x outside the α of mN × mN pixels at the center of the region is unknown by IDCT, it is corrected to a predetermined value “L”. As a result, the state shown in FIG.

This operation is called spatial area limitation. DCT of the image of FIG. 7F in which the extended area has been corrected in step S11 results in the frequency component b shown in FIG.
Can be obtained.

In step S12, in the low frequency region of the frequency component b obtained in step S11, the known value a is replaced with the state shown in FIG. 7 (H).

In step S13, the area composed of the frequency components a and b is subjected to IDCT to obtain an image shown in FIG.
In step S14, it is determined whether the number of repetitions has exceeded the set value. If NO, the processes in steps S9 to S13 are repeated.

If "YES" in the step S14, the image enlarged in the step S15 is output, and all the operations are ended.

In the conventional technique described above, a method of converting an original image at once without dividing it is described. However, DCT conversion of a large-sized image requires an enormous amount of processing time and is not practical.

Therefore, a method has been proposed in which an original image is once divided into small-sized image blocks, and a resolution conversion process is performed on each block.

FIG. 8 is a diagram schematically showing block extraction of an original image and setting of an extended area in the conventional IM-GPCTC processing.

Referring to FIG. 8, an original image (# 601) is divided into an image of predetermined N × N pixels by a block division process (# 602). Here, the extracted block to be processed is referred to as a target block (# 60
3). All the original images are processed by setting all the blocks as target blocks. Here, an example is described in which a block near the center of the character of “image” in the original image, which is a character image, is cut out. doing.

An extension area of nN × nN pixels is added to the block of interest (# 604), and the subsequent resolution conversion processing is performed (# 605).

As described above, the data of the extended area is fixed to a specific value. However, in the conventional method, the data of the extended area is generally "0", "255", or the image data of the block of interest. The average value is set.

[0045]

However, the IM-GPCT method described above has a problem that the processing time is relatively long. Biline, on the other hand, has a relatively short processing time
When the ar method or the like is used, high-frequency components of an image are not restored.

The present invention has been made to solve the above-described problem, and has as its object to improve the processing speed of a pixel interpolation device capable of restoring high-frequency components.

[0047]

According to one aspect of the present invention, there is provided a pixel interpolating apparatus for interpolating a pixel with respect to an input image, wherein the input image has an edge portion. A determination unit that determines whether the input image is present; and a switching unit that switches a processing method of the input image based on a determination result of the determination unit.

More preferably, the switching means of the pixel interpolation device selects the first image processing method when an edge portion exists in the input image, and selects the second image processing method when the edge portion does not exist in the input image. In the first image processing method, the information in the pass frequency range is correct and the spread of the image is limited in the process of repeating the normal transform and the inverse transform by orthogonal transform of the image. Using two constraints,
This is a method for restoring high-frequency components lost at the time of sampling, and the second image processing method is an interpolation method that does not reproduce high-frequency components of image data.

More preferably, the pixel interpolating apparatus further includes block extracting means for extracting a block from the input image, and the determining means determines whether or not an edge portion exists in the extracted block.

According to the present invention, the processing method of the input image is switched based on the result of determining whether or not an edge portion exists, so that the processing speed of the apparatus can be improved.

[0051]

FIG. 1 is a block diagram of an image processing circuit according to a preferred embodiment of the present invention. This image processing circuit executes image enlargement processing by performing pixel interpolation.

Referring to the figure, an image processing circuit includes an input interface unit 401 for inputting image data to be processed.
And an input image memory unit 402 for temporarily storing input image data, and a resolution conversion unit 40 for converting the resolution.
3, an output image memory unit 405 for temporarily storing an output image, an output interface unit 406 for outputting image data, and a CPU 407 for controlling the entire image processing apparatus.

Image data input from an external device via the input interface unit 401 is temporarily stored in the input image memory unit 402. The stored image data is appropriately read out according to the processing process, and is processed by the resolution conversion unit 403. At this time, the buffer memory 404 stores part of the image data in order to buffer the image during the conversion process. After the completion of all the processes, the image subjected to the predetermined resolution conversion is stored in the output image memory unit 405 and output to a subsequent processing unit such as a printing device via the output interface unit 406.

FIG. 2 is a block diagram showing a specific configuration of the resolution converter 403 of FIG. Resolution conversion unit 403
Is a block extracting unit 4030 for extracting a block from image data stored in the input image memory unit 402, and an edge determining unit 4031 for determining whether an edge component is included in the extracted block. And an IM-GPDCT unit 4032 that performs pixel interpolation processing on the cut-out block image using the IM-GPDCT method, and a Cubic Convolution method (hereinafter, referred to as a CC method) on the cut-out block image. CC section 4 for performing pixel interpolation processing according to
033 and the output of the IM-GPDCT unit 4032 and the CC unit 4032 based on the determination result from the edge determination unit 4031.
A selector 40 for selecting a preferred output from the outputs of
34.

That is, the image processing apparatus according to the present embodiment, when an edge component exists in an image of a cut-out block, in the process of repeating normal and inverse transforms by orthogonal transform of the image, The information of the pass frequency range is correct,
In addition, the IM-GPCT method, which is a method of restoring the high-frequency component lost at the time of sampling by using the two constraint conditions that the spread of the image is limited, is employed. The CC method, which is an interpolation method that does not reproduce high-frequency components, is employed. This allows
In the pixel processing device according to the present embodiment, it is possible to restore a lost high-frequency component and to perform image processing at a higher speed.

In the present embodiment, Cubic Convolution is employed as an interpolation method that does not reproduce high-frequency components.
A processing unit that employs the estNeighbor method or the Bilinear method may be employed instead of the CC unit 4033.

FIG. 3 is a flowchart showing an image enlargement process performed by the image processing apparatus according to the present embodiment.

Referring to the figure, in step S101, IM-
The number of repetitions and the magnification in the GPDCT method are set. In step S102, an input image is read into the input image memory unit 402 via the input interface unit 401. In step S103, the block extracting unit 4030
Thus, a block is cut out from the image data stored in the input image memory unit 402.

It is determined whether or not an edge portion (edge component) exists in the image of the block extracted in step S104. If YES, the same processing (IM-GPD) as steps S4 to S15 in FIG.
Image processing using the CT method).

On the other hand, if NO in step S104,
In step S117, a method of not reproducing high frequency components (CC
The conversion of the resolution is performed by the method. Then, in step S116, the processed image is stored in the selector 4034.
Is stored in the output image memory unit 405 through

FIG. 4 is a diagram for explaining a resolution conversion method that does not reproduce high-frequency components performed in step S117 of FIG. In the figure, (A) Nearest Ne
(B) shows processing by Bilinear method, and (C) shows processing by CubicConvolution. In each figure, pixel positions x = -1, 0,
1, the value of the image data (density of the pixel) f (x) is determined, and between these pixels (for example, x = −0.
5, 0.5) is assumed.

Referring to (A), in the Nearest Neighbor method, the value of the pixel closest to the pixel to be interpolated (the point of interest) is directly used as the image data of the pixel to be interpolated.

Referring to (B), in the Bilinear method,
The value of a neighboring pixel is changed linearly according to the distance from the pixel to be interpolated, and the value on the straight line is adopted as image data of the pixel to be interpolated.

Referring to (C), in Cubic Convolution, the degree of reflection of image data of neighboring pixels is changed in a curve according to the distance from the pixel to be interpolated, and the data on the curve is used.

In the above embodiment, blocks are cut out and the image processing method is switched depending on the presence or absence of an edge component in the cut out blocks. Alternatively, it may be determined whether or not any part of the input image data has an edge component, and the processing method of the entire image may be switched according to the determination result.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a preferred embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a resolution conversion unit 403 in FIG.

FIG. 3 is a flowchart of a resolution conversion process according to the embodiment.

FIG. 4 is a diagram for describing a resolution conversion method that does not reproduce high-frequency components.

FIG. 5 is a diagram for explaining the principle of the IM-GPDCT method.

FIG. 6 is a flowchart showing a conventional resolution conversion process.

FIG. 7 is a schematic diagram of the IM-GPDCT method.

FIG. 8 is a diagram for explaining processing performed in a conventional IM-GPDCT method.

[Explanation of symbols]

 403 Resolution conversion unit 404 Buffer memory 4030 Block extraction unit 4031 Edge discrimination unit 4032 IM-GPDCT unit 4033 CC unit 4034 Selector

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Junji Nishigaki 2-3-1-13 Azuchicho, Chuo-ku, Osaka City Inside Osaka International Building Minolta Co., Ltd. (72) Kazuhiro Ishiguro 2-chome, Azuchicho, Chuo-ku, Osaka-shi No. 3-13 Osaka International Building Minolta Co., Ltd. (72) Inventor Takamoto Nabeshima 2-3-1 Azuchicho, Chuo-ku, Osaka City Osaka International Building Minolta Co., Ltd. (72) Inventor Takashi Yamauchi Osaka City Central Osaka International Building Minolta Co., Ltd. 2-3-13 Azuchicho, Ward

Claims (3)

[Claims] 1. A pixel interpolating device that performs pixel interpolation on an input image, comprising: a determination unit that determines whether an edge portion exists in the input image; A pixel interpolation device comprising: a switching unit configured to switch a processing method of the input image. 2. The switching means selects a first image processing method when an edge portion exists in the input image, and selects a second image processing method when an edge portion does not exist in the input image. However, in the first image processing method, in the process of repeating the normal transform and the inverse transform by the orthogonal transform, the two constraints that the information of the pass frequency band is correct and the spread of the image is limited. The pixel interpolation device according to claim 1, wherein the method is a method of restoring a high-frequency component lost at the time of sampling using a condition, and wherein the second image processing method is an interpolation method that does not reproduce a high-frequency component of image data. . 3. The apparatus according to claim 1, further comprising a block extracting unit configured to extract a block from the input image, wherein the determining unit determines whether an edge portion exists in the extracted block. 3. The pixel interpolation device according to 2.