KR20070119482A - Image resampling method - Google Patents

Abstract

An image re-sampling method is provided to improve an image before interpolation and to remove artifacts derived from a JPEG format when color and gray-scale images having high resolution are enlarged. Binary images are improved in order to remove artifacts with respect to coded images having losses. The mesh of plural coordinate points is formed with respect to the improved image. Neighboring pixels most close to each pixel of the mesh are determined, and an edge map is formed based on absolute values of partial derivation values(202). Interpolation is performed according to the edge map. Interpolation is performed in relation to each color component. The image is improved thereafter such that the artifacts are reduced. Different interpolation processes are employed based on the edge information about each pixel.

Description

Image Resampling Method

1 is a block diagram of an edge interpolation method in an image resampling method according to an embodiment of the present invention.

2 is a flowchart of edge map forming steps for an edge interpolation method in an image resampling method according to an embodiment of the present invention.

3 is a flowchart of edge map processing steps for an edge direction interpolation method in an image resampling method according to an embodiment of the present invention.

4 is a flowchart of resampling method steps for an edge interpolation method in an image resampling method according to an embodiment of the present invention.

FIG. 5 is a diagram of interpolated schematic edges, with indexes 1 through 4 as mentioned in step 406 of FIG.

FIG. 6 is a plot of interpolated schematic edges at indexes 5 through 8 as mentioned in step 406 of FIG.

7 is a flowchart of post-processing stages in an image resampling method according to an embodiment of the present invention.

8 is a hue mapping curve graph for different values of control parameter n.

9 is a 4x magnification image before edge enhancement produced by the proposed scheme for the initial image, an 4x magnification image with edge enhancement, and a 4x magnification image with conventional bi-cubic interpolation.

The present invention relates to an image resampling method. Image resampling according to the present invention relates to a method for computing the luminance values of points that do not belong to a starting point set, given luminance values.

Image resampling is the basic behavior in image processing and deinterlacing not only image enlargement and reduction, but also image rotation, wrapping, affinity transformation, and increasing time resolution of video sequences. Deinterlacing is commonly used in applications. For example, interpolation is used for image and video resizing. Interpolation is also required in single-chip charge combiners, which are widely used in digital cameras and camcorders, and affects the general image processing pipeline. Many interpolation algorithms have been proposed so far, and there is a big difference in complexity and performance.

Conventional methods, such as bilinear and bicubic methods, create some obstacles to artifacts around the edges on the image. Representative artifacts that can be seen include jagging artifacts or checkerboard patterns, blurring of images due to camouflage of the smoothness of the image, and ringing due to the removal of high frequency portions of the image spectrum. (ringing) artifacts, etc.

Typically 2D interpolation is a series of 1D interpolations and is performed in a direction perpendicular to each other. For example, the solution disclosed in US Pat. No. 6,915,026 proposes a separable method, which first enlarges an image in a column direction, then enlarges an image in a row direction, and then precomputes and stores interpolation coefficients in advance. I suggest that. These techniques have the disadvantage that all 1D interpolation directions aligned as image coordinate axes and their artifacts tend to reveal the underlying sampling grid. Some solutions suggest preliminarily filtering the input image with a low pass filter in two diagonal directions, but this method degrades the image quality. U.S. Pat.Nos. 5,719,967 and 6,606,093 provide special techniques for suppressing jagging artifacts.

The solution disclosed in US Pat. No. 5,446,804 uses a precomputed subpixel edge map. For every pixel that is interpolated, the median is computed using two diagonal neighbors based on the edge map. The value located on the other side of the edge is then replaced with the composite value computed using the intermediate value computed with the neighboring pixels. This solution gives clear results but is not free from jaggies.

Then, solution [1] approaches the problem of jaggies by leaving the image sampling grid. This solution proposes to perform a triangulation that treats grayscale values of the image as points on the surface. In this case, the intensity means a rise. Proposers construct triangulation meshes and then reconstruct images from the triangulation mesh. Their approach adopted traditional data dependency triangulation (DDT), optimizing with a new cost function. The data dependent triangulation thus matches the edges in the image, improving the reconstructed image. This method is relatively complex and computationally expensive, and triangulation is performed repeatedly.

U. S. Patent No. 6,928, 196 discloses a solution in which edge directional interpolation is performed. Here, an edge map is constructed by first calculating edge intensities for each of the individual sample values of the image data and the direction of the edge. Then, the edge map and edge direction A also apply morphological procedures to erase and extend them. The direction map is then quantified in one of four directions: horizontal (0), diagonal (π / 4), vertical (π / 2) and nondiagonal (3π / 4). The interpolation kernel is then selected according to the edge strength and the edge direction.

Typically the digital image magnification process is performed in two steps. First, it computes the corrupted pixels of a high resolution image. Second, processing is performed for the purpose of edge improvement. For example, US Pat. No. 6,714,688 discloses a solution that proposes a method for post-sharpening an image. According to a preferred embodiment, first, bilinear interpolation is used, and then the edge enhancement technique is applied in a window-based manner. The size of the window is determined from the resizing parameters. First, high frequency components are extracted by low-passing the image portion inside the window. The extracted high frequency component is added to the low pass portion highlighted by the factor obtained from the local luminance statistics. Thereafter, further modifications are made to sharpen the edges using edge values and a special tone reproduction curve.

The solution disclosed in EP1533899A1 provides a method for resolution transformation which constitutes edge directional interpolation and back edge enhancement. First magnify the image by 2n times and then reduce it to the desired magnification Then, the edge improvement technique is applied. This solution gives good results but is too computationally complex.

Additional references:

[1] "Image Reconstruction Using Data-Dependent Triangulation," by Yu, X., Morse, B., and T. Sederberg, in IEEE Computer Graphics and Applications, Volume 21, pages 62-68, 2001.

Conventional image resampling methods typically perform 2D interpolation, as a series of 1D interpolations, in directions perpendicular to each other (eg, as in US Pat. No. 6,915,026). Thus the 1D interpolation direction edges are all aligned in line with the image coordinate axes, and their artifacts reveal the underlying sampling grid.

Methods that suggest deviating from interpolating along image coordinate axes are computationally complex and require too much processing time.

Some of the developed edge-oriented interpolation algorithms require a fixed structure of the mesh, which means to enlarge, in particular, to a power of two.

For example, in the case of image zooming for JPEG coded images, as are typical, visually identifiable artifacts or frames of MPEG video sequences become more pronounced. Because they expand well with the rest of the image.

Accordingly, the present invention has been made to solve the above-described problem, in the case of image magnification for color and grayscale images that provide high quality sharp images at high resolution, the image is improved before interpolation and lossy compression is performed. It is an object to provide an image resampling strategy for removing artifacts borne by (such as block artifacts of ringing in the case of JPEG compression algorithms).

In order to achieve the above object, the present invention provides an image resampling method for calculating luminance values at points that do not belong to an initial set of points given luminance values, in order to eliminate typical artifacts for lossy coded images. Improving the previous image, constructing a mesh of a plurality of coordinate points with respect to the interpolated image, for each pixel position of the constructed mesh, determining 16 nearest neighbors on the low resolution image, Constructing an integer edge map from 1 to 8 in eight directions based on absolute values, performing interpolation according to the edge map, repeating the previous steps for each color component, reducing artifacts, Post-improving the image to see and improve the sharpness of the edges. Characterized in that it also.

Here, different interpolation procedures are applied to each pixel based on the edge information.

A diagonal line is selected for each edge direction interpolation, 1D interpolation is performed in parallel with the image coordinate axes, and then performed at an angle with respect to the image coordinate axis.

In addition, the image enlargement is performed stepwise by the enlargement ratio R, but first constitute n intermediate images, where the kth image (k = 1 ... n) is the first image magnified by Tk times, and Tk <R It is characterized by.

Here, for image improvement, the post-processing is applied using tonal mapping, and for the local contrast image improvement, the tonal mapping curve characteristic is controlled by a parameter.

Here, the mapping function is characterized in that the mapping function having the control parameter n is as follows.

The image resampling method according to the present invention is based on the construction of the edge map. Here, the edges are oriented in eight directions. In addition, interpolation is performed in three steps. That is, the first two are aligned with the image coordinate axes, and the third one constitutes an angle with respect to the image coordinate axes, where the angle depends on the value from the edge map. Using a special tone mapping curve, post-sharpening the image improves the local contrast of the edge regions. Sharpening is controlled by the curve slope. As a result, the image appears sharper than conventional bicubic interpolation while the jaggies also decrease in the images. The recoveries can also be suppressed by multiple application of the image resampling method according to the present invention. That is, when performing image enlargement step by step at an enlargement ratio R, first, n intermediate images are constructed. Here, the k-th image (k = 1 ... n) is the first image magnified by Tk times, and Tk <R. Normally n = 2 is sufficient. At the same time, this method is very fast and does not require excessive memory resources.

In the case of image magnification, the image resampling method according to the present invention improves the image before interpolation and removes artifacts burdened by lossy compression (equivalent to the block artifacts of ringing in the case of JPEG compression algorithms). Suggest to do.

1 is a block diagram of an edge interpolation method in an image resampling method according to an embodiment of the present invention.

As shown in Fig. 1, the system operation process of performing the edge interpolation method according to the present invention is controlled by the processor 101 executing the program code stored in the memory 102. A source grayscale or color photograph is also stored in the memory 102. The image is expanded and transferred to the display device 104. Acceptance of the conversion performed is ascertained by the user using the input device 105. The compensated image is transferred to the printing device 103. Information exchange is performed via the data bus 106.

The following describes the process of forming an image.

2 is a flowchart of edge map forming steps for an edge interpolation method in an image resampling method according to an embodiment of the present invention.

As shown in Figure 2, Figure 2 illustrates the basic steps of forming a direction map. Let I be an input image with a size of 4 ㅧ 4 pixels or more. For each pixel of the input image, the following procedure is performed until the condition of step 201 is satisfied. Sixteen nearest neighbors are extracted (step 202). The norms for the direction derivatives in the eight directions are calculated as follows (step 203). Assume that the four nearest neighbors of the map are I (i, j), I (i + Ij), I (ij + 1) and I (i + Ij + 1). then,

Here, Ix, Iy, Id1, Id2, Ixxy, Iyxx, Ixyy, Iyyx are the norms of the direction derivatives in the eight directions. In step 204, the smallest value among the direction derivatives is selected. In step 205, the index of the corresponding direction is stored in the direction map. Accordingly, the artifacts for [Ix, Iy, Id1, Id2, Ixxy, Iyxx, Ixyy, Iyyx] are set as [1,2,3,4,5,6,7,8].

3 is a flowchart of edge map processing steps for an edge direction interpolation method in an image resampling method according to an embodiment of the present invention.

3 shows preprocessing of the direction map. First, the direction map is removed from the expansion directions. Windows 3x3 pixels are used for this purpose (step 302). If there are at least three different directions with five or more artifacts, the artifact at the midpoint is substituted in the following way. That is, 5 and 6 are substituted with 1 and 7 and 8 are substituted with 2 (step 303). Next, in steps 304-307, the morphological action (after erosion with the same component is extended) in which small isolated edges (having five or more artifacts in the edge map), which can be attributed to noise in the image, are closed. To be removed. Morphological erosion behavior marks the central pixel as non-zero only when all non-zero elements in the component are aligned with non-zero elements in the image. The morphological erosion operation marks the central pixel as nonzero only when at least one nonzero element in the component is aligned with the nonzero elements in the image. The components are selected according to the edge artifacts in the following manner.

Then, interpolation is performed according to the direction map.

4 is a flowchart of resampling method steps for an edge interpolation method in an image resampling method according to an embodiment of the present invention.

4 shows an interpolation procedure. First, a mesh is constructed for the interpolated image (step 400). If the current color component is not the last one (condition of step 401), for each point A of the constructed mesh, the following procedure is performed. The coordinates of A are determined (step 403). Sixteen nearest neighbors are extracted for point A of the input image (step 404). Next, the coordinates of the nearest upper left neighbor are read (step 404), the corresponding edge direction is read from the direction map (step 405), and interpolation is performed according to the edge direction value. This process (step 406) is described in more detail later. After the last point in the constructed mesh has been processed, the condition of step 402 is satisfied. The above procedure, in particular steps 402-409, is repeated for each color component in the RGB or YCrCb color space. After the condition of step 401 is satisfied, the final image is output.

FIG. 5 is a plot of interpolated schematic edges, with indexes 1 through 4, as mentioned in step 406 of FIG.

FIG. 6 is a plot of interpolated schematic edges at indexes 5 through 8 as mentioned in step 406 of FIG.

 5 and 6 show a basic configuration diagram for performing interpolation as mentioned in step 406. Edge-wise artifacts are shown in the left column, and interpolation layouts for a given artifact are demonstrated in the adjacent right column. Let point A of the constructed mesh (marked with cross (×)) be surrounded by 16 nearest neighbors. At points P1 and P2, line P1P2 passes point A, and its tilt angle is 45 °, -45 °, arctg (0.5), -arctg (0.5), arctg (0.5) + 90 °, arctg (0.5) Obtain it by -90 °. Luminance values at points P1 and P2 (corresponding edges are marked with bold lines) are obtained using 1D linear interpolation, ie using two points for P1 and two points for point P2. The methods for 1D interpolation are not limited to those described above and may use other kernels and may involve more or fewer points to compute interpolation coefficients.

If necessary, the image is post processed to emphasize the edges.

7 is a flowchart of post-processing stages in an image resampling method according to an embodiment of the present invention.

7 shows the basic steps for edge improvement. The corresponding current edge thickness in the horizontal and vertical directions, Kh and Kv is first determined as approximations of the enlargement ratio along the horizontal and vertical directions (step 700). Next, construct a look-up table (LUT) for the hue mapping curve for local contrast adjustment (step 701). The hue mapping curve is a mapping of the segment [0,1] itself, and its derivatives at point 0.5 must be at least one. The final edge sharpness depends on the curve shape, in particular its tilt angle at 0.5. That is, the larger the angle, the sharper the edge, i.e. the sharper. The function on the curve is selected as follows:

Here, the curve slope angle at x = 0.5 is controlled by parameter n.

8 is a hue mapping curve graph for different values of control parameter n.

8 shows hue mapping curves for several ns. For each pixel of the image, the following procedure is performed.

Find channel c1. This value is the maximum of two other things in color space RGB or YCrCb. In addition, for each pixel in steps 704-708 (until the condition of step 703 is satisfied), the values of this channel Ic1 (i, j) are processed. Obtain the minimum and maximum values of the pixels in the local neighborhood, scale the input pixel Ic1 (i, j) and set three- [0,1] as follows (step 705):

The mapped value d is then obtained at step 706 as f (scaled value), and then it is segmented at step 705 as follows (local-minimum, local-maximum). -maximum)] to scale back:

의 비와 그의 입력 값의 곱으로서 구하고, 채널의 초기값을 다음과 같이 처리한다:In the case of a color image, the values of the remaining channels

Obtained as the product of the ratio of and its input value, the initial value of the channel is processed as follows:

As mentioned above, Ic1 (i, j) is selected as the maximum value among Ic1 (i, j), Ic2 (i, j) and Ic3 (i, j). If Ic1 (i, j) is zero, all remaining components are equally zero.

9 is a 4x magnification image before edge enhancement produced by the proposed scheme for the initial image, an 4x magnification image with edge enhancement, and a 4x magnification image with conventional bi-cubic interpolation. 9 illustrates the application of the method for 5x magnification of any image as a comparison to regular bicubic interpolated results.

Other aspects of the invention will be apparent from the drawings and description of the preferred embodiments of the invention. Those skilled in the art will appreciate that other embodiments of the invention are possible and the description of the invention can be modified in various ways without departing from the spirit of the invention. Accordingly, the drawings and descriptions presented herein are to be regarded as illustrative and not restrictive.

The proposed image resampling method of the present invention can be realized in printing software or firmware.

As described above, the image resampling method according to the present invention can be applied for improving photographs during printing in photo printers and MFPs.

Claims (6)

An image resampling method for computing luminance values at points not belonging to an initial point set at which luminance values are given, (a) improving the previous image to remove artifacts for the lost coded images; (b) constructing a mesh of a plurality of coordinate points for the improved image; (c) for each pixel position of the constructed mesh, determining nearest neighbors on a low resolution image and constructing an edge map based on absolute values of some derived values; (d) performing interpolation according to the edge map; (e) repeating step (d) for each color component; (f) post-improving the image such that the artifacts are reduced. The method of claim 1, And in step (d), a different interpolation procedure is applied to each pixel based on the edge information. The method of claim 1, Selecting a diagonal line for each edge direction interpolation, performing 1D interpolation in parallel with the image coordinate axes, and then performing an angle with respect to the image coordinate axis. The method of claim 1, The image enlargement is performed stepwise by an enlargement ratio R, wherein first n intermediate images are constructed, and the k th image (k = 1 ... n) is the first image magnified by Tk times, and Tk <R. Image resampling method. The method of claim 1, Apply post-processing using hue mapping to improve the image, and the hue mapping curve characteristic is controlled by a parameter for local contrast image enhancement. The method of claim 5, The mapping function has a control parameter n and has the following equation.

Images