KR101721632B1 - Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method - Google Patents

Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method Download PDF

Info

Publication number
KR101721632B1
KR101721632B1 KR1020160007579A KR20160007579A KR101721632B1 KR 101721632 B1 KR101721632 B1 KR 101721632B1 KR 1020160007579 A KR1020160007579 A KR 1020160007579A KR 20160007579 A KR20160007579 A KR 20160007579A KR 101721632 B1 KR101721632 B1 KR 101721632B1
Authority
KR
South Korea
Prior art keywords
image
pixels
unit
pixel
method
Prior art date
Application number
KR1020160007579A
Other languages
Korean (ko)
Inventor
전광길
Original Assignee
인천대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 인천대학교 산학협력단 filed Critical 인천대학교 산학협력단
Priority to KR1020160007579A priority Critical patent/KR101721632B1/en
Application granted granted Critical
Publication of KR101721632B1 publication Critical patent/KR101721632B1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/40Scaling the whole image or part thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/40Scaling the whole image or part thereof
    • G06T3/4015Demosaicing, e.g. colour filter array [CFA], Bayer pattern
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image

Abstract

A computer readable recording medium on which an image interpolation method, a method, and a method are recorded using a modified kernel regression method includes generating a low resolution image by downsampling an input image, upsamples a low resolution image, And calculates and interpolates pixel values using a modified kernel regression method for empty pixels between pixels.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for interpolating an image using a modified kernel regression method, and a computer-readable recording medium on which the method is recorded.

In particular, the present invention relates to a method of interpolating an image, and more particularly, to a method of generating a low-resolution image by downsampling an input image, upsampling a low-resolution image to fill an empty pixel with zero, And a computer-readable recording medium on which the method is recorded. 2. Description of the Related Art

Image interpolation is a technique for reconstructing an input low resolution image into a high resolution image. The image interpolation method is used for detailed analysis of single images such as satellite and medical images, and has been studied for a long time to apply to video and the like of high resolution enlargement of video.

In recent years, mobile devices having various screen resolutions are widely used. Therefore, the image interpolation method is also used to apply the same application or moving picture to different resolutions suitable for each device, and the importance thereof is further increased.

Commonly used image interpolation methods are filter-based image interpolation methods such as Bilinear interpolation (Binary) or Bicubic interpolation (Bicubic). However, these techniques cause blurring or jagging in the edge part of the image and considerably deteriorate the visual quality in consideration of the human visual structure which is sensitive to the boundary information of the image.

In order to solve such problems, the present invention provides a method of generating a low-resolution image by downsampling an input image, up-sampling a low-resolution image to fill an empty pixel with zero, And a computer readable recording medium on which the method is recorded. The present invention also provides a computer readable recording medium on which an image interpolation apparatus, a method, and a method are provided.

According to an aspect of the present invention, there is provided an apparatus for interpolating an image using a modified kernel regression method,

An image sensor unit having a color filter array to receive a color image;

A sampling filter for sampling a pixel-by-pixel unit of a specific row and a column in an input image to grasp a position and an intensity value of the pixel;

A down-sampling unit for down-sampling an image acquired by the sampling filter to generate a low-resolution image;

An upsampling unit for upsampling the low resolution image generated by the downsampling unit to generate a high resolution image and interleaving the empty pixels of the image to zero;

The luminance value of the empty pixel between the pixels obtained in the up-sampling unit is calculated by the modified kernel regression method which applies the weight information including the difference between the luminance value of the pixel obtained in the up-sampling unit and the position between the two pixels, A channel interpolator interpolating the liver; And

And an image generating unit for interpolating pixels in the channel interpolating unit and outputting the estimated upsampled image.

According to an aspect of the present invention, there is provided an image interpolation method using a modified kernel regression method,

Receiving a color image from an image sensor unit having a color filter array;

Sampling pixels of a specific row and column in the input image by a sampling filter and determining a position and an intensity value of the pixel;

Generating a low-resolution image by down-sampling an image acquired by the sampling filter;

Generating a high resolution image by upsampling the generated low resolution image and interleaving the empty pixels of the image to zero;

A brightness value of an empty pixel between the obtained pixels is calculated by a modified kernel regression method that applies weight information including the difference between the brightness values of the pixels obtained by the upsampling and the positions of the two pixels, step; And

And outputting an estimated upsampled image by interpolating between pixels.

According to the above-described configuration, the present invention can perform image interpolation more similar to the original image than the conventional image interpolation method by performing image interpolation using the modified kernel regression method.

FIG. 1 is a block diagram illustrating an image interpolation apparatus using a modified kernel regression method according to an embodiment of the present invention. Referring to FIG.
2 is a diagram showing an example using a general kernel regression method.
3 is a diagram illustrating a method of interpolating an image using a modified kernel regression method according to an embodiment of the present invention.
4 is a diagram showing an example of a test image according to an embodiment of the present invention.
5 to 7 are diagrams showing ranges in which weight information is applied according to the value of h according to the embodiment of the present invention.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

FIG. 1 is a block diagram illustrating an image interpolation apparatus using a modified kernel regression method according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example using a general kernel regression method.

The image interpolation apparatus 100 using the modified kernel regression method according to the embodiment of the present invention includes an image sensor unit 110, a downsampling unit 120, an upsampling unit 130, a channel interpolation unit 140, And a generating unit 150.

The image sensor unit 110 includes a color filter array to receive a color image.

The sampling filter 112 samples a pixel-by-pixel pixel in the Cartesian plane coordinates of a specific row and column of the input image, and determines the position of the sampled pixel and the intensity of the pixel.

The downsampling unit 120 downsamples the pixels acquired by the sampling filter 112 to generate a low-resolution image.

The upsampling unit 130 upsamples the low resolution image generated by the downsampling unit 120 to generate a high resolution image and interleaves the empty pixels of the image to zero.

The channel interpolating unit 140 calculates and interpolates pixel values between pixels by using a kernel regression transformed between the pixels extracted by sampling pixels in the Cartesian plane coordinates. The modified kernel regression method is described in detail in FIG. 3 below.

The kernel regression method will be briefly described as a known image interpolation method as follows. The kernel regression produces a regression equation f (X, Y) that can be estimated as accurately as possible for the data set, and the kernel regression equation used in the kernel regression method is a kernel-based function generated using previously observed data, It is made up of weights that multiply the function.

This weight is calculated in proportion to the distance between the point X at which the dependent variable value Y is to be estimated and the observation data X. [

The kernel-based function takes as arguments the diameter or width between the observations X and the locations x close to them. Thus, the kernel regression is the sum of the regression equations to which the individual local weights are applied.

The one-dimensional kernel regression method is expressed by the following equation (1).

Figure 112016007006240-pat00001

Figure 112016007006240-pat00002
Is an unknown regression function
Figure 112016007006240-pat00003
Gaussian noise in position
Figure 112016007006240-pat00004
P < / RTI > measurements with
Figure 112016007006240-pat00005
.

In this mathematical model,

Figure 112016007006240-pat00006
Close to
Figure 112016007006240-pat00007
Can be expressed by an Taylor series of N dimensions, as shown in the following Equation (2).

Figure 112016007006240-pat00008

In Equation (2)

Figure 112016007006240-pat00009
Lt; RTI ID = 0.0 >
Figure 112016007006240-pat00010
As shown in Fig.

Figure 112016007006240-pat00011
The minimum square expression is used as shown in the following Equation (3).

Figure 112016007006240-pat00012

K () is a kernel function that uses a function whose importance decreases as the distance increases. h is the smoothing parameter, the range or bandwidth to which the weight of the kernel function is applied.

When a mathematical model (kernel regression method) of [Expression 1], [Expression 2] and [Expression 3] is applied to image interpolation, upsampling is performed on pixels of a specified number of (x, y) And calculates and interpolates the pixel values using the kernel regression method for the empty pixels.

An example of image interpolation using the linear regression method is shown in FIG.

FIG. 3 is a diagram illustrating a method of interpolating an image using a modified kernel regression method according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating an example of a test image according to an embodiment of the present invention.

The image sensor unit 110 includes a color filter array to receive a color image as shown in FIG. 4 (S100).

The sampling filter 112 samples a specific row and column of the input image in pixel units in Cartesian plane coordinates. For example, filter 58 rows 1 through 20 of the image of FIG.

When sampling a pixel, the sampling filter 112 determines the position of the pixel and the intensity of the pixel.

The y1 value is the intensity of the pixel as shown in [Table 1], and the x-coordinate of y1 is shown in [Table 2].

Figure 112016007006240-pat00013

Figure 112016007006240-pat00014

The downsampling unit 120 downsamples the image sampled by the sampling filter 112 to generate a low-resolution image (S102).

The upsampling unit 130 upsamples the low-resolution image generated by the downsampling unit 120 to generate a high-resolution image, and interleaves the empty pixels of the image to zero (S104).

For example, if row 58 and row 1 through column 20 of the image of FIG. 4 are selected and blanked for each column, y1 is the original value and y2 is the downsampling value of 160, 157, 161, 162, 166, 169 , 170, 165, and 169, respectively.

The upsampling unit 130 performs interleaving so as to fill the empty pixels of the image with 0, as shown in the following [Table 3].

Figure 112016007006240-pat00015

In this case, the x2 coordinates of y2 are 1, 3, 5, 7, 9, 11, 13, 15, 17,

x and xs are the positions of the pixels, y is the intensity of the point at the x position, and ys is the intensity of the point at the xs position.

The value of xs is a value corresponding to xs (1) = 1 to xs (20) = 20, and is shown in Table 4 below.

Figure 112016007006240-pat00016

The channel interpolating unit 140 calculates a ys value, which is an intensity of a point at an xs position, by using a kernel regression transformed between pixels extracted by the sampling filter 112 when the xs value is 1 to 20 .

The modified kernel regression method is represented by Equation (4) and Equation (5) as Y, a matrix x, which is a y matrix, and a regression equation to which weight information is applied.

Figure 112016007006240-pat00017

Figure 112016007006240-pat00018

Y is an integer of 1 to n, Y (i) is a y matrix representing a luminance value of a pixel obtained in the up-sampling unit, X is an x matrix indicating a position (x) of a pixel, z is weight information, h Represents the range or bandwidth to which the weight information is applied as the smoothing parameter and is set to any value from 1 to 11. [

For example, when xs (i) = 20, xs (i) -X is 19, 17, 15, 13, 11, 9, 7, 5, 3, At this time, z (i) is 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0044, 0.2420.

Y is 160, 157, 161, 161, 162, 166, 169, 170, 165 and 169. Thus, the final result value ys2 is calculated as follows using Equations (4) and .

(1 to 20)

169.4280, 169.3600, 167.5270, 165.9594, 167.0272, 168.5236, 168.9281,

It can be seen that the final result value, ys2, is not significantly different from the original value y1, and the difference between ys2 and y1 is not large.

(1 to 20)

-0.3572, -2.4773, 0.7464, 0.0270, -0.4258, 0.9730, 0.1067, 0.5360, 0.3209, 0.0180, -0.1067, 0.4730, -0.2158, 1.4280, -0.6400, -0.4730, 0.9594, 1.0272, -0.4764,

The channel interpolating unit 140 acquires the luminance value of the pixel obtained by the upsampling unit 130 and the upsampling unit 130 using the modified kernel regression method that applies the weight information including the difference in distance between the two pixels, And interpolates between pixels by calculating a luminance value of an empty pixel between the pixels (S106).

The image generating unit 150 outputs the estimated upsampled image by interpolating the pixels in the channel interpolating unit 140 (S108).

5 to 7 are diagrams showing ranges in which weight information is applied according to the value of h according to the embodiment of the present invention.

5 (a) is a result of interpolating the image by the modified kernel regression method applying h = 1. 5 (b) is a result of interpolating the image by the modified kernel regression method applying h = 3.

FIG. 6C shows the result of interpolating the image by the modified kernel regression method using h = 5. FIG. 6 (d) shows the result of interpolating the image by the modified kernel regression method applying h = 7.

FIG. 7 (e) is a result of interpolating the image by the modified kernel regression method applying h = 9. FIG. 7 (f) shows the result of interpolating the image by the modified kernel regression method applying h = 11.

The above-described steps S100 to S108 may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.

Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CDROMs and DVDs; magneto-optical media such as Floptical disks; Hardware devices that are specifically configured to store and execute program instructions such as media (megneto-optical media) and ROM (ROM), RAM (RAM), flash memory and the like.

Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The embodiments of the present invention described above are not implemented only by the apparatus and / or method, but may be implemented through a program for realizing functions corresponding to the configuration of the embodiment of the present invention, a recording medium on which the program is recorded And such an embodiment can be easily implemented by those skilled in the art from the description of the embodiments described above.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

100: image interpolator
110: Image sensor unit
112: sampling filter
120: Downsampling unit
130: up-sampling unit
140: Channel interpreter
150:

Claims (5)

  1. An image sensor unit having a color filter array to receive a color image;
    A sampling filter for sampling a pixel unit pixel of a specific row and column in the input image to grasp a position and a luminance value of the pixel;
    A down-sampling unit for down-sampling an image acquired by the sampling filter to generate a low-resolution image;
    An upsampling unit for upsampling the low resolution image generated by the downsampling unit to generate a high resolution image and interleaving the empty pixels of the image to zero;
    A luminance value of an empty pixel between pixels acquired by the up-sampling unit is calculated by a modified kernel regression method that applies weight information including the difference between the luminance value of the pixel obtained in the up-sampling unit and the position of the two pixels A channel interpolator for interpolating pixels; And
    And an image generator for outputting an estimated upsampled image by interpolating pixels in the channel interpolator,
    In the modified kernel regression method, the luminance values of the empty pixels between the pixels extracted by the up-sampling unit are calculated using the following Equations (1) and (2), and weight information is applied according to the value of h Wherein the estimated kernel recurrence result is flattened as the value of h increases.
    [Equation 1]
    Figure 112016111787466-pat00030

    Here, i is an integer from 1 to n, and Y (i) is a y matrix representing a luminance value of a pixel obtained by the upsampling unit.
    &Quot; (2) "
    Figure 112016111787466-pat00031

    Here, xs (i) and x are the positions of the pixels, X is the matrix indicating the position (x) of the pixel, z is the weight information, h is the smoothing parameter and indicates the range or bandwidth to which the weight information is applied. ≪ / RTI >
  2. delete
  3. delete
  4. delete
  5. delete
KR1020160007579A 2016-01-21 2016-01-21 Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method KR101721632B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160007579A KR101721632B1 (en) 2016-01-21 2016-01-21 Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160007579A KR101721632B1 (en) 2016-01-21 2016-01-21 Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method

Publications (1)

Publication Number Publication Date
KR101721632B1 true KR101721632B1 (en) 2017-03-30

Family

ID=58503068

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160007579A KR101721632B1 (en) 2016-01-21 2016-01-21 Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method

Country Status (1)

Country Link
KR (1) KR101721632B1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090015566A (en) * 2007-08-09 2009-02-12 연세대학교 산학협력단 Method and apparatus for color interpolation
KR20150046113A (en) * 2012-08-21 2015-04-29 펠리칸 이매징 코포레이션 Systems and methods for parallax detection and correction in images captured using array cameras

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090015566A (en) * 2007-08-09 2009-02-12 연세대학교 산학협력단 Method and apparatus for color interpolation
KR20150046113A (en) * 2012-08-21 2015-04-29 펠리칸 이매징 코포레이션 Systems and methods for parallax detection and correction in images captured using array cameras

Similar Documents

Publication Publication Date Title
JP5861949B2 (en) Digital processing method and system for optical flow determination
JP4688432B2 (en) System for improving display resolution
Zhang et al. A super-resolution reconstruction algorithm for surveillance images
US20070217713A1 (en) Robust reconstruction of high resolution grayscale images from a sequence of low resolution frames
US7783130B2 (en) Spatial standard observer
US20100086227A1 (en) Image super-resolution using gradient profile prior
Kim et al. Accurate image super-resolution using very deep convolutional networks
US6816166B2 (en) Image conversion method, image processing apparatus, and image display apparatus
JP3915563B2 (en) An image processing apparatus and an image processing program
JP4874904B2 (en) Image processing apparatus and method
CN102150418B (en) Image amplification apparatus, a method and an integrated circuit
US20080122858A1 (en) Image masks
US20060290950A1 (en) Image superresolution through edge extraction and contrast enhancement
Khan et al. Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique
JP5450600B2 (en) Improving image resolution
Rajan et al. Simultaneous estimation of super-resolved scene and depth map from low resolution defocused observations
Chaudhuri Super-resolution imaging
US8655109B2 (en) Regression-based learning model for image upscaling
KR20060014879A (en) Resolution conversion method and apparatus
WO2004068862A1 (en) Method for creating high resolution color image, system for creating high resolution color image and program for creating high resolution color image
Lu et al. A revisit to MRF-based depth map super-resolution and enhancement
US20100002948A1 (en) Image enhancement
CN102770887A (en) Method and system for generating an output image of increased pixel resolution from an input image
JP2001512265A (en) Texture mapping in 3d computer graphics
CN101163224A (en) Super-resolution device and method

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant