KR101721632B1  Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method  Google Patents
Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method Download PDFInfo
 Publication number
 KR101721632B1 KR101721632B1 KR1020160007579A KR20160007579A KR101721632B1 KR 101721632 B1 KR101721632 B1 KR 101721632B1 KR 1020160007579 A KR1020160007579 A KR 1020160007579A KR 20160007579 A KR20160007579 A KR 20160007579A KR 101721632 B1 KR101721632 B1 KR 101721632B1
 Authority
 KR
 South Korea
 Prior art keywords
 image
 pixels
 unit
 pixel
 method
 Prior art date
Links
 239000002609 media Substances 0 title 1
Images
Classifications

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T3/00—Geometric image transformation in the plane of the image
 G06T3/40—Scaling the whole image or part thereof

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06F—ELECTRIC DIGITAL DATA PROCESSING
 G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
 G06F17/10—Complex mathematical operations

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T3/00—Geometric image transformation in the plane of the image
 G06T3/40—Scaling the whole image or part thereof
 G06T3/4015—Demosaicing, e.g. colour filter array [CFA], Bayer pattern

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T2207/00—Indexing scheme for image analysis or image enhancement
 G06T2207/10—Image acquisition modality
 G06T2207/10024—Color image
Abstract
Description
In particular, the present invention relates to a method of interpolating an image, and more particularly, to a method of generating a lowresolution image by downsampling an input image, upsampling a lowresolution image to fill an empty pixel with zero, And a computerreadable recording medium on which the method is recorded. 2. Description of the Related Art
Image interpolation is a technique for reconstructing an input low resolution image into a high resolution image. The image interpolation method is used for detailed analysis of single images such as satellite and medical images, and has been studied for a long time to apply to video and the like of high resolution enlargement of video.
In recent years, mobile devices having various screen resolutions are widely used. Therefore, the image interpolation method is also used to apply the same application or moving picture to different resolutions suitable for each device, and the importance thereof is further increased.
Commonly used image interpolation methods are filterbased image interpolation methods such as Bilinear interpolation (Binary) or Bicubic interpolation (Bicubic). However, these techniques cause blurring or jagging in the edge part of the image and considerably deteriorate the visual quality in consideration of the human visual structure which is sensitive to the boundary information of the image.
In order to solve such problems, the present invention provides a method of generating a lowresolution image by downsampling an input image, upsampling a lowresolution image to fill an empty pixel with zero, And a computer readable recording medium on which the method is recorded. The present invention also provides a computer readable recording medium on which an image interpolation apparatus, a method, and a method are provided.
According to an aspect of the present invention, there is provided an apparatus for interpolating an image using a modified kernel regression method,
An image sensor unit having a color filter array to receive a color image;
A sampling filter for sampling a pixelbypixel unit of a specific row and a column in an input image to grasp a position and an intensity value of the pixel;
A downsampling unit for downsampling an image acquired by the sampling filter to generate a lowresolution image;
An upsampling unit for upsampling the low resolution image generated by the downsampling unit to generate a high resolution image and interleaving the empty pixels of the image to zero;
The luminance value of the empty pixel between the pixels obtained in the upsampling unit is calculated by the modified kernel regression method which applies the weight information including the difference between the luminance value of the pixel obtained in the upsampling unit and the position between the two pixels, A channel interpolator interpolating the liver; And
And an image generating unit for interpolating pixels in the channel interpolating unit and outputting the estimated upsampled image.
According to an aspect of the present invention, there is provided an image interpolation method using a modified kernel regression method,
Receiving a color image from an image sensor unit having a color filter array;
Sampling pixels of a specific row and column in the input image by a sampling filter and determining a position and an intensity value of the pixel;
Generating a lowresolution image by downsampling an image acquired by the sampling filter;
Generating a high resolution image by upsampling the generated low resolution image and interleaving the empty pixels of the image to zero;
A brightness value of an empty pixel between the obtained pixels is calculated by a modified kernel regression method that applies weight information including the difference between the brightness values of the pixels obtained by the upsampling and the positions of the two pixels, step; And
And outputting an estimated upsampled image by interpolating between pixels.
According to the abovedescribed configuration, the present invention can perform image interpolation more similar to the original image than the conventional image interpolation method by performing image interpolation using the modified kernel regression method.
FIG. 1 is a block diagram illustrating an image interpolation apparatus using a modified kernel regression method according to an embodiment of the present invention. Referring to FIG.
2 is a diagram showing an example using a general kernel regression method.
3 is a diagram illustrating a method of interpolating an image using a modified kernel regression method according to an embodiment of the present invention.
4 is a diagram showing an example of a test image according to an embodiment of the present invention.
5 to 7 are diagrams showing ranges in which weight information is applied according to the value of h according to the embodiment of the present invention.
Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.
FIG. 1 is a block diagram illustrating an image interpolation apparatus using a modified kernel regression method according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example using a general kernel regression method.
The image interpolation apparatus 100 using the modified kernel regression method according to the embodiment of the present invention includes an image sensor unit 110, a downsampling unit 120, an upsampling unit 130, a channel interpolation unit 140, And a generating unit 150.
The image sensor unit 110 includes a color filter array to receive a color image.
The sampling filter 112 samples a pixelbypixel pixel in the Cartesian plane coordinates of a specific row and column of the input image, and determines the position of the sampled pixel and the intensity of the pixel.
The downsampling unit 120 downsamples the pixels acquired by the sampling filter 112 to generate a lowresolution image.
The upsampling unit 130 upsamples the low resolution image generated by the downsampling unit 120 to generate a high resolution image and interleaves the empty pixels of the image to zero.
The channel interpolating unit 140 calculates and interpolates pixel values between pixels by using a kernel regression transformed between the pixels extracted by sampling pixels in the Cartesian plane coordinates. The modified kernel regression method is described in detail in FIG. 3 below.
The kernel regression method will be briefly described as a known image interpolation method as follows. The kernel regression produces a regression equation f (X, Y) that can be estimated as accurately as possible for the data set, and the kernel regression equation used in the kernel regression method is a kernelbased function generated using previously observed data, It is made up of weights that multiply the function.
This weight is calculated in proportion to the distance between the point X at which the dependent variable value Y is to be estimated and the observation data X. [
The kernelbased function takes as arguments the diameter or width between the observations X and the locations x close to them. Thus, the kernel regression is the sum of the regression equations to which the individual local weights are applied.
The onedimensional kernel regression method is expressed by the following equation (1).
Is an unknown regression function Gaussian noise in position P < / RTI > measurements with .
In this mathematical model,
Close to Can be expressed by an Taylor series of N dimensions, as shown in the following Equation (2).
In Equation (2)
Lt; RTI ID = 0.0 > As shown in Fig.The minimum square expression is used as shown in the following Equation (3).
K () is a kernel function that uses a function whose importance decreases as the distance increases. h is the smoothing parameter, the range or bandwidth to which the weight of the kernel function is applied.
When a mathematical model (kernel regression method) of [Expression 1], [Expression 2] and [Expression 3] is applied to image interpolation, upsampling is performed on pixels of a specified number of (x, y) And calculates and interpolates the pixel values using the kernel regression method for the empty pixels.
An example of image interpolation using the linear regression method is shown in FIG.
FIG. 3 is a diagram illustrating a method of interpolating an image using a modified kernel regression method according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating an example of a test image according to an embodiment of the present invention.
The image sensor unit 110 includes a color filter array to receive a color image as shown in FIG. 4 (S100).
The sampling filter 112 samples a specific row and column of the input image in pixel units in Cartesian plane coordinates. For example, filter 58 rows 1 through 20 of the image of FIG.
When sampling a pixel, the sampling filter 112 determines the position of the pixel and the intensity of the pixel.
The y1 value is the intensity of the pixel as shown in [Table 1], and the xcoordinate of y1 is shown in [Table 2].
The downsampling unit 120 downsamples the image sampled by the sampling filter 112 to generate a lowresolution image (S102).
The upsampling unit 130 upsamples the lowresolution image generated by the downsampling unit 120 to generate a highresolution image, and interleaves the empty pixels of the image to zero (S104).
For example, if row 58 and row 1 through column 20 of the image of FIG. 4 are selected and blanked for each column, y1 is the original value and y2 is the downsampling value of 160, 157, 161, 162, 166, 169 , 170, 165, and 169, respectively.
The upsampling unit 130 performs interleaving so as to fill the empty pixels of the image with 0, as shown in the following [Table 3].
In this case, the x2 coordinates of y2 are 1, 3, 5, 7, 9, 11, 13, 15, 17,
x and xs are the positions of the pixels, y is the intensity of the point at the x position, and ys is the intensity of the point at the xs position.
The value of xs is a value corresponding to xs (1) = 1 to xs (20) = 20, and is shown in Table 4 below.
The channel interpolating unit 140 calculates a ys value, which is an intensity of a point at an xs position, by using a kernel regression transformed between pixels extracted by the sampling filter 112 when the xs value is 1 to 20 .
The modified kernel regression method is represented by Equation (4) and Equation (5) as Y, a matrix x, which is a y matrix, and a regression equation to which weight information is applied.
Y is an integer of 1 to n, Y (i) is a y matrix representing a luminance value of a pixel obtained in the upsampling unit, X is an x matrix indicating a position (x) of a pixel, z is weight information, h Represents the range or bandwidth to which the weight information is applied as the smoothing parameter and is set to any value from 1 to 11. [
For example, when xs (i) = 20, xs (i) X is 19, 17, 15, 13, 11, 9, 7, 5, 3, At this time, z (i) is 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0044, 0.2420.
Y is 160, 157, 161, 161, 162, 166, 169, 170, 165 and 169. Thus, the final result value ys2 is calculated as follows using Equations (4) and .
(1 to 20)
169.4280, 169.3600, 167.5270, 165.9594, 167.0272, 168.5236, 168.9281,
It can be seen that the final result value, ys2, is not significantly different from the original value y1, and the difference between ys2 and y1 is not large.
(1 to 20)
0.3572, 2.4773, 0.7464, 0.0270, 0.4258, 0.9730, 0.1067, 0.5360, 0.3209, 0.0180, 0.1067, 0.4730, 0.2158, 1.4280, 0.6400, 0.4730, 0.9594, 1.0272, 0.4764,
The channel interpolating unit 140 acquires the luminance value of the pixel obtained by the upsampling unit 130 and the upsampling unit 130 using the modified kernel regression method that applies the weight information including the difference in distance between the two pixels, And interpolates between pixels by calculating a luminance value of an empty pixel between the pixels (S106).
The image generating unit 150 outputs the estimated upsampled image by interpolating the pixels in the channel interpolating unit 140 (S108).
5 to 7 are diagrams showing ranges in which weight information is applied according to the value of h according to the embodiment of the present invention.
5 (a) is a result of interpolating the image by the modified kernel regression method applying h = 1. 5 (b) is a result of interpolating the image by the modified kernel regression method applying h = 3.
FIG. 6C shows the result of interpolating the image by the modified kernel regression method using h = 5. FIG. 6 (d) shows the result of interpolating the image by the modified kernel regression method applying h = 7.
FIG. 7 (e) is a result of interpolating the image by the modified kernel regression method applying h = 9. FIG. 7 (f) shows the result of interpolating the image by the modified kernel regression method applying h = 11.
The abovedescribed steps S100 to S108 may be implemented in the form of a program command that can be executed through various computer means and recorded in a computerreadable medium. The computerreadable medium may include program instructions, data files, data structures, and the like, alone or in combination.
The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.
Examples of computerreadable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CDROMs and DVDs; magnetooptical media such as Floptical disks; Hardware devices that are specifically configured to store and execute program instructions such as media (megnetooptical media) and ROM (ROM), RAM (RAM), flash memory and the like.
Examples of program instructions include machine language code such as those produced by a compiler, as well as highlevel language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.
The embodiments of the present invention described above are not implemented only by the apparatus and / or method, but may be implemented through a program for realizing functions corresponding to the configuration of the embodiment of the present invention, a recording medium on which the program is recorded And such an embodiment can be easily implemented by those skilled in the art from the description of the embodiments described above.
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.
100: image interpolator
110: Image sensor unit
112: sampling filter
120: Downsampling unit
130: upsampling unit
140: Channel interpreter
150:
Claims (5)
 An image sensor unit having a color filter array to receive a color image;
A sampling filter for sampling a pixel unit pixel of a specific row and column in the input image to grasp a position and a luminance value of the pixel;
A downsampling unit for downsampling an image acquired by the sampling filter to generate a lowresolution image;
An upsampling unit for upsampling the low resolution image generated by the downsampling unit to generate a high resolution image and interleaving the empty pixels of the image to zero;
A luminance value of an empty pixel between pixels acquired by the upsampling unit is calculated by a modified kernel regression method that applies weight information including the difference between the luminance value of the pixel obtained in the upsampling unit and the position of the two pixels A channel interpolator for interpolating pixels; And
And an image generator for outputting an estimated upsampled image by interpolating pixels in the channel interpolator,
In the modified kernel regression method, the luminance values of the empty pixels between the pixels extracted by the upsampling unit are calculated using the following Equations (1) and (2), and weight information is applied according to the value of h Wherein the estimated kernel recurrence result is flattened as the value of h increases.
[Equation 1]
Here, i is an integer from 1 to n, and Y (i) is a y matrix representing a luminance value of a pixel obtained by the upsampling unit.
&Quot; (2) "
Here, xs (i) and x are the positions of the pixels, X is the matrix indicating the position (x) of the pixel, z is the weight information, h is the smoothing parameter and indicates the range or bandwidth to which the weight information is applied. ≪ / RTI >  delete
 delete
 delete
 delete
Priority Applications (1)
Application Number  Priority Date  Filing Date  Title 

KR1020160007579A KR101721632B1 (en)  20160121  20160121  Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method 
Applications Claiming Priority (1)
Application Number  Priority Date  Filing Date  Title 

KR1020160007579A KR101721632B1 (en)  20160121  20160121  Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method 
Publications (1)
Publication Number  Publication Date 

KR101721632B1 true KR101721632B1 (en)  20170330 
Family
ID=58503068
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

KR1020160007579A KR101721632B1 (en)  20160121  20160121  Apparatus and Method for Interpolating Video Using Modified Kernel Regression and Computer Recordable Medium Storing the Method 
Country Status (1)
Country  Link 

KR (1)  KR101721632B1 (en) 
Citations (2)
Publication number  Priority date  Publication date  Assignee  Title 

KR20090015566A (en) *  20070809  20090212  연세대학교 산학협력단  Method and apparatus for color interpolation 
KR20150046113A (en) *  20120821  20150429  펠리칸 이매징 코포레이션  Systems and methods for parallax detection and correction in images captured using array cameras 

2016
 20160121 KR KR1020160007579A patent/KR101721632B1/en active IP Right Grant
Patent Citations (2)
Publication number  Priority date  Publication date  Assignee  Title 

KR20090015566A (en) *  20070809  20090212  연세대학교 산학협력단  Method and apparatus for color interpolation 
KR20150046113A (en) *  20120821  20150429  펠리칸 이매징 코포레이션  Systems and methods for parallax detection and correction in images captured using array cameras 
Similar Documents
Publication  Publication Date  Title 

JP5861949B2 (en)  Digital processing method and system for optical flow determination  
JP4688432B2 (en)  System for improving display resolution  
Zhang et al.  A superresolution reconstruction algorithm for surveillance images  
US20070217713A1 (en)  Robust reconstruction of high resolution grayscale images from a sequence of low resolution frames  
US7783130B2 (en)  Spatial standard observer  
US20100086227A1 (en)  Image superresolution using gradient profile prior  
Kim et al.  Accurate image superresolution using very deep convolutional networks  
US6816166B2 (en)  Image conversion method, image processing apparatus, and image display apparatus  
JP3915563B2 (en)  An image processing apparatus and an image processing program  
JP4874904B2 (en)  Image processing apparatus and method  
CN102150418B (en)  Image amplification apparatus, a method and an integrated circuit  
US20080122858A1 (en)  Image masks  
US20060290950A1 (en)  Image superresolution through edge extraction and contrast enhancement  
Khan et al.  Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique  
JP5450600B2 (en)  Improving image resolution  
Rajan et al.  Simultaneous estimation of superresolved scene and depth map from low resolution defocused observations  
Chaudhuri  Superresolution imaging  
US8655109B2 (en)  Regressionbased learning model for image upscaling  
KR20060014879A (en)  Resolution conversion method and apparatus  
WO2004068862A1 (en)  Method for creating high resolution color image, system for creating high resolution color image and program for creating high resolution color image  
Lu et al.  A revisit to MRFbased depth map superresolution and enhancement  
US20100002948A1 (en)  Image enhancement  
CN102770887A (en)  Method and system for generating an output image of increased pixel resolution from an input image  
JP2001512265A (en)  Texture mapping in 3d computer graphics  
CN101163224A (en)  Superresolution device and method 
Legal Events
Date  Code  Title  Description 

E701  Decision to grant or registration of patent right  
GRNT  Written decision to grant 