TWI406187B - Fast and high quality image/video interpolation method and apparatus - Google Patents

Fast and high quality image/video interpolation method and apparatus Download PDF

Info

Publication number
TWI406187B
TWI406187B TW99127730A TW99127730A TWI406187B TW I406187 B TWI406187 B TW I406187B TW 99127730 A TW99127730 A TW 99127730A TW 99127730 A TW99127730 A TW 99127730A TW I406187 B TWI406187 B TW I406187B
Authority
TW
Taiwan
Prior art keywords
image
pixels
resolution image
weight
pixel difference
Prior art date
Application number
TW99127730A
Other languages
Chinese (zh)
Other versions
TW201209755A (en
Inventor
Jar Ferr Yang
King-Hong Chung
Original Assignee
Univ Nat Cheng Kung
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Univ Nat Cheng Kung filed Critical Univ Nat Cheng Kung
Priority to TW99127730A priority Critical patent/TWI406187B/en
Publication of TW201209755A publication Critical patent/TW201209755A/en
Application granted granted Critical
Publication of TWI406187B publication Critical patent/TWI406187B/en

Links

Landscapes

  • Image Processing (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Television Systems (AREA)

Abstract

The invention provides a fast and high quality image/video interpolation method and apparatus. An up-sampling step first estimates the diagonal missing pixels and interpolates the remaining missing pixels based on the estimation result. An edge-sensing interpolator is provided to guide the interpolation automatically along the edges, where local gradients are non-linearly mapped to achieve the weights. A post-processing refinement step then updates the interpolated high resolution image iteratively, by adding the error between the observed low resolution image and the simulated low resolution image, to attain a clear, natural and visually pleasuring high resolution output.

Description

快速高畫質視頻影像內插方法及裝置Fast high-definition video image interpolation method and device

本發明係關於影像處理之技術領域,尤指一種快速高畫質視頻影像內插方法及裝置。The invention relates to the technical field of image processing, in particular to a method and a device for interpolating fast high-definition video images.

隨著顯示面板技術的高速發展,大型超高解析度(Quad Full HD,即3841×2160解析度)多媒體平面電視及播放器將成未來重點開發的電子產品項目之一。為使現有的高解析度影像及視頻媒體能在這些超高解析度顯示面板上播放,快速高畫質影像內插方法之研發則是未來研發趨勢。With the rapid development of display panel technology, large-scale ultra-high resolution (Quad Full HD, ie 3841×2160 resolution) multimedia flat-panel TVs and players will become one of the key electronic product projects to be developed in the future. In order to enable existing high-resolution image and video media to be played on these ultra-high-resolution display panels, the development of fast high-quality image interpolation methods is a future development trend.

美國第US5,001,563號專利案公告中,利用欲插值點的鄰近影像進行濾波以產生插值點,其係著重在濾波器的設計上。然而此種利用濾波器方法由於只參考插值點之鄰近影像,而對於需要較遠之影像訊息才能進行插值的影像,例如傾斜度比較低以及細斜線的影像,由於影像訊息不足,因此所插值的影像會因誤判,而將原本細斜線的影像產生鋸齒狀(zigzag)及影像模糊。In the US Patent No. 5,001,563, the adjacent image to be interpolated is filtered to generate an interpolation point, which focuses on the design of the filter. However, the use of the filter method only refers to the adjacent image of the interpolation point, and the image that can be interpolated for the image information that needs to be far away, such as the image with low inclination and thin oblique line, because of insufficient image information, the interpolated value The image will be jagged and zigzag due to misjudgment.

為了彌補因影像訊息不足而造成影像誤判以及模糊等問題,美國第US6,133,957號專利案公告中,提出使用紋路變化最小法(Least texture variance method)之設計,其係利用尋找欲插值影像中紋路變化最小者當作插值的影像來源。同時,為了彌補因影像訊息不足而造成影像誤判以及模糊的缺失,紋路變化最小法則藉由尋找影像中紋路變化最小者,當作插值的影像來源的方式。可是由於該法則往往會出現一些混亂的情形,如出現多於一個紋路變化最小者等,在此情形下,影像誤判的機率也大增,因而產生不自然的影像。而為避免誤判,其加入了可信度(Confidence)判斷的機制,此機制利用紋路變化複雜度的高低而給予可信度的大小,當可信度小時則採用垂直方向做插值的影像來源,以避免可能的誤判。然而此方法對於複雜的影像,由於其大多採用垂直方向做插值,因此造成影像的不連續性。In order to compensate for the problem of image misjudgment and ambiguity caused by insufficient image information, the US Patent No. 6,133,957 proposes the use of the Least texture variance method, which is used to find the texture in the image to be interpolated. The smallest change is used as the source of the interpolated image. At the same time, in order to compensate for the lack of image misjudgment and the lack of blur due to insufficient image information, the minimum rule of texture change is to find the source of the interpolated image by finding the smallest variation in the image. However, due to this law, there are often some confusing situations, such as the occurrence of more than one trace change, etc. In this case, the probability of image misjudgment is also greatly increased, resulting in unnatural images. In order to avoid misjudgment, it adds a mechanism of Confidence judgment, which uses the complexity of the texture change to give the degree of credibility. When the credibility is small, the image source is interpolated in the vertical direction. To avoid possible misjudgments. However, this method causes image discontinuity due to the fact that most of the images are interpolated in the vertical direction.

由前述可知,習知影像內插方法只針對畫質的追求,往往忽略對運算量的考慮,加上近年顯示面板技術進歩快速,面板的解析度越來越高,使得習知高運算量的內插法在即時的應用上的可行性越來越低。因此,習知增加影像解析度之內插技術有諸多缺失而有予以改進之必要。It can be seen from the foregoing that the conventional image interpolation method only focuses on the pursuit of image quality, and often neglects the consideration of the amount of calculation. In addition, the display panel technology has been rapidly developed in recent years, and the resolution of the panel is getting higher and higher, which makes the conventional high computational amount. The feasibility of interpolation in real-time applications is getting lower and lower. Therefore, there are many shortcomings in the conventional interpolation technique for increasing image resolution, and it is necessary to improve it.

本發明之主要目的係在提供一種快速高畫質視頻影像內插法,可利用加權平均內插法,依著影像圖案之紋理計算出該空白像素之內插值,並反覆地利用原始輸入低解析度影像與內插後高解析度影像模擬出來之低解析影像之間的誤差,對高解析度影像進行修正,令輸出影像更清晰、自然及像真。The main object of the present invention is to provide a fast high-definition video image interpolation method, which can calculate the interpolation value of the blank pixel according to the texture of the image pattern by weighted average interpolation method, and repeatedly utilize the original input low resolution. The error between the image and the low-resolution image simulated by the high-resolution image after interpolation, the high-resolution image is corrected to make the output image clearer, more natural and true.

依據本發明之一特色,本發明提出一種快速高畫質視頻影像內插法,係使用於一倍頻器(SCALER)中,以於一低解析度影像中插入像素而增加該低解析度影像的解析度,俾產生一高解析度影像,該低解析度影像係由排列成二維陣列的多數個像素所組成,其中包含像素X 2 i ,2 j X 2 i ,2 j +2X 2 i +2,2 j +2X 2 i +2,2 j i ,j 為正整數,該影像內插法包含下列步驟:(A)計算一斜向空白像素X 2 i +1,2 j +1 其鄰近的向左上、向右上、向右下及向左下四個方向之絕對像素差;(B)依據步驟(A)中的絕對像素差及一非線性函數以計算該斜向空白像素X 2 i +1,2 j +1 的向左上、向右上、向右下及向左下四個方向之權重;(C)依據步驟(B)中的權重以計算該斜向空白像素之值;(D)計算一餘下空白像素X 2 i ,2 j +1 其鄰近四角已知像素方向之絕對像素差;(E)依據步驟(D)中的絕對像素差及該非線性函數以計算該餘下空白像素X 2 i ,2 j +1 之向上權重、向下權重W D 、向左權重W L 及向右權重W R ;(F)計算該餘下空白像素X 2 i ,2 j +1 的水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV );(G)依據步驟(F)中的水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV )以計算該餘下空白像素X 2 i ,2 j +1 之值。According to a feature of the present invention, the present invention provides a fast high-definition video image interpolation method for use in a single frequency multiplier (SCALER) to insert pixels in a low-resolution image to increase the low-resolution image. The resolution, 俾 produces a high-resolution image consisting of a plurality of pixels arranged in a two-dimensional array, including pixels X 2 i , 2 j , X 2 i , 2 j +2 , X 2 i +2,2 j +2 and X 2 i +2,2 j, i, j is a positive integer, the image interpolation method comprising the steps of: (a) calculating a diagonal blank pixels X 2 i +1 , 2 j +1 the absolute pixel difference of its neighboring upper left, upper right, lower right, and lower left directions; (B) calculating the oblique according to the absolute pixel difference in step (A) and a nonlinear function The weights to the upper left, upper right, lower right, and lower left directions of the blank pixels X 2 i +1 , 2 j +1 ; (C) calculate the oblique blank pixels according to the weights in step (B) (D) calculating the absolute pixel difference of a remaining blank pixel X 2 i , 2 j +1 in the direction of the known pixel adjacent to the four corners; (E) according to the absolute pixel difference in step (D) and the non-line Of the function to calculate the remaining blank pixels X 2 i, up right 2 j +1 heavy, the weight W D downward, leftward, the weight W L and the right weight W R; (F) calculates the remaining blank pixels X 2 i, 2 j +1 horizontal absolute pixel difference (Δ H ) and vertical absolute pixel difference (Δ V ); (G) absolute pixel difference (Δ H ) in the horizontal direction and absolute pixel difference in the vertical direction in step (F) (Δ V ) to calculate the value of the remaining blank pixels X 2 i , 2 j +1 .

依據本發明之另一特色,本發明提出一種快速高畫質視頻影像內插裝置,係接收一低解析度影像並於該低解析度影像中插入像素而增加其解析度,俾產生一高解析度影像,該低解析度影像係由排列成二維陣列的多數個像素所組成,其中包含像素X 2 i ,2 j i ,j 為正整數,該影像內插裝置包含一視頻影像內插處理器、一線緩衝器、一表格裝置、及一影像強化後處理器。該視頻影像內插處理器接收該低解析度影像,並使用一非線性指數函數(exponential function)及一斜向空白像素X 2 i +1,2 j +1 其鄰近的向左上、向右上、向右下及向左下四個方向之絕對像素差,計算出該斜向空白像素X 2 i +1,2 j +1 及鄰近像素之權值,並根據該權值及該斜向空白像素X 2 i +1,2 j +1 鄰近之像素值,對該斜向空白像素X 2 i +1,2 j +1 作加權平均內插。該線緩衝器連接至該視頻影像內插處理器,以暫存該視頻影像內插處理器所產生之資料。該表格裝置連接至該視頻影像內插處理器,依據視頻影像內插處理器輸出之指標,利用查表方式,以對該指數函數作即時運算。該影像強化後處理器連接至該線緩衝裝置,以對該高解析度影像進行影像強化後處理。According to another feature of the present invention, the present invention provides a fast high-definition video image interpolation device that receives a low-resolution image and inserts pixels into the low-resolution image to increase the resolution thereof, thereby generating a high resolution. a low-resolution image consisting of a plurality of pixels arranged in a two-dimensional array, wherein the pixels X 2 i , 2 j , i , j are positive integers, and the image interpolation device includes a video image interpolation A processor, a line buffer, a table device, and an image enhancement post processor. The interpolation within video image processor receives the low-resolution image, and using a non-linear exponential function (exponential function), and a blank pixel X on the obliquely adjacent left, upper right 2 i +1,2 j +1, the left and right pixel absolute difference of the four directions, right +1,2 j + 1 is calculated and the values of the neighboring pixels X 2 I oblique blank pixels, and weights based on the diagonal of the blank and the pixel X 2 i +1,2 j +1 value adjacent to the pixel, the oblique blank pixels X 2 i +1,2 j +1 as a weighted average interpolation. The line buffer is coupled to the video image interpolating processor to temporarily store the data generated by the video image interpolating processor. The table device is connected to the video image interpolating processor, and uses the look-up table method to perform an instant operation on the exponential function according to the video image interpolating processor output index. The image enhancement post processor is coupled to the line buffer device for image post-processing of the high resolution image.

本發明提供一快速視頻影像內插方法。本發明首先利用一像素提升技術,對輸入低解析度影像進行放大,然後利用一影像強化後處理技術,對放大後之高解析度影像反覆進行修正。The invention provides a fast video image interpolation method. The invention firstly uses a pixel lifting technique to amplify the input low-resolution image, and then uses an image enhancement post-processing technique to repeatedly correct the amplified high-resolution image.

圖1係本發明一種快速高畫質視頻影像內插裝置之方塊圖。該內插裝置100接收一低解析度影像I L 並於該低解析度影像中插入像素而增加其解析度,俾產生一高解析度影像I H (0),該低解析度影像I L 如圖2A所示,係由排列成二維陣列的多數個像素X 0,0X 0,2X 0,4 ...;X 2,0X 2,2X 2,4 ...;X 4,0X 4,2X 4,4 ...所組成,其中包含像素X 2 i ,2 j X 2 i ,2 j +2X 2 i +2,2 j +2X 2 i +2,2 j i ,j 為正整數,而經插入像素所產生之高解析度影像I H (0)如圖2B所示,則成為由排列成二維陣列的多數個像素X 0,0X 0,1X 0,2 ...;X 1,0X 1,1X 1,2 ...;X 2,0X 2,1X 2,2 ...;X 3,0X 3,1X 3,2 ...;X 4,0X 4,1X 4,2 ...所組成,該影像內插裝置包含一視頻影像內插處理器110、一線緩衝器120、一表格裝置130、及一影像強化後處理器140。1 is a block diagram of a fast high definition video image interpolation device of the present invention. The interpolation device 100 receives a low-resolution image I L and inserts pixels into the low-resolution image to increase the resolution thereof, and generates a high-resolution image I H (0), and the low-resolution image I L 2A is a plurality of pixels X 0,0 , X 0,2 , X 0,4 ... ; X 2,0 , X 2,2 , X 2,4 . . arranged in a two-dimensional array. .; X 4,0, X 4,2, X 4,4 ... , where the contents including the pixel X 2 i, 2 j, X 2 i, 2 j +2, X 2 i +2,2 j + 2 and X 2 i +2 , 2 j , i , j are positive integers, and the high-resolution image I H (0) generated by the inserted pixels is as shown in FIG. 2B, and becomes a majority arranged by a two-dimensional array. Pixels X 0,0 , X 0,1 , X 0,2 ... ;X 1,0 , X 1,1 , X 1,2 ... ;X 2,0 , X 2,1 , X 2 , 2 ... ; X 3,0 , X 3,1 , X 3,2 ... ;X 4,0 , X 4,1 , X 4,2 ..., the image interpolation device includes A video image interpolating processor 110, a line buffer 120, a table device 130, and an image enhancement post processor 140.

該視頻影像內插處理器110接收該低解析度影像I L ,並使用一非線性指數函數(exponential function)及一斜向空白像素X 2 i +1,2 j +1 其鄰近的向左上、向右上、向右下及向左下四個方向之絕對像素差,計算出該斜向空白像素X 2 i +1,2 j +1 及鄰近像素之權值,並根據該權值及該斜向空白像素X 2 i +1,2 j +1 鄰近之像素值,對該斜向空白像素X 2 i +1,2 j +1 作加權平均內插。The interpolation within video image processor 110 receives the low-resolution image I L, and the use of a non-linear exponential function (exponential function), and a diagonal blank pixels X 2 i +1,2 j +1 adjacent to the left, upper right, lower right and lower left of absolute pixel differences of the four directions calculated 2 i +1,2 j +1 and right adjacent pixel value to the slant of blank pixels X, and based on the weight and slant of the 2 i +1,2 j +1 pixel values of neighboring pixels of the blank X, the oblique blank pixels X 2 i +1,2 j +1 as a weighted average interpolation.

該線緩衝器120連接至該視頻影像內插處理器110,以暫存該視頻影像內插處理器110所產生之資料。該表格裝置130連接至該視頻影像內插處理器110,依據視頻影像內插處理器110輸出之指標,利用查表方式,以對該指數函數作即時運算。該影像強化後處理器140連接至該線緩衝裝置120,以對該高解析度影像I H (0)進行影像強化後處理。The line buffer 120 is coupled to the video image interpolating processor 110 to temporarily store the data generated by the video image interpolating processor 110. The table device 130 is connected to the video image interpolating processor 110, and performs an instant operation on the exponential function by using a look-up table according to the index output by the video image interpolating processor 110. The image enhancement processor 140 is coupled to the line buffer device 120 to perform image enhancement post processing on the high resolution image I H (0).

本發明所使用之像素提升技術是利用一非線性指數函數(exponential function)及鄰近像素間之絕對差,計算出空白像素向鄰近四方已知像素之權值,並跟據該權值及其鄰近已知之像素值,對該空白像素作加權平均內插。為使內插過程更有效率,本內插法首先對斜向之空白像素進行內插,然後跟據其內插結果及原始圖像之像素,對餘下之空白像素進行內插。The pixel lifting technique used in the present invention uses a nonlinear exponential function and an absolute difference between adjacent pixels to calculate the weight of a blank pixel to a known pixel of the neighboring square, and according to the weight and its neighboring Known pixel values, weighted average interpolation of the blank pixels. In order to make the interpolation process more efficient, the interpolation method first interpolates the oblique blank pixels, and then interpolates the remaining blank pixels according to the interpolation result and the pixels of the original image.

圖2係本發明一種快速高畫質視頻影像內插法的流程圖。該影像內插法係使用於一倍頻器(SCALER)中,以於一低解析度影像I L 中插入像素而增加該低解析度影像的解析度,俾產生一高解析度影像I H (0),該低解析度影像I L 係由排列成二維陣列的多數個像素所組成,其中包含像素X 2 i ,2 j X 2 i ,2 j +2X 2 i +2,2 j +2X 2 i +2,2 j i ,j 為正整數。2 is a flow chart of a fast high quality video image interpolation method of the present invention. The image interpolation method is used in a single frequency multiplier (SCALER) to insert a pixel into a low resolution image I L to increase the resolution of the low resolution image, and generate a high resolution image I H ( 0), the low-resolution image I L lines arranged in a two-dimensional array consisting of a plurality of pixels, wherein the pixel comprises X 2 i, 2 j, X 2 i, 2 j +2, X 2 i +2,2 j +2 and X 2 i +2,2 j, i, j is a positive integer.

一併參照圖4係本發明一高解析度影像I H (0)中一斜向空白像素X 2 i +1,2 j +1 及其鄰近已知像素的示意圖。在本內插法中,該斜向空白像素X 2 i +1,2 j +1 的內插值為其鄰近四角已知像素X 2 i ,2 j X 2 i ,2 j +2X 2 i +2,2 j +2X 2 i + 2,2 j 加權平均內插後之所得其中,該像素X 2 i ,2 j X 2 i ,2 j +2X 2 i +2,2 j +2X 2 i +2,2 j 係為低解析度影像I L 中已知的像素。Referring also to FIG. 4 The present invention is a high resolution image I H (0) in a diagonal blank pixels X 2 i +1,2 j +1 Schematic known neighboring pixels. In this interpolation, the oblique blank pixels X 2 i +1,2 j +1 adjacent the four corners of the known pixel interpolation, X 2 i, 2 j, X 2 i, 2 j +2, X 2 i +2,2 j +2 and X 2 i + 2,2 j are obtained after weighted average interpolation, wherein the pixels X 2 i , 2 j , X 2 i , 2 j +2 , X 2 i +2, 2 j +2 and X 2 i +2,2 j-based low-resolution image I L is known in the pixel.

首先於步驟(A)中,計算該空白像素X 2 i + 1,2 j +1 其鄰近四角已知像素方向之絕對像素差,即向左上絕對像素差D UL 、向右上絕對像素差D UR 、向右下絕對像素差D DR 及向左下絕對像素差D DL 。其算式分別為:First, in step (A), the absolute pixel difference of the adjacent pixel direction of the blank pixel X 2 i + 1, 2 j +1 is calculated, that is, the upper left absolute pixel difference D UL and the upper right absolute pixel difference D UR , the absolute pixel difference D DR to the lower right and the absolute pixel difference D DL to the left. The formulas are:

D UL =|X 2 i ,2 j -X 2 i +2,2 j +2 |+|X 2 i ,2 j -2 -X 2 i +2,2 j |+|X 2 i -2,2 j -X 2 i ,2 j +2 |、 D UL =| X 2 i ,2 j - X 2 i +2,2 j +2 |+| X 2 i ,2 j -2 - X 2 i +2,2 j |+| X 2 i -2, 2 j - X 2 i , 2 j +2 |,

D UR =|X 2 i ,2 j +2 -X 2 i +2,2 j |+|X 2 i -2,2 j +2 -X 2 i ,2 j |+|X 2 i ,2 j +4 -X 2 i +2,2 j +2 |、 D UR =| X 2 i ,2 j +2 - X 2 i +2,2 j |+| X 2 i -2,2 j +2 - X 2 i ,2 j |+| X 2 i ,2 j +4 - X 2 i +2,2 j +2 |,

D DR =|X 2 i ,2 j -X 2 i +2,2 j +2 |+|X 2 i +2,2 j -X 2 i +4,2 j +2 |+|X 2 i ,2 j +2 -X 2 i +2,2 j +4 |、 D DR = | X 2 i, 2 j - X 2 i +2,2 j +2 | + | X 2 i +2,2 j - X 2 i +4,2 j +2 | + | X 2 i, 2 j +2 - X 2 i +2,2 j +4 |,

D DL =|X 2 i ,2 j +2 -X 2 i +2,2 j |+|X 2 i ,2 j -X 2 i +2,2 j -2 |+|X 2 i +2,2 j +2 -X 2 i +4,2 j |。 D DL =| X 2 i ,2 j +2 - X 2 i +2,2 j |+| X 2 i ,2 j - X 2 i +2,2 j -2 |+| X 2 i +2, 2 j +2 - X 2 i +4,2 j |.

於步驟(B)中,依據步驟(A)中的絕對像素差及一非線性函數以計算該斜向空白像素X 2 i +1,2 j +1 的向左上、向右上、向右下及向左下四個方向之權重。其中,該步驟(B)中之非線性函數係一指數函數。In step (B), a step based on the absolute pixel difference (A), and a nonlinear function to calculate a diagonal blank pixels X 2 i +1,2 j +1 upper left, upper right, lower right, and The weight to the left four directions. Wherein, the nonlinear function in the step (B) is an exponential function.

亦即,依據步驟(A)之計算結果,利用一指數函數,計算該空白像素X 2 i +1,2 j +1 其鄰近四角已知像素方向之權重,即向左上權重W UL 、向右上權重W UR 、向右下權重W DR 及向左下權重W DL 。其算式分別為That is, according to calculation step (A) of the results, by using an exponential function, calculates weights of the white pixel X direction pixel adjacent corners of known weight 2 i +1,2 j +1, i.e., to the left the weighting W UL, rightward The weight W UR , the right lower weight W DR and the lower left weight W DL . The formulas are

W UL =exp(-D UL /h )、 W UL =exp(- D UL / h ),

W UR =exp(-D UR /h )、 W UR =exp(- D UR / h ),

W DR =exp(-D DR /h )、 W DR =exp(- D DR / h ),

W DL =exp(-D DL /h ), W DL =exp(- D DL / h ),

當中,h 為一調節參數,0<h ,於本較佳實施例中,h 之值為48。Where h is an adjustment parameter, 0 < h , in the preferred embodiment, the value of h is 48.

於步驟(C)中,依據步驟(B)中的權重以計算該斜向空白像素之值。該斜向空白像素X 2 i +1,2 j +1 的內插值為:In step (C), the value of the oblique blank pixel is calculated according to the weight in step (B). The interpolated pixel X oblique blank 2 i +1,2 j +1 is:

從上述步驟可知,權重與像素差之間是成非線性反比關係,故鄰近已知之像素會因其對應方向之絕對像素差變大,而使其自己參與該內插之比重降低,反之亦然。It can be seen from the above steps that the weight and the pixel difference are nonlinearly inversely proportional, so that the adjacent pixel will become larger due to the absolute pixel difference of its corresponding direction, and the proportion of its own participation in the interpolation is reduced, and vice versa. .

因此,本發明之內插法能自動沿著影像之邊緣對斜向之空白像素作內插,令影像之圖案紋理在影像放大後仍能保持清晰完整。另外,也由於使用非線性反比函數作權重計算之原故,本發明之內插法所輸出之高解析度影像會比傳統利用線性反比函數之加權平均內插法所輸出之高解析度影像更清晰銳利。Therefore, the interpolation method of the present invention can automatically interpolate the oblique blank pixels along the edge of the image, so that the image texture of the image can remain clear and complete after the image is enlarged. In addition, because of the use of the nonlinear inverse ratio function for weight calculation, the high resolution image output by the interpolation method of the present invention is clearer than the high resolution image output by the conventional weighted average interpolation method using the linear inverse ratio function. sharp.

依據前述步驟(A)至步驟(C),對所有斜向之空白像素執行內插運算後,本發明的內插法接著對餘下之空白像素進行內插。圖5係本發明一高解析度影像I H (0)中一每個餘下之空白像素X 2 i ,2 j +1 及其鄰近已知像素的示意圖。如圖5所示,該高解析度影像I H (0)經過步驟(A)至步驟(C),而對所有斜向之空白像素X 2 i +1,2 j +1 執行內插運算後的像素之排列。如圖5所示,每個餘下之空白像素X 2 i ,2 j +1 均已被四個鄰近已知之像素從上下左右四個方向所包圍。為免邊緣在影像放大後出現拉鍊效應(zipper effect),本發明之內差法會首先跟據鄰近地區水平及垂直方向之絕對像素差,決定該餘下之空白像素X 2 i ,2 j +1 之內插方向,然後利用權平均內插法,沿著該已決定的方向,對該餘下之空白像素X 2 i ,2 j +1 進行內插。According to the foregoing steps (A) to (C), after the interpolation operation is performed on all the oblique blank pixels, the interpolation method of the present invention then interpolates the remaining blank pixels. FIG. 5 is a schematic diagram of each of the remaining blank pixels X 2 i , 2 j +1 and its neighboring known pixels in a high-resolution image I H (0) of the present invention. As shown in the high-resolution image I H (0) through steps (A) to step (C), the sum of all the diagonal-blank pixel interpolation operation performed X 2 i +1,2 j +1 5 The arrangement of the pixels. As shown in FIG. 5, each of the remaining blank pixels X 2 i , 2 j +1 has been surrounded by four adjacent known pixels from four directions of up, down, left, and right. In order to avoid the zipper effect after the image is enlarged, the internal difference method of the present invention first determines the remaining blank pixels X 2 i , 2 j +1 according to the absolute pixel difference between the horizontal and vertical directions of the adjacent area. The interpolation direction is then interpolated by the weighted average interpolation method along the determined direction to the remaining blank pixels X 2 i , 2 j +1 .

於步驟(D)中,計算該餘下空白像素X 2 i , 2 j +1 其鄰近四角已知像素方向之絕對像素差。其中,該步驟(D)中係使用下列公式,以計算該餘下空白像素X 2 i ,2 j +1 其鄰近的向上絕對像素差D U 、向下絕對像素差D D 、向左絕對像素差D L 及向右絕對像素差D R In step (D), the absolute pixel difference of the remaining blank pixels X 2 i , 2 j +1 in the direction of the known pixel adjacent to the four corners is calculated. Wherein, in the step (D), the following formula is used to calculate the adjacent absolute pixel difference D U , the downward absolute pixel difference D D , and the absolute left pixel difference of the remaining blank pixels X 2 i , 2 j +1 D L and absolute right pixel difference D R :

D U =|X 2 i -1,2 j +1 -X 2 i +1,2 j +1 |+|X 2 i -2,2 j -X 2 i ,2 j |+|X 2 i -2,2 j +2 -X 2 i ,2 j +2 |、 D U =| X 2 i -1,2 j +1 - X 2 i +1,2 j +1 |+| X 2 i -2,2 j - X 2 i ,2 j |+| X 2 i - 2,2 j +2 - X 2 i ,2 j +2 |,

D D =|X 2 i -1,2 j +1 -X 2 i +1,2 j + 1 |+|X 2 i ,2 j -X 2 i +2,2 j |+|X 2 i ,2 j +2 -X 2 i +2,2 j +2 |、 D D =| X 2 i -1,2 j +1 - X 2 i +1,2 j + 1 |+| X 2 i ,2 j - X 2 i +2,2 j |+| X 2 i , 2 j +2 - X 2 i +2,2 j +2 |,

D L =|X 2 i ,2 j -X 2 i ,2 j +2 |+|X 2 i -1,2 j -1 -X 2 i -1,2 j +1 |+|X 2 i +1,2 j -1 -X 2 i +1,2 j +1 |、 D L =| X 2 i ,2 j - X 2 i ,2 j +2 |+| X 2 i -1,2 j -1 - X 2 i -1,2 j +1 |+| X 2 i + 1,2 j -1 - X 2 i +1,2 j +1 |,

D R =|X 2 i ,2 j -X 2 i ,2 j +2 |+|X 2 i -1,2 j +1 -X 2 i -1,2 j +3 |+|X 2 i +1,2 j +1 -X 2 i +1,2 j +3 |, D R =| X 2 i ,2 j - X 2 i ,2 j +2 |+| X 2 i -1,2 j +1 - X 2 i -1,2 j +3 |+| X 2 i + 1,2 j +1 - X 2 i +1,2 j +3 |,

當中,X 2 i - 1,2 j +1X 2 i +1,2 j -1X 2 i -1,2 j -1X 2 i +1,2 j +1X 2 i -1,2 j +3X 2 i +1,2 j +3 係該步驟(C)中所算出該斜向空白像素之值。 Among, X 2 i - 1,2 j +1 , X 2 i +1,2 j -1, X 2 i -1,2 j -1, X 2 i +1,2 j +1, X 2 i - 1,2 j +3, X 2 i +1,2 j +3 lines the step (C), the calculated value of the diagonal of the blank pixels.

於步驟(E)中,依據步驟(D)中的絕對像素差及該非線性函數以計算該餘下空白像素X 2 i ,2 j +1 之向上權重W U 、向下權重W D 、向左權重W L 及向右權重W R 。其算式分別為:In step (E), the absolute weight W U , the downward weight W D , and the left weight are calculated according to the absolute pixel difference in the step (D) and the nonlinear function to calculate the remaining blank pixels X 2 i , 2 j +1 W L and rightward weight W R . The formulas are:

W U =exp(-D U /h )、 W U =exp(- D U / h ),

W D =exp(-D D /h )、 W D =exp(- D D / h ),

W L =exp(-D L /h )、 W L =exp(- D L / h ),

W R =exp(-D R /h ), W R =exp(- D R / h ),

當中,h 為前述之調節參數。Where h is the aforementioned adjustment parameter.

於步驟(F)中,計算該餘下空白像素X 2 i ,2 j +1 的水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV )。其中,該步驟(F)中係使用下列公式,以計算該餘下空白像素X 2 i ,2 j +1 的水平方向絕對像素差(ΔH )及垂直方向之絕對像素差(ΔV ):In step (F), the calculation of the remaining blank pixels X 2 i, 2 j +1 in the horizontal direction of the absolute pixel differences (Δ H) and the vertical direction of the absolute pixel differences (Δ V). Wherein the step (F) in the system using the following formula to calculate the remaining blank pixels X 2 i, 2 j +1 in the horizontal direction of the absolute pixel differences (Δ H) and the vertical direction of the absolute pixel differences (Δ V):

於步驟(G)中,依據步驟(F)中的水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV )以計算該餘下空白像素X 2 i ,2 j + 1 之值。其中,該步驟(G)中係比較水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV )之數值,決定該餘下空白像素X 2 i ,2 j +1 之內插方向及其內插值。In the step (G), the values of the remaining blank pixels X 2 i , 2 j + 1 are calculated according to the horizontal absolute pixel difference (Δ H ) and the vertical absolute pixel difference (Δ V ) in the step (F). Wherein the step (G) in Comparative horizontal line direction of the absolute pixel differences (Δ H) and the vertical direction of the absolute pixel differences (Δ V) of the value, determining the remaining blank pixels X 2 i, 2 j +1 interpolation direction and the Its interpolation value.

當水平方向絕對像素差(ΔH )大於垂直方向絕對像素差(ΔV ),則該餘下空白像素X 2 i ,2 j +1 以垂直方向作內插,其內插值為:When the horizontal direction of the absolute pixel differences (Δ H) is greater than the vertical direction of the absolute pixel differences (Δ V), the remaining blank pixels X 2 i, 2 j +1 in the vertical direction for interpolation, which is interpolated:

當水平方向絕對像素差(ΔH )小於垂直方向絕對像素差(ΔV ),則該餘下空白像素X 2 i ,2 j + 1 以水平方向作內插 其內插值為:When the horizontal direction of the absolute pixel differences (Δ H) is smaller than the vertical direction of the absolute pixel differences (Δ V), the remaining blank pixels X 2 i, 2 j + 1 in the horizontal direction for interpolation, which is interpolated:

當水平方向絕對像素差(ΔH )等於垂直方向絕對像素差(ΔV ),則該餘下空白像素X 2 i ,2 j +1 之內插值為:When the horizontal direction absolute pixel differences (Δ H) is equal to absolute vertical pixel differences (Δ V), the remaining blank pixels X 2 i, 2 j +1 within the interpolation is:

重複執行該步驟(A)至步驟(G),則可產生該高解析度影像I H (0)。為了達到實際應用時低運算量之要求,本發明避免對該指數函數作即時運算,取而代之為利用一查表法來獲得內插時所需要的權重值。By repeating this step (A) to step (G), the high-resolution image I H (0) can be generated. In order to achieve the low computational complexity in practical applications, the present invention avoids performing an instant operation on the exponential function, and instead uses a look-up table method to obtain the weight values required for interpolation.

以一8位元(8-bit)影像為例,從上述步驟(A)及步驟(D)中,可得知每個絕對像素差的有效範圍為0-765,故本發明只需利用一個766欄位的該表格裝置130作查詢,便可在不需要運算量的情形下即時獲得所需要之權重,其中查找該表格裝置130中每一欄位為exp(-k /h )之運算結果,k 為0,1,2,...,765。Taking an 8-bit image as an example, from the above steps (A) and (D), it can be known that the effective range of each absolute pixel difference is 0-765, so the present invention only needs to utilize one The table device 130 of the 766 field makes a query, so that the required weight can be obtained instantly without the need of an amount of calculation, wherein the operation result of each field in the table device 130 is exp( -k / h ). , k is 0, 1, 2, ..., 765.

另外,若輸入影像為一16位元(16-bit)影像,本發明先將從步驟(A)及步驟(D)中所獲得的絕對像素差除以256,並將其結果四捨五入至整數,然後利用該整數作索引,從該表格裝置130找出所需要之權重值作內插。In addition, if the input image is a 16-bit (16-bit) image, the present invention first divides the absolute pixel difference obtained in steps (A) and (D) by 256, and rounds the result to an integer. The integer is then used as an index to find the desired weight value from the table device 130 for interpolation.

依據以上步驟(D)至步驟(G)對所有餘下之空白像素內插後,所有高解析度影像I H (0)中空白之像素均已被獲得,整個像素提升過程亦因此而完成。After all the remaining blank pixels are interpolated according to the above steps (D) to (G), all the pixels in the high-resolution image I H (0) have been obtained, and the entire pixel lifting process is completed.

接著,本發明執行步驟(H),以對該放大後高解析度影像執行一強化後處理,本發明之強化後處理技術反覆進行修正,以增強輸出影像之清晰度及像真度。Next, the present invention performs step (H) to perform a post-enhancement processing on the enlarged high-resolution image, and the enhanced post-processing technique of the present invention repeatedly corrects to enhance the sharpness and image quality of the output image.

本發明之強化後處理技術是一種疊代的(iterative)影像修正技術,它反覆地利用放大後高解析度影像模擬出來之低解析影像與原始輸入低解析度影像之間的誤差,對高解析度影像進行修正。The enhanced post-processing technique of the present invention is an iterative image correction technique, which repeatedly uses the error between the low-resolution image simulated by the amplified high-resolution image and the original input low-resolution image, and the high resolution The image is corrected.

圖6係本發明所使用之影像強化後處理技術之運作原理之示意圖。為方便說明,本發明以I L 代表原始輸入之低解析度影像,I H (n )代表第n次修正後之高解析度影像輸出,其中n =0,1,2,...,NN 為疊代總次數。在本方法中,為降低運算量,N 之預設值為5。圖7係本發明步驟(H)的詳細流程圖。Fig. 6 is a schematic view showing the operation principle of the image enhancement post-processing technique used in the present invention. For convenience of description, the present invention uses I L to represent the low-resolution image of the original input, and I H ( n ) represents the high-resolution image output after the nth correction, where n =0, 1, 2, ..., N , N is the total number of iterations. In the method, in order to reduce the amount of calculation, the preset value of N is 5. Figure 7 is a detailed flow diagram of step (H) of the present invention.

於步驟(H1)中,設定一初始高解析度影像I H (0),其中I H (0)為前述步驟(A)至步驟(G)輸出的高解析度影像。In the step (H1), an initial high-resolution image I H (0) is set, wherein I H (0) is the high-resolution image outputted from the foregoing steps (A) to (G).

於步驟(H2)中,計算該低解析度影像I L 與一由該高解析度影像I H (n )模擬出來之一模擬低解析影像之間的誤差e ,當中,I H (n )為第n 次疊代時的高解析度影像。In step (H2), an error e between the low-resolution image I L and a simulated low-resolution image simulated by the high-resolution image I H ( n ) is calculated, where I H ( n ) is High resolution image at the nth iteration.

其中,步驟(H2)係利用下列公式,以計算該誤差eWherein, step (H2) uses the following formula to calculate the error e :

e =I L -(f *I H (n ))↓2, e = I L -( f * I H ( n ))↓2,

當中,(‧)↓2為降冪取樣(down-sampling)程序,f 為一2×2平均濾波器(averaging filter)。Among them, (‧) ↓ 2 is a down-sampling program, and f is a 2×2 averaging filter.

於步驟(H3)中,依據該誤差e ,對高解析度影像I H (n )進行修正,並產生一修正高解析度影像I H (n +1)。其中,步驟(H3)係利用下列公式,以對該高解析度影像I H (n )進行修正,並產生該修正高解析度影像I H (n +1):In step (H3), the high-resolution image I H ( n ) is corrected according to the error e , and a corrected high-resolution image I H ( n +1) is generated. Wherein, the step (H3) uses the following formula to correct the high-resolution image I H ( n ) and generate the corrected high-resolution image I H ( n +1):

I H (n +1)=I H (n )+g *(e )↑2, I H ( n +1)= I H ( n )+ g *( e )↑2,

當中,運算函數↑2為零次升冪取樣(zero order up-sampling)程序,g 為一5×5高斯濾波器(Gaussian filter),其數值為:Among them, the operation function ↑2 is a zero order up-sampling program, and g is a 5×5 Gaussian filter whose values are:

於步驟(H4)中,重覆步驟(H1)至步驟(H3)直至一預定運算次數。於本實施例中,該預定運算次數為5。In the step (H4), the step (H1) to the step (H3) are repeated until a predetermined number of operations. In this embodiment, the predetermined number of operations is five.

隨著本發明之強化後處理步驟完成,整個快速高畫質視頻影像內插過程亦告完結。With the completion of the enhanced post-processing steps of the present invention, the entire fast high-definition video image interpolation process is also completed.

由前述說明可知,本發明揭示一種快速視頻影像內插方法,其方法包括一像素提升技術及一影像強化後處理技術。該像素提升技術首先對斜向之空白像素進行內插,然後依據其內插結果及原始圖像之像素值,對餘下之空白像素進行內插。它利用一非線性函數及鄰近像素間之絕對差,計算出對一空白像素鄰近四方已知像素之權值,並根據該權值及其鄰近已知之像素值,利用加權平均內插法,依著影像圖案之紋理計算出該空白像素之內插值。該影像強化後處理技術則反覆地利用原始輸入低解析度影像與內插後高解析度影像模擬出來之低解析影像之間的誤差,對高解析度影像進行修正,令輸出影像更清晰、自然及像真。It can be seen from the foregoing description that the present invention discloses a fast video image interpolation method, which includes a pixel lifting technique and an image enhancement post processing technique. The pixel lifting technique first interpolates the oblique blank pixels, and then interpolates the remaining blank pixels according to the interpolation result and the pixel values of the original image. It uses a nonlinear function and the absolute difference between adjacent pixels to calculate the weight of a known pixel adjacent to a blank pixel, and uses the weighted average interpolation method according to the weight and its neighboring known pixel values. The texture of the image pattern calculates the interpolation value of the blank pixel. The image enhancement post-processing technique repeatedly uses the error between the original input low-resolution image and the low-resolution image simulated by the interpolated high-resolution image to correct the high-resolution image to make the output image clearer and more natural. And like truth.

為了達到實際應用時高畫質低運算量的要求,本發明之內插法一改其傳統純內插的設計模式,以先內插後強化的概念為大綱,並在這大綱下,對每個技術部分再以簡單高效能的設計為目標。故跟現有的內插法相比,本內插法不單能提供即時的影像放大,而且還提供一個比較清晰及自然之影像輸出,達到高畫質低運算量之要求。In order to achieve the requirements of high image quality and low computational complexity in practical applications, the interpolation method of the present invention changes its traditional purely interpolated design mode, and the concept of post-interpolation and post-enhancement is taken as an outline, and under this outline, for each The technical part is aimed at simple and efficient design. Therefore, compared with the existing interpolation method, the interpolation method not only provides instant image enlargement, but also provides a clear and natural image output, achieving high image quality and low computational complexity.

由上述可知,本發明無論就目的、手段及功效,在在均顯示其迥異於習知技術之特徵,極具實用價值。惟應注意的是,上述諸多實施例僅係為了便於說明而舉例而已,本發明所主張之權利範圍自應以申請專利範圍所述為準,而非僅限於上述實施例。From the above, it can be seen that the present invention is extremely useful in terms of its purpose, means, and efficacy, both of which are different from those of the prior art. It should be noted that the various embodiments described above are merely illustrative for ease of explanation, and the scope of the invention is intended to be limited by the scope of the claims.

100...內插裝置100. . . Interpolation device

110...視頻影像內插處理器110. . . Video image interpolation processor

120...線緩衝器120. . . Line buffer

130...表格裝置130. . . Table device

140...影像強化後處理器140. . . Image enhancement processor

步驟(A)~步驟(H)Step (A) ~ Step (H)

步驟(H1)~步驟(H4)Step (H1)~Step (H4)

圖1係本發明一種快速高畫質視頻影像內插裝置之方塊圖。1 is a block diagram of a fast high definition video image interpolation device of the present invention.

圖2A本發明低解析度影像的示意圖。2A is a schematic diagram of a low resolution image of the present invention.

圖2B本發明高解析度影像的示意圖。2B is a schematic diagram of a high resolution image of the present invention.

圖3係本發明一種快速高畫質視頻影像內插法的流程圖。3 is a flow chart of a fast high quality video image interpolation method of the present invention.

圖4係本發明一高解析度影像中一斜向空白像素及其鄰近已知像素的示意圖。4 is a schematic diagram of an oblique blank pixel and its adjacent known pixels in a high resolution image of the present invention.

圖5係本發明一高解析度影像中一餘下之空白像素及其鄰近已知像素的示意圖。FIG. 5 is a schematic diagram of a remaining blank pixel and its neighboring known pixels in a high resolution image of the present invention.

圖6係本發明所使用之影像強化後處理技術之運作原理之示意圖。Fig. 6 is a schematic view showing the operation principle of the image enhancement post-processing technique used in the present invention.

圖7係本發明快速高畫質視頻影像內插法之影像強化後處理的詳細流程圖。FIG. 7 is a detailed flowchart of image post-enhancement processing of the fast high-definition video image interpolation method of the present invention.

100...內插裝置100. . . Interpolation device

110...視頻影像內插處理器110. . . Video image interpolation processor

120...線緩衝器120. . . Line buffer

130...表格裝置130. . . Table device

140...影像強化後處理器140. . . Image enhancement processor

Claims (12)

一種快速高畫質視頻影像內插法,係使用於一倍頻器(SCALER)中,以於一低解析度影像中插入像素而增加該低解析度影像的解析度,俾產生一高解析度影像,該低解析度影像係由排列成二維陣列的多數個像素所組成,其中包含像素X 2 i ,2 j X 2 i ,2 j +2X 2 i +2,2 j +2X 2 i +2,2 j i ,j 為正整數,該影像內插法包含下列步驟:(A) 計算一斜向空白像素X 2 i +1,2 j +1 其鄰近的向左上、向右上、向右下及向左下四個方向之絕對像素差;(B) 依據步驟(A)中的絕對像素差及一非線性函數以計算該斜向空白像素X 2 i +1,2 j +1 的向左上、向右上、向右下及向左下四個方向之權重;(C) 依據步驟(B)中的權重以計算該斜向空白像素之值;(D) 計算一餘下空白像素X 2 i ,2 j +1 其鄰近四角已知像素方向之絕對像素差;(E) 依據步驟(D)中的絕對像素差及該非線性函數以計算該餘下空白像素X 2 i ,2 j +1 之向上權重W U 、向下權重W D 、向左權重W L 及向右權重W R ;(F) 計算該餘下空白像素X 2 i ,2 j +1 的水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV );以及(G) 依據步驟(F)中的水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV )以計算該餘下空白像素X 2 i ,2 j +1 之值。A fast high-definition video image interpolation method used in a single frequency multiplier (SCALER) to insert pixels in a low-resolution image to increase the resolution of the low-resolution image, thereby generating a high resolution image, the low-resolution image of a plurality of pixel lines arranged in the two-dimensional array, where the pixels comprising X 2 i, 2 j, X 2 i, 2 j +2, X 2 i +2,2 j +2 and X 2 i +2,2 j, i, j is a positive integer, the image interpolation method comprising the steps of: (a) calculated on a diagonal blank pixels X 2 i +1,2 j +1 adjacent leftward Absolute pixel difference in the four directions of the upper right, the lower right, and the lower left; (B) calculating the oblique blank pixel X 2 i +1, 2 according to the absolute pixel difference in step (A) and a nonlinear function j +1 the weights of the upper left, upper right, lower right, and lower left directions; (C) calculate the value of the oblique blank pixel according to the weight in step (B); (D) calculate a remaining blank The pixels X 2 i , 2 j +1 are absolute pixel differences of the known pixel directions adjacent to the four corners; (E) calculating the remaining blank pixels X according to the absolute pixel difference in the step (D) and the nonlinear function 2 i , 2 j +1 up weight W U , down weight W D , left weight W L and right weight W R ; (F) calculating the horizontal direction of the remaining blank pixels X 2 i , 2 j +1 Absolute pixel difference (Δ H ) and vertical absolute pixel difference (Δ V ); and (G) calculated according to the horizontal absolute pixel difference (Δ H ) and the vertical absolute pixel difference (Δ V ) in step (F) The value of the remaining blank pixels X 2 i , 2 j +1 . 如申請專利範圍第1項所述之快速高畫質視頻影像內插法,其更包含步驟:(H) 對該高解析度影像執行影像強化後處理。The fast high-definition video image interpolation method according to claim 1, further comprising the step of: (H) performing image enhancement post-processing on the high-resolution image. 如申請專利範圍第2項所述之快速高畫質視頻影像內插法,其中,該步驟(A)至步驟(H)係重複執行。The fast high-definition video image interpolation method described in claim 2, wherein the steps (A) to (H) are repeatedly performed. 如申請專利範圍第3項所述之快速高畫質視頻影像內插法,其中,該步驟(A)中係使用下列公式,以計算該斜向空白像素X 2 i +1,2 j +1 其鄰近的向左上絕對像素差D UL 、向右上絕對像素差D UR 、向右下絕對像素差D DR 及向左下絕對像素差D DL D UL =|X 2 i ,2 j -X2 i +2,2 j +2 |+|X 2 i ,2 j -2 -X 2 i +2,2 j |+|X 2 i -2,2 j -X 2 i ,2 j +2 |、D UR =|X 2 i ,2 j +2 -X 2 i +2,2 j |+|X 2 i -2,2 j +2 -X 2 i ,2 j |+|X 2 i ,2 j +4 -X 2 i +2,2 j +2 |、D DR =|X 2 i ,2 j -X 2 i +2,2 j +2 |+|X 2 i +2,2 j -X 2 i +4,2 j +2 |+|X 2 i ,2 j +2 -X 2 i +2,2 j +4 |、D DL =|X 2 i ,2 j +2 -X 2 i +2,2 j |+|X 2 i ,2 j -X 2 i +2,2 j -2 |+|X 2 i +2,2 j +2 -X 2 i +4,2 j |。The interpolation fast high quality video images of the item 3 of patent application range, wherein the step (A) is based using the following formula to calculate the diagonal blank pixels X 2 i +1,2 j +1 Its adjacent left upper absolute pixel difference D UL , upper right absolute pixel difference D UR , right lower absolute pixel difference D DR and lower left absolute pixel difference D DL : D UL =| X 2 i , 2 j -X 2 i +2,2 j +2 |+| X 2 i ,2 j -2 - X 2 i +2,2 j |+| X 2 i -2,2 j - X 2 i ,2 j +2 |, D UR =| X 2 i ,2 j +2 - X 2 i +2,2 j |+| X 2 i -2,2 j +2 - X 2 i ,2 j |+| X 2 i ,2 j + 4 - X 2 i +2,2 j +2 |, D DR =| X 2 i ,2 j - X 2 i +2,2 j +2 |+| X 2 i +2,2 j - X 2 i +4,2 j +2 |+| X 2 i ,2 j +2 - X 2 i +2,2 j +4 |, D DL =| X 2 i ,2 j +2 - X 2 i +2, 2 j |+| X 2 i ,2 j - X 2 i +2,2 j -2 |+| X 2 i +2,2 j +2 - X 2 i +4,2 j |. 如申請專利範圍第4項所述之快速高畫質視頻影像內插法,其中,該步驟(B)中之非線性函數係一指數函數,即該斜向空白像素X 2 i +1,2 j +1 的向左上權重W UL 、向右上權重W UR 、向右下權重W DR 及向左下權重W DL 之計算分別為:W UL =exp(-D UL /h )、W UR =exp(-D UR /h )、W DR =exp(-D DR /h )、W DL =exp(-D DL /h ),當中,h 為一調節參數,0<hThe fast high-definition video image interpolation method according to claim 4, wherein the nonlinear function in the step (B) is an exponential function, that is, the oblique blank pixel X 2 i +1, 2 The calculation of the left upper weight W UL , the upper right weight W UR , the right lower weight W DR and the lower left weight W DL of j +1 are respectively: W UL =exp(- D UL / h ), W UR =exp( - D UR / h ), W DR =exp(- D DR / h ), W DL =exp(- D DL / h ), where h is an adjustment parameter, 0 < h . 如申請專利範圍第5項所述之快速高畫質視頻影像內插法,其中,該步驟(C)中係使用下列公式,以計算該斜向空白像素之值X 2 i +1,2 j +1 The fast high-definition video image interpolation method described in claim 5, wherein the following formula is used in the step (C) to calculate the value of the oblique blank pixel X 2 i +1, 2 j +1 : 如申請專利範圍第6項所述之快速高畫質視頻影像內插法,其中,該步驟(D)中係使用下列公式,以計算該餘下空白像素X 2 i ,2 j +1 其鄰近的向上絕對像素差D U 、向下絕對像素差D D 、向左絕對像素差D L 及向右絕對像素差D R D U =|X 2 i - 1,2 j +1 -X 2 i +1,2 j +1 |+|X 2 i - 2,2 j -X 2 i ,2 j |+|X 2 i -2,2 j +2 -X 2 i ,2 j +2 |、D D =|X 2 i -1,2 j +1 -X 2 i + 1,2 j +1 |+|X 2 i ,2 j -X 2 i +2,2 j |+|X 2 i ,2 j +2 -X 2 i +2,2 j +2 |、D L =|X 2 i ,2 j -X 2 i ,2 j +2 |+|X 2 i -1,2 j -1 -X 2 i -1,2 j +1 |+|X 2 i +1,2 j -1 -X 2 i +1,2 j +1 |、D R =|X 2 i ,2 j -X 2 i ,2 j +2 |+|X 2 i -1,2 j +1 -X 2 i -1,2 j +3 |+|X 2 i +1,2 j +1 -X 2 i +1,2 j + 3 |,當中,X 2 i -1,2 j +1X 2 i +1,2 j -1X 2 i -1,2 j -1X 2 i +1,2 j +1X 2 i -1,2 j +3X 2 i +1,2 j +3 係該步驟(C)中所算出該斜向空白像素之值。The fast high-definition video image interpolation method according to claim 6, wherein the step (D) uses the following formula to calculate the remaining blank pixels X 2 i , 2 j +1 adjacent thereto Upward absolute pixel difference D U , downward absolute pixel difference D D , left absolute pixel difference D L and rightward absolute pixel difference D R : D U =| X 2 i - 1,2 j +1 - X 2 i + 1,2 j +1 |+| X 2 i - 2,2 j - X 2 i ,2 j |+| X 2 i -2,2 j +2 - X 2 i ,2 j +2 |, D D =| X 2 i -1,2 j +1 - X 2 i + 1,2 j +1 |+| X 2 i ,2 j - X 2 i +2,2 j |+| X 2 i ,2 j +2 - X 2 i +2,2 j +2 |, D L =| X 2 i ,2 j - X 2 i ,2 j +2 |+| X 2 i -1,2 j - 1 - X 2 i -1,2 j +1 |+| X 2 i +1,2 j -1 - X 2 i +1,2 j +1 |, D R =| X 2 i ,2 j - X 2 i ,2 j +2 |+| X 2 i -1,2 j +1 - X 2 i -1,2 j +3 |+| X 2 i +1,2 j +1 - X 2 i +1,2 j + 3 |, where X 2 i -1,2 j +1 , X 2 i +1,2 j -1 , X 2 i -1,2 j -1 , X 2 i +1,2 j +1 , X 2 i -1,2 j +3, X 2 i +1,2 j +3 lines the step (C), the calculated value of the diagonal of the blank pixels. 如申請專利範圍第7項所述之快速高畫質視頻影像內插法,其中,該步驟(E)中係使用下列公式,以計算該餘下空白像素X 2 i ,2 j +1 的向上權重W U 、向下權重W D 、向左權重W L 及向右權重W R W U =exp(-D U /h )、W D =exp(-D D /h )、W L =exp(-D L /h )、W R =exp(-D R /h ),當中,h 為該調節參數。The fast high-definition video image interpolation method according to claim 7, wherein the following formula is used in the step (E) to calculate the upward weight of the remaining blank pixels X 2 i , 2 j +1 W U , downward weight W D , left weight W L and right weight W R : W U =exp(- D U / h ), W D =exp(- D D / h ), W L =exp( - D L / h ), W R =exp(- D R / h ), where h is the adjustment parameter. 如申請專利範圍第8項所述之快速高畫質視頻影像內插法,其中,該步驟(F)中係使用下列公式,以計算該餘下空白像素X 2 i ,2 j +1 的水平方向絕對像素差(ΔH )及垂直方向之絕對像素差(ΔV ): For example, the fast high-definition video image interpolation method described in claim 8 wherein the step (F) uses the following formula to calculate the horizontal direction of the remaining blank pixels X 2 i , 2 j +1 Absolute pixel difference (Δ H ) and absolute pixel difference (Δ V ) in the vertical direction: 如申請專利範圍第9項所述之快速高畫質視頻影像內插法,其中,該步驟(G)中係比較水平方向絕對像素差(ΔH )及垂直方向絕對像素差(ΔV )之數值,決定該餘下空白像素X 2 i ,2 j +1 之內插方向及其內插值。The patentable scope of application of the nine fast the high quality video image interpolation method, wherein the step (G) in Comparative horizontal line direction of the absolute pixel differences (Δ H) and the vertical direction of the absolute pixel differences (Δ V) of The value determines the interpolation direction of the remaining blank pixels X 2 i , 2 j +1 and its interpolation value. 如申請專利範圍第10項所述之快速高畫質視頻影像內插法,其中,步驟(H)更包含下列步驟:(H1) 設定一初始高解析度影像I H (0);(H2) 計算該低解析度影像I L 與一由該高解析度影像I H (n )模擬出來之一模擬低解析影像之間的誤差e ;(H3) 依據該誤差e ,對高解析度影像I H (n )進行修正,並產生一修正高解析度影像I H (n +1);以及(H4) 重覆步驟(H1)至步驟(H3)直至一預定運算次數。The fast high-definition video image interpolation method according to claim 10, wherein the step (H) further comprises the following steps: (H1) setting an initial high-resolution image I H (0); (H2) Calculating an error e between the low-resolution image I L and a simulated low-resolution image simulated by the high-resolution image I H ( n ); (H3) according to the error e , for the high-resolution image I H ( n ) performing correction and generating a corrected high-resolution image I H ( n +1); and (H4) repeating steps (H1) through (H3) until a predetermined number of operations. 一種快速高畫質視頻影像內插裝置,係接收一低解析度影像並於該低解析度影像中插入像素而增加其解析度,俾產生一高解析度影像,該低解析度影像係由排列成二維陣列的多數個像素所組成,其中包含像素X 2 i ,2 j i ,j 為正整數,該影像內插裝置包含:一視頻影像內插處理器,其接收該低解析度影像,並使用一非線性指數函數(exponential function)及一斜向空白像素X 2 i +1,2 j +1 其鄰近的向左上、向右上、向右下及向左下四個方向之絕對像素差,計算出該斜向空白像素X 2 i + 1,2 j + 1 及鄰近像素之權值,並根據該權值及該斜向空白像素X 2 i +1,2 j +1 鄰近之像素值,對該斜向空白像素X 2 i +1,2 j +1 作加權平均內插;一線緩衝器,連接至該視頻影像內插處理器,以暫存該視頻影像內插處理器所產生之資料;一表格裝置,連接至該視頻影像內插處理器,依據視頻影像內插處理器輸出之指標,利用查表方式,以對該指數函數作即時運算;以及一影像強化後處理器,連接至該線緩衝裝置,以對該高解析度影像進行影像強化後處理。A fast high-definition video image interpolation device receives a low-resolution image and inserts a pixel into the low-resolution image to increase the resolution thereof, thereby generating a high-resolution image, and the low-resolution image is arranged Forming a plurality of pixels in a two-dimensional array, wherein the pixels X 2 i , 2 j , i , j are positive integers, the image interpolation device includes: a video image interpolation processor that receives the low-resolution image and using a non-linear exponential function (exponential function), and a diagonal blank pixels X 2 i +1,2 j +1 on adjacent left, upper right, lower left and lower right absolute four directions of the pixel difference calculated i + 1,2 j + 1 and the weight values of neighboring pixels of the blank pixels obliquely X 2, and 2 i +1,2 j +1 value adjacent to a pixel based on the weight of the inclined gap and the pixel X , the oblique blank pixels X 2 i +1,2 j +1 as a weighted average interpolation; line buffer, coupled to the interpolation processor of the video image to the interpolation processor temporarily storing video images arising Data; a table device connected to the video image interpolation processor, according to the video shadow The interpolation processor output indicators, the use of table lookup to real time of the exponential function computation; and the latter image enhancement processor, coupled to the line buffer means, to the high-resolution images for the image enhancement processing.
TW99127730A 2010-08-19 2010-08-19 Fast and high quality image/video interpolation method and apparatus TWI406187B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW99127730A TWI406187B (en) 2010-08-19 2010-08-19 Fast and high quality image/video interpolation method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW99127730A TWI406187B (en) 2010-08-19 2010-08-19 Fast and high quality image/video interpolation method and apparatus

Publications (2)

Publication Number Publication Date
TW201209755A TW201209755A (en) 2012-03-01
TWI406187B true TWI406187B (en) 2013-08-21

Family

ID=46763748

Family Applications (1)

Application Number Title Priority Date Filing Date
TW99127730A TWI406187B (en) 2010-08-19 2010-08-19 Fast and high quality image/video interpolation method and apparatus

Country Status (1)

Country Link
TW (1) TWI406187B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10402943B2 (en) * 2016-10-20 2019-09-03 Htc Corporation Image enhancement device and method for convolutional network apparatus
CN111200755A (en) * 2018-11-20 2020-05-26 晨星半导体股份有限公司 Image pixel lifting device and method
GB2617220A (en) * 2022-09-30 2023-10-04 Imagination Tech Ltd Upsampling blocks of pixels

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5001563A (en) * 1988-09-27 1991-03-19 U.S. Philips Corporation Method of and circuit for processing a picture signal
US5125042A (en) * 1989-06-16 1992-06-23 Eastman Kodak Company Digital image interpolator using a plurality of interpolation kernals
WO2000060532A1 (en) * 1999-04-01 2000-10-12 University Of Maryland Baltimore County Method and apparatus for restoration of low resolution images
US6133957A (en) * 1997-10-14 2000-10-17 Faroudja Laboratories, Inc. Adaptive diagonal interpolation for image resolution enhancement
WO2005008589A1 (en) * 2003-07-16 2005-01-27 Hewlett-Packard Development Company, L.P. High resolution image reconstruction
TW200514429A (en) * 2003-10-06 2005-04-16 Sunplus Technology Co Ltd Directional interpolation method and device for increasing resolution of an image
TW200513984A (en) * 2003-10-13 2005-04-16 Leadtek Research Inc Fast edge-oriented image interpolation algorithm
EP2045773A1 (en) * 2007-10-02 2009-04-08 Vestel Elektronik Sanayi ve Ticaret A.S. Method and apparatus for changing the spatial resolution of a digital image

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5001563A (en) * 1988-09-27 1991-03-19 U.S. Philips Corporation Method of and circuit for processing a picture signal
US5125042A (en) * 1989-06-16 1992-06-23 Eastman Kodak Company Digital image interpolator using a plurality of interpolation kernals
US6133957A (en) * 1997-10-14 2000-10-17 Faroudja Laboratories, Inc. Adaptive diagonal interpolation for image resolution enhancement
WO2000060532A1 (en) * 1999-04-01 2000-10-12 University Of Maryland Baltimore County Method and apparatus for restoration of low resolution images
WO2005008589A1 (en) * 2003-07-16 2005-01-27 Hewlett-Packard Development Company, L.P. High resolution image reconstruction
TW200514429A (en) * 2003-10-06 2005-04-16 Sunplus Technology Co Ltd Directional interpolation method and device for increasing resolution of an image
TW200513984A (en) * 2003-10-13 2005-04-16 Leadtek Research Inc Fast edge-oriented image interpolation algorithm
EP2045773A1 (en) * 2007-10-02 2009-04-08 Vestel Elektronik Sanayi ve Ticaret A.S. Method and apparatus for changing the spatial resolution of a digital image

Also Published As

Publication number Publication date
TW201209755A (en) 2012-03-01

Similar Documents

Publication Publication Date Title
JP4351394B2 (en) Non-linear and linear methods for enlarging or reducing image resolution conversion
JP6016061B2 (en) Image generation apparatus, image display apparatus, image generation method, and image generation program
US7881563B2 (en) Distortion correction of images using hybrid interpolation technique
CN103500435B (en) A kind of adapting to image interpolation method of edge guiding and VLSI implement device thereof
JP2003274157A (en) Image processor, image processing method, image processing program, and computer readable recording medium with image processing program recorded
CN106780336B (en) Image reduction method and device
KR20070091559A (en) Image processing apparatus, image processing method, recording medium, and program
US8711231B2 (en) Digital image processing device and processing method thereof
KR100860968B1 (en) Image-resolution-improvement apparatus and method
JP4582666B2 (en) Method and apparatus for interpolating and correcting an image and scanner having the apparatus
CN111968033A (en) Image scaling processing method and device
TWI406187B (en) Fast and high quality image/video interpolation method and apparatus
CN109325909B (en) Image amplification method and image amplification device
JP2001028051A (en) Device and method for image processing, and image processing program recording medium
Gour et al. Hardware accelerator for real-time image resizing
JP6532151B2 (en) Super-resolution device and program
US9741095B2 (en) Method for electronic zoom with sub-pixel offset
US20180218477A1 (en) Data interpolation device, method therefor, and image processing apparatus
JP2000187726A (en) Data interpolation method and its device and storage medium
JP5085589B2 (en) Image processing apparatus and method
Zemčík et al. Accelerated image resampling for geometry correction
JP5387289B2 (en) Image processing apparatus, image processing method and program thereof
JP6326914B2 (en) Interpolation apparatus and interpolation method
JP5387288B2 (en) Image processing apparatus, image processing method and program thereof
JP2004536385A (en) Method and apparatus for performing affine or non-affine transformations