KR20230083519A - System for Interpolating Color Image Intelligent and Method for Deinterlacing Using the Same - Google Patents

Abstract

An intelligent color image restoration system and a deinterlacing method using the same apply different fuzzy weights to each edge direction using a fuzzy concept, multiply the weight value by an interpolation target pixel value, and interpolate to a final deinterlaced pixel, thereby interpolating good quality By selectively applying an image interpolation method while obtaining an image, there is an effect of improving objective and subjective image quality and algorithm calculation speed.

Description

Intelligent color image restoration system and deinterlacing method using the same {System for Interpolating Color Image Intelligent and Method for Deinterlacing Using the Same}

The present invention relates to a color image reconstruction system, and more particularly, to reflect different fuzzy weights differently for each edge direction using a fuzzy concept, and to interpolate a final deinterlaced pixel by multiplying a weight value by an interpolation target pixel value. It relates to an intelligent color image restoration system and a deinterlacing method using the same.

While most image sources (DVD, SDTV, HDTV) transmit interlaced scanning images, receiver displays such as DLP, LCD, LCOS (SXRX, D-ILA), and PDP use interlaced scanning. Since an image is reproduced using a progressive scanning method, de-interlacing technology is being developed to solve this problem.

Deinterlacing is the process of reworking an interlaced video signal into a non-interlaced form. That is, deinterlacing represents a process of changing an interlaced (interlaced scanning) method image into a progressive (sequential scanning) method.

The deinterlacing technology is a scanning format conversion technology that converts an image input by forward scan into a forward scan image and outputs it. A representative example of the use of deinterlacing technology is a case of viewing a television signal, which is a public wave broadcast, on a computer. This deinterlacing technique makes one complete frame by predicting missing pixel values in one field. Deinterlacing techniques can be largely classified into two types: inter-field interpolation and intra-field interpolation.

An example of interpolation between fields is Weave, and one frame is created by combining a top-field and a bottom-field. That is, one frame is implemented by simply inserting the lines of the previous field between the lines of the current field. This can be easily implemented when interpolating an image without motion compensation, but when interpolating an image with motion compensation, there is a problem in that horizontal lines are generated or the screen is deteriorated.

Intra-field interpolation, commonly known as bob, uses each line of a field twice to make one frame. In other words, a new frame is implemented by inserting average data of two lines into an area between two lines in one field. This intra-field interpolation method prevents horizontal lines from appearing in motion-compensated images. However, when interpolating still images, the screen deteriorates, and complex and fine screens vibrate up and down at 30Hz.

Korean Registered Patent No. 10-0909032

In order to solve this problem, the present invention uses a fuzzy concept to differently reflect different fuzzy weights for each edge direction, and multiplies the weight value by the pixel value to be interpolated to interpolate to the final deinterlaced pixel Intelligent color image restoration Its purpose is to provide a system and a deinterlacing method using the same.

An intelligent color image restoration system according to a feature of the present invention for achieving the above object,

를 계산하는 퍼지 메트릭 계산부;Displays the missing pixel, existing pixel, and interpolation target pixel within the spatial axis window set for the input image frame, and the edge direction angle of the pixel located on both edges with the interpolation target pixel as the center

a fuzzy metric calculation unit that calculates

에 대한 퍼지 가중치

를 다르게 설정하는 가중치 계산부; 및The edge direction angle in the spatial axis window

Fuzzy weight for

a weight calculation unit that sets differently; and

을 보간 대상 픽셀값에 곱하여 최종 디인터레이싱된 픽셀을 생성하는 보간 구현부를 포함한다.Purge weight set above

and an interpolation implementation unit that multiplies the pixel value to be interpolated to generate a final deinterlaced pixel.

A deinterlacing method using a color image restoration system according to a feature of the present invention,

displaying a missing pixel, an existing pixel, and a pixel to be interpolated within a spatial axis window set for the input image frame by a fuzzy metric calculation unit;

를 계산하는 단계;The weight calculator determines the edge direction angles of pixels located on both edges of the interpolation target pixel as a center

Calculating ;

에 따라 퍼지 가중치

를 다르게 설정하는 단계; 및The weight calculator calculates the edge direction angle in the spatial axis window.

fuzzy weights according to

Setting differently; and

을 보간 대상 픽셀값에 곱하여 최종 디인터레이싱된 픽셀을 생성하는 단계를 포함한다.The interpolation implementation unit set the fuzzy weight

and multiplying by the pixel value to be interpolated to generate a final deinterlaced pixel.

According to the configuration described above, the present invention can obtain an interpolation image of good quality using the edge direction angle of the image, and at the same time selectively apply the interpolation method of the image to improve the objective and subjective image quality and algorithm calculation speed. It works.

1 is a diagram showing the configuration of an intelligent color image restoration system using a perceptual fuzzy concept according to an embodiment of the present invention.
2 is a diagram illustrating a deinterlacing method based on an edge direction according to an embodiment of the present invention.
3 is a diagram illustrating a triangle membership function according to an embodiment of the present invention.
FIG. 4 is a diagram showing a comparison result of visual performance using a conventional deinterlacing method according to an embodiment of the present invention and a deinterlacing method of the present invention.
5 is a diagram illustrating a deinterlacing method using an intelligent color image restoration system according to an embodiment of the present invention.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

1 is a diagram showing the configuration of an intelligent color image restoration system using a perceptual fuzzy concept according to an embodiment of the present invention, and FIG. 2 is a diagram showing a deinterlacing method based on an edge direction according to an embodiment of the present invention, 3 is a diagram showing a triangular membership function according to an embodiment of the present invention, and FIG. 4 is a diagram showing a result of comparing visual performance using a conventional deinterlacing method according to an embodiment of the present invention and a deinterlacing method of the present invention. .

An intelligent color image restoration system 100 using a perceptual fuzzy concept according to an embodiment of the present invention includes a fuzzy metric calculation unit 110, a weight calculation unit 120, a fuzzy membership calculation unit 130, and an interpolation implementation unit 140. , It includes a control unit 150 and a display unit 160.

As shown in FIG. 2 , a two-dimensional window in which white rectangles indicate missing pixels, dark gray rectangles indicate existing pixels, and light gray rectangles indicate pixels to be interpolated.

Parameters c and r represent the column number and row number in the spatial axis window. Since x represents a pixel, x(c,r) means the pixel value at the position (c,r). k is an integer of -3, -2, -1, -, 1, 2, 3;

가 있으며, 예를 들면, θ₃ = tan1(2/6) for x(c+3,r+1)and x(c-3,r-1)이고, θ₂ = tan1(2/4) for x(c+2, r+1) and x(c-2, r-1), θ₁ = tan1(2/2) for x(c+1, r+1) and x(c-1, r-1), θ₀ = tan1(2/0) for x(c, r+1) and x(c, r-1), θ₁ = tan1(-2/2) for x(c-1, r+1) and x(c+1, r-1), θ₂ = tan1(-2/4) for x(c-2, r+1) and x(c+2, r-1), and θ₃ = tan1(-2/6) for x(c-3, r+1) and x(c+3, r-1)이다.2 shows several edge direction angles

There is, for example, θ ₃ = tan1(2/6) for x(c+3,r+1)and x(c-3,r-1), and θ ₂ = tan1(2/4) for x(c+2, r+1) and x(c-2, r-1), θ ₁ = tan1(2/2) for x(c+1, r+1) and x(c-1, r -1), θ ₀ = tan1(2/0) for x (c, r+1) and x(c, r-1), θ ₁ = tan1(-2/2) for x(c-1, r +1) and x(c+1, r-1), θ ₂ = tan1(-2/4) for x(c-2, r+1) and x(c+2, r-1), and θ ₃ = tan1(-2/6) for x(c-3, r+1) and x(c+3, r-1).

Since the conventional edge-based line average (ELA)-based interlacing method selects only one edge direction, good interpolation quality cannot be obtained regardless of edge characteristics or motion characteristics of an image. To overcome this problem, the present invention uses edge direction information in all directions of 360 degrees centered on an interpolation target pixel.

The fuzzy metric calculation unit 110 displays the missing pixel, the existing pixel, and the pixel to be interpolated within the spatial axis window set for the input image frame, and the edge direction angle of the pixel located on both edges of the pixel to be interpolated as the center Calculate The edge direction angle means the angle at the point where a line is drawn between two pixels and meets the horizontal line.

The missing pixel is calculated as the average value of two pixel values located on both edges of the missing pixel as the center.

에 대한 퍼지 가중치

를 다르게 설정한다. 각각의 에지 방향 각도는 퍼지 가중치를 가정하는 서로 다른 중요도를 가지고 있다.The weight calculator 120 calculates seven edge direction angles in the spatial axis window.

Fuzzy weight for

set differently. Each edge orientation angle has a different importance for assuming fuzzy weights.

와

)를 계산한다.The weight calculation unit 120 uses Equations 1 and 2 below to determine the first and second parameters (

and

) is calculated.

는 두 개의 픽셀값의 차이를 나타낸다.Here, the first parameter

represents the difference between two pixel values.

는 모든

중 가장 큰 값을 나타낸다.Here, the second parameter

is all

represents the largest value among them.

가 작을수록 올바른 에지 방향의 가능성이 높고,

가 클수록 올바른 에지 방향의 가능성이 낮다고 가정한다.The weight calculation unit 120

The smaller is the higher the probability of correct edge orientation,

It is assumed that the probability of the correct edge direction is low as .

가 기설정된 기준 임계값 이상인 경우, 에지 방향 각도

를 제거한다.The weight calculation unit 120

If is greater than or equal to the preset reference threshold, the edge direction angle

Remove

The fuzzy set membership function is an implementation of the indicator function for Humanistic Sets. Since fuzzy truth means being a member of an ambiguously interpreted set, the degree of truth is similar to probability. L.Zadeh proposed a fuzzy set that performs in the domain of all possible values using a membership function in the range (0, 1).

The fuzzy membership calculator 130 calculates a fuzzy membership value using a triangular membership function. 3 shows the triangle membership function.

로 설명된다. 매개변수

와

는 멤버쉽 함수의 피트(Feet)를 나타내고, 매개변수

는 멤버쉽 함수의 피크(Peak)를 나타낸다. 본 발명의 삼각형 멤버쉽 함수에서는

로 가정하고 있다.where the membership function parameter is a vector

is explained by parameter

and

represents the feet of the membership function, and the parameters

represents the peak of the membership function. In the triangle membership function of the present invention,

is assumed to be

The interpolation implementation unit 140 calculates a weighted moving average pixel (xWMA) at the position (c, r) using Equation 4 below. A weighted moving average is an average with a multiplicative factor that gives different weights to data at different locations in the sample window.

The interpolation implementation unit 140 calculates a weighted moving average pixel value by multiplying the interpolation target pixel value by the weight value calculated by the weight calculation unit 120 according to Equation 4 above.

)을 생성한다.The interpolation implementation unit 140 substitutes the calculated weighted moving average pixel value into Equation 5 below to obtain the final deinterlaced pixel of the interpolation target pixel (

) to create

)에 보간하여 영상 프레임을 디스플레이부(160)에 출력한다.The controller 150 generates the final deinterlaced pixel (

) and outputs the image frame to the display unit 160.

It was performed on well-known test images of progressive formats such as Akiyo, Flower, Foreman, Mobile, News, Stefan and Table Tennis. All video frames are 352 (horizontal) x 288 (vertical) pixels in size.

If the simulation process is performed using video frames such as Akiyo, Mobile, and News, the following is done.

(Step 1) A video frame of 1/30 sec is first divided into two interlace fields of 1/60 by alternate subsampling. If we consider deinterlacing in odd fields, the even field information we want to fill is missing.

(Step 2) The interlaced signals are deinterlaced in various deinterlacing methods.

(Step 3) The deinterlaced signal is finally compared with the original video signal.

Objective performance is evaluated with the peak signal-to-noise ratio (PSNR) metric defined by Equation 6 below.

The control unit 150 calculates the PSNR of deinterlacing signals using various deinterlacing methods on well-known test images in progressive format, such as Akiyo, Flower, Foreman, Mobile, News, Stefan, and Table Tennis, by Equations 6 and 7 below. Calculate.

는 디인터레이스된 비디오 프레임이다. 255는 그레이 스케일 영상의 최대 명암값을 나타내며, C와 R은 가로 세로 방향의 이미지 길이이다. 본 발명에서는 C=352, R=288이다.where x is the original video frame,

is a deinterlaced video frame. 255 represents the maximum contrast value of the gray scale image, and C and R are the image lengths in the horizontal and vertical directions. In the present invention, C = 352 and R = 288.

As shown in FIG. 4, the controller 150 uses the bike image to determine (a) Bob, (b) ELA, (c) NEDI, (d) LCID, (e) MELA, (f) MADLSCD, (g) ), LABI, (h) FMELA, and (i) the visual performance of bicycle images is compared by applying the deinterlacing method based on the edge direction of the present invention. Here, the conventional deinterlacing methods are Bob, ELA, NEDI, LCID, MELA, MADLSCD, LABI, and FMELA, and detailed descriptions thereof are omitted.

As another embodiment, the controller 150 applies Bob, ELA, NEDI, LCID, MELA, MADLSCD, LABI, FMELA, the deinterlacing method of the present invention to the sample image, calculates the PSNR of the deinterlacing signal (Equation 6, Equation 7), the deinterlacing method having the highest calculated PSNR is selected.

Table 1 shows PSNR results of deinterlacing signals using various deinterlacing methods.

As can be seen in Table 2, the deinterlacing method of the present invention is -0.421 (Bob), -1.611 (ELA), -0.92 (NEDI), -0.365 (LCID), -0.37 (MELA), -0.899 (MADLSCD), -0.003 (LABI) and -0.301 (FMELA), which showed better performance than other methods.

5 is a diagram illustrating a deinterlacing method using an intelligent color image restoration system according to an embodiment of the present invention.

The fuzzy metric calculation unit 110 displays missing pixels, existing pixels, and pixels to be interpolated within the spatial axis window set for the input image frame (S100).

를 계산한다(S110).The weight calculation unit 120 calculates the edge direction angle of pixels located on both edges of the pixel to be interpolated as the center.

Calculate (S110).

에 따라 퍼지 가중치

를 다르게 설정한다(S120).The weight calculator 120 calculates the edge direction angle in the spatial axis window.

fuzzy weights according to

Set differently (S120).

와 제2 파라미터

를 계산한다.The weight calculation unit 120 uses the above-described Equations 1 and 2 to determine the first parameter.

and the second parameter

Calculate

The fuzzy membership calculation unit 130 calculates the fuzzy membership value using the triangular membership function of Equation 3 described above.

을 보간 대상 픽셀값에 곱하여 최종 디인터레이싱된 픽셀을 생성한다(S130).The interpolation implementation unit 140 sets fuzzy weights

A final deinterlaced pixel is generated by multiplying by the pixel value to be interpolated (S130).

/

를 삼각형 멤버쉽 함수(도 3)에 입력하여 가중치

를 출력한다.The weight calculation unit 120

/

to the triangle membership function (Fig. 3) to weight

outputs

를 계산하고,

를 전술한 수학식 5에 대입하여 보간 대상 픽셀의 최종 디인터레이싱된 픽셀(

)을 생성한다.The interpolation implementation unit 140 calculates the weighted moving average pixel value at the position (c, r) by using Equation 4 described above.

calculate,

The final deinterlaced pixel of the pixel to be interpolated by substituting into Equation 5 above (

) to create

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

100: intelligent color image restoration system
110: fuzzy metric calculation unit
120: weight calculation unit
130: fuzzy membership calculation unit
140: interpolation implementation unit
150: control unit
160: display unit

Claims (11)

Displays the missing pixel, existing pixel, and interpolation target pixel within the spatial axis window set for the input image frame, and the edge direction angle of the pixel located on both edges with the interpolation target pixel as the center

a fuzzy metric calculation unit that calculates
The edge direction angle in the spatial axis window

Fuzzy weight for

a weight calculation unit that sets differently; and
Purge weight set above

An intelligent color image restoration system including an interpolation implementation unit that multiplies y by an interpolation target pixel value to generate a final deinterlaced pixel. The method of claim 1,
The weight calculation unit uses Equations 1 and 2 below to determine the first parameter

and the second parameter

An intelligent color image restoration system that calculates
[Equation 1]

Here, the first parameter

is the difference between two pixel values, c and r are the column and row numbers in the spatial axis window, x is a pixel, x(c,r) represents the pixel value at the position (c,r), k is An integer of -3, -2, -1, -, 1, 2, 3.
[Equation 2]

Here, the second parameter

is all

is the largest value among them. The method of claim 2,
An intelligent color image restoration system further comprising a fuzzy membership calculation unit that calculates a fuzzy membership value using a triangular membership function of Equation 3 below.
[Equation 3]

where the membership function parameter is a vector

is the parameter

and

represents the feet of the membership function, and the parameters

Is the peak of the membership function, and in the triangle membership function

Assumed to be. The method of claim 3,
The weight calculation unit

/

to the triangle membership function to enter the weight

An intelligent color image restoration system that outputs The method of claim 1,
The interpolation implementation unit uses Equation 4 below to calculate the weighted moving average pixel value at the position (c, r)

Calculate, and the

The final deinterlaced pixel of the pixel to be interpolated by substituting into Equation 5 below (

), an intelligent color image restoration system that generates
[Equation 4]

[Equation 5]

The method of claim 1,
The fuzzy metric calculator calculates edge direction angles in all directions of 360 degrees centered on the interpolation target pixel.

An intelligent color image restoration system that calculates displaying a missing pixel, an existing pixel, and a pixel to be interpolated within a spatial axis window set for the input image frame by a fuzzy metric calculation unit;
The weight calculator determines the edge direction angles of pixels located on both edges of the interpolation target pixel as a center

Calculating ;
The weight calculator calculates the edge direction angle in the spatial axis window.

fuzzy weights according to

Setting differently; and
The interpolation implementation unit set the fuzzy weight

A deinterlacing method using a color image restoration system, comprising generating final deinterlaced pixels by multiplying by a pixel value to be interpolated. The method of claim 7,
The weight calculation unit uses Equations 1 and 2 below to determine the first parameter

and the second parameter

A deinterlacing method using a color image restoration system, further comprising the step of calculating .
[Equation 1]

Here, the first parameter

is the difference between two pixel values, c and r are the column and row numbers in the spatial axis window, x is a pixel, x(c,r) represents the pixel value at the position (c,r), k is An integer of -3, -2, -1, -, 1, 2, 3.
[Equation 2]

Here, the second parameter

is all

is the largest value among them. The method of claim 8,
A deinterlacing method using a color image restoration system, further comprising calculating a fuzzy membership value using a triangular membership function of Equation 3 below in a fuzzy membership calculation unit.
[Equation 3]

where the membership function parameter is a vector

is the parameter

and

represents the feet of the membership function, and the parameters

Is the peak of the membership function, and in the triangle membership function

Assumed to be. The method of claim 9,
The weight calculation unit

/

to the triangle membership function to enter the weight

A deinterlacing method using a color image restoration system further comprising the step of outputting. The method of claim 7,
The interpolation implementation unit uses Equation 4 below to calculate the weighted moving average pixel value at the position (c, r)

Calculate, and the

The final deinterlaced pixel of the pixel to be interpolated by substituting into Equation 5 below (

) Deinterlacing method using a color image restoration system further comprising the step of generating.
[Equation 4]

[Equation 5]

Images