KR101469513B1 - Video processing method and apparatus based on multiple texture image using video excitation signal - Google Patents

Abstract

A multi-texture image-based image processing technique using an image excitation signal capable of processing an image at an optimum image quality at a low data rate is disclosed. To this end, the multi-texture image-based image processing method according to the present invention includes the steps of: classifying an input image into a shot unit image and selecting one of a plurality of frames of the shot unit image as a seed image; Detecting a plurality of feature points in the seed image; A step of tracking a plurality of feature points in a plurality of frames of the shot unit image to calculate a space-time position conversion parameter for each of the feature points; Defining a plurality of texture images using feature points corresponding to spatiotemporal transformation parameters; And defining each of the plurality of texture images as a sum of a plurality of texture blocks that are outputs of a texture synthesis filter to which an image excitation signal is input.

Description

[0001] The present invention relates to a multi-texture image-based image processing method and apparatus using an image excitation signal,

The present invention relates to a multi-texture image-based image processing method and apparatus using an image excitation signal. More particularly, the present invention relates to a multi-texture image-based image processing method and apparatus using an image excitation signal capable of processing an image at an optimal image quality at a low data rate.

A general image processing method is based on a method in which a signal subjected to motion estimation processing between image frames is subjected to transform domain processing using discrete cosine transform or the like. However, a general image processing method has a problem that it is difficult to express characteristics of various real images due to inaccurate estimation and modeling of image characteristics. As a result, the difference between the converted video signal and the original video signal increases and the bit rate of the video signal increases. To solve this problem, image compression standards such as MPEG1 / 2/4 and H.261 / 263/264 have been proposed, but image quality degradation is still serious in low bit rate image compression such as 1/500 bit rate compared to the original size . In addition, 1 / n pixel motion estimation and compensation method, adaptive block size conversion region image processing method, multiple reference frame motion estimation and compensation method, and generalized B-frame processing methods have been proposed and used. However, Image degradation is serious.

It is an object of the present invention to represent a variety of image characteristics through a plurality of texture images and space-time location transformation parameters of the texture images.

It is another object of the present invention to provide a compressed image which is significantly reduced in size relative to an original size by compressing an original image using only a plurality of texture images and a plurality of space-time location transformation variables corresponding thereto. In addition, the present invention defines each of a plurality of texture images as a sum of an image excitation signal expressed by a Gaussian function and a texture block, which is an output of a texture synthesis filter, to provide a compressed image that is significantly larger in size than the original size .

In addition, the present invention aims at reducing the size of a compressed image by approximating a plurality of predetermined texture images using similarity of space-time location conversion parameters.

In addition, the present invention aims at processing an image at an optimal image quality at a low transmission rate.

According to an aspect of the present invention, there is provided a method of processing a multi-texture image based image, the method comprising: classifying input images into shot image units and selecting one of a plurality of frames of the shot unit image as a seed image; Detecting a plurality of feature points in the seed image; Tracking the plurality of feature points in the plurality of frames of the shot unit image to calculate a space-time position conversion parameter for each of the feature points; Defining a plurality of texture images using the feature points corresponding to the space-time location transformation parameters; And defining each of the plurality of texture images as a sum of a plurality of texture blocks that are outputs of a texture synthesis filter to which an image excitation signal is input.

At this time, the image excitation signal can be expressed by a two-dimensional Gaussian function.

The method may further include compressing space-time location transformation variables corresponding to each of the image excitation signal, the texture synthesis filter parameter, and the plurality of texture images of each of the plurality of texture blocks defining the plurality of texture images do.

In this case, the compressing step compresses the image excitation signal, the variable of the texture synthesis filter, and the space-time location conversion parameter using a bit stream compression method.

At this time, in the plurality of texture images, texture images having space-time location transformation variables whose similarities calculated by obtaining the correlation characteristic of the texture image signals have values within a predetermined threshold value are classified into one texture image And further includes a step of approximating.

At this time, the step of detecting the plurality of feature points detects, as the feature points, a point having a variation amount equal to or greater than a predetermined value in the plurality of frames.

Decompressing the compressed image excitation signal, the parameters of the texture synthesis filter, and the space-time location conversion variable corresponding to each of the texture images; Generating the plurality of texture blocks using the image excitation signal and the parameters of the texture synthesis filter, and summing the texture blocks to generate the plurality of texture images; Matching the texture image with the space-time location transformation parameter corresponding to the texture image; Generating a visual texture using the texture image and the space-time location transformation parameter; And combining the visual textures generated corresponding to each of the texture images.

The method further includes filtering and correcting artifacts at the joint boundaries of the visual textures.

According to another aspect of the present invention, there is provided a multi-texture image-based image processing apparatus for classifying an input image into a shot unit image and selecting one of a plurality of frames of the shot unit image as a seed image, An image selection unit; A feature point detector for detecting a plurality of feature points in the seed image; A variable calculation unit for tracking the plurality of feature points in the plurality of frames of the shot unit image to calculate a space-time location conversion parameter for each of the feature points; A texture image defining unit that defines a plurality of texture images using the feature points corresponding to the space-time location conversion parameters; And a texture block defining unit that defines each of the plurality of texture images as a sum of a plurality of texture blocks that are outputs of a texture synthesis filter that receives an image excitation signal.

At this time, the image excitation signal can be expressed by a two-dimensional Gaussian function.

The image processing apparatus may further include a compression unit that compresses the image excitation signal of each of the plurality of texture blocks defining the plurality of texture images, the variable of the texture synthesis filter, and the space-time position conversion variable corresponding to each of the plurality of texture images do.

At this time, the compression unit separately compresses the image excitation signal, the texture synthesis filter, and the space-time location conversion variable by a bitstream compression method.

At this time, in the plurality of texture images, texture images having space-time location transformation variables whose similarities calculated by obtaining the correlation characteristic of the texture image signals have values within a predetermined threshold value are classified into one texture image And further includes an approximate approximation unit.

At this time, the minutia point detecting unit detects, as the minutiae points, a point having a variation amount equal to or larger than a predetermined value in the plurality of frames.

A decompression unit decompressing the compressed image excitation signal, the parameters of the texture synthesis filter, and the space-time location conversion parameters corresponding to the respective texture images; A texture image generation unit for generating the texture blocks by using the image excitation signal and the parameters of the texture synthesis filter, and adding the plurality of texture blocks to generate the texture image; A matching unit for matching the texture image with the space-time location conversion parameter corresponding to the texture image; A visual texture generator for generating a visual texture using the texture image and the space-time location transformation parameter; And a visual texture combining unit for combining the visual textures generated corresponding to the respective texture images.

In this case, the image processing apparatus further includes a correction unit for filtering and correcting artifacts at the boundary of the visual textures.

According to the present invention, various image characteristics can be expressed through a plurality of texture images and space-time location transformation parameters of the texture images.

In addition, the present invention is capable of compressing an original image using only a plurality of texture images and a plurality of space-time location conversion variables corresponding thereto, thereby providing a compressed image that is significantly reduced in size relative to the original size. In addition, the present invention defines each of a plurality of texture images as a sum of an image excitation signal expressed by a Gaussian function and a texture block, which is an output of a texture synthesis filter, to provide a compressed image that is significantly larger in size than the original size .

In addition, the present invention can further reduce the size of a compressed image by approximating a plurality of texture images determined using the similarity of space-time location conversion parameters.

In addition, the present invention can process an image at an optimal image quality at a low transmission rate. That is, the present invention can minimize image deterioration at a low transmission rate such as 1/500 bit rate.

FIG. 1 is a flowchart illustrating an encoding method in a multi-texture image-based image processing method according to the present invention.
2 is a diagram for explaining a method of encoding in a multi-texture image-based image processing method according to the present invention.
FIG. 3 is a flowchart illustrating a decoding method in a multi-texture image-based image processing method according to the present invention.
4 is a diagram for explaining a decoding method in the multi-texture image-based image processing method according to the present invention.
FIG. 5 is a block diagram of a multi-texture image-based image processing apparatus according to the present invention.

The present invention will now be described in detail with reference to the accompanying drawings. Hereinafter, a repeated description, a known function that may obscure the gist of the present invention, and a detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

Hereinafter, an encoding method in a multi-texture image-based image processing method according to the present invention will be described.

FIG. 1 is a flowchart illustrating an encoding method in a multi-texture image-based image processing method according to the present invention. 2 is a diagram for explaining a method of encoding in a multi-texture image-based image processing method according to the present invention.

Referring to FIGS. 1 and 2, an encoding method of a multi-texture image-based image processing method according to the present invention receives an image 200 composed of a plurality of frames (S10).

The input image 200 is classified into a shot unit image, and one of a plurality of frames of the shot unit image is selected as the seed image 210 (S11). The remaining frames except for the seed image 210 of the shot unit image are defined as the residual frame image 220. That is, when the shot unit image is composed of k frames, one seed image is selected and the remaining k-1 frames are defined as the residual frame image 220. [ At this time, the shot unit video corresponds to an image shot by one camera consecutively.

A plurality of feature points are detected in the seed image 210 selected in step S11 (S12). At this time, in a plurality of frames of the shot unit image, points having a variation amount equal to or greater than a predetermined value can be detected as feature points. That is, if a specific point in the seed image 210 and the residual frame image 220 shows a change of more than a predetermined value, the specific point can be detected as a feature point.

Then, a plurality of feature points are traced in a plurality of frames of the shot unit image to calculate a space-time position conversion parameter for each of the feature points (S13). That is, the space-time location conversion parameter that defines the change of the feature point in the seed image 210 and the residual frame image 220 is calculated. The space-time location conversion parameter may be a function representing the amount of change in position of the feature point with time.

A plurality of texture images 211a, 212a, 213a, 214a, and Na are defined using mutually corresponding minutiae corresponding to the space-time location conversion variables 211b, 212b, 213b, 214b, and Nb calculated in step S13 ). At this time, one texture image can be defined by connecting the same minutiae to each other through the space-time location conversion parameters 211b, 212b, 213b, 214b, and Nb.

Then, in the plurality of texture images, texture images having similar space-time location conversion parameters are combined into one texture image and approximated (S15). At this time, the similarity between the space-time location conversion variables can be calculated by obtaining the correlation characteristic of the texture image signal. Then, the texture images having the similarity between the space-time location conversion variables and the values within the predetermined threshold value can be combined into one texture image. In FIG. 2, a first texture image 211a and a second texture image 212a, which are assumed to have a high similarity, are combined with each other, and corresponding to the first space time position conversion parameter 211b and the second space time position By combining the transform variables 212b, a first approximate texture image 211a 'and a first approximate space-time position transform variable 211b' are generated. The third texture image 213a and the fourth texture image 214a are combined and the third space-time position transform variable 213b and the fourth space-time position transform variable 214b are combined, The approximate texture image 213a 'and the second approximate space-time location conversion variable 213b' are generated.

Then, each of the plurality of texture images 211a, 212a, 213a, 214a, and Na is defined as a sum of a plurality of texture blocks (S16). If step S15 is performed, each of the plurality of approximated texture images 211 ', 213', and Na 'may be defined as a sum of a plurality of texture blocks. At this time, the texture block can be defined as the output of the texture synthesis filter which receives the image excitation signal. Then, the image excitation signal can be expressed by a two-dimensional Gaussian function. The image excitation signal, that is, the Gaussian function, has a magnitude variable G, a mean value variable m, and a bearer value a as model parameters. The texture synthesis filter has a transform domain filter coefficient as a model parameter as shown in Equation 1 below.

The parameters of the image excitation signal, i.e., G, m, a values and the variable h values of the texture synthesis filter are obtained to minimize the difference between the texture estimate value and the original texture signal value in the transform domain. The texture estimation signal R in the transform domain is expressed by the following equation (2).

E and H denote the excitation signal vector and the texture synthesis filter coefficient vector in the transform domain and '·' denotes the product of each component of the vector. The excitation signal vector E is approximated by a two-dimensional Gaussian function, and the texture synthesis filter H has a characteristic in which most of the variable values are 0 and the variable values are only in some regions according to the texture transformation region characteristic. Therefore, the image processing method according to the present invention enables compression at a very low bit rate using a variable length encoder or an arithmetic encoder.

The image excitation signal of each of the plurality of texture blocks defining the plurality of texture images 211a, 212a, 213a, 214a, and Na, the parameters of the texture synthesis filter, and the plurality of space- The conversion variables 211b, 212b, 213b, 214b, and Nb are compressed (S17). In step S17, the image excitation signal of each of the plurality of texture blocks defining each of the plurality of approximated texture images 211a ', 213a', and Na 'is determined based on the approximation step S15 of the texture image, A plurality of approximate space-time position transform parameters 211b ', 213b', and Nb 'corresponding to the variables of the texture synthesis filter and the plurality of approximated texture images may be compressed. At this time, compression may be performed by a bit stream compression method.

Hereinafter, an encoding method in a multi-texture image-based image processing method according to the present invention will be described with reference to mathematical expressions.

에 대하여, 자기 상관 매트릭스(Autocorrelation matrix)

를 계산한다. 여기서,

는 {x,y}가

를 만족할 때의 포인트의 주변 윈도우 신호이다. 그리고, x와 y는 각각 x축 방향과 y축 방향의 픽셀 포인트이며,

는 통계적 기대 함수(Statistical expectation operator)로 정의된다. 또한, w와 w'은 입력 영상 프레임의 가로 픽셀 개수이고, h와 h'는 입력 영상 프레임의 세로 픽셀 개수이다.The minutiae points can be detected as follows. First, an input image composed of k frames

For example, an autocorrelation matrix,

. here,

Is { x , y }

Is the surrounding window signal of the point when it is satisfied. X and y are pixel points in the x-axis direction and the y-axis direction, respectively,

Is defined as a statistical expectation operator. Also, w and w 'are the number of horizontal pixels of the input image frame, and h and h ' are the number of vertical pixels of the input image frame.

를 통하여 계산된 고유치들 즉,

및

로부터, 텍스쳐 포인트 매트릭스

를 다음의 수학식 3과 같이 구할 수 있다. At pixel point { x , y }

The eigenvalues computed through Eq.

And

From the texture point matrix

Can be obtained by the following equation (3).

및

는 기 설정된 임계치에 해당한다. 상기 수학식 1에서 특정 픽셀 위치의

및

가 임계치

및

보다 큰 경우, 해당 특정 픽셀을 1로 정의한다. 그리고, 특정 픽셀 위치의

및

가 임계치

및

보다 작은 경우, 해당 특정 픽셀을 0으로 정의하여 텍스쳐 포인트 매트릭스를 구한다.
here,

And

Corresponds to a predetermined threshold value. In Equation 1,

And

Lt; / RTI >

And

, The specific pixel is defined as " 1 ". Then,

And

Lt; / RTI >

And

, The texture point matrix is obtained by defining the specific pixel as zero.

A plurality of space-time location transformation variables and texture images corresponding to each texture image may be defined as Equation (4).

는 다음의 수학식 5와 같이 정의될 수 있다.here,

Can be defined by the following equation (5).

는 다음의 수학식 6과 같이, N개의 텍스쳐 이미지의 합으로써 정의될 수 있다. Then, an input image composed of k frames

Can be defined as the sum of N texture images, as shown in Equation (6).

In Equation (6), the i- th divided texture image can be expressed by approximation as shown in Equation (7).

는 전달 함수를,

은 입력 영상의 l번째 프레임의 i번째 분할된 텍스쳐 이미지를,

는 x축 방향과 y축 방향의 위치변환 벡터를,

는

에서의 개략적인 추정 오류 신호를 나타낸다. 그리고, 수학식 7에서, 프레임 넘버 k는 k+1부터 l+M의 범위에 속한다. 수학식 7은 Taylor expansion에 의하여 다음의 수학식 8과 같이 근사화될 수 있다.here,

Is a transfer function,

Th texture image of the l & lt; th > frame of the input image,

A position conversion vector in the x-axis direction and the y-axis direction,

The

Which represents the approximate estimated error signal at. In Equation (7), the frame number k belongs to the range of k + 1 to 1 + M. Equation (7) can be approximated by the following equation (8) by Taylor expansion.

및

각각은

의 x축 방향과 y축 방향의 경사도(Gradient value)의 합을 나타낸다. 그리고, 추정 오류 신호의 제곱합에 대한 정리는 다음의 수학식 9와 같이 나타낼 수 있다. here,

And

Each

Axis direction and a y-axis direction gradient value (Gradient value). The sum of squares of the estimated error signals can be expressed by the following equation (9).

크기의 최소화를 가정하여,

의 값을 구할 수 있다. 즉, 다음의 수학식 10 및 수학식 11을 계산하여,

의 값을 구한다.Here, the square sum of the estimated error signal

Assuming a minimization of size,

Can be obtained. That is, by calculating the following equations (10) and (11)

.

가 항등변환(Identity transform)식임을 가정하면, 수학식 9와 수학식 10을 통해, 다음의 수학식 12를 얻을 수 있다.here,

Is an identity transform equation, the following equation (12) can be obtained from the equations (9) and (10).

에 대한 다음의 수학식 13을 얻을 수 있다.Then, solving Equation (12)

The following equation (13) can be obtained.

및 수학식 11을 사용하여 다음의 수학식 14와 같은 변환함수

를 얻을 수 있다.Then, the equation

And Equation (11), a transformation function < RTI ID = 0.0 >

Can be obtained.

의 변환함수

으로부터

를 얻을 수 있다. 그리고,

의 변환함수

으로부터

를 얻을 수 있다. 또한, 수학식 3 내지 수학식 14를 통해,

를 시드 이미지

및 변환함수

로써 표현할 수 있다. 그리고,

간의 유사성을 계산하여 텍스쳐 이미지의 근사화가 이루어질 수 있다.
(10) and (11) are summarized,

Conversion function of

From

Can be obtained. And,

Conversion function of

From

Can be obtained. Further, through Equations 3 to 14,

Seed image

And conversion functions

. And,

The approximation of the texture image can be approximated.

Hereinafter, a decoding method in the multi-texture image-based image processing method according to the present invention will be described.

FIG. 3 is a flowchart illustrating a decoding method in a multi-texture image-based image processing method according to the present invention. 4 is a diagram for explaining a decoding method in the multi-texture image-based image processing method according to the present invention.

Referring to FIGS. 3 and 4, a decoding method of a multi-texture image-based image processing method according to the present invention receives a compressed image signal (S30). In this case, the compressed video signal may be a signal in which a plurality of texture blocks defining a plurality of texture images, a video excitation signal of each of the plurality of texture blocks, a variable of a texture synthesis filter, and a plurality of space- . Of course, the compressed video signal may be a signal in which the image excitation signal of each of the plurality of texture blocks defining the plurality of approximated texture images, the parameters of the texture synthesis filter, and the plurality of approximated space-time position transformation parameters are compressed. Also, in the compressed video signal, the image excitation signal, the parameters of the texture synthesis filter, and the plurality of space-time location conversion parameters may be compressed by a bitstream compression method.

Then, the compressed video signal is decompressed (S31). That is, the compressed image excitation signal, the parameters of the texture synthesis filter, and the plurality of space-time location conversion variables corresponding to each texture image are decompressed.

Then, a plurality of texture blocks are generated by using the compressed image excitation signal and the parameters of the texture synthesis filter, and a texture image is generated by combining the plurality of texture blocks (S32). The texture image at this time may be an approximated texture image in the encoding process.

In each of the generated plurality of texture images and the plurality of space-time location conversion variables, each texture image and the space-time location conversion variables corresponding to the texture image are matched one-to-one (S33). Of course, each approximated texture image and an approximate space-time location transformation parameter corresponding to the approximated texture image may be matched. 4, the first approximate texture image 211a 'and the first approximate space-time position transform variable 211b' are matched and the second approximate texture image 213a 'and the second approximate space-time position transform variable 213b' And the Nth approximated texture image Na 'is matched with the Nth approximated space-time location transformation parameter Nb'.

In step S33, a visual texture is generated using the matched texture image and space-time location transformation parameters (S34). Specifically, a temporal / spatial transformation parameter defining temporal / spatial motion of minutiae points is applied to a texture image, and a visual texture composed of a plurality of frames for the texture image is generated. Of course, a visual texture may be generated using a matched approximated texture image and an approximate space-time location transformation parameter. In FIG. 4, a first visual texture 211 composed of a plurality of frames is generated using the first approximate texture image 211a 'and the first approximate space-time location conversion variable 211b'. Then, a second visual texture 213 composed of a plurality of frames is generated using the second approximate texture image 213a 'and the second approximate space-time position conversion parameter 213b'. Also, an Nth visual texture N composed of a plurality of frames is generated using the Nth approximated texture image Na 'and the Nth approximated space-time location transformation parameter Nb'.

In step S34, the visual textures corresponding to the respective texture images are combined (S35). By combining the visual textures, a plurality of frames of the shot unit image are restored as a whole. In FIG. 4, a first visual texture 211, a second visual texture 213, and an Nth visual texture N are combined.

At step S35, the artifacts at the joint boundaries of the plurality of combined visual textures are filtered and corrected (S36). That is, the combined plurality of visual textures in step S35 are reconstructed as a simple sum, and a defect may occur at the boundary between the visual textures. By performing an elimination filtering operation on these defects, a final corrected restored image is generated.

The restored images of the shot units may be combined with the restored image of the shot unit obtained through step S36 to generate the final image 200 '.

Hereinafter, the structure and operation of a multi-texture image-based image processing apparatus according to the present invention will be described.

FIG. 5 is a block diagram of a multi-texture image-based image processing apparatus according to the present invention.

Referring to FIG. 5, the multi-texture image-based image processing apparatus 500 according to the present invention may include an encoding unit 510 and a decoding unit 520.

The encoding unit 510 includes a seed image selecting unit 511, a feature point detecting unit 512, a variable calculating unit 513, a texture image defining unit 514, and a texture block defining unit 516. The encoding unit 510 may further include an approximation unit 515 and a compression unit 517.

The seed image selection unit 511 classifies the input image into a shot unit image, and selects one of a plurality of frames of the shot unit image as a seed image. The seed image selection unit 511 defines the remaining frames except the seed image of the shot unit image as a residual frame image. That is, when the shot unit image is composed of k frames, the seed image selector 511 selects one seed image and defines the remaining k-1 frames as the residual frame image 220. [ At this time, the shot unit video corresponds to an image shot by one camera consecutively.

The feature point detection unit 512 detects a plurality of feature points in the seed image selected by the seed image selection unit 511. At this time, the minutia detection unit 512 can detect a point having a variation amount equal to or larger than a predetermined value in a plurality of frames of the shot unit image, as minutiae. That is, if the specific point in the seed image and the residual frame image shows a change of more than a predetermined value, the feature point detector 512 can detect the specific point as a feature point.

The variable calculating unit 513 tracks a plurality of feature points in a plurality of frames of the shot unit image to calculate a space-time position conversion parameter for each of the feature points. That is, the variable calculating unit 513 calculates a space-time location conversion parameter that defines the change of the minutiae in the seed image and the residual frame image. The space-time location conversion parameter may be a function representing the amount of change in position of the feature point with time.

The texture image defining unit 514 defines a plurality of texture images using the mutually corresponding minutiae corresponding to the space-time location conversion variables calculated by the variable calculating unit 513. [ At this time, the texture image defining unit 514 may define one texture image by associating the same minutiae with the same space-time location conversion variables.

The approximation unit 515 combines and approximates texture images having similar space-time location transformation parameters into one texture image for a plurality of texture images. That is, the approximation unit 515 may generate a plurality of approximated texture images and a plurality of approximated space-time position transformation variables in which a plurality of texture images, a plurality of space-time location transformation variables are approximated, and a plurality of approximate space- In this case, the approximation unit 515 can calculate the similarity between the space-time location transformation variables by obtaining a correlation characteristic of the texture image signal. The approximation unit 515 may combine the texture images whose similarities between the space-time location conversion variables have values within a predetermined threshold value into one texture image.

The texture block defining unit 516 defines each of the plurality of texture images as a sum of a plurality of texture blocks. At this time, the texture block can be defined as the output of the texture synthesis filter which receives the image excitation signal. Then, the image excitation signal can be expressed by a two-dimensional Gaussian function. Of course, the texture block defining unit 516 may define each of the plurality of approximated texture images as a sum of a plurality of texture blocks.

The compression unit 517 compresses the parameters of the image excitation signal and the texture synthesis filter and a plurality of space-time location conversion variables corresponding to the respective texture images. Of course, the compression unit 517 may compress the image excitation signal of the plurality of approximated texture images, the parameters of the texture synthesis filter, and the plurality of approximated space-time position transformation variables.

The decoding unit 520 includes a decompression unit 521, a texture image generation unit 522, a matching unit 523, a visual texture generation unit 524, and a visual texture combination unit 525. The decoding unit 520 may further include a correction unit 526. [

The decompression unit 521 receives the compressed video signal from the encoding unit 510 and decompresses the compressed video signal. The decompression unit 521 decompresses the parameters of the image excitation signal and the texture synthesis filter that define each of the compressed plurality of texture images, and the plurality of space-time location conversion variables corresponding to the respective texture images.

The texture image generating unit 522 generates a plurality of texture blocks using the parameters of the image excitation signal and the texture synthesis filter, and combines the plurality of texture blocks to generate a texture image.

The matching unit 523 matches the texture image and the space-time location transformation parameters corresponding to the texture image in a one-to-one correspondence, using the texture image generated by the texture image generation unit 522 and a plurality of space-time location transformation variables. Of course, the matching unit 523 may match the approximate space-time location conversion parameters corresponding to the respective approximated texture images and the corresponding approximated texture images.

The visual texture generator 524 generates a visual texture using the matched texture image and the space-time location transformation parameters. Specifically, the visual texture generator 524 generates a visual texture composed of a plurality of frames for the texture image by applying a space-time location transformation parameter defining a time-to-motion and the like of the feature points to the texture image. Of course, the visual texture generator 524 may generate the visual texture using the matched approximated texture image and the approximate space-time location transformation parameters.

The visual texture combining unit 525 combines the visual textures generated corresponding to the respective texture images by the visual texture generating unit 524. By combining the visual textures, a plurality of frames of the shot unit image are restored as a whole.

The corrector 526 filters and corrects artifacts at the combined boundary of the plurality of combined visual textures. That is, the plurality of visual textures combined by the visual texture combining unit 525 are reconstructed as a simple sum, and defects may occur at the boundary between the visual textures. The correction unit 526 performs an elimination filtering operation on these defects to generate a final corrected reconstruction image.

As described above, the multi-texture image-based image processing method and apparatus according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways All or some of the embodiments may be selectively combined.

500; Multi-texture image-based image processing device
510; The encoding unit
511; A seed image selection unit 512; The feature point detector
513; A variable calculating unit 514; Texture image definition section
515; An approximation unit 516; Texture block definition section
517; Compression section
520; The decoding unit
521; Decompression unit 522; The texture image generation unit
523; A matching unit 523; The visual texture generator
524; A visual texture combining unit 525; [0040]

Claims (16)

Classifying an input image into a shot unit image and selecting one of a plurality of frames of the shot unit image as a seed image;
Detecting a plurality of feature points in the seed image;
Tracking the plurality of feature points in the plurality of frames of the shot unit image to calculate a space-time position conversion parameter for each of the feature points;
Defining a plurality of texture images using the feature points corresponding to the space-time location transformation parameters; And
And defining each of the plurality of texture images as a sum of a plurality of texture blocks that are outputs of a texture synthesis filter to which an image excitation signal is input. The method according to claim 1,
Wherein the image excitation signal is represented by a two-dimensional Gaussian function. The method according to claim 1,
Compressing the space-time location transformation parameter corresponding to each of the image excitation signal, the variable of the texture synthesis filter, and each of the plurality of texture images of each of the plurality of texture blocks defining the plurality of texture images The image processing method comprising: The method of claim 3,
Wherein the compressing comprises:
Wherein the image excitation signal, the variable of the texture synthesis filter, and the space-time location transformation parameter are compressed by a bit stream compression method. Claim 1
In the plurality of texture images, texture images having space-time location transformation variables having similarity calculated within a predetermined threshold by calculating a correlation characteristic of a texture image signal are summed and approximated into one texture image The method comprising the steps of: (a) inputting a plurality of texture images; The method according to claim 1,
Wherein the detecting of the plurality of feature points comprises:
Wherein a point having a variation amount equal to or greater than a predetermined value is detected as the feature point in the plurality of frames. The method of claim 3,
Decompressing the compressed image excitation signal and the variable of the texture synthesis filter and the space-time location conversion variable corresponding to each of the texture images;
Generating the plurality of texture blocks using the image excitation signal and the parameters of the texture synthesis filter, and summing the plurality of texture blocks to generate the texture image;
Matching the texture image with the space-time location transformation parameter corresponding to the texture image;
Generating a visual texture using the texture image and the space-time location transformation parameter; And
And combining the visual textures generated corresponding to each of the texture images. The method of claim 7,
Further comprising filtering and correcting artifacts at the joint boundaries of the visual textures. A seed image selection unit for classifying an input image into a shot unit image and selecting one of a plurality of frames of the shot unit image as a seed image;
A feature point detector for detecting a plurality of feature points in the seed image;
A variable calculation unit for tracking the plurality of feature points in the plurality of frames of the shot unit image to calculate a space-time location conversion parameter for each of the feature points;
A texture image defining unit that defines a plurality of texture images using the feature points corresponding to the space-time location conversion parameters; And
And a texture block defining unit that defines each of the plurality of texture images as a sum of a plurality of texture blocks that are outputs of a texture synthesis filter that receives an image excitation signal. The method of claim 9,
Wherein the image excitation signal is represented by a two-dimensional Gaussian function. The method of claim 9,
And a compression unit for compressing the space-time location conversion parameter corresponding to each of the image excitation signal, the texture synthesis filter parameter, and the plurality of texture images of each of the plurality of texture blocks defining the plurality of texture images The image processing apparatus comprising: The method of claim 11,
Wherein the compression unit comprises:
Wherein the compression unit compresses the image excitation signal, the variable of the texture synthesis filter, and the space-time location conversion parameter using a bit stream compression method. The method of claim 9,
In the plurality of texture images, texture images having space-time location transformation variables having similarity calculated within a predetermined threshold by calculating a correlation characteristic of a texture image signal are summed and approximated into one texture image Wherein the image processing unit further includes an approximation unit. The method of claim 9,
Wherein the minutiae point detecting unit comprises:
Wherein a point having a variation amount equal to or greater than a predetermined value is detected as the feature point in the plurality of frames. The method of claim 11,
A decompression unit decompressing the compressed image excitation signal, the parameters of the texture synthesis filter, and the space-time location conversion variable corresponding to each of the texture images;
A texture image generation unit for generating the texture blocks by using the image excitation signal and the parameters of the texture synthesis filter, and adding the plurality of texture blocks to generate the texture image;
A matching unit for matching the texture image with the space-time location conversion parameter corresponding to the texture image;
A visual texture generator for generating a visual texture using the texture image and the space-time location transformation parameter; And
And a visual texture combining unit for combining the visual textures generated corresponding to the respective texture images. 16. The method of claim 15,
Further comprising a correction unit for filtering and correcting artifacts at a boundary of the visual textures.

Images