KR20180088188A - Method and apparatus for adaptive image transformation - Google Patents

Abstract

A method and apparatus for performing adaptive transformations on an image are disclosed. In the disclosed method and apparatus for transforming an image according to an embodiment of the present invention, a restoration frame in which an image frame before a current image frame is restored is divided into a plurality of blocks. Each block is classified into one or more classes based on the distribution of frequency components of each block in the plurality of blocks to determine a block set for each class. A base vector for each class is generated with regard to the block set for each class. The base vector to be sued for transforming the current block in the current image frame is selected among the base vectors for each class. The current block is transformed by using the selected base vector. Accordingly, the present invention can perform the adaptive transformation on the image by selecting the base vector to be used in a block to be transformed.

Description

[0001] METHOD AND APPARATUS FOR ADAPTIVE IMAGE TRANSFORMATION [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for converting an image, and more particularly, to a technique for improving compression performance by applying an adaptive transformation to an image.

BACKGROUND ART [0002] With the development and dissemination of hardware capable of reproducing and storing high-resolution or high-definition video content, there is an increasing need for a video codec that effectively encodes or decodes high-definition or high-definition video content. Frequency conversion and inverse transformation of video codec compression techniques are indispensable technologies.

Typical standard compression schemes include MPEG2, H.264 / AVC, and HEVC. As a method of image compression, a motion estimation and compensation process is performed to reduce the temporal correlation between consecutive images and the spatial correlation within the image, and a difference between the current image and the current image is generated after the current image is generated from the past image. At this time, in the conversion step of a general image compression technique, discrete cosine transform (hereinafter referred to as " DCT ") operation is performed on image data input in block units. For example, a pixel component of a block is projected using a cosine eigen vector stored in advance as a basis vector in order to perform DCT transform.

On the other hand, the transformation indicating the optimum performance among the image transformation methods is known as Karhuhen-Loeve Transform (hereinafter referred to as " KLT "). The KLT transform method is a method of using an eigenvector that best represents the signal based on the statistical characteristics of the signal. The KLT transformation has an advantage that the image can be adaptively transformed and exhibits excellent energy cohesion effect. However, since the KLT transform needs to transmit the eigenvectors separately, the advantage of improving compression performance is offset by the occurrence of overhead. In addition, since KLT conversion requires calculation of an eigenvector for each image frame, it is difficult to apply it to an apparatus for encoding and decoding an image in real time due to increased complexity. Therefore, despite the excellent energy cohesion performance of KLT, DCT conversion has been used in a general compression method due to overhead and complexity problems.

Accordingly, there is a need for a technique capable of reducing overhead and complexity while performing adaptive conversion to an image.

A problem to be solved is to provide a method and apparatus for generating a basis vector for each block set classified from a reconstructed image frame, selecting a base vector to be used for a block to be transformed, and performing adaptive transformation on the image.

According to an aspect of the present invention, there is provided a method of transforming an image, the method comprising: dividing a reconstructed frame reconstructed from an image frame preceding a current image frame into a plurality of blocks; Classifying each block into one or more classes based on a distribution of frequency components of each block in the plurality of blocks to determine a block set for each class; Generating a class-basis vector for the class-specific block set; Selecting a base vector to be used for transforming a current block in a current image frame among the basis vectors for each class; And transforming the current block using the selected basis vector.

In addition, in the image transform method according to an embodiment, the class-basis vector may be generated based on the reconstructed frame stored in the reconstructed picture buffer.

Also, in the image transform method according to an exemplary embodiment, the reconstructed frame may be a scene transformed image frame of the image frames preceding the current image frame, DCT transformed and quantized, inverse quantized, and DCT inversely transformed.

The method may further include determining whether the current image frame is a scene-switched image frame, and determining whether the current image frame is a scene-switched image frame In this case, the current block may be transformed based on a transform vector embedded in the image transform apparatus.

Also, in the image transform method according to an embodiment, the determining may include comparing a pixel brightness difference value or a brightness histogram difference value between the current image frame and an image frame before the current image frame to a predetermined threshold value . ≪ / RTI >

In addition, in the image transform method according to an exemplary embodiment, the step of determining a block set for each class may include: determining a position of a coefficient having a maximum absolute value among frequency components excluding a direct current (DC) ; And classifying blocks of the plurality of blocks having the same index corresponding to the searched position into the same class.

In addition, in the image transform method according to an exemplary embodiment, the step of generating the basis vector for each class may include: calculating a covariance matrix of the block set for each class; And generating the class-basis vector based on an eigenvector of the covariance matrix of the class-specific block set and an eigenvalue of each class.

Also, in the image transform method according to an exemplary embodiment, the basis vector used for transforming the current block may be selected as a basis vector of a class that minimizes a difference in brightness value from the current block among the basis vectors for each class.

In addition, in the image transform method according to an embodiment, the basis vector used for transforming the current block may be selected as the basis vector of the class corresponding to the current block.

According to an embodiment of the present invention, there is provided an image conversion apparatus comprising: a receiver for receiving a current image frame to perform conversion; Dividing a restored frame in which an image frame before the current image frame is reconstructed into a plurality of blocks, classifying each block into one or more classes based on distribution of frequency components of each block in the plurality of blocks, A control unit for generating a class-basis vector for the class-specific block set and selecting a basis vector to be used for transforming a current block in the current image frame, among the class-based basis vectors; And a transform unit for transforming the current block using the selected basis vector.

In addition, in the image transform apparatus according to the embodiment, the class-based basis vector may be generated based on the reconstructed frame stored in the reconstructed picture buffer .

Also, in the image transforming apparatus according to an embodiment, the reconstructed frame may be one obtained by rearranging DCT and DCT transformed and quantized image frames after the current image frame, and then performing inverse quantization and inverse DCT.

In addition, in the image conversion apparatus according to an embodiment, the controller determines whether the current image frame is a scene-switched image frame, and when the current image frame is determined as a scene-switched image frame, Can be converted based on the conversion vector embedded in the image conversion apparatus.

In addition, in the image conversion apparatus according to an exemplary embodiment, the controller may compare a pixel brightness difference value or a brightness histogram difference value between the current image frame and an image frame before the current image frame with a predetermined threshold value.

Also, in the image transforming apparatus according to an exemplary embodiment, the controller searches for a position of a coefficient having a maximum absolute value among frequency components excluding a direct current (DC) component for each block, Blocks having the same index corresponding to the searched position may be classified into the same class.

Also, in the image transforming apparatus according to an embodiment, the controller calculates a covariance matrix of the block set for each class, calculates an eigenvector of a covariance matrix of the block set for each class, Value of the class-basis vector.

In addition, in the image transforming apparatus according to an embodiment, the basis vector used for transforming the current block may be selected as a basis vector of a class that minimizes a difference in brightness value from the current block among the basis vectors for each class.

In addition, in the image transform apparatus according to the embodiment, the basis vector used for transforming the current block may be selected as the basis vector of the class corresponding to the current block.

FIG. 1 shows a schematic block diagram of an image conversion apparatus 100 according to an embodiment.
FIG. 2 is a block diagram of an image encoding apparatus 200 according to an embodiment.
FIG. 3 is a block diagram of an image decoding apparatus 300 according to an embodiment.
Figure 4 is a diagram illustrating various basis patterns for NxN blocks.
5 is a view for explaining a method of generating a class-basis vector according to an embodiment.
6 is a diagram for explaining a method of classifying a block based on a distribution of frequency components according to an embodiment.
FIG. 7 is a diagram illustrating features of a block classified according to an embodiment.
8 is a view for explaining a method of generating a class-basis vector according to an embodiment.
9 is a diagram for explaining a method of selecting a basis vector to be used for transforming a current block according to an embodiment.
10 is a flowchart illustrating an image conversion method according to an embodiment.
11 is a flowchart for explaining an image conversion method according to another embodiment.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms " part, " " module, " and the like, as used herein, refer to a unit that processes at least one function or operation, and may be implemented in hardware or software or a combination of hardware and software.

As used herein, the phrase " one embodiment " or " an embodiment " means features, structures, features, and the like described in connection with the embodiments included in at least one embodiment. Therefore, the appearances of the phrase " in one embodiment " or " in an embodiment " appearing in various places throughout this specification are not necessarily all referring to the same embodiment.

In the present specification, coding may be interpreted as encoding or decoding as occasion demands, and information is understood to include both values, parameters, coefficients, elements, and the like .

A 'slice', a frame, or the like refers to a unit of a picture in a real video signal coding. The 'picture' or 'picture' And may be used in combination with a picture if necessary.

'Pixel', 'pixel' or 'pel' means the smallest unit of an image. In addition, 'sample' can be used as a term indicating the value of a specific pixel. The sample can be divided into luminance (Luma) and chroma (chroma) components, but generally it can be used as a term including all of them. The chrominance component represents a difference between predetermined colors, and is generally composed of Cb and Cr.

A 'unit' refers to a specific unit of an image processing unit or image, such as a coding unit (CU), a prediction unit (PU), and a conversion unit (TU) Or " area " and the like. The block may also be used as a term indicating a set of samples or transform coefficients consisting of M columns and N rows.

The 'current block' may refer to a block of an image to be encoded or decoded.

The 'neighboring block' indicates at least one block that is coded or decoded neighboring the current block. For example, a neighboring block may be located at the top of the current block, the upper right of the current block, the left side of the current block, or the upper left or lower left of the current block. It may also include temporally neighboring blocks as well as spatially neighboring blocks. For example, a co-located block that is a temporally neighboring block may include a block at the same position as the current block of the reference picture or a neighboring block adjacent to the block at the same position.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring now to FIGS. 1-11, a method and apparatus for transforming an image using an adaptive transformation base on an image according to various embodiments is disclosed.

FIG. 1 shows a schematic block diagram of an image conversion apparatus 100 according to an embodiment. The image conversion apparatus 100 includes a receiving unit 110, a control unit 120, and a conversion unit 130. The controller 120 includes a scene change determining unit 122, a basis vector generating unit 124, and a basis vector selecting unit 126.

The image conversion apparatus 100 according to an exemplary embodiment may be included in an encoder or a decoder to perform operations for converting or inversely converting an image.

According to one embodiment, the receiving unit 110 receives a current image frame to perform conversion. For example, the receiving unit 110 can receive image data of a predetermined data unit of an image. The image data of a predetermined data unit may be an image frame, or may be data of a square block or a rectangular block obtained by dividing an image frame into a predetermined size. On the other hand, when the image converting apparatus 100 is included in the decoder, the receiving unit 110 can receive the bit stream compressed with the image frame.

The controller 120 according to an exemplary embodiment performs operations to determine an adaptive transformation base on an image frame prior to performing a transformation or an inverse transformation on the current image frame. Adaptive to an image frame means that a transform function is newly defined according to the input image frame depending on the statistical characteristics of the input signal. For example, an adaptive conversion base for an image frame may refer to a KLT transformed basis.

The transformation algorithms widely used in image compression technology can be divided into two types: block-based transformation and image-based transformation. Examples of block-based transformations include KLT transforms and DCT transforms. Here, it is important to increase the compression effect by reducing the number of transform coefficients obtained through the transform. In other words, in order to extract the minimum loss in the post-conversion quantization step, it is important to distribute the important information of the frame intensively to several coefficients after the conversion. The conversion algorithm that satisfies this condition is KLT. Although KLT is theoretically known as the ideal conversion method that is optimized most for the compression due to its excellent energy convergence characteristics of the image signal, it is required to newly define the conversion function according to the input image depending on the statistical characteristics of the input signal, There is a problem that it is difficult to apply the present invention to an apparatus for encoding and decoding an image in real time.

A method of reducing overhead and complexity while enabling adaptive conversion and inverse transformation of an image through the operation of the controller 120 according to an embodiment is proposed. Hereinafter, a specific operation performed by the scene change determining unit 122, the basis vector generating unit 124, and the basis vector selecting unit 126 in the control unit 120 will be described.

According to an exemplary embodiment, the scene change determination unit 122 determines whether the current image frame received through the receiver 110 is a scene change image frame or an image frame that has not been changed. Computing the basis vectors every frame increases the overhead and may increase the bit-rate to transmit it. Therefore, it is necessary to determine whether the current image frame is a scene-switched image frame in order to use the basis vector of the previous image frame as a basis vector in the conversion of the currently input image frame. If a scene change occurs, only the basis vectors of the first scene after the scene change are obtained, and the base vectors of the subsequent frames belonging to the same scene group can be transformed using the base vectors of the first frame.

Various scene transition detection techniques can be utilized as a method for determining whether a scene change is to be performed or not. As a typical method for determining whether or not the scene change is to be performed, a method of comparing a pixel brightness difference value or a brightness histogram difference value between a current image frame and a previous image frame with a predetermined threshold value may be used.

For example, the following Equation 1 can be used to derive the pixel brightness difference value.

In Equation _{1, d DC (X, Y} ) represents a pixel brightness difference between the X-frame and the Y frame, C _X (m, n) and C _Y (m, n) is the X-frame and a Y frame, respectively It is a value for indicating pixel brightness. X frame and the Y-frame is divided into 8x8 blocks, respectively there may represent each block of m columns and n rows, C _X (m, n) and C _Y (m, n) is the following formula (2)) and ( 3 < / RTI >

The scene change determining unit 122 may compare the pixel brightness difference value derived through Equations 1 to 3 with a predetermined threshold value Th. If the pixel brightness difference value is greater than the predetermined threshold value, the scene change determination unit 122 may determine that the scene change has been performed. If the pixel brightness difference value is smaller than or equal to the predetermined threshold value, .

을 도출할 수도 있다.For example, the scene change discrimination unit 122 may calculate the histogram difference value between the frames X and Y

.

In Equation (4), H _X (j) denotes an empty value of the histogram of the X frame, H _Y (j) denotes the bin value of the histogram of the Y frame, and K denotes the total number of the beans.

The scene change determining unit 122 may compare the histogram difference value derived through Equation (4) with a predetermined threshold value Th. If the histogram difference value is greater than the predetermined threshold value, it is possible to determine that the scene change has been performed. If the histogram difference value is smaller than or equal to the predetermined threshold value, the scene change determination section 122 can determine that the scene has not been changed .

In addition, the scene change determining unit 122 may determine whether the current image frame is a scene-switched image frame using various known scene change detection techniques.

The basis vector generating unit 124 according to an embodiment generates a basis vector that can represent a block set in a predetermined block unit (for example, 32x32, 16x16, 8x8, etc.). The basis vector generator 124 according to an embodiment generates a basis vector using a reconstructed frame rather than an input image frame. Thus, the basis vectors can be acquired on-line within the device without having to be transmitted from the outside. A detailed process of generating the basis vector by the basis vector generating unit 124 will be described later with reference to FIGs. 5 to 8. FIG.

The basis vector selector 126 according to an embodiment selects a basis vector that can represent the current block to be transformed among the at least one basis vector generated by the basis vector generator 124. [ A specific method by which the basis vector selection unit 126 selects a basis vector will be described later with reference to FIG.

The transforming unit 130 transforms or inversely transforms the current block using the basis vector selected by the basis vector selecting unit 126 of the controller 120. [

FIG. 2 is a block diagram of an image encoding apparatus 200 according to an embodiment.

The image encoding apparatus 200 according to an embodiment performs operations to encode the converted image in the image conversion apparatus 100 of FIG. The image encoding apparatus 200 includes a receiver 210, a controller 220, a transform unit 230, a reconstruction picture buffer 235, an inter prediction unit 240, an intra prediction unit 245, An entropy encoding unit 255, an inverse quantization unit 260, an inverse transform unit 265, a deblocking unit 270, and an SAO performing unit 275. The control unit 220 may include a scene change determining unit 222, a basis vector generating unit 224, and a basis vector selecting unit 226. The receiving unit 210, the control unit 220 and the converting unit 230 of FIG. 2 may correspond to the receiving unit 110, the control unit 120, and the converting unit 130 of FIG. 1, respectively. The scene change determination unit 222, the basis vector generation unit 224, and the basis vector selection unit 226 shown in FIG. 2 correspond to the scene change determination unit 122, the basis vector generation unit 124, And may correspond to the vector selection unit 126. Therefore, the operation of the scene change determining unit 222, the basis vector generating unit 224, and the basis vector selecting unit 226 shown in FIG. 2 is the same as that described above with reference to FIG. 1, and therefore a detailed description thereof will be omitted.

The intra prediction unit 245 performs intra prediction on the prediction unit of the intra prediction mode for the intra prediction mode of the input image 205. The inter prediction unit 240 predicts the input image 205 And the reference image obtained in the reconstructed picture buffer 235. The inter- The input image 205 may be divided into a maximum encoding unit and then sequentially encoded. At this time, encoding can be performed on the encoding unit in which the maximum encoding unit is divided into a tree structure.

The residue data is generated by calculating the difference between the data of the encoding unit to be encoded of the input image 205 and the prediction data of the encoding unit of each mode output from the intra prediction unit 245 or the inter prediction unit 240 do. The residue data is output as a transform coefficient quantized by the transform unit 230 and the quantization unit 250. The quantized transform coefficients are restored to residue data in the spatial domain through the inverse quantization unit 260 and the inverse transform unit 265. The residue data of the reconstructed spatial region is added to the prediction data for the coding unit of each mode output from the intra prediction unit 245 or the inter prediction unit 240 to generate spatial data for the coding unit of the input image 205 Data is restored. The reconstructed spatial domain data is generated as a reconstructed image through the deblocking unit 270 and the SAO performing unit 275. The generated restored image is stored in the restored picture buffer 235. The restored images stored in the restored picture buffer 235 may be used as reference images for inter prediction of other images. In addition, a basis vector for transforming the next image frame may be generated from the reconstructed images stored in the reconstructed picture buffer 235. A detailed operation of generating the base vector using the reconstructed image stored in the reconstructed picture buffer 235 by the basis vector generation unit 224 in the control unit 220 will be described later with reference to FIG. 5 through FIG. The transform coefficients quantized by the transforming unit 230 and the quantizing unit 250 may be output to the bitstream 280 via the entropy encoding unit 255. [

The inter prediction unit 240, the intra prediction unit 245, the transform unit 230, the quantization unit 250, the entropy coding unit 255, the inverse quantization unit 255, The inverse transform unit 260, the inverse transform unit 265, the deblocking unit 270, and the SAO performing unit 275 can perform operations based on the respective encoding units among the encoding units according to the tree structure for each maximum encoding unit.

In particular, the intra-prediction unit 245 and the inter-prediction unit 240 determine the partition mode and the prediction mode of each coding unit among the coding units according to the tree structure in consideration of the maximum size and the maximum depth of the current maximum coding unit , The transform unit 230 can determine whether or not the transform unit according to the quadtree in each encoding unit among the encoding units according to the tree structure is divided.

FIG. 3 is a block diagram of an image decoding apparatus 300 according to an embodiment.

The image decoding apparatus 300 according to an embodiment performs operations for decoding an image. The image decoding apparatus 300 can perform decoding operations based on the image inversely transformed by the image transforming apparatus 100 of FIG. The image decoding apparatus 300 according to an embodiment includes an entropy decoding unit 310, an inverse quantization unit 315, a controller 320, an inverse transform unit 330, a reconstruction picture buffer 335, an inter prediction unit 340, An intraprediction unit 345, a deblocking unit 350, and an SAO performing unit 355. The control unit 320 may include a scene change determining unit 322, a basis vector generating unit 324, and a basis vector selecting unit 326. The control unit 320 and the inverse transform unit 330 of FIG. 3 may correspond to the control unit 120 and the transform unit 130 of FIG. 1, respectively. The scene change determining unit 322, the basis vector generating unit 324 and the basis vector selecting unit 326 shown in FIG. 3 correspond to the scene change determining unit 122, the basis vector generating unit 124, And may correspond to the vector selection unit 126. Therefore, the operation of the scene change determination unit 322, the basis vector generation unit 324, and the basis vector selection unit 326 of FIG. 3 are the same as those described above with reference to FIG. 1, and therefore detailed description thereof will be omitted.

The entropy decoding unit 310 obtains the encoded image data to be decoded and the encoding information necessary for decoding from the bitstream 305. The encoded image data is a quantized transform coefficient, and the inverse quantization unit 315 and the inverse transform unit 330 reconstruct the residue data from the quantized transform coefficients.

The intraprediction unit 345 performs intraprediction on a prediction unit basis for an intra-mode encoding unit. The inter-prediction unit 340 performs inter-prediction using the reference picture obtained in the reconstructed picture buffer 335 for each inter-mode coding unit of the current picture, on a predictive unit basis.

The spatial data for the coding unit of the current image is restored by adding the prediction data and the residue data for the coding unit of each mode through the intra prediction unit 345 or the inter prediction unit 340, Data may be output as a reconstructed image through the deblocking unit 350 and the SAO performing unit 355. [ The restored images stored in the restored picture buffer 335 may be output as a reference image. In addition, a basis vector for inverse transformation of the next image frame may be generated from reconstructed images stored in the reconstructed picture buffer 335. The specific operation of generating the base vector using the reconstructed image stored in the reconstructed picture buffer 335 by the basis vector generating unit 324 in the control unit 320 will be described later with reference to FIG. 5 through FIG.

The entropy decoding unit 310, the inverse quantization unit 315, the inverse transform unit 330, the intra prediction unit 345, the inter prediction unit 340, the deblocking unit 350, And the SAO performing unit 355 can perform an operation based on each encoding unit among the encoding units according to the tree structure for each maximum encoding unit.

In particular, the intra prediction unit 345 and the inter prediction unit 340 determine a partition mode and a prediction mode for each coding unit among the coding units according to the tree structure, and the inverse transform unit 330 performs a quad- It is possible to determine whether or not the conversion unit according to < RTI ID = 0.0 >

The inter-prediction unit 240 of FIG. 2 and the inter-prediction unit 340 of FIG. 3 perform intra-prediction in the reference picture to generate a sub-pixel sample value when the motion vector indicates a sub- Interpolation filtering can be performed on the reference samples in the integer pixel unit.

Figure 4 is a diagram illustrating various basis patterns for NxN blocks.

Referring to FIG. 4, there may be one or more base patterns for the NxN block 415 in the image frame 410. The basis pattern is expressed as a basis vector, and a basis vector is a set of vectors that are independent of each other and can generate all the coordinates of the n-dimensional space through a primary combination {V1, V2, ... , Vn}. For example, when the pixel values in the NxN block 415 are connected in a line and expressed as an n-dimensional vector, the n-dimensional space may have a maximum of n linearly independent vectors, and n The vectors that are independent of each other act as a basis for generating an n-dimensional space. 4, the basis pattern for the NxN block 415 may be represented as a DCT basis vector 420, a KLT basis vector 430, or the like.

Hereinafter, a DCT and KLT conversion process for deriving the DCT basis vector 420 and the KLT basis vector 430 will be described.

The DCT and KLT classify the input image frame 410 into low frequency and high frequency components. As a result of the conversion, energy is concentrated on the low frequency component, so that the high frequency component can be easily removed during the quantization. Since human vision is more sensitive to loss of low frequency components than loss of high frequency components, image compression is possible without deteriorating large image quality even if high frequency components are removed.

More specifically, the DCT performs a column-wise transform and a row-wise transform on the NxN block to generate an NxN coefficient block. Assuming that the NxN block is X, the NxN DCT transformation matrix is A, and the Y is NxN coefficient block, the forward DCT is defined as Y = A X A ^T. The first matrix multiplication A · X corresponds to performing a one-dimensional DCT on each column of X, which is an N × N block, and multiplying A · X by a binary matrix A ^T is performed for each row of X This corresponds to performing one-dimensional DCT.

Α _ij component of the (i, j) of the NxN DCT transform matrix A is the same as Equation 5.

When DCT is performed on the NxN block, an NxN coefficient block composed of DCT coefficients is generated. These DCT coefficients correspond to the weights of the DCT basis vector 420 pattern set as shown in FIG. The pattern of the DCT basis vector 420 consists of a combination of a horizontal cosine function and a vertical cosine function. The image block may be reconstructed by multiplying the DCT coefficients corresponding to each pattern of the DCT basis vector 420 and then combining the respective patterns.

KLT is one of the orthogonal transformation coding methods, and is a method in which the transformed base is optimized in accordance with the characteristics of the input signal among the transform coding methods for transforming the image signal on the time axis into a frequency axis and encoding the transformed image. Since the eigenmatrix of the blocked image signal is obtained and the orthogonal transformation is performed using the eigenmatrix, there is no correlation between the transformed signals. KLT is based on the statistical properties of vector representations.

KLT consists of calculating a covariance matrix, calculating a basis vector from the calculated covariance matrix, and calculating the transform coefficients in blocks of the input image frame. That is, KLT obtains a covariance matrix of input vectors and uses the eigenvectors as a transformation matrix by aligning the eigenvectors according to the eigenvalues of the covariance matrix.

When there is a population of block-level pixel values having the form of Equation (6), the mean vector of the population is defined as Equation (7)

At this time, the covariance matrix of the vector population is expressed by Equation (8).

T represents the transposition of the vector, X is the n-th difference, and C _x and (X m _x ) (X m _x ) ^T are the nxn differences. The element c _ij of the matrix C _x is the covariance between the elements x _i and x _j of the vectors, and C _x has a symmetric real value. If x _i and x _j are not correlated, then the covariance of these elements is zero, and thus c _ij = c _ji = 0.

The covariance matrix is expressed by the following equation (9).

이고, Cx가 대칭행렬의 성질을 만족할 때, n개의 고유벡터는 항상 구할 수 있다. 고유벡터들과 대응되는 고유값 i의 크기에 따라 고유벡터들을 정렬하여 만든 새로운 변환 행렬 A를 이용하여 벡터 X를 Y로 변환하는 KLT 변환식을 만들 수 있다.here,

, And when Cx satisfies the property of the symmetric matrix, n eigenvectors can always be obtained. A KLT transformation formula for transforming the vector X to Y can be made using a new transformation matrix A created by aligning the eigenvectors according to the magnitudes of the eigenvalues i corresponding to the eigenvectors.

변환을 통해 얻어진 Y 벡터는 다음과 같은 특성을 갖는다. Y 벡터의 평균은 0이 되고, 수학식 10과 같이 공분산 행렬은 C_X 로부터 구한 고유값만으로 이루어진 대각 행렬이 된다. 이는 KLT 가 이산변수를 상관관계가 0인 계수로 변환한다는 것을 의미하며, 이 때, C_X 및 C_Y 는 동일한 고유값과 고유 벡터를 갖는다.

The Y vector obtained through the transformation has the following characteristics. The mean of the Y vector becomes 0, and the covariance matrix becomes a diagonal matrix consisting of eigenvalues obtained from C _X as shown in Equation (10). This means that KLT converts the discrete variable to a coefficient with a correlation of zero, where C _X and C _Y have the same eigenvalue and eigenvector.

Equations (11) and (12) below are derived from the inverse transformation of KLT and show that only eigenvectors corresponding to K largest eigenvalues can be restored with only small errors.

The error of the mean square of x and hat x can be expressed by Equation (13) below.

Here, in order to appropriately select the size of K, the relative ratio of the specific eigenvalues to the sum of the total eigenvalues is used. The contribution d _i of any unique solution λ _i is given by:

The appropriate K is determined according to the number of relatively large values of the contribution d _i values calculated through Equation (14). As can be seen from this process, the eigenvector of the covariance matrix obtained from the vector obtained in the image frame is the KLT basis vector 430. [

As described above, the KLT transformation method can adaptively convert the image using a eigenvector that can best represent the signal based on the statistical characteristics of the signal. However, calculating an eigenvalue with a kind of eigenvector for all blocks in an image frame has a problem that the adaptability is degraded for blocks having various characteristics. Accordingly, there is a need for a method of reducing overhead while calculating a plurality of kinds of basis vectors adaptively according to an input image for efficient conversion. In this specification, instead of calculating a basis vector for all blocks in an image frame, a method of calculating a basis vector on a block set classified by classifying blocks in an image frame according to a certain standard is proposed.

5 is a view for explaining a method of generating a class-basis vector according to an embodiment.

The class-basis vectors according to an embodiment may be generated by the basis vector generating units 124, 224, and 324 in the controllers 120, 220, and 320, respectively. The basis vector generating units 124, 224 and 324 according to an embodiment divide the image frame 510 into a predetermined block unit (for example, 32x32, 16x16, 8x8, etc.). In this case, the image frame 510 may be a restored frame of the scene-switched image frame. That is, the image frame 510 may be a restored frame stored in the restored picture buffers 235 and 335. The base vector generation units 124, 224, and 324 classify the divided blocks according to classes and perform DCT transform 520 for each of the divided blocks to generate a block set 530 for each class. Here, the divided blocks can be classified based on the distribution of frequency components of each block. For example, the criterion by which each block is classified may be based on a position of a coefficient having a maximum absolute value among frequency components of each block. That is, the blocks having the same index corresponding to the position of the coefficient having the maximum absolute value can be classified into the same class. At this time, since the direct current (DC) component of each block always represents a large value on the basis of the average energy, only the frequency component excluding the DC component can be considered. Therefore, in the case of the 8x8 block, there are 63 indices excluding the index corresponding to the DC component position, so that 8x8 blocks can be classified into a maximum of 63 classes. According to one embodiment, the basis vector generators 124, 224, and 324 generate a class-specific basis vector 540 for each generated block set 530 of classes. In the case of 8x8 block, up to 63 base class vectors 540 can be generated.

A detailed process of classifying the blocks into classes will be described later with reference to FIGS. 6 and 7, and a detailed process of generating the class-basis vectors 540 will be described later with reference to FIG.

6 is a diagram for explaining a method of classifying a block based on a distribution of frequency components according to an embodiment.

According to an embodiment, the basis vector generating units 124, 224, and 324 in the controllers 120, 220, and 320 may classify blocks of image frames to determine a block set for each class. Referring to FIG. 6, the distribution of frequency components for any 8x8 block 600 segmented from an image frame is illustrated. The basis vector generating units 124, 224, and 324 according to one embodiment determine the class of the block 600 based on the distribution of the frequency components of the block 600.

First, the base vector generating units 124, 224, and 324 search for positions of coefficients having the maximum absolute value among the frequency components excluding the DC component of the block 600. Referring to FIG. 6, there are respective indices corresponding to each position of the frequency component in block 600. For example, an index corresponding to a position having a value of '16 .9828' may be 'Index 2', and an index corresponding to a position having a value of '0.2262' may be an index 3 '. In the case of block 600 of FIG. 6, the position of the coefficient having the maximum absolute value among the frequency components is a position having a value of '16 .9828'. Accordingly, the index 610 representing the block 600 is 'index 2' which is an index corresponding to a position having a value of '16 .9828'.

The basis vector generating units 124, 224, and 324 classify each block according to an index representative of each block. That is, blocks having the same representative index can be classified into the same class. In the case of the 8x8 block, there are 63 indices excluding the index corresponding to the DC component position. Therefore, 8x8 blocks can be classified into a maximum of 63 classes.

FIG. 7 is a diagram illustrating features of a block classified according to an embodiment.

As described above with reference to FIG. 6, the basis vector generating units 124, 224, and 324 according to the embodiment may classify the classes into one or more classes based on the frequency components of the blocks. For example, a class according to one embodiment can be classified into a class including a low frequency block 710, a class including an intermediate frequency block 720, a class including a high frequency block 730, and the like. For example, the low frequency block 710 may correspond to a flat region of an image frame, the intermediate frequency block 720 may correspond to an edge region of an image frame, May correspond to a texture or noise region of an image frame. In the above-described embodiment, the class including the low-frequency block 710 may be a set of blocks whose indexes corresponding to the position of the coefficient having the maximum absolute value of the frequency have low values. On the other hand, the class including the high-frequency block 730 may be a set of blocks whose indexes corresponding to the position of the coefficient having the maximum absolute value of frequency have a high value.

In the above-described embodiment, the block group for each class is classified into a high frequency, an intermediate frequency, a low frequency, and the like. For example, a class according to an embodiment can be classified into a maximum of 63 classes for 8x8 blocks.

The reason for determining the block set for each class is that it is effective to reduce the overhead and the bit rate by calculating the basis vector for each block set having a similar frequency distribution instead of calculating the basis vector for each block. Hereinafter, a process of generating a class-basis vector will be described.

8 is a view for explaining a method of generating a class-basis vector according to an embodiment.

The basis vector generation units 124, 224, and 324 according to the embodiment may perform Principal Component Analysis (hereinafter, referred to as "PCA") on a determined block set for each class to generate a basis vector for each class .

PCA is a technique for finding the principal component of distributed data. Specifically, referring to FIG. 8, n point data (x1, y1), (x2, y2), ..., , (xn, yn) are distributed in an elliptical shape, the characteristics of this data distribution 810 can be described by two vectors 812 and 814. Knowing the direction and size of the two vectors 812 and 814 can be the simplest and most effective way to grasp what the data distribution 810 is. In other words, PCA is not a component for each piece of data, but a method of analyzing the main component of data distribution when various data are gathered to form one distribution.

Here, the principal component means a direction vector having the largest variance of data in the direction. Among the two vectors 812 and 814 in FIG. 8, the direction in which the dispersion of data along the vector 812 direction is largest and the direction in which the dispersion of data next to the vector 812 is largest is the vector 814).

For example, if the number of blocks belonging to an arbitrary class is N, and the size of each block is 8x8, the component values of the respective blocks are connected in series to form a vector. Then, each block is divided into 8x8 = . That is, each block corresponds to a point (coordinate) in the 64-dimensional space. If PCA is performed based on N 64-dimensional data, the number of principal component vectors (i.e., eigenvectors) equal to the number of dimensions of the data can be obtained. The principal component vectors thus obtained are perpendicular to each other. This means that principal component vectors can serve as a basis for generating a 64-dimensional space. That is, the principal component vectors obtained by PCA are e1, e2, ... , e64, any 64-dimensional data x is x = c1e1 + c2e2 + ... + c64e64 can be expressed as a linear combination of e _i . In this case, the constant coefficients c _i can be computed as the inner product of x and e _i , ie, c _i = x · e _i . Thus, expressing data of a certain data set as a primary combination of their principal component vectors is called KLT.

In other words, the process of calculating the principal component vector by performing the PCA is the same as the calculation process of the KLT basis vector described above with reference to FIG. That is, as a method of generating a class-basis vector for a block-by-class block set, the calculation process of Equations (5) to (14) described above can be directly applied.

Therefore, when a covariance matrix for a class-specific block set R _xx is expressed as shown in Equation (15), a relationship between R _xx and an eigenvector and an eigenvalue can be expressed by Equation (16).

는 i 번째 클래스의 고유 값(eigen value),

는 i 번째 클래스의 고유 벡터(eigen vector), 즉 기저 벡터를 의미한다.In Equation (16), i denotes a class number,

Is the eigenvalue of the ith class,

Denotes an eigenvector of an i-th class, that is, a basis vector.

The base vector generation units 124, 224, and 324 according to an exemplary embodiment may generate and update class basis vectors for a block set for each class. The updated class-basis vectors can be used for transforming the input image frames until the next scene change is made.

9 is a diagram for explaining a method of selecting a basis vector to be used for transforming a current block according to an embodiment.

The basis vector selection unit 126, 226, and 326 according to an exemplary embodiment selects a basis vector to be used for transforming or inversely transforming a current block in a current image frame among base vectors generated by the generated class.

In the case of the image coding apparatus 200, the basis vector selection unit 226 performs an operation of finding a basis vector that can represent the current block. In the case of the video decoding apparatus 300, the base vector selection unit 326 selects a base vector necessary for inverse transformation of the current block, based on the base vector selection unit 226 of the video encoding apparatus 200, . For example, in order for the image decoding apparatus 300 to invert the 8x8 block, it is necessary to know which class of the maximum of 63 classes the current block has been converted to. Accordingly, the information about the class to which the current block corresponds can be signaled from the video encoding device 200 to the video decoding device 300. [ Information on the class to which the current block corresponds can be generated as additional information in the image encoding apparatus 200 and transmitted to the image decoding apparatus 300.

The base vector used for transforming or inverse transforming the current block can be selected based on whether or not the current image frame including the current block is a scene-switched image.

If the current video frame is the first frame of the scene change, the base vector generated in the previous video frame can not be used, so only the transform vector embedded in the video converter can be selected. For example, as the transform vector embedded in the image transform apparatus, a transform vector such as DCT or DST (Discrete Sine Transform) embedded in the image coding apparatus or the image decoding apparatus can be selected.

If the current image frame is a non-scene-switched frame, the basis vector selection unit 226 of the image encoding apparatus 200 generates a current block to be transformed, which is generated from the reconstructed frame of the previous image, It is possible to select the base vector of the class that can be used. According to another embodiment, the basis vector selection unit 226 may further include one DCT basis vector in addition to the class-specific basis vectors (maximum of 63) generated from the reconstructed frame of the previous image, It is also possible to select a base vector of a class that can represent a block.

를 사용하여 구한 고유 값이고, 수학식 18과 같이, Y_i가 역변환을 통해 얻어진 현재 블록의 밝기 값이라고 한다.As shown in Equation 17, if y _i is the eigenvector of class _i for the current (original) block Y _org

, And Y _i is the brightness value of the current block obtained through inverse transform as shown in Equation (18).

As a first embodiment for selecting a base vector, a method of selecting a base vector of a class that minimizes a difference between the brightness values of the current block and the base vectors of the classes may be used. The class i of the basis vector that minimizes the difference from the current block can be expressed by Equation (19).

As a second embodiment for selecting a basis vector, a method of selecting a basis vector of a class corresponding to the current block may be used. The second embodiment directly uses a learned base vector with a class corresponding to a maximum absolute value through a frequency analysis of a current block in a faster manner than the first embodiment. The process of obtaining the class i corresponding to the current block can be expressed by Equation (20).

According to one embodiment, when the basis vector selection unit 226 of the image encoding apparatus 200 selects a basis vector to be used for transforming the current block, the transform unit 230 transforms the current block using the selected basis vector .

According to one embodiment, information on the class of the base vector selected in the basis vector selection unit 226 may be compressed into a bitstream as additional information and transmitted to the video decoding apparatus 300. At this time, the information on the class of the basis vector selected by the basis vector selection unit 226 may be included in the side information as a form of a class index of a block unit. According to one embodiment, bits may be allocated for each block to represent a class index. For example, in the case of an 8x8 block, if the total number of classes is 63, 6 bits are required per block as additional information for indicating the class index. However, if the total number of classes is reduced to 8, the additional information can be transmitted with only 3 bits per block.

According to one embodiment, the basis vector selection unit 326 of the video decoding apparatus 300 inversely transforms the current block based on the base vector corresponding to the class of the current block, based on the additional information obtained from the bitstream.

10 is a flowchart illustrating an image conversion method according to an embodiment.

Referring to FIG. 10, in step S1010, a restored frame in which an image frame preceding a current image frame is reconstructed is divided into a plurality of blocks.

In step S1020, each block is classified into one or more classes based on the distribution of frequency components of each block in the plurality of blocks to determine a block set for each class.

In step S1030, a class-specific basis vector is generated for each class-based block set.

In step S1040, a base vector to be used for transforming the current block in the current image frame among the class-basis vectors is selected.

In step S1050, the current block is transformed using the selected basis vector.

11 is a flowchart for explaining an image conversion method according to another embodiment.

Referring to FIG. 11, in step S1110, a current image frame to be converted is received.

In step S1120, it is determined whether the current image frame is a scene-switched image frame.

In step S1130, the base vector to be used for transforming the current block is selected from the basis vectors of the classes generated for the previous image frame, and the current vector is selected using the basis vector selected in step S1140. Converts the block. The operation of steps S1130 to S1140 may correspond to S1040 to S1050 of Fig.

If the current image frame is a scene-switched image frame, the current block is converted based on the transform vector embedded in the image transform device in step S1150. For example, the current block may be converted based on a conversion vector such as DCT, DST, or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Modification is possible. Accordingly, the spirit of the present invention should be understood only in accordance with the following claims, and all of the equivalent or equivalent variations will fall within the scope of the present invention. In addition, the system according to the present invention can be embodied as computer-readable codes on a computer-readable recording medium.

In addition, the computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like. The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

Claims (19)

Dividing a restored frame in which an image frame previous to a current image frame is reconstructed into a plurality of blocks;
Classifying each block into one or more classes based on a distribution of frequency components of each block in the plurality of blocks to determine a block set for each class;
Generating a class-basis vector for the class-specific block set;
Selecting a base vector to be used for transforming a current block in a current image frame among the basis vectors for each class; And
And transforming the current block using the selected basis vector. The method according to claim 1,
And the class-basis vector is generated based on the restored frame stored in the restored picture buffer. The method according to claim 1,
Wherein the reconstructed frame is an inverse quantized and DCT inversely transformed and quantized after transformed image frames of an image frame preceding the current image frame. The method according to claim 1,
The method comprises:
Further comprising the step of determining whether the current video frame is a scene-switched video frame,
Wherein if the current image frame is determined as a scene-switched image frame, the current block is transformed based on a transform vector embedded in the image transform device. 5. The method of claim 4,
Wherein the determining step comprises:
Comparing the pixel brightness difference value or the brightness histogram difference value between the current image frame and the image frame preceding the current image frame to a predetermined threshold value. The method according to claim 1,
Wherein the step of determining the class-
Searching for a position of a coefficient having a maximum absolute value among frequency components excluding a direct current (DC) component for each block; And
And classifying blocks of the plurality of blocks having the same index corresponding to the searched position into the same class. The method according to claim 1,
The step of generating the class-
Calculating a covariance matrix of the block set for each class; And
Class basis vector based on an eigenvector and an eigenvalue of a covariance matrix of the block-by-class block set. The method according to claim 1,
Wherein a basis vector used for transforming the current block is selected as a basis vector of a class that minimizes a difference between brightness values of the class and the current block. The method according to claim 6,
Wherein a basis vector to be used for transforming the current block is selected as a basis vector of a corresponding class of the current block. A receiving unit for receiving a current image frame to be converted;
Dividing a restored frame in which an image frame before the current image frame is reconstructed into a plurality of blocks, classifying each block into one or more classes based on distribution of frequency components of each block in the plurality of blocks, A control unit for generating a class-basis vector for the class-specific block set and selecting a basis vector to be used for transforming a current block in the current image frame, among the class-based basis vectors; And
And a conversion unit for converting the current block using the selected basis vector. 11. The method of claim 10,
Further comprising a restored picture buffer for storing a reference picture for prediction,
And the class-basis vector is generated based on the reconstruction frame stored in the reconstruction picture buffer. 11. The method of claim 10,
Wherein the reconstructed frame is an inverse quantized and inverse transformed DCT transformed and quantized image frame of a previous image frame of the current image frame. 11. The method of claim 10,
Wherein,
The method of claim 1, further comprising: determining whether the current image frame is a scene-switched image frame, and converting the current block based on a transformed vector included in the image transform device when the current image frame is determined as a scene- Conversion device. 14. The method of claim 13,
Wherein,
And compares a pixel brightness difference value or a brightness histogram difference value between the current image frame and an image frame before the current image frame with a predetermined threshold value. 11. The method of claim 10,
Wherein,
Searching for a position of a coefficient having a maximum absolute value among frequency components excluding a direct current (DC) component for each of the blocks, and searching for a position of a coefficient having a maximum absolute value, Into the same class. 11. The method of claim 10,
Wherein,
Class basis vector on the basis of an eigen vector and an eigen value of a covariance matrix of the block set for each class, Device. 11. The method of claim 10,
Wherein a basis vector used for transforming the current block is selected as a basis vector of a class that minimizes a difference between brightness values of the class and the current block. 16. The method of claim 15,
Wherein a basis vector to be used for transforming the current block is selected as a basis vector of a corresponding class of the current block. A recording medium on which a program for causing a computer to execute the method according to claim 1 is recorded.

Images