WO2017003240A1

WO2017003240A1 - Image conversion device and image conversion method therefor

Info

Publication number: WO2017003240A1
Application number: PCT/KR2016/007080
Authority: WO
Inventors: 김문철; 최재석
Original assignee: 한국과학기술원
Priority date: 2015-06-30
Filing date: 2016-06-30
Publication date: 2017-01-05

Abstract

The present invention relates to an image conversion device and an image conversion method therefor. The disclosed image conversion method comprises the steps of: dividing a low-resolution image so as to generate a plurality of low-resolution image patches having N (wherein N is a natural number of 2 or more) pixels; classifying the plurality of low-resolution image patches into a plurality of image categories; converting the plurality of low-resolution image patches by using a conversion kernel corresponding to each image category among a plurality of prestored conversion matrices, so as to generate a plurality of high-resolution image patches having L (wherein L is a natural number) pixels, which is less than N pixels; and arranging the plurality of high-resolution image patches so as to generate a high-resolution image. According to one embodiment of the present invention, an operation quantity can be reduced in comparison to a prior art, and thus hardware having low complexity can be implemented. In addition, a high quality and high-resolution image can be generated regardless of characteristic information of an image signal.

Description

Video conversion device and video conversion method

The present invention relates to an image conversion apparatus and an image conversion method thereof. More particularly, the present invention relates to an image converting apparatus for converting a low resolution image into a high resolution image having a relatively higher resolution, and an image conversion method thereof.

As is well known, the conversion of a low resolution image to a higher resolution image with a relatively higher resolution is called super resolution or up-scaling.

According to the prior art, in converting a low resolution image into a high resolution image, the low resolution image is extended to high resolution through an interpolation process such as bi-cubic and converted into a high resolution image. In addition, a high quality image remaining at a low quality level is repeatedly improved by using a real conversion kernel such as a floating-point matrix to generate a high quality image of high quality.

However, the prior art requires an interpolation process for a low resolution image and requires a lot of operations because it uses a real conversion kernel, which is difficult to implement with low complexity hardware.

In addition, there is a problem in that a satisfactory high definition image cannot be generated for a low resolution image having a specific image signal characteristic because the characteristic information such as edge or texture of the image signal is not used or not sufficiently utilized.

According to an exemplary embodiment of the present invention, there is provided an image conversion apparatus and an image conversion method for converting a low resolution image directly into a high resolution image without undergoing an interpolation process and a repetitive image quality improvement process for the low resolution image.

The problem to be solved of the present invention is not limited to those mentioned above, and another problem to be solved which is not mentioned will be clearly understood by those skilled in the art from the following description.

According to a first aspect of the present invention, an image conversion method using an image conversion apparatus for converting a low resolution image into a high resolution image having a relatively higher resolution comprises dividing the low resolution image into N (where N is a natural number of 2 or more). Generating a plurality of low resolution image patches having pixels, classifying the plurality of low resolution image patches into a plurality of image categories, and corresponding the plurality of low resolution image patches by the image categories among a plurality of stored transformation matrices. Converting using a conversion kernel to generate a plurality of high resolution image patches having L (where L is a natural number) pixels, and arranging the plurality of high resolution image patches to generate the high resolution image. It may include a step.

The classifying of the plurality of image categories may include classifying the at least one information among edge direction, edge intensity, texture, brightness, first or more differential signals, color, object or object type, and frequency of the low resolution image patch. Can be.

The classifying of the plurality of image categories may include classifying the N pixels included in the low resolution image patch into a plurality of pixel groups, and a plurality of direction categories in which the edge direction is preset for the pixel group. The method may include identifying which direction category corresponds to the direction category, and classifying the low resolution image patch into any one image category among the plurality of image categories based on the identified direction category of the pixel group.

Some of the N pixels may be classified in a different pixel group among the plurality of pixel groups.

The step of determining which direction category corresponds to may use an edge filter.

The edge filter may include a vertical pixel conversion matrix and a horizontal pixel conversion matrix.

Determining which direction category corresponds to one quadrant on a rectangular coordinate plane in which a horizontal category, a vertical category, a horizontal direction and a vertical direction are orthogonal to each other, using the vertical pixel transformation matrix and the horizontal pixel transformation matrix. It is possible to determine which direction category corresponds to a first diagonal direction across the three quadrants, a second diagonal direction across the quadrant and the quadrant on the rectangular coordinate plane, and a non-directional category having no directionality.

The determining of which direction category corresponds to the direction category may include calculating the magnitude value and the angle value in the edge direction, and determining the direction direction category if the magnitude value is less than a preset threshold value, and the magnitude value is the threshold value. If it is equal to or greater than the value, it may be determined as any one of the horizontal category, the vertical category, the first diagonal category, and the second diagonal category according to the angle value.

In the classifying of the plurality of image categories, the N pixels may be classified into first to fourth pixel groups, and the direction category may be determined based on the first to fourth pixel groups. The low resolution image patch may be classified into any one image category among 625 image categories using a decimal number.

The low resolution image patch converted using the conversion kernel may be an original size not enlarged.

The plurality of transform kernels are calculated using a pair of training image patches corresponding to the low resolution image patch and a correct image patch corresponding to the high resolution image patch, wherein the training image patch is an original size that is not enlarged. Can be.

The plurality of transform kernels may be a linear transform matrix or a nonlinear transform matrix.

The generating of the plurality of high resolution image patches may include multiplying the linear transformation matrix of integer values by the low resolution image patch and obtaining a pixel value of the high resolution image patch by a bit shift operation.

The generating of the high resolution image may include overlapping the N pixels included in the low resolution image patch with the high resolution image, and the high resolution image is located at a peripheral position around the center of the N pixels. L pixels included in the image patch may be disposed.

In the generating of the high resolution image, when N of 9 and L is 4, the low resolution image patch may be a 3 × 3 polygon, and the high resolution image patch may be a 2 × 2 polygon.

In the generating of the high resolution image, the high resolution image may be generated by arranging the plurality of high resolution image patches so as not to overlap each other.

According to a second aspect of the present invention, it is possible to provide a computer program stored in a computer-readable recording medium for the processor to perform the image conversion method.

According to a third aspect of the present invention, a computer-readable recording medium having a computer program stored therein for causing a processor to perform the image conversion method may be provided.

According to a fourth aspect of the present invention, an image converting apparatus for converting a low resolution image into a higher resolution image having a higher resolution includes: an input unit receiving the low resolution image and converting the low resolution image input to generate the high resolution image And a processing unit to output the generated high resolution image, wherein the processing unit divides the low resolution image to generate a plurality of low resolution image patches having N pixels (where N is a natural number of two or more), The plurality of low resolution image patches are classified into a plurality of image categories, and the plurality of low resolution image patches are converted using a transform kernel corresponding to each of the image categories from among a plurality of stored transformation matrices, where L is less than N (where: L is a natural number). , It is possible to arrange the plurality of high resolution image patch to produce the high resolution image.

The processor may generate the high resolution image by arranging the plurality of high resolution image patches so as not to overlap each other.

According to an embodiment of the present invention, a low resolution image is immediately converted into a high resolution image without undergoing an interpolation process for the low resolution image and an iterative image quality improvement process. Therefore, the amount of computation is reduced compared to the prior art, which requires an interpolation process, which can be implemented with low complexity hardware.

Furthermore, since the low resolution image is classified into a plurality of image categories according to the characteristic information, and then converted into a high resolution image using an integer conversion kernel selected for each category, the amount of computation can be further reduced.

In addition, when generating a high resolution image by arranging a plurality of high resolution image patches, by arranging adjacent high resolution image patches so that they do not overlap, an additional operation such as pixel value calculation for the overlapped region is not necessary, thereby further increasing the amount of calculation. Can be reduced.

Since the low resolution image is classified into a plurality of image categories according to the characteristic information, and then converted into a high resolution image using a conversion kernel selected for each image category, a high resolution image of high quality can be generated regardless of the characteristic information of the image signal. It has an effect.

1 is a block diagram of an image conversion apparatus capable of performing an image conversion method according to an exemplary embodiment of the present invention.

2 is a flowchart illustrating an image conversion method according to an embodiment of the present invention.

3 is a flowchart illustrating a process of classifying an image category using edge direction information according to an embodiment of the present invention.

4 is a flowchart illustrating a process of determining a direction category using edge direction information according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a vertical pixel conversion matrix V and a horizontal pixel conversion matrix H applied to a low resolution image patch divided from a low resolution image according to an exemplary embodiment of the present invention.

6 is a diagram illustrating a direction category classified using edge direction information according to an embodiment of the present invention.

7 is a diagram illustrating a process of converting a high resolution image patch by applying a conversion kernel to a low resolution image patch according to an embodiment of the present invention.

8 is a diagram illustrating a process of generating a high resolution image by arranging a plurality of high resolution image patches according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, and only the embodiments make the disclosure of the present invention complete, and the general knowledge in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, terms to be described below are terms defined in consideration of functions in the embodiments of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the contents throughout the specification.

As illustrated, the image conversion apparatus 100 according to the embodiment includes an input unit 110, a processor 120, and an output unit 130.

The input unit 110 receives the low resolution image and provides it to the processor 120. For example, the input unit 110 may be implemented by various communication interfaces such as a serial port, a universal serial bus (USB) port, or the like, or a receiver capable of receiving a data stream.

The processor 120 performs a process of converting a low resolution image provided from the input unit 110 into a high resolution image. The processor 120 may be implemented as a processor such as a central processing unit.

The processor 120 divides the low resolution image and generates a plurality of low resolution image patches having N pixels (where N is a natural number of two or more). Subsequently, the plurality of low resolution image patches are classified into a plurality of image categories. The plurality of low resolution image patches are converted using a transform kernel corresponding to each image category among a plurality of stored transformation matrices, thereby generating a plurality of high resolution image patches having L (where L is a natural number) pixels smaller than N. . Also, a plurality of high resolution image patches are arranged to generate a high resolution image.

When the processor 120 classifies a plurality of low resolution image patches into a plurality of image categories, the processor 120 may include at least one of an edge direction, an edge intensity, a texture, a brightness, a first or more differential signal, a color, an object or an object type, and a frequency of the low resolution image patch. It can be classified using one or more pieces of information.

For example, the processor 120 may classify an image category using edge direction information. In this case, N pixels included in the low resolution image patch are classified into a plurality of pixel groups. In this case, some of the N pixels may be classified in a different pixel group among the plurality of pixel groups. Next, the direction of the edge group corresponds to the direction category among a plurality of preset direction categories for the pixel group. The low resolution image patch is classified into any one image category among the plurality of image categories based on the identified direction category of the pixel group.

Here, the processor 120 may use an edge filter including a vertical pixel transformation matrix and a horizontal pixel transformation matrix to determine the direction category of the pixel group.

For example, the processor 120 may determine which direction category an edge direction component of a pixel group corresponds to by using a vertical pixel transformation matrix and a horizontal pixel transformation matrix in a state of setting five direction categories in total. The five direction categories may include a horizontal category and a vertical category. The first and second quadrants may include a first diagonal direction that crosses one quadrant and three quadrants on an orthogonal coordinate plane perpendicular to the horizontal direction and the vertical direction. It may also include a second diagonal category across the moving and quadrants on the Cartesian coordinate plane. And, it may include a non-directional category having no directionality.

For example, the processor 120 may use a preset threshold value when determining the direction category of the pixel group. After calculating the size value and the angle value in the edge direction, if the size value is less than the predetermined threshold value can be determined as a non-directional category. However, if the magnitude value is greater than or equal to the threshold value, it may be determined as any one direction category among the horizontal category, the vertical category, the first diagonal category, and the second diagonal category according to the angle value.

Meanwhile, when the processing unit 120 converts a plurality of low resolution image patches using a transform kernel corresponding to each image category from among a plurality of stored transform matrices, a linear transform matrix or a nonlinear transform matrix may be used as the transform kernel. Here, the low resolution image patch used by the transform kernel may be an original size not enlarged. In addition, the plurality of transform kernels used by the processor 120 may be obtained through a separate training process. The processor 120 may calculate a plurality of transform kernels through training using a pair of training image patches corresponding to the low resolution image patch and correct answer image patches corresponding to the high resolution image patch. Here, the training image patch used by the processor 120 may be an original size that is not enlarged. Original size that is not enlarged may mean that the interpolation process has not been performed.

In addition, the processor 120 may generate a high resolution image by arranging a plurality of high resolution image patches without overlapping. Here, when N pixels included in the low resolution image patch are superimposed on a high resolution image, L pixels included in the high resolution image patch may be disposed at a peripheral position centering on the position of the pixel among the N pixels. have.

The output unit 130 constituting the image conversion apparatus 100 according to the exemplary embodiment of the present invention outputs a high resolution image generated by the processor 120. For example, the output unit 130 may be implemented as various communication interfaces such as a serial port, a USB port, or the like, or may be implemented as an image display device capable of displaying a high resolution image on a screen.

Hereinafter, an image conversion method for converting a low resolution image into a high resolution image having a relatively high resolution will be described with reference to FIGS. 1 to 8.

First, the input unit 110 of the image conversion apparatus 100 receives a low resolution image and provides it to the processing unit 120 of the image conversion apparatus 100 (S210).

Then, the processor 120 divides the low resolution image provided from the input unit 110 and generates a plurality of low resolution image patches having N pixels (where N is a natural number of two or more) (S220). For example, the processor 120 may generate a plurality of low resolution image patches having nine pixels. That is, as shown in FIG. 4, the low resolution image may be divided into a 3 × 3 polygonal low resolution image patch 301.

Subsequently, the processor 120 classifies the plurality of low resolution image patches into the plurality of image categories using the characteristic information (S230). Here, the number of image categories is determined in advance through a training process using a pair of training image patches corresponding to the low resolution image patches and correct answer image patches corresponding to the high resolution image patches. The classification criteria of the image category may use at least one or more information among the edge direction, edge intensity, texture, brightness, first or more differential signal, color, object or object type, and frequency of the low resolution image patch. Alternatively, the processor 120 may classify a plurality of low resolution image patches into a plurality of image categories using a known k-means clustering algorithm. In this case, the k-means clustering algorithm may be executed with a 30-dimensional feature vector using a PCA (Principal Component Analysis).

Here, the processor 120 may classify the image category by using the edge direction information. In this case, the processor 120 divides the N pixels included in the low resolution image patch into L pixels and classifies them into a plurality of pixel groups (S231). In this case, some of the N pixels may be classified in a different pixel group among the plurality of pixel groups. As shown in FIG. 4, the low-resolution image patch 301 having a 3 × 3 polygon shape is divided into 2 × 2 polygon shapes to form a first pixel group 311, a second pixel group 312, and a third pixel group 313. The fourth pixel group 314 may be classified.

Subsequently, the processor 120 determines which direction category the edge direction corresponds to among the plurality of preset direction categories for the pixel groups (S232).

In this case, the processor 120 may use an edge filter including a vertical pixel conversion matrix V and a horizontal pixel conversion matrix H, as illustrated in FIG. 5, to determine the direction category of the pixel group.

Further, the processor 120 uses the vertical pixel conversion matrix V and the horizontal pixel conversion matrix H to set the edge direction components of the pixel groups in a state in which five direction categories are set as shown in FIG. 6. Find out which direction category it belongs to.

The five direction categories may include a horizontal category 401 and a vertical category 402. And a first diagonal category 403 across one quadrant and three quadrants on an orthogonal coordinate plane perpendicular to the horizontal direction and the vertical direction. It may also include a second diagonal category 404 across the moving and quadrants on the Cartesian coordinate plane. And, it may include a non-directional category 405 having no directionality.

In addition, the processor 120 may use a preset threshold value when determining the direction category of the pixel group. After the magnitude value and the angle value of the edge direction are calculated, if the magnitude value is less than the preset threshold value, the non-direction category 405 may be determined. However, if the magnitude value is greater than or equal to the threshold value, the direction category of any one of the horizontal category 401, the vertical category 402, the first diagonal category 403, and the second diagonal category 404 according to the angle value. Can be determined.

Equation 1 below illustrates a process in which the processor 120 represents the first pixel group 311 as a matrix P ₁₁ , and calculates the magnitude value m ₁₁ and the angle value d _{11 in the} edge direction. It is shown as.

Equation 2 below illustrates a process in which the processor 120 represents the second pixel group 312 as a matrix P ₁₂ and calculates the magnitude value m ₁₂ and the angle value d _{12 in the} edge direction. It is shown as.

Equation 3 below illustrates a process in which the processor 120 represents the third pixel group 313 as a matrix P ₂₁ and calculates the magnitude value m ₂₁ and the angle value d _{21 in the} edge direction. It is shown as.

Equation 4 below illustrates a process in which the processor 120 represents the fourth pixel group 314 as a matrix P ₂₂ , and calculates the magnitude value m ₂₂ and the angle value d _{22 in the} edge direction. It is shown as.

For example, when the magnitude value and the angle value of the edge direction are calculated through the processes of Equation 1 to Equation 4, the processor 120 compares the magnitude value with a preset threshold value, and if the magnitude value is less than the preset threshold value, The direction category 405 is determined, and the index is designated as "0".

However, if the calculated size value is greater than or equal to a preset threshold, the processor 120 determines a horizontal category 401, a vertical category 402, a first diagonal category 403, and a second diagonal category according to the angle value. It may be determined as one of the direction categories from the (404). For example, if "-22.5 <angle value <22.5", it is determined by the horizontal direction category 401, and the index is designated as "1". If "22.5 ≤ angle value <67.5", the first pre-direction category 403 is determined, and the index is designated as "2". If "67.5 ≤ angle value <112.5", it is determined by the vertical direction category 402, and the index is designated as "3". If "112.5 <angle value <157.5", the second pre-direction category 404 is determined, and the index is designated as "4".

Subsequently, the processor 120 classifies the low resolution image patch into any one image category among the plurality of image categories based on the direction category of the pixel group identified above (S233).

4, an index of the first pixel group 311 is designated as "3", an index of the second pixel group 312 is designated as "2", and an index of the third pixel group 313 is designated as "1". Is a case where the index of the fourth pixel group 314 is designated as "2". Then, the processor 120 classifies the low resolution image patch into the image category corresponding to the class 288 among the 625 image categories by using a decimal number.

Next, the processor 120 converts a plurality of low resolution image patches using a transform kernel corresponding to each image category among a plurality of stored transformation matrices, and has a plurality of L (where L is a natural number) pixels smaller than N. A high resolution image patch is generated (S240).

FIG. 7 illustrates a case where a 3 × 3 polygon low resolution image patch 501 is transformed using a k th transform kernel M _k corresponding to a k th image category to generate a 2 × 2 polygon high resolution image patch 502. It is shown.

Meanwhile, when the processing unit 120 converts a plurality of low resolution image patches using a transform kernel corresponding to each image category from among a plurality of stored transform matrices, a linear transform matrix or a nonlinear transform matrix may be used as the transform kernel. Here, the low resolution image patch used by the transform kernel may be an original size not enlarged. In addition, the plurality of transform kernels used by the processor 120 may be obtained through a separate training process.

The processor 120 may calculate a plurality of transform kernels through training using a pair of training image patches corresponding to the low resolution image patch and correct answer image patches corresponding to the high resolution image patch. Here, the training image patch used by the processor 120 may be an original size that is not enlarged. Original size that is not enlarged may mean that the interpolation process has not been performed. However, if the amount of computation increases to some extent, the interpolation process for the low resolution image patch may be performed.

Here, the processor 120 may multiply the linear transformation matrix composed of integer values by the low resolution image patch and obtain a pixel value of the high resolution image patch by a bit shift operation.

For example, the conversion kernels are previously stored in a separate external memory in the form of a look-up table so as to correspond to the plurality of image categories in a one-to-one correspondence, and the processor 120 converts the conversion kernels for each image category of the low resolution image patch from the external memory. The pixel value of the high resolution image patch can be obtained through the multiplication operation and the bit shift operation using the read and read conversion kernels. Here, if a nonlinear transformation matrix is used, the low resolution image patch may be converted into a high resolution image patch through MLP (Multi-layer Perceptron) mapping after a preprocessing process such as noise mitigation or color space conversion. For example, information about the number of shift bits for the bit shift operation may be stored in the external memory, and the processor 120 may process the bit shift operation with reference thereto.

Thereafter, the processor 120 generates a high resolution image by arranging the plurality of high resolution image patches so as not to overlap each other.

Here, when N pixels included in the low resolution image patch are superimposed on a high resolution image, L pixels included in the high resolution image patch may be disposed at a peripheral position centering on the position of the pixel among the N pixels. have.

8 is a diagram illustrating a process of generating a high resolution image by arranging a plurality of high resolution image patches. In (a), the first low resolution image patch 601 is converted to generate and arrange the first high resolution image patch 701, and in (b) the second low resolution image patch 602 is converted to the second high resolution image patch ( 702) is generated and an example of generating and arranging the third high resolution image patch 703 by converting the third low resolution image patch 603 in (c). As shown in FIG. 8, in the first high resolution image patch 701, the second high resolution image patch 702, and the third high resolution image patch 703, pixels do not overlap between adjacent high resolution image patches. Similarly, later-defined high resolution image patches also do not overlap pixels between adjacent high resolution image patches.

Next, the output unit 130 of the image conversion apparatus 100 outputs a high resolution image generated by the processor 120 (S250). For example, the output unit 130 may output a high resolution image in a data stream form or display a high resolution image on a screen through various communication interfaces such as a serial port and a USB port.

As described above, according to the exemplary embodiment of the present invention, the low resolution image is immediately converted into a high resolution image without undergoing an interpolation process for the low resolution image and an iterative image quality improvement process. Therefore, the amount of computation is reduced compared to the prior art, which requires an interpolation process, which can be implemented with low complexity hardware.

Since the low resolution image is classified into a plurality of image categories according to the characteristic information, and then converted into a high resolution image using a conversion kernel selected for each image category, a high resolution image of high quality can be generated regardless of the characteristic information of the image signal. have.

Combinations of the steps of each flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment such that the instructions performed through the processor of the computer or other programmable data processing equipment are described in each step of the flowchart. It will create a means to perform them. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored therein to produce an article of manufacture containing instruction means for performing the functions described in each step of the flowchart. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions for performing the processing equipment may also provide steps for executing the functions described in each step of the flowchart.

In addition, each step may represent a module, segment or portion of code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions noted in the steps may occur out of order. For example, the two steps shown in succession may in fact be performed substantially simultaneously or the steps may sometimes be performed in the reverse order, depending on the function in question.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas falling within the scope of the present invention should be construed as being included in the scope of the present invention.

Claims

An image conversion method by an image conversion apparatus for converting a low resolution image into a high resolution image having a relatively higher resolution,

Generating a plurality of low resolution image patches having N pixels (where N is a natural number of two or more) by dividing the low resolution image;

Classifying the plurality of low resolution image patches into a plurality of image categories;

The plurality of low resolution image patches are converted using a transform kernel corresponding to each image category from among a plurality of stored transformation matrices, thereby generating a plurality of high resolution image patches having L pixels (where L is a natural number) of pixels smaller than N. To do that,

Generating the high resolution image by arranging the plurality of high resolution image patches

Image conversion method comprising a.
The method of claim 1,

The classifying into the plurality of image categories may include:

The low resolution image patch is classified using at least one or more information among an edge direction, edge intensity, texture, brightness, first or more differential signals, color, object or object type, and frequency.

Video conversion method.
The method of claim 2,

The classifying into the plurality of image categories may include:

Classifying the N pixels included in the low resolution image patch into a plurality of pixel groups;

Identifying which direction category the edge direction corresponds to among the preset plurality of direction categories for the pixel group;

Classifying the low resolution image patch into any one image category among the plurality of image categories based on the identified direction category of the pixel group

Image conversion method comprising a.
The method of claim 3, wherein

Some of the N pixels are classified in duplicate in different pixel groups among the plurality of pixel groups.

Video conversion method.
The method of claim 3, wherein

Determining which direction category corresponds to,

Using edge filters

Video conversion method.
The method of claim 5,

The edge filter includes a vertical pixel conversion matrix and a horizontal pixel conversion matrix.

Video conversion method.
The method of claim 5,

Determining which direction category corresponds to,

A horizontal diagonal category, a vertical category, a first diagonal category that crosses one quadrant and three quadrants on an orthogonal coordinate plane in which a horizontal direction and a vertical direction are perpendicular to each other, using the vertical pixel transformation matrix and the horizontal pixel transformation matrix; Determine which direction category corresponds to the second diagonal category across the moving and quadrants on the Cartesian coordinate plane, or the non-directional category with no directionality.

Video conversion method.
The method of claim 7, wherein

Determining which direction category corresponds to,

After calculating the size value and the angle value in the edge direction, if the size value is less than a predetermined threshold value, the non-directional category is determined. If the size value is more than the threshold value, the horizontal category is determined according to the angle value. Determine one of the vertical categories, the first diagonal category, and the second diagonal category.

Video conversion method.
The method of claim 8,

The classifying into the plurality of image categories may include:

The N pixels are classified into first to fourth pixel groups, the direction category is determined based on the first to fourth pixel groups, and the low resolution image patch is 625 using a decimal number. Categorized into one video category from the video categories

Video conversion method.
The method of claim 1,

The low resolution image patch converted using the conversion kernel is an original size that is not enlarged.

Video conversion method.
The method of claim 1,

The plurality of transform kernels are calculated using a pair of training image patches corresponding to the low resolution image patch and a correct answer image patch corresponding to the high resolution image patch,

The training video patch, the original size is not enlarged

Video conversion method.
The method of claim 1,

The plurality of transform kernels are linear transform matrix or nonlinear transform matrix.

Video conversion method.
The method of claim 12,

The generating of the plurality of high resolution image patches may include:

Multiplying the linear transformation matrix of integer values by the low resolution image patch and obtaining a pixel value of the high resolution image patch by a bit shift operation.

Video conversion method.
The method of claim 1,

Generating the high resolution image,

When the N pixels included in the low resolution image patch are superimposed on the high resolution image, L pixels included in the high resolution image patch are located at a peripheral position centering on the position of the center pixel among the N pixels. Posted

Video conversion method.
The method of claim 14,

Generating the high resolution image,

When N of 9 and L is 4, the low resolution image patch is a 3 × 3 polygon, and the high resolution image patch is a 2 × 2 polygon.

Video conversion method.
The method of claim 1,

Generating the high resolution image,

Generating the high resolution image by arranging the plurality of high resolution image patches so as not to overlap

Video conversion method.
A computer program stored in a computer-readable recording medium for causing a processor to perform the image conversion method of any one of claims 1 to 16.
A computer-readable recording medium having a computer program stored therein for causing a processor to perform the image conversion method of any one of claims 1 to 16.
An image conversion apparatus for converting a low resolution image into a higher resolution image having a relatively higher resolution.

An input unit configured to receive the low resolution image;

A processor configured to convert the inputted low resolution image into the high resolution image;

An output unit for outputting the generated high resolution image,

The processing unit,

Splitting the low resolution image to generate a plurality of low resolution image patches having N pixels (where N is a natural number of 2 or more),

Classifying the plurality of low resolution image patches into a plurality of image categories,

The plurality of low resolution image patches are converted using a transform kernel corresponding to each image category from among a plurality of stored transformation matrices, thereby generating a plurality of high resolution image patches having L pixels (where L is a natural number) of pixels smaller than N. and,

Generating the high resolution image by arranging the plurality of high resolution image patches

Video conversion device.
The method of claim 19,

The processing unit,

Generating the high resolution image by arranging the plurality of high resolution image patches so as not to overlap

Video conversion device.