KR102552589B1 - Super resolution imaging method - Google Patents

Abstract

The present invention provides a technology for generating a high-resolution image directly from an input low-resolution image without an intermediate image, applying multiple linear mappings to one patch by utilizing local information of sub-patches in the low-resolution image, and generating an image at this time. A technique for generating a high-quality, high-resolution image patch by applying one global transformation matrix to candidate patches is provided.

Description

Super resolution method {SUPER RESOLUTION IMAGING METHOD}

The present invention relates to image processing technology.

In converting a low-resolution image into a high-resolution image, a conventional technique utilizes interpolation to generate an intermediate image and converts the intermediate image into a high-resolution image using linear mapping. In this conventional technique, since the number of patches of the interpolated intermediate image is greater than the number of patches of the low-resolution image, it takes a long time to generate a high-resolution image.

Meanwhile, in the conventional technique, when one image is generated by combining patches of a high-resolution image obtained by converting patches of an intermediate image, overlapping inevitably occurs. If a high-resolution image is generated by averaging the overlapping patches, the image quality is improved, but additional memory and computation are required.

Against this background, an object of the present invention is to provide a technique for reducing the time required to convert a low-resolution image into a high-resolution image.

In order to achieve the above object, in one aspect, the present invention provides a super-resolution method for converting a low-resolution image into a high-resolution image, the steps of storing a global transformation matrix and sub-patch transformation matrices for each image category; Determining an image category for each of a plurality of sub-patches included in the patch of the low-resolution image (low-resolution image patch), and selecting the sub-patch transformation matrix corresponding to each of the plurality of sub-patches according to the determined image category. , generating a high-resolution image patch by applying the global transformation matrix and the selected sub-patch transformation matrix to the low-resolution image patch, and generating the high-resolution image using the high-resolution image patch. do.

In another aspect, the present invention provides a super-resolution method for converting a low-resolution image into a high-resolution image, comprising: converting a test high-resolution image into a test low-resolution image; extracting a plurality of high-resolution image patches from the test high-resolution image; extracting a plurality of low-resolution image patches from a low-resolution image; determining an image category for each of a plurality of sub-patches included in the low-resolution image patch for each low-resolution image patch; and selecting the low-resolution image patch for each image category. Learning a sub-patch transformation matrix for transforming into a high-resolution image patch; generating a plurality of high-resolution image candidate patches by applying the sub-patch transformation matrix to each of the plurality of sub-patches for each low-resolution image patch; Learning a global transformation matrix for transforming a candidate high-resolution image patch into the high-resolution image patch; determining the image category for each of a plurality of sub-patches included in the patch of the low-resolution image; and learning the sub-patches for each image category. It provides a super-resolution method comprising the step of converting the low-resolution image into the high-resolution image using a patch transformation matrix and the global transformation matrix.

As described above, according to the present invention, a low-resolution image can be converted into a high-resolution image within a short period of time. Also, according to the present invention, a low-resolution image can be converted into a high-resolution image using less storage space.

1 is a configuration diagram of a display device to which embodiments may be applied.
2 is a flowchart of a super-resolution method according to an embodiment.
3 is a first exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.
4 is a second exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.
5 is a third exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.
6 is a fourth exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.
7 is a diagram illustrating generating a feature vector and determining an image category of a sub-patch according to an exemplary embodiment.
8 is a diagram illustrating conversion of a 7×7 low-resolution image patch into a high-resolution image patch according to an exemplary embodiment.
9 is a flowchart of a super-resolution method according to another embodiment.
10 is an exemplary diagram illustrating a process of learning a sub-patch transformation matrix in a super-resolution method according to another embodiment.
11 is an exemplary diagram illustrating a process of learning a global transformation matrix in a super-resolution method according to another embodiment.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

Also, terms such as first, second, A, B, (a), and (b) may be used in describing the components of the present invention. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is directly connected or connectable to the other element, but there is another element between the elements. It will be understood that elements may be “connected”, “coupled” or “connected”.

1 is a configuration diagram of a display device to which embodiments may be applied.

Referring to FIG. 1 , a display device 100 may include an image processing device 110, a data driving circuit 120, a gate driving circuit 130, a timing controller 140, a display panel 150, and the like. .

A plurality of data lines DL and a plurality of gate lines GL may be disposed on the display panel 150, and a plurality of pixels (P) may be disposed.

The gate driving circuit 130 may supply a gate driving signal of a turn-on voltage or a turn-off voltage to the gate line GL. When the gate driving signal of the turn-on voltage is supplied to the pixel P, the corresponding pixel P is connected to the data line DL. Then, when the gate driving signal of the turn-off voltage is supplied to the pixel P, the connection between the corresponding pixel P and the data line DL is released.

The data driving circuit 120 supplies a data voltage to the data line DL. The data voltage supplied to the data line DL is supplied to the pixel P according to the gate driving signal.

The timing controller 140 may supply control signals to the gate driving circuit 130 and the data driving circuit 120 . For example, the timing controller 140 may transmit a gate control signal GCS for starting a scan to the gate driving circuit 120 . Also, the timing controller 140 may output the image data RGB to the data driving circuit 120 . Also, the timing controller 140 may transmit a data control signal DCS that controls the data driving circuit 120 to supply data voltages to each pixel P.

The image processing device 110 may generate and transmit image data RGB to the timing controller 140 . The image processing device 110 may generate image data RGB by converting a low-resolution image into a high-resolution image.

2 is a flowchart of a super-resolution method according to an embodiment.

Referring to FIG. 2 , the image processing apparatus may store a matrix for global regression (MR) and a matrix for sub-patch conversion (MS) for each image category (S200). The global transformation matrix (MR) and the subpatch transformation matrix (MS) may be learned by training. An embodiment related to generation of the global transformation matrix (MR) and the subpatch transformation matrix (MS) will be described later.

The image processing device may convert an image in units of patches constituting the low-resolution image (LRP: Low Resolution Patch).

The image processing apparatus may determine an image category for each of a plurality of sub-patches included in the low-resolution image patch (LRP) (S202). Then, the image processing apparatus may select a sub-patch transformation matrix corresponding to each of a plurality of sub-patches according to the determined image category (S204).

Then, the image processing apparatus generates a high-resolution image patch (SRP: Super Resolution Patch) by applying the pre-stored global transformation matrix and the sub-patch transformation matrix determined according to the image category to the low-resolution image patch (S206).

Then, the image processing device generates a high-resolution image using the high-resolution image patch (SRP) (S208).

3 is a first exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.

The image processing device may convert an image in units of patches. Here, a patch is a partial image including pixels P, and may include at least one pixel P.

Referring to FIG. 3 , the image processing apparatus may select and convert low resolution patches LRPa, LRPb, and LRPc constituting a low resolution image LR or an image obtained by extending the low resolution image LR.

For example, the image processing apparatus generates a first high resolution patch (HRPa) by converting a first low resolution patch (LRPa) having a size of 5 x 5 pixels, and converts a second low resolution patch (LRPb) to generate a second high resolution patch. (HRPb) may be generated, and a third high resolution patch (HRPc) may be generated by converting the third low resolution patch (LRPc).

The low-resolution patches LRPa, LRPb, and LRPc may be selected to partially overlap. For example, the first low resolution patch LRPa and the second low resolution patch LRPb may be selected such that 20 pixels P overlap each other. In this case, the generated high-resolution patches HRPa, HRPb, and HRPc may not overlap each other. The image processing apparatus may convert the low-resolution patches (LRPa, LRPb, and LRPc) into high-resolution patches (HRPa, HRPb, and HRPc) at different locations. According to this embodiment, a calculation process for overlapping patches in the prior art is not required, so memory can be reduced and conversion speed can be increased.

4 is a second exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.

Referring to FIG. 4 , the image processing apparatus may select a low resolution image patch (LRP) for each pixel (P) of the low resolution image (LR) and convert the selected low resolution image patch (LRP) into a high resolution image patch (HRP). there is. When the low-resolution image patch (LRP) is selected and converted for each pixel (P) in this way, since the high-resolution image patch (HRP) corresponds to the pixel (P) of the low-resolution image (LR) on a one-to-one basis, all pixels (P) It is possible to convert images without overlapping for .

The image processing apparatus may simultaneously select and convert a plurality of low-resolution image patches (LRPs), and may sequentially select and convert low-resolution image patches (LRPs) while moving pixels (P).

For example, the image processing apparatus may select and convert a low-resolution image patch (LRP) of a certain size to the periphery of the central pixel (CP) located in the center. At this time, the image processing apparatus may select the low resolution image patch LRP while sequentially moving the central pixel CP in the horizontal and vertical directions.

When the central pixel CP is located outside the low resolution image LR, the low resolution image patch LRP may further include a virtual extended pixel VP. The virtual extended pixel VP may be formed identically to the outermost pixel of the low-resolution image LR or may be formed by transforming the outermost pixel.

5 is a third exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.

Referring to FIG. 5 , a low-resolution image patch (LRP) may include a plurality of sub-patches (SPa, SPb, ..., SPn). The image processing apparatus may form a high-resolution image patch (HRP) by generating high-resolution image candidate patches (HRPCa, HRPCa, ..., HRPCn) for each sub-patch (SPa, SPb, ..., SPn).

The low-resolution image patch (LRP) may have a pixel size of N×M (N and M are natural numbers greater than or equal to 2). In the example of FIG. 5, N is 7 and M is 7.

The sub-patches (SPa, SPb, ..., SPn) have a pixel size of P x Q (where P is a natural number smaller than N and Q is a natural number smaller than M), and a plurality of sub-patches (SPa, SPb, ..., SPn) may be selected such that some regions overlap. For example, the sub-patches (SPa, SPb, ..., SPn) are selected inside the low-resolution image patch (LRP), except that adjacent sub-patches (SP) are horizontally oriented or vertically oriented. and may overlap in the rest of the area. If the subpatches (SPa, SPb, ..., SPn) are selected in this way, (N-P+1) x (M-Q+1) subpatches (SPa, SPb, ..., SPn) can be formed.

The image processing apparatus determines sub-patch transformation matrices (MS1, MS2, ..., MSn) corresponding to each of the plurality of sub-patches (SPa, SPb, ..., SPn), and determines the determined sub-patch transformation matrices (MS1, MSn). MS2, ..., MSn) may be applied to the low-resolution image patch (LRP) to generate a plurality of high-resolution image candidate patches (HRPCa, HRPCa, ..., HRPCn). Here, applying the sub-patch transformation matrices (MS1, MS2, ..., MSn) to the low-resolution image patch (LRP) may be linear mapping of the low-resolution image patch (LRP). As a specific example, the image processing apparatus forms a vector for each pixel value of the low-resolution image patch (LRP) and multiplies the vector by the sub-patch transformation matrix (MS1, MS2, ..., MSn) to obtain a high-resolution image candidate patch (HRPCa, HRPCa, ..., HRPCn) can be generated.

Also, the image processing apparatus may generate the high-resolution image patch (HRP) by applying the global transformation matrix (MR) to the high-resolution image candidate patches (HRPCa, HRPCa, ..., HRPCn). As a specific example, the image processing device generates a high-resolution image patch (HRP) by combining vectors corresponding to a plurality of high-resolution image candidate patches (HRPCa, HRPCa, ..., HRPCn) and multiplying by a global transformation matrix (MR). can

6 is a fourth exemplary diagram for explaining conversion of a low-resolution image patch into a high-resolution image patch according to an embodiment.

Unlike the example shown in FIG. 5 , the image processing apparatus may convert a low-resolution image patch (LRP) into a high-resolution image patch (HRP) after first generating a matrix for patch conversion (MP).

The image processing apparatus determines sub-patch transformation matrices (MS1, MS2, ..., MSn) corresponding to each of a plurality of sub-patches (SPa, SPb, ..., SPn), and converts the plurality of sub-patch transformation matrices (MR) to A patch transformation matrix (MP) can be generated by calculating a transformation matrix obtained by combining the sub-patch transformation matrices (MS1, MS2, ..., MSn) of .

In addition, a high-resolution image patch (HRP) may be generated by applying the patch transformation matrix (MP) to the low-resolution image patch (LRP).

Meanwhile, the image processing apparatus may classify the image categories of the sub-patch SP and determine a different sub-patch transformation matrix for each image category. The image category may be determined according to information such as image edge direction information, edge strength information, texture information, brightness information, first- or higher-order differential value information, color information, object or object type information, and frequency information of a sub-patch. .

As a specific example, the image processing apparatus may generate a feature vector for each of a plurality of sub-patches, and determine an image category of each of the plurality of sub-patches according to the feature vector.

7 is a diagram illustrating generating a feature vector and determining an image category of a sub-patch according to an exemplary embodiment.

Referring to FIG. 7 , the image processing apparatus may extract a feature vector from a sub-patch (SP) using a filter. For example, the image processing apparatus may extract a feature vector by applying a filter consisting of h1 = [1, -1] or h2 = [1, -1]^(-1) to the sub-patch SP. .

As a specific example, the image processing apparatus may generate the first feature vector CVa by applying a filter composed of h1 and h2 to the sub-patch SP. In addition, the image processing apparatus may generate a second feature vector CVb by multiplying a principal component analysis (PCA) by the first feature vector CVa.

In this way, the image processing apparatus extracts a plurality of feature vectors from a plurality of subpatches—a plurality of subpatches selected from a plurality of low-resolution images—and clusters the extracted feature vectors into a plurality of image categories to generate class vectors. can In addition, when a feature vector is extracted for a specific sub-patch, a class number assigned to a class vector having a proximity to the extracted feature vector may be determined as an image category of the corresponding sub-patch.

The image processing apparatus may cluster feature vectors extracted from a plurality of sub-patches using a k-means clustering technique. Also, the image processing apparatus may determine an image category to which the feature vector of the sub-patch belongs according to the proximity calculation method of the k-means clustering technique.

8 is a diagram illustrating conversion of a 7×7 low-resolution image patch into a high-resolution image patch according to an exemplary embodiment.

Referring to FIG. 8 , the image processing apparatus determines image categories of a plurality of sub-patches included in a low-resolution image patch (LRP) (S802). If a sub-patch consisting of 3 x 3 is selected from a low-resolution image patch (LRP) consisting of 7 x 7, (7 - 3 + 1) x (7 - 3 + 1) = 25 sub-patches are generated. For each of these 25 sub-patches, image categories (k1, k2, ..., k25) are determined according to feature vectors.

Then, the image processing apparatus may select a sub-patch transformation matrix according to the determined image category (k1, k2, ..., k25) (S804). At this time, the image processing apparatus may select a sub-patch transformation matrix in consideration of the location of the sub-patch within the low-resolution image patch (S804). For example, the image processing apparatus may select a sub-patch transformation matrix (MS) corresponding to each sub-patch according to a location in a low-resolution image patch and an image category. The image processing apparatus may select different sub-patch transformation matrices MS for a sub-patch positioned at the upper left and a sub-patch positioned at the lower right even in the same image category. This process can be understood as classifying each sub-patch by analyzing local information of the sub-patch.

To this end, the image processing apparatus may store a sub-patch transformation matrix (MS) for each location in the low-resolution image patch and each image category.

The image processing apparatus may generate a patch transformation matrix MP by combining the sub-patch transformation matrix MS selected for each sub-patch and performing an operation with the global transformation matrix MR (S806).

The image processing apparatus may vectorize the low-resolution image patch (LRP) (S808) and generate a high-resolution image vector (HRV) by calculating the vector with the patch transformation matrix (MP) (S810).

Also, the image processing device may convert the high-resolution image vector (HRV) into a matrix form to generate a high-resolution image patch (HRP).

Meanwhile, a global transformation matrix used for transformation and a sub-patch transformation matrix for each image category may be generated by learning.

9 is a flowchart of a super-resolution method according to another embodiment.

Referring to FIG. 9 , the learning device may convert a test high-resolution image into a test low-resolution image (S902). The learning device may convert a test high-resolution image into a test low-resolution image by removing some pixels from the test high-resolution image or averaging small patches to configure one pixel.

The learning device may extract a plurality of high-resolution image patches from the test high-resolution image and extract a plurality of low-resolution image patches from the test low-resolution image (S904). When extracting a plurality of high-resolution image patches from the test high-resolution image, the learning device may extract the high-resolution image patches so as not to overlap each other. In addition, the learning device extracts a low-resolution image patch corresponding to the high-resolution image patch from the test low-resolution image, and extracts an adjacent low-resolution image patch so that a partial region overlaps.

The learning device may determine an image category for each of a plurality of sub-patches included in the low-resolution image patch for each low-resolution image patch (S906). Here, the image categories may be generated by a k-means clustering technique using a plurality of subpatches.

Then, the learning device may learn a sub-patch transformation matrix for converting a low-resolution image patch into a high-resolution image patch for each image category (S908). In this case, the learning device may learn the sub-patch transformation matrix for each image category and position within the low-resolution image patch.

The learning apparatus may generate a plurality of high-resolution image candidate patches by applying a sub-patch transformation matrix for each of a plurality of sub-patches for each low-resolution image patch (S910). Then, the learning device may learn a global transformation matrix for converting a plurality of high-resolution image candidate patches into high-resolution image patches (S912).

Then, the image processing apparatus determines an image category for each of a plurality of sub-patches included in the patch of the input low-resolution image, and converts the low-resolution image into a high-resolution image by using the sub-patch transformation matrix and the global transformation matrix learned for each image category. can be converted

Here, the learning device and the image processing device may be the same device or may be separate devices. Also, each step may be performed by different devices.

10 is an exemplary diagram illustrating a process of learning a sub-patch transformation matrix in a super-resolution method according to another embodiment.

Referring to FIG. 10, the learning device may form a training pair by preparing a test high-resolution image and a test low-resolution image, extracting a plurality of high-resolution image patches from the test high-resolution image, and extracting a plurality of low-resolution image patches from the test low-resolution image. Yes (S1002).

The learning device may determine an image category for each of a plurality of sub-patches included in the low-resolution image patch for each low-resolution image patch (S1004). At this time, the learning device may generate a sub-patch transformation matrix for each video category, or may generate and train a sub-patch transformation matrix for each video category and location in the video patch as shown in FIG. 10 . The learning device can classify image categories by clustering sub-patches for each location.

In addition, the learning device applies a low-resolution image patch and a high-resolution image patch to a linear mapping calculation method (e.g., kernel ridge regression, etc.) for each group-groups grouped by location in the image patch and image category to obtain a sub-patch transformation matrix (MS) can be calculated (S1006).

When the calculation of the sub-patch transformation matrix (MS) for each video category at one location is completed, the learning device may sequentially calculate the sub-patch transformation matrix (MS) for each video category for other locations (S1008).

The learning device may calculate the global transformation matrix after calculating the subpatch transformation matrix MS.

11 is an exemplary diagram illustrating a process of learning a global transformation matrix in a super-resolution method according to another embodiment.

Referring to FIG. 11 , the learning apparatus may generate a plurality of high-resolution image candidate patches by applying a sub-patch transformation matrix (MS) to each of the plurality of sub-patches for each low-resolution image patch (S1102).

Then, the learning device may calculate a global transformation matrix (MR) by applying the high-resolution image candidate patch and the high-resolution image patch to a linear mapping calculation method (eg, kernel ridge regression, etc.) (S1104).

Meanwhile, in the foregoing embodiment, the high-resolution image patch has been described based on the 2x2 embodiment, but the high-resolution image patch can be expanded to various sizes such as 3x3 and 4x4. Preferably, the pixel size (number) of the low-resolution image patch is larger than the pixel size (number) of the high-resolution image patch.

Also, in the above-described embodiment, conversion of a low-resolution image into a high-resolution image by linear mapping has been described, but in some embodiments, a low-resolution image may be converted into a high-resolution image by non-linear mapping.

In addition, in the above-described embodiment, the low-resolution image patch has been described based on a rectangular embodiment, but the low-resolution image patch may have a polygonal shape of another shape (for example, a hexagonal shape). At this time, the high-resolution image patch preferably has the same or smaller polygonal shape.

As described above, according to the present embodiment, a low-resolution image can be converted into a high-resolution image within a short period of time. Also, according to the present embodiment, a low-resolution image can be converted into a high-resolution image using less storage space.

Terms such as "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless otherwise stated, and therefore do not exclude other components. It should be construed that it may further include other components. All terms, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the meaning in the context of the related art, and unless explicitly defined in the present invention, they are not interpreted in an ideal or excessively formal meaning.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims (12)

In the super-resolution method of converting a low-resolution image into a high-resolution image,
storing a global transformation matrix and sub-patch transformation matrices for each image category;
Determining an image category for each of a plurality of sub-patches included in the patch of the low-resolution image (low-resolution image patch), and selecting the sub-patch transformation matrix corresponding to each of the plurality of sub-patches according to the determined image category. ;
generating a high-resolution image patch by applying the global transformation matrix and the selected sub-patch transformation matrix to the low-resolution image patch; and
Generating the high-resolution image using the high-resolution image patch
including,
In the step of storing the sub-patch transformation matrix, the sub-patch transformation matrix is stored differently according to the image category and the location of the sub-patch in the low-resolution image patch;
In the step of selecting the sub-patch transformation matrix, the sub-patch transformation matrix corresponding to each of the plurality of sub-patches is selected according to the image category and the location of the sub-patch in the low-resolution image patch. According to claim 1,
The super-resolution method of selecting the low-resolution image patch for each pixel of the low-resolution image and converting the high-resolution image patch. According to claim 1,
The low-resolution image patch has a pixel size of NxM (N and M are natural numbers greater than or equal to 2), and the sub-patch has a pixel size of PxQ (P is a natural number smaller than N and Q is a natural number smaller than M). , The plurality of sub-patches are selected such that a partial region overlaps, and (N-P+1) x (M-Q+1) sub-patches are formed in the low-resolution image patch. According to claim 1,
In the step of generating the high-resolution image patch,
A plurality of high-resolution image candidate patches are generated by applying the sub-patch transformation matrix of each of the plurality of sub-patches to the low-resolution image patch, and the high-resolution image patch is obtained by applying the global transformation matrix to the plurality of high-resolution image candidate patches. How to generate super-resolution. According to claim 1,
In the step of generating the high-resolution image patch,
The super-resolution method of generating the high-resolution image patch by applying a patch transformation matrix generated by calculating a transformation matrix obtained by combining the global transformation matrix with the sub-patch transformation matrix of each of the plurality of sub-patches to the low-resolution image patch. According to claim 1,
In the step of selecting the subpatch transformation matrix,
The super-resolution method of generating a feature vector for each of the plurality of sub-patches, and determining an image category of each of the plurality of sub-patches according to the feature vector. According to claim 1,
In the step of selecting the subpatch transformation matrix,
Using at least one information of image edge direction information of the sub-patch, edge strength information, texture information, brightness information, first- or higher-order differential value information, color information, object or object type information, and frequency information, the plurality of A super-resolution method for determining the image category of each sub-patch of . delete According to claim 1,
The global transformation matrix and the sub-patch transformation matrix are learned and determined using a test high-resolution image and a test low-resolution image. In the super-resolution method of converting a low-resolution image into a high-resolution image,
converting the test high-resolution image into a test low-resolution image;
extracting a plurality of high-resolution image patches from the test high-resolution image and extracting a plurality of low-resolution image patches from the test low-resolution image;
determining an image category for each of a plurality of sub-patches included in the low-resolution image patch for each low-resolution image patch;
learning a sub-patch transformation matrix for transforming the low-resolution image patch into the high-resolution image patch for each image category;
A global transformation matrix for generating a plurality of high-resolution image candidate patches by applying the sub-patch transformation matrix for each of the plurality of sub-patches for each low-resolution image patch, and converting the plurality of high-resolution image candidate patches into the high-resolution image patch Step of learning; and
The image category for each of a plurality of sub-patches included in the patch of the low-resolution image is determined, and the low-resolution image is converted into the high-resolution image using the sub-patch transformation matrix learned for each image category and the global transformation matrix. step to do
including,
In the step of learning the subpatch transformation matrix,
The super-resolution method of learning the sub-patch transformation matrix for each location in the low-resolution image patch and for each image category. delete According to claim 10,
The super-resolution method in which the image category is generated by a k-means clustering technique using a plurality of sub-patches.

Images