JP4956464B2 - Image high resolution device, learning device and method - Google Patents

Description

  The present invention relates to an image resolution increasing apparatus, a learning apparatus, and a method for converting image data captured by a camera and image data received by a television into image data with higher resolution.

  In recent years, display devices such as televisions and displays with a large number of pixels have become widespread. If the number of pixels of the display device is large, it is possible to display up to a fine part of the image. That is, a high-resolution image can be displayed. When the number of pixels of the image data to be displayed is smaller than that of the display device, it is necessary to display after increasing the resolution by increasing the number of pixels.

As a technique for increasing the resolution, there is one in which pixels of a high-resolution image to be created are created by linear combination of values of pixels located in the vicinity in the time direction or the spatial direction in the low-resolution image (for example, Patent Document 1, Patent Document 2). The coefficient at the time of the linear combination is learned using a learning image prepared in advance. Since the coefficient is learned so as to minimize an error with respect to the learning image, an output high resolution image obtained by increasing the resolution of the input low resolution image using the coefficient is compared with an unknown true high resolution image. It can be expected that the error will be reduced.
Japanese Patent No. 3702464 Japanese Patent No. 4001143

  However, there is a problem that the image quality is still low even when the above-described high resolution technique is used.

  An object of the present invention is to provide an image high-resolution device, a learning device, and a method for creating a high-quality high-resolution image.

  In order to solve the above-described problem, an image resolution increasing apparatus according to the present invention includes an input unit that inputs a low-resolution image having a first size indicating the number of vertical and horizontal pixels, and a vertical size of the first size. The number of pixels and the number of vertical pixels of a second size larger than the first size, and the number of horizontal pixels of the first size and the number of horizontal pixels of the second size, And a first block size corresponding to the low-resolution image and a second block size corresponding to the high-resolution image of the second size in a processing unit for increasing the resolution. From the one or more sets of a determination means for determining the size of a certain block and a first learning block having the first block size and a second learning block having the second block size, the first learning block Apply the first coefficient to the block One or more first coefficients learned so as to reduce an error between the block calculated in this way and the second learning block are held, and the first block size and the second block size are determined from the one or more first coefficients. Corresponding to the first position of the block of the first block size in the low-resolution image and the first position of the block of the second block size. A setting means for sequentially setting a second position in the high-resolution image, and a second block at the second position is created by applying the second coefficient to the first block at the first position to create the high-resolution image. And creating means.

  The learning device of the present invention includes an input unit that inputs a high-resolution image for learning whose size indicating the number of vertical and horizontal pixels is a high size, and a low-resolution image for learning whose size is a low size smaller than the high size. From the learning high-resolution image, and from the learning low-resolution image and the learning high-resolution image, a first learning block having a low block size and a second learning block having a high block size, Extraction means for extracting a set of the first learning block, the first learning block of the low block size, and the high of the error between the block calculated by multiplying the first learning block by a first coefficient and the second learning block First calculation means for calculating the first coefficient that minimizes the sum of the block size and the second learning block, the low block size and the high block size; Holding means for associating and holding the first coefficient, wherein the high block size and the low block size are such that the number of vertical pixels of the high size and the number of vertical pixels of the low size are mutually prime. And the determined number of pixels having the same size as the number of horizontal pixels of the high size and the number of horizontal pixels of the low size. .

  In addition, the learning device of the present invention includes an input unit that inputs a high-resolution image for learning having a high size indicating the number of vertical and horizontal pixels, and a learning low-size that is a low size smaller than the high size. A creation means for creating a resolution image from the learning high-resolution image, and a first learning block having a low block size and a second learning block having a high block size from the learning low-resolution image and the learning high-resolution image. A vector belonging to a set of vectors indicating the high-block size blocks that become the first learning block by reducing the resolution of the first vector indicating the first learning block by extracting means for extracting a pair with the block Of the coefficient matrix to be converted into the vector obtained by multiplying the coefficient matrix by the first vector from the right and the vector indicating the second learning block. First calculation means for calculating the coefficient matrix having the smallest sum for the set of the first learning block having the low block size and the second learning block having the high block size, and the low block size and Holding means for associating and holding the high block size and the coefficient matrix, wherein the high block size and the low block size are the number of vertical pixels of the high size and the vertical pixels of the low size. The number is determined based on a relatively prime number, and the number of horizontal pixels of the high size and the number of horizontal pixels of the low size are relatively prime, or the determined number is input. It is characterized by that.

  According to the image resolution increasing device, the learning device, and the method of the present invention, a high-quality high-resolution image can be created.

  Hereinafter, an image resolution increasing apparatus, a learning apparatus, and a method according to an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, the same numbered parts are assumed to perform the same operation, and repeated description is omitted. Further, the embodiment is not limited to the embodiment described later, and may be changed. For example, you may change by the example of a below-mentioned change, and may change by combining those examples of change.

First, terms are briefly explained.
A digital image consists of values having color and brightness information at each position. A unit having this value is called a pixel, and the value is called a pixel value. Hereinafter, in order to simplify the description, a case will be described in which only the pixel value has a value representing luminance. In this case, there is one pixel value per pixel. As a case where a value having color information is given as a pixel value, for example, a value in an RGB color space in which color information about red, green and blue is expressed in a three-dimensional space may be given as a pixel value. In this case, three values for red, green, and blue are pixel values per pixel. Instead of the RGB color space, a value in the YUV space may be given as the pixel value.

  In order to increase the resolution of a digital image, it is necessary to increase the number of pixels. Increasing the number of pixels is sometimes called increasing the resolution of an image. In some cases, enlargement of an image and higher resolution are used in the same meaning, and reduction of an image and lower resolution are used in the same meaning.

  A rectangular partial image in the image is called a block. In the following, a block is represented by a vertical vector in which pixel values of the block are arranged. The representation of the block may be a horizontal vector instead of a vertical vector, but here it is a vertical vector. For simplification of explanation, a vector may be called a block or a block may be called a vector without distinguishing the block and the vector.

  A block that is expanded and has pixels in the temporal direction is called a spatiotemporal block. In order to distinguish from a spatiotemporal block, a block having no pixels in the temporal direction may be referred to as a spatial block. In some cases, the spatial block and the spatio-temporal block are simply referred to as a block without being distinguished.

The image resolution increasing apparatus of this embodiment will be described with reference to FIG.
The image resolution increasing apparatus 100 includes a block size determination unit 101, a coefficient data holding unit 102, a block position control unit 103, and a creation unit 104.
The input to the image resolution increasing apparatus 100 is a low resolution image 105 and size data 106. The size data 106 is data of a size that is the number of vertical and horizontal pixels of the low-resolution image 105 and data of the size of the high-resolution image 110 to be created. Hereinafter, the number of vertical pixels of the image is referred to as height, and the number of horizontal pixels is referred to as width. The width and height of the low resolution image 105 are denoted by w _l and h _l , respectively. The width and height of the high resolution image 110 are represented by _wh and _hh , respectively. The low resolution image 105 is sent to the creation unit 104. Size data 106 _{(w l} and _{h l} and _{w h} and _{h h)} is sent to the block size determination unit 101 and the block position controller 103. The output from the image resolution increasing device 100 is a high resolution image 110. The high resolution image 110 is output from the creation unit 104. An example of the low resolution image 105 and the high resolution image 110 is shown in FIGS. 2 (a) and 2 (b), respectively. In this _example, a _{w l = 9, h l =} 12, w h = 24, h h = 27.

Block size determining unit 101, the size data 106 input _{(w l} and _{h l} and _{w h} and _{h h)} to determine the block size data 107 is output, sends it to the coefficient data holding unit 102 and the block position controller 103 . The block size data 107 is data indicating the size of a block which is a unit of high resolution processing. The block size data 107 includes data indicating the size of a block corresponding to the low resolution image 105 and the size of a block corresponding to the high resolution image 110. Data. Hereinafter, blocks in the low resolution image 105 are referred to as low resolution blocks, and blocks in the high resolution image 110 are referred to as high resolution blocks. The width and height of the low-resolution block necessary for creating the high-resolution block are represented by c _l + i and r _l + i, respectively. c _l a _{r l} and i block position control unit 103 after about, described (1-1). Respectively the width and height of the high-resolution block represented by c _h a r _h. That is, the block size data 107 includes _{c l} and _{r l} and i and _{c h} a _{r h.} Method for determining the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} will be described in (1-1) after.

The coefficient data holding unit 102 reads the corresponding coefficient data 108 from the input block size data 107 (c ₁ , r ₁ , i, c _h and r _h ), and sends it to the creation unit 104. The coefficient data 108 is a coefficient for creating a high resolution block from a low resolution block, and is represented by F. A (c _l + i) (r _l + i) dimensional vertical vector in which the values of each pixel of the low resolution block are arranged is represented by z. A c _h r _h dimensional vertical vector in which the estimated values of the pixels of the high resolution block are arranged is represented by x. The coefficient data 108 (F) is a coefficient matrix for calculating x from z with the formula x = Fz. The size of F is c _h r _h rows (c _l + i) (r _l + i) columns. F is learned in advance, and a plurality of F are held in the coefficient data holding unit 102. The learning method of F will be described later in (1-1).

Block position control section 103, since the size data 106 input _{(w l} and _{h l} and _{w h} and _{h h)} block size data 107 input and _{(c l} and _{r l} and i and _{c h} a _{r h),} The position data 109 is controlled so as to be sequentially set, output, and sent to the creation unit 104. The position data 109 is data indicating the position of the low resolution block (z or y described later) in the low resolution image 105 and the position of the high resolution block (x) in the high resolution image 110 to be created. The block position control unit 103 controls the position data 109 so as to fill the high resolution image 110 with high resolution blocks, for example. The block position control unit 103 controls the position of the high resolution block in the order shown in FIG. 3, for example. The position of the low resolution block is synchronized with the position of the high resolution block. For this purpose, the block position control unit 103 moves the high-resolution block in the horizontal direction by c _h, controlled by moving each r _h in the vertical direction, the low resolution required to create the high-resolution block Go by c _l a block in the horizontal direction, the vertical direction may be controlled by moving one by r _l. Hereinafter, an example of this control will be described. However, the block position control unit 103 of the present embodiment is not limited to this case. The position data 109 may be controlled so that only some high-resolution blocks in the high-resolution image 110 are created according to this embodiment. For example, only the high-resolution block located near the center in the high-resolution image 110 may be created in this embodiment, and the other portions may be created by another method. Further, the high-resolution image 110 is separated into a flat part, a part with many edges, and a part with a texture, and the position data 109 is controlled so that a high-resolution block with a texture is created in this embodiment. May be. For this purpose, it is necessary to input the separation result to the block position control unit 103 or to input the low-resolution image 105 so that the separation can be performed.

  The creation unit 104 creates and outputs a high resolution image 110 from the low resolution image 105, coefficient data 108 (F), and position data 109 according to the following formula.

x = Fz
When certain position data 109 is given, x indicated by the position data 109 is created by multiplying z indicated by the position data 109 from the left. Each time the position data 109 controlled by the block position control unit 103 changes, the creation unit 104 sequentially creates high-resolution blocks in the high-resolution image 110, and all the high-resolution blocks in the high-resolution image 110 are created. The high resolution image 110 is created.

Next, an example of the operation of the image resolution increasing apparatus in FIG. 1 will be described with reference to FIG.
At S401, the block size determination unit 101 determines the size data 106 _{(w l} and _{h l} and _{w h} and _{h h)} from the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h),} S402 Proceed to

In S402, reads out the coefficient data 108 coefficient data holding unit 102 corresponds from the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} (F), the process proceeds to S403.

  In step S403, the block position control unit 103 uses the size data 106 and the block size data 107 to determine the position of the high resolution block in the high resolution image 110 to be created and the low resolution image 105 necessary for creating the high resolution block. The position data 109 indicating the position of the middle low resolution block is determined, and the process proceeds to S404.

  In S404, the creation unit 104 creates a high-resolution block indicated by the position data 109 in the high-resolution image 110 to be created from the low-resolution block (z) and the coefficient data 108 (F) in the low-resolution image 105 indicated by the position data 109. (X) is created by the equation (x = Fz), and the process proceeds to S405.

  In S405, the creation unit 104 determines whether or not the high-resolution block (x) indicated by the position data 109 is controlled to fill the high-resolution image 110. If the high-resolution block 110 is filled, the process ends. If not, the process returns to S403. Here, the processing termination condition is whether or not the high-resolution block (x) indicated by the position data 109 is controlled to fill the high-resolution image 110, but may be determined based on other criteria.

(1-1) Method for Determining Block Size Data 107 (c _l , r _l , i, c _h , r _h ) Next, block size data 107 (c _l , r _l , i determined by the block size determining unit 101) And the determination method of c _h and r _h ) will be described.
Let c _h and c _{l be} values in which w _h and w _l are prime. For example lowest terms fractional that w _{h /} w _l showing a high resolution rate of lateral to irreducible fraction. Euclidean mutual division can be used for reduction. The denominator is the molecule of the irreducible fraction number and c _h a c _l. Let r _h and r _{l be} values in which h _h and h _l are relatively prime. For example, a fraction of h _h / h _l indicating the high resolution ratio in the vertical direction is reduced to an irreducible fraction. Let the numerator of the irreducible fraction be r _h and the denominator be r _l .

Next, referring to FIG. 5A and FIG. 5B for the low resolution block and the high resolution block in the case of i = 0 in the low resolution image (a) and the high resolution image (b) of FIG. 2, respectively. To explain. When i = 0, the low resolution block and the high resolution block are blocks showing the same part of the subject. Was calculated disjoint in value is to calculate the size of the low-resolution block and the high-resolution block is block diagram showing the same portion of the object (c _l and r _l and c _h a r _h). In this case, the low resolution block and the high resolution block are spread on the low resolution image 105 and the high resolution image 110 without overlapping. Determine the size of the low-resolution block and the high-resolution block indicating the same part of the subject (c _l and r _l and c _h a r _h) are those having a low resolution of the high-resolution block created, input of a low resolution This is to approach the block. When the resolution of the created high resolution block does not match the input low resolution block, an error corresponding to the mismatch does exist in the created high resolution block. In order to prevent the occurrence of the error to determine the size of the low-resolution block and the high-resolution block indicating the same part of the subject (c _l and r _l and c _h a r _h).

  Next, for the low resolution block and the high resolution block in the case of i = 2 in the low resolution image (a) and the high resolution image (b) of FIG. 2, respectively refer to FIG. 6 (a) and FIG. 6 (b). To explain. When i is positive, the low resolution block indicates a wider range than the high resolution block. In order to create a certain high resolution block, a low resolution block showing a wider range than that is used, so that high image quality can be obtained.

Next, FIG. 7A and FIG. 7B are respectively shown for the low resolution block and the high resolution block in the case of i = −2 in the low resolution image (a) and the high resolution image (b) of FIG. The description will be given with reference. When i is negative, the low resolution block shows a narrower range than the high resolution block. In order to create a certain high-resolution block, only a low-resolution block showing a narrower range is used, so that high image quality cannot be obtained.
Therefore, in order to obtain high image quality, the block size determination unit 101 may determine the block size data 107 by setting i to 0 or more.

A vector in which the pixel values of the low resolution block in the case of i = 0 is arranged is represented by y. y is a c _l r _l- dimensional vector. It is desirable that the high resolution block x be similar to the low resolution block y due to the reduction in resolution. When they do not match, the reason why they do not match is that x includes an error corresponding to the mismatch. Therefore, in order to obtain a high image quality, it is only necessary that the generated x with a reduced resolution approaches y.

  In the following, it is assumed that the resolution is reduced according to the area ratio when the high resolution block and the low resolution block are overlapped. Here, the resolution reduction according to the area ratio is assumed, but other resolution reduction processes may be assumed. For example, it is possible to assume a process of lowering the resolution by blurring a high resolution block and subsampling to become a low resolution block. You can easily change the size of the vector or matrix if you note that it can change.

The reduction in the resolution of the high resolution block x according to the area ratio when the high resolution block and the low resolution block are overlaid is represented by Wx. W is a matrix indicating the process of resolution reduction according to the area ratio. x is a vertical vector of c _h r _h dimension, and W is a matrix of c _l r _l rows c _h r _h columns. FIG. 8 is an example of resolution reduction in the case of c _h = r _h = 4 and c _l = r _l = 1. In this case, x is a 16-dimensional vector, and W is a 1 × 16 matrix. Since the elements of W are determined according to the area ratio, they are all 1/16. If c _l and r _l and c _h a r _h is the other value, depending on its value, the values of the matrix size and the elements of W changes.

  When the high resolution block x coincides with the low resolution block y due to the reduction in resolution, the following equation is established.

y = Wx
The general form of x that satisfies this equation is expressed by the following equation.

x = W ⁺ y + (I-W ⁺ W) v
Here, + represents a general inverse matrix. The size of the matrix of W ⁺ is c _h r _h rows c _l r _l columns. I represents a unit matrix of c _h r _h rows and c _h r _h columns. v is an arbitrary vector of the same dimension as x. Since an arbitrary vector v is included in this general form, it can be seen that x that satisfies the expression (y = Wx) is not uniquely determined. Figure 9 is a high-resolution block x ₁ and x ₂ are the low resolution, which is an example showing how the both of the same low-resolution block y. Also from this example, it can be seen that x satisfying the equation (y = Wx) cannot be uniquely obtained from y.

The first term on the right side of the formula (x = W ⁺ y + (I−W ⁺ W) v) includes y. Therefore, in order to obtain x that satisfies the equation (y = Wx), y is required. y is a vector in which the values of the pixels of the low resolution block when i is 0 are arranged. Therefore, in order to obtain a high-resolution image 110 with high image quality, the block size determination unit 101 may determine the block size data 107 by setting i to 0 or more.

  In the conventional method, a coefficient is learned and the resolution is increased by using the coefficient. However, no index has been given as to which area in the low-resolution image should be used to create a certain pixel in the high-resolution image. For example, consider a case where 16 pixels in the vicinity (4 pixels each in the vertical and horizontal directions) in the low resolution image are used in order to create a certain pixel in the high resolution image. Thereby, the pixel in the high resolution image can be calculated. The same calculation can be performed for other pixels. However, when the created high-resolution image is reduced in resolution, there is no guarantee that it matches the input low-resolution image. If they do not match, an error that does not match exists in the created high-resolution image, and the image quality is low. In order to prevent the error from occurring, conventionally, it is necessary to determine pixels to be used in a low-resolution image by trial and error. On the other hand, according to the present embodiment, it is clear which pixel in the low resolution image should be used in order to calculate the value of the pixel in the high resolution image.

(1-2) First Learning Method for Coefficient Matrix F Next, a first learning device and method for the coefficient matrix F will be described.
The learning apparatus 1000 for the coefficient matrix F will be described with reference to FIG.
The coefficient matrix F learning apparatus 1000 includes a low-resolution image creation unit 1001, a block extraction unit 1002, a coefficient data learning unit 1003, and a coefficient data holding unit 102. The input of the coefficient matrix F to the learning apparatus 1000 is size data 1004, a learning high resolution image 1005, and block size data 1006. The learning high resolution image 1005 is a learning high resolution image used for learning the coefficient matrix F. The learning high-resolution image 1005 is sent to the low-resolution image creation unit 1001 and the block extraction unit 1002. The size data 1004 is the size (width and height) of the learning high resolution image 1005. The size data 1004 is sent to the low resolution image creation unit 1001. Block size data 1006 is _{c l} and _{r l} and _{c h} a _{r h} and i. The block size data 1006 is sent to the low resolution image creation unit 1001, the block extraction unit 1002, the coefficient data learning unit 1003, and the coefficient data holding unit 102. The learning apparatus 1000 for the coefficient matrix F causes the coefficient data holding unit 102 to hold the coefficient data 1009.

Low-resolution image generating unit 1001, the size data 1004 input, the high resolution image 1005 for learning input, from _{c l} and _{r l} and _{c h} a _{r h} of the block size data 1006 input, low learning A resolution image 1007 is created and sent to the block extraction unit 1002. The learning low resolution image 1007 is created by reducing the resolution of the learning high resolution image 1005. The learning low-resolution image 1007 may be created so as to satisfy the equation (y = Wx). The learning low resolution image 1007 may be further blurred. When the resolution is increased using a coefficient learned to convert a blurred low-resolution image into a high resolution, the created high-resolution image becomes sharper. Here, x represents a high resolution block having a width c _h and a height r _h in the learning high resolution image 1005. y has a width and a height c _l in the low-resolution image 1007 for learning the x and position corresponding represents a low-resolution block located at r _l. Since the low resolution image creating unit 1001 does not use i, it is not necessary to send i to the low resolution image creating unit 1001.

The block extraction unit 1002 extracts the learning block 1008 from the input learning high resolution image 1005, the input block size data 1006 and the input learning low resolution image 1007, and sends the learning block 1008 to the coefficient data learning unit 1003. Learning block 1008, the width in the training for the high-resolution image 1005 is a learning high-resolution block height c _h is r _h, in the low-resolution image 1007 for learning corresponding to the high-resolution block for learning It consists of a set with a low resolution block for learning whose width is (c ₁ + i) and height is (r ₁ + i). If the number of learning high-resolution images 1005 is increased, the number of sets can be increased. The greater the number of pairs, the more effective the learning.

The coefficient data learning unit 1003 learns coefficient data 1009 from the block size data 1006 and the learning block 1008 and sends it to the coefficient data holding unit 102. The coefficient data 1009 is a coefficient matrix F described below. Of the learning block 1008, a c _h r _h- dimensional vertical vector indicating a learning high-resolution block is represented by x _n (n = 1, 2,..., N), and indicates a learning low-resolution block (c _l + i) (r _l + i) A dimensional vertical vector is represented by z _n (n = 1, 2,..., N). Here, N represents the number of learning sets. The low-resolution block _{z n} in the case of the i = 0 expressed by _{y n.} The low resolution block y _n is a part of the low resolution block z _n where i is positive. Low-resolution block y _n and the high-resolution block x _n represent the same part of the subject. Square error e _n when the low-resolution block z _n and high resolution with coefficient matrix F is expressed by the following equation.

Here, F is an unknown number. The sum E of square errors e _n (n = 1, 2,..., N) for all learning block sets is expressed by the following equation.

If F that minimizes E is held in the coefficient data holding unit 102, the square error with respect to the unknown true high-resolution block when the low-resolution block of the unknown input is increased in resolution by F will be obtained. Expect to be smaller. In order to obtain F that minimizes E, when E is partially differentiated by F, the following equation is obtained.

Here, T represents transposition of a matrix or a vector.

Is set to 0, the following formula

Is obtained.

If N is large and z _n are not similar to each other, it becomes a regular matrix and has an inverse matrix. Therefore, F that minimizes E can be calculated by the following equation.

Here, −1 represents an inverse matrix. If i is greater than or equal to 0, the high-resolution block x obtained by F in this way often satisfies the equation (y = Wx). But there is no guarantee of satisfaction. if,

If is not regular, F may be calculated by the following equation.

Here, + represents a general inverse matrix.

Coefficient data holding unit 102 holds the block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i) and the coefficient data 1009 (F) a set. Thus, the coefficient data 1009 corresponding to a certain block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i) (F) is held.

Next, an example of the operation of the learning apparatus 1000 in FIG. 10 will be described with reference to FIG.
In S1101, it is input to the block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i) a low-resolution image generating unit 1001 and the block extracting unit 1002 and the coefficient data learning unit 1003 and the coefficient data holding unit 102 .

  In S1102, one of the high-resolution images prepared for learning is input as a learning high-resolution image 1005 to the low-resolution image creating unit 1001 and the block extracting unit 1002, and the size of the learning high-resolution image 1005 is the size. Input as data 1004.

In step S <b> 1103, the low-resolution image creation unit 1001 determines the learning low-resolution image 1007 from the size data 1004, the learning high-resolution image 1005, and the c _l , r _l , c _h , and r _{h of the} block size data 1006. Create

  In S1104, the block extraction unit 1002 extracts a set of a learning high resolution block and a corresponding learning low resolution block from the learning high resolution image 1005, the block size data 1006, and the learning low resolution image 1007. As the number of pairs to be extracted increases, the learning effect increases. However, it is not necessary to extract a block (flat block) in which the fluctuation of the pixel value is small in the block.

  In S1105, it is confirmed whether or not all the high resolution images prepared for learning have been input. If they have been input, the process proceeds to S1106, and if not, the process returns to S1102. This confirmation work is executed by user input from outside the learning apparatus 1000. Alternatively, the number of high-resolution images prepared for learning is included in the size data 1004, and the function of counting the number of learning high-resolution images 1005 input by the low-resolution image creation unit 1001 is provided. The resolution image creation unit 1001 may be executed.

In step S1106, the coefficient data learning unit 1003 calculates coefficient data 1009 (F) from the block size data 1006 and all of the extracted learning high-resolution blocks and corresponding learning low-resolution blocks (learning blocks 1008). learn coefficient data holding unit 102 holds in its coefficient data 1009 (F) and a set block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i).

(1-3) Second Learning Method for Coefficient Matrix F The high-resolution block x by F learned by the first learning method for the coefficient matrix F is not guaranteed to satisfy the equation (y = Wx). A second learning device and method for the coefficient matrix F that can satisfy the equation (y = Wx) will be described.
The second learning device for the coefficient matrix F is the same as the first learning device, as shown in FIG. However, only the calculation method of F in the coefficient data learning unit 1003 is different. Further, the flow of processing of the second learning method of the coefficient matrix F is the same as that of the first learning method, as shown in FIG. However, only the calculation method of F in S1106 is different. Therefore, only the calculation method of F will be described.

In order to explain the calculation method of F, decomposition of the c _l r _l- dimensional vertical vector will be described. column vector _c h _{r h} dimensions such as x _{n is,} c h _{r h} dimensional Euclidean space

It is one of the points. This space is an orthogonal direct sum of R (W ^T ) and N (W). Here, R (W ^T ) is
R (W ^T ) = {x | x = W ^T y}
A range of defined W ^T by. N (W) is
N (W) = {x | Wx = 0}
The zero space of W defined by, where 0 represents a c _l r _l dimensional vertical vector with all elements being 0. From this orthogonal direct sum relationship, the c _h r _h dimensional Euclidean space

Arbitrary c _h r _h dimensional vertical vectors x can be uniquely decomposed into R (W ^T ) elements and N (W) elements as follows:

x = W ⁺ Wx + (I−W ⁺ W) x
Here, W ⁺ W is an orthogonal projection matrix to R (W ^T ), and I − W ⁺ W is an orthogonal projection matrix to N (W).

P = I-W ⁺ W
Then, the vertical vector x of the c _h r _h dimension is expressed as x = W ⁺ Wx + Px
And can be decomposed uniquely.

  The general form of the coefficient matrix F that satisfies the equation (y = Wx) is given by the following equation.

F = S + PX
Where S is the Sz ⁼ W + Wx = ^{W +} y satisfies _c h _{r h} line _{(c l + i) (r} l + i) column of the matrix, with a ^{W +} components and 0 to the element . X is an arbitrary matrix of c _h r _h rows (c _l + i) (r _l + i) columns. F is a set of (c _l + i) (r _l + i) -dimensional vertical vectors z representing low-resolution blocks, and c _h r _h- dimensional vertical vectors indicating high-resolution blocks that become y by resolution reduction. It is a general form of a coefficient matrix that converts to a vector belonging to a certain linear manifold. F is learned by obtaining PX that minimizes E under the constraint that F is represented by this formula (F = S + PX). Since P is an orthogonal projection matrix, P ^T = P and P ² = P are established. Therefore,

It becomes. Since P is a constant, if E is partially differentiated with X instead of PX, the following equation is obtained:

Is obtained.

Is set to 0, the following formula

Is obtained.

If N is large and z _n are not similar to each other, an inverse matrix is provided, so that PX that minimizes E can be calculated by the following equation.

A value obtained by substituting this into the formula (F = S + PX) is F to be obtained. The high-resolution block x by F satisfies the expression (y = Wx). if,

If does not have an inverse matrix, PX may be calculated by the following equation.

The fact that the low resolution block WFz which has been reduced in resolution by applying W from the left to Fz obtained by increasing the resolution of the low resolution block z by F thus obtained matches y is that WW ⁺ is a unit matrix, It is clear from W (I−W ⁺ W) being a zero matrix.

(1-4) Effects of the Present Embodiment According to the present embodiment, which pixel in the low-resolution image should be used to calculate the value of the pixel in the high-resolution image becomes clear. When a coefficient for increasing the resolution is learned using the pixel and the resolution of a high-resolution block created using the coefficient is reduced, the input approaches or coincides with the input. This reduces the error. That is, there is an effect of improving the image quality.

(1-5) First modification example: Modification example in which blocks that cannot be expressed by linear combination are learned and added
A modification example for further increasing the resolution of this embodiment will be described. In the present embodiment, the resolution is increased by the equation (x = Fz). With this high resolution, only a high resolution block represented by a linear combination of a low resolution block and a coefficient can be created. However, true and unknown high-resolution blocks are not always represented by their linear combination. Therefore, a component that cannot be expressed by the linear combination is learned and changed to be added.
An image resolution increasing apparatus 1200 according to a first modification will be described with reference to FIG. A connection between the addition data holding unit 1201 and the addition data holding unit 1201 is added to the configuration of FIG. Only the changes will be described.

Block size determining unit 101, the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h),} and sends to the addition data holding section 1201.

Addition data holding section 1201, the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h),} reads the sum data 1203 corresponding to this data, and sends the creating unit 1202. Here, the addition data 1203 is learned for each block size data 107 (c ₁ , r ₁ , i, c _h and r _h ) and held in the addition data holding unit 1201. A learning method for the addition data 1203 will be described later.

  The creation unit 1202 creates and outputs a high-resolution image 110 from the input low-resolution image 105, the input coefficient data 108 (F), the input position data 109, and the input addition data 1203. Each time the position data 109 controlled by the block position control unit 103 changes, high-resolution blocks x in the high-resolution image 110 are sequentially created.

  A method of creating the high-resolution block x in the addition data 1203 and the creation unit 1202 will be described. In the first modification, a high resolution block x is created by the following formula.

x = Fz + h
Here, h is a vertical vector having the same dimension as x. F is learned by the method described above. This h is learned after learning F. The h learned in advance is held in the addition data holding unit 1201 as addition data 1203.

A learning method of the addition data 1203 (h) will be described. Square error e _n when the high resolution low-resolution block z _n at F and h is expressed by the following equation.

Here, F is a constant and h is an unknown number. Learning h that minimizes the sum E of square errors e _n (n = 1, 2,..., N) for all learning block sets. By partial differentiation of E with h,

Is obtained.

If is set to 0, the following equation is obtained.

Therefore, h that minimizes E can be learned by the following equation.

Next, an example of the operation of the image resolution increasing device 1200 of FIG. 12 will be described with reference to FIG. In FIG. 13, S1301 is added to the processing flow of FIG. Only the changes will be described.
After S402, the process proceeds to S1301, not S403.
In S1301, the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h),} reads the corresponding added data 1203 (h), the process proceeds to S403.
In S404, the position data 109 in the high-resolution image 110 to be created is created from the low-resolution block (z) in the low-resolution image 105 indicated by the position data 109, the coefficient data 108 (F), and the addition data 1203 (h). The resolution block (x) is created by the equation (x = Fz + h), and the process proceeds to S405.

  Next, a learning apparatus 1400 for the coefficient matrix F and the addition data 1203 will be described with reference to FIG. In the learning device 1400, an addition data learning unit 1401, an addition data holding unit 1201, and a connection between them are added to the configuration of FIG. Only the changes will be described.

Block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i) are sent to addition data learning unit 1401 and the addition data holding section 1201. The learning block 1008 is also sent to the addition data learning unit 1401. The coefficient data 1009 (F) is also sent to the addition data learning unit 1401.

Addition data learning unit 1401, since the block size data 1006 and _{(c l} and _{r l} and _{c h} a _{r h} and i) the learning block 1008 and the coefficient data 1009 (F), to learn the added data 1402, adds the data Send to learning unit 1401. The addition data 1402 is h. The learning method is as described above.

Addition data holding section 1201, and holds the block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i) and the sum data 1402 (h) to set. Thus, a certain block size data 1006 _{(c l} and _{r l} and _{c h} a _{r h} and i) corresponding to the added data 1402 (h) is maintained.

An example of the operation of the learning device 1400 in FIG. 14 will be described with reference to FIG. In FIG. 15, S1501 is added to the processing flow of FIG. Only that part will be described.
At S1501, the block size data 1006 and _{(c l} and _{r l} and _{c h} a _{r h} and i) from the learning block 1008 and the coefficient data 1009 (F), to learn the added data 1402, the block size data 1006 _{(c l} , R _l , c _h , r _h and i) and the addition data 1402 (h) are held in pairs.

  According to the first modification, two-stage learning is performed in which additional data is learned using the learned coefficient data. Since the added data is a component that cannot be expressed by linear combination using coefficient data, there is an effect of increasing the resolution.

(1-6) Second modification example: Modification example in which blocks that cannot be expressed by linear combination are learned and added
In the first modification, a large amount of learning data (x _n , z _n, and y _n ) is necessary to learn h. This is because each element of h can have a wide range, and h is a high-dimensional vector of c _h r _h dimensions. So, was changed in order to obtain high image quality even with a small number N of data for learning (x _n and z _n and y _n) is an this modification.

Also in this modified example, the configuration of the image resolution increasing apparatus is as shown in FIG. Only the differences from the first modification will be described. In the present modification example, h learned by the method described later is held in the added data holding unit 1201 as the added data 1203. The creation unit 104 creates a high-resolution block x by the following formula using the added data holding unit 1201 (h learned by a method described later).
x = Fz + ‖M‖h
Here, F is learned by the above method. M is a vertical vector having the same dimension as x. For example, M is defined by the following formula.
M = Fz−w
Here, w is a vertical vector of the same dimension as x indicating a high resolution block obtained by increasing the resolution of z by nearest neighbor interpolation, or
w = W ⁺ y
Is a vector defined by When learning h, M is calculated from the low-resolution blocks for learning (z _n and y _n ). Since it depends on n, M at the time of learning is expressed as M _n .

M _n = Fz _n - w _n
Here, w _n is a vertical vector of the same dimension as x _n indicating a high resolution block obtained by increasing the resolution of z _n by nearest neighbor interpolation, or
w _n = W ⁺ y _n
Is a vector defined by Square error e _n when the high resolution low-resolution block z _n at F and h is expressed by the following equation.

Here, F and M _n are constants, and h is an unknown number. Learning h that minimizes the sum E of square errors e _n (n = 1, 2,..., N) for all learning block sets. By partial differentiation of E with h,

Is obtained.

If is set to 0, the following equation is obtained.

Therefore, h that minimizes E can be learned by the following equation.

Also in the second modified example, the flow of processing of the image high-resolution method is as shown in FIG. Only the differences from the first modification will be described.
In S404, the position data 109 in the high-resolution image 110 to be created is created from the low-resolution block (z) in the low-resolution image 105 indicated by the position data 109, the coefficient data 108 (F), and the addition data 1203 (h). The resolution block (x) is created by the equation (x = Fz + ‖M‖h), and the process proceeds to S405.

  The configuration of the learning device of the added data holding unit 1201 of the second modification is the same as that of the first modification, as shown in FIG. However, the learning method of the addition data 1402 (h) in the addition data learning unit 1401 is changed from the method described in the first modification to the method shown here.

  Furthermore, the processing flow of the learning method of the addition data holding unit 1201 of the second modification is the same as that of the first modification, as shown in FIG. However, the learning method of the addition data 1402 (h) in S1501 is changed from the method described in the first modification to the method shown here.

According to the second modification, it is possible to efficiently learn components that cannot be expressed by linear combination using coefficient data. Therefore, high image quality can be obtained when the number N of learning data (x _n , z _n, and y _n ) is not large.

(1-7) Third modification example: Modification example for emphasizing a low-resolution image
In order to create a sharp high-resolution image 110, a modification example in which a low-resolution image is emphasized will be described.

  In the present modification example, the creation unit 104 emphasizes the low resolution image 105. The low resolution block is a block in the emphasized low resolution image 105. An unsharp mask or Laplacian filter can be used for emphasis. Since the low resolution image 105 is sharply emphasized, the created high resolution image 110 becomes sharp.

(1-8) Fourth modification example: Modification example in which an image for learning is emphasized
In order to create a sharp high-resolution image 110, a modification example in which a learning image is emphasized will be described.

In the present modification example, the high-resolution block _xn for learning is sharply emphasized, and then the coefficient data and the addition data are learned. An unsharp mask or Laplacian filter can be used for emphasis. The high-resolution block _xn for learning may be emphasized, or the original image including _xn may be emphasized. Thereby, coefficient data and addition data that can create a sharper high-resolution block are learned. Therefore, a sharper high-resolution image 110 can be created.

(1-9) Fifth modification example: Modification example using labeling
In order to further improve the image quality, a modified example using labeling will be described. By using labeling, coefficient data and addition data are learned for each label. Thereby, the image quality can be further improved. Here, labeling is used in the same meaning as classification.

  In the present modification, a case where labeling is further used in the second modification will be described. However, the change using labeling is not only applicable to the second modification example. The present embodiment is not applicable to other modified examples. Further, the present invention can be applied to other modified examples.

An image resolution increasing apparatus 1600 according to a fifth modification will be described with reference to FIG.
The connection between the labeling unit 1601 and the labeling unit 1601 is added to the configuration of the first modified example in FIG. Only changes from the second modification will be described.

The low resolution image 105 is also sent to the labeling unit 1601. The block position control unit 103 also sends the position data 109 to the labeling unit 1601.
The labeling unit 1601 determines and outputs the label data 1604 of the low resolution block z or y indicated by the position data 109 according to a predetermined rule, and sends it to the coefficient data holding unit 1602 and the addition data holding unit 1603. As a rule for determining the label data 1604, for example, vector quantization, ADRC, incremental code, or the like is used. Since it is only necessary to label the low-resolution block, for example, edge detection may be performed on the low-resolution block, and vector quantization, ADRC, incremental code, or the like may be used for the result. The labeling unit 1601 converts the low resolution block z or y according to a predetermined rule, and outputs data obtained by labeling (classifying) the converted block according to the value included in the block as label data 1604. Since the converted one shows a characteristic amount peculiar to the conversion, the labeling unit 1601 labels the low resolution block z or y according to the characteristic amount. The labeling unit 1601 performs ADRC on, for example, a low-resolution block (represented by a vector), further binarizes, and labels low-resolution blocks corresponding to the number of vectors that can be obtained by binarization. The label data 1604 of the block z or y is output. For example, when a low-resolution block is represented by N columns of vertical vectors, it can be finally clustered to 2 ^N by binarization.

Coefficient data holding section 1602, the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} and the label data 1604, reads the corresponding coefficient data 108 (F), and sends the creating unit 1202. Here, the coefficient data 108 (F) is learned for each of the block size data 107 (c ₁ , r ₁ , i, c _h and r _h ) and the label data 1604 and held in the coefficient data holding unit 1602. The learning method for F will be described later.

The addition data holding unit 1603 reads the corresponding addition data 1203 (h) from the block size data 107 (c ₁ , r ₁ , i, c _h and r _h ) and the label data 1604, and sends it to the creation unit 1202. Here, the addition data 1203 (h) is learned for each of the block size data 107 (c ₁ , r ₁ , i, c _h and r _h ) and the label data 1604 and held in the addition data holding unit 1201. The learning method for h will be described later.

  The creation unit 1202 creates and outputs a high-resolution image 110 from the low-resolution image 105, the coefficient data 108 (F), the position data 109, and the addition data 1203 (h). Each time the position data 109 controlled by the block position control unit 103 changes, high-resolution blocks x in the high-resolution image 110 are sequentially created. The high resolution block x is created by the equation (x = Fz + ‖M‖h).

Next, an example of the operation of the image resolution increasing apparatus in FIG. 16 will be described with reference to FIG.
In S1701, it determines the block size data 107 block size determining unit 101 from the size data 106 _{(w l} and _{h l} and _{w h} and _{h h),} the process proceeds to S1702.

  In S1702, the block position control unit 103 determines the position of the high resolution block (x) in the high resolution image 110 to be created and the low resolution in the low resolution image 105 necessary for creating the high resolution block (x). The position data 109 indicating the position of the block (z or y) is determined, and the process proceeds to S1703.

  In S1703, the labeling unit 1601 determines the label data 1604 of the low resolution block (z or y) indicated by the position data 109 according to a predetermined rule, and the process proceeds to S1704.

In S1704, it reads the coefficient data 108 coefficient data holding unit 1602 corresponds the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} and the label data 1604 (F), the process proceeds to S1705.

In S1705, read the added data 1203 addition data holding unit 1603 corresponds the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} and the label data 1604 (h), the process proceeds to S1706.

  In S1706, the creation unit 1202 creates a position in the high-resolution image 110 to be created from the low-resolution block (z), the coefficient data 108 (F), and the addition data 1203 (h) in the low-resolution image 105 indicated by the position data 109. The high resolution block (x) indicated by the data 109 is created by the equation (x = Fz + ‖M‖h), and the process proceeds to S1707.

  In S1707, the creation unit 1202 determines whether or not the high resolution block (x) indicated by the position data 109 is controlled so as to fill the high resolution image 110. If the high resolution block 110 is filled, the processing ends and the filling is completed. If not, the process returns to S1702. Here, the processing termination condition is whether or not the high-resolution block (x) indicated by the position data 109 is controlled to fill the high-resolution image 110, but may be determined based on other criteria.

Next, learning of F and h will be described.
Determining the label data for learning (x _n and z _n and y _n), to learn the F using the learning data for each label. Specifically, the first low-resolution label block z _n or y _n for learning, determined by the same rules as the labeling unit 1601. Next, the label is used as a label for a high-resolution block (x _n ) for learning and a label for a low-resolution block (z _n and y _n ) for learning. Finally, from the high-resolution block (x _n ) for learning of each label and the low-resolution block (z _n and y _n ) for learning, the method of (1-2) and (1-3) The label F is learned, and the label h is learned by the method in the second modification. Thus, F and h is learned and for each label data 1604 block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h).}
The flowchart showing the processing flow of the learning method for F and h is simply the addition of the step of calculating the label in FIG. The learning device for F and h is merely the addition of the means for calculating the label in FIG.

  According to the fifth modification, the label of the low resolution block (z or y) is determined, and the resolution is increased by F and h learned according to the label. Therefore, an adaptive increase in resolution according to the label of the low resolution block (z or y) is performed. By this adaptive increase in resolution, the image quality of the created high-resolution block (x) is improved, and the image quality of the high-resolution image 110 is increased.

(1-10) Sixth modification example: Other modification examples using labeling
This modified example describes a modified example different from the fifth modified example using labeling.
Here, a case where labeling is further used in the second modification will be described. However, the change using labeling is not only applicable to the second change example. The present embodiment is not applicable to other modified examples. Further, the present invention can be applied to other modified examples.

An image resolution increasing apparatus 1800 according to this modification will be described with reference to FIGS.
The first labeling unit 1801, the second labeling unit 1802, and the connection between them are added to the configuration of FIG. 12.

  The first labeling unit 1801 and the second labeling unit 1802 have the same processing contents as the labeling unit 1601 described in the fifth modification. The first labeling unit 1801 and the second labeling unit 1802 have different input images.

Next, an example of the operation of the image resolution increasing apparatus in FIG. 18 will be described with reference to FIG.
In S <b> 1701, the low resolution image 105 is also sent to the block position control unit 103, the coefficient data holding unit 102, the addition data holding unit 1201, the first labeling unit 1801, and the second labeling unit 1802. The size data 106 _{(w l} and _{h l} and _{w h} and _{h h)} is sent to the block size determination unit 101 and the block position controller 103. Then, the block size determination unit 101 determines the size data 106 _{(w l} and _{h l} and _{w h} and _{h h),} the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h),} The block position control unit 103, the first labeling unit 1801, the second labeling unit 1802, the coefficient data holding unit 1602, the addition data holding unit 1603, and the creation unit 1202 are sent, and the process proceeds to S1702.

At S1702, the block position control unit 103, since the size data 106 _{(w l} and _{h l} and _{w h} and _{h h)} block size data 107 (and _{c l r l} and i and _{c h} a _{r h),} creating The position indicating the position of the high resolution block (x) in the high resolution image 110 and the position of the low resolution block (z or y) in the low resolution image 105 necessary for creating the high resolution block (x) The data 109 is determined and sent to the first labeling unit 1801 and the creating unit 1202, and the process proceeds to S1901.

  In S1901, the first labeling unit 1801 determines the first label data 1803 of the low resolution block (z or y) indicated by the position data 109 from the low resolution image 105 and the position data 109 according to a predetermined rule, and coefficient data Then, the process proceeds to S1704. As a rule for determining the first label data 1803, for example, vector quantization, ADRC, incremental code, or the like can be used. Since it is only necessary to label the low-resolution block, for example, edge detection may be performed on the low-resolution block, and vector quantization, ADRC, incremental code, or the like may be used for the result.

In S1704, the coefficient data holding unit 1602 reads from the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} the first label data 1803, corresponding coefficient data 108 (F), creation unit Then, the process proceeds to S1903. Here, the coefficient data 108 (F) is learned for each of the block size data 107 (c ₁ , r ₁ , i, c _h and r _h ) and the first label data 1803 and is held in the coefficient data holding unit 102. ing. The learning method for F will be described later.

  In S1902, the creation unit 1202 applies F from the left to the z indicated by the position data 109 from the low-resolution image 105, the coefficient data 108 (F), and the position data 109, so that the high-resolution block ( Fz) is calculated and sent to the second labeling unit 1802 as block data 1804, and the process proceeds to S1903.

In S1903, the second labeling portion 1802, from _{c h} a _{r h} and the block data 1804 of block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} (Fz), the block data 1804 ( Fz) second label data 1805 is determined according to a predetermined rule, sent to the addition data holding unit 1603, and the process proceeds to S1705. Since the block size determination unit 101, the second labeling section 1802 has been to send a _{c l} and _{r l} and i and _{c h} a _{r h,} which only uses the second labeling section 1802 in _{c h} a _{r h,} You may make it send only them.

In S1705, the sum data holding section 1603 reads from the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} and the second label data 1805, corresponding sum data 1203 (h), creation unit The process proceeds to 1202, and the process proceeds to S1904. Here, the sum data 1203 (h) is held to the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} and is learned every second label data 1805 addition data holding section 1201 Yes. The learning method for h will be described later.

  In S1904, the creation unit 1202 uses the low-resolution image 105, the coefficient data 108 (F), the position data 109, the addition data 1203 (h), and the Fz calculated in S1902 to display the high-resolution block (x ) Is calculated by the equation (x = Fz + ‖M‖h), and the process proceeds to S1707. Each time the position data 109 controlled by the block position control unit 103 changes, high-resolution blocks x in the high-resolution image 110 are sequentially created.

  In step S1707, the creation unit 1202 determines whether or not the high resolution block (x) indicated by the position data 109 is controlled to fill the high resolution image 110. 110 is output to end the process, and if not filled, the process returns to S1702. Here, the processing termination condition is whether or not the high-resolution block (x) indicated by the position data 109 is controlled to fill the high-resolution image 110, but may be determined based on other criteria.

Next, a learning method for F will be described.
Determining a first label data for learning (x _n and z _n and y _n), by using the learning data of the first label, to learn F. Specifically, the first low-resolution block first label z _n or y _n for learning, determined by the same rule as the first labeling section 1801. Next, the first label is a first label of a learning high-resolution block (x _n ) and a first label of a learning low-resolution block (z _n and y _n ). Finally, from the high-resolution block (x _n ) for learning of each first label and the low-resolution block (z _n and y _n ) for learning, the method (1-2) or (1-3) is used. , F of the first label is learned.

Next, a learning method for h will be described.
Determining a second label data for learning (x _n and z _n and y _n), by using the learning data of the second label, to learn h. Specifically, first, the second label of the high resolution block Fz _n calculated from the learning low resolution block z _{n is} determined according to the same rule as the second labeling unit 1802. Here, it should be noted that F is a coefficient matrix corresponding to the first label. Next, the second label is set as the second label of the learning high-resolution block (x _n ) and the second label of the learning low-resolution block (z _n and y _n ). Finally, h of the second label is learned from the high-resolution block (x _n ) for learning of each second label and Fz _n by the method of the second modification.

  Note that the flowchart showing the processing flow of the learning method for F and h is simply the addition of the step for calculating the first label and the step for calculating the second label in FIG. The learning apparatus for F and h is the same as that shown in FIG. 10 except that means for calculating the first label and means for calculating the second label are added.

  According to the sixth modification, the resolution is increased by F learned according to the first label of the low resolution block z or y and h learned according to the second label of the high resolution block Fz. Therefore, adaptively high resolution is made according to the first label and the second label. By this adaptive increase in resolution, the image quality of the created high-resolution block (x) is improved, and the image quality of the high-resolution image 110 is increased.

  In the fifth modification example and the sixth modification example, an example of increasing the resolution by F and h learned according to the label is shown. In addition, it is possible to learn only F according to the label, or to learn only h according to the label.

(1-11) Seventh modification example: Modification example for creating coefficient data and addition data for low magnification from coefficient data and addition data learned for high magnification
In the present embodiment and its modified examples, the coefficient data 108 (F) and the addition data 1203 (h) are learned and held in advance. If F and h have the same high resolution ratio (c _h / c ₁ times in the horizontal direction and r _h / r ₁ times in the vertical direction), the same can be used. However, the F and h cannot be used for other high resolution ratios. In order to realize high resolution at many high resolution ratios, when learning and holding many F and h, it takes time for learning and requires a lot of memory for holding these F and h. There is a problem of becoming. Therefore, a description will be given of a modification example in which F and h for a higher resolution rate lower than the F and h learned for a higher resolution rate are created.
Here, an example in which the second modification example is further modified will be described. However, the seventh modification example cannot be applied only to the second modification example. The present embodiment is not applicable to other modified examples. Further, the present invention can be applied to other modified examples.

The diagram showing the configuration of the image resolution increasing apparatus of the seventh modification is the same as that of the second modification, and is FIG. However, the operations of the coefficient data holding unit 102 and the addition data holding unit 1201 are different. Only that part will be described.
The coefficient data holding unit 102 reads out the corresponding coefficient data 108 (F) from the block size data 107 (c _l , r _l , i, c _h and r _h ) and creates it if it does not exist. And sent to the creation unit 1202. A method of creating F when it does not exist will be described later.

The addition data holding unit 1201 reads out the corresponding addition data 1203 (h) from the block size data 107 (c _l , r _l , i, c _h, and r _h ) and creates it if it does not exist. And sent to the creation unit 104. A method of creating h when it does not exist will be described later.

A diagram showing a flow of processing of the image resolution increasing apparatus of the seventh modified example is the same as that of the second modified example, and is as shown in FIG. However, the contents of the processing in S402 and S1301 are different. Only that part will be described.
In S402, the coefficient data holding unit 102 reads out the corresponding coefficient data 108 (F) from the block size data 107 (c _l , r _l , i, c _h, and r _h ) if it does not exist. Create and proceed to S1301.

In S1301, creation unit 1202 creates the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h),} if there reads it if there is a corresponding added data 1203 (h) The process proceeds to S403.

Next, a method for creating F when the corresponding coefficient data 108 (F) does not exist will be described.
First, from the coefficient data holding unit 102, among the coefficient data 108 corresponding to the high resolution increasing rate than the high-resolution rate (r _{h /} r _l fold laterally c _{h /} c _l times the vertical direction) , I are searched for. The coefficient data 108 is represented by F ′, and the block size data 107 corresponding to F ′ is represented by c ₁ ′, r ₁ ′, c _h ′, and r _h ′.

Next, 'calculates the least common multiple c of, _{r l} and _{r l' c l} and _{c l} for calculating the least common multiple r of. The least common multiple can be calculated using the Euclidean algorithm. Next, a block a of c + i pixels in the horizontal direction and r + i pixels in the vertical direction is changed to a block b of cc _h '/ c _l ' pixels in the horizontal direction and rr _h '/ r _l ' pixels in the vertical direction. A matrix G to be increased in resolution is created from F ′. G is a matrix of cc _h 'rr _h ' / (c _l 'r _l ') rows (c + i) (r + i) columns. G has an element of F ′ and 0 as elements.

Figure _{20, i = 4, c h} = 4, c l = 3, c h '= 3, c l' = 2, r h = 4, r l = 3, r h '= 3, r l' = 2 shows an example of a block a, a block b, and a later-described block d. In this example, c = r = 6. Next, create a matrix V of lower resolution in block d of _rr h / _{r l} pixels longitudinally _cc h / _{c l} pixel block b in the lateral direction. V is a matrix of _{cc h rr h / (c l} r l) line _{cc h 'rr h' / (} c l 'r l') column. The element of V is determined according to the area ratio when the block b and the block d are overlapped. Finally, F is created from VG. VG is a _{cc h rr h / (c l} r l) line (c + i) (r + i)) column of the matrix. Focusing on a thick line in FIG. 20, in the longitudinal direction c _{l +} i pixels in the horizontal direction of the r _{l +} i pixel block, which is a block of r _h pixels in the horizontal direction longitudinally c _h pixels. F is created by extracting elements related to the bold line from VG. In the example of FIG. 20, in addition to the portion indicated by the bold line, a block of c _l + i pixels in the horizontal direction and r _l + i pixels in the vertical direction, and c _h pixels in the horizontal direction and r _h in the vertical direction. There are three locations that are pixel blocks. They are shown in FIGS. 21, 22, and 23, respectively. F may be created by taking the average of the elements corresponding to these figures.

Next, a method of creating h when the corresponding addition data 1203 (h) does not exist will be described.
First, the added data holding unit 1201, in addition data 1203 corresponding to the higher resolution increasing rate than the high-resolution rate _(r h / _{r l} fold laterally _c h / _{c l} times the vertical direction) , I are searched for. The added data 1203 is represented by h ′, and the block size data 107 corresponding to h ′ is represented by c ₁ ′, r ₁ ′, c _h ′, and r _h ′. Next, 'calculates the least common multiple c of, _{r l} and _{r l' c l} and _{c l} for calculating the least common multiple r of. The least common multiple can be calculated using the Euclidean algorithm.

Next, a block f in which c ′ / c ₁ ′ h ′ is laid in the horizontal direction and r / r ₁ ′ in the vertical direction is created. FIG. 24 shows a case where c _h = 4, c _l = 3, c _h '= 3, c _l ' = 2, r _h = 4, r _l = 3, r _h '= 3 and r _l ' = 2. An example of a block f and a block g described later is shown. In this example, c = r = 6. The thick line in the block f in FIG. 24 represents h ′ that has been spread. Next, the block f, to a low resolution into blocks g of _rr h / _{r l} pixels in the horizontal direction longitudinally _cc h / _{c l} pixels. Finally, create a h by extracting a block of r _h pixels in the vertical direction c _h pixels in the horizontal direction from g. In the example of FIG. 24, a block indicated by a thick line in the block g is a candidate for h. The average of these may be h.

  According to the seventh modification example, F and h having a high resolution rate that has not been learned can be created, so that the learning cost of F and h and the memory for holding F and h can be reduced.

(1-12) Eighth Modification Example: Modification Example Regarding Speedup
In this embodiment, equation (y = Wx) in order to obtain a high image quality by obtaining the x that satisfies the block size data 107 _{(c l} and _{r l} and i and _{c h} a _{r h)} coefficient data corresponding to the The resolution was increased by using 108 (F). In order to increase the speed, elements having a small absolute value among the elements of the coefficient data 108 (F) may be ignored.

  If 0 is included in the element of the coefficient data 108 (F), the calculation with the element does not affect the result of Fz. Therefore, it is possible to increase the speed by ignoring the element and omitting the calculation. However, if it is determined whether each element of F is 0, calculation time for the determination is required. Its To reduce the calculation time of the determination, advance to determine in advance which elements to learned F is 0, it is preferable to create a circuit or program that calculates Fz for the F dedicated. Thereby, it is possible to increase the speed without degrading the image quality.

If the image quality can be degraded to the extent that humans cannot perceive it, it is advisable to ignore the F elements having a small absolute value. For example, consider the case where F = (0.004−0.1 0.9−0.1 0.004) and z = (255 100 50 40 10) ^T. In this case, Fz = 32.06 as shown in the following equation.

Fz = 0.004 × 255−0.1 × 100 + 0.9 × 50−0.1 × 40 + 0.004 × 10
= 1.02 -10 + 45 -4 + 0.04
= 32.06
When the pixel value is expressed by 8 bits, that is, an integer from 0 to 255, and the high resolution block x is obtained by the equation (x = Fz), the value is 32.06 to 32. In this case, since the number of product-sum operations is reduced, the speed can be increased. If the calculations of 0.004 × 255 and 0.004 × 10 are ignored, Fx is 31 and the result is almost unchanged. To the extent that the pixel value changes by one, humans cannot perceive the difference. Even if the value of a pixel changes by 4, it is often impossible to perceive the difference. Therefore, if elements having a small absolute value among the elements of F are ignored, the speed can be increased with almost no deterioration in image quality.

  Even when x is obtained by the equation (x = Fz + h) or the equation (x = Fz + ‖M‖h), the speed can be increased by ignoring 0 or a small absolute value in F. Similarly, h can be speeded up by ignoring 0 in h and those having a small absolute value.

  Since F and h have element values known at the stage of learning, information on which pixel in the low resolution block should be ignored is a method, apparatus, or program for increasing the resolution of an image. Can be reflected. It is not necessary to determine whether the absolute values of the elements F and h are smaller than the threshold value. If a circuit dedicated to F and h that does not perform an operation on an element having a small absolute value of the element F or h is designed, the circuit scale is reduced correspondingly, and the calculation cost can be reduced. Similarly, if an image high-resolution method or program that does not perform operations on elements having small absolute values of F and h is used, the calculation cost can be reduced accordingly.

(1-13) Ninth modification example: Modification example for extending from a space block to a space-time block
Flickering in the time direction is perceived when playing a movie created by connecting each frame of the movie as input, increasing the resolution with the apparatus or method of this embodiment that does not apply this modification, and connecting the output high-resolution images. May be. A modification example for reducing the flicker will be described.

  In the present embodiment, an example in which a certain image is input and a single image with a high resolution is output is shown. On the other hand, t images whose shooting times are close to the time direction may be input, and one image obtained by increasing the resolution of one image may be output. In this case, the low resolution block z is changed to a space-time block. The spatiotemporal block has t pixels in the time direction in addition to the spatial direction. Note that the low resolution block y and the high resolution block x do not change and have no pixels in the time direction. When distinguishing between a block having no pixels in the temporal direction and a block having pixels, the one having no pixels is referred to as a space block, and the one having a pixel is referred to as a space-time block.

  For example, a case where two images are input and a high resolution image of the second image is output will be taken as an example, and only changes from the present embodiment when no other modification examples are applied will be described. In this example, t = 2. The configuration of the image resolution increasing apparatus of this modification is the same as that shown in FIG.

The low resolution image 105 is t images. In this example, there are two images.
The low resolution block z has pixels in the time direction. The size of the low resolution block z is c _l + i in the horizontal direction, r _l + i in the vertical direction, and t in the time direction. z is expressed by a vertical vector, and the dimension is (c ₁ + i) (r ₁ + i) t. Figure 25 _is an example of a _{c l = 3, r l =} 4, i = 2, t = 2 for the low-resolution block z. 25 in FIG. 25 has 5 pixels in the horizontal direction, 6 in the vertical direction, and 2 ((a) and (b)) in the time direction, and has a total of 60 pixels.

  The block size data 107 is not changed. T is included in the size of the low resolution block z. Since t is a constant, it is not necessary to include t in the block size data 107, and each means that needs the value of t may hold it. Therefore, there is no change in the block size determination unit 101.

The coefficient data holding unit 102 holds coefficient data 108 (F) having a different size in accordance with changing z to a spatio-temporal block. Other than that, it doesn't change. F is a matrix of c _h r _h rows (c _l + i) (r _l + i) t columns. F of this size is learned and held.

W will be described with reference to the example of FIG. W, a c _h r _h dimensional vertical vector indicating a high resolution block (spatial block), and a c _l r _l t dimensional vertical vector indicating the low resolution block (spatial block) in FIGS. A matrix of c _l r _l t rows and c _h r _h columns to reduce the resolution to a vector. When W is compared between t = 1 and t = 2, the difference is the number of rows of W. W in the case of t = 2 has two elements of W in the case of t = 1. As the learning method of F, the above-described method can be applied only by changing the sizes of F, W, and z.
The block position control unit 103 does not change.
The creation unit 104 is different in that the input low resolution image 105 and the coefficient data 108 (F) are changed. When certain position data 109 is given, x indicated by the position data 109 is created by multiplying z indicated by the position data 109 from the left. This z becomes a spatiotemporal block, and the size of F changes, but the equation (x = Fz) does not change. Each time the position data 109 controlled by the block position control unit 103 changes, high-resolution blocks in the high-resolution image 110 are sequentially created.

  The flow of the processing for increasing the resolution of the image in this modified example remains as in FIG. There is no change except that some data has increased data in the time direction.

  Flickering in the time direction is perceived when playing a movie created by connecting each frame of the movie as input, increasing the resolution with the apparatus or method of this embodiment that does not apply this modification, and connecting the output high-resolution images. May be. On the other hand, when the ninth modification example is applied, time direction flicker is reduced because information in the time direction is used.

  However, when the subject is moving, the image quality is lowered by setting t to 2 or more. In this case, in a pixel having a change in the time direction in the low-resolution image, the pixel may be changed to be ignored. Specifically, a moving pixel in the low resolution image is detected by the inter-frame difference method. Then, the pixel value that has moved in the low resolution block may be replaced with the pixel value of the low resolution block at the same time as the high resolution block to be obtained. Alternatively, a block including pixels that have moved may be processed as t = 1. Alternatively, pixels that have moved may be ignored. In order to ignore it, it is only necessary to delete the F, W, and z elements relating to the pixel that has moved. Deletion reduces the size of F, W, and z. As a result, it is possible to prevent image quality deterioration in pixels with movement while suppressing flickering in the time direction of pixels without movement.

  Here, an example in which a space block is changed to a spatio-temporal block is shown, but the same effect can be obtained if pixels are provided in the time direction instead of the spatio-temporal block.

(1-14) Others The above modification examples can be used in combination. As a result, higher image quality can be obtained and processing can be performed at high speed. For example, the present embodiment is modified so that blocks that cannot be expressed by linear combination like the first modification example and the second modification example are learned and added, and further, the third modification example and the fourth modification are added. Change to enhance the image as in the example, change to use labeling as in the fifth and sixth modifications, and further, for high magnification as in the seventh modification Coefficient data that is modified to create low-magnification coefficient data and addition data from the learned coefficient data and addition data, and further has little influence on the pixel value of the high-resolution block created as in the eighth modification example And the addition data can be ignored, and the low resolution block for calculating the high resolution block can be changed from the spatial block to the spatiotemporal block as in the ninth modification.

  According to the embodiment described above, it is clear which pixel in the low resolution image should be used to calculate the value of the pixel in the high resolution image. If a high-resolution block created using that coefficient is reduced in resolution, the error will be reduced compared to the conventional image because it approaches the input low-resolution image. A resolution image can be obtained.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is a block diagram of an image resolution increasing apparatus according to an embodiment. The figure which shows an example of the low-resolution image which is an input of the image high-resolution apparatus of FIG. 1, and the high-resolution image which is an output. The figure which shows an example of the position of the high resolution block which the block position control part of FIG. 1 controls in order. 3 is a flowchart showing an example of the operation of the image resolution increasing device in FIG. 1. FIG. 3 is a diagram illustrating an example of a low resolution block and a high resolution block in the case of i = 0 in FIG. 2. FIG. 3 is a diagram illustrating an example of a low resolution block and a high resolution block when i = 2 in FIG. 2. FIG. 3 is a diagram illustrating an example of a low resolution block and a high resolution block when i = −2 in FIG. 2. The figure which shows an example of resolution reduction. The figure which shows an example in which two high resolution blocks are reduced in resolution to one low resolution block. The block diagram of the learning apparatus of the coefficient matrix of embodiment. 11 is a flowchart illustrating an example of the operation of the learning device in FIG. The block diagram of the image resolution increasing apparatus of the 1st modification. 13 is a flowchart showing an example of the operation of the image resolution increasing apparatus in FIG. 12. The block diagram of the coefficient matrix of the 1st modification and the learning apparatus of addition data. The flowchart which shows an example of operation | movement of the learning apparatus of FIG. The block diagram of the image resolution increasing apparatus of the 5th modification. The flowchart which shows an example of operation | movement of the image resolution increasing apparatus of FIG. The block diagram of the image resolution increasing apparatus of the 6th modification. The flowchart which shows an example of operation | movement of the image resolution increasing apparatus of FIG. The figure for demonstrating the example which produces a new coefficient matrix from the learned coefficient matrix of the 7th modification. In addition to FIG. 20, shows the locations longitudinally c _{l +} i pixels in the horizontal direction of the r _{l +} i pixel block, which is longitudinally c _h pixels in the horizontal direction into blocks of r _h pixels . 20, portions in other figures 21, in the longitudinal direction transversely c _{l +} i pixels r _{l +} i pixel block, which is longitudinally transversely c _h pixels into blocks of r _h pixels FIG. Figure 20, Figure 21, in addition to FIG. 22, in the longitudinal direction _c l + i pixels in the horizontal direction of the _r l + i pixel block, is a block of _{r h} pixels in the horizontal direction longitudinally _{c h} pixels The figure which shows the location. The figure for demonstrating the example which produces new addition data from the learned addition data of the 7th modification. The figure which shows an example of the low resolution block which is a spatiotemporal block of the 9th modification.

Explanation of symbols

100, 1200, 1600, 1800... Image high resolution device, 101... Block size determination unit, 102, 1602 .. coefficient data holding unit, 103... Block position control unit, 104, 1202. Creation unit 105: Low resolution image 106, 1004 ... Size data 107, 1006 ... Block size data 108 ... Coefficient data 109 ... Position data 110 ... High Resolution image, 1000, 1400... Learning device, 1001... Low resolution image creation unit, 1002... Block extraction unit, 1003... Coefficient data learning unit, 1005. ..Block size data, 1007... Learning low resolution image, 1008... Learning block, 1009. , 1603 ... addition data holding unit, 1203, 1402 ... addition data, 1401 ... addition data learning unit, 1601 ... labeling unit, 1604 ... label data, 1801 ... first labeling unit , 1802 ... second labeling unit, 1803 ... first label data, 1804 ... block data, 1805 ... second label data.

Claims (14)

An input means for inputting a low-resolution image whose size indicating the number of vertical and horizontal pixels is the first size;
First obtained by dividing the first number of pixels in the vertical of the first size, the first number of pixels in the first greatest common divisor of the second number of pixels of the vertical larger second size than said first size set the value, the second number of pixels to calculate a second value obtained by dividing the first common divisor, fourth number of pixels next to the side of the third pixel count and said second size of said first size And a third value obtained by dividing the third pixel number by the second greatest common divisor, and a fourth value obtained by dividing the fourth pixel number by the second greatest common divisor is calculated. and the corresponding first block size, the number of vertical pixels is first value and the horizontal number of pixels is determined to be larger block size than the block is a third value, high-resolution image of the second size and a second block size corresponding to the vertical number of pixels in the second value, and number of horizontal pixels determined to fourth value, said first block size and the second Determining means for a block size as a processing unit for high-resolution,
A block calculated by multiplying the first learning block by a first coefficient from one or more sets of the first learning block having the first block size and the second learning block having the second block size. A coefficient corresponding to the first block size and the second block size from one or more first coefficients, wherein one or more first coefficients learned so as to reduce an error between the first learning block and the second learning block are retained. First reading means for reading as a second coefficient;
Setting means for sequentially setting a first position in the low-resolution image of the block of the first block size and a second position in the high-resolution image corresponding to the first position of the block of the second block size. When,
An image resolution enhancement apparatus, comprising: a creation unit configured to create the high-resolution image by creating the second block at the second position by applying the second coefficient to the first block at the first position. . An input means for inputting a low-resolution image whose size indicating the number of vertical and horizontal pixels is the first size;
A first value obtained by dividing the first pixel number by a first greatest common divisor between the number of vertical pixels of the first size and the number of vertical second pixels of a second size that is larger than the first size . set, the second number of pixels to calculate a second value obtained by dividing the first greatest common divisor, and the fourth number of pixels next to the side of the third pixel count and said second size of said first size A third value obtained by dividing the third number of pixels by the second greatest common divisor is set, and a fourth value obtained by dividing the fourth number of pixels by the second greatest common divisor is calculated to correspond to the low-resolution image. and a first block size, the number of vertical pixels is first value and the horizontal number of pixels is determined to be larger block size than the block is a third value, the high-resolution image is the second size and a corresponding second block size, the number of vertical pixels to the second value, the number of horizontal pixels determined to fourth value, said first block size and the second Determining means for determining a block size as a block size of a processing unit for higher resolution,
A block calculated by multiplying the first learning block by a first coefficient from one or more sets of the first learning block having the first block size and the second learning block having the second block size. One or more first coefficients learned so as to reduce an error between the first learning block and the second learning block, and the first block size and the second block size represented by the first block size are represented by one or more first coefficients. First reading means for reading a coefficient corresponding to a second higher resolution ratio higher than the higher resolution ratio as a second coefficient;
Setting means for sequentially setting a first position in the low-resolution image of the block of the first block size and a second position in the high-resolution image corresponding to the first position of the block of the second block size. When,
A third coefficient corresponding to the first high resolution ratio is created from the second coefficient, the third coefficient is reset as the second coefficient, and the reset second coefficient is set to the second coefficient of the first position. An image high-resolution apparatus, comprising: a creating unit that creates the second block at the second position over one block to create the high-resolution image. A first set of the first learning block having the first block size and a second learning block having the second block size and the first coefficient are multiplied by the first coefficient to the first learning block. One or more first addition data learned so as to reduce an error between the block calculated by adding the addition data and the second learning block is held, and the first block size is determined from the one or more first addition data. And second reading means for reading the first addition data corresponding to the second block size as the second addition data,
In the creating means, instead of creating the second block by applying the second coefficient to the first block at the first position, the second coefficient is applied to the first block at the first position, and the second addition data is further applied. The image resolution increasing device according to claim 1 or 2, wherein the image resolution increasing device according to claim 1 or 2 is added. The first label data of the low resolution block or the third block is determined according to a rule relating to a value included in the low resolution block of the first position or the third block generated by applying the second coefficient to the first block of the first position. A calculation means for calculating,
The second reading means is calculated by the rule of the first learning block or the fourth block calculated by multiplying the first learning block by the first coefficient instead of the first addition data. Instead of holding one or more third addition data learned for each label data and reading the second addition data from the first addition data, the first block size and the second block are obtained from the third addition data. 4. The image high-resolution device according to claim 3, wherein addition data corresponding to a size and the first label data is read as the second addition data. 5.   The calculation is omitted by regarding the first element whose absolute value of the first coefficient is smaller than the first threshold as 0 and regarding the second element whose absolute value of the first added data is smaller than the second threshold as 0. The image high-resolution apparatus according to claim 3 or 4,   In the creating means, instead of the first block being a block of the first block size at the first position in the low-resolution image, the first position in the image in which the low-resolution image is enhanced The image high-resolution apparatus according to claim 1, wherein the first block size blocks are located at the same position.   The image resolution increasing apparatus according to any one of claims 1 to 6, wherein the block includes a plurality of pixels in a time direction in addition to a spatial direction. Input means for inputting a learning high-resolution image having a high size indicating the number of vertical and horizontal pixels;
Creating means for creating a learning low resolution image having a size smaller than the high size from the learning high resolution image;
Extraction means for extracting a set of a first learning block having a low block size and a second learning block having a high block size from the learning low resolution image and the learning high resolution image;
A first learning block having a low block size and a second learning block having a high block size, which are errors of a block calculated by multiplying the first learning block by a first coefficient and the second learning block; First calculating means for calculating the first coefficient with the smallest sum for the set of
Holding means for holding the low block size and the high block size in association with the first coefficient, and
The number of vertical pixels of the high block size is determined by dividing the number of vertical pixels of the high size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size , the number of horizontal pixels of the high-block size is determined by dividing the number of pixels next to the high size greatest common divisor of the number of pixels in the horizontal beside the the number of pixels low size of the high sizes, The number of vertical pixels of the low block size is a value obtained by dividing the number of vertical pixels of the low size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size. The number of horizontal pixels of the low block size is larger than the value obtained by dividing the number of horizontal pixels of the low size by the greatest common divisor of the number of horizontal pixels of the high size and the number of horizontal pixels of the low size. are determined or characterized by what is determined is input Learning device to be.   A first learning block having a low block size, which is an error between a block obtained by adding the calculated first coefficient to the first learning block and adding data to the block, and the second learning block; The learning apparatus according to claim 8, further comprising second calculation means for calculating the addition data that minimizes the sum of a pair with a second learning block having a high block size. Input means for inputting a learning high-resolution image having a high size indicating the number of vertical and horizontal pixels;
Creating means for creating a learning low resolution image having a size smaller than the high size from the learning high resolution image;
Extraction means for extracting a set of a first learning block having a low block size and a second learning block having a high block size from the learning low resolution image and the learning high resolution image;
Due to the reduction in resolution, the first learning block is included in a vector belonging to a set of vectors indicating the high block size blocks that are included in the first learning block and become smaller in size than the first learning block . from the coefficient matrix to convert the first vector indicating a block, the error between the vector representing the right to the coefficient matrix and vector calculated by multiplying the first vector second learning block, the low block First calculation means for calculating the coefficient matrix having the smallest sum for a set of the first learning block of the size and the second learning block of the high block size;
Holding means for holding the low block size and the high block size in association with the coefficient matrix; and
The number of vertical pixels of the high block size is determined by dividing the number of vertical pixels of the high size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size , the number of horizontal pixels of the high-block size is determined by dividing the number of pixels next to the high size greatest common divisor of the number of pixels in the horizontal beside the the number of pixels low size of the high sizes, The number of vertical pixels of the low block size is a value obtained by dividing the number of vertical pixels of the low size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size. The number of horizontal pixels of the low block size is larger than the value obtained by dividing the number of horizontal pixels of the low size by the greatest common divisor of the number of horizontal pixels of the high size and the number of horizontal pixels of the low size. are determined or characterized by what is determined is input Learning device to be.   The low block size error of the block obtained by adding the calculated coefficient matrix to the vector indicating the first learning block plus the addition data to the block indicated by the vector and the second learning block 11. The apparatus according to claim 10, further comprising second calculation means for calculating the addition data that minimizes a sum of a set of the first learning block and the second learning block having the high block size. Learning device. Input a low resolution image whose size indicating the number of vertical and horizontal pixels is the first size,
A first number of pixels in the vertical of the first size, than the value obtained by dividing the first number of pixels in the first greatest common divisor of the second number of pixels of the vertical larger second size than said first size also set a large first value, the second number of pixels to calculate a second value obtained by dividing the first common divisor, next to next to the third pixel count and said second size of said first size A third value larger than a value obtained by dividing the third pixel number by the second greatest common divisor with the fourth pixel number is set, and a fourth value obtained by dividing the fourth pixel number by the second greatest common divisor is set. calculated, the as the first block size corresponding to the low-resolution image, the number of vertical pixels in the first value, the number of horizontal pixels determined to be the third value, the high-resolution image of the second size and a second block size corresponding to the vertical number of pixels in the second value, and number of horizontal pixels determined to fourth value, said first block size and the second block size Was processed Unit for high resolution,
A block calculated by multiplying the first learning block by a first coefficient from one or more sets of the first learning block having the first block size and the second learning block having the second block size. And holding means for holding one or more first coefficients learned so as to reduce an error between the second learning block and the second learning block,
A coefficient corresponding to the first block size and the second block size is read as a second coefficient from the one or more first coefficients;
Sequentially setting a first position in the low resolution image of the block of the first block size and a second position in the high resolution image corresponding to the first position of the block of the second block size;
A method for increasing the resolution of an image, wherein the second resolution is generated by applying the second coefficient to the first block at the first position to generate the second block at the second position. Input a high-resolution learning image with a high size indicating the number of vertical and horizontal pixels,
A learning low-resolution image having a size smaller than the high size is created from the learning high-resolution image,
A set of a first learning block having a low block size and a second learning block having a high block size are extracted from the learning low resolution image and the learning high resolution image;
A first learning block having a low block size and a second learning block having a high block size, which are errors of a block calculated by multiplying the first learning block by a first coefficient and the second learning block; Calculating the first coefficient with the smallest sum for the set of
Preparing holding means for holding the low block size and the high block size in association with the first coefficient;
The number of vertical pixels of the high block size is determined by dividing the number of vertical pixels of the high size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size , the number of horizontal pixels of the high-block size is determined by dividing the number of pixels next to the high size greatest common divisor of the number of pixels in the horizontal beside the the number of pixels low size of the high sizes, The number of vertical pixels of the low block size is a value obtained by dividing the number of vertical pixels of the low size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size. The number of horizontal pixels of the low block size is larger than the value obtained by dividing the number of horizontal pixels of the low size by the greatest common divisor of the number of horizontal pixels of the high size and the number of horizontal pixels of the low size. are determined or characterized by what is determined is input Learning how to. Input a high-resolution learning image with a high size indicating the number of vertical and horizontal pixels,
A learning low-resolution image having a size smaller than the high size is created from the learning high-resolution image,
A set of a first learning block having a low block size and a second learning block having a high block size are extracted from the learning low resolution image and the learning high resolution image;
Due to the reduction in resolution, the first learning block is included in a vector belonging to a set of vectors indicating the high block size blocks that are included in the first learning block and become smaller in size than the first learning block . from the coefficient matrix to convert the first vector indicating a block, the error between the vector representing the right to the coefficient matrix and vector calculated by multiplying the first vector second learning block, the low block Calculating the coefficient matrix having the smallest sum for a set of the first learning block of the size and the second learning block of the high block size,
Preparing holding means for holding the low block size and the high block size in association with the coefficient matrix;
The number of vertical pixels of the high block size is determined by dividing the number of vertical pixels of the high size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size , the number of horizontal pixels of the high-block size is determined by dividing the number of pixels next to the high size greatest common divisor of the number of pixels in the horizontal beside the the number of pixels low size of the high sizes, The number of vertical pixels of the low block size is a value obtained by dividing the number of vertical pixels of the low size by the greatest common divisor of the number of vertical pixels of the high size and the number of vertical pixels of the low size. The number of horizontal pixels of the low block size is larger than the value obtained by dividing the number of horizontal pixels of the low size by the greatest common divisor of the number of horizontal pixels of the high size and the number of horizontal pixels of the low size. are determined or characterized by what is determined is input Learning how to.

Images