JP2016115313A - Super-resolution device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning type super-resolution device capable of suppressing a database capacity.SOLUTION: The pixel values of object pixels which are pixels in a prescribed-shaped area in an inputted image and are pixels obtained by excluding an input reference pixel group composed of a plurality of pixels in prescribed orders when the pixels in the area are arranged according to magnitude are normalized on the basis of pixel values of the pixels included in the input reference pixel group, a high resolution image corresponding to the area is acquired by referring to a database on the basis of information representing the arrangement of the normalized pixel values of the object pixels, and the inputted image is subjected to high resolution by correcting each pixel value of the acquired high resolution image on the basis of pixel values of the input reference pixel group.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a super-resolution device and a program.

When converting the resolution of an image, an interpolation function is convolved with the input image, and sampling is performed at a sampling period different from that of the input image. In particular, when the resolution of the output image is higher than the resolution of the input image, a super-resolution technique that obtains a fine output image by supplementing the signal component exceeding the Nyquist frequency of the input image may be used. is there.
The super-resolution technique includes a technique for improving the fineness of appearance by artificially synthesizing a signal component exceeding the Nyquist frequency from one image, and is called a single frame super-resolution technique.

  One type of single-frame super-resolution technique is a learning-type super-resolution technique that uses a database that has been learned in advance with a pair of a low-resolution image and a high-resolution image. In learning-type super-resolution technology, a pair of a partial region (low-resolution patch) obtained by dividing a low-resolution image into blocks and a partial region (high-resolution patch) of a high-resolution image corresponding to the partial region is obtained. A large amount is stored in advance in the database. When the super-resolution processing is executed, the corresponding high-resolution patch is obtained by referring to the above-described database with each low-resolution patch obtained by dividing the input low-resolution image into blocks. A high resolution image is generated by pasting the acquired high resolution patches. In learning super-resolution, there is also a technique of referring to a database in order to estimate a high-frequency component from a middle-frequency component of an input low-resolution patch (for example, see Patent Document 1).

Japanese Patent No. 4140690

  However, in the learning-type super-resolution technique such as Patent Document 1, it is necessary to register a large number of pairs in the database in advance according to the density patterns of various low-resolution patches. There is a problem that the capacity of the (only memory) may become enormous.

  The present invention has been made in view of such circumstances, and provides a learning-type super-resolution device and a program that can reduce the database capacity.

(1) The present invention has been made to solve the above-described problems, and one aspect of the present invention is a pixel in a region having a predetermined shape in an input image, and the pixel of the pixel in the region. Normalizing a pixel value of a target pixel that is a pixel excluding an input reference pixel group composed of a plurality of pixels in a predetermined order when values are arranged according to a size, based on a pixel value of a pixel included in the input reference pixel group;
A reference consisting of a plurality of pixels in the predetermined shape when the pixel values are aligned in the image, based on information representing the normalized arrangement of pixel values of the target pixel. A high-resolution image corresponding to the region by referring to a database that stores information indicating the arrangement of pixel values of pixels excluding the reference pixel group and an image having a higher resolution than the image having the predetermined shape. And resolving each pixel value of the acquired high-resolution image based on the pixel value of the input reference pixel group, thereby increasing the resolution of the input image. Device.

(2) According to another aspect of the present invention, in the super-resolution device according to (1), the plurality of pixels in the predetermined order are a first pixel having a maximum pixel value, a pixel The second pixel having the minimum value is characterized in that

(3) According to another aspect of the present invention, there is provided the super-resolution device according to (2), wherein the arrangement is an arrangement of pixels based on the first pixel and the second pixel. It is characterized by.

(4) According to another aspect of the present invention, there is provided the super-resolution device according to (3), wherein the input image is divided into the predetermined shape to generate a divided image. A normalization unit for normalizing pixel values of the divided image based on the maximum pixel value and the minimum pixel value in the divided image for each of the divided images; and for each divided image, In the database, the information representing the arrangement of the normalized pixel values of the pixels excluding the pixel having the maximum pixel value and the pixel having the minimum pixel value with reference to the first pixel and the second pixel is used as a key. Referring to a search unit that acquires a high-resolution image corresponding to the divided image, and a super-resolution image obtained by increasing the resolution of the input image using the high-resolution image acquired by the search unit. And a super resolving unit for generating.

(5) According to another aspect of the present invention, in the super-resolution device according to (4), the key is an arrangement of a pixel having a maximum pixel value and a minimum pixel value in the divided image. It includes the information indicating the type.

(6) Further, another aspect of the present invention is the super-resolution device according to (3) or (4), in which the super-resolution unit interpolates each of the divided images and performs the search. Generating the super-resolution image by adding the high-resolution image acquired by the search unit to each of the divided images interpolated and interpolated with the same resolution as the high-resolution image acquired by the unit. Features.

(7) Further, according to another aspect of the present invention, there is provided a plurality of computers when pixels in a predetermined shape area in an input image and the pixel values of the pixels in the area are aligned according to size. The pixel values of the target pixels that are pixels excluding the input reference pixel group composed of pixels of the predetermined order are normalized based on the pixel values of the pixels included in the input reference pixel group, and the normalized pixel value array of the target pixels On the basis of the information indicating the pixel values of the pixels excluding the reference standard pixel group consisting of the plurality of pixels in the predetermined order when the pixel values are aligned in the image. The high resolution image corresponding to the region is acquired by referring to the database that stores the information indicating the arrangement and the high resolution image having a higher resolution than the image of the predetermined shape, and stores the acquired high resolution image. Each pixel of the resolution image The, by correcting on the basis of the pixel values of the input reference pixel group is a program for functioning the input image as a super-resolution device for high resolution.

  According to the present invention, the database capacity can be suppressed.

1 is a schematic block diagram showing a configuration of a super-resolution system 10 according to an embodiment of the present invention. It is a figure which shows the forward transformation and reverse transformation of geometric transformation in the embodiment. It is a figure which shows a response | compatibility with arrangement | positioning of the pixel with the largest and the smallest pixel value in the embodiment, and the geometric transformation to apply. It is a figure explaining operation | movement of the learning apparatus 1 in the embodiment. It is a schematic block diagram which shows the structure of the super-resolution apparatus 3 in the same embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of a super-resolution system 10 according to an embodiment of the present invention. The super-resolution system 10 includes a learning device 1, a database 2, and a super-resolution device 3. The super-resolution device 3 uses the database 2 previously constructed by the learning device 1 to increase the resolution of the image.

  In conventional learning-type super-resolution, a pair of a low-resolution image partial region (low-resolution patch K) and a corresponding high-resolution image partial region (high-resolution patch G) is stored in a database. Whereas the database 2 according to the present embodiment is recorded, numerical data obtained by normalizing and reducing the dimension of the low-resolution patch K (hereinafter referred to as a low-resolution key N ′) and the corresponding high-resolution patch A pair with a patch (high resolution residual patch E) that is a material for generating the image is recorded.

  Below, the structure of the learning apparatus 1 for constructing | assembling the database 2 is demonstrated. The learning device 1 includes a dividing unit 11, a dividing unit 12, a normalizing unit 13, a converting unit 14, an interpolating unit 15, a subtracting unit 16, and a registering unit 17, and a low solution input for learning. Learning is performed by registering and updating data in the database 2 based on the image image LS and the high-resolution image HS.

  The dividing unit 11 cuts out a low-resolution patch K having a predetermined shape from the low-resolution image LS. The dividing unit 12 cuts out a portion corresponding to the low resolution patch K in the high resolution image HS as the high resolution patch G. The dividing unit 11 and the dividing unit 12 may cut out only one patch from each of the low resolution image LS and the high resolution image HS, or may cut out patches sequentially while scanning the screen. Further, the pair of the low resolution image LS and the high resolution image HS may be one pair or plural pairs. When the patches are cut out sequentially, the operations from the normalization unit 13 to the registration unit 17 are repeatedly executed to update the database 2 sequentially.

  When the low-resolution image LS or the high-resolution image HS is a color image, the database 2 may be provided independently for each color component and may be learned independently, or the database 2 common to all color components may be provided. This may be learned for all color components.

The normalizing unit 13 obtains the minimum value m and the maximum value M of the pixel values of the pixels constituting the low resolution patch K, normalizes the pixel value level (hereinafter referred to as level conversion) based on these values, and applies the patch J Is generated. For example, the normalization unit 13 performs level conversion so that the minimum value m is “0.0” and the maximum value is “1.0”.
In the patch K, the pixel value at the image coordinates (i, j) is K (i, j) (other images and patches are also expressed in the same manner). The level conversion at this time is, for example, conversion expressed by (Expression 1).

  The normalizing unit 13 further performs geometric transformation (rotation transformation or mirror image transformation) based on the arrangement of pixels having pixel values of 0.0 and 1.0 in the patch J so that the pixel arrangement becomes a prescribed arrangement. A pixel having a pixel value of 0.0 is a pixel having the minimum pixel value, and a pixel having a pixel value of 1.0 is a pixel having the maximum pixel value. An example of geometric transformation of the normalizing unit 13 is illustrated as a pair (conversion identifiers T = 0 to T = 7) of “before conversion” and “after forward conversion” in FIG.

  For example, the normalization unit 13 collates the arrangement of the value 0.0 (0 in the figure) and the value 1.0 (1 in the figure) in the patch J with the template (pattern of the patch J) shown in FIG. The normalized low-resolution patch N is obtained by applying the geometric transformation (see FIG. 2) of the transformation identifier T associated with the matching one. If the arrangement of the value 0.0 and the value 1.0 in the patch J matches a plurality of templates in FIG. 3, the normalization unit 13 selects one according to a predetermined rule, and The geometric transformation of the transformation identifier T associated with the selected template is performed. For example, when the arrangement of the value 0 and the value 1 in the patch J matches a plurality of templates in FIG. 3, if the arrangement type S has the smallest numerical value and still cannot be narrowed down only The one with the smallest numerical value of the conversion identifier T is selected.

  The normalization unit 13 outputs a patch after applying the geometric transformation to the patch J as a normalized low-resolution patch N and outputs a normalization parameter R = (m, M, S, T). . The normalization parameter R includes a minimum value m, a maximum value M, a value 0 and a value 1 in the patch J (that is, a maximum value and a minimum value in the low-resolution patch K). ) And a conversion identifier T of geometric transformation.

  The conversion unit 14 performs level conversion shown in (Expression 2) on the high-resolution patch G to generate a patch D.

Note that the minimum value m and the maximum value M in (Equation 2) are the minimum value and the maximum value of the low resolution patch K (not the minimum value and maximum value of the high resolution patch G).
Subsequently, the conversion unit 14 performs no conversion, rotation, or forward conversion of the mirror image specified by the conversion identifier T, and as a result, the patch F is obtained.

The interpolation unit 15 obtains a patch P by performing high-resolution interpolation interpolation so that the normalized low-resolution patch N has the same resolution as the high-resolution patch G. For interpolation, for example, 0th order interpolation (nearest neighbor interpolation) is used, but higher order interpolation may be used.
For example, when the horizontal and vertical resolutions of the high resolution patch are twice and twice the horizontal and vertical resolutions of the low resolution patch, the interpolation unit 15 performs the conversion represented by (Equation 3).

  The subtraction unit 16 obtains a difference image obtained by subtracting the patch P from the patch F, and outputs the result as a high-resolution residual patch E. That is, the subtracting unit 16 performs the conversion represented by (Expression 4).

The registration unit 17 is a sequence of pixel values obtained by scanning a pixel column excluding the minimum and maximum pixel values used to specify the arrangement type S and the conversion identifier T in a predetermined order (alignment). (Hereinafter referred to as low resolution key N ′). This predetermined order is predetermined for each arrangement type S. That is, the order is based on the arrangement of the minimum and maximum pixel values. For example, when the low-resolution patch is 2 horizontal pixels × 2 vertical pixels, 2 pixels are picked up from 4 pixels constituting the normalized low-resolution patch N, and the low-resolution key N that is a sequence of two terms 'Create. Specifically, N ′ = (N (0,1), N (1,1)) when the arrangement type S = 0 or the arrangement type S = 1, and N when the arrangement type S = 2. Let '= (N (0,1), N (1,0)).
Subsequently, the registration unit 17 registers the pair of the arrangement type S and the low resolution key N ′ and the high resolution residual patch E in the database 2.

The registration unit 17 records, for example, the arrangement type S and the low resolution key N ′ in the database 2 in association with each other so that a high resolution residual patch E can be obtained when given as a search key.
When the high-resolution residual patch E corresponding to the arrangement type S and the low-resolution key N ′ is not yet registered in the database 2, the registration unit 17 associates the arrangement type S with the low-resolution key N ′. To register a high-resolution residual patch E.

  When the high resolution residual patch E corresponding to the arrangement type S and the low resolution key N ′ is already registered in the database 2, the registration unit 17 has already registered the arrangement type S and the low resolution key N ′. The associated high resolution residual patch is discarded and a new high resolution residual patch E is registered. Alternatively, when the high resolution residual patch E corresponding to the arrangement type S and the low resolution key N ′ is already registered in the database 2, the registration unit 17 sets the arrangement type S and the low resolution key N ′. The registration is updated to a combination of the high-resolution residual patch and the high-resolution residual patch E already associated with (for example, the arithmetic average of both patches).

  After updating the contents of the database 2 with various pairs of the low resolution patch K and the high resolution patch G, the high resolution residual is still applied to all combinations of the arrangement type S and the low resolution key N ′. The difference patch E is not necessarily associated. In such a case, the registration unit 17 is closest to the low resolution key N ′ of the combination and is associated with the low resolution key N ′ in the same arrangement type S as the combination that is not associated. It may be operated to search for '' and register a high-resolution residual patch corresponding to the low-resolution key N '' as a high-resolution residual patch E corresponding to the combination. Alternatively, the registration unit 17 selects a plurality of low-resolution keys N ″ that are associated with the low-resolution keys N ′ that are not associated with each other from the shortest distance, and associates them with these. Registered as a high-resolution residual patch E corresponding to the low-resolution key N ′ with the result of computation (for example, arithmetic mean by weighting according to distance) performed on the high-resolution residual patch It does not matter if it operates. Note that the distance between N ′ and N ″ is measured by the Euclidean distance, for example.

Hereinafter, the case where the low resolution patch K is composed of a total of 4 pixels of 2 horizontal pixels and 2 vertical pixels, while the high resolution patch G is composed of 16 pixels of 4 vertical pixels and 4 horizontal pixels, see FIG. However, the operation of the learning device 1 is illustrated.
The learning device 1 first cuts out the low-resolution patch K in the dividing unit 11. An example of the low resolution patch K is shown at 100 in FIG. The normalization unit 13 performs three processes (minimum value calculation 101, maximum value calculation 102, and geometric transformation determination 103) for obtaining a normalization parameter, a level conversion 104 process, and a geometric conversion 106 process. .

  In the minimum value calculation 101, the normalization unit 13 obtains the minimum value m from the low resolution patch K (m = 2 in this example). In the maximum value calculation 102, the normalization unit 13 obtains the maximum value M from the low resolution patch K (M = 6 in this example). In the level conversion 104, the normalization unit 13 obtains the patch J by performing the conversion of (Equation 1) on the low resolution patch K based on the minimum value m and the maximum value M. An operation example of the patch J is shown in 105.

  In the geometric transformation determination 103, the normalization unit 13 specifies the arrangement type S and the transformation identifier T of the transformation to be applied in the geometric transformation 106 from the arrangement of the value 0 and the value 1 of the patch J. In this example, the arrangement type S = 1 and the conversion identifier T = 4. In the geometric transformation 106, the normalization unit 13 performs a geometric transformation of the transformation identifier T on the patch J. In the geometric transformation 106, the normalized low-resolution patch N indicated by 107 is obtained as a result of the transformation from “before transformation” in FIG. 2 to “after forward transformation” at T = 4.

  The conversion unit 14 performs level conversion 111 processing and geometric conversion 113 processing. In the level conversion 111, the conversion unit 14 is based on the minimum value m (m = 2 in this example) and the maximum value M (M = 6 in this example) obtained by the normalization unit 13 according to (Equation 2). Conversion is performed to obtain a patch D indicated by 112 in FIG. In the geometric transformation 113, the transformation unit 14 performs no transformation, rotation, or mirror image forward transformation designated by the transformation identifier T specified in the geometric transformation judgment 103 on the patch D, and as a result, the patch indicated by 114 is obtained. Get F.

  The interpolating unit 15 interpolates the normalized low-resolution patch N 107 (0-order interpolation in this example), and generates a patch P indicated by 109. The subtracting unit 16 subtracts 109 patches P for each pixel from 114 patches F, and obtains a high-resolution residual patch E indicated by 115 as a result. The registration unit 17 determines from the normalized low resolution patch N to the low resolution key N ′ = (0.25, 0.50) (108 in FIG. 4) based on the arrangement type S = 1 specified in the geometric transformation determination 103. 2 pixels within the dotted line).

  Further, the registration unit 17 performs the low-resolution key N ′ = (0.25, 0.50) indicated by 108 in FIG. 4, the high-resolution residual patch E indicated by 115 in FIG. 4, and the arrangement type S = One set is registered / updated in the database 2.

  Next, the configuration of the super-resolution device 3 according to the present embodiment will be described with reference to FIG. FIG. 5 is a schematic block diagram showing the configuration of the super-resolution device 3. The super-resolution device 3 increases the resolution of the input low-resolution image L with reference to the database 2 and outputs the high-resolution image H. The super-resolution device 3 includes a dividing unit 31, a normalizing unit 32, a database 2, a search unit 33, an interpolation unit 34, an addition unit 35, an inverse conversion unit 36, and a synthesis unit 37.

  The dividing unit 31 cuts out the high resolution patch K from the low resolution image L. The dividing unit 31 sequentially cuts out patches (low-resolution patches K) while scanning the screen of the low-resolution image L. At this time, the subsequent operations from the normalization unit 32 to the inverse conversion unit 36 are executed each time the dividing unit 31 cuts out the low-resolution patch L, and the result is synthesized by the synthesis unit 37 so that the high-resolution image H is synthesized. Composed.

_Let the horizontal resolution of the low resolution image L be C _x pixels and the vertical resolution be _Cy pixels. _{Let the} number of pixels of the low resolution patch K be horizontal _Sx pixels and vertical _Sy pixels. When the dividing unit 31 sequentially cuts out the low-resolution patch K while scanning the screen of the low-resolution image L, for example, when cutting out the low-resolution patch K having the image coordinates (X, Y) as the upper left point. The dividing unit 31 executes (Expression 5).

The normalization unit 32 performs level conversion and geometric conversion on the low-resolution patch K input from the division unit 31 by the same calculation as the normalization unit 13 in FIG. The normalization parameter R = (m, M, S, T) is output.
The search unit 33 selects a pixel excluding the minimum and maximum pixel values from the normalized low resolution patch N based on the arrangement type S of the normalization parameter R, and obtains the low resolution key N ′. Specifically, N ′ = (N (0,1), N (1,1)) when the arrangement type S = 0 or the arrangement type S = 1, and N when the arrangement type S = 2. Let '= (N (0,1), N (1,0)).

  The search unit 33 also queries the database 2 using the arrangement type S and the low resolution key N ′ as a search query, and obtains a high resolution residual patch E corresponding to the combination of the arrangement type S and the low resolution key N ′. obtain. The search unit 33 refers to the database 2 using the arrangement type S and the low resolution key N ′ as a search query, and as a result, the corresponding high resolution residual patch E is not obtained (registered in the database 2). If not, the low resolution key N ″ closest to and associated with the low resolution key N ′ among the low resolution keys of the same arrangement type S is searched for, and the low resolution key N The high resolution residual patch corresponding to ″ may be regarded as the high resolution residual patch E corresponding to the low resolution key N ′ and output.

  Alternatively, the search unit 33 sets the distance between the low-resolution key N ″ that is associated with the surrounding low-resolution key N ′ that is not associated among the low-resolution keys of the same arrangement type S. Are selected from the nearer ones, and the result of arithmetic (for example, arithmetic mean by weighting according to distance) corresponding to these is applied to the low-resolution key N ′. Operation may be performed so that the corresponding high-resolution residual patch E is output. Note that the distance between N ′ and N ″ is measured by the Euclidean distance, for example.

  The interpolation unit 34 obtains a patch P by interpolating the normalized low-resolution patch N by high-resolution by the same calculation as the interpolation unit 15 of FIG. The adding unit 35 performs the calculation represented by (Expression 6). That is, the adding unit 35 adds the high-resolution residual patch E from the searching unit 33 and the patch P from the interpolating unit 34 for each pixel to obtain a patch F.

The inverse conversion unit 36 performs an operation corresponding to the inverse conversion of the conversion unit 14 on the patch F based on the minimum value m, the maximum value M, and the conversion identifier T among the normalization parameters R from the normalization unit 32. Then, a high resolution patch G is obtained.
For example, the inverse transformation unit 36 first performs an operation corresponding to the inverse transformation of the geometric transformation 113 on the patch F to obtain the patch D. That is, no conversion, rotation, or reverse conversion of the mirror image specified by the conversion identifier T is performed on the patch F in the pattern shown in FIG. Subsequently, the inverse conversion unit 36 performs an operation corresponding to the inverse conversion of the level conversion 111 on the patch D to obtain a high resolution patch G. The inverse conversion of the level conversion 111 is expressed by (Equation 7).

  In response to the division unit 31 scanning the inside of the screen of the low resolution image L, the synthesizing unit 37 scans the corresponding region in the high resolution image and pastes the high resolution patch G together. A high resolution image H is constructed.

The horizontal resolution of the high resolution image H is A _x · C _x pixels, and the vertical resolution is A _y · C _y pixels. _{Let the} number of pixels of the high-resolution patch G be horizontal A _x · S _x pixels and vertical A _y · S _y pixels. Incidentally, A _x is the ratio obtained by dividing the number of horizontal pixels of the high resolution image H in the number of horizontal pixels of the low resolution image L, also, A _y are low-resolution images the number of vertical pixels of the high resolution image H This is the ratio divided by the number of vertical pixels of L. When the dividing unit 31 sequentially cuts out the low-resolution patch K while scanning the screen of the low-resolution image L, for example, when the low-resolution patch K having the image coordinate (X, Y) as the upper left point is cut out In response to this, the synthesizer 37 constructs the high-resolution image H by executing the processing represented by (Equation 8).

In the above-described embodiment, the super-resolution device 3 and the database 2 are different devices, but the super-resolution device 3 may include the database 2.
Further, in the super-resolution device 3, the processing order of the addition unit 35 and the inverse conversion unit 36 may be reversed. In this case, the interpolation unit 34 interpolates the low resolution patch K to obtain a high resolution and obtains the patch P.
In the above-described embodiment, the shape of the low-resolution patch K is a square of 2 pixels × 2 pixels, but is not limited thereto. A square or rectangle with a larger number of pixels may be used, or a shape other than a rectangle may be used.

  Thus, in the database 2 used by the super-resolution device 3, when learning a pair of the low-resolution patch K and the high-resolution patch G, the low-resolution patch K is used as information representing the low-resolution patch K. Among them, information representing the arrangement of pixel values of pixels excluding the pixels with the largest and smallest pixel values is used. For this reason, the number of bits of information representing the low-resolution patch K is decreased by the maximum and minimum pixel values. That is, since the range of values that can be taken by the information representing the low-resolution patch K is small, the number of records in the database 2 can be suppressed and the capacity can be suppressed.

  The arrangement in the information representing the arrangement of the pixel values is an arrangement of pixels based on the pixels having the largest and smallest pixel values by performing geometric transformation such as rotation and mirror image transformation. As a result, it is sufficient to learn one low-resolution patch K for a set of low-resolution patches K that can be converted by geometric transformation. That is, when learning the database (registering pairs), if the number of pairs of low-resolution patches and high-resolution patches is insufficient, overfitting of learning (overlearning; When learning with, a phenomenon in which an inappropriate model is constructed depending on the bias and noise of individual data) occurs, and the image quality of the super-resolution processing result using the database is unlikely to deteriorate. Become.

  1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. The learning device 1, the database 2, and the super-resolution device 3 may be realized by executing them. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

In addition, each functional block of the learning device 1, the database 2, and the super-resolution device 3 in FIG. 1 described above may be individually chipped, or a part or all of them may be integrated into a chip. Further, the method of circuit integration is not limited to LSI, and implementation using a dedicated circuit or a general-purpose processor is also possible. Either hybrid or monolithic may be used. Some of the functions may be realized by hardware and some by software.
In addition, when a technology such as an integrated circuit that replaces an LSI appears due to progress in semiconductor technology, an integrated circuit based on the technology can be used.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like within a scope not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Learning apparatus 2 ... Database 3 ... Super-resolution apparatus 10 ... Super-resolution system 11, 12 ... Dividing part 13 ... Normalization part 14 ... Conversion part 15 ... Interpolation part 16 ... Subtraction part 17 ... Registration part 31 ... Dividing part 32 ... Normalization unit 33 ... Search unit 34 ... Interpolation unit 35 ... Addition unit 36 ... Inverse conversion unit 37 ... Synthesis unit

Claims (7)

Pixels in a region of a predetermined shape in the input image, excluding an input reference pixel group composed of a plurality of pixels in a predetermined order when pixel values of the pixels in the region are aligned according to size Normalizing the pixel value of the target pixel based on the pixel values of the pixels included in the input reference pixel group;
A reference consisting of a plurality of pixels in the predetermined shape when the pixel values are aligned in the image, based on information representing the normalized arrangement of pixel values of the target pixel. By referring to a database that stores information indicating the arrangement of pixel values of pixels excluding the reference pixel group and a high-resolution image having a resolution higher than that of the image having the predetermined shape, the high value corresponding to the region is stored. Get the resolution image,
A super-resolution device that increases the resolution of the input image by correcting each pixel value of the acquired high-resolution image based on a pixel value of the input reference pixel group.   2. The super-resolution device according to claim 1, wherein the plurality of pixels having a predetermined order are a first pixel having a maximum pixel value and a second pixel having a minimum pixel value.   The super-resolution apparatus according to claim 2, wherein the arrangement is an arrangement of pixels based on the first pixel and the second pixel. A dividing unit that divides the input image into the predetermined shape to generate a divided image;
For each divided image, a normalization unit that normalizes the pixel value of the divided image based on the maximum pixel value and the minimum pixel value in the divided image;
For each of the divided images, the normalized pixel values of the pixels excluding the pixel having the maximum pixel value and the pixel having the minimum pixel value in the divided image are arranged on the basis of the first pixel and the second pixel. A search unit that obtains a high-resolution image corresponding to the divided image with reference to the database using information representing
The super-resolution part which produces | generates the super-resolution image which raised the resolution of the said input image using the said high-resolution image which the said search part acquired. Super-resolution device.   The super-resolution device according to claim 4, wherein the key includes information representing an arrangement type of a pixel having a maximum pixel value and a minimum pixel value in the divided image. The super-resolution unit interpolates each of the divided images to have the same resolution as the high-resolution image acquired by the search unit, and the search unit acquires the divided image obtained by interpolation interpolation. The super-resolution device according to claim 4 or 5, wherein the super-resolution image is generated by adding images having high resolution. Computer
Pixels in a region of a predetermined shape in the input image, excluding an input reference pixel group composed of a plurality of pixels in a predetermined order when pixel values of the pixels in the region are aligned according to size Normalizing the pixel value of the target pixel based on the pixel values of the pixels included in the input reference pixel group;
A reference consisting of a plurality of pixels in the predetermined shape when the pixel values are aligned in the image, based on information representing the normalized arrangement of pixel values of the target pixel. By referring to a database that stores information indicating the arrangement of pixel values of pixels excluding the reference pixel group and a high-resolution image having a resolution higher than that of the image having the predetermined shape, the high value corresponding to the region is stored. Get the resolution image,
A program for functioning as a super-resolution device that increases the resolution of the input image by correcting each pixel value of the acquired high-resolution image based on the pixel value of the input reference pixel group.

Images