JP2017175347A - Image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of acquiring a high-resolution image with high definition.SOLUTION: The image processing system includes: acquiring a plurality of differential values obtained by taking space differentials with a plurality of phases from a low-resolution image input at a high-region generation part for each pixel; generating a high-region component on the basis of a differential value which make its absolute value minimum, of the plurality of differential values; synthesizing the low-resolution image and the high-region component at a band synthesis part to acquire a high-resolution image for output.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus, and more particularly to an apparatus for improving the resolution of an image.

  As a technique for improving the resolution of an image, a technique by interpolation is used. Several fine resolution conversion techniques with less blurring than interpolation interpolation have been put to practical use as super-resolution techniques by complementing frequency components exceeding the Nyquist frequency of signals.

  As one of the super-resolution methods, there is a method that realizes high resolution by assuming conversion from a high-resolution image to a low-resolution image and introducing a regularization term to enable reverse conversion ( Patent Document 1).

  As another super-resolution technique, a method of synthesizing a frequency component exceeding the Nyquist frequency from a high frequency component (high frequency component) of an input image has been proposed. As such a method, a frequency-resolved image generated by a plurality of frequency-resolved filters is synthesized to create a multiplexed image, and the multiplexed image is filtered (Patent Document 2). There is a method (Patent Document 3) that performs a conversion process and then performs a sharpening process using a non-linear operation. For synthesis of frequency components exceeding the Nyquist frequency, a digital filter sequence (Patent Document 2) or a nonlinear function (Patent Document) A relatively simple calculation such as 3) is used.

  Further, there is a case-based super-resolution technique in which pairs of low-resolution patches and high-resolution patches are stored in a database, and a high-resolution image is obtained from a low-resolution image by referring to the database (Patent Document 4, etc.) ).

Japanese Patent No. 4214409 JP 2014-119949 A Japanese Patent No. 5629902 Japanese Patent No. 5743615

  However, the conventional super-resolution technique has not yet obtained a satisfactory image. For example, in the super-resolution method using regularization, especially when trying to obtain a high-resolution image from a single frame with low resolution, the number of dimensions of the unknown variable greatly exceeds the number of dimensions of the constraint, and the regularization term It is impossible to solve the inverse problem by introducing. As a result, a high-resolution result of a flat image tone in which approximate pixel values are continuous is obtained.

  As in the super-resolution methods of Patent Document 2 and Patent Document 3, a method of synthesizing a frequency component exceeding the Nyquist frequency (also referred to as a super-Nyquist component) from a high-frequency component of the input signal surely adds the super-Nyquist component. Therefore, it is possible to obtain a fine sense of appearance. However, since the synthesis method of the super Nyquist component from the high frequency component of the input image is simple, there is a possibility that an unnatural super Nyquist component is generated. In particular, in the method of Patent Document 2, the edge of the super Nyquist component pattern tends to be thicker than the pattern that should be.

  As a method of referring to a database, a relatively natural super-resolution result can be obtained if a sufficient database is constructed. However, if the database is insufficient, high resolution is performed based on the high resolution patch corresponding to the low resolution patch that does not match well, and a high resolution image that looks strange is generated. End up. The high-resolution patch used at this time is a pattern that actually exists as an image, and it is misused (for example, by applying an image of another person's eye to the face image), so the sense of incongruity is enormous. It is.

  Therefore, an object of the present invention made in view of the above problems is to provide an image processing apparatus capable of obtaining a more natural and fine high-resolution image.

  In order to solve the above problems, an image processing apparatus according to the present invention includes a high-frequency generation unit that generates a high-frequency component of an image from an input low-resolution image, the low-resolution image, and the high-frequency image A band synthesis unit that generates and outputs a high-resolution image with a higher resolution than the low-resolution video by band-combining the components, and the high-frequency generation unit obtains a plurality of spatial differences having a phase difference with respect to the image The plurality of acquired difference values are compared for each pixel position, and as a result of the comparison, a high frequency component value at the pixel position is determined based on a difference value having a minimum absolute value of the difference value. .

  In the image processing device, the high frequency generator includes a first high frequency generator that generates a horizontal high frequency component, and a second high frequency generator that generates a vertical high frequency component. A third high-frequency generating unit that generates an angular high-frequency component, and the band synthesizing unit receives the input low-resolution image, the horizontal high-frequency component, the vertical high-frequency component, It is desirable to generate each pixel of the high-resolution image by performing an operation based on the diagonal high-frequency component.

  In the image processing apparatus, it is preferable that the operation of the band synthesizing unit is band synthesis based on a two-dimensional Haar basis.

  In the image processing apparatus, the first high frequency generation unit and the second high frequency generation unit obtain at least two types of spatial differences having a phase difference, and the third high frequency generation unit It is desirable to acquire at least four types of spatial differences having a phase difference.

  In the image processing apparatus, the first to third high frequency generation units may obtain two difference values having the same absolute value but different signs as the difference value having the smallest absolute value of the difference value. In addition, it is desirable to further include a process of setting the high frequency component value of the high frequency generation unit to 0.

  According to the image processing apparatus of the present invention, the frequency component exceeding the Nyquist frequency is generated from the input low-resolution video, and the phenomenon that the edge of the processed image becomes thicker than the original pattern should be suppressed. A high-resolution image with a feeling can be obtained.

It is a block diagram which shows an example of the image processing apparatus of this invention. It is a block diagram which shows an example of the high region production | generation part of the image processing apparatus of this invention. It is a figure explaining spatial difference calculation. It is a block diagram which shows an example of the high region production | generation part of a conventional system. It is the comparison of the result of having performed image enlargement by the image processing apparatus which concerns on interpolation interpolation, the conventional system, and this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment)
FIG. 1 is a block diagram showing an example of an image processing apparatus according to the present invention. The image processing apparatus 1 increases the resolution of the input low resolution image LL and outputs it as a high resolution image S. In FIG. 1, the image processing apparatus 1 includes three high-frequency generation units 2 (2 _{1 to} 2 ₃ ) and a band synthesis unit 3.

High-frequency creating unit 2 ₁ a high-frequency component HL, the high-frequency creating unit 2 ₂ high-frequency component LH, also the high-frequency creating unit 2 ₃ respectively generate high-frequency component HH. For example, the high frequency component HL is a horizontal high frequency component, ie, a vertical edge component, the high frequency component LH is a vertical high frequency component, ie, a horizontal edge component, and the high frequency component HH is a diagonal high frequency component, ie, Each diagonal edge component is represented.

  The band synthesizing unit 3 performs band synthesis on the input low resolution image LL and the generated high frequency components HL, LH, and HH, and generates a high resolution image S that is higher in resolution than the low resolution image LL. And output.

  Typically, all the resolutions of the generated high-frequency components HL, LH, and HH are the same as the resolution of the low-resolution image LL, and the high-resolution image is 2 in both the horizontal and vertical resolutions of the low-resolution image. The resolution is double (that is, four times the number of pixels). Hereinafter, the resolution of the low resolution image LL and the high frequency components HL, LH, and HH are all assumed to be horizontal resolution A and vertical resolution B, and the resolution of the high resolution image S is assumed to be horizontal resolution 2A and vertical resolution 2B. Although A and B are both natural numbers, the conditions under which the image processing apparatus 1 according to the present invention operates effectively are A ≧ 3 and B ≧ 3.

  Hereinafter, the pixel value of the image P at the image coordinate (x, y) is described as P (x, y), and the image coordinate is expressed as an integer of 0 or more. The image coordinate (0, 0) is the pixel position at the upper left corner of the image, and the first component x of the image coordinate (x, y) is in the horizontal right direction and the second component y is in the vertical downward direction.

  The band synthesizing unit 3 performs band synthesis using, for example, a two-dimensional Haar basis. That is, the band synthesizing unit 3 generates and outputs a high resolution image S according to the following equation (1).

  Or you may produce | generate the high-resolution image S by following Formula (2). Expressions (1) and (2) are equivalent. By the processing of the band synthesizing unit 3, a frequency component exceeding the Nyquist frequency is generated.

  Note that the band synthesis method is not limited to the above-described two-dimensional Haar base, and may be, for example, band synthesis based on an nth-order Daubechies base. Note that the use of the Haar basis equation (1) simplifies the circuit configuration and reduces the circuit scale.

FIG. 2 is a block diagram showing an example of the high frequency generator 2 (2 _{1 to} 2 ₃ ) in FIG. The high-frequency generation unit 2 in FIG. 2 includes N spatial difference calculation units 4 (4 _{1 to} 4 _N ), N absolute value calculation units 5 (5 _{1 to} 5 _N ), and a comparison unit 6. A unit 7 and an amplifying unit 8 are provided.

N is a natural number of 2 or more. The value of N to not be the same all the high-frequency creating unit 2 ₁ to 2 _3, may be the same. Most effective is, N = 2 for the high-frequency creating unit 2 ₁ to generate the horizontal high-frequency component, for high-frequency creating unit 2 ₂ which generates a vertical high-frequency component N = 2, Further, for the high-frequency creating unit 2 ₃ for generating a diagonal direction high frequency component and N = 4.

The spatial difference calculation unit 4 (4 _{1 to} 4 _N ) obtains spatial differences (pixel value differences) at different phases (positions).

The absolute value calculation unit 5 _n (n is an integer greater than or equal to 1 and less than or equal to N) takes and outputs an absolute value for each pixel position with respect to the spatial difference value output by the corresponding spatial difference calculation unit 4 _n .

Comparing unit 6 compares each absolute value output of the absolute value calculation unit 5 (5 ₁ to 5 _N) for each pixel position, identify the smallest absolute value and outputs the absolute value calculating unit 5 _Nmin among them The identifier N _min is output for each pixel position.

The selection unit 7 selects and outputs the output specified by the identifier N _min from among the spatial difference values output from the spatial difference calculation unit 4 (4 _{1 to} 4 _N ) for each pixel position.

The amplifying unit 8 multiplies the output of the selecting unit 7 by a gain a and outputs the result as a high frequency component output. The gain a is a real number and may be a positive value, a negative value, or 0. The value of the gain a is selected according to the degree of emphasizing the edge portion. Further, the gain a may not be all the same in the high frequency generators 2 _{1 to} 2 ₃ (independent gains a _HL , a _LH , and a _HH for the high frequency components HL, LH, and HH, respectively). May be set) or the same. However, the gains a for the high-frequency generation units 2 _{1 to} 2 ₃ must not be all zero.

Hereinafter, for each of the high-frequency generation units 2 _{1 to} 2 ₃ , each of the spatial difference calculation unit 4 (4 _{1 to} 4 _N ), the absolute value calculation unit 5 (5 _{1 to} 5 _N ), the comparison unit 6, and the selection unit 7. A specific example of the implementation will be given.

(Implementation of the high-frequency creating unit _{2 1)}
High-frequency creating unit 2 _1, as comprising N = 2 pieces of spatial difference calculation section 4 (4 ₁ and 4 _2), described their implementation.

Both spatial difference calculating unit 4 ₁ and spatial difference calculating section 4 ₂ performs spatial difference calculation in the horizontal direction, but performs difference calculation shifted by one pixel in the horizontal direction. For example, spatial difference HL 1 the output of the spatial difference calculating unit 4 _1, when the output of the spatial difference calculating section 4 ₂ placed between the spatial difference HL _2, the following equation (3), to obtain respective results.

FIG. 3 is a diagram illustrating the spatial difference calculation. FIG. 3A shows the positional relationship of the pixels related to the calculation of the spatial differences HL ₁ and HL ₂ when the pixel of interest A is the coordinates (1, 1), and FIG. The example of the calculation result of a difference (x direction) is shown. HL ₁ is a difference value between the target pixel A and the pixel on the left side thereof, and HL ₂ is a difference value between the target pixel A and the pixel on the right side thereof.

Absolute value calculation unit _{5 1,} to the input value _HL 1 (x, y), the absolute value _{| HL 1 (x, y)} | outputs a. The absolute value calculating unit _{5 2,} to the input value _HL 2 (x, y), the absolute value _{| HL 2 (x, y)} | outputs a.

The comparison unit 6 compares the absolute value | HL ₁ (x, y) | and the absolute value | HL ₂ (x, y) | for each pixel position (x, y), and an identifier N that minimizes the value. _{Find min} (x, y). That is, N _min (x, y) is expressed by the following equation (4). In Equation (4), arg min is an abbreviation for argument of the minimum (argument that gives the minimum value), and is a set formed by the entire domain of the domain in which the function takes the minimum value.

In the example of FIG. 3B, N _min (1,1) = 2 since | HL ₁ (1,1) |> | HL ₂ (1,1) |.

Selector 7, the pixel position (x, y) for each, spatial difference value outputted from the spatial difference calculating unit _{4 1 HL} 1 (x, y) and the spatial difference value _HL 2 which outputs the spatial difference calculating unit _{4 2} ( The output of the spatial difference calculation unit 4 _Nmin specified by the identifier N _min (x, y) is selected from x, y), and the result is output as HL ′ (x, y). Therefore, HL ′ (x, y) is expressed by the following equation (5).

  Here, the selection of a value having a small absolute value of the spatial difference value means that the edge of the pattern is narrowed in the processed image.

The amplification unit 8 multiplies the output value HL ′ (x, y) of the selection unit 7 by the gain a _HL to obtain the high frequency component HL (x, y). The value of the gain a _HL is, for example, a _HL = 0.5. This value is desirably obtained experimentally so that a clear high-definition image can be obtained. In general, a value of about 0.3 to 0.7 is selected. The high frequency component HL (x, y) is expressed by Equation (6).

In normal image processing, the calculation of the high frequency component HL is as described above, but the absolute values of the spatial difference value HL ₁ (x, y) and the spatial difference value HL ₂ (x, y) are the same. When the signs are opposite, it is desirable to add a process for setting the value of the high frequency component HL (x, y) to 0. This is because when one of the adjacent pixels has the same absolute value of the spatial difference value and the sign is inverted, adopting one of the spatial difference values may generate an unnatural image. As such processing, when the comparison unit 6 determines that the two spatial difference values have the same absolute value and opposite signs, HL ′ (x, y) is replaced with 0 in Expression (5). Alternatively, only under such conditions, the value of the gain a _HL may be set to 0 in Equation (6).

The absolute value calculation section 5 (5 ₁ and 5 _2), the comparing unit 6, summarizes the operation of the selector 7, and the amplification unit 8, it is also possible to calculate by one means, the following equation (7) It may be executed (implemented based on the equation (7)).

  In this implementation example, the difference is obtained from the pixels adjacent to the target pixel in the horizontal direction with N = 2. For example, when N = 4, the pixels adjacent to the target pixel in the vertical and diagonal directions are also used. Thus, obtaining the horizontal difference is one method.

(Implementation of the high-frequency creating unit _{2 2)}
High-frequency creating unit 2 _2, as comprising N = 2 pieces of spatial difference calculation section 4 (4 ₁ and 4 _2), described their implementation.

Both spatial difference calculating unit 4 ₁ and spatial difference calculating section 4 ₂ performs spatial difference calculation in the vertical direction, but performs difference calculation shifted by one pixel in the vertical direction. For example, spatial difference LH 1 the output of the spatial difference calculating unit 4 _1, when the output of the spatial difference calculating section 4 ₂ placed between the spatial difference LH _2, the following equation (8), to obtain respective results.

FIG. 3A shows the positional relationship of the pixels related to the calculation of the spatial differences LH ₁ and LH ₂ when the pixel A of interest is the coordinate (1, 1). LH ₁ is the difference value between the target pixel A and the pixel above it, and LH ₂ is the difference value between the target pixel A and the pixel below it.

Absolute value calculation unit _{5 1,} to the input value _LH 1 (x, y), the absolute value _{| LH 1 (x, y)} | outputs a. The absolute value calculating unit _{5 2,} to the input value _LH 2 (x, y), the absolute value _{| LH 2 (x, y)} | outputs a.

The comparison unit 6 compares the absolute value | LH ₁ (x, y) | and the absolute value | LH ₂ (x, y) | for each pixel position (x, y), and an identifier N that minimizes the value. _{Find min} (x, y). That is, N _min (x, y) is expressed by the following equation (9).

Selector 7, the pixel position (x, y) for each spatial difference computing section _{4 1} of the output spatial difference value _LH 1 (x, y) and the spatial difference value _LH 2 to output spatial difference calculating unit _{4 2} ( The output of the spatial difference calculation unit 4 _Nmin specified by the identifier N _min (x, y) is selected from x, y), and the result is output as LH ′ (x, y). Therefore, LH ′ (x, y) is expressed by the following equation (10).

The amplifying unit 8 multiplies the output value LH ′ (x, y) of the selecting unit 7 by a gain a _LH to obtain a high frequency component LH (x, y). The value of the gain a _LH is, for example, a _LH = 0.5. This value is desirably obtained experimentally so that a clear high-definition image can be obtained. In general, a value of about 0.3 to 0.7 is selected. The high frequency component LH (x, y) is expressed by Equation (11).

As for the calculation of the high frequency component LH, when the absolute values of the spatial difference value LH ₁ (x, y) and the spatial difference value LH ₂ (x, y) are the same and the signs are opposite, the high frequency component LH ( It is desirable to add a process for setting the value of x, y) to 0. As such processing, when the comparison unit 6 determines that the absolute values of the two spatial difference values are the same and the signs are opposite, LH ′ (x, y) is replaced with 0 in Expression (10). Or, only under such conditions, the value of gain a _LH may be set to 0 in equation (11). This is the same as the horizontal component processing.

The absolute value calculation section 5 (5 ₁ and 5 _2), the comparing unit 6, summarizes the operation of the selector 7, and the gain portion 8, it is also possible to calculate by one means, the following equation (12) You can do it.

  In this implementation example, the difference is obtained from the pixels adjacent in the vertical direction of the target pixel when N = 2, but for example, pixels adjacent in the left and right and diagonal directions of the target pixel are also used when N = 4. Thus, obtaining the vertical difference is one method.

(Implementation of the high-frequency creating unit _{2 3)}
High-frequency creating unit 2 _3, as comprising N = 4 pieces of spatial difference calculation section 4 (4 ₁ to 4 _4), is described for their implementation.

While performing the spatial difference calculation of the spatial difference computing unit 4 ₁ to spatial difference calculating unit 4 ₄ also pairs Any angular and spatial difference calculating section 4 ₂ to spatial difference calculating section 4 ₄ against spatial difference calculating unit 4 ₁ Difference calculation is performed by shifting one pixel in the horizontal, vertical, and horizontal / vertical directions. For example, when the output of the space difference calculation unit 4 _n is set as the space difference HH _n (nε {1, 2, 3, 4}), the result of each calculation unit is obtained by the following equation (13).

FIG. 3A shows the positional relationship of pixels related to the calculation of the spatial differences HH ₁ , HH ₂ , HH ₃ , and HH ₄ when the pixel of interest A is the coordinates (1, 1).

The absolute value calculation unit 5 _n (nε {1, 2, 3, 4}) outputs the absolute value | HH _n (x, y) | for the input value HH _n (x, y).

For each pixel position (x, y), the comparison unit 6 calculates an absolute value | HH ₁ (x, y) |, an absolute value | HH ₂ (x, y) |, and an absolute value | HH ₃ (x, y) | And the absolute value | HH ₄ (x, y) | are obtained, and an identifier N _min (x, y) that minimizes the value is obtained. That is, N _min (x, y) is expressed by the following equation (14).

Selector 7, the pixel position (x, y) for each spatial difference value _HH 1 to the output of the spatial difference calculating unit _{4 1} to spatial difference calculating unit _{4 4} (x, _y), HH 2 (x, y), The output of the spatial difference calculation unit 4 _Nmin specified by the identifier N _min (x, y) is selected from HH ₃ (x, y) and HH ₄ (x, y), and the result is _expressed as HH ′ ( x, y). Therefore, HH ′ (x, y) is expressed by the following equation (15).

The amplifying unit 8 multiplies the output value HH ′ (x, y) of the selecting unit 7 by a gain a _HH to obtain a high frequency component HH (x, y). The value of the gain a _HH is, for example, a _HH = 0.1. It is desirable to obtain this value experimentally so that a clear high-definition image can be obtained.

In addition, for the calculation of the high frequency component HH, there are two spatial difference values HH _n (x, y) whose absolute values are minimized, and the absolute values of the two spatial difference values HH _n are the same and the signs are opposite. In this case, it is desirable to add a process for setting the value of the high frequency component HH (x, y) to 0. As such processing, when the comparison unit 6 determines that the absolute values of the two spatial difference values are the same and the signs are opposite, HH ′ (x, y) is replaced with 0 in Expression (15). Or, only under such conditions, the value of the gain a _HH may be set to 0 in the equation (16).

The absolute value calculation section 5 (5 ₁ to 5 _4), the comparator 6 summarizes the operation of the selector 7, and the gain portion 8, it is also possible to calculate by one means, the following equation (17) You can do it.

  Hereinafter, a comparative example of the result of high resolution by the image processing apparatus according to the present invention and the result of high resolution by the conventional method (conventional wavelet super-resolution) will be shown.

Here, as a conventional method, a case where the conventional high frequency generator 20 shown in FIG. 4 is used instead of the high frequency generator 2 (2 _{1 to} 2 ₃ ) of FIG. 1 is shown. The conventional high frequency generator 20 includes a spatial difference calculator 21 and a convolution unit 22. The convolution unit 22 may be mounted as a simple gain multiplication.

  In the conventional method, the Gaussian is convolved in the convolution unit 22, but the problem that the edge becomes excessively thick in the conventional high-frequency generation is regarded as a problem to be solved in the present invention. Accordingly, in the following, the convolution unit 22 performs simple gain multiplication (corresponding to impulse multiplication) with the convolutional Gaussian as the limit of sharpness.

Incidentally, the conventional high frequency generator 20 of the high-frequency creating unit 2 for _1, a conventional high frequency generator 20 of the high-frequency creating unit conventional high frequency generator 20 for 2 _2, and the high-frequency generator 2 ₃ The high-frequency components HL, LH, and HH that are respectively output are defined as shown in the following equation (18), and a comparative experiment was performed. Equation (18) corresponds to one of the wavelet super-resolutions using the Haar basis.

  FIG. 5 shows the result of the comparative experiment. FIG. 5A shows the result of up-sampling and 2 × 2 times image enlargement by interpolation using Lanczos-3 as an interpolation function. Interpolation results in a noticeable blur because information above the Nyquist frequency in the image before enlargement cannot be reproduced.

  FIG. 5B shows the result of image enlargement of 2 × 2 times by the conventional method (conventional wavelet super-resolution) shown in FIG. 4 and Equation (18). Although there is a finer feeling than the result of FIG. 5 (a), there is a feeling of edging near the outline, a white line outside the boundary, a black line inside, and the outline tends to be thicker. . This is due to the fact that the edge of the generated high frequency component is fat.

  FIG. 5C shows the result of image enlargement of 2 × 2 times by the image processing apparatus according to the present invention. There is a finer feeling than in FIGS. 5A and 5B, and the edge-like artifacts seen in FIG. 5B are also reduced. Therefore, the contour portion looks sharper and a natural image is obtained. The present invention can greatly improve an image with a small number of operations (small circuit scale).

  Note that a computer can be suitably used to cause the image processing apparatus 1 to function as described above, and such a computer stores a program describing processing contents for realizing each function of the image processing apparatus 1. This program can be realized by reading out and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiments can be combined into one, or one constituent block can be divided.

First image processing apparatus _{2, 2 1,} 2 2, _{2 3} high frequency generator 3 band synthesis section _{4, 4 1,} 4 2, ..., _{4 N} spatial difference calculating unit _{5,5 1,} 5 2, ..., _{5 N} Absolute value calculation unit 6 Comparison unit 7 Selection unit 8 Amplification unit 20 Conventional high frequency generation unit 21 Spatial difference calculation unit 22 Convolution unit

Claims (5)

A high-frequency generation unit for generating a high-frequency component of the image from the input low-resolution image;
A band synthesizing unit that generates and outputs a high-resolution image having a higher resolution than the low-resolution video by band-combining the low-resolution image and the high-frequency component;
The high frequency generation unit acquires a plurality of spatial differences having a phase difference with respect to an image, compares the plurality of acquired difference values for each pixel position, and as a result of the comparison, a difference value having a minimum absolute value of the difference value An image processing apparatus characterized by determining a high frequency component value of the pixel position based on   2. The image processing apparatus according to claim 1, wherein the high frequency generator includes a first high frequency generator that generates a horizontal high frequency component, and a second high frequency generator that generates a vertical high frequency component. And a third high-frequency generating unit that generates a diagonal high-frequency component, and the band synthesizing unit includes the input low-resolution image, the horizontal high-frequency component, and the vertical high-frequency component. An image processing apparatus that generates a pixel of a high-resolution image by performing a calculation based on a band component and the diagonal high-frequency component.   The image processing apparatus according to claim 2, wherein the operation of the band synthesizing unit is band synthesis based on a two-dimensional Haar basis.   4. The image processing apparatus according to claim 2, wherein the first high-frequency generation unit and the second high-frequency generation unit acquire at least two types of spatial differences having a phase difference, and The high frequency generation unit acquires at least four types of spatial differences having a phase difference.   5. The image processing apparatus according to claim 2, wherein the first to third high-frequency generation units have the same absolute value and the same sign as the difference value having the smallest absolute value of the difference value. An image processing apparatus, further comprising a process of setting a high frequency component value of the high frequency generation unit to 0 when two different difference values are obtained.