KR20230047687A - Apparatus for Restoring Image - Google Patents

Abstract

Disclosed is a video restoration device. The embodiment provides the video restoration device, which implements the ability to discriminate single distortion which occurs in all synthetic low-quality images including low-level noise from mosaics, and real low- quality images, the complex distortion consisting of numerous combinations, real distortion, and unlearned distortion during the learning process, and whether it can be restored or not, into one integrated artificial neural network to restore videos.

Description

Image Restoration Apparatus {Apparatus for Restoring Image}

An embodiment of the present invention relates to an image restoration device.

The contents described below merely provide background information related to the present embodiment and do not constitute prior art.

In general, a technique for restoring a low-resolution image to a high-resolution image is classified according to the number of input images used for reconstruction or a reconstruction technique. Depending on the number of input images, it is divided into single image super-resolution restoration technology and continuous image super-resolution restoration technology.

In general, single-image super-resolution image restoration technology has a faster processing speed than continuous image super-resolution image restoration, but the quality of image restoration is low because information necessary for restoration is insufficient.

Since the continuous image super-resolution image restoration technology uses various features extracted from a plurality of consecutively acquired images, the quality of the restored image is superior to that of the single image super-resolution image restoration technology, but the algorithm is complex and the amount of computation is large, so real-time It is difficult to process.

Depending on the restoration technology, there is a technology using interpolation, a technology using edge information, a technology using frequency characteristics, and a technology using machine learning such as deep learning. The technique using the interpolation method has a high processing speed, but has the disadvantage of blurring the edges.

The technology using edge information is fast and can restore an image while maintaining the sharpness of the edge, but has a disadvantage in that it may include a visually noticeable restoration error when the edge direction is incorrectly estimated.

The technology using frequency characteristics can restore the image while maintaining the sharpness of the edge like the technology using edge information using high-frequency components, but has the disadvantage of generating Ringing Artifact near the boundary. Finally, techniques using machine learning such as example-based or deep learning have the highest quality of restored images, but the processing speed is very slow.

As described above, among various existing high-resolution image restoration technologies, the continuous image super-resolution image restoration technology can be applied to fields requiring a digital zoom function using an existing interpolation method, and can produce images of superior quality compared to interpolation-based image restoration technologies. to provide. However, the existing super-resolution image restoration technology is limited in its application due to the complex amount of calculation in electro-optical equipment requiring limited resources and real-time processing.

Existing single image-based super-resolution image restoration technology capable of real-time processing has a problem in that performance is significantly reduced compared to continuous image-based restoration technology when an image needs to be enlarged at a higher magnification than a preset multiple.

In this embodiment, synthetic low-quality images including mosaics and low-level noises, single distortions occurring in all real low-quality images, complex distortions composed of numerous combinations and intensities, actual distortions, unlearned distortions in the learning process, whether or not restoration is possible, or An object of the present invention is to provide an image restoration device capable of restoring an image by implementing the ability to determine whether restoration is impossible as an integrated artificial neural network.

According to an aspect of the present embodiment, a structure for constructing a plurality of neural networks independently learning image reconstruction for each of a plurality of image scales and using the image restoration results of the lower scale neural networks as additional information for image restoration of the current scale neural network an inference network block having; A network module having a structure for applying different weights for each scale in consideration of a trade-off between image distortion restoration and image content preservation when using the image restoration results of the plurality of lower-scale neural networks as additional information; Provided is a video restoration network characterized in that it has a structure for performing image restoration by gradually connecting the inference network block and the network module from the lowest scale to the highest scale.

According to another aspect of the present embodiment, the first scale level upscaled by receiving only the image (I ^k-1 _content , k=1) downscaled to the input scale of the corresponding step, reflecting the artificial intelligence learning result, and then upscaling by a preset multiple. PD ^k-1 (k=1) (Pretrained Decoder) outputting a reconstructed image (I ^k _detail , k=1); and an image (I ^L→k-1 _da-content , k>1, (k-1)>L≥0) by scaling the DAF output images included in the lower step to the input scale of the corresponding step (where k=2 , I ^0→1 _da-content = I ^0→1 _content ), an image downscaled to the input scale of the corresponding step (I ^k-1 _content , k>1), and an output reconstructed image of the previous step PD (I ^k- _{A plurality} of DAF modules (DAF ^{k -1 ,} k>1), DAF output information (I ^k-1 _da-content , k>1) and the PD output of the previous step are received as inputs and output the kth scale level upscaled reconstructed image (I ^k _detail , k>1) It provides an image restoration device characterized in that it includes a PD ^k-1 (k>1) (Pretrained Decoder).

As described above, according to the present embodiment, a synthetic low-quality image including mosaic and low-level noise, a single distortion occurring in all of the actual low-quality images, a complex distortion consisting of numerous combinations and intensities, actual distortion, and unlearning in the learning process There is an effect of restoring the image by implementing the ability to determine whether the distortion, restoration is possible or not, as one integrated artificial neural network.

1 is a diagram illustrating an image processing method in a PPD type according to an embodiment.
2 is a diagram illustrating an image processing method in the DAF application structure according to the present embodiment.
3A and 3B are diagrams illustrating an image processing method of the DAF ^k module according to the present embodiment.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an artificial intelligence network configuration and a learning method for a PPD-type image restoration device according to a first embodiment.

The artificial intelligence network for the image restoration apparatus of the present invention has a PPD (Pretrained Progressive Decoder) form in which a plurality of PDs (Pretrained Decoders) independently learned for each image scale are connected in a cascade manner. . The image restoration apparatus to which the artificial intelligence network and the learning method of the present invention are applied performs a process of inputting inference results obtained by each image scale in a plurality of PDs to the next connected PD.

[Explanation of PPD (inference) network structure]

In other words, the inference network block of step k (PD ^{k -1} , I ^{k - 1} _detail , I ^{k - 1} _content (or DAF ^k-1 (in the case of the second embodiment)) and the scale level of the inference result image of the corresponding step k (I ^k _detail ) is set, PD and DAF (including the case of the second embodiment) are configured in each step, and the lower step is sequentially connected to the next upper step. However, since only one input scaled by the corresponding scale of the input image can be applied to the PD at the lowest stage, parameter inflating may not be applied at the lowest stage PD. can

That is, at k>1 (a stage other than the lowest stage), the result inferred from the PD of the previous stage (ie, I ^{k - 1} _detail ) is input as the PD of the corresponding stage (ie, PD ^{k - 1} ), and the In the PD, the inference result (ie, I ^{k - 1} _detail ) of the previous PD, which is two (exemplarily two, but may be multiple) inputs, and the input image are obtained through parameter inflating. I ^{k - 1} _content (or, in the case of FIG. 2 , the output of the DAF (ie, DAF ^{k - 1} ) of the corresponding step), which is the result of scaling to the input scale of the corresponding step, is received as an input, and the scale level upscaled in the corresponding step It has an artificial intelligence network structure that infers k images (I ^k _detail ).

^A plurality of PDs (Pretrained Decoders) according to this embodiment include PD ^{k -1} (k=1) and PD ^{k -1} _I (k>1). The image restoration apparatus to which the artificial intelligence network of the present invention is applied has a PPD (Pretrained Progressive Decoder) structure in which PD ^{k -1} (k = 1) to PD ^{k - 1} _I (k > 1) are connected in a cascade manner. In a structure in which PD ^{k -1} (k = 1) and PD ^{k - 1} _I (k > 1) are connected, inference results for each scale level of an image through each PD are input to the upper stage PD. That is, an image of the highest scale level upscaled with respect to an input image can be restored through the output of the highest level PD of the artificial intelligence network composed of PPDs.

PD ^{k - 1} (k = 1) receives only the image (I ^{k - 1} _content , k = 1) downscaled to the input scale of the corresponding step, reflects the artificial intelligence learning result, and upscales it by a preset multiple. A 1-scale level upscaled reconstructed image (I ^k _detail , k=1) is output.

PD ^{k - 1} _I (k>1) is an image (I ^k-1 _content , k>1) downscaled to the input scale of the corresponding step and an output reconstructed image (I ^k-1 _detail , k>1) of the previous PD. receive input PD ^{k - 1} _I (k>1) is an image (I ^k-1 _content , k>1) downscaled to the input scale of the corresponding step and an output reconstructed image (I ^k-1 _detail , k>1) of the previous PD. A kth scale level upscaled reconstructed image (I ^k _detail , k>1) upscaled by a predetermined multiple by reflecting the artificial intelligence learning result is output.

Hereinafter, an operation process of PD ⁰ corresponding to PD ^k−1 (k=1) will be exemplarily described.

PD ⁰ is included in the first step and receives only the 0th scale level image (I ⁰ _content ) downscaled to the corresponding scale level. PD ⁰ reflects the artificial intelligence learning result on the 0th scale level image (I ⁰ _content ) and upscales it by a preset multiple, and outputs a first scale-level reconstructed image (I ¹ _detail ).

Hereinafter, operation processes of PD ¹ _I , PD ² _I , PD ³ _I , and PD ⁴ _I corresponding to PD ^{k − 1} _I (k>1) will be exemplarily described.

PD ¹ _I is included in the second step, and receives an image (I ¹ _content ) downscaled to the input scale of the corresponding step and an image (I ¹ _detail ) reconstructed at the first scale level. PD ¹ _I reflects the artificial intelligence learning result on the image (I ¹ _content ) downscaled to the input scale of the corresponding step and the reconstructed first scale level image (I ¹ _detail ), and reconstructs the second scale level upscaled by a preset multiple. Outputs an image (I ² _detail ).

PD ² _I is included in the third step, and receives an image (I ² _content ) downscaled to the input scale of the corresponding step and a second scale level reconstructed image (I ² _detail ). PD ² _I reflects the result of artificial intelligence learning on the downscaled image (I ² _content ) and the reconstructed second scale level image (I ² _detail ) to the input scale of the corresponding step, and upscales the reconstructed third scale level image by a preset multiple. (I ³ _detail ) is output.

PD ³ _I is included in the fourth step, and receives an image (I ³ _content ) downscaled to the input scale of the corresponding step and a reconstructed image (I ³ _detail ) at a third scale level. PD ³ _I reflects the artificial intelligence learning result on the image (I ³ _content ) downscaled to the input scale of the corresponding step and the reconstructed 3rd scale level image (I ³ _detail ), and up-scales the fourth scale level reconstructed image by a predetermined multiple. (I ⁴ _detail ) is output.

PD ⁴ _I is included in the fifth step, and receives an image (I ⁴ _content ) downscaled to the input scale of the corresponding step and a reconstructed image (I ⁴ _detail ) at a fourth scale level. PD ⁴ _I reflects the result of artificial intelligence learning on the downscaled image (I ⁴ _content ) and the 4th scale level reconstructed image (I ⁴ _detail ) downscaled to the input scale of the corresponding step, and upscales the 5th scale level reconstructed image by a preset multiple. (I ⁵ _detail ) is output.

In this embodiment, the fifth scale level reconstructed image (I ⁵ _detail ) is used as an output of an image upscale restoration apparatus to which the artificial intelligence network and learning method of the present invention are applied.

[How to learn PPD network]

A plurality of PDs (Pretrained Decoders) according to the present embodiment independently perform initial learning for each scale of the image, and then sequentially connect upper-level PDs in a cascade in a PPD format at each step while connecting upper-level PDs sequentially. Whenever a PD corresponding to is added, tuning learning is performed for the added PD, and at this time, the PD for which tuning learning has already been completed in the previous step is not learned again.

The configuration of the entire learning network is composed of a plurality of PDs, and the plurality of PDs include PD ^{k -1} (k=1) and PD ^{k - 1} _I (k>1). In this case, when the subscript I is indicated, it indicates that parameter inflating is applied to the corresponding PD. Parameter inflating is a method for expanding a number of input channels of a pretrained network. The method consists of a process of copying a plurality of pre-learned input channels, a process of concatenating the plurality of copied channels, and a process of adjusting the scale of parameters in the concatenated channels.

The image restoration device network of the present invention configures stages for each scale level of the image and connects PD ^k−1 (k=1) to PD ^k−1 _I (k>1) in a step-by-step cascade manner.

Parameter inflating may be applied to PD ^{k - 1} _I (k>1) included in the image restoration device network to accommodate a plurality of inputs.

In order to learn the network of the PPD structure, tuning learning is performed for each PD at each stage while progressively connecting PD ^{k -1} (k = 1) to PD ^{k - 1} _I (k > 1) independently learned to train the entire network. In other words, in a network structure such as PPD, it is difficult to learn PDs included in a plurality of steps for each scale level at once. In order to solve this problem, while sequentially connecting the PDs for each step, which are network components, tuning learning is performed to tune the existing initial learning parameters for the newly added PD according to the added network, Complete the learning of the connected PD. At this time, the pre-connected PD for which tuning learning has been completed while being connected to the previous step is not learned, and only the PD to be added is learned.

The process of learning a plurality of scales can be independently performed for each scale and then connected to each other. Based on the configuration up to this point, tuning learning is performed on the newly connected PD to sequentially connect the networks of additional connection steps for each scale level.

In this way, PD ^{k -1} (k = 1) and PD ^{k - 1} _I (k > 1) are set by learning the respective network parameters. In the PPD, the PD at each step uses available image information, such as the image scaled from the original as the input of the step and the reconstructed image upscaled through the PD of the previous step, as the input of the PD at the step, learning goal As an image, learning is performed by applying an image obtained by scaling the first clean original image to be the same as the output scale level of the corresponding step PD. When learning the artificial intelligence network of the present invention, an input image reflected with various distortions can be used as the original input image, and the first clean original image, which is the original input image without distortion, can be used as the input original image with these distortions reflected The artificial intelligence network must be trained so that it can be output.

PD ^{k - 1} (k = 1) receives as input only the image (I ^{k - 1} _content , k = 1) scaled by the input scale of the corresponding step, and outputs the k (k ) output of PD ^{k -1} (k = 1) = 1) Tuning learning is performed with an image obtained by scaling the first clean original image to the scale level of the scale level reconstructed image (I ^k _detail , k=1) as a target image.

PD ^{k - 1} _I (k>1) is an image (I ^k-1 _content , k>1) downscaled to the input scale of the corresponding step and an output scale level reconstructed image (I ^k-1 _detail , k> of the previous step PD) 1) as an input, scaling the first clean original image to the scale level of the _kth (k>1) scale level reconstructed image (I ^k _detail , k>1), which is the output of PD ^{k -1 I} (k>1) Tuning learning is performed using one image as a target image.

PD ^k _I , the next step, is an image (I ^k _content , k>1) scaled by the input scale of the corresponding step and an output scale level restored image (I ^k _detail , k> 1) of the current step PD ^{k − 1} _I (k>1). 1) as an input, an image obtained by scaling the first clean original image to the scale level of the k+1th (k>1) scale level reconstructed image (I ^k+1 _detail , k>1), which is the output of PD ^k _I Tuning learning is performed using the target image.

2 is a diagram illustrating an artificial intelligence network configuration and a learning method to which DAF is applied according to a second embodiment.

[PPD and DAF (inference) network structure description]

The structure of the artificial intelligence network of the present invention according to the second embodiment is a structure in which a Domain Aware Fusion (DAF) module is additionally applied to each step from the structure of the first embodiment. (See Fig. 2)

That is, the second embodiment has a structure in which a DAF is added to the front end of the PD for each stage in the structure of the first embodiment. In other words, in the structure of the first embodiment, the input image scaled by the corresponding step input scale, which is one of the inputs of the PD of each step, is not used as an input. Instead, after adding the DAF module to the corresponding input position, the corresponding step input This is a structure in which the optimal information for obtaining the PD output of the corresponding step is generated using various information available in the corresponding step, including the input image scaled by the scale, and then used as the input of the PD of the corresponding step.

That is, a plurality of DAF modules (DAF ^k-1 , k>1) scale DAF output images included in lower stages to the input scale of the corresponding stage (I ^L→k-1 _da-content , k>1, (k-1)>L≥0) (however, if k=2, I ^{0→ 1} _{da -content} = I ^{0→ 1} _content ), the image downscaled to the input scale of the corresponding step (I ^k-1 _content , k>1), DAF (DAF ^k -1, k>1) of the current step is performed based on available information such as the output reconstructed image (I ^k-1 _detail , k>1) of the previous step PD, Optimum DAF output information (I ^k-1 _da-content , k>1) for obtaining the optimal output of the PD of the step is generated.

[Description of DAF Structure and Function]

The DAF of the second embodiment is a module that generates information necessary to obtain an optimal upscaling reconstructed image output through the PD of the corresponding step, and modules having various structures can be applied. As an example, the structure and function of FIG. 3 will be described.

A plurality of DAF modules, an image obtained by scaling an input image to the input scale of the corresponding step, images obtained by scaling DAF outputs of sub-steps of the corresponding step to the input scale of the corresponding step, and a restored PD output image of the previous step, input information for each step receive as

The DAF module at each stage generates an image scaled by the input scale of the corresponding stage, images obtained by scaling the DAF outputs of the lower stages of the corresponding stage to the input scale of the corresponding stage, and a difference between the PD output reconstructed image of the previous stage, respectively. yield

The DAF module performs group convolution on each of the subtraction values and inputs them to Softmax so that the total sum becomes 1 when the output values are summed. The DAF module multiplies and sums each output value that has passed through softmax by the previous PD output reconstructed image, generates DAF output information (I ^k-1 _da-content , k>1), and inputs it to the PD connected afterwards. .

The DAF output information output through this process performs a function to maintain the characteristics of the actual input image information without weakening the PD output information of the lower stage through the stages. That is, the DAF module can be designed, learned, and applied to perform necessary functions in the network of the present invention, and, as in the example of FIG. You can also do

[How to learn PPD and DAF networks]

The network learning method of the second embodiment uses a progressive method like the learning method of the first embodiment, and the difference from the first embodiment is that DAF is added, and DAF is added along with PD of each step during step-by-step learning. is to learn together.

A plurality of PDs (Pretrained Decoders) according to the second embodiment independently perform initial learning for each scale of the image, and then sequentially connect upper-level PDs and DAFs in a cascade in a PPD format at lower levels. , when PDs and DAFs corresponding to each step are added, tuning learning is performed for PDs and DAFs added, and at this time, PDs and DAFs for which tuning learning has already been completed in the previous step are not learned again.

The configuration of the entire learning network consists of a plurality of PDs and DAFs, the plurality of PDs include PD ^{k -1} (k = 1), PD ^{k - 1} _I (k > 1), and the plurality of DAFs are DAF ^{k - 1} (k>1). In this case, when the subscript I is indicated, it indicates that parameter inflating is applied to the corresponding PD. Parameter inflating is a method for expanding a number of input channels of a pretrained network. The method consists of a process of copying a plurality of pre-learned input channels, a process of concatenating the plurality of copied channels, and a process of adjusting the scale of parameters in the concatenated channels.

In order to learn the network of the PPD structure, PD ^{k -1} (k = 1) to PD ^{k - 1} _I (k > 1) independently learned, and DAF ^{k -1} (k > 1) for each stage initialized , sequentially and progressively at each stage, and learning the entire network by performing tuning learning for PD at each stage and learning for DAF at each stage. In other words, in a network structure such as PPD, it is difficult to learn PDs and DAFs included in a plurality of steps at once. In order to solve this problem, while sequentially connecting the PD and DAF for each stage, which are network components, tuning learning is performed to tune the existing initial learning parameters to the newly added network for the newly added PD, and newly added For the DAF to be initialized, learning is performed according to the network to be added, and learning of the newly connected PD and DAF is completed for the corresponding step. At this time, PDs and DAFs connected up to the previous step and completed learning are not learned, but only PDs and DAFs that are added are learned.

The process of learning a plurality of scales can be independently performed for each scale and then connected to each other. Based on the configuration up to the network to be connected, tuning learning is performed for the newly connected PD, and learning is performed for the additionally connected DAF to sequentially connect the networks for each step.

In this way, each network parameter of PD ^{k −1} (k=1), PD ^{k − 1} _I (k>1) and DAF ^{k −1} (k>1) is learned and set.

In the network of the PPD structure of Example 2, the PD at each stage uses the output image of the previous stage PD and the DAF output information of the corresponding stage as inputs, and uses the first clean original image as the learning target image as the PD of the corresponding scale level. Learning is performed by applying an image scaled to be the same as the output scale of .

At this time, the DAF of the corresponding step is also learned at the same time, and the DAF includes information obtained by scaling DAF outputs of a lower step than the corresponding step to the DAF input scale of the current step, the output of the upscaled reconstructed image of the PD of the previous step, and the like. After applying the information as the input of the DAF of the corresponding stage, the output information is applied as the input of the PD of the corresponding stage, and the output of the PD of the corresponding stage is the learning target image, the first clean original image scaled to the corresponding scale level. DAF learning is performed together with the PD of the corresponding stage so that

PD ^{k - 1} (k = 1) receives only the image (I ^{k - 1} _content , k = 1) scaled by the input scale of the corresponding step as input, and outputs the k (k ) output of PD ^{k -1} (k = 1). = 1) Tuning learning is performed with an image obtained by scaling the first clean original image to the scale level of the scale level reconstructed image (I ^k _detail , k=1) as a target image.

DAF ^{k -1} (k>1) is an input, and can use various information available from up to a lower step of the corresponding step. For example, as shown in FIG. 2, scaling I ⁰ _content to the input scale of the current step The image, information obtained by scaling DAF output information (DAF ^k-1 output information, all k where the current step>k>1) of all sub-steps before the current step to the input scale of the current step, and the original input image of the current step All available information, such as an image downscaled to the input scale (I ^k-1 _content , k>1) and an output reconstructed image (I ^k-1 _detail , k>1) of a previous stage PD, can be used as input. Learning about DAF ^{k -1} (k > 1) is learned to output optimal image information that enables PD ^{k -1} (k > 1) of the corresponding step to generate a learning target image with respect to this input information. .

PD ^{k - 1} _I (k>1) receives the output information of DAF ^{k - 1} (k>1) and the output reconstructed image (I ^k-1 _detail , k>1) of the previous PD as input, and PD ^{k -} The target image is an image obtained by scaling the first clean original image to the scale level of the k (k > 1) scale level upscaled reconstructed image (I ^k _detail , k > 1), _{which is} ^the output of 1 I (k > 1). Perform tuning learning.

The image restoration apparatus according to the present embodiment includes a learning unit.

The learning unit independently performs learning according to a scale of an image for each of a plurality of pretrained decoders (PDs).

The learning unit performs learning while connecting a plurality of PDs and a plurality of DAFs from the lower scale level to the highest scale level step by step. The learning unit prevents learning from being performed again for PDs and DAFs learned in a previous step for each of a plurality of PDs and DAFs.

The learning unit selects a PD used to obtain each scale level image output among a plurality of PDs step by step. The learning unit performs learning independently for each scale of the input image at the time of initial learning in the PD used in each step. The learning unit performs learning step by step from the lower scale level on the structure that connects the PD for which learning has been completed for each scale level and the DAF in front of each PD. The learning unit performs progressive learning so that the previous step is not learned and updated when learning the newly added step after additionally connecting the next step in the network until the learning is completed.

The learning unit performs parameter inflating for each PD used in the remaining steps except for the PD used in the first scale level among the plurality of PDs, and receives the output of the PD of the previous step and the output of the DAF of the corresponding step to perform learning.

The above description is merely an example of the technical idea of the present embodiment, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of this embodiment should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of this embodiment.

Claims (7)

An inference network block having a structure that configures a plurality of neural networks independently learning image reconstruction for each image scale and uses image restoration results of lower scale neural networks as additional information for image restoration of a current scale neural network;
A network module having a structure for applying different weights for each scale in consideration of a trade-off between image distortion restoration and image content preservation when using the image restoration results of the plurality of lower-scale neural networks as additional information;
The image restoration network characterized in that it has a structure for performing image restoration by gradually connecting the inference network block and the network module from the lowest scale to the highest scale. ^The first scale level upscaled reconstructed image ⁽ _I ^k _detail ^, k = 1) outputting PD ^k-1 (k = 1) (Pretrained Decoder); and
An image (I ^L→k-1 _da-content , k>1, (k-1)>L≥0) by scaling the DAF output images included in the lower step to the input scale of the step (provided that k=2 In this case, I ^{0→ 1} _{da -content} = I ^{0→ 1} _content ), an image downscaled to the input scale of the corresponding step (I ^k-1 _content , k>1), and an output reconstructed image of the previous step PD (I ^k-1 A plurality of DAF modules (DAF k-1, k to output optimal DAF output information (I ^k-1 _da-content , k>1) of the current stage based on available information including _detail , k> ¹ ) >1)
Image restoration device characterized in that it comprises a. According to claim 2,
The output (I k-1 _da-content , k>1) of the DAF module (DAF ^{k -1} , k>1) of the corresponding step and the reconstructed output image (I ^k-1 _detail , k>1) of the previous PD Receive input, output (I k-1 _da-content , k>1) of the DAF module (DAF ^{k-1 ,} k>1) of the corresponding step and output reconstructed image (I ^k-1 _detail , k>1) of the PD of the previous step PD k - 1 I (k>1) outputting a kth scale level upscaled reconstructed image (I ^k _detail , k ^{> 1} ) upscaled by _a preset multiple by reflecting the artificial intelligence learning result in ^k >1)
Image restoration device characterized in that it further comprises. According to claim 3,
Having a PPD (Pretrained Progressive Decoder) form in which the PD ^{k-1 I} ₍ k>1) are sequentially connected in a cascade manner after the PD k-1 (k = 1) independently learned Image restoration device characterized in that. According to claim 2,
The plurality of DAF modules (DAF ^k-1 , k>1)
The image (I ^k-1 _content , k=1) downscaled to the input scale of the corresponding step and the optimal DAF output information (I ^k-1 _da-content , k>1) of the lower step of the corresponding step are converted to the input scale of the corresponding step. An image restoration apparatus characterized in that the images scaled by , and the previous stage PD output reconstruction image are input at each stage. According to claim 5,
The plurality of DAF modules (DAF ^k-1 , k>1)
The image (I ^k-1 _content , k=1) downscaled to the input scale of the corresponding step and the optimal DAF output information (I ^k-1 _da-content , k>1) of the lower step of the corresponding step are converted to the input scale of the corresponding step. An image restoration device characterized in that each calculates a difference value between images scaled by , and a previous stage PD output reconstruction image. According to claim 6,
The plurality of DAF modules (DAF ^k-1 , k>1)
Group convolution is performed on each of the subtraction values, input to Softmax so that the sum of the output values is 1, and then each output value that has passed through the softmax is multiplied by the restored PD output image of the previous step and then summed. and generating the optimal DAF output information (I ^k-1 _da-content , k>1) and then inputting the information to the connected PD.

Images