KR102102182B1 - Apparatus and method for restoring image - Google Patents

Abstract

An image reconstruction apparatus according to an embodiment of the present invention includes a noise vector generator for generating a noise vector using a noise level, and an image feature map and a noise vector generated from an input image including noise in each of a plurality of residual blocks And a noise canceling neural network to remove noise included in the input image by calculating.

Description

Image restoration apparatus and method {APPARATUS AND METHOD FOR RESTORING IMAGE}

The present invention relates to an image restoration apparatus and method. More specifically, it relates to an image restoration apparatus and method capable of restoring an image using a trained neural network to minimize memory storage space.

Recently, due to the development of devices, images can be easily and conveniently created, and various images are widely used. Due to this, noise may be mixed into the image due to a photographing device that photographs the image, and noise may be mixed into the image in a process of processing an image such as compression or decompression.

In this way, noise mixed in the image is a factor that degrades the image quality. Moreover, due to the advancement of the device, such noise is an important factor to be eliminated in order to improve image quality in image processing because it uses a high resolution in terms of image resolution.

In relation, Korean Patent Registration No. 10-1578426, which is a prior art document, selects a modeling model corresponding to input image information from among a plurality of modeling information in order to remove noise components included in the input image information, and the characteristics of modeling. As a basis, a signal processing apparatus for removing noise components included in image information is described. However, such an image processing apparatus needs to include various information included in a plurality of modeling in order to remove noise from an image. Moreover, the multiple modeling has a problem in that the memory share is high because data collected for each of various components constituting an image is used. In addition, since various electronic devices have a limited memory capacity due to portability, it is difficult to apply a noise reduction method using a plurality of modeling information.

Therefore, a technique for solving the above-described problems is needed.

On the other hand, the above-mentioned background technology is the technical information acquired by the inventor for the derivation of the present invention or acquired in the derivation process of the present invention, and is not necessarily a known technology disclosed to the general public before filing the present invention. .

The embodiments disclosed herein have an object to provide an image reconstruction apparatus and method capable of minimizing the occupied memory capacity of a neural network used to remove noise from an image.

The embodiments disclosed herein have an object to provide an image restoration apparatus and method for minimizing memory capacity when applying a neural network for image restoration.

The embodiments disclosed herein have an object to provide an image reconstruction apparatus and method for learning a neural network capable of removing noise in an image.

As a technical means for achieving the above-described technical problem, according to an embodiment, the image reconstruction apparatus includes a noise vector generating unit that generates a noise vector using a noise level, and noises in each of a plurality of residual blocks. And a noise canceling neural network that removes noise included in the input image by calculating an image feature map generated from an input image and the noise vector.

According to another embodiment, an image reconstruction method performed by an image reconstruction apparatus includes generating a noise vector using a noise level, and in each of a plurality of residual blocks, an image feature map generated from an input image including noise. And learning the noise canceling neural network by calculating the noise vector to remove noise included in the input image.

According to another embodiment, a computer-readable recording medium in which a program performing an image restoration method is recorded, generating a noise vector using a noise level, and in each of the plurality of residual blocks, an input image including noise And learning the noise canceling neural network by removing the noise included in the input image by calculating the image feature map generated from the noise vector and the noise vector.

According to another embodiment, a computer program performed by an image reconstruction apparatus and stored in a medium to perform an image providing method, comprising: generating a noise vector using a noise level, and in each of the plurality of residual blocks, And learning a noise canceling neural network by removing the noise included in the input image by calculating the image feature map generated from the input image including noise and the noise vector.

According to any one of the above-described problem solving means of the present invention, it is possible to propose an image restoration apparatus and method capable of minimizing the occupied memory capacity of a neural network used to remove noise from an image.

In addition, according to any one of the problem solving means of the present invention, it is possible to propose an image restoration apparatus and method for minimizing memory capacity when applying a neural network for image restoration.

In addition, according to any one of the problem solving means of the present invention, it is possible to present an image restoration apparatus and method for learning a neural network capable of removing noise in an image.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description. will be.

1 is a block diagram illustrating an image restoration apparatus according to an embodiment.
2 is a block diagram illustrating a control unit for learning a noise canceling neural network according to an embodiment.
3 is a block diagram illustrating a noise vector generator according to an embodiment.
4 is a block diagram illustrating a noise canceling neural network according to an embodiment.
5 is a block diagram illustrating a residual block according to an embodiment.
6 is a diagram illustrating an operation in which noise is coupled to an image block according to an embodiment.
7 is a flowchart illustrating an operation of learning a noise canceling neural network in an image reconstruction apparatus according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below may be embodied in various different forms. In order to more clearly describe the features of the embodiments, detailed descriptions of the matters well known to those skilled in the art to which the following embodiments pertain are omitted. In the drawings, parts irrelevant to the description of the embodiments are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a configuration is said to be "connected" to another configuration, this includes not only a case of 'directly connecting', but also a case of 'connecting another configuration in between'. In addition, when a configuration is said to "include" a configuration, this means that unless otherwise stated, other configurations may be excluded and other configurations may be further included.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

However, before explaining this, the meanings of terms used below are first defined.

Hereinafter, the 'noise cancellation neural network' is a neural network for removing noise included in an image. The noise canceling neural network is an artificial neural network, and may be modeled to remove noise of an input image.

'Image feature map' and 'noise feature map' refer to a feature map generated using an artificial neural network, for example, a convolutional neural network. Here, the image feature map means a feature map generated by receiving an image, and the noise feature map means a feature map generated by receiving a noise component.

In addition to the terms defined above, terms that require explanation will be described separately below.

1 is a block diagram illustrating an image restoration apparatus according to an embodiment.

Referring to FIG. 1, the image restoration apparatus 100 may be connected to a remote server through a network, or may be implemented as an electronic device that can be connected to another electronic device or a server or may be implemented as a server.

In this case, the electronic device may be implemented as a computer, a portable terminal, a television, or a wearable device. Here, the computer includes, for example, a laptop equipped with a web browser (WEB Browser), a desktop (desktop), a laptop (laptop), and the like, and the portable terminal is, for example, a wireless communication device that guarantees portability and mobility. , PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile Telecommunication) -2000, CDMA (Code Division Multiple Access (2000), W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro), Smart Phone, Mobile Worldwide Interoperability for Microwave Access (WiMAX), etc. (Handheld) -based wireless communication device. In addition, the television may include Internet Protocol Television (IPTV), Internet Television (TV), terrestrial TV, and cable TV. Furthermore, the wearable device is a type of information processing device that can be directly worn on the human body, for example, a watch, glasses, accessories, clothing, shoes, etc., or connects to a remote server through a network directly or through another information processing device or other terminal. And can be connected.

The image restoration apparatus 100 may include an input / output unit 110, a communication unit 120, a storage unit 130, and a control unit 140.

The input / output unit 110 may include an input unit for receiving input from a user, and an output unit for displaying information such as a result of performing a job or a state of the image restoration apparatus 100. For example, the input / output unit 110 may include an operation panel receiving a user input, a display panel displaying a screen, and the like.

Specifically, the input unit may include devices capable of receiving various types of user input, such as a keyboard, a physical button, a touch screen, a camera, or a microphone. Also, the output unit may include a display panel or a speaker. However, the present invention is not limited thereto, and the input / output unit 110 may include a configuration supporting various input / output.

The communication unit 120 may perform wired / wireless communication with other devices or networks. To this end, the communication unit 120 may include a communication module supporting at least one of various wired and wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

The wireless communication supported by the communication unit 120 may be, for example, Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth (Bluetooth), UWB (Ultra Wide Band), or NFC (Near Field Communication). In addition, the wired communication supported by the communication unit 120 may be, for example, USB or High Definition Multimedia Interface (HDMI). The above-described communication is only an example, and various communication techniques enabling the image restoration apparatus 100 to perform communication are possible.

The storage unit 130 may install and store various types of data such as files, applications, and programs. Data stored in the storage unit 130 may be accessed and used by the control unit 140 to be described later, or new data may be stored by the control unit 140. Also, the storage unit 130 may store a program that can be executed by the control unit 140. According to an embodiment, a program for learning a noise canceling neural network and a program for performing an operation for removing noise in an image using the noise canceling neural network may be stored in the storage unit 130.

On the other hand, the control unit 140 controls the overall operation of the image restoration apparatus 100, and may include a processor such as a CPU. The control unit 140 may control other components included in the image restoration apparatus 100 to perform an operation corresponding to a user input received through the input / output unit 110.

According to this embodiment, the control unit 140 may learn a noise canceling neural network. To this end, the control unit 140 may generate a noise vector from the noise level using a neural network. The control unit 140 may generate a noise vector while varying the noise level.

The controller 140 may learn a noise canceling neural network using a noise vector. When the noise canceling neural network learned by the controller 140 receives an image containing noise, it generates an image feature map and calculates an image feature map and noise vectors corresponding to various noise levels in each of the plurality of residual blocks to remove noise. You can train neural networks.

Since the controller 140 learns using a noise vector generated as a noise level in the noise canceling neural network, the controller 140 may perform learning so as not to be biased to a specific noise level. In the noise canceling neural network, the control unit 140 generates an image feature map from an image containing noise and computes and combines the image feature map and the noise vector from a plurality of residual blocks. The controller 140 may learn to output a noise-removed image as an output image by inputting a noise vector from the noise-reduction neural network.

The controller 140 may train the noise canceling neural network by varying the noise level, and when the learned neural network is used, an image obtained by removing noise from an input image in which noise is mixed may be output.

In particular, the control unit 140 may perform learning by using a noise level in which information on noise expressed only in an input image is more clearly divided into numbers. In addition, the noise level is a coefficient capable of adjusting the convolution filter of the noise canceling neural network, so that the controller 140 may change the structure of the noise level neural network to some adaptive form according to the noise level.

For this reason, the controller 140 can learn various noise levels without being focused on a certain noise level. For this reason, the control unit 140 may ensure image restoration performance even in an image having a low noise level or a complex image.

In addition, since the control unit 140 has a constant performance even at a noise level outside the learning range even when only one noise canceling neural network is used, the controller 140 can process the noise level outside the learning range.

As described above, the image reconstruction apparatus 100 may learn a noise canceling neural network, and store the trained noise canceling neural network in an internal storage unit 130 or the like to use to remove noise from an input image.

Accordingly, the image reconstruction apparatus 100 may perform only a function related to learning of a noise canceling neural network, and may be implemented to perform only a function of removing noise of an input image by applying a trained noise canceling neural network.

For this reason, the proposed image reconstruction apparatus 100 uses one noise canceling neural network to remove the noise from the noise image, thereby eliminating the need to store all the learning coefficients of the noise canceling neural network for various noise levels. Since only one learning coefficient is required, the storage 140, that is, the storage space of the memory can be reduced.

2 is a block diagram illustrating a control unit for learning a noise canceling neural network according to an embodiment.

Referring to FIG. 2, the control unit 140 may include a noise vector generator 141 and a noise canceling neural network 142.

Here, the noise vector generator 141 may receive a noise level that is noise information numerically numerically input. For example, the noise level is information that can be divided into numbers such as 1, 5, 10, 20, 25, and 30.

The noise vector generator 141 may be configured in the form of a neural network or a convolutional neural network. The noise vector generator 141 has a structure in which two fully connected layers are connected to generate a noise vector from a noise level. The noise vector generator 141 may generate a noise vector having a dimension corresponding to an image feature map in the neural network. At this time, the noise vector is input to the noise canceling neural network 142.

The noise canceling neural network 142 may receive an input image containing noise, and may perform learning by receiving a noise vector. Through this, the noise canceling neural network outputs the noise-removed output image.

The noise canceling neural network 142 generates an image feature map from the input image, and the dimension of the image feature map may be the same as that of the noise vector. The noise canceling neural network 142 includes a structure in which a plurality of residual blocks having an image feature map as an input are connected, and may calculate and output an image feature map and a noise vector from each of the remaining blocks.

3 is a block diagram illustrating a noise vector generator according to an embodiment.

The noise vector generator 141 may be in the form of a convolutional neural network, and generates a noise vector from noise levels represented by numbers. The noise vector generation unit 141 may include a first full connection layer unit 210 and a second full connection layer unit 220.

The first whole connection layer unit 210 may generate a one-dimensional scalar noise level as a noise feature map having a predetermined dimension in order to display a noise vector. For example, a predetermined dimension may be 64 dimensions. The first full connection layer unit 210 may output a 64-dimensional noise feature map to the second full connection layer unit 220.

The second entire connection layer unit 220 may generate a 128-dimensional noise feature map, that is, a 1x1x128-type noise feature map from a 64-dimensional noise feature map. The second entire connection layer unit 220 may output a noise feature map in the form of 1x1x128 as a noise vector to the noise canceling neural network 142.

Here, as an example, a structure in which two entire connection layer units 210 and 220 are connected to generate a noise vector having an order equal to that of an image feature map processed in a residual block in the noise cancellation neural network 142 Although it is described on the basis of, a larger number of all connection layer parts may be used or only one all connection layer may be used.

As such, the noise vector generator 141 may generate a noise vector in a form that is convenient for processing when learning information about the noise level.

4 is a block diagram illustrating a noise canceling neural network according to an embodiment.

Referring to FIG. 4, the noise canceling neural network 142 includes a first convolution layer unit 310, residual blocks 320, 330, 340, an adder 350, and a second convolution layer unit 360 You can.

The first convolution layer unit 310 may generate a feature map in which an input image having a dimension (or channel) of 1 is expanded into a plurality of dimensions (D). Here, the dimension may be 128-dimensional, and assuming that the dimension (D) of the input image is 1, the dimension (D) of the feature map is 128. Therefore, assuming that the input image is 'HxWx1', the feature map may be 'HxWx128'.

The first residual block, the i residual block, and the nth residual block 320, 330, and 340 may calculate and output a noise vector on the feature map, respectively, and include the first residual block, the i residual block, and the nth The feature map output from each of the remaining blocks 320, 330, and 340 may maintain a 128-dimensional shape (HxWx128). At this time, since the noise vector input to each of the first residual block, the i-th residual block, and the n-th residual block 320, 330, 340 is '1x1x128' and has 128 dimensions, learning according to the noise level is easy.

The adder 350 adds and outputs a feature map output from the first convolution layer unit 310 and a noise vector output from the last residual block, that is, the nth residual block 340.

The second convolution layer unit 360 converts an input value in the form of a feature map having a plurality of dimensions (128 dimensions) into a one-dimensional image identical to the input image. Through this, the second convolution layer unit 360 may output the output image of 'WxHx1'.

In the case of the convolution layer according to the proposed embodiment, a kernel having a predetermined size may be used, and as an example, a 3x3 kernel may be used.

Through this, the noise canceling neural network 142 may learn an operation of obtaining an output image from which noise components included in the input image are removed.

5 is a block diagram illustrating a residual block according to an embodiment.

Referring to FIG. 5, the i-th residual block 330 may include a first convolution block 331, a nonlinear function (ReNU) block 332, a second convolution block 333 and a gate 334. have. The other residual blocks 320 and 340 may have a structure similar to that of the i-th residual block 330, and input and output a feature map having the same size (eg, HxWx128).

The first convolution block 331 may perform a first convolution layer operation on the image feature map. Also, the first convolution block 331 may have a function of a low pass filter. The first convolution block 331 may output the feature map computed with the first convolution layer to the nonlinear function block 332.

The nonlinear function block 332 applies a nonlinear function (eg, f (x) = max (0, x)) to the first convolutional layer computed feature map. Through this, the nonlinear function block 332 may generate a nonlinear feature map that is not linear and cannot be predicted, that is, a new type of feature map. The nonlinear function block 332 outputs the feature map to which the nonlinear function is applied to the second convolution block 333.

The second convolution block 333 may perform a second convolution operation on the feature map to which the nonlinear function is applied. The second convolution block 333 outputs the second convolution computed feature map to the gate 334.

As such, the first convolution block 331 and the second convolution block 333 correspond to the convolutional layer of the neural network, without having a continuous structure, and the nonlinear function block 332 can be positioned therebetween. have. Through this, since a new image feature map is used in each convolution layer (for example, an effect of increasing a depth), image reconstruction performance for noise reduction can be improved.

The gate 334 may output a second convolution computed image feature map and a noise vector by element wise multiplication. Since the size of the dimension of the image feature map and the noise vector are generated to be the same, the gate 334 may output the image feature map through the component product operation.

Meanwhile, the first residual block 320 is a residual block located in the first, and outputs the image feature map as a second residual block by using the image feature map output from the first convolution layer unit 310 of FIG. 4 as an input. . Alternatively, the image feature map may be output to the adder 350 by inputting the image feature map from the previously located residual block as the remaining block where the nth residual block 340 is located last. As such, the remaining blocks may have a connection structure that takes the output from the previous block as an input.

6 is a diagram illustrating an operation in which noise is coupled to an image block according to an embodiment.

Referring to FIG. 6, the i-th residual block 330 may receive the image feature map 410 as an input. Thereafter, the i-th residual block 330 outputs the image feature map 440 by computing the component product 450 of the image feature map 420 and the noise vector 430 output through the two convolution blocks. In addition, since the intensity of the feature map represented by the HxW component of the noise vector can be adjusted, the calculation processes 411, 421, 431, and 441 for each component can be confirmed.

The operation of the i-th residual block 330 may be expressed as Equation 1 below.

는 학습을 위한 계수이고, tanh는 비선형 함수이다. 또한, ○는 성분곱 연산이고, s()는 시그모이드 함수(비선형 함수를 사용하여 잡음 벡터를 정규화 함)이고,

는 잡음 벡터이다.Here, (Y) ^k _{i, out} represents the output of the residual block, and F (Y) ^k _{i, 0} is the input of the residual block. Here, i is the index of the residual block, k is the index of the pixel (k is 1 to HxW). F (Y) ^k _{i, 2} is a feature map that passes through two convolution blocks,

Is a coefficient for learning, and tanh is a nonlinear function. In addition, ○ is a component product operation, s () is a sigmoid function (normalizes a noise vector using a nonlinear function),

Is a noise vector.

7 is a flowchart illustrating an operation of learning a noise canceling neural network in an image reconstruction apparatus according to an embodiment.

Referring to FIG. 7, the image reconstruction apparatus 100 may generate a noise vector using the digitized noise level (S510). Here, the noise vector may be generated by transforming each of 2 dimensions, and may be generated in the form of a feature map through a convolutional neural network for calculation with an image component. Therefore, the image reconstruction apparatus 100 does not directly apply a noise vector in the image, but may apply a separately generated noise vector in the learning step when learning the image.

The image reconstruction apparatus 100 may generate an image feature map by expanding an image including noise to a plurality of dimensions (or channels) (S520).

The image reconstruction apparatus 100 may calculate and output an image feature map and a noise vector from each of the plurality of residual blocks (S530). Here, the image reconstruction apparatus 100 uses a nonlinear function between the convolutional layer operations of the image feature map in each of the remaining blocks. In addition, the image reconstruction apparatus 100 outputs the convolutional layer computed image feature map and noise vector as a component product.

The image reconstruction apparatus 100 converts the noise vector and the calculated feature map into an image corresponding to one dimension, and outputs an image from which noise is removed (S540). The image reconstruction apparatus 100 may perform training of a noise canceling neural network for each noise vector for a noise level set to various values. The image reconstruction apparatus 100 does not require a plurality of noise canceling neural networks according to an image, and when a single noise canceling neural network is used, it can be used to remove noise for all types of images.

The term '~ unit' used in this embodiment means software or hardware components such as a field programmable gate array (FPGA) or ASIC, and '~ unit' performs certain roles. However, '~ wealth' is not limited to software or hardware. The '~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, '~ unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, and procedures. , Subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays, and variables.

The functionality provided within components and '~ units' may be combined into a smaller number of components and '~ units', or separated from additional components and '~ units'.

In addition, the components and '~ unit' may be implemented to play one or more CPUs in the device or secure multimedia card.

In addition, the image restoration method according to an embodiment of the present invention may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor and may be implemented in a high-level programming language, object-oriented programming language, assembly language, or machine language. . In addition, the computer program may be recorded on a tangible computer-readable recording medium (eg, memory, hard disk, magnetic / optical medium, or solid-state drive (SSD), etc.).

Therefore, the image restoration method according to an embodiment of the present invention may be implemented by executing the computer program as described above by a computing device. The computing device may include at least a portion of a processor, a memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and may be mounted on a common motherboard or mounted in other suitable ways.

Here, the processor is capable of processing instructions within the computing device, such as for displaying graphical information for providing a graphical user interface (GUI) on an external input or output device, such as a display connected to a high-speed interface. Examples include instructions stored in memory or storage devices. In other embodiments, multiple processors and / or multiple buses may be used with multiple memories and memory types as appropriate. In addition, the processor may be implemented as a chipset formed by chips including a plurality of independent analog and / or digital processors.

Memory also stores information within computing devices. In one example, the memory may be comprised of volatile memory units or a collection thereof. As another example, the memory may be composed of non-volatile memory units or a collection thereof. The memory may also be other types of computer readable media, such as magnetic or optical disks.

And the storage device can provide a large storage space for the computing device. The storage device may be a computer-readable medium or a configuration including such a medium, and may include, for example, devices within a storage area network (SAN) or other configurations, and may include floppy disk devices, hard disk devices, optical disk devices, Or a tape device, flash memory, or other similar semiconductor memory device or device array.

The above description of the present invention is for illustration only, and a person having ordinary knowledge in the technical field to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

100: image restoration device 110: input and output unit
120: communication unit 130: storage unit
140: downlink scheduler 141: noise vector generator
142: noise canceling neural network 210: the first whole connection layer portion
220; 2nd whole connection layer part 310: 1st convolution layer part
320, 330, 340: residual blocks 350: adder
360: second convolution layer unit 331: first convolution block
332: nonlinear function block 333: second convolution block
334: gate

Claims (14)

A noise vector generator that generates a noise vector using a noise level; And
In each of the plurality of residual blocks, an image feature map generated from an input image including noise and a noise canceling neural network for removing noise included in the input image by calculating the noise vector,
The noise cancellation neural network,
A first convolutional layer unit that expands the input image to a plurality of dimensions to generate the image feature map;
Calculating and outputting the image feature map with the noise vector, a plurality of residual blocks each corresponding to a layer having a predetermined depth; And
And a second convolution layer unit that converts the image feature map output by calculating the noise vector into a single-dimensional image and outputs an image from which noise is removed. According to claim 1,
The noise vector generation unit,
A first overall connection layer unit for generating a noise feature map having a first dimension using the noise level; And
And a second entire connection layer unit for generating the noise vector having a second dimension using the noise feature map,
The first dimension has an image reconstruction apparatus having a smaller dimension than the second dimension. delete According to claim 1,
And an adder configured to add and output the output of the last remaining block among the image feature map and the plurality of residual blocks to the second convolution layer. According to claim 1,
Each of the plurality of residual blocks,
A first convolution block for computing the image feature map as a first convolution layer;
A nonlinear function block applying a nonlinear function to the convolutional layer computed image feature map;
A second convolution block for calculating a second convolution layer on the image feature map to which the nonlinear function is applied; And
And a gate for calculating and outputting the noise vector on the second convolutional feature map. According to claim 1,
Each of the remaining blocks
An image reconstruction apparatus for calculating the image feature map and the noise vector based on the following equation.
[Mathematics]

Here, F (Y) ^k _{i, out} is the output of the residual block, F (Y) ^k _{i, 0} is the input of the residual block, i is the index of the residual block, k is the index of the pixel, and F (Y ) ^k _{i, 2} is a feature map that passes through two convolution blocks,

Is a coefficient for learning, tanh is a nonlinear function, ○ is a component product operation, s () is a sigmoid function,

Is a noise vector. In the image restoration method performed by the image restoration device,
Generating a noise vector using the noise level; And
In each of the plurality of residual blocks, a step of learning a noise canceling neural network by removing the noise included in the input image by calculating the image feature map and the noise vector generated from the input image including the noise,
The step of learning the noise canceling neural network,
Generating the image feature map by expanding the input image into a plurality of dimensions;
Calculating and outputting the image feature map from each of the plurality of residual blocks with the noise vector; And
And converting the image feature map output by calculating the noise vector into a single-dimensional image to output a noise-removed image. The method of claim 7,
Generating the noise vector,
Generating a noise feature map having a first dimension using the noise level; And
Generating the noise vector having a second dimension using the noise feature map,
The first dimension has a smaller dimension than the second dimension image reconstruction method. delete The method of claim 7,
After calculating and outputting the noise vector,
And adding and outputting the output of the last residual block among the image feature map and the plurality of residual blocks. The method of claim 7,
The calculating and outputting the image feature map with the noise vector may include:
Calculating a first convolutional layer of the image feature map;
Applying a nonlinear function to the convolutional layer computed image feature map;
Calculating a second convolutional layer on the image feature map to which the nonlinear function is applied; And
And calculating and outputting the noise vector on the second convolution computed feature map by element wise multiplication. The method of claim 7,
The calculating and outputting the image feature map with the noise vector may include:
An image reconstruction method for calculating the image feature map and the noise vector based on the following equation.
[Mathematics]

Here, F (Y) ^k _{i, out} is the output of the residual block, F (Y) ^k _{i, 0} is the input of the residual block, i is the index of the residual block, k is the index of the pixel, and F (Y ) ^k _{i, 2} is a feature map that performs two convolutional layer calculations,

Is a coefficient for learning, tanh is a nonlinear function, ○ is a component product operation, s () is a sigmoid function,

Is a noise vector. A computer-readable recording medium on which a program for performing the method according to claim 7 is recorded. A computer program performed by an image restoration device and stored in a medium to perform the method according to claim 7.

Images