KR102206792B1 - Method for image denoising using parallel feature pyramid network, recording medium and device for performing the method - Google Patents

Abstract

A method of removing noise in a network image by using a parallel feature pyramid includes: generating a feature map by performing a preset number of convolution operations on an input image; generating n-layer pyramid feature maps by downsizing the feature map to have mutually different scales through a pyramid (spatial pyramid pooling) module; converting the n-layer pyramid feature maps having mutually different scales into unique scales of corresponding layers through n multi-scale context aggregation (MSCA) modules; and integrating information of the feature maps output from the MSCA modules to output the integrated information as one image data. Accordingly, in consideration of various scales in the image, noise removal performance is increased, and unnecessary artifacts are reduced, so that an image having excellent quality is provided.

Description

A method of removing noise in a network image using a parallel feature pyramid, a recording medium and a device for performing it

The present invention relates to a method for removing noise in a network image using a parallel feature pyramid, a recording medium and an apparatus for performing the same, and more particularly, based on a DeNoise CNN (DnCNN) among a convolutional neural network (CNN). It relates to a technology for improving the noise reduction performance of an image.

[Explanation of nationally supported R&D]

This study was conducted with the support of the Information Communication Planning and Evaluation Institute with the financial resources of the government (Ministry of Science, ICT, and ICT) in 2019 (No. 2014-3-00077, Global multi-object tracking and situation based on large-scale real-time video analysis. Development of predictive skills).

Noise removal in the image is an essential technology in equipment using cameras. Although camera sensors are developing a lot, it is still inevitable that noise is mixed in the image during the image acquisition process. Therefore, a technology for removing noise from an image is also applied to an image signal processing module of a camera.

The most important point in image noise removal technology is to remove only image noise while maintaining image detail. If you only pay attention to noise removal, the details of the image may disappear, resulting in a blurry image.

In recent years, the use of images is very high, and since various technologies using images are developing, the importance of obtaining a clean image is increasing more and more.

Various technologies have been researched for decades to remove noise in images. These techniques remove noise from the image and apply it to the image by creating a filter that preserves detail, and sometimes remove the noise through optimization techniques.

Recently, a convolutional neural network (CNN) is used to improve image noise removal performance. DnCNN (DeNoise CNN) is the most representative of CNN-based image noise networks.

DnCNN utilizes batch normalization and residual learning. Through residual learning, the network learns the difference between the noisy image and the clean image. Through this process, the network erases the contents of the image from the noisy image and leaves only the noise component. DnCNN is a widely used technique because it shows good performance.

Meanwhile, DnCNN uses a single-scale feature map to remove noise in the image. However, since there are contents of various scales in an image, a feature map of a single scale has a limitation in that it is difficult to capture all the features of such an image.

US 9,984,325 B1 JP 6103243 B2 KR 10-1938945 B1

K.Zhang,W.Zuo,Y.Chen,D.Meng,andL.Zhang,,"Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising," IEEE Trans. on Image Process., vol. 26, no. 7, pp. 3142-3155, July 2017 K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional network for visual recognition," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 346-361 S.-W. Kim, H.-K Kook, J.-Y. Sun, M.-C. Kang, and S.-Jea Ko, "Parallel feature pyramid network for object detection," in Proc. Eur. Conf. Comput. Vis., 2018, pp. 234-350

Accordingly, the technical problem of the present invention is conceived in this respect, and an object of the present invention is to provide a method for removing noise in a network image using a parallel feature pyramid to reflect various scales in the image.

Another object of the present invention is to provide a recording medium in which a computer program for performing a method for removing noise in a network image using the parallel feature pyramid is recorded.

Another object of the present invention is to provide an apparatus for performing a method for removing noise in a network image using the parallel feature pyramid.

A method for removing noise in a network image using a parallel feature pyramid according to an embodiment for realizing the object of the present invention is to generate a feature map by performing a predetermined number of convolution operations on an input image. Generating; Generating n-layer pyramid feature maps by downsizing the feature maps to have different scales through a pyramid (Spatial pyramid pooling) module; converting the pyramid feature maps of n layers having different scales into a unique scale of each corresponding layer through n multi-scale context aggregation (MSCA) modules; And integrating information on the feature maps output from each of the MSCA modules and outputting one image data.

In an embodiment of the present invention, the converting of the n-layer pyramid feature maps to a unique scale of each corresponding layer may include performing convolution calculations of the n-layer pyramid feature maps having different scales in each MSCA module; Converting each of the convolutional feature maps into a unique scale of each corresponding layer; Performing a concatenation operation of the feature maps converted to the intrinsic scale; And performing a convolution operation and refinement of the convolutional feature map.

In an embodiment of the present invention, the step of integrating information of feature maps output from each MSCA module and outputting the information as one image data includes: converting feature maps output from each MSCA module to a reference scale; Performing a concatenation operation of feature maps converted to the reference scale; Performing a convolution operation on the joint-operated feature map by a preset number; And adding and outputting the convolutional feature map.

인 경우,

(여기서, n은 피라미드의 층 수이며, 1층은 n=0으로 정의됨)의 크기를 갖는 특징 맵들을 생성할 수 있다.In an embodiment of the present invention, the step of converting the n-layer pyramid feature maps to a unique scale of each corresponding layer includes a size of the input image

If is,

(Where n is the number of layers of the pyramid, and the first layer is defined as n=0), feature maps having a size can be generated.

In an embodiment of the present invention, the generating of the n-layer pyramid feature maps includes: generating a first pyramid feature map having the same scale as the input image; Downsizing the first pyramid feature map to generate a second pyramid feature map having a 1/2 scale of the input image; And generating a third pyramidal feature map having a 1/4 scale of the input image by downsizing the second pyramidal feature map.

In an embodiment of the present invention, the converting of the n-layer pyramid feature maps to a unique scale of each corresponding layer includes: downsizing after a convolution operation of the first pyramid feature map; Performing a convolution operation of the second pyramidal feature map; And upsizing after a convolution operation of the third pyramid feature map.

In an embodiment of the present invention, the step of converting the n-layer pyramid feature maps to a unique scale of each corresponding layer comprises: a downsized first pyramid feature map, a second pyramid feature map, and an upsized third pyramid feature map. Performing a concatenation operation of; And performing a convolution operation and refinement of the convolutional feature map.

In an embodiment of the present invention, the step of integrating information of feature maps outputted from each MSCA module and outputting it as one image data may include: upsizing feature maps output from each MSCA module as necessary; Performing a concatenation operation of the upsized feature maps; Performing a convolution operation on the joint-operated feature map by a preset number; And adding and outputting the convolutional feature map.

In a computer-readable storage medium according to an embodiment for realizing another object of the present invention, a computer program for performing a method of removing noise in a network image using the parallel feature pyramid is recorded.

In an embodiment for realizing another object of the present invention, an apparatus for removing noise in a network image using a parallel feature pyramid is characterized by performing a predetermined number of convolution operations on an input image. A feature map generator for generating a map; A pyramid (Spatial pyramid pooling) module for generating n-layer pyramid feature maps by downsizing the feature maps to have different scales; N multi-scale context aggregation (MSCA) modules for converting pyramid feature maps of n layers having different scales into unique scales of each corresponding layer; And a fusion unit for integrating information of feature maps output from each MSCA module and outputting one image data.

In an embodiment of the present invention, each MSCA module of the n MSCAs performs convolutional operation on each of the n-layer pyramid feature maps having different scales, and downsizing or upsizing each of the convolutional feature maps. Each layer can be converted to its own scale.

In an embodiment of the present invention, each MSCA module of the n MSCAs may refine the feature maps converted to the intrinsic scales of the respective layers by concatenation and convolution operations.

In an embodiment of the present invention, the fusion unit may upsize feature maps output from each of the MSCA modules to convert them to a reference scale, perform concatenation and convolution operations, and then add and output the feature maps. have.

According to such a method for removing noise in a network image using a parallel feature pyramid, not only has high noise removal performance in consideration of various scales in the image, but also reduces unnecessary artifacts to provide an image of excellent quality.

1 is a block diagram of an apparatus for removing noise in a network image using a parallel feature pyramid according to an embodiment of the present invention.
2 is a conceptual diagram showing a general network structure of DnCNN.
3 is a detailed configuration diagram of an apparatus for removing noise in a network image using a parallel feature pyramid of FIG. 1.
4 is a detailed configuration diagram of the MSCA module of FIG. 3.
5 is a flowchart of a method for removing noise in a network image using a parallel feature pyramid according to an embodiment of the present invention.
6 is a diagram showing an example of noise reduction of a conventional technique and a method proposed by the present invention for a test image.
7 is a table comparing the noise reduction performance in an image between the conventional technique and the method proposed by the present invention.

DETAILED DESCRIPTION OF THE INVENTION The detailed description of the present invention to be described later refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in detail sufficient to enable a person skilled in the art to practice the present invention. It is to be understood that the various embodiments of the present invention are different from each other, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention in relation to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description to be described below is not intended to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all scopes equivalent to those claimed by the claims. Like reference numerals in the drawings refer to the same or similar functions over several aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a block diagram of an apparatus for removing noise in a network image using a parallel feature pyramid according to an embodiment of the present invention.

The apparatus 10 for removing noise in a network image using a parallel feature pyramid according to the present invention may be a learner based on a DnCNN (DeNoise CNN) that considers information of different scales in the image.

Referring to FIG. 1, an apparatus 10 according to the present invention includes a feature map generator 100, a spatial pyramid pooling (SPP) module 300, and n multi-scale context aggregation (MSCA) It includes a module 500 and a fusion unit 700.

The device 10 of the present invention may be executed by installing software (application) for performing noise reduction in a network image using a parallel feature pyramid, and the feature map generator 100, the SPP module 300, The configurations of the n MSCA modules 500 and the fusion unit 700 may be controlled by software for performing noise reduction in a network image using the parallel feature pyramid executed in the device 10.

The device 10 may be a separate terminal or some modules of the terminal. In addition, the configuration of the feature map generation unit 100, the SPP module 300, the n MSCA modules 500, and the fusion unit 700 may be formed as an integrated module or one or more modules. . However, on the contrary, each component may be formed as a separate module.

The device 10 may be mobile or stationary. The device 10 may be in the form of a server or an engine, and may be a device, an apparatus, a terminal, a user equipment (UE), a mobile station (MS), and a wireless device. (wireless device), portable device (handheld device), etc. can be called by other terms.

The device 10 may execute or manufacture various software based on an operating system (OS), that is, a system. The operating system is a system program for allowing software to use the hardware of the device, and mobile computer operating systems such as Android OS, iOS, Windows Mobile OS, Bada OS, Symbian OS, Blackberry OS, Windows series, Linux series, Unix series, It can include all computer operating systems such as MAC, AIX, and HP-UX.

DnCNN, recently studied in the field of image noise reduction, shows good performance as an intra-image noise reduction network using CNN. DnCNN uses a single-scale feature map to remove noise in the image. However, since there are contents of various scales in an image, a feature map of a single scale is difficult to capture all the features of such an image. Therefore, the present invention focuses on containing contents of various scales in an image.

2 is a structure of an existing network, DnCNN. When an input image is given, the existing network generates feature maps of the same size as the image through a convolution operation, and each feature maps extracts only the noise component by erasing the background from the image containing noise.

However, since content components of various sizes exist in an image, it is not easy to erase only the background component from an image containing noise with this single scale.

Accordingly, the present invention focuses on the network structure including information of different scales in an image. Hereinafter, each configuration of the present invention will be described in detail.

The feature map generator 100 generates a feature map by performing a convolution operation as many as a preset number on the input image.

The SPP module 300 downsizes the feature maps to have different scales to generate n-layer pyramid feature maps, and the n MSCA modules 500 generate n-layer pyramid feature maps having different scales. Convert to a unique scale for each layer.

The fusion unit 700 integrates information of feature maps output from each MSCA module and outputs one image data.

The SPP module 300 is a technique used in the field of existing object detection, and the SPP module 300 of the present invention resizes the existing feature map to various sizes. In the present invention, the previously created feature map is downsized to a different size using the SPP module 300.

라고 하면, 상기 SPP 모듈(300)의 결과물은

의 크기를 갖는다. 여기서, n은 피라미드의 층 수에 해당하며, 피라미드의 1층은 n=0으로, 피라미드의 2층은 n=1으로 피라미드의 3층은 n=2으로 정의된다. 이렇게 만들어진 각각의 특징 맵들은 영상 내에 서로 다른 스케일의 정보를 포함하게 된다. For example, the size of the first input image

If so, the result of the SPP module 300 is

Has the size of. Here, n corresponds to the number of layers of the pyramid, the first layer of the pyramid is defined as n=0, the second layer of the pyramid is n=1, and the third layer of the pyramid is defined as n=2. Each of the feature maps created in this way includes information of a different scale in the image.

When including each scale information, the context information of the image is often helpful. Accordingly, in the present invention, the n MSCA modules 500 are created and applied to each pyramid layer, and different scale information in different layers is fused and converted into scale information suitable for the layer.

Since the inputs of the n MSCA modules 500 are feature maps of different pyramid layers, their sizes are different. Accordingly, in the n MSCA modules 500, the size of the feature map is first adjusted to fit the layer and refined by a convolution operation.

After that, the degree of abstraction a few times is slightly higher through the convolution operation, and the results are combined into one to finally produce a single result. This structure can obtain a more robust result in an image having a lot of different scale information than the conventional structure.

3 is a detailed configuration diagram of an apparatus for removing noise in a network image using a parallel feature pyramid of FIG. 1. 4 is a detailed configuration diagram of the MSCA module of FIG. 3.

Referring to FIG. 3, the feature map generator 100 is composed of a convolution operation 110 block and generates a feature map for creating a pyramid through several convolution operations.

라고 하면, 상기 SPP 모듈(300)은 특징 맵을 다른 사이즈로 다운 사이징하여

의 크기를 갖는 결과물을 출력한다.The SPP 300 generates n-layer pyramid feature maps by resizing the feature maps to have different scales. The size of the first input image

If so, the SPP module 300 downsizes the feature map to a different size

Outputs the result with the size of.

For example, when generating a three-layered pyramid feature map, the first pyramid feature map ( f ₀ ) and the first pyramid feature map ( f ₀ ) having the same scale as the input image are downsized to A second pyramid feature map ( f ₁ ) having a 1/2 scale of the input image, and a third pyramid feature map having a 1/4 scale of the input image by downsizing the second pyramid feature map ( f ₁ ). ( f ₂ ) can be created.

The n MSCA modules 500 convert the pyramid feature maps of n layers having different scales into a unique scale of each corresponding layer. Since the inputs of the n MSCA modules 500 are feature maps of different pyramid layers, their sizes are different. Accordingly, in the n MSCA modules 500, the size of the feature map is first adjusted to fit the layer and refined by a convolution operation.

For example, the n MSCA modules 500 may have three of a first MSCA module 510, a second MSCA module 530, and a third MSCA module 550, and each MSCA module The scale is changed after a convolution operation is performed on the first pyramidal feature map ( f ₀ ), the second pyramidal feature map ( f ₁ ), and the third pyramidal feature map ( f ₂ ).

Specifically, referring to FIG. 4, the second MSCA module 530 includes the first pyramid feature map ( f ₀ ), the second pyramid feature map ( f ₁ ), and the third pyramid feature map ( f ₂ ). Operation is performed through the first to third convolution operation blocks 501, 503, and 505, respectively.

Thereafter, the result of the first convolution operation block 501 is downsized through the downsizing block 502, and the result of the third convolution operation block 505 is upsized through the upsizing block 506 Thus, the scale information of the second pyramid feature map f ₁ suitable for the corresponding layer is changed. In this case, the result of the second convolution operation block 503 does not need to be downsized or upsized.

In addition, the result of the down sizing block 502, the result of the second convolution operation block 503, and the result of the up sizing block 506 are joined through a concatenation operation block 507 and then synthesized. It is refined and output through the product operation block 509.

As a result, the second MSCA module 530 outputs a patent map ( g ₁ ) having a scale of a corresponding layer, and the first MSCA module 510 and the third MSCA module 550 are also A layer corresponding to the scale transformation of the first pyramid feature map ( f ₀ ), the second pyramid feature map ( f ₁ ), and the third pyramid feature map ( f ₂ ), including the same configuration as the module 530 By converting to the scale of, a feature map ( g ₀ ) and a feature map ( g ₂ ) are output, respectively.

The fusion unit 700 slightly increases the degree of abstraction several times through a convolution operation, and finally integrates the results into one in order to produce a single result. This structure can obtain a more robust result in an image having a lot of different scale information than the conventional structure.

Specifically, the fusion unit 700 is a feature map ( f _0, g _1, g ) output from the first MSCA module 510, the second MSCA module 530, and the third MSCA module 550, respectively. ₂ ) is upsized as necessary (711 and 712), and a concatenation operation 703 is performed.

Thereafter, a convolution operation 704 is performed on the conjugated feature maps by a preset number, and an addition operation 705 is performed on the convolutional feature maps to output a final image.

Accordingly, according to the present invention, by including information of different scales in an image, noise removal performance is high, and unnecessary artifacts are reduced to provide an image of excellent quality.

5 is a flowchart of a method for removing noise in a network image using a parallel feature pyramid according to an embodiment of the present invention.

The method for removing noise in a network image using a parallel feature pyramid according to the present embodiment may be performed in substantially the same configuration as the apparatus 10 of FIG. 3. Accordingly, the same components as those of the device 10 of FIG. 3 are given the same reference numerals, and repeated descriptions are omitted.

Further, the method for removing noise in a network image using a parallel feature pyramid according to the present embodiment may be executed by software (application) for performing noise removal in a network image using a parallel feature pyramid.

Referring to FIG. 5, in the method for removing noise in a network image using a parallel feature pyramid according to the present embodiment, when an input image for starting learning is given, a predetermined number of convolutions are calculated on the input image. To generate a feature map (step S10).

When the feature map is generated, the feature map is downsized to have different scales through a pyramid (Spatial pyramid pooling) module to generate n-layer pyramid feature maps (step S20).

Pyramid feature maps of n layers having different scales are converted into unique scales of each corresponding layer through n multi-scale context aggregation (MSCA) modules (step S30).

인 경우,

(여기서, n은 피라미드의 층 수이며, 1층은 n=0으로 정의됨)의 크기를 갖는 특징 맵들을 생성할 수 있다.As a result, the size of the input image

If is,

(Where n is the number of layers of the pyramid, and the first layer is defined as n=0), feature maps having a size can be generated.

In step S30, each MSCA module performs a convolution operation of each of the pyramid feature maps of n layers having different scales, converting the convolutional feature maps to a unique scale of each corresponding layer, and converting the convolutional feature maps to a unique scale of each corresponding layer. It may include performing a concatenation operation of the transformed feature maps, and refining the conjugated feature maps by performing a convolution operation.

The information on the feature maps output from each MSCA module is combined and output as one image data (step S40).

Step S40 is a step of converting a feature map output from each of the MSCA modules to a reference scale, performing a concatenation operation of feature maps converted to the reference scale, and performing a concatenation operation of the feature maps by a preset number. It may include performing a convolution operation and performing an addition operation and outputting the convolutional feature map.

When network learning is completed in the above process, an image of excellent quality may be output using the learning result for the real-time input image. In the following, the experimental results for evaluating the performance of the present invention will be described.

6 is a diagram showing an example of noise reduction of a conventional technique and a method proposed by the present invention for a test image. In FIG. 6, (a) an image without noise, (b) an image with noise, (c) a noise reduction result of DnCNN, and (d) a noise reduction result of the network proposed by the present invention.

Referring to FIG. 6, an example of noise reduction of the existing technique and the proposed technique for a test image of the berkeley segmentation dataset (BSD) is shown. In the case of DnCNN, which is an existing technique, unnecessary artifacts are generated in addition to noise removal, such as an enlarged area.

On the other hand, in the result of the method proposed in the present invention, it can be confirmed that there are few unnecessary artifacts. The PSNR result of FIG. 3 is 14.99dB in the case of (b), 28.38 in the case of (c), and 28.88 in the case of (d), and it can be seen that the PSNR of the method proposed in the present invention is also higher.

7 is a table comparing the noise reduction performance in an image between the conventional technique and the method proposed by the present invention.

Various levels of noise were added in the image for the experiment, and the performance of the method proposed by the present invention was the best at all levels of noise. The performance evaluation was measured through the PSNR, which is officially used in the field of noise reduction in the image, and the performance was verified through the BSD 68 set, which is widely used as a data set in existing detection fields including data sets.

Such a method of removing noise in a network image using a parallel feature pyramid may be implemented as an application or implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, and may be known and usable to those skilled in the computer software field.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks. media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

Although the above has been described with reference to embodiments, it is understood that those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the following claims. You can understand.

The present invention not only has high noise removal performance in consideration of various scales in an image, but also reduces unnecessary artifacts to provide an image of excellent quality.

10: Device for removing noise in network image using parallel feature pyramid
100: feature map generator
300: SPP module
500: MSCA module
700: fusion unit

Claims (13)

Generating a feature map by performing a predetermined number of convolution operations on the input image;
Generating n-layer pyramid feature maps by downsizing the feature maps to have different scales through a pyramid (Spatial pyramid pooling) module;
converting the pyramid feature maps of n layers having different scales into a unique scale of each corresponding layer through n multi-scale context aggregation (MSCA) modules; And
Including; integrating information of feature maps output from each MSCA module and outputting as one image data; Including,
The step of integrating information of feature maps output from each MSCA module and outputting it as one image data,
Converting a feature map output from each MSCA module into a reference scale;
Performing a concatenation operation of feature maps converted to the reference scale;
Performing a convolution operation on the joint-operated feature map by a preset number; And
Comprising; adding and outputting the convolutional feature map. A method of removing noise in a network image using a parallel feature pyramid.
The method of claim 1, wherein converting the n-layer pyramid feature maps to a unique scale of each corresponding layer,
Performing a convolution operation of each of the n-layer pyramid feature maps having different scales in each MSCA module;
Converting each of the convolutional feature maps into a unique scale of each corresponding layer;
Performing a concatenation operation of the feature maps converted to the intrinsic scale; And
Comprising, Comprising; Comprising the convolutional computation of the spliced feature map to remove noise in a network image using a parallel feature pyramid.
delete The method of claim 1, wherein converting the n-layer pyramid feature maps to a unique scale of each corresponding layer,
The size of the input image

If is,

(Here, n is the number of pyramidal layers, and the first layer is defined as n=0) A noise removal method in a network image using a parallel feature pyramid to generate feature maps.
The method of claim 1, wherein generating the n-layer pyramid feature maps comprises:
Generating a first pyramid feature map having the same scale as the input image;
Downsizing the first pyramid feature map to generate a second pyramid feature map having a 1/2 scale of the input image; And
Generating a third pyramidal feature map having a 1/4 scale of the input image by downsizing the second pyramidal feature map. Including, Method for removing noise in a network image using a parallel feature pyramid.
The method of claim 5, wherein converting the n-layer pyramid feature maps to a unique scale of each corresponding layer,
Downsizing after a convolution operation of the first pyramid feature map;
Performing a convolution operation of the second pyramidal feature map; And
Comprising, Upsizing after a convolution operation of the third pyramidal feature map. A method of removing noise in a network image using a parallel feature pyramid.
The method of claim 6, wherein converting the n-layer pyramid feature maps to a unique scale of each corresponding layer,
Performing a concatenation operation of the downsized first pyramid feature map, the second pyramid feature map, and the upsized third pyramid feature map; And
Comprising, Comprising; Comprising the convolutional computation of the spliced feature map to remove noise in a network image using a parallel feature pyramid.
The method of claim 7, wherein the step of integrating information of feature maps output from each MSCA module and outputting it as one image data,
Upsizing a feature map output from each of the MSCA modules;
Performing a concatenation operation of the upsized feature maps;
Performing a convolution operation on the joint-operated feature map by a preset number; And
Comprising; adding and outputting the convolutional feature map. A method of removing noise in a network image using a parallel feature pyramid.
A computer-readable storage medium in which a computer program for performing the method for removing noise in a network image using the parallel feature pyramid according to any one of claims 1 to 2 and 4 to 8 is recorded.
A feature map generator configured to generate a feature map by performing convolution operations as many as a preset number on the input image;
A pyramid (Spatial pyramid pooling) module for generating n-layer pyramid feature maps by downsizing the feature maps to have different scales;
N multi-scale context aggregation (MSCA) modules for converting pyramid feature maps of n layers having different scales into unique scales of each corresponding layer; And
Containing; a fusion unit for integrating the information of the feature maps output from each MSCA module and outputting as one image data,
The fusion unit,
An apparatus for removing noise in a network image using a parallel feature pyramid for upsizing the feature maps output from each MSCA module and converting them to a reference scale, performing concatenation and convolution of feature maps, and then performing addition operation. .
The method of claim 10, wherein each MSCA module of the n MSCAs,
Network image using a parallel feature pyramid in which each of the pyramid feature maps of n layers having different scales is convoluted, and each of the convolutional feature maps is down-sized or up-sized to convert to a unique scale of each corresponding layer My noise canceling device.
The method of claim 10, wherein each MSCA module of the n MSCAs,
An apparatus for removing noise in a network image using a parallel feature pyramid, which refines the feature maps converted to the intrinsic scale of each corresponding layer by performing concatenation and convolution.
delete

Images