KR102325898B1 - System and method for multi-domain image restoration - Google Patents

Abstract

The disclosed embodiments are intended to provide technical means for performing image restoration on a plurality of domains selectable by a user, rather than on one specific domain. Disclosed are a system and method for multi-domain image restoration. According to one embodiment of the present invention, a system for multi-domain image restoration comprises: a domain-specific image restoration device comprising a plurality of image restoration modules receiving an image and performing restoration on different areas or domains within the image; and an image blending device for generating a final restored image by synthesizing a plurality of primary restored images received from the domain-specific image restoration device.

Description

Multi-domain image restoration system and method {SYSTEM AND METHOD FOR MULTI-DOMAIN IMAGE RESTORATION}

Disclosed embodiments relate to techniques for restoring or improving quality of images including still images and moving images.

Image restoration refers to restoring a damaged image or improving image quality, such as removing noise from an image and increasing the resolution. The image restoration technology may be subdivided into super resolution, deblurring, denoising, inpainting, and the like. Super resolution is a method of recovering and generating a high-resolution image from a low-resolution image. Deblurring is an image blur caused by object motion blur, camera shake blur, defocus blur, or complex elements in the image. way to remove it. Denoising is a method of reconstructing an actual image by separating noise, edges, and texture, which are high-frequency components of an image, and removing only the noise. Inpainting is a method of reconstructing an image by naturally filling in a damaged or blank area in an input image.

Existing image restoration techniques (super-resolution, deblurring, denoising, inpainting, etc.) were developed with a focus on improving overall image performance. Therefore, in the case of conventional techniques, overall image restoration is well performed, but there is a limit to restoration of a detailed object region. Accordingly, an image restoration method specialized for a specific domain, such as a face or a license plate, has been proposed. However, the method can only achieve high performance within a specific domain, and it is difficult to achieve that much performance in other domains. In particular, when a plurality of domains are included in one image, it is very difficult to evenly improve the image quality for each domain using the conventional image restoration technique.

The disclosed embodiments are intended to provide technical means for performing image restoration on a plurality of domains selectable by a user, rather than on one specific domain.

According to an exemplary embodiment, there is provided an apparatus comprising: a domain-specific image restoration apparatus including a plurality of image restoration modules that receive an image and perform restoration on different regions or domains in the image; and an image blending apparatus for generating a final restored image by synthesizing a plurality of primary restored images received from the domain-specific image restoration apparatus is provided.

The image blending apparatus may include: an image quality measurement module for generating a plurality of image quality maps corresponding to each of the plurality of primary reconstructed images; a superpixel map generating module for generating a superpixel map from a selected one of the plurality of primary reconstructed images; and generating a plurality of image quality superpixel maps using the plurality of image quality maps and the superpixel maps, and based on the image quality score for each superpixel of the plurality of image quality superpixel maps, the plurality of primary and an image synthesizing module for generating the final reconstructed image by synthesizing the reconstructed image.

The image quality map may include a quality score for each pixel of the primary reconstructed image.

The superpixel map generating module may generate the superpixel map by using a primary reconstructed image corresponding to an image quality map having the largest average value of the quality scores for each of the plurality of image quality maps.

The plurality of image quality superpixel maps may be generated by mapping, for each of the plurality of image quality maps, a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map.

The plurality of image quality superpixel maps are generated by mapping, with respect to each of the plurality of image quality maps, a value obtained by rounding a representative value of the image quality score of the corresponding region for each region corresponding to each superpixel of the superpixel map. can be

The image synthesizing module may generate the final reconstructed image by summing the element products for each pixel of the plurality of primary reconstructed images and the image quality superpixel map corresponding thereto.

According to another exemplary embodiment, an image quality measurement module for generating a plurality of image quality maps corresponding to each of a plurality of primary restored images generated by a plurality of domain-specific image restoration apparatuses; a superpixel map generating module for generating a superpixel map from a selected one of the plurality of primary reconstructed images; and generating a plurality of image quality superpixel maps using the plurality of image quality maps and the superpixel maps, and based on the image quality score for each superpixel of the plurality of image quality superpixel maps, the plurality of primary An image blending apparatus is provided, including an image synthesizing module for generating the final reconstructed image by synthesizing the reconstructed image.

The image quality map may include a quality score for each pixel of the primary reconstructed image.

The superpixel map generating module may generate the superpixel map by using a primary reconstructed image corresponding to an image quality map having the largest average value of the quality scores for each of the plurality of image quality maps.

The plurality of image quality superpixel maps may be generated by mapping, for each of the plurality of image quality maps, a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map.

The plurality of image quality superpixel maps are generated by mapping, with respect to each of the plurality of image quality maps, a value obtained by rounding a representative value of the image quality score of the corresponding region for each region corresponding to each superpixel of the superpixel map. can be

The image synthesizing module may generate the final reconstructed image by summing the element products for each pixel of the plurality of primary reconstructed images and the image quality superpixel map corresponding thereto.

According to another exemplary embodiment, there is provided a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, receiving an image, and generating a plurality of primary restored images by performing restoration on different regions or domains, respectively; and an image blending step of synthesizing the plurality of primary reconstructed images to generate a final reconstructed image.

The image blending step may include: an image quality measuring step of generating a plurality of image quality maps corresponding to each of the plurality of primary reconstructed images; a superpixel map generating step of generating a superpixel map from a selected one of the plurality of primary reconstructed images; and generating a plurality of image quality superpixel maps using the plurality of image quality maps and the superpixel maps, and based on the image quality score for each superpixel of the plurality of image quality superpixel maps, the plurality of primary and synthesizing the reconstructed image to generate the final reconstructed image.

The image quality map may include a quality score for each pixel of the primary reconstructed image.

The generating of the superpixel map may include generating the superpixel map using a primary reconstructed image corresponding to an image quality map having the largest average value of the quality scores for each of the plurality of image quality maps.

The plurality of image quality superpixel maps may be generated by mapping, for each of the plurality of image quality maps, a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map.

The plurality of image quality superpixel maps are generated by mapping, with respect to each of the plurality of image quality maps, a value obtained by rounding a representative value of the image quality score of the corresponding region for each region corresponding to each superpixel of the superpixel map. can be

The image synthesizing step may generate the final reconstructed image by summing the element products for each pixel of the plurality of primary reconstructed images and the image quality superpixel map corresponding thereto.

According to the disclosed embodiment, there is an advantage in that image restoration can be performed with higher accuracy on a plurality of domains selectable by a user rather than on one specific domain.

1 is a block diagram illustrating a multi-domain image restoration system 100 according to an embodiment.
2 is a block diagram for explaining in more detail the image blending apparatus 104 of the multi-domain image restoration system 100 according to an embodiment.
3 is an exemplary diagram of a plurality of primary restored images generated by the domain-specific image restoration apparatus 102 according to an embodiment;
4 is an exemplary diagram of a plurality of image quality maps generated from the plurality of primary reconstructed images shown in FIG. 3 ;
5 is an exemplary diagram of a superpixel map according to an embodiment;
6 is an exemplary diagram of a plurality of image quality superpixel maps according to an embodiment;
7 is an exemplary diagram of a plurality of image quality superpixel maps according to another embodiment;
8 is an exemplary view of a final reconstructed image generated based on the image quality superpixel map of FIGS. 6 and 7 ;
9 is a flowchart illustrating a multi-domain image restoration method 900 according to an embodiment.
10 is a flowchart illustrating in more detail an image blending process 904 of a multi-domain image restoration method 900 according to an embodiment.
11 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and/or systems described herein. However, this is merely an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, if it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. And, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the content throughout this specification. The terminology used in the detailed description is for the purpose of describing embodiments of the present invention only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. In this description, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, acts, elements, some or a combination thereof, one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

1 is a block diagram illustrating a multi-domain image restoration system 100 according to an embodiment. As shown, the multi-domain image restoration system 100 according to an embodiment includes a domain-specific image restoration apparatus 102 and an image blending apparatus 104 .

The domain-specific image restoration apparatus 102 receives an original image and generates a plurality of primary restored images from the input original image. To this end, the domain-specific image restoration apparatus 102 includes a plurality of image restoration modules. Each of the plurality of image restoration modules is specialized in image restoration for different regions or domains in the original image. In some embodiments, at least some of the plurality of image restoration modules may be trained to perform image restoration on a region selected by a user. In other embodiments, at least some of the plurality of image restoration modules may be specialized in image restoration for a specific object (eg, a license plate or a human face) found in the original image. In still other embodiments, at least some of the plurality of image reconstruction modules perform super resolution, deblurring, denoising, and inpainting of all or part of an original image. can be learned to perform at least one of Note that in the disclosed embodiments, the domain-specific image restoration apparatus 102 may include, without limitation, various types of image restoration modules as necessary, and is not limited to a specific type of image restoration module.

In an embodiment, the user of the multi-domain image restoration system 100 may perform image restoration on the original image among I image restoration modules (I is an integer greater than or equal to 1) included in the domain-specific image restoration apparatus 102 M. M number of image restoration modules (1≤M≤I) may be selected, and M primary restored images may be generated from the selected M image restoration modules.

Next, the image blending apparatus 104 generates one final restored image by synthesizing a plurality of primary restored images received from the domain-specific image restoration apparatus 102 .

2 is a block diagram for explaining in more detail the image blending apparatus 104 of the multi-domain image restoration system 100 according to an embodiment. As shown, the image blending apparatus 104 according to an embodiment includes an image quality measurement module 202 , a superpixel map generation module 204 , and an image synthesis module 206 .

The image quality measurement module 202 generates a plurality of image quality maps corresponding to each of the plurality of primary restored images generated by the domain-specific image restoration apparatus 102 . In this case, the image quality map includes a quality score for each pixel of the corresponding primary reconstructed image. This will be described in more detail as follows.

Let the M plurality of primary reconstructed images generated by the domain-specific image reconstructing apparatus 102 be P ₁ , P ₂ , ??, and P _M , respectively. In this case, the size of the k-th primary reconstructed image P _k (1≤k≤M) may be defined as _{(C p,k} * H _p,k * W _{p,k ).} In this case, C _p,k denotes the number of channels _{of P k , H p,k} denotes the vertical image size _{of P k , and W p,k} denotes the horizontal image size _{of P k .}

In an embodiment, all of the M plurality of primary reconstructed images may have the same size. That is, each primary reconstructed image may have the following relationship.

C _p,1 = C _p,2 = ... = C _p,M

H _p,1 = H _p,2 = ... = H _p,M

W _p,1 = W _p,2 = ... = W _p,M

Any pixel of the k-th primary reconstructed image may be referred to as _{p k,c,h,w.} p _k,c,h,w denotes a pixel on the c-th channel, vertical position h, and horizontal position _{w of the k-th primary reconstructed image (1≤c≤C p,k} , 1≤h≤H _{p ,k} , 1≤w≤W _p,k , 1≤k≤M)

3 is an exemplary diagram of a plurality of primary restored images generated by the domain-specific image restoration apparatus 102 according to an embodiment. The illustrated example shows an example of generating two primary restored images indicated by (a) and (b) from the domain-specific image restoration apparatus 102 .

The image quality measurement module 202 generates M image quality maps by using the M primary reconstructed images. Let the image quality maps corresponding to the primary reconstructed images P ₁ , P ₂ , ..., P _{M be Q 1} , Q ₂ , ..., Q _M . In this case, the size of the k-th image quality map Q _k (1≤k≤M) may be defined as _{(C q,k} * H _q,k * W _{q,k ).} In this case, C _q,k denotes the number of channels _{of Q k , H q,k} denotes the vertical image size _{of Q k , and W q,k} denotes the horizontal image size _{of Q k , respectively.}

In an embodiment, the size of the image quality map may be set to be the same as the size of the corresponding primary reconstructed image. This can be expressed as follows.

C _p,k = C _q,k , H _p,k = H _q,k , W _p,k = W _q,k (1≤k≤M)

Any pixel of the k-th image quality map may be referred to as _{q k,c,h,w.} q _k,c,h,w means pixels on the c-th channel, vertical position h, and horizontal position _{w of the k-th image quality map (1≤c≤C q,k} , 1≤h≤H _{q, k} , 1≤w≤W _q,k , 1≤k≤M)

Each pixel of the image quality map q _1,c,h,w , q _2,c,h,w , ..., q _M,c,h,w is the corresponding point of the primary reconstructed image, that is, q _{1,c ,h,w} , q _2,c,h,w , ..., q Represents the image quality score of _M,c,h,w. That is, q _k,c,h,w means the image quality score of p _k,c,h,w. In the disclosed embodiments, the image quality score means the degree to which a corresponding pixel of the primary restored image contributes to the final restored image R. That is, the image quality score is information on which pixel should be selected in order to increase the quality of the final reconstructed image R among M pixels existing on the same position (c, h, w) of the M primary reconstructed images. indicates In other words, the higher the image quality score, the higher the probability or the inclusion ratio of the pixel to be included in the final reconstructed image R. In this case, the image quality score for each pixel included in the image quality map may have the following relationship.

In an embodiment, the image quality measurement module 202 may input a plurality of primary reconstructed images using the learned artificial neural network and generate an image quality map including an image quality score for each pixel therefrom.

4 is an exemplary diagram of a plurality of image quality maps generated from the plurality of primary reconstructed images shown in FIG. 3 . Specifically, (a) of FIG. 4 is an image quality map generated from the first reconstructed image corresponding to (a) of FIG. 3 , and (b) of FIG. 4 is the first reconstructed image corresponding to (b) of FIG. 3 . Each of the image quality maps generated from . The brighter the color of each pixel in the image quality map, the higher the image quality score of the corresponding position.

Next, the superpixel map generating module 204 generates a superpixel map from a selected one of the plurality of primary reconstructed images. In an embodiment, the superpixel map generating module 204 may generate a superpixel map using a primary reconstructed image corresponding to an image quality map having the largest average value of the quality scores for each of the plurality of image quality maps. .

In the disclosed embodiments, a superpixel means one large pixel by grouping pixels having similar visual characteristics among adjacent pixels. The superpixel map generation module 204 may configure a superpixel using various algorithms, such as a SEEDS or SLIC algorithm.

The superpixel map S is s ₁ , s ₂ , … , may be composed of N superpixels including _{s N .} In this case, the size of N may be set to an appropriate number in consideration of characteristics of the original image. In addition, the size of the superpixel map _{may be defined as (C S} * H _S * W _S ), which may be determined to be the same as the size of the primary reconstructed image and the image quality map. 5 is an exemplary diagram of a superpixel map according to an embodiment.

Next, the image synthesis module 206 generates a plurality of image quality superpixel maps by using the plurality of image quality maps and superpixel maps, and adds the image quality score for each superpixel of the plurality of image quality superpixel maps. Based on the synthesizing of the plurality of primary reconstructed images, the final reconstructed image is generated. This will be described in more detail as follows.

First, the image synthesis module 206 generates a plurality of image quality superpixel maps by using the plurality of image quality maps and superpixel maps. Specifically, the image synthesis module 206 maps, for each of the plurality of image quality maps, a representative value (average value or median value, etc.) of the image quality score of the corresponding region for each region corresponding to each superpixel of the superpixel map. The image quality superpixel map may be generated. At this time, the image synthesis module 206 Q ₁ , Q ₂ , ... , from M image quality maps corresponding to _{Q M , M superpixel maps T 1} , T ₂ , … , T _M can be created.

n≤N) 과 동일한 위치에 있는 퀄리티 스코어를 각각 q_k,n 이라 하자. 이 경우 해당 슈퍼필셀에 대응되는 영상 퀄리티 슈퍼픽셀 맵의 퀄리티 스코어의 대표값은 다음과 같이 계산될 수 있다.For the kth image quality map Q _{k , superpixel s n} (1) belonging to the superpixel map S

Let the quality scores in the same position as n≤N) be q _{k,n, respectively.} In this case, the representative value of the quality score of the image quality superpixel map corresponding to the corresponding superpixel may be calculated as follows.

In an embodiment, the image synthesis module 206 may generate an image quality superpixel map by using _{the t k,n as it is (soft mapping).} In another embodiment, the image synthesizing module 206 may _{generate an image quality superpixel map using the rounded value of t k,n} as follows (ie, 0 or 1) (hard mapping).

6 is an exemplary diagram of a plurality of image quality superpixel maps according to an exemplary embodiment, and FIG. 7 is an exemplary diagram of a plurality of image quality superpixel maps according to another exemplary embodiment. 6 is an image quality superpixel map generated by the soft mapping method described above from the image quality map shown in FIGS. 4A and 4B, and FIG. Each of the image quality superpixel maps generated by the above-described hard mapping method from the illustrated image quality map is indicated.

Thereafter, the image synthesis module 206 generates the final reconstructed image by synthesizing the plurality of primary reconstructed images based on the image quality score for each superpixel of the plurality of image quality superpixel maps. Specifically, the image synthesis module 206 may generate the final reconstructed image by summing the element products for each pixel of the plurality of primary reconstructed images and the image quality superpixel map corresponding thereto.

The size of the final reconstructed image R _{is defined as (C r} * H _r * W _r ), which is the same as the size of the primary reconstructed image and the image quality map. In this case, the final reconstructed image may be generated by the following equation.

8 is an exemplary diagram of a final reconstructed image generated based on the image quality superpixel map of FIGS. 6 and 7 . (a) of FIG. 8 shows a final reconstructed image generated based on the image quality superpixel map of FIG. 6, and (b) shows a final reconstructed image generated based on the image quality superpixel map of FIG. 7, respectively.

9 is a flowchart illustrating a multi-domain image restoration method 900 according to an embodiment. The illustrated method may be performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, for example, the multi-domain image restoration system 100 described above. In the illustrated flowchart, the method or process is described by dividing the method or process into a plurality of steps, but at least some of the steps are performed in a reversed order, are performed in combination with other steps, are omitted, are performed in separate steps, or are shown. One or more steps not included may be added and performed.

In step 902, the domain-specific image restoration apparatus 102 of the multi-domain image restoration system 100 receives an image, and performs restoration on different regions or domains in the image to obtain a plurality of primary restored images. create

In operation 904, the image blending apparatus 104 generates a final restored image by synthesizing the plurality of primary restored images.

10 is a flowchart illustrating in more detail the image blending process 904 of the multi-domain image restoration method 900 according to an embodiment. In the illustrated flowchart, the method or process is described by dividing the method or process into a plurality of steps, but at least some of the steps are performed in a reversed order, are performed in combination with other steps, are omitted, are performed in separate steps, or are shown. One or more steps not included may be added and performed.

In operation 1002, the image quality measurement module 202 generates a plurality of image quality maps corresponding to each of the plurality of primary reconstructed images.

In operation 1004 , the superpixel map generating module 204 generates a superpixel map from a selected one of the plurality of primary reconstructed images.

In step 1006, the image synthesis module 206 generates a plurality of image quality superpixel maps by using the plurality of image quality maps and the superpixel maps.

In step 1008, the image synthesis module 206 generates the final reconstructed image by synthesizing the plurality of primary reconstructed images based on the image quality score for each superpixel of the plurality of image quality superpixel maps.

11 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, the computing device 12 may be the multi-domain image restoration system 100 according to embodiments of the present invention. Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium accessed by computing device 12 and capable of storing desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 102 as a separate device distinct from the computing device 12 . may be

Meanwhile, an embodiment of the present invention may include a program for performing the methods described in this specification on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include program instructions, local data files, local data structures, etc. alone or in combination. The medium may be specially designed and configured for the present invention, or may be commonly used in the field of computer software. Examples of computer-readable recording media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and program instructions specially configured to store and execute program instructions such as ROMs, RAMs, flash memories, and the like. Hardware devices are included. Examples of the program may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those generated by a compiler.

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art will understand that various modifications are possible without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by the claims described below as well as the claims and equivalents.

Meanwhile, an embodiment of the present invention may include a program for performing the methods described in this specification on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include program instructions, local data files, local data structures, etc. alone or in combination. The medium may be specially designed and configured for the present invention, or may be commonly used in the field of computer software. Examples of computer-readable recording media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and program instructions specially configured to store and execute program instructions such as ROMs, RAMs, flash memories, and the like. Hardware devices are included. Examples of the program may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those generated by a compiler.

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art will understand that various modifications are possible without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by the claims described below as well as the claims and equivalents.

100: multi-domain image restoration system
102: domain-specific image restoration device
104: image blending device
202: image quality measurement module
204: Superpixel map generation module
206: image synthesis module

Claims (20)

a domain-specific image restoration apparatus including a plurality of image restoration modules receiving an image and performing restoration on different regions or domains in the image; and
and an image blending apparatus for generating a final restored image by synthesizing a plurality of primary restored images received from the domain-specific image restoration apparatus,
The image blending device,
an image quality measurement module for generating a plurality of image quality maps corresponding to each of the plurality of primary reconstructed images;
a superpixel map generating module for generating a superpixel map from a selected one of the plurality of primary reconstructed images; and
A plurality of image quality superpixel maps are generated using the plurality of image quality maps and the superpixel maps, and the image quality score for each superpixel of the plurality of primary restored images and the image quality superpixel map corresponding thereto is added. and an image synthesizing module configured to generate the final reconstructed image by synthesizing the plurality of primary reconstructed images by summing the factor products for each.
delete The method according to claim 1,
The image quality map is
A multi-domain image restoration system, including a quality score for each pixel of the primary restored image.
4. The method according to claim 3,
The superpixel map generating module is configured to generate the superpixel map by using a primary reconstructed image corresponding to an image quality map having the largest average value of the quality score for each of the plurality of image quality maps.
4. The method according to claim 3,
The plurality of image quality superpixel maps, for each of the plurality of image quality maps, are generated by mapping a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map. video restoration system.
4. The method according to claim 3,
The plurality of image quality superpixel maps are generated by mapping, with respect to each of the plurality of image quality maps, a value obtained by rounding a representative value of the image quality score of the corresponding region for each region corresponding to each superpixel of the superpixel map. Being a multi-domain image restoration system.
delete an image quality measuring module for generating a plurality of image quality maps corresponding to each of a plurality of primary restored images generated by a plurality of domain-specific image restoration apparatuses;
a superpixel map generating module for generating a superpixel map from a selected one of the plurality of primary reconstructed images; and
A plurality of image quality superpixel maps are generated using the plurality of image quality maps and the superpixel map, and the image quality score for each superpixel of the plurality of primary restored images and the image quality superpixel map corresponding thereto is obtained. and an image synthesizing module for generating a final reconstructed image by synthesizing the plurality of primary reconstructed images by summing the factor products for each.
9. The method of claim 8,
The image quality map is
An image blending apparatus including a quality score for each pixel of the primary reconstructed image.
10. The method of claim 9,
The superpixel map generating module is configured to generate the superpixel map by using a primary reconstructed image corresponding to an image quality map having the largest average value of the quality score for each of the plurality of image quality maps.
10. The method of claim 9,
The plurality of image quality superpixel maps are generated by mapping a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map with respect to each of the plurality of image quality maps. Device.
10. The method of claim 9,
The plurality of image quality superpixel maps are generated by mapping, with respect to each of the plurality of image quality maps, a value obtained by rounding a representative value of the image quality score of the corresponding region for each region corresponding to each superpixel of the superpixel map. A video blending device.
delete one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
generating a plurality of primary restored images by receiving an image and performing restoration on different regions or domains in the image; and
An image blending step of synthesizing the plurality of primary reconstructed images to generate a final reconstructed image,
The image blending step is
an image quality measurement step of generating a plurality of image quality maps corresponding to each of the plurality of primary reconstructed images;
a superpixel map generating step of generating a superpixel map from a selected one of the plurality of primary reconstructed images; and
A plurality of image quality superpixel maps are generated using the plurality of image quality maps and the superpixel map, and the image quality score for each superpixel of the plurality of primary restored images and the image quality superpixel map corresponding thereto is obtained. and an image synthesizing step of generating the final reconstructed image by synthesizing the plurality of primary reconstructed images by summing the factor products for each.
delete 15. The method of claim 14,
The image quality map is
A multi-domain image restoration method, including a quality score for each pixel of the primary restored image.
17. The method of claim 16,
The step of generating the superpixel map includes:
A multi-domain image restoration method of generating the superpixel map by using a primary restored image corresponding to an image quality map having the largest average value of the quality score for each of the plurality of image quality maps.
17. The method of claim 16,
The plurality of image quality superpixel maps,
For each of the plurality of image quality maps, the multi-domain image restoration method is generated by mapping a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map.
17. The method of claim 16,
The plurality of image quality superpixel maps,
A multi-domain image restoration method generated by mapping a value obtained by rounding a representative value of an image quality score of a corresponding region for each region corresponding to each superpixel of the superpixel map with respect to each of the plurality of image quality maps.
delete

Images