KR20230090485A - Image restoration method using segmentation loss - Google Patents

Abstract

The present invention relates to an image restoration method using segmentation loss. The image restoration method according to various embodiments may include an operation of restoring an image to its original state based on segmentation information. Therefore, the image can be restored like the original.

Description

Image restoration method using segmentation loss {IMAGE RESTORATION METHOD USING SEGMENTATION LOSS}

The disclosure below relates to an image reconstruction method using segmentation loss.

The background art described above is possessed or acquired by the inventor in the process of deriving the disclosure of the present application, and cannot necessarily be said to be known art disclosed to the general public prior to the present application.

Various embodiments may provide a technique of restoring an image to an original image based on segmentation information.

However, the technical challenges are not limited to the above-described technical challenges, and other technical challenges may exist.

An image restoration method according to various embodiments may include an operation of restoring an image to an original image based on segmentation information.

1 to 3 are diagrams for explaining an image restoration method according to various embodiments.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be changed and implemented in various forms. Therefore, the form actually implemented is not limited only to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

□ Contents and technical characteristics of the present invention

The present invention relates to a technique for restoring an image to an original image based on segmentation information. The present invention 1) reconstructs an image with an encoder-decoder model, extracts segmentation information of the reconstructed image, adds segmentation information to the reconstructed image based on this, learns, and 2) makes the reconstructed image better able to use the segmentation information. It consists of two key skills that help you reflect.

The technique proposed by the present invention is differentiated from the existing system for restoring an image using only an encoder-decoder, in that it additionally uses the semantic segmentation information of the image to restore the image, and shows higher improvement performance through this technique. There is a difference in

1) Extraction of segmentation information of the restored image

The most important problem with the restored image is that the information necessary for image restoration is very lacking. So it is important to solve this. Image restoration is assisted by using the fact that the restored image must have the same semantic segmentation as the original image. 1) After restoring the original image using the encoder-decoder model, semantic segmentation information is extracted from the restored image through the learned semantic segmentation model. The obtained semantic segmentation information is reflected in model learning through Equation (1). 2) A semantic segmentation discriminator is added to help restore the semantic segmentation result of an image more realistically. Corresponding information is transmitted to the model through Equation (2).

[Equation 1]

[Equation 2]

Figure 2 shows the overall model structure. The latest model, Dense Prediction Transformer (DPT), was used for the semantic segmentation process of the reconstructed image, but other models may be used.

2) Module for reflecting segmentation results to images

In this study, the implemented model was designed by adding a loss to reflect the characteristic that it generally has similar texture and color when belonging to the same segmentation label in the learning process. Through Equation (3), the corresponding information is reflected in the learning process of the model.

[Equation 3]

The image restoration model developed through the present invention showed better image restoration performance than the model using only the encoder-decoder. A performance comparison between these systems can be seen in Figure 4.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. A computer readable medium may store program instructions, data files, data structures, etc. alone or in combination, and program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in the art of computer software. there is. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (1)

An operation of restoring an image to an original image based on segmentation split information
Including, image restoration method.

Images