KR20200071866A - Method and apparatus for machine-learning based processing of high-resolution image - Google Patents

Abstract

The present invention relates to a method and a device for processing a high-resolution image by using machine learning. According to one embodiment of the present invention, the device for processing the high-resolution image by using machine learning comprises: a downsampling part generating a low-resolution image by reducing a spatial resolution of a high-resolution input image; a machine learning processing part calculating a low-resolution output image by using the generated low-resolution image as input of a machine learning network; and an image conversion part converting the high-resolution input image and the calculated low-resolution output image according to the characteristics of the calculated low-resolution output image to output a high-resolution result image. Therefore, a computational complexity of processing high-resolution images through machine learning can be reduced.

Description

METHOD AND APPARATUS FOR MACHINE-LEARNING BASED PROCESSING OF HIGH-RESOLUTION IMAGE}

The present invention relates to a method and apparatus for processing high-resolution images using machine learning.

Machine learning techniques can be used to apply techniques such as changing image style and improving image quality through a series of processing on a given input image. The machine learning technology outputs a new type of image from the input image using the principles of a convolutional neural network (CNN), an auto encoder, and a generative adversarial network (GAN).

In such a machine learning-based image processing process, in the related art, an image is cut into blocks, and then each block is input to a machine learning network to combine output block images. This is to process an image having a size higher than High Definition (HD) due to the high spatial and computational complexity of the machine learning network used. In this process, the machine learning process must be repeated as many as the number of blocks calculated from the image, and there is a problem in that the computational complexity increases due to the process of combining the blocks. In addition, there is a problem of image quality deterioration in which a block boundary observed in a result image due to block-based image processing is visible.

As described above, due to the high computational complexity of the machine learning network itself in the image processing technique using the machine learning technique, a high specification/expensive image processing system must be established or a long computation time must be performed in order to handle high-resolution images of HD level or higher. did. Otherwise, standard definition (SD)-class low-resolution images had to be processed.

Embodiments of the present invention are to provide a method and apparatus for processing a high resolution image using machine learning, which reduces computational complexity in a process of processing a high resolution image through machine learning.

Embodiments of the present invention are to provide a method and apparatus for processing a high resolution image using machine learning, which calculates a high resolution result image using a relationship between a high resolution input image and an image calculated through machine learning.

Embodiments of the present invention are to provide a method and apparatus for processing a high-resolution image using machine learning, which generates a high-resolution output image through a series of downsampling, machine learning processing, and image restoration processes using high-resolution images as input.

According to an embodiment of the present invention, a down-sampling unit for generating a low-resolution image by reducing the spatial resolution of the high-resolution input image; A machine learning processor configured to calculate the low-resolution output image by using the generated low-resolution image as an input to the machine learning network; And an image converting unit for converting the high-resolution input image and the calculated low-resolution output image according to the characteristics of the calculated low-resolution output image and outputting a high-resolution result image. I want to.

According to embodiments of the present invention, through a process of processing a high-resolution image using machine learning, the machine learning process of an input image of a high-definition or higher-definition image is performed only once, or a high-resolution image is output only by performing a few times depending on the application field. This makes it possible to reduce computational complexity.

1 is a view for explaining the configuration of a high-resolution image processing apparatus using machine learning according to an embodiment of the present invention.
FIG. 2 is a view for explaining the relationship between input and output to the machine learning processing unit of FIG. 1.
FIG. 3 is a diagram for explaining a relationship between input and output to the image converter of FIG. 1.
4 is a view for explaining a case in which a high-resolution image processing apparatus according to an embodiment of the present invention is applied to an embodiment of a separation technique for visible and near-infrared regions.
FIG. 5 is a view for explaining a configuration in which the machine learning processing unit of FIG. 4 is further embodied.
6 is a view for explaining the internal processing of the image converter of FIG. 4.
7 is a view for explaining a visible area and a near-infrared separation network according to an embodiment of the present invention.
8 is a view for explaining a high-resolution visible region and a near-infrared image acquisition process according to an embodiment of the present invention.
9 is a view for explaining the experimental results for the method according to an embodiment of the present invention.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components. For example, the first component may be referred to as a second component without departing from the scope of the present invention, and similarly, the second component may be referred to as a first component. The term and/or includes a combination of a plurality of related described items or any one of a plurality of related described items.

When an element is said to be "connected" or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but there may be other components in between. It should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the overall understanding in describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

1 is a view for explaining the configuration of a high-resolution image processing apparatus using machine learning according to an embodiment of the present invention.

As shown in FIG. 1, the apparatus 100 for processing a high-resolution image using machine learning according to an embodiment of the present invention includes a downsampling unit 110, a machine learning processing unit 120, and an image conversion unit 130. Includes. However, not all of the illustrated components are essential components. It may be implemented by more components than the illustrated components, or may be implemented by fewer components.

The apparatus 100 for processing a high resolution image using machine learning according to an embodiment of the present invention performs a downsampling process of reducing the high resolution input image to a spatial resolution suitable for an input size of the machine learning network. The high-resolution image processing apparatus 100 calculates a low-resolution output image by using the down-sampled low-resolution image as an input to the machine learning network. The number of times the machine learning network is performed in this process may vary depending on the application, but is performed only once in one embodiment of the present invention.

In addition, the apparatus 100 for processing a high resolution image using machine learning according to an embodiment of the present invention calculates a high resolution result image through an image processing process using a high resolution input image and a low resolution image calculated through a machine learning network. . In this process, in one embodiment of the present invention, an image segmentation process and a color/brightness conversion process are performed within an image processing process.

Hereinafter, a detailed configuration and operation of each component of the high-resolution image processing apparatus 100 using the machine learning of FIG. 1 will be described.

1 is a structural diagram of a device representing the entire structure according to an embodiment of the present invention. 1 shows a structure for calculating a high resolution result image from a high resolution input image.

The high-resolution image processing apparatus 100 reduces the spatial resolution of the high-resolution input image through the down-sampling unit 110 to generate a low-resolution image. The downsampling unit 110 performs a process of reducing the spatial resolution of the high resolution image. In this process, any method can be used as long as it can reduce the spatial resolution of the image.

The machine learning processing unit 120 outputs a low-resolution image converted from the input low-resolution image. The machine learning method used here is a conventional machine such as a convolutional neural network (CNN), an autoencoder, and a generative adversarial network (GAN) that can receive images and output images. The use of learning networks is possible.

The image converter 130 receives a high-resolution input image and a low-resolution machine learning result image, and outputs a high-resolution result image in a form similar to the brightness, color, and texture of the machine learning result image. The processing method of the image conversion unit 130 may vary according to an embodiment, and in the present invention, the image conversion unit 130 according to a processing method for separating a visible region and a near infrared region image is described in FIG. 6.

FIG. 2 is a view for explaining the relationship between input and output to the machine learning processing unit of FIG. 1.

2 is a diagram illustrating a relationship between an input and an output to the machine learning processing unit 120 of the apparatus 100 for processing high-resolution images according to an embodiment of the present invention. The machine learning processing unit 120 used in an embodiment of the present invention calculates a low-resolution machine learning output image through a machine learning processing process according to an embodiment from a low-resolution input image.

FIG. 3 is a diagram for explaining a relationship between input and output to the image converter of FIG. 1.

3 is a diagram illustrating a relationship between an input and an output to the image converter 130 of the apparatus 100 for processing a high resolution image according to an embodiment of the present invention. The image converter 130 used in an embodiment of the present invention receives a high-resolution input image and a low-resolution machine learning output image and outputs a high-resolution result image through a process of changing the image according to the embodiment.

4 to 6 are for another embodiment of the present invention, showing that the separation from the visible and near-infrared region mixed input image into visible and near-infrared images.

4 is a view for explaining a case in which a high-resolution image processing apparatus according to an embodiment of the present invention is applied to an embodiment of a separation technique for visible and near-infrared regions.

4 is a diagram illustrating a case in which the apparatus 100 for processing a high-resolution image according to an embodiment of the present invention is applied to an embodiment of a technique for separating visible and near-infrared regions.

The input image in which the high-resolution visible region and the near-infrared region, which is the input of FIG. 4 are mixed, refers to an image captured after removing the infrared cut filter inside the conventional camera.

The high-resolution visible region result image and the high-resolution near-infrared region result image are calculated through a series of processes of the down-sampling unit 110, the machine learning processing unit 120, and the image conversion unit 130 of FIG.

FIG. 5 is a view for explaining a configuration in which the machine learning processing unit of FIG. 4 is further embodied.

5 is a detailed embodiment of the machine learning processing unit 120 of the apparatus 100 for processing high-resolution images according to an embodiment of the present invention. The machine learning processing unit 120 includes a machine learning based visible region separated image calculator 121 and a machine learning based near infrared region separated image calculator 122. FIG. 5 is a diagram illustrating a case in which the machine learning processing unit 120 of FIG. 2 is applied to an embodiment of a technique of separating visible and near-infrared regions.

The input image of the low-resolution visible region and the near-infrared region, which is the input of FIG. 5, is an image calculated after the down-sampling process of the high-resolution visible region and the near-infrared region mixed input image.

The input image of FIG. 5 is input to the machine learning based visible region separated image calculator 121 and the near infrared region separated image calculator 122 to calculate a low resolution visible region output image and a low resolution near infrared region output image from each calculator.

6 is a view for explaining the internal processing of the image converter of FIG. 4.

FIG. 6 shows an internal processing process of the image conversion unit 130 of FIG. 4 and schematically illustrates a case in which FIG. 3 is applied to an embodiment of an image separation technique for visible and near-infrared regions.

The high-resolution visible and near-infrared region mixed input image of FIG. 6 is the same as the input image of FIG. 4.

The low-resolution visible region output image and the low-resolution near-infrared region output image of FIG. 6 are the same as the output image of FIG. 5.

The high-resolution visible region result image and the high-resolution near-infrared region result image of FIG. 6 are the same as the output image of FIG. 4.

The color space converter (RGB to HSV) 131 of FIG. 6 performs a process of converting an RGB color space image into an HSV color space image. The color space conversion unit (HSV to RGB) 132 converts the HSV color space image into an RGB color space image. The color space converters 131 and 132 used herein are not limited to converting to HSV, and a color space suitable for an embodiment can be used.

The upsampling 133 of FIG. 6 receives a low resolution image and outputs a high resolution image. The upsampling method only needs to be able to convert a low spatial resolution image to a high spatial resolution, and does not limit the method.

The image region division unit 134 of FIG. 6 outputs a region division map from the input image. In another embodiment of the present invention, the K-average clustering based region division method is used, but the method capable of region division is not limited, and any method may be used.

The color/brightness conversion unit 135 of FIG. 6 changes the color and brightness of the high-resolution input image and outputs the changed high-resolution image. In this process, a low-resolution image and an area division map calculated through a machine learning process are used.

Hereinafter, a specific method related to the embodiment of the present invention will be described.

The model of visible and near-infrared separation used in the embodiment of the present invention will be described.

In order to obtain an image in which the visible region and the near infrared ray are mixed, an embodiment of the present invention removes the IR hot-mirror in the general camera. The image obtained from the camera can be expressed by the following equation (1). Model of VIS+NIR mixed image.

(1)

(One)

,

및

은 R,G,B 각 채널별 픽셀 값을 나타낸다.

,

및

는 RGB컬러필터배열에 의한 투과응답 함수이며,

는 광원에 대한 함수이다.

는 물체에 의한 반사특성 함수이다. 위의 식으로부터, 카메라 전면에 가시영역 밴드통과 필터를 장착한다음 영상을 촬영하는 경우는 다음과 같이 표현 가능하다. 가시영역 영상의 모델(Model of VIS image)이다. here,

,

And

Indicates a pixel value for each channel of R, G, and B.

,

And

Is the transmission response function by the RGB color filter arrangement,

Is a function for the light source.

Is a function of the reflection characteristic by an object. From the equation above, when attaching the visible band pass filter to the front of the camera, the following can be expressed when shooting an image. Model of VIS image.

(2)

(2)

는 가시영역 대역통과필터에 대한 응답함수이며, 가시영역 내에서

가 일정한 경우에는

와 같이 표현 가능하다. 근적외선 영상의 경우도 마찬가지로 아래와 같은 표현이 가능하다. 근적외선영역 영상의 모델(Model of NIR image)이다. here,

Is the response function for the visible band pass filter and within the visible area

If is constant

Can be expressed as In the case of a near-infrared image, the following expression is also possible. It is a model of near infrared region image.

(3)

(3)

는 근적외선영역 대역통과필터의 응답함수이며, 근적외선영역에서

이 일정하면

과 같이 표현 가능하다. 또한, 이상적인 경우 근적외선 대역에서의

는 동일하며,

하나로 표현 가능하다.here,

Is the response function of the band pass filter in the near infrared region, and in the near infrared region

If this is constant

Can be expressed as Also, in the ideal case, in the near-infrared band

Is the same,

It can be expressed as one.

From the above equation, it is possible to acquire the visible and near-infrared region images through the following separation formula.

(4)

(4)

는 분리식을 통해 획득한 가시영역 및 근적외선 영상이며,

는 각각 가시영역 및 근적외선 영역의 대역통과필터에 의한 가중치이다. 위의 식은 압축센싱을 이용한 방법 또는 딥러닝을 이용한 방법으로 풀이가 가능하며, 본 발명의 실시 예에서는 딥러닝을 이용한 방법을 통해 가시영역 및 근적외선 영역을 분리하였다.here,

Is a visible region and near-infrared image obtained through a separation equation,

Is a weight by a band pass filter in the visible and near infrared regions, respectively. The above equation can be solved by a method using compression sensing or a method using deep learning. In the embodiment of the present invention, a visible region and a near infrared region are separated through a method using deep learning.

Hereinafter, with reference to FIGS. 7 and 8, an entire block diagram of a visible region and near-infrared separation process according to an embodiment of the present invention will be described.

7 is a view for explaining a visible area and a near-infrared separation network according to an embodiment of the present invention.

7 is a diagram of a deep learning network separating a low-resolution visible region and a near infrared image from a high-resolution input image. In the deep learning network used in the embodiment of the present invention, a conditional generative adversarial network (cGAN) was applied to each of the visible and near-infrared images, and after downsampling the high-resolution visible and near-infrared mixed images, Used as input for each network.

8 is a view for explaining a high-resolution visible region and a near-infrared image acquisition process according to an embodiment of the present invention.

8 illustrates a process of generating a high-resolution visible region and a near-infrared image using a low-resolution visible region and a near-infrared image calculated through a high-resolution mixed image and a network. The basic concept of an embodiment of the present invention is to change the brightness and color of a high-resolution mixed image to match a separate visible-region image in order to generate a high-resolution visible region. To this end, RGB to HSV color space conversion, image segmentation, and color/brightness transfer are performed.

Meanwhile, a process of converting color space from RGB to HSV will be described. The image generated from the camera has three channel information of R, G, and B for each pixel. In an embodiment of the present invention, color, hue, saturation, and brightness are used to distinguish brightness and color gamut. Convert to HSV color space with 3 pieces of information. The transformed HSV color space information is used for image segmentation and color/brightness transfer.

가 이와 관련이 있어 가시영역에서는 밝게 나오는 부분이 근적외선 영역에서는 어둡게 나올 수가 있다. 본 발명의 실시 예에서는 영상영역분할(segmentation)과정을 이용하여 해당 특성이 나오는 부분을 분할화 하여 각 영역별로 밝기 값을 다르게 변화시키는 과정을 수행한다. 영상영역분할(Segmentation)을 하는 과정에서 분할에 중요한 영향을 미치는 요소는 색상이며, 이를 위해 분리된 가시영역 영상의 휘도(H), 채도(S) 정보 및 근적외선 영상의 밝기 정보를 영상영역분할(Segmentation)에 필요한 특징벡터로 구성한다. 그 다음, K-평균 클러스터링(K-means clustering) 방법을 통해 영역분할 맵(Segmentation map)을 생성한다. K-평균 클러스터링(K-mean clustering) 방법은 다음의 식을 통해 산출 가능하다.The image segmentation process will be described. The visible region and the near-infrared image have different wavelength bands, which is related to the reflectance of objects in the image. In equations (2) and (3)

As this is related, the bright part in the visible region may appear dark in the near infrared region. In an embodiment of the present invention, a process in which a brightness value is differently changed for each area is performed by dividing a part in which a corresponding characteristic appears using an image area segmentation process. In the process of segmenting the image, the factor that has an important effect on segmentation is color, and for this purpose, the luminance ( H ), saturation ( S ) information of the separated visible region image and the brightness information of the near-infrared image are divided into the image region ( Segmentation). Next, a segmentation map is generated through a K-means clustering method. The K-mean clustering method can be calculated by the following equation.

는

내 데이터의 평균이다.Here, S is the cluster set to be searched, K is the number of clusters, x is the data set,

The

It is the average of my data.

The process of brightness transfer will be described. The brightness of the mixed image is changed as follows for each region created in the image region segmentation process.

,

,

는 각각 분할영역 S _k 에 속하는 i, j번째 픽셀에 대한 고해상도 가시영역영상, 고해상도 근적외선영역영상, 고해상도 가시영역 및 근적외선영역 혼합영상의 명도 값을 의미한다.

,

,

는 각각 분할영역 S _k 에 속하는 저해상도 가시영역영상, 저해상도 근적외선영역 영상, 저해상도 가시영역 및 근적외선영역 혼합영상의 명도에 대한 표준편차 값을 의미한다.

및

는 각각 분할영역 S _k 에 속하는 저해상도 가시영역영상, 고해상도 가시영역 및 근적외선영역 혼합영상의 명도에 대한 평균값을 의미한다. 위의 과정을 통해 혼합영상으로부터 각 분리된 영상에 맞는 명도값을 보정할 수 있으며 이를 통해 고해상도 가시영역 밝기영상 및 고해상도 근적외선 밝기영상을 획득할 수 있다.here,

,

,

Denotes brightness values of a high-resolution visible region image, a high-resolution near-infrared region image, a high-resolution visible region, and a near-infrared region mixed image for the i and j- th pixels belonging to the divided region S _k , respectively.

,

,

Denotes a standard deviation value for the brightness of a low-resolution visible region image, a low-resolution near-infrared region image, and a low-resolution visible and near-infrared region mixed image, respectively, belonging to the divided region S _k .

And

Denotes an average value for the brightness of the low-resolution visible region image, the high-resolution visible region, and the near-infrared region mixed image, respectively, belonging to the divided region S _k . Through the above process, the brightness value suitable for each separated image can be corrected from the mixed image, thereby obtaining a high-resolution visible region brightness image and a high-resolution near-infrared brightness image.

The color transfer process will be described. The process, the resolution of the luminance of the low-resolution visible region image (H) and saturation (S) values in order to identify and separate the color of the isolated high resolution visible region image low-resolution color in the visible region image intensity in the visible region image (H), and Replace it with the saturation ( S ) value. Up-sampling process is performed to convert the low-resolution image to high-resolution, and linear interpolation is used in the embodiment of the present invention. Through this process, the luminance ( H ) and saturation ( S ) values for the high-resolution result image can be obtained, and the HSV color space for the high-resolution result image together with the brightness ( V ) value obtained through the brightness transfer process It is possible to acquire the whole.

The process of color space conversion from HSV to RGB will be described. In order to display a high-resolution visible area HSV image obtained through a color/brightness transfer process, a process of converting it to an RGB color space is performed. When this process is performed, a high resolution visible area image is finally obtained.

Meanwhile, the experimental results will be described.

The experimental conditions are as follows. In order to generate the high-resolution visible region and near-infrared image proposed in the embodiment of the present invention, experimental conditions under the conditions as shown in [Table 1] were set.

9 is a view for explaining the experimental results for the method according to an embodiment of the present invention.

The results of the experiment are as follows.

Evaluation of computational complexity: The computational complexity of the method according to the embodiment of the present invention was compared with the conventional method. In the conventional method, in order to perform deep learning on a high-resolution image, a high-resolution input image is divided into 3978 256x256 sized blocks, and a deep learning process is performed for each block. On the system used in this experiment, since 10 images per second can be calculated in the deep learning process, the conventional method performs a deep learning process of about 398 seconds per sheet, and a stitching process to combine the calculated images into one. Essay took about 40 seconds to produce. Therefore, in order to acquire two images of the visible region and the near-infrared ray using a conventional method, it takes about 876 seconds. Since the method according to the embodiment of the present invention only needs to execute a single deep learning network, it takes 0.1 seconds for deep learning to be performed per page. Subsequent image segmentation and color/brightness transfer ) Process took about 4 seconds per sheet. Therefore, in the method according to the embodiment of the present invention, the time taken to separate two visible and near infrared images does not exceed 9 seconds, which is about 97 times faster than the conventional method.

Subjective image quality evaluation: Figure 9 shows the experimental results of the method according to an embodiment of the present invention.

As shown in the experiment results of FIG. 9, it can be seen that the color of the method according to the embodiment of the present invention is different from the target comparison image, because the learning result of the deep learning network is not sufficient. It can be seen that the color of the separated high-resolution visible region image of the method according to the embodiment of the present invention is similar to the low-resolution visible region deep learning output image. Therefore, in order to improve the image quality of the separated image, it is expected that more precise design of the network and learning through a large amount of samples will be required. However, when the method according to the embodiment of the present invention is compared with the image using the conventional method, in the method according to the embodiment of the present invention, image quality deterioration with visible block boundaries is not observed, whereas in the conventional method, the block boundary is There is a clearly visible problem. In addition to this, it can be seen that the method according to the embodiment of the present invention is improved over the conventional method in addition to computational complexity.

In an embodiment of the present invention, a method of obtaining a high-resolution visible region and a near-infrared image from a single visible and near-infrared mixed image is provided. According to an embodiment of the present invention, a low-resolution input image generated by down-sampling a high-resolution input image is input to a deep learning network, and then image segmentation, color/brightness conversion (Color) is calculated for the calculated low-resolution output image. /Brightness transfer) to create a high-resolution result image. The method according to the embodiment of the present invention showed a computational complexity that is 97 times faster than that of the conventional method, and it was also confirmed that block deterioration does not occur in the resulting image.

The method of processing a high-resolution image using machine learning according to the above-described embodiments of the present invention can be implemented as a computer-readable code on a computer-readable recording medium. The method of processing a high-resolution image using machine learning according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable recording medium.

Computer-readable recording media includes all kinds of recording media storing data that can be read by a computer system. For example, there may be a read only memory (ROM), a random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, and an optical data storage device. In addition, the computer-readable recording medium may be distributed over computer systems connected through a computer communication network, and stored and executed as code readable in a distributed manner.

Specifically, the described features can be implemented in digital electronic circuitry, or computer hardware, firmware, or combinations thereof. Features may be implemented in a computer program product implemented in storage in a machine-readable storage device, eg, for execution by a programmable processor. And the features can be performed by a programmable processor executing a program of instructions for performing the functions of the described embodiments by operating on input data and generating output. The described features include at least one programmable processor, at least one input device, and at least one output coupled to receive data and directives from the data storage system and to transmit data and directives to the data storage system. It can be executed in one or more computer programs that can be executed on a programmable system including a device. A computer program includes a set of directives that can be used directly or indirectly within a computer to perform a specific action on a given result. A computer program is written in any form of programming language, including compiled or interpreted languages, and is included as a module, element, subroutine, or other unit suitable for use in other computer environments, or as a standalone program. Can be used in any form.

Suitable processors for the execution of the program of instructions include, for example, both general purpose and special purpose microprocessors, and either a single processor or multiple processors of other types of computers. Also suitable for implementing computer program instructions and data embodying the described features are storage devices suitable for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic devices such as internal hard disks and removable disks. Devices, magneto-optical disks and all forms of non-volatile memory including CD-ROM and DVD-ROM disks. The processor and memory can be integrated within application-specific integrated circuits (ASICs) or added by ASICs.

The present invention described above has been described based on a series of functional blocks, but is not limited by the above-described embodiments and the accompanying drawings, and various substitutions, modifications and changes without departing from the spirit of the present invention. It will be apparent to those skilled in the art that the present invention is possible.

Combinations of the above-described embodiments are not limited to the above-described embodiments, and various forms of combinations may be provided as well as the above-described embodiments according to implementation and/or needs.

In the above-described embodiments, the methods are described based on a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and some steps may occur in a different order than the steps described above or simultaneously. have. In addition, those of ordinary skill in the art are aware that the steps shown in the flowcharts are not exclusive, other steps may be included, or one or more steps in the flowcharts may be deleted without affecting the scope of the present invention. You will understand.

The above-described embodiments include examples of various aspects. It is not possible to describe all possible combinations for representing various aspects, but those skilled in the art will recognize that other combinations are possible. Accordingly, the present invention will be said to include all other replacements, modifications and changes that fall within the scope of the following claims.

100: high-resolution image processing device
110: downsampling unit
120: machine learning processing unit
130: video converter
121: visible area separated image calculation unit
122: near infrared region separated image calculation unit
131: color space conversion unit (RGB to HSV)
132: color space conversion unit (HSV to RGB)
133: upsampling
134: image area division unit
135: color / brightness conversion unit

Claims (1)

A down-sampling unit that generates a low-resolution image by reducing the spatial resolution of the high-resolution input image;
A machine learning processor configured to calculate the low-resolution output image by using the generated low-resolution image as an input to the machine learning network; And
And an image conversion unit for converting the high-resolution input image and the calculated low-resolution output image according to the characteristics of the calculated low-resolution output image and outputting a high-resolution result image.

Images