KR102631631B1 - Hyperspectral image visualization method and appratus using neural network - Google Patents

Abstract

A device for visualizing hyperspectral images using a neural network-based model is disclosed.
An image visualization device according to an embodiment includes a receiver and a processor that receive a first image corresponding to a hyperspectral image, and the processor includes intensity information of each wavelength band for each pixel of the first image. extract the original data,
Generate prediction data including pixel values to generate an RGB image corresponding to the first image based on the original data and a pre-trained neural network-based model, and generate the first image based on the generated prediction data. Visualize, and the pre-trained neural network-based model outputs prediction data for generating an RGB image corresponding to the second image by inputting a second image containing fewer wavelength bands than the first image. It can be learned in advance to do so.

Description

Visualization method and device for hyperspectral images using neural network {HYPERSPECTRAL IMAGE VISUALIZATION METHOD AND APPRATUS USING NEURAL NETWORK}

The following embodiments relate to a method and device for visualizing hyperspectral images using a neural network.

While RGB has three channels, hyperspectral images (HSIs) measure a wide range of wavelengths, up to 200 bands or more. To achieve this, HSI detects complex features that cannot be found in RGB images. RGB images only have spatial features, but HSIs have both spectral features and spatial features, showing excellent performance in classification. Therefore, HSIs can be classified for similar colors and thus can be applied to a wide range of fields, including remote sensing.

Recently, HSI using machine learning and deep learning has been adopted in areas such as meat quality, land cover mapping, object detection, change detection, and satellite image mapping.

Machine learning involves mathematical models that learn patterns of data classification or linear regression. Deep learning is widely used in various fields such as computer vision or translation. Deep neural networks used in deep learning include multiple layers.

Non-patent Reference 1: M. Magnusson, J. Sigurdsson, S. E. Armansson, M. O. Ulfarsson, H. Deborah and J. R. Sveinsson, "Creating RGB Images from Hyperspectral Images Using a Color Matching Function, "　IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium, 2020, pp. 2045-2048, doi: 10.1109/IGARSS39084.2020.9323397

Embodiments may provide image visualization technology using a neural network.

However, technical challenges are not limited to the above-mentioned technical challenges, and other technical challenges may exist.

An apparatus for visualizing a hyperspectral image using a neural network-based model according to an embodiment includes a receiver that receives a first image corresponding to the hyperspectral image; and a processor, wherein the processor extracts original data including intensity information of each wavelength band for each pixel of the first image, and extracts the original data based on the original data and a previously learned neural network-based model. Generating prediction data including pixel values to generate an RGB image corresponding to the first image, visualizing the first image based on the generated prediction data, and the pre-trained neural network-based model, It can be trained in advance to output prediction data for generating an RGB image corresponding to the second image by using a second image including fewer wavelength bands than one image as input.

The processor generates preprocessing data including intensity information corresponding to the wavelength band of the second image through interpolation of the wavelength band of the first image, and inputs the preprocessing data into the neural network-based model, Predictive data can be generated.

The neural network-based model may be trained in advance to output the prediction data for generating the RGB image by using the wavelength band corresponding to the visible light region among the wavelength bands of the second image as input.

The processor may apply a sharpening filter to the generated prediction data and visualize the first image based on a result of applying the sharpening filter.

An image visualization method for a hyperspectral image according to an embodiment includes receiving a first image corresponding to a hyperspectral image; extracting original data including intensity information of each wavelength band for each pixel of the first image; Generating prediction data for generating an RGB image corresponding to the first image based on the original data and a pre-trained neural network-based model; and visualizing the first image based on the generated prediction data.

Embodiments can increase the usability of hyperspectral images by clearly visualizing hyperspectral images with high similarity to reality through a neural network-based model learned through multispectral images. Additionally, compared to artificial neural networks that simply take hyperspectral images as input and output visualization results, embodiments can provide a means of performing learning with less load.

Figure 1 shows a schematic block diagram of an image visualization device according to an embodiment.
Figure 2 is an example of image visualization.
Figure 3 shows a flowchart of an image visualization method according to one embodiment.
FIG. 4 is a diagram illustrating a method of learning a neural network-based model used in a visualization method according to an embodiment.
Figure 5 shows a sampling method of original data according to one embodiment.
FIGS. 6A and 6B are diagrams illustrating an experimental example to which a visualization method according to an embodiment is applied.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are merely illustrative for the purpose of explaining the embodiments according to the concept of the present invention. They may be implemented in various forms and are not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can make various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes changes, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component, for example, a first component may be named a second component, without departing from the scope of rights according to the concept of the present invention, Similarly, the second component may also be referred to as the first component.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between. Expressions that describe the relationship between components, such as “between”, “immediately between” or “directly adjacent to”, should be interpreted similarly.

The terminology used herein is only used to describe specific embodiments and is not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate the presence of a described feature, number, step, operation, component, part, or combination thereof, and one or more other features or numbers, It should be understood that this does not exclude in advance the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. No.

A module in this specification may mean hardware that can perform functions and operations according to each name described in this specification, or it may mean computer program code that can perform specific functions and operations. , or it may mean an electronic recording medium loaded with computer program code that can perform specific functions and operations, for example, a processor or microprocessor.

In other words, a module may mean a functional and/or structural combination of hardware for carrying out the technical idea of the present invention and/or software for driving the hardware.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or limited by these examples. The same reference numerals in each drawing indicate the same members.

Figure 1 shows a schematic block diagram of an image visualization device according to an embodiment.

Referring to FIG. 1, the image visualization device 10 may perform visualization of the first image below. The first image may be a hyperspectral image (HSIs). The first image may be comprised of information in a form that can be processed by a computer. The image visualization device 10 may perform visualization of the first image by receiving the first image and processing the first image using a neural network.

Neural networks (or artificial neural networks) can include statistical learning algorithms that mimic neurons in biology in machine learning and cognitive science. A neural network can refer to an overall model in which artificial neurons (nodes), which form a network through the combination of synapses, change the strength of the synapse connection through learning and have problem-solving capabilities.

Neurons in a neural network can contain combinations of weights or biases. A neural network may include one or more layers consisting of one or more neurons or nodes. Neural networks can infer the results they want to predict from arbitrary inputs by changing the weights of neurons through learning.

Neural networks may include deep neural networks. Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), and LSTM. (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield) Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network) ), Liquid State Machine (LSM), Extreme Learning Machine (ELM), Echo State Network (ESN), Deep Residual Network (DRN), Differential Neural Computer (DNC), Neural Turning Machine (NTM), Capsule Network (CN), Those skilled in the art will understand that it may include any neural network, including, but not limited to, KN (Kohonen Network) and AN (Attention Network).

The image visualization device 10 may be implemented with a printed circuit board (PCB) such as a motherboard, an integrated circuit (IC), or a system on chip (SoC). For example, the image visualization device 10 may be implemented with an application processor.

Additionally, the image visualization device 10 may be implemented within a personal computer (PC), a data server, or a portable device.

Portable devices include laptop computers, mobile phones, smart phones, tablet PCs, mobile internet devices (MIDs), personal digital assistants (PDAs), and enterprise digital assistants (EDAs). , digital still camera, digital video camera, portable multimedia player (PMP), personal navigation device or portable navigation device (PND), handheld game console, e-book ( It can be implemented as an e-book) or a smart device. A smart device may be implemented as a smart watch, smart band, or smart ring.

The image visualization device 10 according to an embodiment may include a receiver 100 and a processor 200, and may further include a memory 300.

The receiver 100 according to one embodiment may receive the first image. Receiver 100 may include a receiving interface. The receiver 100 may output the received first image to the processor 200.

The processor 200 according to one embodiment may process data stored in the memory 300. The processor 200 may execute computer-readable code (eg, software) stored in the memory 300 and instructions triggered by the processor 200 .

The processor 200 according to one embodiment may be a data processing device implemented in hardware that has a circuit with a physical structure for executing desired operations. For example, the intended operations may include code or instructions included in the program.

For example, data processing devices implemented in hardware include microprocessors, central processing units, processor cores, multi-core processors, and multiprocessors. , ASIC (Application-Specific Integrated Circuit), and FPGA (Field Programmable Gate Array).

Specifically, the processor 200 can generate preprocessed data through interpolation for the input first image, and apply the preprocessed data to a neural network-based model to generate prediction data necessary for visualization of the first image. there is.

More specifically, the processor 200 extracts original data including intensity information of individual wavelength bands constituting each pixel from the first image, obtains pre-processed data through interpolation on the extracted original data, and obtains pre-processed data. Predicted data can be generated by inputting sampling data obtained from a neural network-based model.

A neural network-based model according to one embodiment may be composed of an encoder of the Auto-Encoder model. A neural network-based model may be trained in advance to take the second image as input and output an RGB image corresponding to the second image. The second image may be an image composed of fewer wavelength bands than the first image, and may be an example of a multispectral image (MSIs) image. For example, the second image may be an image composed of 10 or less wavelength bands. The neural network-based model predicts the RGB value of each pixel by using the intensity information of each wavelength band in the visible light region (for example, the wavelength band between 400nm and 700nm) among the wavelength bands that make up each pixel of the second image as input. It can be learned in advance.

Since the wavelength of the wavelength band constituting the second image used for learning the neural network-based model is different from the wavelength of the wavelength band constituting the first image, the processor 200 generates the generated through interpolation for the first image. Visualization of the first image can be performed using preprocessed data. More specifically, the processor 200 configures each pixel of the second image through interpolation of intensity information for the first wavelength band constituting each pixel of the first image (intensity information of each wavelength band constituting each pixel). Preprocessing data can be generated by calculating intensity information corresponding to the second wavelength band. For example, in a situation where the individual pixels constituting the first image are composed of wavelength bands that divide the entire wavelength band into 100, and the individual pixels that constitute the second image are composed of wavelength bands that divide the entire wavelength band into 10 , the processor 200 may perform interpolation on the intensity values of the 10th wavelength band from the 1st wavelength band of the first image to obtain intensity information of the 1st wavelength band constituting the preprocessing data. In the same way, intensity information corresponding to each wavelength band constituting the preprocessing data can be derived from the first image. For example, the processor 200 may perform interpolation on the intensity values of the 11th wavelength band to the 20th wavelength band of the second image in order to obtain intensity information of the second wavelength band constituting the preprocessing data. In the same way, in order to generate preprocessing data corresponding to the 550 nm wavelength band of the second image, 10 wavelength bands of the first image can be interpolated through the method described above. In the process of interpolating the 10 wavelength bands, the processor 200 may perform interpolation by assigning a weight inversely proportional to the difference between the wavelength of each of the 10 wavelength bands used for interpolation and 550 nm.

In another example, the processor 200 determines a predetermined number of wavelength bands of the first image that have the highest similarity to the wavelengths of the wavelength bands included in the second image, and generates preprocessing data through interpolation between the determined wavelength bands. It may be possible. Similarly, the processor 200 may determine a wavelength band of the first image whose similarity to the wavelength of the wavelength band included in the second image is greater than or equal to a predetermined threshold, and generate preprocessing data through interpolation between the determined wavelength bands. . For example, in order to generate preprocessing data corresponding to the 550nm wavelength band included in the second image, the processor 200 selects a predetermined number (e.g., 5) of the wavelength bands with the closest wavelength to 550nm. And, preprocessing data can be generated based on interpolation between selected wavelength bands. In the process of performing interpolation, weights may be assigned depending on the degree of proximity to 550 nm. Those skilled in the art will understand that the method of performing interpolation is not limited to the presented example, and that any interpolation method may be applied.

In another example, the processor 200 selects the wavelength band of the second image in which a wavelength band having a wavelength having a similarity greater than a predetermined threshold among the wavelength bands included in the second image is present in the first image to the 2-1 wavelength band. , and the remaining wavelength band can be determined as the 2-2 wavelength band. The processor 200 uses the wavelength band of the first image determined to have the highest similarity for the 2-1 wavelength band as is, but performs interpolation on the first image only for data corresponding to the 2-2 wavelength band. By doing so, preprocessing data can be generated. Through this, the present invention can save calculations in the pre-processing process.

The processor 200 acquires data corresponding to the wavelength of the wavelength band used for learning the neural network model among the preprocessed data as sampling data, and inputs the obtained sampling data into the neural network-based model to generate prediction data. there is. The processor 200 may obtain prediction data by acquiring intensity information of a wavelength band corresponding to the visible light band among the preprocessed data as sampling data and inputting the sampling data into a neural network-based model. Prediction data may be configured in the form of RGB values of each pixel. Based on the prediction data, the first image may be visualized in RGB form.

The memory 300 according to one embodiment may store instructions (or programs) executable by the processor 200. For example, the instructions may include instructions for executing the operation of the processor and/or the operation of each component of the processor.

The memory 300 may be implemented as a volatile memory device or a non-volatile memory device.

Volatile memory devices may be implemented as dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM).

Non-volatile memory devices include EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM (MRAM), and Conductive Bridging RAM (CBRAM). , FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM (Nanotube RRAM), Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, molecular electronic memory device, or insulation resistance change memory.

Figure 2 is an example of image visualization.

Referring to Figure 2, an RGB visualization image 210 and a hyperspectral visualization image 220 appear.

When visualization is performed using data from images captured by an RGB camera, visualization can be achieved in colors similar to what we actually see with our eyes, as shown in image 210, by visualizing the image using three standardized channels. . On the other hand, when using data from images taken by a hyperspectral camera, intensity data for multiple wavelength bands is included, and in order to visualize this, three bands can be selected among many bands for visualization. In this case, distortion may occur due to the selection of data, resulting in visualization that is different from the actual image, such as image 220.

In order to minimize distortion of image visualization and enable hyperspectral visualization images to be similar to actual images, the hyperspectral image visualization method according to the embodiment of FIG. 3 may be used.

Figure 3 shows a flowchart of an image visualization method according to one embodiment.

The image visualization method of FIG. 3 may be performed by the image visualization device 10 of FIG. 1 .

Specifically, the image visualization device 10 may receive the first image through the receiver 100 (S310). For example, the first image may include a hyperspectral image implemented with multiple wavelength bands exceeding 10 for each pixel.

The image visualization device 10 according to an embodiment may extract original data including intensity information of an individual wavelength band of each pixel from the received first image (S320).

The original data may include intensity information for each pixel and individual wavelength bands that make up each pixel. Depending on the implementation, the original data may consist of intensity information for each of as few as 11 wavelength bands, and in some cases, more than 200 wavelength bands. It can be.

The image visualization device 10 according to an embodiment may generate preprocessed data from original data and obtain sampling data from the preprocessed data (S330). The preprocessed data may include intensity information of multiple wavelength bands for each pixel and may be generated through interpolation of the original data. The sampling data may include intensity information of a wavelength band in the visible light region among the wavelength bands included in the preprocessing data. The method of acquiring sampling data through steps S320 and S330 will be described in more detail with reference to FIG. 5 below.

The image visualization device 10 may obtain prediction data by inputting the acquired sampling data into a neural network-based model. The neural network-based model takes as input a second image (e.g., multispectral image) consisting of fewer wavelength bands than the first image, and predicts corresponding RGB values for each pixel to visualize the multispectral image as an RGB image. It can be trained in advance to output data. A detailed description of the neural network-based model can be explained in more detail through FIG. 4 attached below.

The image visualization device 10 may perform visualization of the first image using previously acquired prediction data. The first image may be visualized as an RGB image through visualization based on prediction data. According to one embodiment, the image visualization device 10 may further improve the clarity of the RGB image by applying a sharpening filter to previously acquired prediction data. A sharpening filter is a filter that improves the sharpness of an RGB image. For example, a Laplacian filter can be used. However, the type of sharpening filter that can be used is not limited to this, and any type of sharpening filter that can improve the sharpness of the image can be used. Those skilled in the art will understand that image processing filters may be utilized.

FIG. 4 is a diagram illustrating a method of learning a neural network-based model used in a visualization method according to an embodiment.

Referring to FIG. 4, the neural network-based model 420 used in the visualization method according to one embodiment includes prediction data for visualizing the second image as an RGB image from the second image (e.g., multispectral image) input. It can be learned in advance to output .

More specifically, the neural network-based model 420 may be composed of an encoder of an auto-encoder model. The neural network-based model 420 outputs a prediction result 430 from learning data 410 generated based on a multispectral image, and the loss value calculated through the prediction result 430 and the target result 440 is minimized. It can be learned in a way that works. The learning data 410 may include information on the intensity of each wavelength band for each pixel of the multispectral image, and the wavelength band included in the learning data may be a wavelength band in the visible light band among all wavelength bands. Prediction data 430 may be composed of RGB values for each pixel. The target result 440 may be an RGB image obtained for the same object as the training data 410. The neural network-based model 420 can be learned in a way that minimizes the loss value corresponding to Equation 1.

is the loss value of the neural network-based model 420, N is the number of pixels included in the image, is input information corresponding to the intensity information of individual wavelength bands of individual pixels included in the learning data 410, is the input information The prediction result (430) of the neural network-based model (420) according to is the input information It may be a pixel value (RGB value) of the target result 440 corresponding to .

Figure 5 shows a sampling method of original data according to one embodiment.

Referring to FIG. 5, the processor generates preprocessing data corresponding to the second image 520 corresponding to the multispectral image used for learning the neural network-based model from the first image 510 corresponding to the hyperspectral image. can do. Exemplarily, the first image 510 may include intensity values 512 of 100 wavelength bands for each pixel 511. The second image 520 may be composed of intensity values of a smaller number of wavelength bands for each pixel 512 than the first image 510, for example, intensity values 522 of 10 wavelength bands. ) may include. As shown, since the wavelengths of the wavelength bands included in the first image 510 and the second image 520 are different, the processor processes preprocessed data corresponding to the wavelength of the second image 520 from the first image 510. can be created. More specifically, the processor performs interpolation on the intensity values of the first to tenth wavelength bands 513 of the pixel 511 to generate preprocessing data corresponding to the first wavelength band 523 of the pixel 521. can do. The method of performing interpolation may be based on the method described above. In the same way, the processor may sequentially interpolate the intensity values 512 of the wavelength bands included in the pixel 511 to generate preprocessing data corresponding to the second to tenth wavelength bands of the pixel 521. . The processor may generate preprocessed data by performing the same interpolation for all pixels other than pixel 511.

In another example, in order to generate preprocessing data corresponding to the wavelength band 523, the processor selects a predetermined number (e.g., You can also select five (5) wavelength bands and generate preprocessing data through interpolation of the selected wavelength bands. In this case, in the process of generating preprocessing data, the higher the similarity between the selected wavelength band and the wavelength band 523, a higher weight may be assigned to generate preprocessing data.

The processor can generate sampling data by extracting the intensity value of the wavelength band corresponding to the visible light region from the generated preprocessed data, and obtain prediction data by inputting the sampling data into a neural network-based model. The processor may perform visualization of the first image 510 based on RGB values included in the prediction data.

In addition, as described above, the processor determines a wavelength band to perform interpolation and a wavelength band to be used as is without interpolation, based on the similarity between the wavelength of the wavelength band of the second image and the wavelength of the wavelength band of the first image, Preprocessing may be performed based on the decision result.

FIGS. 6A and 6B are diagrams illustrating an experimental example to which a visualization method according to an embodiment is applied.

In order to confirm the effectiveness of the visualization method according to one embodiment, hyperspectral images were visualized in three different ways. The first comparative visualization method may be a method of selecting and visualizing three different wavelength bands among a plurality of wavelength bands included in a hyperspectral image. The second comparative visualization method is to visualize hyperspectral images using a color matching function, and is described in non-patent reference 1 (M. Magnusson, J. Sigurdsson, S. E. Armansson, M. O. Ulfarsson, H. Deborah and J. R. Sveinsson, "Creating RGB Images from Hyperspectral Images Using a Color Matching Function, "IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium, 2020, pp. 2045-2048, doi: 10.1109/IGARSS39084.2020.9323397) can be easily implemented by a person skilled in the art. .

The camera used in the experiment below may correspond to FX10 Specim (400~1000nm).

Figure 6a shows an experimental example of visualizing a hyperspectral image for a target object corresponding to dried radish. Image 611 may be an image taken of a target object, dried radish, through any RGB camera, such as a mobile phone camera.

The image 612 may be an image in which a hyperspectral image of a target object is visualized through the first comparative visualization method described above.

The image 613 may be an image in which a hyperspectral image of a target object is visualized through the second comparative visualization method described above.

The image 614 may be an image in which the target object has been visualized by applying a visualization method according to an embodiment.

When comparing the presented images (611, 612, 613, and 614), it can be seen that the image (614) on which visualization according to the present invention was performed has a high degree of similarity to the general RGB image (611) and that the actual target object was visualized with high clarity. there is.

Figure 6b shows an example of an experiment visualizing a hyperspectral image for a target object corresponding to a rat gun. Image 621 may be an image taken of a rat, which is a target object, through any RGB camera, such as a mobile phone camera.

The image 622 may be an image in which a hyperspectral image of a target object is visualized through the first comparative visualization method described above.

The image 623 may be an image in which a hyperspectral image of a target object is visualized through the second comparative visualization method described above.

The image 624 may be an image in which the target object has been visualized by applying a visualization method according to an embodiment.

When comparing the presented images (621, 622, 623, 624), it can be seen that the image (624) on which visualization according to the present invention was performed has a high degree of similarity to the general RGB image (621) and that the actual target object was visualized with high clarity. there is.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (5)

In a device for visualizing hyperspectral images using a neural network-based model,
A receiver that receives a first image corresponding to a hyperspectral image; and
processor
Including,
The processor,
Extract original data including intensity information of each wavelength band for each pixel of the first image,
Generating prediction data including pixel values to generate an RGB image corresponding to the first image based on the original data and a pre-trained neural network-based model,
Visualize the first image based on the generated prediction data,
The pre-trained neural network-based model is,
Learning in advance to output prediction data for generating an RGB image corresponding to the second image by using a second image including a smaller number of wavelength bands than the first image as input,
The processor,
Generate preprocessing data including intensity information corresponding to the wavelength band of the second image through interpolation of the wavelength band of the first image,
Generating the prediction data by inputting the preprocessed data into the neural network-based model,
The neural network-based model is,
Trained in advance to output the prediction data for generating the RGB image by using a wavelength band corresponding to the visible light region among the wavelength bands of the second image as input,
The processor,
In the process of generating the preprocessing data corresponding to the second wavelength band included in the second pixel of the second image,
Select a wavelength band whose similarity to the wavelength of the second wavelength band is more than a predetermined threshold among the wavelength bands included in the first pixel of the first image corresponding to the second pixel, and based on interpolation for the selected wavelength band An image visualization device that generates the preprocessed data. delete According to paragraph 1,
The processor,
In the case of a wavelength band whose similarity exceeds a predetermined threshold, the preprocessing data is generated by using the wavelength band exceeding the threshold without interpolation. According to paragraph 1,
The processor,
Apply a sharpening filter to the generated prediction data,
An image visualization device that visualizes the first image based on a sharpening filter application result. In an image visualization method for a hyperspectral image,
Receiving a first image corresponding to a hyperspectral image;
extracting original data including intensity information of each wavelength band for each pixel of the first image;
Generating prediction data for generating an RGB image corresponding to the first image based on the original data and a pre-trained neural network-based model;
Visualizing the first image based on the generated prediction data
Including,
The pre-trained neural network-based model is,
Learning in advance to output prediction data for generating an RGB image corresponding to the second image by using a second image including a smaller number of wavelength bands than the first image as input,
The step of generating the prediction data is,
Generate preprocessing data including intensity information corresponding to the wavelength band of the second image through interpolation of the wavelength band of the first image,
Generating the prediction data by inputting the preprocessed data into the neural network-based model,
The neural network-based model is,
Trained in advance to output the prediction data for generating the RGB image by using a wavelength band corresponding to the visible light region among the wavelength bands of the second image as input,
In the process of generating the preprocessing data corresponding to the second wavelength band included in the second pixel of the second image,
Select a wavelength band whose similarity to the wavelength of the second wavelength band is more than a predetermined threshold among the wavelength bands included in the first pixel of the first image corresponding to the second pixel, and based on interpolation for the selected wavelength band. An image visualization method for generating the preprocessed data.