JP5807111B2 - Image generation system and image generation method - Google Patents

Description

  The present invention relates to an image generation system for generating a hyperspectral image, and more particularly to a technique for generating a hyperspectral image with a high wavelength resolution from a multispectral image with a low wavelength resolution.

  Features installed on the ground (buildings, trees, roads, water bodies, etc.) reflect sunlight. A sensor mounted on an aircraft, an artificial satellite, or the like acquires reflected light from a feature, separates the acquired reflected light for each frequency band, and generates a multispectral image of a plurality of frequency bands (see FIG. 7). . Among multispectral images, spectral images with high wavelength resolution (including hundreds of narrow bands) are called hyperspectral images.

  Hyperspectral images contain many spectral features and have a high potential for identifying features, so that their importance is increasing. However, hyperspectral images are not widely used. According to current sensor technology, hyperspectral images with high wavelength resolution are obtained from aircraft, and satellites only provide multispectral images with low wavelength resolution. That is, a sensor mounted on an aircraft for generating the spectrum image has a high wavelength resolution, and thus a hyperspectral image including several hundred narrow bands can be obtained.

  Here, the difference between a general multispectral image and a hyperspectral image will be described with reference to FIG. The multispectral image includes an image of a plurality of frequency bands between purple and infrared, and the width of one frequency band is a wavelength of 40 to 100 nm. A black-and-white image is obtained by detecting visible light (about 400 to 900 nm) as one band having a width of several hundreds of nm and detecting the intensity of light within the band. A black and white image may be used in combination with the multispectral image described above. The hyperspectral image includes an image of a plurality of frequency bands between purple and infrared as in the case of the multispectral image, but the width of one frequency band is a wavelength of 20 to 50 nm, and the wavelength resolution is higher than that of the multispectral image. .

  However, a sensor mounted on an aircraft is expensive, and an aircraft flies at a low altitude as compared with an artificial satellite. Therefore, the imaging range is narrow, and the cost for imaging a wide area increases. On the other hand, although the sensor mounted on the artificial satellite has a low resolution, the artificial satellite flies at a high altitude, so that an image of a wide area can be obtained at a low cost with a wide imaging range. Therefore, one problem is that the cost of acquiring a hyperspectral image is high.

N. Keshava and J. Mustard, "Spectral unmixing", IEEE Signal Processing Magazine, vol. 19, pp. 44-57, Jan. 2002

  The features installed on the ground have different spectral characteristics of reflected light depending on the type (building, tree, road, water area, etc.). However, in a multispectral image that includes only a few measured values within the visible light band, it is difficult to determine the difference in features from the spectral characteristics of the reflected light. On the other hand, if a hyperspectral image including several tens to several hundreds of measurement values in the visible light band is used, the type of the feature can be specified. Furthermore, the kind of tree can be specified, and the growth status of the crop can be confirmed.

  For this reason, it is required to generate a hyperspectral image with a high wavelength resolution from a multispectral image taken by an artificial satellite.

  In order to solve this problem, a method of generating a hyperspectral image from a multispectral image has been proposed. Of these, the simplest method is interpolation (for example, linear interpolation). However, the spectrum obtained by linear interpolation is rough, and it is unlikely that the original characteristics of the spectrum can be found. For this reason, a spectrum unmixing technique for decomposing a measured spectrum into a plurality of spectrum components has been proposed (see Non-Patent Document 1).

  However, spectrum unmixing needs to know the number and type of feature types, but in many cases it is difficult to actually obtain the necessary information. For this reason, spectral unmixing is not sufficient to generate hyperspectral images, is unreliable and impractical.

  An object of the present invention is to reconstruct a spectrum curve by reflected light from a feature and to provide information for finding a spectrum characteristic.

  A typical example of the invention disclosed in the present application is as follows. That is, an image generation system that generates a spectral image with high wavelength resolution from an original spectral image, a calculation unit that performs a calculation for generating a spectral image, and a memory that stores a program executed by the calculation unit The calculation unit obtains a regression curve that applies to the original spectrum image, generates a spectrum image by multiplying the obtained regression curve by a response characteristic function of a sensor, and generates the spectrum image and the element Calculating a difference with a spectral image, comparing the calculated difference with a predetermined threshold, and if the result of the comparison is that the difference is less than the predetermined threshold, the response of the sensor to the determined regression curve A spectral image is generated by multiplying by a characteristic function, and if the difference is larger than the predetermined threshold as a result of the comparison, the spectral image is generated. To perform operation again spectral image as the original spectral image to generate a spectral image by multiplying the response characteristic function of the sensor to the determined regression curve.

  According to a typical aspect of the present invention, a hyperspectral image reproducing light reflected from a feature can be obtained by a simple method.

It is a block diagram which shows the structure of the image generation system of embodiment of this invention. It is a figure explaining the method to acquire the multispectral image of embodiment of this invention. It is a figure explaining the concept of the production | generation of the spectrum super-resolution image by the image generation system of embodiment of this invention. It is a figure explaining the concept of the production | generation of the spectrum super-resolution image by the image generation system of embodiment of this invention. It is a flowchart which shows the process of the image generation system of embodiment of this invention. It is a figure explaining the change of the spectrum by the process of the image generation system of embodiment of this invention. It is a figure explaining the spectrum produced | generated by the image generation system of embodiment of this invention. It is a figure explaining the method of acquiring a multispectral image. It is a figure explaining a multispectral image and a hyperspectral image.

  The main feature of the embodiment of the present invention is to create a spectrum reproducing the reflected light from the feature by using a sensor response function (SRF). Conventional methods mainly focus on images and their attributes, and the influence of sensors has been ignored. The present invention provides a method for restoring a spectrum based on a physical model of a sensor. This is an ideal continuous spectrum curve that eliminates the effects of the sensor. The hyperspectral image is generated by sampling points on the generated ideal spectral curve.

  FIG. 1 is a block diagram showing a configuration of an image generation system according to an embodiment of the present invention.

  The image generation system according to the present embodiment includes a calculation unit 10, a storage device 20, a communication interface 30, and a medium driver 40.

  The arithmetic unit 10 includes a processor (CPU) 101 that executes a program, a ROM 102 that is a nonvolatile storage element, and a RAM 103 that is a volatile storage element. The ROM 102 stores an invariant program (for example, BIOS). The RAM 103 temporarily stores a program stored in the storage device 20 and data used when the program is executed.

  The storage device 20 is a large-capacity non-volatile storage device such as a magnetic storage device or a flash memory, and stores a program executed by the processor 101 and data used when the program is executed. That is, the program executed by the processor 101 is read from the storage device 20, loaded into the RAM 103, and executed by the processor 101.

  The communication interface 30 is a network interface device that controls communication with other devices in accordance with a predetermined protocol. The medium driver 40 is an interface (for example, an optical disk drive, a USB port) for reading a recording medium 50 in which a program and data introduced into the image generation system are stored.

  Next, the concept of generating a spectral super-resolution image disclosed in the present application will be described.

  FIG. 2 is a diagram for explaining a method of acquiring the spectrum image S according to the embodiment of the present invention.

  As described above, features (buildings, trees, roads, water areas, etc.) reflect sunlight. A sensor mounted on an artificial satellite or the like acquires reflected light R from a feature, separates the acquired reflected light for each frequency band, and generates a spectrum image S including a plurality of frequency bands.

The sensitivity of a sensor that generates a spectral image varies depending on the frequency, and the change in sensitivity is expressed as a sensor response characteristic function (SRF). When this SRF is used, the spectrum image S is expressed by the equation (1).
S = R × SRF + N (1)
In Expression (1), N is noise.

  When a plurality of sensor elements are mounted, the response characteristic function is usually different for each sensor element. In this case, in a process described later, different spectral images are generated by using different response characteristic functions for each sensor element.

  3A and 3B are diagrams illustrating the concept of hyperspectral image generation by the image generation system according to the embodiment of the present invention.

  Multispectral including multiple frequency bands by photographing ground features (buildings, trees, roads, water areas, etc.) with a predetermined spatial resolution (m / pixel) and a predetermined wavelength resolution (nm / band) Get an image. Thereafter, each pixel is converted into a hyperspectral image having a frequency characteristic with a higher wavelength resolution.

  FIG. 3B shows frequency characteristics of a captured image (multispectral image) having a low wavelength resolution and frequency characteristics of a hyperspectral image having a high wavelength resolution. As shown in the figure, the hyperspectral image has high wavelength resolution, and each frequency bandwidth (wavelength width) included in the hyperspectral image is narrow.

  FIG. 4 is a flowchart showing processing of the image generation system according to the embodiment of the present invention. 5 and 6 are diagrams for explaining changes in the spectrum due to the processing shown in FIG.

  First, an image spectrum taken by an artificial satellite or the like is defined as S (0) (S1). S (0) is a discrete function, and is indicated by a solid line and ● in FIG.

  Thereafter, a curve is fitted to the measured value of S (0) to obtain a regression curve R (0) (S2). R (0) is given by a continuous function and is indicated by a broken line in FIG. At this time, the parameter i is 0. For the curve fitting, a polynomial curve fitting by the least square method can be used. S (i) is a discrete value (discrete function) of the image spectrum, and R (i) is a spectrum value (continuous function) of the reflected light.

Thereafter, after adding 1 to the parameter i (S3), the image spectrum S (1) is calculated using the equation (2) (S4).
S (i) = R (i−1) × SRF (2)
In Expression (2), SRF is a response characteristic function of the sensor, and indicates a frequency characteristic of the sensitivity of the sensor within the band of the image spectrum S (i).

Next, a difference E (i) from the previous calculated value is calculated using Equation (3) (S5).
E (i) = S (i) -S (i-1) (3)
In FIG. 5, E (0) is indicated by an interval between ● and ▲.

  Thereafter, the calculated difference E (i) is compared with a predetermined threshold (Error) (S6). As a result, if the difference E (i) is smaller than a predetermined threshold (Error), it is determined that the image spectrum S (i) has sufficiently converged, and the regression curve R (i) is finally obtained as a curve C. (S9). In FIG. 6, a curve C is a continuous function indicated by a broken line and has a wavelength resolution higher than that of the image spectrum S (0). That is, since the curve C is a continuous function, a spectrum value (signal intensity) can be obtained at any wavelength.

  Then, a hyperspectral image (Image) can be obtained by multiplying the determined curve C by SRF (S10).

On the other hand, if the difference E (i) is greater than or equal to a predetermined threshold (Error), it is determined that the image spectrum S (i) has not yet converged, and the difference E (i) is calculated using Equation (4). A value multiplied by a predetermined coefficient K is added to the image spectrum S (i−1) to obtain the image spectrum S (i) (S7).
S (i) = S (i−1) + K × E (i) (4)
The coefficient K may be a value greater than 0 and less than or equal to 1. By selecting a large value for the coefficient K, the convergence calculation can be accelerated and the amount of calculation can be reduced. On the other hand, by selecting a small value for the coefficient K, the accuracy of the obtained hyperspectral image (Image) can be improved.

  In FIG. 5, KE (0) is indicated by the interval between ● and ◆. In addition, although illustration is abbreviate | omitted, image spectrum S (1) is a discrete function which connected *.

  Thereafter, a regression curve R (i) is obtained by fitting a curve to each value included in S (i) (discrete function S (1)) by the same method as in step S2 (S8). In FIG. 5, R (1) is a continuous function indicated by a one-dot chain line. Thereafter, the process returns to step S3, and the next loop is executed.

  The hyperspectral image generated by the present embodiment can restore a sufficient characteristic of the spectrum due to the reflected light from the feature. Usually, no prior preparation is required except for SRF provided with multi-spectral images from the provider. Therefore, this embodiment is practical and provides a highly reliable hyperspectral image. Until now, only hyperspectral images could be used to find spectral features, but this embodiment generates hyperspectral images from multispectral images, so spectral features from multispectral images. Provides a new way to find out.

  In addition, the method of the embodiment of the present invention can be performed by a program executed on a stand-alone system (preferably a server computer). For example, an operator that provides data obtained by remote sensing can provide a service for generating and analyzing a hyperspectral image.

  Although the present invention has been described in detail with reference to the accompanying drawings, the present invention is not limited to such specific configurations, and various modifications and equivalents within the spirit of the appended claims Includes configuration.

Claims (6)

An image generation system for generating a spectral image with high wavelength resolution from an original spectral image,
A calculation unit for performing calculation for generating a spectrum image;
A memory for storing a program executed by the arithmetic unit,
The computing unit is
Find a regression curve that applies to the original spectral image;
A spectral image is generated by multiplying the obtained regression curve by the response characteristic function of the sensor,
Calculating the difference between the generated spectral image and the original spectral image;
Comparing the calculated difference with a predetermined threshold;
As a result of the comparison, when the difference is smaller than the predetermined threshold, a spectral image is generated by multiplying the obtained regression curve by a response characteristic function of the sensor,
If the difference is greater than the predetermined threshold as a result of the comparison, the calculated regression curve is multiplied by the response characteristic function of the sensor in order to perform the calculation again using the generated spectrum image as the original spectrum image. And generating a spectral image. The image generation system according to claim 1,
The said calculating part produces | generates the original spectrum image for performing the said calculation again by adding the value which multiplied the predetermined error to the said calculated error to the said original spectrum image, The image generation system characterized by the above-mentioned. The image generation system according to claim 1,
The sensor has a plurality of sensor elements having different response characteristic functions,
The image generation system characterized in that the calculation unit generates different spectral images using different response characteristic functions for the sensor elements. An image for generating a spectral image with high wavelength resolution from an original spectral image using an image generation system having a calculation unit for performing calculation for generating a spectral image and a memory for storing a program executed by the calculation unit A generation method,
A first step of determining a regression curve that applies to the original spectral image;
A second step of generating a spectral image by multiplying the obtained regression curve by a response characteristic function of a sensor;
A third step of calculating a difference between the generated spectral image and the original spectral image;
A fourth step of comparing the calculated difference with a predetermined threshold;
As a result of the comparison, when the difference is smaller than the predetermined threshold, a fifth step of generating a spectrum image by multiplying the obtained regression curve by a response characteristic function of the sensor;
If the difference is greater than the predetermined threshold as a result of the comparison, the calculated regression curve is multiplied by the response characteristic function of the sensor in order to perform the calculation again using the generated spectrum image as the original spectrum image. And a sixth step of generating a spectrum image. The image generation method according to claim 4,
The sixth step includes generating an original spectrum image for performing the calculation again by adding a value obtained by multiplying the calculated error to a predetermined count to the original spectrum image. . The image generation method according to claim 4,
The sensor has a plurality of sensor elements having different response characteristic functions,
In the second step and the sixth step, a different response characteristic function is used for each sensor element.

Images