JP6208076B2 - Image processing apparatus, image processing method, and recording medium recording program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a recording medium on which a program is recorded, and relates to an image processing apparatus, an image processing method, and a program for processing an optical satellite image obtained by a sensor mounted on a flying object such as an artificial satellite. It is suitable for application to a recorded recording medium.

  An optical satellite image obtained by a sensor mounted on a flying object such as an artificial satellite includes a high-resolution monochrome image (hereinafter, the monochrome image will be referred to as a panchromatic image) and a low-resolution color image ( Hereinafter, the color image will be referred to as a multispectral image, and the multispectral image stores color information divided into layers for each observation wavelength band such as a blue band, a green band, a red band, and an invisible band. is there. Traditionally, the pan-sharpening process creates a pan-sharpened image with high resolution and color information by synthesizing the color of the multispectral image with the pixel values of the panchromatic image using the features of both images. Is done.

  For example, Patent Document 1 discloses performing pan sharpening processing by aligning radar image data approximated to a panchromatic image and an optical satellite image captured by an optical satellite. According to Patent Literature 1, it is possible to generate a composite image in which features can be easily distinguished regardless of the weather.

  In Patent Document 2, high-resolution color radar image data having color information can be obtained from low-resolution multi-polarization radar image data to which color information is assigned and high-resolution single-polarization radar image. It is disclosed. According to Patent Document 2, it is possible to acquire information including features and shapes of features with high resolution and color, and therefore feature information can be read with high accuracy.

JP 2009-47516 A JP 2013-96807 A

  However, due to the difference in image characteristics between the panchromatic image and the multispectral image, there is a problem that the color of the pan-sharpened image may be different from that of the multispectral image. Specifically, when pan-sharpening processing is performed, if the effect of atmospheric scattering occurs in the multispectral image or if minute pixel values occur in the background area other than the scene, the color of the feature will change. A pan-sharpened image different from the original color will be generated. A panchromatic image is an image that combines the intensities of light observed not only in the wavelength band of visible light but also in the wavelength band including the wavelength band of invisible light. When combined with the color observed in the wavelength band, an unnaturally bright pan-sharpened image is generated in the region where the invisible light is strong.

  The present invention has been made in consideration of the above points, and an image processing apparatus and an image processing method capable of generating a pan-sharpened image in consideration of a difference in image characteristics between a panchromatic image and a multispectral image And a recording medium on which the program is recorded.

In order to solve such a problem, the present invention provides an image processing apparatus for processing an optical satellite image taken from a flying object, wherein the panoramic image overlaps a multispectral image of the optical satellite image and a region of the multispectral image. An input unit for reading a chromatic image, a noise removal unit for removing pixels corresponding to a predetermined range of pixel value distribution of the multispectral image and the pixel value, and a minimum pixel of the image from which noise has been removed by the noise removal unit Based on the ratio of each band of the multispectral image and the atmospheric scatter correction unit that subtracts the pixel value of the multispectral image and the value as the atmospheric scattering component, the influence of invisible light from the pixel value of the panchromatic image An observation wavelength bandwidth correction unit that performs correction to reduce, and the multis corrected by the atmospheric scattering correction unit. With a pan-sharpened processing unit to create a pan-sharpened image by combining said panchromatic image corrected by vector image and the observation wavelength band width correction unit, wherein the noise removing unit, the color image A histogram showing the distribution of pixel values in each observation wavelength band is created, the pixel value corresponding to the distribution of pixel values isolated in the low value of the histogram in each observation wavelength band is set to 0, and the atmospheric scattering correction unit is The minimum pixel value of each observation wavelength band of the color image in which the pixel value corresponding to noise is set to 0 by the noise removing unit, and the minimum pixel of each observation wavelength band is determined from the pixel value of each observation wavelength band of the color image. The observation wavelength bandwidth correction unit subtracts a value to obtain a spectral intensity for each wavelength band by multiplying the color image by a ratio of the observation wavelength bandwidth specific to the sensor. The pixel value, characterized that you multiply the ratio of the pixel value of the observed wavelength bandwidth of visible light other than visible light to the pixel values of all observations wavelength band width of the color image, the image processing apparatus provided Is done.

  According to the present invention, high-definition is generated by generating a pan-sharpened image in consideration of the difference in image characteristics between a panchromatic image (monochrome image of an optical satellite image) and a multispectral image (color image of an optical satellite image). Thus, an image having clear color information can be provided.

It is a block diagram which shows the hardware constitutions of the image processing apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows the program and data which are stored in the memory | storage device concerning the embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the image processing apparatus according to the embodiment. It is a chart which shows an example of the noise parameter preservation | save part concerning the embodiment. It is a graph which shows an example of the observation wavelength band data storage part concerning the embodiment. It is a flowchart which shows the outline | summary of the image processing concerning the embodiment. It is a flowchart which shows the noise removal process concerning the embodiment. It is a flowchart which shows the increase / decrease determination process of the frequency concerning the embodiment. It is a flowchart which shows the determination process of the increase in the differential value frequency concerning the embodiment. It is a flowchart which shows the atmospheric scattering correction | amendment process concerning the embodiment. It is a flowchart which shows the observation wavelength bandwidth correction process concerning the embodiment. It is a flowchart which shows the pan sharpening process concerning the embodiment. It is a conceptual diagram explaining the example of a display of the pan sharpen image concerning the embodiment.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) Outline of the present embodiment Conventionally, pan sharpening processing for creating a pan sharpened image having high resolution and color information using the features of both images has been performed. Due to the difference in image characteristics from the multispectral image, there is a problem that the color of the pan-sharpened image may be different from that of the multispectral image. Hereinafter, three problems in the pan sharpening process will be described.

  As a first problem, there is a problem that the color of the pan-sharpened image is affected when the influence of atmospheric scattering occurs in the multispectral image. When the influence of atmospheric scattering occurs, the color with a shorter wavelength (blue) blurs, so that the multispectral image is an overall bluish image. For this reason, when a multispectral image affected by atmospheric scattering and a panchromatic image are synthesized, the feature color in the pan-sharpened image may not be the original color.

  As a second problem, a satellite image is composed of a foreground area where a scene is reflected and a background area other than that, but there is a problem that minute pixel values in the background area affect the color of the pan-sharpened image. In the satellite image, the background pixel value is set to 0 in order to distinguish the foreground and the background. However, a minute pixel value other than 0 may be generated in the background region due to the influence of noise generated in the characteristics of the sensor and image processing. Since the pan sharpening process is performed only on the foreground area, the statistical value used for the pan sharpening process needs to be calculated only from the foreground area. However, when a minute pixel value other than 0 is generated in the background area, the pixel value is processed as a part of the foreground area even though it is originally the background area. For this reason, the calculation of statistical values may not be performed correctly, which may affect the color of the pan-sharpened image.

  As a third problem, since the observation wavelength bands of the panchromatic image and the multispectral image are different, there is a problem that an area where the invisible light is strong in the pan-sharpened image becomes unnaturally bright. A panchromatic image is an image that combines the intensities of light observed not only in the visible light wavelength band but also in the wavelength band including the invisible light wavelength band. When the pixel values of the panchromatic image are combined, the intensity of the light observed in the invisible light wavelength band is also combined. For this reason, the region (for example, vegetation) where the invisible light is strong in the pan-sharpened image may become unnaturally bright.

  Therefore, in the present embodiment, by solving the above-described problem, it is possible to generate a pan-sharpened image having high-definition and clear color information that matches the color of the original feature.

  Specifically, as a method for solving the first problem, the influence of atmospheric scattering is reduced by using the minimum pixel value of the multispectral image. None of the conventionally proposed methods for reducing the influence of atmospheric scattering from optical satellite images has to set an atmospheric model for each image, and cannot be applied uniformly to various images. Therefore, in the present embodiment, the minimum value of the pixel value in the foreground is regarded as the atmospheric scattering component, and the atmospheric scattering effect is reduced without setting the atmospheric model for each image by subtracting the atmospheric scattering component from the entire foreground. It is possible to reduce.

  That is, the minimum pixel value in the foreground is considered to be a value such as a shadow that originally has a pixel value of 0, but has a pixel value greater than 0 due to atmospheric scattering. For this reason, the minimum value of the pixel values in the foreground, that is, the bottom value of the foreground histogram of the multispectral image is regarded as the atmospheric scattering component. Then, the atmospheric scattering component is subtracted from the pixel value of the multispectral image, and the histogram of the multispectral image is shifted to the lower pixel value by the atmospheric scattering component. Thus, since the bottom value of the histogram is the atmospheric scattering component, the influence of atmospheric scattering on the multispectral image can be reduced without setting an atmospheric model for each image.

  As a method for solving the second problem, the influence of noise generated in the background is removed using the distribution of pixel values of the multispectral image. It is known that the pixel value of noise generated in the background is extremely lower than the pixel value of the foreground. For this reason, in the frequency distribution of pixel values of an image including noise, the frequency due to noise is distributed in a range where the pixel value is small, and the frequency due to the foreground pixels is distributed in a range where the pixel value is large. When calculating the statistical value using the characteristics of this distribution, the influence of noise can be removed by excluding the pixels corresponding to the distribution in the range of small pixel values from the calculation target.

  Further, as a method for solving the third problem, the influence of invisible light is reduced by multiplying the panchromatic image by the ratio of the spectral intensity for each wavelength band of the multispectral image. For this purpose, first, the pixel value of the multispectral image normalized to a certain number of bits is multiplied by the observation wavelength bandwidth, thereby converting the pixel value into a spectral intensity. The pixel value of the multispectral image is a value obtained by normalizing the spectral intensity to a fixed number of bits for each observation wavelength band for the convenience of storing it in an electronic file. It is necessary to multiply the normalized pixel value by the observation wavelength bandwidth to convert the pixel value into a spectral intensity. Then, the panchromatic image is multiplied by the ratio of the total spectral intensity of the visible light wavelength band of the multispectral image to the total spectral intensity of the visible light and invisible light wavelength bands, and the panchromatic image This reduces the influence of invisible light on the pixel value. The same effect can be obtained not by multiplying the pixel value by the observation wavelength bandwidth but by obtaining the spectral intensity ratio by multiplying the pixel value by the observation wavelength bandwidth ratio. .

(2) Hardware Configuration of Image Processing Device Next, the hardware configuration of the image processing device 10 will be described with reference to FIG. As shown in FIG. 1, the image processing apparatus 10 mainly includes a CPU (Central Processing Unit) 101, an auxiliary storage device 102, a storage device 103, an input / output device 104, and a display device 105.

  The CPU 10 functions as an arithmetic processing device and a control device, and controls the overall operation in the image processing device 10 according to various programs.

  The auxiliary storage device 102 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The ROM stores programs and calculation parameters used by the CPU 10, and the RAM primarily stores programs used in the execution of the CPU 10, parameters that change as appropriate in the execution, and the like. These are connected to each other by a host bus including a CPU bus.

  The storage device 103 can include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 103 is composed of, for example, an HDD (Hard Disk Drive). The storage device 103 drives the HDD and stores programs executed by the CPU 10 and various data. Further, the storage device 103 stores image data and the like which will be described later.

  The input / output device 104 is a device that transmits / receives data to / from various devices, and connects to a drive that reads information recorded in the storage device 103, a connection port that is an interface connected to an external device, or a network. For example, a communication device.

  The display device 105 includes a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Display) device, and a lamp. For example, the display device 105 displays the combined pan-sharp image on the display device.

  As shown in FIG. 2, the auxiliary storage device 102 stores a plurality of programs having various functions. Specifically, the auxiliary storage device 102 includes an input unit 111, a histogram creation unit 112, a histogram smoothing unit 113, a frequency increase / decrease determination unit 114, a histogram differentiation unit 115, a differential value / frequency increase determination unit 116, Noise removal image processing unit 117, output unit 118, minimum pixel value calculation unit 119, atmospheric scattering correction image processing unit 120, observation wavelength band correction image processing unit 121, invisible band correction image processing unit 122, visible band pan sharpening processing unit 123, programs that function as the invisible band pan sharpening processing unit 124 and the pixel value range optimization processing unit 125 are stored. The function of each part will be described in detail later.

  The storage device 103 stores various data. Specifically, the storage device 103 stores an image data storage unit 131, a noise parameter storage unit 132, and an observation wavelength band data storage unit 133. Data stored in each unit will be described in detail later.

(3) Functional Configuration of Image Processing Device Next, a functional configuration of the image processing device 10 will be described with reference to FIG. Various programs stored in the auxiliary storage device 102 are used to solve the above three problems. Therefore, in FIG. 3, the functional units for solving the three problems are generically named as follows.

  That is, the functional unit that reduces the influence of atmospheric scattering, which is the first problem, is referred to as an atmospheric scattering correction unit 152. Further, a functional unit that removes the influence of noise, which is the second problem, is referred to as a noise removing unit 151. Further, a function unit that reduces the influence of the wavelength band of invisible light, which is the third problem, is referred to as an observation wavelength band correction unit 153.

  An overview of each function will be described. First, the input unit 111 acquires a panchromatic image and a multispectral image stored in the image data storage unit 131 and provides them to the noise removal unit 151.

  The noise removal unit 151 includes a histogram creation unit 112, a histogram smoothing unit 113, a frequency increase / decrease determination unit 114, a histogram differentiation unit 115, a differential value / frequency increase determination unit 116, and a noise removal image processing unit 117. The

  In the noise removal unit 151, the histogram creation unit 112 creates a histogram for the panchromatic image and multispectral image input from the input unit 111, and the histogram smoothing unit 113 smoothes the histogram. Then, noise is removed from the panchromatic image and the multispectral image using the result of determining the increase or decrease in the frequency of the smoothed histogram. This will be described in detail below.

  The histogram creation unit 112 creates a histogram H (i) having the pixel value i of the band to be processed of the panchromatic image and the multispectral image as a class. Here, the histogram of an image is a graph showing the frequency (number of pixels) of each gray level in a certain image. That is, the histogram H (i) of the multispectral image is represented by a graph in which the horizontal axis is the pixel value and the vertical axis is the pixel frequency. The histogram creation unit 112 provides the created histogram to the histogram smoothing unit 113.

  Then, the histogram smoothing unit 113 creates a smoothed histogram Hs (i) by smoothing the histogram created by the histogram creating unit 112 using the following equation (1). Here, the smoothing of the histogram is to perform conversion so that the frequency of the histogram becomes a weighted average in a certain class range of the frequency of the original histogram. By smoothing the histogram, the frequency fluctuation can be smoothed. When the frequency distribution of noise is divided into a plurality of cases, all of them may not be detected as noise. By combining the frequency distribution of noise divided into a plurality of smooth frequency variations, all of them can be detected as noise. The histogram smoothing unit 113 provides the smoothed histogram to the frequency increase / decrease determination unit 114.

  Here, the weights w 1, w 2, w 3, w 4, and w 5 in Expression (1) are stored in the noise parameter storage unit 132. FIG. 4 is an example of the noise parameter 1320 stored in the noise parameter storage unit 132. As shown in FIG. 4, the noise parameter 1320 includes a threshold used for frequency determination by the frequency increase / decrease determination unit 114, the differential value / frequency increase determination unit 116, and the like.

  The frequency increase / decrease determination unit 114 uses the histogram smoothed by the histogram smoothing unit 113 to determine whether the frequency is increasing or decreasing in the order of the pixel values. That is, the frequency increase / decrease determination unit 114 determines the frequency increase / decrease sequentially one by one from the class of the pixel value i = 0 of the histogram Hs (i) to the maximum pixel value i_noise_max that the noise can take. The pixel value i when first exceeding a predetermined threshold value h is assumed to be i0. Here, the maximum pixel value i_noise_max of noise is included in the noise parameter 1320 of the noise parameter storage unit 132 shown in FIG. Then, Hs (i) is determined in order from i = i0, and the pixel value i when first falling below a predetermined threshold value h is set to i1. The histogram smoothing unit 113 provides the smoothed histogram to the histogram differentiating unit 115. Here, the predetermined threshold value h is stored in the noise parameter storage unit 132.

  The histogram differentiator 115 calculates a differential Hs ′ (i) with respect to i of the histogram H (i) with a predetermined class width di. Here, the predetermined class width di is stored in the noise parameter storage unit 132. The histogram differentiation unit 115 provides the differentiated histogram to the differential value / frequency increase determination unit 116.

  Using the histogram Hs ′ (i) and the histogram Hs (i) differentiated by the histogram differentiator 115, the differential value / frequency increase determiner 116 sequentially determines the increase in frequency from i = i1. Specifically, the first i in which Hs ′ (i) exceeds a predetermined threshold hp and Hs (i) exceeds a predetermined h is defined as a noise value i_noise. Here, the predetermined threshold value hp is also included in the noise parameter 1320 of the noise parameter storage unit 132 shown in FIG. The differential value / frequency increase determination unit 116 provides the noise value i_noise to the noise removal image processing unit 117.

  The noise-removed image processing unit 117 inspects all pixel values of the image, overwrites the pixel value lower than i_noise as the noise value with 0, and removes the influence of noise in the panchromatic image and the multispectral image. . As described above, in the optical satellite image, the background pixel value is set to 0 in order to distinguish the foreground and the background. Therefore, by setting the pixel value lower than i_noise as the noise value to 0, it is possible to remove noise generated in the background area. The noise removal image processing unit 117 provides the panchromatic image and multispectral image from which noise has been removed to the minimum pixel value calculation unit 119.

  The atmospheric scattering correction unit 152 includes a minimum pixel value calculation unit 119 and an atmospheric scattering correction image processing unit 120. The atmospheric scattering correction unit 152 regards the minimum pixel value calculated by the minimum pixel value calculation unit 119 as an atmospheric scattering component, and reduces the influence of atmospheric scattering by subtracting the atmospheric scattering component from the entire image. This will be described in detail below.

  The minimum pixel value calculation unit 119 calculates the minimum pixel value i_min of the foreground of the panchromatic image and the multispectral image provided from the noise removal image processing unit 117. Since the pixel value of the background of the panchromatic image and the multispectral image is set to 0 by the noise-removed image processing unit 117, the minimum pixel value i_min of the panchromatic image and the multispectral image is the minimum value of the pixel values excluding 0. The minimum pixel value calculation unit 119 provides the atmospheric scatter correction image processing unit 120 with the minimum pixel value i_min of the panchromatic image and the multispectral image.

  The atmospheric scattering corrected image processing unit 120 subtracts the minimum pixel value i_min from the pixel value of each band of the multispectral image to be processed. In this embodiment, it is assumed that the minimum pixel value in the foreground is originally a pixel value of 0, such as a shadow, and has a pixel value greater than 0 due to atmospheric scattering, and the minimum class of the histogram is It is regarded as an atmospheric scattering component. Therefore, the histogram of each band is shifted to the lower pixel value by the minimum class regarded as the atmospheric scattering component, that is, by subtracting the minimum pixel value i_min from the pixel value of each band, The influence of atmospheric scattering components can be reduced. The atmospheric scattering correction image processing unit 120 provides the corrected multispectral image to the observation wavelength band correction image processing unit 121.

  The observation wavelength band correction unit 153 includes an observation wavelength band correction image processing unit 121 and an invisible band correction image processing unit 122. The observation wavelength band correction unit 153 performs processing for reducing the influence of invisible light on the panchromatic image in order to reduce the influence of the difference in wavelength band between the panchromatic image and the multispectral image. This will be described in detail below.

  The observed wavelength bandwidth correction image processing unit 121 sets the observed wavelength bandwidths of the blue band, the green band, the red band, and the invisible band of the multispectral image as λb, λg, λr, and λn, respectively. Is multiplied by λb, all pixel values in the green band are multiplied by λg, all pixel values in the red band are multiplied by λr, and all pixel values in the invisible band are multiplied by λn. That is, the spectral intensity for each wavelength band of the multispectral image is obtained by multiplying the pixel value of the multispectral image by the ratio of the observation wavelength band specific to the sensor that captures the multispectral image.

  Here, the observation wavelength band unique to the sensor is stored in the observation wavelength band data storage unit 133. FIG. 5 is an example of observation wavelength band data 1330 stored in the observation wavelength band data storage unit 133. As shown in FIG. 5, the observation wavelength band data 1330 stores parameters for the observation wavelength band of each band of the multispectral image. For example, the observation wavelength bandwidth parameter for the blue band is 60, the observation wavelength bandwidth parameter for the green band is 70, the observation wavelength bandwidth parameter for the red band is 35, and the observation wavelength bandwidth parameter for the invisible band is 140. is there. The observation wavelength band parameter may be stored in the observation wavelength band data storage unit 133 as a parameter corresponding to the type of sensor mounted on the flying object.

  The observation wavelength bandwidth correction image processing unit 121 multiplies all pixel values of the multispectral image by the parameter of each observation wavelength bandwidth. The observation wavelength band corrected image processing unit 121 provides the corrected panchromatic image to the invisible band corrected image processing unit 122.

  The invisible band corrected image processing unit 122 performs correction for reducing the influence of invisible light on the panchromatic image. Specifically, the pixel value of the panchromatic image is Pa, the pixel value of the blue band of the multispectral image is B, the pixel value of the green band is G, the pixel value of the red band is R, the pixel value of the invisible band is N, The pixel value Pa ′ of the panchromatic image corrected by the following equation (2) is calculated. The invisible band corrected image processing unit 122 provides the corrected panchromatic image to the pan sharpening processing unit 154.

  The pan sharpening processing unit 154 includes a visible band pan sharpening processing unit 123, an invisible band pan sharpening processing unit 124, and a pixel value range optimization processing unit 125. The pan-sharpening processing unit 154 generates a pan-sharpened image by combining the panchromatic image and the multispectral image processed by the noise removing unit 151, the atmospheric scattering correction unit 152, and the observation wavelength band correction unit 153. . This will be described in detail below.

  The visible band pan-sharpening processing unit 123 calculates the pixel value of the visible band of the pan-sharpened image. Specifically, the pixel value Pb of the blue band, the pixel value Pg of the green band, and the pixel value Pr of the red band are calculated by the following equation (3). The pixel value of the panchromatic image corrected by the invisible band correction image processing unit 122 is Pa ′, the pixel value of the blue band of the multispectral image is B, the pixel value of the green band is G, and the pixel value of the red band is R. It is said.

  Note that the following equation (3) ′ is obtained by substituting equation (2) into the above equation (3).

  The pixel value Pb of the pan-sharpened image in the blue band, the pixel value Pg of the pan-sharpened image in the green band, and the pixel value Pr of the pan-sharpened image in the red band may be calculated by the above equation (3) ′. By doing so, the step of calculating the equations (2) and (3) can be made only to the step of calculating the equation (3), and therefore the pan-sharpened image creation process can be speeded up. .

  The invisible band pan-sharpening processing unit 124 calculates the pixel value Pn of the invisible band of the pan-sharpened image by the following equation (4). The pixel value of the panchromatic image is Pa, the pixel value of the blue band of the multispectral image is B, the pixel value of the green band is G, the pixel value of the red band is R, and the pixel value of the invisible band is N.

  The expressions (3) ′ and (4) differ only in whether the band term to be processed is blue, green, red, or invisible, and the calculation is common to the other terms. For this reason, you may integrate into one calculation process. By integrating into one calculation process, it is not necessary to determine which one of formulas (3) ′ and (4) is used for each band to be processed, and the pan-sharpened image creation process is accelerated. be able to.

  The pixel value range optimization processing unit 125 normalizes the range of pixel values Pb, Pg, Pr, and Pn of the pan-sharpened image to a range of values that can be output. Specifically, the pixel value range optimization processing unit 125 performs each pixel value Pb, Pg, Pr of the pan-sharpened image according to the number of gradations (16 bits, 8 bits, etc.) of the display device 105 such as a display or a printer. Normalize the range of Pn.

(4) Details of Image Processing in Image Processing Device (4-1) Overview of Image Processing Next, details of image processing in the image processing device 10 will be described with reference to FIGS. First, an outline of image processing according to the present embodiment will be described with reference to FIG.

  As shown in FIG. 6, the image processing apparatus 10 first reads a satellite image from a sensor mounted on a flying object by the input unit 111 (S101).

  Then, the noise removing unit 151 removes noise from the image read in step S101 (S102). The process of step S102 will be described later in detail.

  Next, the atmospheric scattering correction unit 152 performs a spectrum correction process based on the influence of atmospheric scattering on the image from which noise has been removed in step S102 (S103). The process of step S103 will be described in detail later.

  Subsequently, the observation wavelength band correction unit 153 performs spectrum correction processing based on the observation wavelength band on the image whose spectrum is corrected based on the influence of atmospheric scattering in step S103 (S104). The process of step S104 will be described in detail later.

  Then, the pan-sharpening processing unit 154 performs pan-sharpening processing on the image whose spectrum is corrected based on the observation wavelength band in step S104 (S105). The process of step S105 will be described in detail later.

(4-2) Details of Noise Removal Processing Next, details of the noise removal processing performed by the noise removal unit 151 executed in step S102 shown in FIG. 6 will be described with reference to FIGS. As described above, the noise removal processing shown in FIG. 7 is executed by each unit included in the noise removal unit 151. Specifically, first, the histogram creation unit 112 creates a histogram H (i) with the pixel value i of the band to be processed of the multispectral image as a class (S111). As described above, the histogram H (i) created by the histogram creation unit 112 is an image in which the pixel value i of the band to be processed of the multispectral image is a class, the horizontal axis is the pixel value, and the vertical axis is the frequency. It is a graph which shows the frequency (number of pixels) of each shading level in the inside.

  Then, the histogram smoothing unit 113 smoothes the histogram created by the histogram creating unit 112 in step S111 using the above equation (1) (S112). Specifically, the histogram smoothing unit 113 creates a smoothed histogram Hs (i) obtained by smoothing the histogram H (i) according to the above equation (1).

  Subsequently, the frequency increase / decrease determination unit 114 performs frequency increase / decrease determination processing shown in FIG. 8 using the smoothed histogram Hs (i) smoothed by the histogram smoothing unit 113 (S113). .

  With reference to FIG. 8, the determination process of the increase or decrease in the frequency of the pixel value in step S113 will be described. As shown in FIG. 8, the frequency increase / decrease determination unit 114 moves the pointer to the first class of the smoothed histogram (S121).

  Then, the frequency increase / decrease determination unit 114 acquires the frequency of the class pointed to by the pointer in step S121 (S122).

  The frequency increase / decrease determination unit 114 determines whether the frequency acquired in step S122 is smaller than a predetermined threshold and has not reached the last class (S123). Here, the predetermined threshold is the threshold h of the histogram of the noise parameter 1320 shown in FIG.

  In step S123, if it is determined that the acquired frequency is smaller than the predetermined threshold value, has not reached the class of the maximum pixel value of noise, and has not reached the final class, The increase / decrease determination unit 114 moves the pointer to the next higher class, and returns to the process of step S122. That is, the frequency increase / decrease determination unit 114 repeats the processing from step S122 to step S124 while adding the frequency of the class one by one until the frequency of the class pointed to by the pointer increases and exceeds the threshold value h.

  On the other hand, in step S123, if the acquired frequency is larger than the predetermined threshold value h, has not reached the class of the maximum pixel value of noise, and has not reached the final class, the frequency is increased. The decrease determination unit 114 moves the pointer to the next higher class i0 (S125), and acquires the frequency of the class i0 pointed to by the pointer (S126).

  Then, the frequency increase / decrease determination unit 114 has the frequency acquired in step S126 greater than the threshold value h, has not reached the class of the maximum pixel value of noise, and has not reached the last class. Is determined (S127).

  If it is determined in step S127 that the frequency acquired in step S126 is greater than the threshold value h, has not reached the class of the maximum pixel value of noise, and has not reached the last class, The frequency increase / decrease determination unit 114 moves the pointer to the next higher class (S128), and returns to the process of step S126. That is, the frequency increase / decrease determination unit 114 repeats the processing from step S126 to step S128 while adding the class frequencies one by one until the class frequency pointed to by the pointer decreases and falls below the threshold value h.

  On the other hand, in step S127, if the acquired frequency is smaller than the predetermined threshold value h, has not reached the class of the maximum pixel value of noise, and has not reached the last class, the frequency is increased. The decrease determination unit 114 records the class i1 pointed to by the pointer (S129).

  As described above, when noise is included in an image, the distribution of pixel values of the image including noise is drawn as a histogram that rises independently with only a low peak due to noise. Therefore, by performing the determination process in step S123 while raising the class one by one in steps S122 to 124, it is possible to find the starting point of the distribution that rises independently with a low value. Further, by performing the determination process in step S127 while increasing the class one by one in steps SS126 to S128, it is possible to find an end point of a distribution that rises isolated with a low value.

  In the determination processing in step S123 or step S127, it is also determined that the maximum pixel value class that noise can take has not been reached. Thereby, when the frequency due to noise is not included in the histogram, it is possible to prevent the foreground from being erroneously removed as noise.

  Further, since the class pointed to by the pointer recorded in step S129 is the end point of the noise peak, it is possible to remove noise from the image by removing pixels below the class.

  Returning to FIG. 7, after the processing of step S113, the histogram differentiator 115 calculates a differential Hs ′ (i) with respect to i of the histogram Hs (i) with a predetermined class width di, and calculates the histogram Hs (i). Is differentiated (S114). The class width used at the time of differentiation in step S114 is stored in the noise parameter storage unit 1320.

  Then, the differential value / frequency increase determination unit 116 uses the smoothed histogram Hs (i) smoothed in step S112 and the differential histogram Hs ′ (i) differentiated in step S114 to use the differential value shown in FIG. The frequency increase determination process is performed (S115).

  With reference to FIG. 9, the determination process of the increase in the differential value / frequency in step S115 will be described. As shown in FIG. 9, the differential value / frequency increase determination unit 116 moves the pointer 1 to the class i1 where the smoothed histogram Hs (i) is recorded (S131), and the differential histogram Hs ′ (i) is recorded. The pointer 2 is moved to the new class i1 (S132). In step S131 and step S132, the recorded class i1 is the class i1 recorded in step S129 described above.

  In step S131 and step S132, the class i1 recorded in step S129 is set as the start position of the pointer for performing the increase determination of the differential value / frequency, and the end position of the frequency distribution of the pixel value detected as noise. You can move the pointer.

  Then, the differential value / frequency increase determination unit 116 acquires the frequency 1 of the class pointed to by the pointer 1 (S133). Further, the differential value / frequency increase determination unit 116 acquires the frequency 2 of the class indicated by the pointer 2 (S134).

  Then, the differential value / frequency increase determination unit 116 has the frequency 1 acquired in step S133 smaller than the threshold 1 and the frequency 2 acquired in step S134 is smaller than the threshold 2 and has reached the last class. It is determined whether there is any (S135). Here, the threshold value 1 is the above-described threshold value h, and the threshold value 2 is the threshold value hp.

  In step S135, when it is determined that the frequency 1 acquired in step S133 is smaller than the threshold value 1, the frequency 2 acquired in step S134 is smaller than the threshold value 2, and the final class has not been reached, The differential value / frequency increase determination unit 116 moves the pointer 1 to the next higher class (S136), and moves the pointer 2 to the next higher class (S137).

  On the other hand, if it is determined in step S135 that the frequency 1 acquired in step S133 is greater than or equal to the threshold value 1, the frequency 2 acquired in step S134 is greater than or equal to the threshold value 2, or the final class has been reached, The frequency increase determination unit 116 records the class pointed to by the pointer 1 (S138).

  The differential value / frequency increase determination unit 116 increases the frequency of the class pointed to by the pointer 1 and the pointer 2 and exceeds the threshold value 1 or threshold value 2 while adding the frequency of each class one by one. The process of S137 is repeated.

  In step S135, when the frequency of the pointer 1 exceeds the threshold 1 or the frequency of the pointer 2 exceeds the threshold 2, the differential value / frequency increase determination unit 116 records the class indicated by the pointer 1 (S138). . By the processing in step S135, the rising position of the foreground pixel value distribution that appears after the low value peak that is noise in the smoothed histogram Hs (i), that is, the minimum pixel value i_noise that is not affected by noise. Can be found.

  Returning to FIG. 7, after the process of step S115, the noise-removed image processing unit 117 inspects all the pixel values of the image, and overwrites the pixel value lower than the pixel value i_noise recorded as the noise value with 0. Then, the influence of the noise of the multispectral image is removed (S116). Here, the pixel value i_noise recorded as the noise value is a pixel value corresponding to the class recorded in step S135 of FIG. 9 described above.

  Then, the noise removing unit 151 determines whether or not the processing of Step S111 to Step S116 has been performed for all the bands of the panchromatic image and the multispectral image (S117). If all the bands of the panchromatic image and the multispectral image are processed in step S117, the noise removing unit 151 ends the noise removing process. On the other hand, if all the bands of the panchromatic image and the multispectral image have not been processed in step S117, the noise removing unit 151 applies to the panchromatic image or the band of the multispectral image that has not been processed yet. The processes after step S111 are repeated.

(4-3) Details of Atmospheric Scattering Correction Processing Next, details of the atmospheric scattering correction processing by the atmospheric scattering correction unit 152 executed in step S103 illustrated in FIG. 6 will be described with reference to FIG. As shown in FIG. 10, the minimum pixel value calculation unit 119 of the atmospheric scattering correction unit 152 calculates the minimum pixel value i_min of the multispectral image provided from the noise removal image processing unit 117 (S201).

  The multispectral image provided from the noise removal image processing unit 117 is an image from which noise has been removed by the noise removal image processing unit 117. Therefore, the minimum pixel value obtained in step S201 is the minimum pixel value of the foreground area excluding noise.

  Subsequently, the atmospheric scattering correction image processing unit 120 performs spectral correction based on the influence of atmospheric scattering (S202). Specifically, the atmospheric scatter correction image processing unit 120 subtracts the minimum pixel value i_min from the pixel value of each band of the multispectral image to be corrected. In step S202, subtracting the minimum pixel value i_min from the pixel value of each band means that the histogram of each band is shifted to the lower pixel value by the minimum class regarded as an atmospheric scattering component.

  Then, the atmospheric scattering correction unit 152 determines whether or not the processing of Step S201 to Step S202 has been performed for all bands of the panchromatic image and the multispectral image (S203). In step S201, when all the bands of the panchromatic image and the multispectral image are processed, the atmospheric scattering correction unit 152 ends the atmospheric scattering correction process. On the other hand, if all the bands of the panchromatic image and the multispectral image have not been processed in step S201, the atmospheric scattering correction unit 152 repeats the processes in and after step S201.

(4-4) Details of Observation Wavelength Bandwidth Correction Processing Next, with reference to FIG. 11, the observation wavelength bandwidth correction processing by the observation wavelength bandwidth correction unit 153 executed in step S104 shown in FIG. explain. As shown in FIG. 11, the observation wavelength band correction image processing unit 121 of the observation wavelength band correction unit 153 uses the observation wavelength band parameter of each band of the multispectral image with respect to the pixel value of the multispectral image. Then, observation wavelength band correction image processing is executed (S301). Specifically, the observation wavelength band correction image processing unit 121 multiplies the pixel value of the multispectral image by the observation wavelength band of each band unique to the sensor that captures the multispectral image. Which sensor is used to capture the image may be acquired from metadata or the like added to the image data, or which sensor is used may be input by the user.

  Then, the observation wavelength band correction unit 153 determines whether or not the observation wavelength band correction image processing using the observation wavelength band parameter is performed on all the bands of the multispectral image (S302).

  If it is determined in step S302 that processing has been performed for all bands of the multispectral image, the observation wavelength bandwidth correction unit 153 performs processing in step S303. On the other hand, if the processing is not performed for all the bands of the multispectral image in step S302, the processing of step S301 is executed for the multispectral image that is not processed.

  Then, the invisible band correction image processing unit 122 of the observation wavelength band correction unit 153 performs correction processing for reducing the influence of invisible light on the panchromatic image (S303). Specifically, the invisible band corrected image processing unit 122 calculates the pixel value Pa ′ of the panchromatic image corrected by the above equation (2).

(4-5) Details of Pan-sharpening Process Next, with reference to FIG. 12, the pan-sharpening process performed by the pan-sharpening processing unit 154 executed in step S105 shown in FIG. As shown in FIG. 12, the pan-sharpening processing unit 154 first determines whether the processing target band of the provided image is a visible band (S401).

  If it is determined in step S401 that the processing target band is a visible band, the visible band pan-sharpening processing unit 123 performs a visible band pan-sharpening process (S402). Specifically, the visible band pan-sharpening processing unit 123 calculates the pixel value of each pan-sharpened image of the blue band, the green band, and the red band according to the above formula (3) or formula (3) ′.

  On the other hand, if it is determined in step S401 that the processing target band is an invisible band, the invisible band pan-sharpening processing unit 124 performs pan-sharpening processing of the invisible band (S403). Specifically, the invisible band pan-sharpening processing unit 124 calculates the pixel value of the pan-sharpened image of the invisible band by the above equation (4).

  Then, the pixel value range optimization processing unit 125 performs pixel value range optimization processing of the pan-sharpened image in the visible band and the pan-sharpened image in the invisible band that have been pan-sharpened in step S402 or step S403. (S404).

  Then, the pan-sharpening processing unit 154 determines whether all the bands of the multispectral image have been processed (S405). If it is determined in step S405 that processing has been performed for all bands of the multispectral image, the processing ends. On the other hand, if it is determined that processing has not been performed for all the bands of the multispectral image, the processing from step S401 is repeated.

  The pan-sharpened image of the visible band created by the above process matches the color of the original feature as described above, and it is a clear color image that can obtain information including the features and shape of the feature. It is. The invisible band pan-sharpened image created by the above processing is an image that can be used for remote sensing such as a distribution of vegetation and a distribution of thermal radiation that cannot be visually detected. In the image processing apparatus 10, the pan-sharpened image of the visible band and the pan-sharpened image of the invisible band created by the above processing may be stored in association with each other.

  Here, a display example of a pan-sharpened image created as a result of executing the above-described process will be described with reference to FIG. As shown in FIG. 13, the display screen 50 of the output unit 118 includes a panchromatic image designation field 51, a multispectral image designation field 52, a band information field 53, an image display field 54, and a pan sharpening execution button 55. .

  In the panchromatic image designation field 51 and the multispectral image designation field 52, the file name of the panchromatic image and the file name of the multispectral image from which the pan-sharpened image is created are input. In the band information column 53, information on the wavelength bandwidth of each band is displayed. In the image display field 54, a synthesized pan-sharpened image is displayed. The pan-sharpening execution button 55 is a button that triggers the start of the pan-sharpening process when pressed after each file name is input to the panchromatic image designation field 51 and the multispectral image designation field 52.

(5) Effects of the present embodiment As described above, according to the present embodiment, the noise removing unit 151 removes pixels corresponding to noise in the panchromatic image and the multispectral image using the histogram, The atmospheric scattering correction unit 152 subtracts the pixel values of each observation wavelength band of the panchromatic image and the multispectral image using the minimum pixel value of the image data from which noise has been removed by the noise removing unit 151 as the atmospheric scattering component, and the observation wavelength The bandwidth correction unit 153 performs correction to reduce invisible light of the panchromatic image based on the ratio of the observation wavelength bandwidth of the multispectral image, and the pan sharpening processing unit 154 is corrected by the atmospheric scattering correction unit 152. The synthesized multispectral image and the panchromatic image corrected by the observation wavelength bandwidth correction unit 153 Create a sharp image. Accordingly, it is possible to generate a pan-sharpened image that is a clear color image that does not have an unnaturally bright area and matches the color of the original feature.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 102 Auxiliary storage apparatus 103 Storage apparatus 104 Input / output apparatus 105 Display apparatus 111 Input section 112 Histogram creation section 113 Histogram smoothing section 114 Decrease determination section 115 Histogram differentiation section 116 Increase determination section 117 Noise removal image processing section 118 Output 119 Minimum pixel value calculation unit 120 Atmospheric scatter correction image processing unit 121 Observation wavelength band correction image processing unit 122 Invisible band correction image processing unit 123 Visible band pan sharpening processing unit 124 Invisible band pan sharpening processing unit 125 Value range optimization processing Unit 131 Image data storage unit 132 Noise parameter storage unit 133 Observation wavelength band data storage unit 151 Noise removal unit 152 Atmospheric scattering correction unit 153 Observation wavelength band width correction unit

Claims (7)

An image processing apparatus for processing an optical satellite image taken from a flying object,
An input unit for reading a color image of the optical satellite image and a monochrome image overlapping the area of the color image;
A noise removing unit that removes, as noise, pixels corresponding to a predetermined distribution of pixel values of the color image;
An atmospheric scattering correction unit that subtracts pixel values of each observation wavelength band of the color image, using the minimum pixel value of the image data from which noise has been removed by the noise removing unit as an atmospheric scattering component;
Based on the spectral intensity ratio of the color image, an observation wavelength bandwidth correction unit that performs correction to reduce the invisible light of the monochrome image;
A pan-sharpening processing unit that combines the color image corrected by the atmospheric scattering correction unit and the monochrome image corrected by the observation wavelength bandwidth correction unit to create a pan-sharpened image;
Equipped with a,
The noise removing unit
A histogram showing the distribution of pixel values in each observation wavelength band of the color image is created, and the pixel value corresponding to the distribution of pixel values isolated by a low value in the histogram of each observation wavelength band is set to 0,
The atmospheric scattering correction unit,
The minimum pixel value of each observation wavelength band of the color image in which the pixel value corresponding to noise is set to 0 by the noise removing unit, and the minimum pixel of each observation wavelength band is determined from the pixel value of each observation wavelength band of the color image. Subtract the value,
The observation wavelength band correction unit is
The color image is multiplied by the ratio of the observation wavelength bandwidth specific to the sensor to obtain the spectral intensity for each wavelength band. An image processing apparatus that multiplies a ratio of pixel values of an observation wavelength bandwidth of visible light other than invisible light . The pan sharpening processing unit
By multiplying the pixel value of the corrected monochrome image by the ratio of the pixel value of each visible light observation wavelength band to the pixel value of the corrected color image, the visible wavelength observation wavelength band of the pan-sharpened image The image processing apparatus according to claim 1 , wherein a pixel value of a width is calculated. The pan sharpening processing unit
Multiplying the pixel value of the corrected monochrome image by the ratio of the pixel value of the observation wavelength band width of the invisible light to the pixel value of the corrected color image, the observation wavelength band of the invisible light of the pan-sharpened image The image processing apparatus according to claim 2 , wherein a pixel value of a width is calculated. An image processing method in an image processing apparatus for processing an optical satellite image taken from a flying object,
An input unit reading a color image of the optical satellite image and a monochrome image overlapping the area of the color image;
A noise removing unit removing pixels corresponding to a predetermined distribution of pixel values of the color image as noise;
A step of air diffuser Ranho positive part, the minimum pixel value of the image data that has been removed of noise by the noise removing unit as atmospheric scattering component, subtracts the pixel value of each observation wavelength bands of the color image,
An observation wavelength bandwidth correction unit performing correction to reduce invisible light of the monochrome image based on a spectral intensity ratio of the color image;
A pan-sharpening processing unit combining the color image corrected by the atmospheric scattering correction unit and the monochrome image corrected by the observation wavelength bandwidth correction unit to create a pan-sharpened image;
Only including,
In the step in which the noise removing unit removes the corresponding pixel as noise,
The noise removing unit creates a histogram indicating a distribution of pixel values in each observation wavelength band of the color image, and selects a pixel value corresponding to a distribution of pixel values isolated by a low value in the histogram of each observation wavelength band. 0,
In the step in which the atmospheric scattering correction unit performs correction to reduce the invisible light of the monochrome image,
The atmospheric scattering correction unit obtains the minimum pixel value of each observation wavelength band of the color image in which the pixel value corresponding to the noise is set to 0 by the noise removal unit, and calculates the pixel value of each observation wavelength band of the color image. Subtract the minimum pixel value of each observation wavelength band,
In the step of performing the correction in which the observation wavelength bandwidth correction unit reduces the invisible light of the monochrome image,
The observation wavelength bandwidth correction unit multiplies the color image by a ratio of an observation wavelength bandwidth specific to the sensor to obtain a spectral intensity for each wavelength band, and further, the pixel value of the monochrome image includes all of the color image. An image processing method characterized by multiplying the ratio of the pixel value of the observation wavelength bandwidth of visible light other than invisible light to the pixel value of the observation wavelength bandwidth of . In the step in which the pan sharpening processing unit creates the pan sharpened image,
The pan-sharpening processing unit multiplies the pixel value of the corrected monochrome image by the ratio of the pixel value of the observation wavelength band of each visible light to the pixel value of the corrected color image, and the pan-sharpened Calculate the pixel value of the observation wavelength bandwidth of the visible light of the image
The image processing method according to claim 4, wherein: In the step in which the pan sharpening processing unit creates the pan sharpened image,
The pan-sharpening processing unit multiplies the pixel value of the corrected monochrome image by a ratio of the pixel value of the observation wavelength bandwidth of invisible light to the pixel value of the corrected color image, and the pan-sharpened Calculate the pixel value of the observation wavelength bandwidth of invisible light in the image
The image processing method according to claim 5, wherein: Computer
An image processing apparatus for processing an optical satellite image taken from a flying object,
An input unit for reading a color image of the optical satellite image and a monochrome image overlapping the area of the color image;
A noise removing unit that removes, as noise, pixels corresponding to a predetermined distribution of pixel values of the color image;
An atmospheric scattering correction unit that subtracts the pixel value of each observation wavelength band of the color image, using the minimum pixel value of the image data removed by the noise removing unit as an atmospheric scattering component;
Based on the spectral intensity ratio of the color image, an observation wavelength bandwidth correction unit that performs correction to reduce the invisible light of the monochrome image;
A pan-sharpening processing unit that combines the color image corrected by the atmospheric scattering correction unit and the monochrome image corrected by the observation wavelength bandwidth correction unit to create a pan-sharpened image;
With
The noise removing unit
A histogram showing the distribution of pixel values in each observation wavelength band of the color image is created, and the pixel value corresponding to the distribution of pixel values isolated by a low value in the histogram of each observation wavelength band is set to 0,
The atmospheric scattering correction unit,
The minimum pixel value of each observation wavelength band of the color image in which the pixel value corresponding to noise is set to 0 by the noise removing unit, and the minimum pixel of each observation wavelength band is determined from the pixel value of each observation wavelength band of the color image. Subtract the value,
The observation wavelength band correction unit is
The color image is multiplied by the ratio of the observation wavelength bandwidth specific to the sensor to obtain the spectral intensity for each wavelength band. recording medium for recording a program for functioning as an image processing apparatus characterized that you multiply the ratio of the pixel value of the observed wavelength bandwidth of visible light other than visible light.