JP5137893B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that processes an image obtained by imaging the earth with an optical sensor mounted on an artificial satellite or an aircraft, and in particular, white balance due to the influence of atmospheric scattering / attenuation that differs between imaging wavelengths generated for multiband images. It is related with the correction of the change.

  When light waves propagate through the atmosphere, scattering and attenuation phenomena occur due to atmospheric molecules and aerosols. When the earth is imaged with an optical sensor mounted on an artificial satellite or an aircraft, a light wave affected by scattering or absorption in the earth atmosphere is received by the optical sensor. The magnitude of the scattering and attenuation of light waves occurring in the atmosphere varies greatly depending on the types of molecules and aerosols in the atmosphere, the season, and weather conditions. In addition, since the magnitudes of light wave scattering and attenuation differ between wavelengths, the white balance of the luminance value varies depending on the imaging conditions in a multiband image, causing deterioration in image quality. From the above, it is necessary to perform correction processing for removing the influence of light wave scattering components and attenuation occurring in the atmosphere on the captured image.

  For example, in Patent Document 1 below, light waves in the near-infrared region have a physical property that water molecules are almost absorbed by water molecules, and reflected light from the ground surface is not captured by pixels that image a place where there is a water region on the ground surface. Since it is not included and only the scattering / reflecting component in the atmosphere is included, the scattering component in the atmosphere in the pixel that captured the land area is corrected using the pixel information. In addition, correction is performed by estimating scattering components in the atmosphere in other wavelength bands using these pieces of pixel information.

Japanese Patent Laid-Open No. 06-300845

  In the above-described conventional correction processing method, the correction amount is calculated by imaging the water area in the near infrared region. However, the correction amount is obtained in an optical sensor having no imaging wavelength band in the near infrared region or near the infrared region. There is a possibility that the accuracy of will deteriorate greatly.

  In addition, in order to correct the land pixels from the pixel information obtained by imaging the water area, the air and weather conditions may be different from the pixels obtained by imaging the water area in areas where there is no water area or far away from the water area. The accuracy of the quantity can be greatly degraded.

  The present invention has been made to solve the above-described problems, and in an image obtained by imaging the earth using an optical sensor mounted on an artificial satellite or an aircraft, particularly in a multiband image, the wavelength band of the captured image is known. If so, an image processing device capable of accurately correcting the influence of light wave scattering and absorption caused by molecules and aerosols in the atmosphere on an image captured from an optical sensor having any imaging wavelength band. The purpose is to provide.

  The present invention provides a histogram analysis unit for performing histogram analysis of image data captured in a plurality of input different wavelength bands, and a wavelength band for obtaining an imaging wavelength of the image data captured in the plurality of input different wavelength bands. From the information output unit and the histogram analysis and imaging wavelength for each image data, the luminance value offset correction amount is calculated to correct the luminance value offset, and the luminance value gain correction amount is calculated to correct the luminance value gain. An image processing apparatus comprising: a wavelength correlation offset / gain correction unit that performs processing.

  In this invention, if the wavelength band of the captured image is known, the effect of light wave scattering and absorption caused by molecules and aerosols in the atmosphere can be accurately applied to the captured image from the optical sensor having any imaging wavelength band. It can be corrected well.

1 is a diagram showing a schematic configuration of an image processing apparatus according to Embodiment 1 of the present invention. It is a figure for demonstrating operation | movement of the image processing apparatus by this invention. It is another figure for demonstrating operation | movement of the image processing apparatus by this invention. It is another figure for demonstrating operation | movement of the image processing apparatus by this invention. It is a figure which shows schematic structure of the image processing apparatus by Embodiment 2 of this invention.

Embodiment 1 FIG.
FIG. 1 is a diagram showing a schematic configuration of an image processing apparatus according to Embodiment 1 of the present invention. An image input unit 1, a histogram analysis unit 2, a wavelength band information output unit 3, a wavelength correlation offset correction unit 4, a wavelength correlation. A gain correction unit 5 and an image output unit 6 are provided.

  In FIG. 1, image data captured in a plurality of different wavelength bands is input from an image input unit 1. The histogram analysis unit 2 analyzes the input image data to calculate a luminance value correction amount. This correction amount is sent to the wavelength correlation offset correction unit 4 and the wavelength correlation gain correction unit 5 in the subsequent stage. Also, the imaging wavelength of each image is sent from the wavelength band information output unit 3 to the wavelength correlation offset correction unit 4 and the wavelength correlation gain correction unit 5 in accordance with the image data. The wavelength correlation offset correction unit 4 calculates the luminance value offset correction amount from the correction amount and the imaging wavelength for each image data, and performs correction processing. Further, the wavelength correlation gain correction unit 5 performs correction processing by calculating a luminance value gain correction amount from the correction and imaging wavelength with respect to the image data. A plurality of pieces of image data subjected to the luminance value correction processing as described above are output from the image output unit 6. The wavelength correlation offset correction unit 4 and the wavelength correlation gain correction unit 5 constitute a wavelength correlation offset / gain correction unit.

  In this embodiment, the luminance value correction processing in the wavelength correlation offset correction processing unit 4 and the wavelength correlation gain correction unit 5 will be described below.

  The magnitude of scattering and attenuation that occurs when a light wave propagates through the atmosphere varies greatly depending on the type of molecules and aerosols in the atmosphere, the season, and weather conditions, but the relationship between wavelengths does not change significantly. This is because the physical properties such as the refractive index of molecules and aerosols in the atmosphere are almost constant in the visible region, so the magnitude of scattering and attenuation of atmospheric molecules and aerosol light waves is the size of the molecules and aerosol particles in the atmosphere It is possible to express by the ratio of the light wavelength to. Therefore, if brightness value correction processing is performed for each image data captured in a plurality of different wavelength bands, if each imaging wavelength band is known, only the image data in a certain wavelength band is overcorrected. It is possible to calculate a luminance value correction amount that is balanced between the two. The wavelength correlation offset correction unit 4 and the wavelength correlation gain correction unit 5 calculate a luminance value correction amount balanced between the wavelengths.

  The wavelength correlation offset correction unit 4 performs luminance value offset correction on the image data. The brightness value offset correction process at this time is expressed by, for example, the following equation (1), where S (x, y) is the image data, λ is the imaging wavelength, and A and m are the correction parameters.

S (x, y) − (A / λ ^m ) 2 <m <4 (1)

Here, simulation calculation is performed on the brightness value offset amount of the captured image generated by scattering in the atmosphere in the atmospheric condition of four patterns in the wavelength band of 400 to 1000 nm, and an exponential function λ ^{−m is obtained} for each simulation result with respect to the wavelength λ. FIG. 2 shows the residual (fitting error) when fitting. In FIG. 2, the horizontal axis represents the correction parameter m, and the vertical axis represents the fitting error as a relative value. From the results shown in FIG. 2, the fitting error is the smallest when the correction parameter m = 3. As a result, the luminance value offset amount of the picked-up image generated by scattering in the atmosphere is approximated by an exponential function λ ^−m that takes the correction parameter m in the region of 2 <m <4 with respect to the wavelength λ. It is possible to reduce. That is, by setting the chromatic dispersion of the luminance value offset correction amount to the exponential function λ- ^m , it is possible to calculate an appropriate inter-wavelength luminance value offset correction amount.

Next, the wavelength correlation gain correction unit 5 performs luminance value gain correction on the image data. The brightness value gain correction processing at this time is expressed by, for example, the following formula (2), where S ′ (x, y) is the image data, the imaging wavelength λ is B, τ ₀ and n are the correction parameters.

S ′ (x, y) / (τ ₀ − (B / λ ⁿ )) 1 <n <3 (2)

Here, the brightness value attenuation of the captured image generated by the atmospheric transmittance is simulated and calculated in four patterns of atmospheric conditions in the wavelength band of 400 to 1000 nm, and the correction function τ ₀ − (B / λ ⁿ ) is calculated for the wavelength λ. FIG. 3 shows a fitting error when fitting to each simulation result. In FIG. 3, the horizontal axis represents the correction parameter n, and the vertical axis represents the fitting error as a relative value. From the results shown in FIG. 3, the fitting error is the smallest when the correction parameter n = 2. Thus, the luminance value attenuation of the captured image caused by the atmospheric transmittance should be approximated by a correction function τ ₀ − (B / λ ⁿ ) that takes the correction parameter n within the region of 1 <n <3 with respect to the wavelength λ. Can reduce the fitting error. That is, by setting the chromatic dispersion of the luminance value gain correction as the correction function τ ₀ − (B / λ ⁿ ), an appropriate inter-wavelength luminance value gain correction amount can be calculated.

Further, the brightness value attenuation of the picked-up image generated by the atmospheric transmittance is simulated and calculated under four patterns of atmospheric conditions in the wavelength band of 400 to 1000 nm, and the correction function τ ₀ − (B / λ ⁿ ) is calculated for each wavelength λ. FIG. 4 shows a fitting error when fitting to the simulation result. In FIG. 4, the horizontal axis represents the correction parameter B, the logarithm of the base 10 and the vertical axis represents the fitting error as a relative value. From the results shown in FIG. 4, the brightness value attenuation of the captured image caused by the atmospheric transmittance is a correction function (τ ₀ − (B that takes the correction parameter B within the region of 4 <log ₁₀ B <5 with respect to the wavelength λ. It is possible to reduce the fitting error by approximating with / λ ⁿ )) ⁻¹ . That is, by setting the chromatic dispersion of the luminance value gain correction to the correction function (τ ₀ − (B / λ ⁿ )) ⁻¹ , it is possible to calculate an appropriate inter-wavelength luminance value gain correction amount.

In this embodiment, the correction parameter A of the equation (1) and the correction parameters τ ₀ and B of the equation (2) are determined from the result of the histogram analysis of each image data by the histogram analysis unit 2. Is possible.

For example, the frequency for each pixel value is obtained by histogram calculation of image data, and the center of gravity of the frequency is calculated. Then, the luminance value offset correction amount and the luminance value gain correction amount for each image data are obtained so that the pixel value serving as the center of frequency coincides between the image data, and the correction function Aλ is added to each image data, that is, the correction amount for each imaging wavelength. By fitting ^−m and (τ ₀ − (B / λ ⁿ )) ⁻¹ , it is possible to obtain the correction parameter A of the equation (1) and the correction parameters τ ₀ and B of the equation (2). .

In addition, the luminance value offset correction amount and the luminance value gain correction amount for each image data are obtained so that the minimum pixel value and the maximum pixel value of the image data are substantially the same between the image data, and the correction is performed for each image data, that is, each imaging wavelength. By fitting the correction function Aλ ^−m and (τ ₀ − (B / λ ⁿ )) ⁻¹ to the quantity, the correction parameter A of the equation (1) and the correction parameter τ ₀ and B of the equation (2) are obtained. It is possible.

  Further, the correction parameter A shown in the equation (1) can be determined from the atmospheric scattering amount measurement values obtained by measuring the atmospheric scattering amount generated when the light wave propagates in the atmosphere at a plurality of different wavelengths. . In this way, the correction parameter A can be determined more strictly by using the actually measured value.

Further, the correction parameters τ ₀ and B shown in the above equation (2) can be determined from measured attenuation values obtained by measuring the attenuation amount generated when the light wave propagates in the atmosphere at a plurality of different wavelengths. is there. In this way, the correction parameters τ ₀ and B can be determined more strictly by using the actually measured values.

  In this embodiment, as means for outputting the imaging wavelength of each image data from the wavelength band information output unit 3, for example, the imaging wavelength is described in advance in the header of the image data file, and the imaging wavelength is read from the header portion. It is possible to output.

  The imaging wavelength may not be described in the header of the image data file. For example, the identification number of each image data and imaging wavelength data are stored as initial data in the wavelength band information output unit 3, and the identification number of the image data input by the image input unit 1 is extracted and collated with the stored data. By doing so, it is possible to output an imaging wavelength corresponding to each image data.

  The image data file may not have a header. For example, the imaging time table and imaging wavelength data of each image data are stored as initial data in the wavelength band information output unit 3, and the imaging data is output by identifying the image data from the time input to the image input unit 1. It is possible.

  In this embodiment, each image data is not a single wavelength but data that is captured in a wavelength band having an arbitrary width, and is used when the wavelength correlation offset correction unit 4 and the wavelength correlation gain correction unit 5 obtain correction amounts. As the imaging wavelength, for example, the center wavelength of the imaging wavelength band is selected. Further, the full width at half maximum of the spectral sensitivity characteristic distribution of the optical sensor in the imaging wavelength band may be obtained, and the center wavelength of the wavelength band corresponding to the full width at half maximum may be selected. Further, for image data captured in a wavelength band having an arbitrary width, for example, a wavelength serving as the center of gravity of the spectral sensitivity characteristic distribution of the optical sensor in the imaging wavelength band may be selected.

  In this embodiment, each image data input from the image input unit 1 is subjected to luminance value correction processing by the wavelength correlation offset correction unit 4 and the wavelength correlation gain correction unit 5, and then each image data is converted to an image output unit 6. In addition to being output from, it is possible to generate a new image by synthesizing a plurality of image data captured in different wavelength bands. For example, it is possible to generate color image data by assigning image data captured near 450 nm to blue, image data captured near 550 nm to green, and image data captured near 650 nm to red. is there.

  In addition, it is possible to visualize light that is normally invisible to the human eye by adding color to image data captured in the wavelength band of the ultraviolet region or infrared region.

  Also, high-resolution panchromatic images (monochromatic images), low-resolution image data captured near 450 nm in blue, image data captured in the vicinity of 550 nm in green, and image data captured in the vicinity of 650 nm in red, It is possible to generate a high-resolution color image by synthesizing with the color image data respectively assigned to.

  As described above, in the present invention, since the brightness value change amount of the captured image due to scattering and absorption of light waves occurring in the atmosphere is represented by a correction function using the wavelength as a parameter, if the wavelength band of the captured image is known, It is possible to correct the influence of light wave scattering and absorption that occur in the atmosphere on an image captured from an optical sensor having any imaging wavelength band.

  Further, when combining image data captured in a plurality of different wavelength bands, it is possible to prevent only a captured image in a certain wavelength band from being overcorrected. For example, if the entire image is forest (green) or the sea surface (blue), the image data captured near 450 nm is blue, the image data captured near 550 nm is green, and the image data captured near 650 nm is red Even when the color image data is assigned to each of the color images, it is possible to generate a color image without causing a bias in luminance value (color) due to overcorrection of a certain wavelength band.

Embodiment 2. FIG.
In Embodiment 1 described above, luminance values are corrected in accordance with imaging wavelength bands for images captured in a plurality of different wavelength bands, and then each image data is combined to generate a multispectral image. Here, each image input from the image input unit 1 is affected by the sensitivity of the optical sensor that differs depending on the imaging wavelength band, and the luminance value varies between imaging wavelength bands. This luminance value variation causes deterioration in image quality of the generated multispectral image.

  Next, an embodiment for removing the influence of the sensitivity characteristics of the optical sensor on the image quality degradation of the multispectral (multiband) image will be described.

  FIG. 5 is a diagram showing a schematic configuration of an image processing apparatus according to Embodiment 2 of the present invention. The apparatus of FIG. 5 is an apparatus in which the influence of the sensitivity of the optical sensor on the image quality degradation of the multiband image is further removed from the apparatus of the first embodiment. In FIG. 5, the same or corresponding parts as those in the apparatus of FIG. Reference numeral 7 in FIG. 5 denotes a sensitivity level correction unit.

  In the second embodiment, a sensitivity level correction unit 7 is arranged at the subsequent stage of the image input unit 1. In accordance with the image data input to the image input unit 1, imaging wavelength band data of the image is input from the wavelength band information output unit 3 to the sensitivity level correction unit 7. The sensitivity level correction unit 7 uses the imaging wavelength of the input image data from the wavelength band information output unit 3 to perform luminance value correction processing of the input image data so that the sensitivity characteristics of the optical sensor are uniform.

  In this embodiment, the sensitivity level correction unit 7 stores the imaging wavelength of each image data and the correction amount for the image data as initial data, and the imaging wavelength input from the wavelength band information output unit 3 and the stored data. Are extracted, and the luminance value correction processing is performed on each image data.

For example, the following means can be considered for the brightness value correction processing performed by the sensitivity level correction unit 7. Assuming that the spectral sensitivity distribution of the optical sensor is R (λ) and the imaging wavelength band of certain image data is [λ ₁ , λ ₂ ], the correction for the imaging wavelength band m is performed so that the sensitivity of the optical sensor becomes a constant value R _const. The quantity C _m is defined as in the following formula (3).

C _m = R _const / (∫ _λ1 ^λ2 R (λ) dλ) (3)

Further, the correction value C _m can be simply expressed as in equation (4) where R ₀ is the average spectral characteristic of the optical sensor and Δλ is the imaging wavelength band of the image data.

C _m = R _const / (τ ₀ Δλ) (4)

Then, by dividing the correction amount C _m input image data, performs the brightness value correction of the image data.

  As described above, from the imaging wavelength band data from the wavelength band information output unit 3 and the spectral sensitivity characteristics of the optical sensor stored in the sensitivity level correction unit 7, for example, the above formula (3) and formula (4) are used. Thus, the sensitivity characteristic correction value can be obtained.

  Then, by correcting the luminance value using sensitivity characteristic correction corresponding to each image, it is possible to remove luminance value variation due to the sensitivity characteristic of the optical sensor being different in the imaging wavelength band.

  As described above, in the image processing apparatus according to this embodiment, the sensitivity level correction unit 7 is arranged at the subsequent stage of the image input unit 1 so that the sensitivity characteristics of the optical sensors different in the imaging wavelength band are deteriorated in the image quality of the multispectral image. Can be removed. It is possible to generate a multispectral image that is not biased toward luminance values (colors) in a certain imaging wavelength band.

  In each of the above embodiments, the luminance value gain correction processing by the wavelength correlation gain correction unit 5 is performed after the luminance value offset correction processing by the wavelength correlation offset correction unit 4 is performed. Either of the luminance value offset correction and the luminance value gain correction may be performed first. However, if the luminance value gain correction is performed first, the luminance value offset correction amount calculated by the histogram analysis unit 2 changes. It is necessary to calculate the value offset correction amount, which increases the calculation processing load. For this reason, it is possible to reduce the calculation processing load by performing the luminance value gain correction processing by the wavelength correlation gain correction unit 5 after performing the luminance value offset correction processing by the wavelength correlation offset correction unit 4.

  1 image input unit, 2 histogram analysis unit, 3 wavelength band information output unit, 4 wavelength correlation offset correction processing unit, 5 wavelength correlation gain correction unit, 6 image output unit, and 7 sensitivity level correction unit.

Claims (5)

A histogram analysis unit that performs histogram analysis of image data captured in a plurality of different input wavelength bands;
A wavelength band information output unit for obtaining an imaging wavelength of image data captured in the plurality of different input wavelength bands;
Wavelength correlation for calculating a luminance value offset correction amount by calculating the luminance value offset correction amount from the histogram analysis and imaging wavelength for each image data, and calculating a luminance value gain correction amount and correcting the luminance value gain. Offset / gain correction means;
An image processing apparatus comprising: A wavelength correlation offset correction unit that performs luminance value offset correction processing of the wavelength correlation offset / gain correction unit uses the imaging wavelength λ, parameters A and m for the image data S (x, y),
S (x, y)-(A / λ ^m ) 2 <m <4
The image processing apparatus according to claim 1, wherein a luminance value offset correction process represented by: The wavelength correlation gain correction unit that performs the luminance value gain correction processing of the wavelength correlation offset / gain correction unit uses the imaging wavelength λ, the correction parameter B, and τ ₀ and n for the image data S ′ (x, y). S ′ (x, y) / (τ ₀ − (B / λ ⁿ )) 1 <n <3
The image processing apparatus according to claim 1, wherein a luminance value gain correction process represented by: The wavelength correlation gain correction unit that performs the luminance value gain correction processing of the wavelength correlation offset / gain correction unit uses the imaging wavelength λ, the correction parameter B, and τ ₀ and n for the image data S ′ (x, y). S ′ (x, y) / (τ ₀ − (B / λ ⁿ )) 4 <log ₁₀ B <5
The image processing apparatus according to claim 1, wherein a luminance value gain correction process represented by:   5. The image processing apparatus according to claim 1, further comprising a sensitivity level correction unit that first corrects sensitivity characteristics of an optical sensor with respect to the input image data. 6.

Images