JP6556409B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that corrects spectral radiance in a panchromatic image.

The contrast of a part of the image acquired from the sensor may be lowered due to the influence of soot or fog contained in the atmosphere.
Since a decrease in contrast causes a decrease in the visibility of an image, an image processing apparatus that corrects a decrease in contrast due to the influence of haze or fog has been developed.

  In Patent Document 1 below, a difference between a signal level of an arbitrary pixel among a plurality of pixels constituting an image and a signal level of a pixel existing in an area around the pixel is measured. An image processing apparatus that raises the signal level of the pixel is disclosed by assuming that the contrast is lowered if is large.

JP 2012-0275547 A

For example, when the image acquired from the sensor is an image obtained by photographing a shoreline, among the plurality of pixels constituting the image, any pixel corresponds to the sea area and is present in the area around the pixel May correspond to the coastal area.
At this time, there is a big difference between the reflectance of the sea and the reflectance of the coast, so even if it is not affected by the haze or fog contained in the atmosphere, the signal level of any pixel and the surroundings of the pixel The difference from the signal level of the pixels existing in the region may increase.
In such a case, assuming that the contrast is lowered and increasing the signal level of the pixel, the visibility of the image is lowered.
For this reason, in addition to the case where the image acquired from the sensor is an image obtained by photographing the coastline, in the case where the image includes a plurality of regions with greatly different reflectances, such as a boundary between a mountainous area and an urban area, There has been a problem that it is difficult to improve the visibility of an image by correcting the contrast that has been lowered due to the influence of fog or the like.

  The present invention has been made to solve the above-described problems. Even when the panchromatic image acquired from the sensor is an image including a plurality of regions having greatly different reflectances, it is possible to reduce the fog or fog. An object of the present invention is to obtain an image processing apparatus capable of correcting a contrast that has been affected and decreased.

  The image processing apparatus according to the present invention includes an image dividing unit that divides a multispectral image including images of a plurality of wavelength bands into a plurality of regions, and a plurality of wavelength bands of the plurality of images included in the multispectral image. From the scattering luminance calculation unit for calculating atmospheric scattering radiance indicating atmospheric scattering in each region divided by the image dividing unit, and from the atmospheric scattering radiance in each region calculated for each wavelength band by the scattering luminance calculation unit A correction value calculation unit that calculates a correction value of spectral radiance in a panchromatic image that is an image in a single wavelength band including a plurality of wavelength bands, and the correction unit calculates the spectral radiation calculated by the correction value calculation unit The spectral radiance in the panchromatic image is corrected using the luminance correction value.

  According to this invention, the spectral radiance in a panchromatic image, which is an image in a single wavelength band including a plurality of wavelength bands, is calculated from the atmospheric scattered radiance in each region calculated by the scattering luminance calculation unit for each wavelength band. Since the correction value calculation unit for calculating the correction value is provided, and the correction unit is configured to correct the spectral radiance in the panchromatic image using the correction value of the spectral radiance calculated by the correction value calculation unit, Even when the panchromatic image acquired from the sensor is an image including a plurality of regions having greatly different reflectances, there is an effect that the contrast that is lowered due to the influence of haze or fog can be corrected. .

1 is a configuration diagram illustrating an image processing apparatus according to Embodiment 1 of the present invention; It is a hardware block diagram which shows the image processing apparatus by Embodiment 1 of this invention. FIG. 2 is a hardware configuration diagram of a computer when the image processing apparatus is realized by software or firmware. It is a flowchart which shows the process sequence in case an image processing apparatus is implement | achieved by software or firmware. It is explanatory drawing which shows the multispectral image divided | segmented by the image division part 1. FIG. It is explanatory drawing which shows the multispectral image divided | segmented by the image division part 1. FIG. It is explanatory drawing which shows the histogram of the small area | region S (i, j) in the band b calculated by the dark part brightness | luminance calculation part 4. FIG. It is explanatory drawing which shows the atmospheric scattering radiance _Lscat (b, i, j) modeled by the scattered luminance calculation process part 5. FIG. It is explanatory drawing which shows the atmospheric transmittance | permeability (tau) (b, i, j) corresponding to the atmospheric scattering radiance _Lscat (b, i, j) of the small area _| region S (i, j) in the band b. It is explanatory drawing which shows the observable wavelength range of a multispectral sensor, and the observable wavelength range of a panchromatic sensor. It is a block diagram which shows the image processing apparatus by Embodiment 2 of this invention. It is a hardware block diagram which shows the image processing apparatus by Embodiment 2 of this invention. It is explanatory drawing which shows the spectral radiance L _sensor (b, i, j) modeled by the reflected luminance calculation part 41. FIG. It is a block diagram which shows the image processing apparatus by Embodiment 3 of this invention. It is a hardware block diagram which shows the image processing apparatus by Embodiment 3 of this invention.

  Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image processing apparatus according to Embodiment 1 of the present invention, and FIG. 2 is a hardware block diagram showing the image processing apparatus according to Embodiment 1 of the present invention.
1 and 2, the image dividing unit 1 is realized by, for example, an image dividing circuit 21 shown in FIG.
The image dividing unit 1 acquires a multispectral image including images of a plurality of wavelength bands from the multispectral sensor, and performs a process of dividing the multispectral image into a plurality of small regions (regions).
The parameter storage unit 2 is realized by, for example, the parameter storage circuit 22 shown in FIG.
The parameter storage unit 2 stores scattering characteristic data indicating scattering characteristics in each wavelength band, scattered transmission characteristic data that is a function indicating atmospheric transmittance in a small area, and wavelength characteristic data indicating a wavelength band that can be observed by the sensor. ing.

The scattered luminance calculation unit 3 is realized by, for example, the scattered luminance calculation circuit 23 illustrated in FIG. 2, and includes a dark portion luminance calculation unit 4 and a scattered luminance calculation processing unit 5.
The scattered luminance calculation unit 3 uses the scattering characteristic data stored in the parameter storage unit 2, and uses the scattering characteristic data stored in the parameter storage unit 2 for each wavelength band of a plurality of images included in the multispectral image. Processing for calculating atmospheric scattering radiance indicating atmospheric scattering in the region is performed.

The dark part luminance calculating unit 4 of the scattered luminance calculating unit 3 calculates a histogram of each small region divided by the image dividing unit 1 for each wavelength band of a plurality of images included in the multispectral image, and the histogram Thus, the process of calculating the spectral radiance of the dark area where the spectral radiance is relatively small in each of the small areas divided by the image dividing unit 1 is performed.
The scattered luminance calculation processing unit 5 of the scattered luminance calculation unit 3 uses the scattering characteristic data stored in the parameter storage unit 2 and the spectral radiance of the dark area calculated by the dark portion luminance calculation unit 4 to generate a multispectral image. For each wavelength band of a plurality of included images, a process of calculating atmospheric scattering radiance in each small region divided by the image dividing unit 1 is performed.

The atmospheric transmittance calculating unit 6 is realized by, for example, the transmittance calculating circuit 24 shown in FIG.
The atmospheric transmittance calculation unit 6 uses the scattered transmission characteristic data stored in the parameter storage unit 2 to calculate from the atmospheric scattering radiance in each small region calculated for each wavelength band by the scattering luminance calculation processing unit 5. For each wavelength band, processing for calculating the atmospheric transmittance of atmospheric scattering in each small region divided by the image dividing unit 1 is performed.

The correction value calculation unit 7 is realized by, for example, the correction value calculation circuit 25 illustrated in FIG. 2, and includes a luminance correction value calculation unit 8, a transmittance correction value calculation unit 9, an image recombination unit 10, and a correction value interpolation unit. 11 is provided.
The correction value calculation unit 7 uses the atmospheric scattering radiance in each small region calculated for each wavelength band by the scattering luminance calculation processing unit 5 in a panchromatic image that is an image in a single wavelength band including a plurality of wavelength bands. Processing for calculating a correction value of the spectral radiance is performed.
Further, the correction value calculation unit 7 calculates the correction value of the spectral radiance in the panchromatic image from the atmospheric transmittance of atmospheric scattering in each small region calculated for each wavelength band by the atmospheric transmittance calculation unit 6. To implement.
The panchromatic image is an image in a single wavelength band including a plurality of image wavelength bands included in the multispectral image.

The luminance correction value calculation unit 8 of the correction value calculation unit 7 uses the wavelength characteristic data stored in the parameter storage unit 2 and the atmospheric air in each small region calculated for each wavelength band by the scattering luminance calculation processing unit 5. A process of calculating the first correction value of the spectral radiance in each small region from the scattered radiance is performed.
The transmittance correction value calculation unit 9 of the correction value calculation unit 7 uses the wavelength characteristic data stored in the parameter storage unit 2 and uses the wavelength characteristic data stored in the parameter storage unit 2 in each small region calculated for each wavelength band. A process of calculating a second correction value of the spectral radiance in each small region from the atmospheric transmittance of atmospheric scattering is performed.

The image recombination unit 10 of the correction value calculation unit 7 combines the first correction values of the spectral radiances in the respective small areas calculated by the luminance correction value calculation unit 8 to obtain a two-dimensional first correction value. A process for generating the array A is performed.
Further, the image recombination unit 10 combines the second correction value of the spectral radiance in each small region calculated by the transmittance correction value calculation unit 9 to generate a two-dimensional array B of the second correction values. Perform the process to generate.
Each element in the two-dimensional arrays A and B corresponds to each small region divided by the image dividing unit 1. Each element in the two-dimensional array A has a first correction value of the spectral radiance in each small area divided by the image dividing unit 1, and each element in the two-dimensional array B is an image division The second correction value of the spectral radiance in each of the small areas divided by the unit 1 is provided.
The correction value interpolation unit 11 of the correction value calculation unit 7 is generated by the image recombination unit 10 so that the resolution of the two-dimensional arrays A and B generated by the image recombination unit 10 matches the resolution of the panchromatic image. A process for interpolating the two-dimensional arrays A and B is performed.

The correction unit 12 is realized by the correction circuit 26 shown in FIG. 2 and includes a radiance correction unit 13 and a transmittance correction unit 14.
The radiance correction unit 13 of the correction unit 12 performs a process of acquiring a panchromatic image from the panchromatic sensor.
The radiance correction unit 13 performs a process of correcting the spectral radiance in the panchromatic image using the first correction value of each element in the two-dimensional array A interpolated by the correction value interpolation unit 11.
The transmittance correction unit 14 performs a process of correcting the spectral radiance in the panchromatic image by using the second correction value of each element in the two-dimensional array B interpolated by the correction value interpolation unit 11.

  In FIG. 1, each of the image dividing unit 1, parameter storage unit 2, scattered luminance calculation unit 3, atmospheric transmittance calculation unit 6, correction value calculation unit 7, and correction unit 12, which are components of the image processing apparatus, is illustrated in FIG. 2. This is assumed to be realized by dedicated hardware as shown in FIG. That is, it is assumed to be realized by the image dividing circuit 21, the parameter storage circuit 22, the scattering luminance calculation circuit 23, the transmittance calculation circuit 24, the correction value calculation circuit 25, and the correction circuit 26.

Here, the parameter storage circuit 22 includes, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory Memory), an EEPROM (Electrically Erasable Memory), and the like. Alternatively, a volatile semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), or the like is applicable.
The image dividing circuit 21, the scattered luminance calculation circuit 23, the transmittance calculation circuit 24, the correction value calculation circuit 25, and the correction circuit 26 are, for example, a single circuit, a composite circuit, a programmed processor, a processor programmed in parallel, An application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof is applicable.

The components of the image processing apparatus are not limited to those realized by dedicated hardware, and the image processing apparatus may be realized by software, firmware, or a combination of software and firmware.
Software or firmware is stored as a program in the memory of a computer. The computer means hardware that executes a program, and includes, for example, a CPU (Central Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, a DSP (Digital Signal Processor), and the like. .

FIG. 3 is a hardware configuration diagram of a computer when the image processing apparatus is realized by software or firmware.
When the image processing apparatus is realized by software or firmware, the parameter storage unit 2 is configured on the memory 31 of the computer, the image dividing unit 1, the scattered luminance calculation unit 3, the atmospheric transmittance calculation unit 6, and the correction value calculation. A program for causing the computer to execute the processing procedures of the unit 7 and the correction unit 12 may be stored in the memory 31, and the processor 32 of the computer may execute the program stored in the memory 31.
FIG. 4 is a flowchart showing a processing procedure when the image processing apparatus is realized by software or firmware.

  2 shows an example in which each component of the image processing apparatus is realized by dedicated hardware, and FIG. 3 shows an example in which the image processing apparatus is realized by software, firmware, etc. Some components in the image processing apparatus may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, or the like.

Next, the operation will be described.
In the first embodiment, the image dividing unit 1 acquires a multispectral image including images of a plurality of wavelength bands from the multispectral sensor, and the radiance correction unit 13 acquires a panchromatic image from the panchromatic sensor. To do.
At this time, each of the wavelength bands of the plurality of images included in the multispectral image has a narrow band, and the wavelength band of the panchromatic image includes the wavelength bands of the plurality of images included in the multispectral image. And
Hereinafter, the wavelength band of a plurality of images included in the multispectral image is referred to as a band b. In the first embodiment, for convenience of explanation, the multispectral image is an image of the band b1, an image of the band b2, and an image of the band b3. It is assumed that the image and the image of band b4 are included.

The image dividing unit 1 acquires a multispectral image from the multispectral sensor (step ST1 in FIG. 4), and divides the multispectral image into N small regions S (i, j) as shown in FIG. Step ST2 in FIG.
FIG. 5 is an explanatory diagram showing a multispectral image divided by the image dividing unit 1.
FIG. 5 shows an example where N = 16, and the multispectral image is divided into 16 pieces. i = 1, 2, 3, 4 and j = 1, 2, 3, 4.
However, the division number N of the multispectral image can be arbitrarily set, and is determined from, for example, the size of the subject included in the multispectral image.

Specifically, the number of divisions N of the multispectral image is set so that the subject is included in a small area that includes a subject included in the multispectral image and a dark area such as a shadow of the subject. To do.
When subjects of various sizes are included in the multispectral image, the multispectral image is divided based on the size of the largest subject among the multiple subjects included in the multispectral image. The number N may be set to reduce the number of small areas S (i, j) that do not include the subject.

Further, when a density distribution indicating a change in density such as haze or fog is drawn in the multispectral image, as shown in FIG. 6, in the density distribution, a multi-value is displayed for each region including a maximum value or a minimum value of density. The spectral image may be divided.
FIG. 6 is an explanatory diagram showing a multispectral image divided by the image dividing unit 1.
FIG. 6 shows an example of N = 16 as in FIG. 5, but N <16, and a small region S (i, j divided for each region including the local maximum value or the local minimum value. ) Is larger than the small region S (i, j) divided as shown in FIG. 5, the dark region in the dark portion luminance calculation unit 4 is more than in the case where the multispectral image is divided as shown in FIG. The number of calculated spectral radiances can be reduced.
Further, in the multispectral image, when the multispectral image is drawn so that the location where the change rate of the density changes can be grasped, the small region S (i, j) indicates the location where the change rate of the density changes. The multispectral image may be divided so as to include it.

The dark part luminance calculating unit 4 of the scattered luminance calculating unit 3 displays a histogram of each small region S (i, j) divided by the image dividing unit 1 for each band b of a plurality of images included in the multispectral image. Each is calculated (step ST3 in FIG. 4).
FIG. 7 is an explanatory diagram showing a histogram of the small area S (i, j) in the band b calculated by the dark part luminance calculation unit 4.
The dark portion luminance calculation unit 4 relatively compares the spectral radiance in each of the small regions S (i, j) divided by the image dividing unit 1 for each of the bands b of the plurality of images included in the multispectral image. Spectral radiance L _D (b, i, j) in the dark area where is small is calculated (step ST4 in FIG. 4).
FIG. 7 shows an example in which the minimum value of the spectral radiance is the spectral radiance L _D (b, i, j) of the dark region in the histogram of the small region S (i, j).

Here, an example is shown in which the dark portion luminance calculation unit 4 calculates a histogram of the small region S (i, j). However, the dark portion luminance calculation unit 4 has a pixel value among a plurality of pixels constituting the multispectral image. There may be pixels that are not included. In such a case, the dark part luminance calculation unit 4 calculates a histogram of the small region S (i, j) by excluding pixels having no pixel value from a plurality of pixels constituting the multispectral image. You may do it.
Also, the dark part luminance calculation unit 4 configures the multispectral image when a plurality of pixels constituting the multispectral image include defective pixels whose pixel values are always near zero. The histogram of the small region S (i, j) may be calculated by excluding defective pixels from a plurality of pixels.

The scattered luminance calculation processing unit 5 of the scattered luminance calculation unit 3 acquires the scattering characteristic data α (b) stored in the parameter storage unit 2 (step ST5 in FIG. 4).
The scattering characteristic data α (b) stored in the parameter storage unit 2 is data indicating the scattering characteristic of the band b and is a value unique to the band.
The scattering characteristic data α (b) can be calculated from the radiance ratio of atmospheric scattering at each wavelength using, for example, a light-wave atmospheric propagation calculation simulator such as MODTRAN (Moderate resolution atomic TRANmission).

The scattered luminance calculation processing unit 5 performs atmospheric scattering radiance L _scat (in each small region S (i, j) divided by the image dividing unit 1 for each band b of the plurality of images included in the multispectral image. b, i, j) is a model of the product of the scattering characteristic data α (b) and the eigenvalue L _scat0 (i, j) of the small region S (i, j) as shown in the following equation (1). Turn into.
L _scat (b, i, j) = α (b) × L _scat0 (i, j) (1)
The atmospheric scattering radiance L _scat (b, i, j) is a spectral radiance indicating atmospheric scattering in band b.
The eigenvalue L _scat0 (i, j) of the small area S (i, j) has no wavelength dependency and is an eigenvalue of the small area S (i, j) depending on the imaging environment.
FIG. 8 is an explanatory diagram showing the atmospheric scattered radiance L _scat (b, i, j) modeled by the scattered luminance calculation processing unit 5.
In FIG. 8, the solid line is a model of atmospheric scattering radiance L _scat (b, i, j), and ◯ is the spectral radiance of the dark area in the small area S (i, j).

Next, the scattered luminance calculation processing unit 5 calculates the spectral radiance L _D (b, i, j) of the dark region and the right side of the expression (1) for each of the bands b of the plurality of images included in the multispectral image. The eigenvalue L _scat0 (i, j) that is the closest to α (b) × L _scat0 (i, j) is estimated by regression analysis.
At this time, the number of measurement points of the spectral radiance L _D (b, i, j) in the dark region is the same as the number of bands that the multispectral sensor has. A typical example of a parameter determination method based on regression analysis is a least square method.

When the scattered luminance calculation processing unit 5 estimates the eigenvalue L _scat0 (i, j), the estimated eigenvalue L _scat0 (i, j) and the scattering characteristic data α (b) are substituted into the equation (1), and the band b Every time, atmospheric scattering radiance L _scat (b, i, j) in each small region S (i, j) is calculated (step ST6 in FIG. 4).
The scattered luminance calculation processing unit 5 outputs the calculated atmospheric scattered radiance L _scat (b, i, j) in each small region S (i, j) to the atmospheric transmittance calculating unit 6 and the luminance correction value calculating unit 8. To do.
However, the scattered luminance calculation processing unit 5 and the measurement point of the spectral radiance L _D (b, i, j) in the dark area in a certain small area S (i, j) and the atmospheric scattered radiance L _scat (b, i , J) is low in correlation with the model, the atmospheric scattering radiance L _scat (b, i, j) in the small region S (i, j) is regarded as an abnormal value, for example, a negative value is output. To.
Thereby, the atmospheric scattering radiance L _scat (b, i) in the small region S (i, j) including the dark region where the measurement point of the spectral radiance L _D (b, i, j) is not similar to the model. , J) can be removed.
In addition, as a case where the correlation between the measurement point and the model is low, for example, a case where an R square value, which is a determination coefficient of regression analysis in the least square method, is equal to or less than a threshold value Rth can be cited.

The atmospheric transmittance calculation unit 6 acquires the scattered transmission characteristic data f (a) stored in the parameter storage unit 2 (step ST7 in FIG. 4). a = (b, L _scat (b, i, j)).
The scattered transmission characteristic data f (a) stored in the parameter storage unit 2 is the atmospheric transmittance τ of the atmospheric scattering radiance L _scat (b, i, j) indicating the atmospheric scattering in the small region S (i, j). It is a function for each band b indicating (b, i, j).
The scattered transmission characteristic data f (a) can be calculated by a light wave atmospheric propagation calculation simulator such as MODTRAN, for example.
FIG. 9 is an explanatory diagram showing the atmospheric transmittance τ (b, i, j) corresponding to the atmospheric scattered radiance L _scat (b, i, j) of the small region S (i, j) in the band b.

As shown in the following formula (2), the atmospheric transmittance calculation unit 6 uses the band b and the small region S (output from the scattering luminance calculation processing unit 5) in the variable a of the scattered transmission characteristic data f (a). By substituting the atmospheric scattering radiance L _scat (b, i, j) for i, j), the atmospheric transmittance τ (b, i) for atmospheric scattering in the small region S (i, j) for each band b. , J) is calculated (step ST8 in FIG. 4).
τ (b, i, j) = f (b, L _scat (b, i, j)) (2)
The atmospheric transmittance calculating unit 6 outputs the calculated atmospheric transmittance τ (b, i, j) of atmospheric scattering in each small region S (i, j) to the transmittance correction value calculating unit 9.
However, if the atmospheric transmittance τ (b, i, j) in a certain small region S (i, j) is separated from the assumed value by several percent or more, the atmospheric transmittance calculating unit 6 For example, a negative value is output as an abnormal value of the atmospheric transmittance τ (b, i, j) in S (i, j).

The luminance correction value calculation unit 8 of the correction value calculation unit 7 acquires the wavelength characteristic data stored in the parameter storage unit 2 (step ST9 in FIG. 4).
The wavelength characteristic data stored in the parameter storage unit 2 is data indicating each of the observable wavelength band of the multispectral sensor and the observable wavelength band of the panchromatic sensor.
The brightness correction value calculation unit 8 uses the acquired wavelength characteristic data, and the atmospheric scattering radiance L _scat (b in each small region S (i, j) for each band b output from the scattering brightness calculation processing unit 5. , I, j) is calculated. Details of the processing for calculating the weighting coefficient ε (b) will be described later.

ｂ＝ｂ１，ｂ２，ｂ３，ｂ４
Ｐａ’は、パンクロマチックセンサにより観測されるパンクロマチック画像に相当する画像の単一波長帯である。
輝度補正値算出部８は、各々の小領域Ｓ（ｉ，ｊ）における分光放射輝度の第１の補正値Ｌ_ｓｃａｔ（Ｐａ’，ｉ，ｊ）を画像再結合部１０に出力する。
なお、輝度補正値算出部８は、散乱輝度算出処理部５から出力された小領域Ｓ（ｉ，ｊ）における大気散乱放射輝度Ｌ_ｓｃａｔ（ｂ，ｉ，ｊ）が異常値である場合、異常値である大気散乱放射輝度Ｌ_ｓｃａｔ（ｂ，ｉ，ｊ）を用いずに、小領域Ｓ（ｉ，ｊ）における分光放射輝度の第１の補正値Ｌ_ｓｃａｔ（Ｐａ’，ｉ，ｊ）を算出するようにする。
例えば、輝度補正値算出部８は、大気散乱放射輝度Ｌ_ｓｃａｔ（ｂ，ｉ，ｊ）が異常値である小領域Ｓ（ｉ，ｊ）の周辺の小領域における大気散乱放射輝度を用いて、小領域Ｓ（ｉ，ｊ）における大気散乱放射輝度Ｌ_ｓｃａｔ（ｂ，ｉ，ｊ）を補間する。
そして、輝度補正値算出部８は、補間した大気散乱放射輝度Ｌ_ｓｃａｔ（ｂ，ｉ，ｊ）を用いて、第１の補正値Ｌ_ｓｃａｔ（Ｐａ’，ｉ，ｊ）を算出するようにする。As shown in the following formula (3), the luminance correction value calculation unit 8 sets the first correction value L _scat (Pa ′, i, j) of the spectral radiance in each small region S (i, j). , The sum of the atmospheric scattering radiance L _scat (b, i, j) in each small region S (i, j) for each band b is calculated using the weight coefficient ε (b) (step ST10 in FIG. 4). ).

b = b1, b2, b3, b4
Pa ′ is a single wavelength band of an image corresponding to the panchromatic image observed by the panchromatic sensor.
The luminance correction value calculation unit 8 outputs the first correction value L _scat (Pa ′, i, j) of the spectral radiance in each small region S (i, j) to the image recombination unit 10.
Note that the luminance correction value calculation unit 8 is abnormal when the atmospheric scattering radiance L _scat (b, i, j) in the small region S (i, j) output from the scattering luminance calculation processing unit 5 is an abnormal value. The first correction value L _scat (Pa ′, i, j) of the spectral radiance in the small region S (i, j) is used without using the atmospheric scattering radiance L _scat (b, i, j) that is a value. Try to calculate.
For example, the brightness correction value calculation unit 8 uses the atmospheric scattering radiance in the small area around the small area S (i, j) where the atmospheric scattered radiance L _scat (b, i, j) is an abnormal value, The atmospheric scattering radiance L _scat (b, i, j) in the small region S (i, j) is interpolated.
Then, the luminance correction value calculation unit 8 calculates the first correction value L _scat (Pa ′, i, j) using the interpolated atmospheric scattering radiance L _scat (b, i, j). .

Here, the calculation process of the weighting coefficient ε (b) will be described.
FIG. 10 is an explanatory diagram showing the observable wavelength band of the multispectral sensor and the observable wavelength band of the panchromatic sensor.
In the example of FIG. 10, the observable wavelength band of the panchromatic sensor is λ ₀ (Pa) to λ ₁ (Pa).
The observable wavelength bands of the multispectral sensor are λ ₀ (b1) to λ ₁ (b1), λ ₀ (b2) to λ ₁ (b2), λ ₀ (b3) to λ ₁ (b3), and λ ₀ ( b4) to λ ₁ (b4).

ｂ＝ｂ１，ｂ２，ｂ３，ｂ４Luminance correction value calculation unit 8, to the observable wavelength band lambda ₀ panchromatic sensor _{(Pa) ~λ 1 (Pa)} , observable wavelength band of the multispectral sensor λ _{'0 (b)} ~λ' By assigning ₁ (b), the weighting coefficient ε (b) is calculated as shown in the following equation (4).

b = b1, b2, b3, b4

Here, the allocation of the wavelength band lambda multispectral sensor _{'0 (b) ~λ' 1} (b) in the luminance correction value calculation unit 8, for example, a wavelength band _{λ '0 (b) ~λ'} 1 (b) Is performed so as to correspond to each of the bands b1, b2, b3, and b4 of the image included in the multispectral image.
That is, the luminance correction value calculation unit 8, assignment waveband lambda to _{'0 (b1) ~λ' 1} (b1) to the band b1, waveband to the band _{b2 λ '0 (b2) ~λ} ' 1 (b2) allocation, and assign a wavelength band _{λ '0 (b3) ~λ'} 1 (b3) to the band b3.
Further, the luminance correction value calculation unit 8 assigns the wavelength bands λ ′ ₀ (b4) to λ ′ ₁ (b4) to the band b4.
In the case of such allocation, for example, the wavelength bands λ ′ ₀ (b1) and λ ′ ₁ (b1) are expressed as in the following equations (5) and (6).

The numerator in the weighting coefficient ε (b) simulates the wavelength integration with respect to the spectral radiance in the assigned wavelength band λ ′ ₀ (b) to λ ′ ₁ (b).
Further, the denominator in the weighting coefficient ε (b) means that the radiance of the panchromatic sensor is converted into the spectral radiance.
The above formula (3) assumes that the spectral radiance in the assigned wavelength band λ ′ ₀ (b) to λ ′ ₁ (b) is constant, and assigns it to the wavelength band of the panchromatic sensor. As the number of bands increases, the first correction value L _scat (Pa ′, i, j) of the spectral radiance can be calculated with higher accuracy.

Here, an example is shown in which the luminance correction value calculation unit 8 calculates the weighting coefficient ε (b) by Expression (4). However, as shown in Expression (7) below, each wavelength in the panchromatic sensor is calculated. The weighting coefficient ε (b) may be calculated based on the spectral sensitivity R (λ) of λ. In this case, even if the panchromatic sensor is a sensor whose sensitivity changes remarkably at the wavelength λ of each band b, the first correction value L _scat (Pa ′, i, j) of the spectral radiance is increased. It can be calculated with accuracy.

Further, the luminance correction value calculation unit 8 may calculate the weighting coefficient ε (b) using a panchromatic image of a real image.
For example, the luminance correction value calculation unit 8 acquires the spectral radiance L (Pa, x, y) of the panchromatic image of the actual image.
(X, y) are coordinates on the panchromatic image, and are different from (i, j) indicating the arrangement of the small regions in the multispectral image.
Then, the luminance correction value calculation unit 8 simulates the spectral radiance first correction value L _scat (Pa ′, i, j) obtained from Equation (3) from the multispectral image. Assuming that the luminance is L (Pa ′, x, y), the difference between the spectral radiance L (Pa ′, x, y) and the spectral radiance L (Pa, x, y) is minimized in Equation (3). Such a weight coefficient ε (b) is calculated.

The transmittance correction value calculation unit 9 of the correction value calculation unit 7 acquires the wavelength characteristic data stored in the parameter storage unit 2 (step ST11 in FIG. 4).
The transmittance correction value calculation unit 9 uses the acquired wavelength characteristic data, and the atmospheric transmittance τ of atmospheric scattering in each small region S (i, j) for each band b output from the atmospheric transmittance calculation unit 6. A weight coefficient ε ′ (b) of (b, i, j) is calculated. Details of the processing for calculating the weighting coefficient ε ′ (b) will be described later.

ｂ＝ｂ１，ｂ２，ｂ３，ｂ４
透過率補正値算出部９は、各々の小領域Ｓ（ｉ，ｊ）における分光放射輝度の第２の補正値τ（Ｐａ’，ｉ，ｊ）を画像再結合部１０に出力する。
なお、透過率補正値算出部９は、大気透過率算出部６から出力された小領域Ｓ（ｉ，ｊ）における大気散乱の大気透過率τ（ｂ，ｉ，ｊ）が異常値である場合、異常値である大気透過率τ（ｂ，ｉ，ｊ）を用いずに、小領域Ｓ（ｉ，ｊ）における分光放射輝度の第２の補正値τ（Ｐａ’，ｉ，ｊ）を算出するようにする。
例えば、透過率補正値算出部９は、大気透過率τ（ｂ，ｉ，ｊ）が異常値である小領域Ｓ（ｉ，ｊ）の周辺の小領域における大気透過率を用いて、小領域Ｓ（ｉ，ｊ）における大気散乱の大気透過率τ（ｂ，ｉ，ｊ）を補間する。
そして、透過率補正値算出部９は、補間した大気透過率τ（ｂ，ｉ，ｊ）を用いて、第２の補正値τ（Ｐａ’，ｉ，ｊ）を算出するようにする。The transmittance correction value calculation unit 9 uses the spectral radiance second correction value τ (Pa ′, i, j) in each small region S (i, j) as shown in the following equation (8). , The sum of the atmospheric transmittance τ (b, i, j) of atmospheric scattering in each small region S (i, j) for each band b is calculated using the weight coefficient ε ′ (b) (FIG. 4). Step ST12).

b = b1, b2, b3, b4
The transmittance correction value calculation unit 9 outputs the second correction value τ (Pa ′, i, j) of the spectral radiance in each small region S (i, j) to the image recombination unit 10.
It should be noted that the transmittance correction value calculation unit 9 has an abnormal value for the atmospheric transmittance τ (b, i, j) of atmospheric scattering in the small region S (i, j) output from the atmospheric transmittance calculation unit 6. The second correction value τ (Pa ′, i, j) of the spectral radiance in the small region S (i, j) is calculated without using the atmospheric transmittance τ (b, i, j) that is an abnormal value. To do.
For example, the transmittance correction value calculation unit 9 uses the atmospheric transmittance in the small region around the small region S (i, j) where the atmospheric transmittance τ (b, i, j) is an abnormal value, to use the small region. Interpolate atmospheric transmittance τ (b, i, j) of atmospheric scattering at S (i, j).
Then, the transmittance correction value calculation unit 9 calculates the second correction value τ (Pa ′, i, j) using the interpolated atmospheric transmittance τ (b, i, j).

光源の照度Ｅ（λ）として、一般的な自然光である太陽の照度を用いることができる。
式（９）は、パンクロマチックセンサに入射される光源からの光のエネルギーに対する割り付け波長のエネルギー比を模擬している。Here, the calculation process of the weighting coefficient ε ′ (b) will be described.
The weighting coefficient ε ′ (b) of the atmospheric transmittance τ (b, i, j) can be calculated by the equation (4) similarly to the luminance correction value calculation unit 8.
The transmittance correction value calculation unit 9 uses the spectral sensitivity R (λ) of each wavelength λ and the illuminance E (λ) of the light source in the panchromatic sensor, as shown in the following formula (9), to give a weighting coefficient ε ′. (B) may be calculated.

The illuminance of the sun, which is general natural light, can be used as the illuminance E (λ) of the light source.
Expression (9) simulates the energy ratio of the assigned wavelength to the energy of light from the light source incident on the panchromatic sensor.

透過率補正値算出部９は、式（８）から得られる第２の補正値τ（Ｐａ’，ｉ，ｊ）を、マルチスペクトル画像から模擬した透過率τ（Ｐａ’，ｘ，ｙ）とみなし、式（８）において、透過率τ（Ｐａ’，ｘ，ｙ）と透過率τ（Ｐａ，ｘ，ｙ）との差が最小になるような重み係数ε’（ｂ）を算出する。Further, the transmittance correction value calculation unit 9 may calculate the weighting coefficient ε ′ (b) using the panchromatic image of the actual image.
For example, the transmittance correction value calculation unit 9 acquires a panchromatic image that is a real image in which a subject showing a known spectral radiance L _sample and a dark area are shown.
Then, the transmittance correction value calculation unit 9 calculates a difference dL between the known spectral radiance L _sample and the spectral radiance in the dark part region, and the difference dL and the known spectral value are calculated as shown in the following equation (10). The transmittance τ (Pa, x, y) is calculated from the radiance L _sample .

The transmittance correction value calculation unit 9 uses the second correction value τ (Pa ′, i, j) obtained from Equation (8) as the transmittance τ (Pa ′, x, y) that simulates the multispectral image. Assuming that in equation (8), the weighting coefficient ε ′ (b) is calculated so that the difference between the transmittance τ (Pa ′, x, y) and the transmittance τ (Pa, x, y) is minimized.

The image recombination unit 10 of the correction value calculation unit 7 includes the first correction value L _scat (Pa ′, i) of the spectral radiance in each small region S (i, j) output from the luminance correction value calculation unit 8. , J) are combined to generate a two-dimensional array A of first correction values (step ST13 in FIG. 4).
Further, the image recombination unit 10 calculates the second correction value τ (Pa ′, i, j) of the spectral radiance in each small region S (i, j) output from the transmittance correction value calculation unit 9. By combining, a two-dimensional array B of second correction values is generated (step ST14 in FIG. 4).
Each element (i, j) in the two-dimensional arrays A and B corresponds to each small area S (i, j) divided by the image dividing unit 1. If the number of divisions of the multispectral image is N and the number of small regions S (i, j) is N, the number of elements (i, j) in the two-dimensional array is also N.
Each element (i, j) in the two-dimensional array A has a first correction value L _scat (Pa ′, i, j) of the spectral radiance in each small region S (i, j). Each element (i, j) in the two-dimensional array B has a second correction value τ (Pa ′, i, j) of the spectral radiance in each small region S (i, j). .
The image recombination unit 10 outputs the generated two-dimensional arrays A and B to the correction value interpolation unit 11.

The correction value interpolation unit 11 of the correction value calculation unit 7 includes an image recombination unit 10 so that each of the resolutions of the two-dimensional arrays A and B output from the image recombination unit 10 matches the resolution of the panchromatic image. 2 are interpolated (step ST15 in FIG. 4).
For example, if the number of pixels in the x direction of the panchromatic image is Mx times the number of elements (i, j) in the x direction in the two-dimensional arrays A and B, the x direction of the two-dimensional arrays A and B is multiplied by Mx. Upsampling to
When the number of pixels in the y direction of the panchromatic image is My times the number of elements (i, j) in the y direction in the two-dimensional arrays A and B, the y direction of the two-dimensional arrays A and B is My times. Upsampling to
Thereby, the resolution of the two-dimensional arrays A and B becomes equal to the resolution of the panchromatic image.

As the interpolation process for the two-dimensional arrays A and B, an interpolation process is preferable in which correction values between the elements of the two-dimensional arrays A and B are continuous. For example, an interpolation process such as bilinear or cubic conversion is performed. Can be considered. In addition, interpolation processing such as a Bezier curve or a Spline curve may be performed so that correction values between elements of the two-dimensional arrays A and B are continuous and draw a curve.
Accordingly, the first correction value L _scat (Pa ′, i, j) is converted into the first correction value L _scat (Pa ′, x, y), and the first correction value L _scat ( Pa ′, x, y) is output from the correction value interpolation unit 11 to the radiance correction unit 13. Further, the second correction value τ (Pa ′, i, j) is converted into the second correction value τ (Pa ′, x, y), and the second correction value τ (Pa ′, x) after conversion. , Y) is output from the correction value interpolation unit 11 to the transmittance correction unit 14.
When the first correction value L _scat (Pa ′, i, j) of a certain element of the two-dimensional array A is an abnormal value, the correction value interpolation unit 11 exists around the element. Based on the correction value possessed by the element, the correction value possessed by the element is interpolated, and then the resolution of the two-dimensional array A is converted.
Further, when the second correction value τ (Pa ′, i, j) of a certain element of the two-dimensional array B is an abnormal value, the correction value interpolation unit 11 exists around the element. Based on the correction value of the element, the correction value of the element is interpolated, and then the resolution of the two-dimensional array B is converted.

The radiance correction unit 13 of the correction unit 12 acquires a panchromatic image from the panchromatic sensor (step ST16 in FIG. 4).
The radiance correction unit 13 performs the first correction output from the correction value interpolation unit 11 from the spectral radiance L (Pa, x, y) of the acquired panchromatic image, as shown in the following equation (11). The spectral radiance in the panchromatic image is corrected by subtracting the value L _scat (Pa ′, x, y) (step ST17 in FIG. 4).
L1 (x, y)
= L (Pa, x, y) -Lscat (Pa ', x, y) (11)
In Expression (11), L1 (x, y) is the spectral radiance after correction.
The radiance correction unit 13 outputs a panchromatic image whose spectral radiance is corrected to the transmittance correction unit 14.

式（１２）において、Ｌ２（ｘ，ｙ）は、補正後の分光放射輝度である。
透過率補正部１４は、分光放射輝度を補正したパンクロマチック画像を出力する。The transmittance correction unit 14 of the correction unit 12 corrects the spectral radiance L1 (x, y) after correction of the panchromatic image output from the radiance correction unit 13 as shown in the following equation (12). By dividing by the second correction value τ (Pa ′, x, y) output from the value interpolation unit 11, the spectral radiance in the panchromatic image is further corrected (step ST18 in FIG. 4).

In Expression (12), L2 (x, y) is the spectral radiance after correction.
The transmittance correction unit 14 outputs a panchromatic image with the spectral radiance corrected.

  As is apparent from the above, according to the first embodiment, from the atmospheric scattered radiance in each region calculated for each wavelength band by the scattering luminance calculation unit 3, a single wavelength band including a plurality of wavelength bands is obtained. A correction value calculation unit 7 that calculates a correction value of spectral radiance in a panchromatic image that is an image is provided, and the correction unit 12 uses the correction value of spectral radiance calculated by the correction value calculation unit 7 to use panchromatic. Since it is configured to correct the spectral radiance in the image, even if the panchromatic image acquired from the panchromatic sensor is an image including a plurality of regions having greatly different reflectances, the influence of haze or fog is affected. There is an effect that it is possible to correct the lowered contrast.

  Further, according to the first embodiment, the correction value calculation unit 7 calculates the spectral radiance of the panchromatic image from the atmospheric transmittance in each region calculated by the atmospheric transmittance calculation unit 6 for each wavelength band. 2 is calculated, and the correction unit 12 is configured to correct the spectral radiance in the panchromatic image using the second correction value. Therefore, only the first correction value is used to correct the panchromatic image. Compared to correcting spectral radiance in an image, the contrast correction accuracy can be improved.

Embodiment 2. FIG.
In the first embodiment, the correction unit 12 uses the first correction value L _scat (Pa ′, x, y) and the second correction value τ (Pa ′, x, y) in the panchromatic image. The example which correct | amends spectral radiance is shown.
In the second embodiment, the third correction value is calculated to remove the offset component included in the panchromatic image, and the spectral radiance in the panchromatic image is further calculated using the third correction value. An example of correction will be described.

FIG. 11 is a block diagram showing an image processing apparatus according to Embodiment 2 of the present invention, and FIG. 12 is a hardware block diagram showing an image processing apparatus according to Embodiment 2 of the present invention.
11 and 12, the same reference numerals as those in FIGS. 1 and 2 indicate the same or corresponding parts, and thus the description thereof is omitted.
The reflection luminance calculation unit 41 is realized by, for example, a reflection luminance calculation circuit 51 shown in FIG.
The reflected luminance calculation unit 41 uses the spectral radiance L _sensor (b, i, j) in each small region S (i, j) for each band b of the multispectral image and the scattered luminance calculation processing unit 5 for each band b. The atmospheric reflection radiance indicating the atmospheric reflection in each small region S (i, j) from the atmospheric scattering radiance L _scat (b, i, j) in each small region S (i, j) calculated in A process of calculating Lr (b, i, j) is performed.

The correction value calculation unit 42 is realized by, for example, the correction value calculation circuit 52 illustrated in FIG. 2, and includes a luminance correction value calculation unit 8, a transmittance correction value calculation unit 9, a reflection correction value calculation unit 43, and an image recombination. Unit 44 and correction value interpolation unit 45.
The reflection correction value calculation unit 43 of the correction value calculation unit 42 is the atmospheric reflection radiance Lr (b, i, j) in each small region S (i, j) calculated for each band b by the reflection luminance calculation unit 41. From this, a process of calculating the third correction value Lr (Pa ′, i, j) of the spectral radiance in each small region is performed.

The image recombination unit 44 of the correction value calculation unit 42 performs the process of generating the two-dimensional array A and the two-dimensional array B, similarly to the image recombination unit 10 of the first embodiment.
Further, the image recombination unit 44 combines the third correction values Lr (Pa ′, i, j) of the spectral radiance in the respective small regions S (i, j) calculated by the reflection correction value calculation unit 43. Then, a process of generating a two-dimensional array C of third correction values is performed.
Each element in the two-dimensional array C corresponds to each small region S (i, j) divided by the image dividing unit 1. Each element in the two-dimensional array C has a third correction value of the spectral radiance in each small area S (i, j) divided by the image dividing unit 1.
The correction value interpolation unit 45 of the correction value calculation unit 42 includes an image recombination unit 44 so that the resolution of the two-dimensional arrays A, B, and C generated by the image recombination unit 44 matches the resolution of the panchromatic image. The process of interpolating the two-dimensional arrays A, B, and C generated by the above is performed.

The correction unit 46 is realized by the correction circuit 53 illustrated in FIG. 12, and includes a radiance correction unit 13, a transmittance correction unit 14, and a reflection correction unit 47.
The reflection correction unit 47 of the correction unit 46 performs a process of correcting the spectral radiance in the panchromatic image using the third correction value of each element in the two-dimensional array C interpolated by the correction value interpolation unit 45. carry out.

  In FIG. 11, the image dividing unit 1, the parameter storage unit 2, the scattered luminance calculation unit 3, the atmospheric transmittance calculation unit 6, the reflection luminance calculation unit 41, the correction value calculation unit 42, and the correction unit 46 that are components of the image processing apparatus. Are assumed to be realized by dedicated hardware as shown in FIG. That is, it is assumed that the image dividing circuit 21, the parameter storage circuit 22, the scattering luminance calculation circuit 23, the transmittance calculation circuit 24, the reflection luminance calculation circuit 51, the correction value calculation circuit 52, and the correction circuit 53 are realized.

The image dividing circuit 21, the scattered luminance calculating circuit 23, the transmittance calculating circuit 24, the reflected luminance calculating circuit 51, the correction value calculating circuit 52, and the correcting circuit 53 are, for example, a single circuit, a composite circuit, a programmed processor, a parallel program An integrated processor, ASIC, FPGA, or a combination thereof is applicable.
The components of the image processing apparatus are not limited to those realized by dedicated hardware, and the image processing apparatus may be realized by software, firmware, or a combination of software and firmware.

When the image processing apparatus is realized by software, firmware, or the like, the parameter storage unit 2 is configured on the memory 31 of the computer shown in FIG. 3, and the image division unit 1, the scattered luminance calculation unit 3, and the atmospheric transmittance calculation unit 6. A program for causing the computer to execute the processing procedure of the reflection luminance calculation unit 41, the correction value calculation unit 42, and the correction unit 46 is stored in the memory 31, and the processor 32 of the computer executes the program stored in the memory 31. What should I do?
12 shows an example in which each component of the image processing apparatus is realized by dedicated hardware, and FIG. 3 shows an example in which the image processing apparatus is realized by software, firmware, or the like. Some components in the image processing apparatus may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, or the like.

Next, the operation will be described.
When the concentration of haze or fog is high, in addition to atmospheric scattering in which light is scattered to atmospheric particles that exist between the subject and the sensor, atmospheric reflection in which light is reflected to atmospheric particles occurs. Sometimes.
When atmospheric reflection occurs, a reflected light component from the atmosphere is superimposed on the multispectral image and panchromatic image as an offset component.
In the second embodiment, the reflection correction unit 47 of the correction unit 46 performs a process of removing the offset component superimposed on the panchromatic image.
Hereinafter, the differences from the first embodiment will be described.

The reflection luminance calculation unit 41 acquires the spectral radiance L _sensor (b, i, j) in each small region S (i, j) for each band b of the multispectral image divided by the image dividing unit 1.
The reflection luminance calculation unit 41 calculates the spectral radiance L _sensor (b, i, j) in each small region S (i, j) by the scattered luminance calculation processing unit 5 as shown in the following equation (13). The calculated atmospheric scattering radiance L _scat (b, i, j) in each small region S (i, j) and the atmospheric reflection radiance Lr (b, i, j) in each small region S (i, j). Model with the sum of j).
L _sensor (b, i, j)
= L _scat (b, i, j) + Lr (b, i, j) (13)
FIG. 13 is an explanatory diagram showing the spectral radiance L _sensor (b, i, j) modeled by the reflected luminance calculation unit 41.
In FIG. 13, the solid line is a model of spectral radiance L _sensor (b, i, j), and ◯ is the spectral radiance of the dark area in the small area S (i, j).

式（１５）において、ρ（λ）は、白色の被写体の反射率、Ｅ（λ）は、光源である太陽の照度である。
式（１３）における大気散乱放射輝度Ｌ_ｓｃａｔ（ｂ，ｉ，ｊ）は、式（１）のように表されるので、式（１３）は、以下の式（１６）のように表される。
Ｌ_{ｓｅｎｓｏｒ}（ｂ，ｉ，ｊ）
＝α（ｂ）Ｌ_{ｓｃａｔ０}（ｉ，ｊ）＋β（ｂ）Ｌ_ｒ０（ｉ，ｊ） （１６）Here, assuming that only the absolute value of the reflected light component changes without changing the spectral characteristics of the reflected light component from the atmosphere, the atmospheric reflected radiance in each small region S (i, j) of the multispectral image. Lr (b, i, j) can be expressed as in the following formula (14).
Lr (b, i, j) = β (b) L _r0 (i, j) (14)
In Expression (14), β (b) is a spectral characteristic component of reflected light, and L _r0 (i, j) is an eigenvalue of the reflected light component.
The spectral characteristic component β (b) of the reflected light can be calculated from the spectral radiance of the white subject, as shown in the following equation (15).

In equation (15), ρ (λ) is the reflectance of the white subject, and E (λ) is the illuminance of the sun as the light source.
Since the atmospheric scattering radiance L _scat (b, i, j) in Expression (13) is expressed as Expression (1), Expression (13) is expressed as Expression (16) below. .
L _sensor (b, i, j)
= Α (b) L _scat0 (i, j) + β (b) L _r0 (i, j) (16)

The reflected luminance calculation unit 41 estimates the eigenvalue L _r0 (i, j) of the reflected light component by regression analysis in the model of Expression (16).
Reflection luminance calculation unit 41, the eigenvalues _L r0 (i, j) of the reflected light component when estimating the eigenvalues _L r0 estimated (i, j) and spectral characteristics ingredient beta (b) and the expression for the reflected light (14 ) To calculate the atmospheric reflection radiance Lr (b, i, j) in each small region S (i, j) of the multispectral image.
The reflection luminance calculation unit 41 outputs the calculated atmospheric reflection radiance Lr (b, i, j) in each small region S (i, j) to the reflection correction value calculation unit 43.

The reflection correction value calculation unit 43 of the correction value calculation unit 42 acquires the wavelength characteristic data stored in the parameter storage unit 2.
The reflection correction value calculation unit 43 uses the acquired wavelength characteristic data to reflect the atmospheric reflection radiance Lr (b, i) in each small region S (i, j) for each band b output from the reflection luminance calculation unit 41. , J) is calculated. The weighting coefficient ε ″ (b) can be calculated in the same manner as the weighting coefficient ε (b) or the weighting coefficient ε ′ (b), for example.

ｂ＝ｂ１，ｂ２，ｂ３，ｂ４
反射補正値算出部４３は、各々の小領域Ｓ（ｉ，ｊ）における分光放射輝度の第３の補正値Ｌｒ（Ｐａ’，ｉ，ｊ）を画像再結合部４４に出力する。The reflection correction value calculation unit 43 uses the spectral radiance third correction value Lr (Pa ′, i, j) in each small region S (i, j) as shown in the following equation (17): Using the weighting coefficient ε ″ (b), the sum of the atmospheric reflection radiance Lr (b, i, j) in each small region S (i, j) for each band b is calculated.

b = b1, b2, b3, b4
The reflection correction value calculation unit 43 outputs the third correction value Lr (Pa ′, i, j) of the spectral radiance in each small region S (i, j) to the image recombination unit 44.

Similar to the image recombination unit 10 of the first embodiment, the image recombination unit 44 of the correction value calculation unit 42 includes the two-dimensional array A of the first correction values and the two-dimensional array B of the second correction values. Generate each.
Further, the image recombination unit 44 combines the third correction values Lr (Pa ′, i, j) of the spectral radiances output from the reflection correction value calculation unit 43 in the respective small regions S (i, j). Then, a two-dimensional array C of third correction values is generated.
Each element (i, j) in the two-dimensional array C corresponds to each small region S (i, j) divided by the image dividing unit 1. If the number of divisions of the multispectral image is N and the number of small regions S (i, j) is N, the number of elements (i, j) of the two-dimensional array C is also N.
Each element (i, j) in the two-dimensional array C has a third correction value Lr (Pa ′, i, j) of the spectral radiance in each small region S (i, j).
The image recombination unit 44 outputs the generated two-dimensional arrays A, B, and C to the correction value interpolation unit 45.

The correction value interpolation unit 45 of the correction value calculation unit 42 interpolates each of the two-dimensional arrays A and B output from the image recombination unit 44, similarly to the correction value interpolation unit 11 of the first embodiment.
The correction value interpolation unit 45 interpolates the two-dimensional array C output from the image recombination unit 44.
For example, if the number of pixels in the x direction of the panchromatic image is Mx times the number of elements (i, j) in the x direction in the two-dimensional array C, the x direction of the two-dimensional array C is upsampled to Mx times. .
As a result, the third correction value Lr (Pa ′, i, j) is converted into the third correction value Lr (Pa ′, x, y), and the third correction value Lr (Pa ′, after conversion) is converted. x, y) is output from the correction value interpolation unit 45 to the reflection correction unit 47.

The reflection correction unit 47 of the correction unit 46 calculates a correction value from the corrected spectral radiance L1 (x, y) of the panchromatic image output from the radiance correction unit 13 as shown in the following equation (18). By subtracting the third correction value Lr (Pa ′, x, y) output from the interpolation unit 45, the spectral radiance in the panchromatic image is corrected.
L3 (x, y) = L1 (x, y) −Lr (Pa ′, x, y) (18)
In Expression (18), L3 (x, y) is the spectral radiance after correction.
The reflection correction unit 47 outputs a panchromatic image whose spectral radiance is corrected to the transmittance correction unit 14.
In the second embodiment, the reflection correction unit 47 is provided at the subsequent stage of the radiance correction unit 13, but the reflection correction unit 47 may be provided at the previous stage of the radiance correction unit 13. .

  As is apparent from the above, according to the second embodiment, the correction value calculation unit 42 uses the atmospheric reflection radiance in each region calculated by the reflection luminance calculation unit 41 for each wavelength band, in the panchromatic image. In addition to calculating the third correction value of the spectral radiance, the correction unit 46 corrects the spectral radiance in the panchromatic image using the first and second correction values calculated by the correction value calculation unit 42. In addition, since the third correction value calculated by the correction value calculation unit 42 is used to correct the spectral radiance in the panchromatic image, the same effects as in the first and second embodiments can be obtained. In addition, there is an effect that the reflected light component from the atmosphere superimposed on the multispectral image and the panchromatic image can be removed.

Embodiment 3 FIG.
In the third embodiment, an example is described in which spectral radiance in a multispectral image is also corrected, and color synthesis is performed between a panchromatic image with corrected spectral radiance and a multispectral image with corrected spectral radiance. To do.

FIG. 14 is a block diagram showing an image processing apparatus according to Embodiment 3 of the present invention, and FIG. 15 is a hardware block diagram showing an image processing apparatus according to Embodiment 3 of the present invention.
14 and 15, the same reference numerals as those in FIGS. 11 and 12 indicate the same or corresponding parts, and thus description thereof is omitted.
In the third embodiment, the correction value calculation unit 42 calculates the correction value of the spectral radiance in the panchromatic image as well as the correction value of the spectral radiance in the multispectral image, as in the second embodiment. Perform the calculation process.
In the third embodiment, the correction unit 46 performs a process of correcting the spectral radiance in the multispectral image in addition to correcting the spectral radiance in the panchromatic image, as in the second embodiment.

The color composition processing unit 48 is realized by the color composition processing circuit 54 shown in FIG.
The color composition processing unit 48 performs color composition processing of the panchromatic image whose spectral radiance is corrected by the correction unit 46 and the multispectral image whose spectral radiance is corrected by the correction unit 46.
FIG. 14 shows an example in which the color composition processing unit 48 is applied to the image processing apparatus in FIG. 11 in the second embodiment. However, the color composition processing unit 48 in FIG. It may be applied to the image processing apparatus.

  In FIG. 14, the image dividing unit 1, the parameter storage unit 2, the scattered luminance calculation unit 3, the atmospheric transmittance calculation unit 6, the reflection luminance calculation unit 41, the correction value calculation unit 42, and the correction unit 46, which are components of the image processing apparatus. Further, it is assumed that each of the color composition processing unit 48 is realized by dedicated hardware as shown in FIG. That is, what is realized by the image dividing circuit 21, the parameter storage circuit 22, the scattering luminance calculation circuit 23, the transmittance calculation circuit 24, the reflection luminance calculation circuit 51, the correction value calculation circuit 52, the correction circuit 53, and the color composition processing circuit 54. Is assumed.

The image dividing circuit 21, the scattered luminance calculation circuit 23, the transmittance calculation circuit 24, the reflection luminance calculation circuit 51, the correction value calculation circuit 52, the correction circuit 53, and the color composition processing circuit 54 are, for example, a single circuit, a composite circuit, a program An integrated processor, a parallel-programmed processor, an ASIC, an FPGA, or a combination thereof is applicable.
The components of the image processing apparatus are not limited to those realized by dedicated hardware, and the image processing apparatus may be realized by software, firmware, or a combination of software and firmware.

  When the image processing apparatus is realized by software, firmware, or the like, the parameter storage unit 2 is configured on the memory 31 of the computer shown in FIG. 3, and the image division unit 1, the scattered luminance calculation unit 3, and the atmospheric transmittance calculation unit 6. A program for causing the computer to execute the processing procedures of the reflection luminance calculation unit 41, the correction value calculation unit 42, the correction unit 46, and the color composition processing unit 48 is stored in the memory 31, and the processor 32 of the computer is stored in the memory 31. The program that is running should be executed.

Next, the operation will be described.
Hereinafter, the differences from the first and second embodiments will be described.
In the third embodiment, the brightness correction value calculation unit 8 uses the spectral radiance first correction value L _scat (Pa ′, i, j) in each small region S (i, j) as an image recombination unit. In addition, the atmospheric scattering radiance L _scat (b, i, j) in each small region S (i, j) is output to the image recombining unit 44 as a fourth correction value.
In the third embodiment, the transmittance correction value calculation unit 9 uses the second correction value τ (Pa ′, i, j) of the spectral radiance in each small region S (i, j) as an image recombination unit. In addition, the atmospheric transmittance τ (b, i, j) in each small region S (i, j) is output to the image recombining unit 44 as a fifth correction value.
In the third embodiment, the reflection correction value calculation unit 43 uses the image recombination unit 44 as the third correction value Lr (Pa ′, i, j) of the spectral radiance in each small region S (i, j). In addition, the atmospheric reflection radiance Lr (b, i, j) in each small region S (i, j) is output to the image recombining unit 44 as a sixth correction value.

Similar to the second embodiment, the image recombining unit 44 of the correction value calculating unit 42 generates the first two-dimensional array A of correction values, and outputs each of the output from the luminance correction value calculating unit 8. L _scat (b, i, j), which is the fourth correction value of spectral radiance in the small region S (i, j), is combined to generate a two-dimensional array D of fourth correction values.
Further, the image recombination unit 44 generates the second correction value two-dimensional array B, and in addition to the spectral radiance in each small region S (i, j) output from the transmittance correction value calculation unit 9. Are combined with the atmospheric transmittance τ (b, i, j) as the fifth correction value to generate a two-dimensional array E of fifth correction values.
Further, the image recombining unit 44 generates the two-dimensional array C of the third correction values, and the spectral radiance in each small region S (i, j) output from the reflection correction value calculating unit 43. The six-correction atmospheric reflection radiance Lr (b, i, j) is combined to generate a two-dimensional array F of sixth correction values.

The correction value interpolation unit 45 of the correction value calculation unit 42 interpolates each of the two-dimensional arrays A, B, and C output from the image recombination unit 44 as in the second embodiment.
Further, the correction value interpolation unit 45 interpolates each of the two-dimensional arrays D, E, and F output from the image recombination unit 44.
As a result, the fourth correction value L _scat (b, i, j) is converted to L _scat (b, x, y), and the converted fourth correction value L _scat (b, x) , Y) is output from the correction value interpolation unit 45 to the radiance correction unit 13.
The atmospheric transmittance τ (b, i, j) that is the fifth correction value is converted into τ (b, x, y), and τ (b, x, y) that is the fifth correction value after conversion. Is output from the correction value interpolation unit 45 to the transmittance correction unit 14.
The atmospheric reflection radiance Lr (b, i, j) that is the sixth correction value is converted into Lr (b, x, y), and Lr (b, x, y) that is the sixth correction value after conversion. ) Is output from the correction value interpolation unit 45 to the reflection correction unit 47.

The radiance correction unit 13 corrects the spectral radiance in the panchromatic image as in the first and second embodiments.
Further, the radiance correction unit 13 is output from the correction value interpolation unit 45 from the spectral radiance L _sensor (b, i, j) in each small region S (i, j) for each band b of the multispectral image. The spectral radiance in the multispectral image is corrected by subtracting L _scat (b, x, y), which is the fourth correction value.

The reflection correction unit 47 corrects the spectral radiance in the panchromatic image as in the second embodiment.
The reflection correction unit 47 corrects the spectral radiance L _sensor (b, i, j) in each small region S (i, j) after correction of the multispectral image output from the radiance correction unit 13. The spectral radiance in the multispectral image is corrected by subtracting Lr (b, x, y), which is the sixth correction value output from the value interpolation unit 45.

The transmittance correction unit 14 corrects the spectral radiance in the panchromatic image as in the first and second embodiments.
Further, the transmittance correction unit 14 corrects the spectral radiance L _sensor (b, i, j) in each small region S (i, j) after correction of the multispectral image output from the reflection correction unit 47. By dividing by the fifth correction value τ (b, x, y) output from the value interpolation unit 45, the spectral radiance in the multispectral image is corrected.

The color composition processing unit 48 performs color composition processing of the panchromatic image whose spectral radiance has been corrected by the correction unit 46 and the multispectral image whose spectral radiance has been corrected by the correction unit 46, and after color synthesis processing Output the image.
As the color composition process, for example, a color composition process called IHS (Intensity Hue Saturation) conversion can be used. In color synthesis processing called IHS conversion, luminance, hue, and saturation, which are color spaces, are calculated from a multispectral image, and the calculated luminance is replaced with a panchromatic image, thereby obtaining luminance information having the resolution of the panchromatic image. , Adding hue and saturation information.
As the color composition processing, brevey conversion, Gram-Schmidt conversion, conversion processing using principal component analysis, or the like can also be used.
An image that combines a high-resolution panchromatic image and a low-resolution multispectral image and has high resolution and color information may be referred to as a pan-sharpened image.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  The present invention is suitable for an image processing apparatus that corrects spectral radiance in a panchromatic image.

  DESCRIPTION OF SYMBOLS 1 Image division part, 2 Parameter memory | storage part, 3 Scattering brightness | luminance calculation part, 4 Dark part brightness | luminance calculation part, 5 Scattering brightness | luminance calculation process part, 6 Atmospheric transmittance | permeability calculation part, 7 Correction value calculation part, 8 Brightness correction value calculation part, 9 Transmittance correction value calculation unit, 10 image recombination unit, 11 correction value interpolation unit, 12 correction unit, 13 radiance correction unit, 14 transmittance correction unit, 21 image division circuit, 22 parameter storage circuit, 23 scattering luminance calculation circuit , 24 transmittance calculation circuit, 25 correction value calculation circuit, 26 correction circuit, 31 memory, 32 processor, 41 reflection luminance calculation unit, 42 correction value calculation unit, 43 reflection correction value calculation unit, 44 image recombination unit, 45 correction Value interpolation unit, 46 correction unit, 47 reflection correction unit, 48 color synthesis processing unit, 51 reflection luminance calculation circuit, 52 correction value calculation circuit, 53 correction circuit, 54 color synthesis processing circuit.

Claims (4)

An image dividing unit for dividing a multispectral image including images of a plurality of wavelength bands into a plurality of regions;
For each wavelength band of a plurality of images included in the multispectral image, a scattering luminance calculating unit that calculates atmospheric scattering radiance indicating atmospheric scattering in each region divided by the image dividing unit,
A correction value for spectral radiance in a panchromatic image, which is an image in a single wavelength band including the plurality of wavelength bands, is calculated from the atmospheric scattered radiance in each region calculated for each wavelength band by the scattered luminance calculation unit. A correction value calculation unit to
An image processing apparatus comprising: a correction unit that corrects spectral radiance in the panchromatic image using the correction value of spectral radiance calculated by the correction value calculation unit. Calculate atmospheric transmittance of atmospheric scattering in each region divided by the image dividing unit for each wavelength band from atmospheric scattering radiance calculated for each wavelength band by the scattering luminance calculating unit. An atmospheric transmittance calculating unit
The correction value calculation unit calculates spectral radiance in the panchromatic image from the atmospheric scattered radiance in each region calculated for each wavelength band by the scattering luminance calculation unit as a correction value of spectral radiance in the panchromatic image. In addition to calculating the first brightness correction value, the second spectral radiance in the panchromatic image is calculated from the atmospheric transmittance of atmospheric scattering in each region calculated for each wavelength band by the atmospheric transmittance calculator. Calculate the correction value of
In addition to correcting the spectral radiance in the panchromatic image using the first correction value calculated by the correction value calculation unit, the correction unit corrects the second correction calculated by the correction value calculation unit. The image processing apparatus according to claim 1, wherein a spectral radiance in the panchromatic image is corrected using a value. From the spectral radiance in each region for each wavelength band of the multispectral image and the atmospheric scattering radiance in each region calculated for each wavelength band by the scattering luminance calculation unit, for each wavelength band, the image A reflection luminance calculation unit for calculating an atmospheric reflection radiance indicating atmospheric reflection in each region divided by the division unit,
In addition to calculating the first and second correction values, the correction value calculation unit calculates the panchromatic image from the atmospheric reflection radiance in each region calculated for each wavelength band by the reflection luminance calculation unit. Calculating a third correction value of the spectral radiance,
The correction unit corrects spectral radiance in the panchromatic image using the first and second correction values calculated by the correction value calculation unit, and also calculates the first calculated by the correction value calculation unit. The image processing apparatus according to claim 2, wherein a spectral radiance in the panchromatic image is corrected using a correction value of 3. The correction value calculation unit calculates a correction value of the spectral radiance in the multispectral image for each wavelength band from the atmospheric scattering radiance in each region calculated for each wavelength band by the scattering luminance calculation unit. ,
The correction unit corrects the spectral radiance in the multispectral image using the correction value of the spectral radiance in the multispectral image calculated by the correction value calculation unit,
2. A color synthesis processing unit that performs color synthesis of a panchromatic image whose spectral radiance is corrected by the correction unit and a multispectral image whose spectral radiance is corrected by the correction unit. The image processing apparatus described.

Images