WO2024121983A1

WO2024121983A1 - Data processing device, data processing method, and data processing program

Info

Publication number: WO2024121983A1
Application number: PCT/JP2022/045130
Authority: WO
Inventors: 智寛國宗; 茂樹赤木
Original assignee: 三菱電機株式会社
Priority date: 2022-12-07
Filing date: 2022-12-07
Publication date: 2024-06-13
Also published as: JP7486696B1

Abstract

An imaging area designation unit (101) designates two areas of an outer space area in which one or more scanning mirrors and a line sensor are provided on the same optical path that are presumed to have a similar brightness temperature, have a known infrared radiation brightness value, and are areas for which the rotational angle of at least one of the one or more scanning mirrors is different each time image capture is performed as two imaging areas on which an imaging device that is in the outer space area is to perform image capture. An image acquisition unit (102) acquires two captured images obtained as a result of the imaging device performing image capture on the two imaging areas. A monitoring data acquisition unit (103) acquires monitoring data that includes angle data that indicates the rotational angle of each scanning mirror each time image capture is performed on the two imaging areas. A coefficient calculation unit (104) uses the two captured images and the monitoring data to calculate a coefficient for an infrared emissivity function for each of the one or more scanning mirrors.

Description

Data processing device, data processing method and data processing program

This disclosure relates to calibrating the infrared emissivity of a scanning mirror installed in an imaging device in space.

The characteristics of an imaging device are prone to change due to changes in the environment. In order to perform appropriate correction processing and obtain higher quality captured images, it is necessary to accurately grasp the characteristics of the imaging device at the time of capturing the image. For this reason, the characteristic evaluation of the imaging device is updated by capturing an image of a specific object while the imaging device is in operation, or by using a special imaging technique.
In an imaging device equipped with a scanning mirror, the observed radiance changes depending on the angle of the scanning mirror when capturing an infrared image. This is because the infrared emissivity of the coating applied to the surface of the scanning mirror depends on the angle. Therefore, in order to accurately estimate the infrared radiance of a subject captured by the imaging device, it is necessary to properly calibrate the infrared emissivity of the scanning mirror to the angle of the scanning mirror.

Note that a technology related to this disclosure is disclosed in Non-Patent Document 1.

The infrared emissivity of a coated scanning mirror depends on the angle of incidence of light on the scanning mirror. It has been confirmed that the infrared emissivity of a scanning mirror installed in an imaging device mounted on a satellite changes due to changes in the environment between on the ground before launch and in space after launch. In other words, the infrared emissivity of a scanning mirror in space after launch will not match the infrared emissivity measured on the ground before launch. Infrared emissivity is used to calculate the infrared radiance of a subject. For this reason, if the infrared emissivity of the scanning mirror after launch is not accurately known, differences in the angle of the scanning mirror during imaging will cause errors in the calculated infrared radiance.

The primary objective of this disclosure is to reduce the errors contained in such infrared radiance.

The data processing device according to the present disclosure comprises:
an imaging area designation unit that designates two imaging areas to be imaged by an imaging device in outer space in which one or more scanning mirrors and a line sensor are provided on the same optical path, the imaging areas being estimated to have a uniform radiance temperature in the outer space, having known infrared radiance values, and having different rotation angles for each imaging of at least one of the one or more scanning mirrors;
an image acquisition unit that acquires two captured images obtained by the imaging device capturing images of the two imaging regions;
a monitor data acquisition unit that acquires monitor data including angle data indicating a rotation angle of each scanning mirror at each time of imaging the two imaging areas;
The imaging device further comprises a coefficient calculation section which calculates a coefficient of an infrared emissivity function for each of the one or more scanning mirrors using the two captured images and the monitor data.

　According to this disclosure, it is possible to reduce errors contained in infrared radiance.

FIG. 1 is a diagram showing an example of a system configuration according to a first embodiment. FIG. 2 is a diagram showing an example of the internal configuration of an artificial satellite according to the first embodiment. 1 is a diagram showing an example of the internal configuration of an imaging device according to a first embodiment; FIG. 2 is a diagram showing an example of the hardware configuration of the emissivity calibration device according to the first embodiment. FIG. 2 is a diagram showing an example of the functional configuration of the emissivity calibration device according to the first embodiment. 4 is a flowchart showing an example of the operation of the emissivity calibration device according to the first embodiment. 4A and 4B are diagrams showing examples of designating an imaging region according to the first embodiment; 4A and 4B are diagrams showing examples of designating an imaging region according to the first embodiment; 4A and 4B are diagrams showing examples of designating an imaging region according to the first embodiment;

Embodiment 1.
Hereinafter, an embodiment will be described with reference to the drawings.

***Configuration Description***
FIG. 1 shows an example of a system configuration according to the present embodiment.
1, the emissivity calibration device 100 is placed on the ground, while the image capturing device 200 is mounted on an artificial satellite 300. In other words, the image capturing device 200 exists in outer space.

FIG. 2 shows an outline of the internal configuration of the satellite 300.
The artificial satellite 300 includes an image capturing device 200 , an image capturing control device 400 , and a monitor device 500 .
2 shows only the internal configuration necessary for explaining this embodiment, i.e., the mechanism for controlling the operation of the artificial satellite 300 and the like are omitted.
In FIG. 2, an imaging control device 400 controls the imaging of the imaging device 200 .
The monitor device 500 monitors the state of the scanning mirror mounted in the imaging device 200 .
In FIG. 2, the image capturing device 200, the image capturing control device 400, and the monitor device 500 are shown as separate devices, but the image capturing device 200, the image capturing control device 400, and the monitor device 500 may be integrated.

Next, an example of the internal configuration of the imaging device 200 will be outlined.
Fig. 3 shows an outline of the internal structure of the imaging device 200. Note that Fig. 3 shows only the internal configuration necessary for explaining this embodiment.
3, the imaging device 200 includes one or more scanning mirrors and a line sensor on the same optical path. Each scanning mirror is a one-axis driven scanning mirror.
3 shows an example in which the imaging device 200 includes a scanning mirror β, which is a primary scanning mirror, and a scanning mirror α, which is a secondary scanning mirror. The scanning mirror α and the scanning mirror β can rotate around a rotation axis. In other words, the rotation angles of the scanning mirror α and the scanning mirror β can be changed. The rotation angle of the scanning mirror is the amount of displacement (angle) from the reference angle (reference position) of the scanning mirror. The rotation axes of the scanning mirror α and the scanning mirror β do not have to be parallel. For example, they may be perpendicular.
The line sensor captures an image of a region in outer space that is designated as an imaging region by the emissivity calibration device 100 .
The arrows shown in FIG. 3 indicate optical paths.
The monitor device 500 shown in FIG. 2 monitors the rotation angle and temperature of each scanning mirror during imaging as the state of each scanning mirror.

As mentioned above, the infrared emissivity of the scanning mirror before launch does not match that after launch. If the infrared emissivity of the scanning mirror after launch is not accurately known, differences in the angle of the scanning mirror during imaging will cause errors in the infrared radiance of the subject.
For this reason, in this embodiment, the emissivity calibration device 100 specifies two areas in which the rotation angle of the scanning mirror differs for each image capture, causes the imaging device 200 to capture images of the two areas, and estimates the infrared emissivity of the scanning mirror based on the difference in the rotation angle of the scanning mirror from the difference between the two captured images.

More specifically, the emissivity calibration device 100 specifies two imaging regions to be imaged by the imaging device 200, and transmits imaging region data 150 indicating the two imaging regions to the imaging control device 400, as shown in Fig. 2. The method of specifying the imaging regions will be described in detail later.
The imaging control device 400 transfers the imaging area data 150 to the imaging device 200 .
Then, the imaging device 200 captures images of the two imaging regions specified by the imaging region data 150 , and transmits the two captured images 250 to the emissivity calibration device 100 .
Furthermore, the monitor device 500 monitors the rotation angle and temperature of each scanning mirror during imaging, and transmits monitor data 550 indicating the monitoring results to the emissivity calibration device 100. The monitor data 550 includes angle data and temperature data of each scanning mirror at each imaging time.
Then, the emissivity calibration device 100 uses the two captured images 250 and the monitor data 550 to estimate the infrared emissivity of the scanning mirror based on the difference in the rotation angle of the scanning mirror.

FIG. 4 shows an example of the hardware configuration of the emissivity calibration device 100 according to the present embodiment.
FIG. 5 shows an example of the functional configuration of the emissivity calibration device 100 according to this embodiment.
The emissivity calibration device 100 is a computer. The emissivity calibration device 100 corresponds to a data processing device. Moreover, the operation procedure of the emissivity calibration device 100 corresponds to a data processing method. Moreover, a program that realizes the operation of the emissivity calibration device 100 corresponds to a data processing program.

As shown in FIG. 4, the emissivity calibration device 100 includes, as hardware, a processor 901, a main memory device 902, an auxiliary memory device 903, a communication device 904, and an input/output device 905.
5, the emissivity calibration device 100 includes, as its functional configuration, an imaging area designation unit 101, an image acquisition unit 102, a monitor data acquisition unit 103, a coefficient calculation unit 104, a coefficient output unit 105, an imaging area database 106, and an apparatus characteristic database 107. The functions of the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105 are realized by, for example, a program.
The auxiliary storage device 903 stores programs for implementing the functions of the imaging area designation unit 101 , the image acquisition unit 102 , the monitor data acquisition unit 103 , the coefficient calculation unit 104 , and the coefficient output unit 105 .
These programs are loaded from the auxiliary storage device 903 to the main storage device 902. Then, the processor 901 executes these programs to perform the operations of an imaging area designation unit 101, an image acquisition unit 102, a monitor data acquisition unit 103, a coefficient calculation unit 104, and a coefficient output unit 105, which will be described later.
FIG. 4 illustrates a state in which the processor 901 is executing a program that realizes the functions of the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105, i.e., a state in which the processor 901 operates as the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105.
Moreover, the imaging area database 106 and the device characteristics database 107 shown in FIG. 4 are realized in the main storage device 902 and/or the auxiliary storage device 903 .

5, an imaging area designation unit 101 designates two imaging areas. The imaging areas are areas to be imaged by the imaging device 200. The two imaging areas are treated as a set. The imaging area designation unit 101 designates a plurality of area sets.
Specifically, the imaging area designation unit 101 designates, as the two imaging areas, two areas that are estimated to have a uniform brightness temperature in outer space, have known infrared radiance values, and have different rotation angles for each imaging of each scanning mirror.
For example, the imaging area designation unit 101 designates two deep space areas as imaging areas.
Furthermore, the imaging area designation unit 101 may designate two areas located symmetrically with respect to an arbitrary area in outer space as the imaging area.
Furthermore, the imaging region designation unit 101 may designate two deep space regions that are symmetrically positioned with respect to an arbitrary region in outer space as the imaging region.
The deep space region is a region of space that is more than 2 million kilometers away from the Earth.
The imaging area designation unit 101 transmits imaging area data 150 indicating the two imaging areas to the imaging control device 400 via the communication device 904 .
Furthermore, the imaging region designation unit 101 stores the imaging region data 150 in the coefficient output unit 105 .
The process performed by the imaging area designation unit 101 corresponds to imaging area designation processing.

The image acquisition unit 102 acquires the captured image 250 from the imaging device 200 via the communication device 904. The captured image 250 is an image obtained by capturing an image of an area designated as an imaging area by the imaging area designation unit 101. The image acquisition unit 102 acquires the captured images 250 of a plurality of area sets.
The image acquisition unit 102 transfers the acquired captured image 250 to the coefficient calculation unit 104 .
The process performed by the image acquisition unit 102 corresponds to an image acquisition process.

The monitor data acquisition unit 103 acquires the monitor data 550 from the monitor device 500 via the communication device 904. The monitor data acquisition unit 103 acquires the monitor data 550 of a plurality of region sets. The monitor data acquisition unit 103 acquires the monitor data 550 in the imaging of a plurality of region sets.
The monitor data acquisition unit 103 transfers the acquired monitor data 550 to the coefficient calculation unit 104. The monitor data 550 includes angle data and temperature data of each scanning mirror at each time when the imaging area designation unit 101 captures the two imaging areas.
2, assume that imaging device 200 has two scanning mirrors (scanning mirror α and scanning mirror β). In this case, the rotation angle of scanning mirror α when capturing image _N1 is set to θ ₁ , the rotation angle of scanning mirror β when capturing image N1 is set to φ ₁ , the temperature of scanning mirror _α when capturing image _N2 is _set to T ₁ , and the temperature of scanning mirror β _when capturing image _{N2 is set to τ 2} _.
At this time, the monitor data acquisition unit 103 acquires ( _θ1 , _φ1 , T1, _τ1 , θ2, _φ2 , _T2 _, _τ2 ) as angle data and temperature data corresponding to a set of captured images ( _N1 , _N2 ). Note that the set of captured images ( _N1 , _N2 ) may be captured at the _same time or at different times.
The process performed by the monitor data acquisition unit 103 corresponds to a monitor data acquisition process.

The coefficient calculation unit 104 acquires two paired captured images 250 from the image acquisition unit 102. The coefficient calculation unit 104 also acquires corresponding monitor data 550 from the monitor data acquisition unit 103.
Furthermore, the coefficient calculation unit 104 acquires device characteristic data 160 from a device characteristic database 107. The device characteristic data 160 is data indicating the characteristics of the image capture device 200. Specifically, the device characteristic data 160 indicates an infrared wavelength and an infrared calibration coefficient for each channel of the line sensor of the image capture device 200.
The coefficient calculation unit 104 calculates coefficients 170 of an infrared emissivity function for each scanning mirror using the captured images 250, the monitor data 550, and the device characteristic data 160. Specifically, it calculates coefficients 170 of an infrared emissivity function that reduces the difference in infrared radiance between the two captured images 250. All of the coefficients 170 of the infrared emissivity function for each scanning mirror of all scanning mirrors provided on the same optical path of the imaging device 200 are calculated by the same physical model using the two captured images 250, the monitor data 550 including the rotation angle data and temperature data of the scanning mirror for each image capture, and the device characteristic data 160.
Then, the coefficient calculation unit 104 outputs the calculated coefficient 170 of the infrared emissivity function of each scanning mirror to the coefficient output unit 105 .
The coefficients of the infrared emissivity function and the physical model will be described in detail later.
The process performed by the coefficient calculation unit 104 corresponds to a coefficient calculation process.

The coefficient output unit 105 outputs the coefficient 170 of the infrared emissivity function of each scanning mirror using the input/output device 905. For example, the coefficient output unit 105 displays the coefficient 170 of the infrared emissivity function of each scanning mirror on a display included in the input/output device 905.

The imaging area database 106 stores imaging area data 150 .
The device characteristic database 107 also stores device characteristic data 160 .

*** Operation Description ***
Next, an example of the operation of the emissivity calibration apparatus 100 according to the present embodiment will be described with reference to FIG.

In STEP 1, the imaging region designation unit 101 designates two imaging regions.
As described above, the imaging area designation unit 101 designates, as imaging areas, two areas that are estimated to have a uniform brightness temperature in outer space, have known infrared radiance values, and have at least one scanning mirror with a different rotation angle each time imaging is performed.
The imaging region designation unit 101 designates a plurality of region sets.

FIG. 7 shows an example of a method for specifying an imaging area by the imaging area specifying unit 101. In FIG.
In FIG. 7, each rectangle around the Earth represents a deep space region.
The imaging region designation unit 101 may designate, for example, two deep space regions located symmetrically in the X-axis direction with respect to the center line of the Earth in the Y-axis direction as the imaging region.
Also, as shown in FIG. 8, the imaging region designation unit 101 may designate, for example, two deep space regions located symmetrically in the Y-axis direction with respect to the center line of the Earth in the X-axis direction as the imaging region.
When two deep space regions at symmetrical positions are imaged as in Figures 7 and 8, it is considered that parameters other than the rotation angle of the scanning mirror and temperature at the time of image capture are common between the captured images. Therefore, when two deep space regions at symmetrical positions are imaged as in Figures 7 and 8, it is considered that the accuracy of calculation of the coefficients of the infrared emissivity function by the monitor data acquisition unit 103 is improved.

However, the method of specifying the imaging area by the imaging area designation unit 101 is not limited to the method of Fig. 7 and Fig. 8. For example, as shown in Fig. 9, the imaging area designation unit 101 may designate two deep space areas that are not symmetrical in either the X-axis direction or the Y-axis direction as the imaging area. Also, any deep space area around the earth and any deep space area not around the earth may be designated as the two imaging areas. Also, two deep space areas not around the earth may be designated as the imaging areas.
In addition, the imaging area designation unit 101 may designate an area other than the deep space area as the imaging area as long as it satisfies the following conditions: "It is estimated to have a uniform brightness temperature, the infrared radiance value is known, and the rotation angle of each scanning mirror is different each time imaging is performed."
The imaging area designation unit 101 can designate the imaging area according to an instruction from a user (hereinafter also referred to as a designer) of the emissivity calibration device 100. The imaging area designation unit 101 may also designate the imaging area by analyzing captured images previously captured by the imaging device 200 and searching for an area "estimated to have a uniform radiance temperature, with a known infrared radiance value, and with a different rotation angle for each imaging of each scanning mirror."

In STEP 2 , the image acquisition unit 102 acquires the captured image 250 from the imaging device 200 .
In this embodiment, the captured image 250 acquired by the image acquisition unit 102 may be a luminance image, or may be a set of data required to generate a luminance image.

In STEP 3, the monitor data acquisition unit 103 acquires the monitor data 550 from the monitor device 500. That is, the monitor data acquisition unit 103 acquires the angle data and temperature data of each scanning mirror when the captured image 250 acquired in STEP 2 was captured.
The monitor data acquisition unit 103 acquires monitor data 550 in capturing images of a plurality of area sets.
The image acquisition unit 102 acquires captured images 250 of a plurality of area sets.

In STEP 4 , the coefficient calculation unit 104 calculates the coefficient 170 of the infrared emissivity function using the captured image 250 , the monitor data 550 and the apparatus characteristic data 160 .
In this embodiment, the captured image 250, monitor data 550, and device characteristic data 160 are input to a physical model M that represents the relationship between the optical path and infrared radiance of the imaging device 200, and coefficients 170 of the infrared emissivity function of each scanning mirror are calculated for all scanning mirrors from the same physical model M. The physical model M is an equation that mathematically represents the physical relationship related to the infrared radiance in the imaging device 200. Details of the physical model M will be described later.

Finally, in STEP 5, the coefficient output unit 105 outputs the coefficient 170 of the infrared emissivity function.

The user (designer) of the emissivity calibration device 100 checks the coefficients 170 of the infrared emissivity function output from the coefficient output unit 105, and judges whether or not, for example, an effect of reducing luminance errors related to the angle dependency of the scanning mirror has been obtained. Specifically, the designer calculates the infrared radiance of the captured image 250 using the coefficients 170 of the infrared emissivity functions of all the scanning mirrors. Then, the designer compares the infrared radiance between the captured images 250 of two regions for each region set, and judges whether or not the luminance errors due to differences in the rotation angles of the scanning mirror have been reduced.
If it is determined that the luminance error reduction effect has been achieved, the designer can calibrate the infrared radiance using the output coefficient 170 of the infrared emissivity function.
On the other hand, if it is determined that the luminance error reduction effect has not been obtained, the designer instructs the emissivity calibration device 100 to, for example, recalculate the infrared emissivity function coefficient 170. When the designer instructs the emissivity calibration device 100 to recalculate the infrared emissivity function coefficient 170, the emissivity calibration device 100 repeats the processes from STEP 4 onwards.
In this case, in STEP 4, the coefficient calculation unit 104 recalculates the coefficient 170 of the infrared emissivity function using the new captured image 250 that has not been used in the calculation and the corresponding unused monitor data 550.

In addition, the imaging area designation unit 101 may specify only one set of imaging areas, the image acquisition unit 102 may acquire only the imaging image 250 of the set of imaging areas, and the monitor data acquisition unit 103 may acquire only the monitor data 550 of the set of imaging areas.
In this case, if the designer issues an instruction to recalculate the coefficient 170 of the infrared emissivity function, the processes from STEP 1 onwards are repeated.

Here, the calculation of the coefficient 170 of the infrared emissivity function performed by the coefficient calculation unit 104 in STEP 4 will be described in detail.
The coefficient calculation unit 104 inputs the captured image 250, the monitor data 550, and the device characteristic data 160 to a physical model M that represents the relationship between the optical path and infrared radiance of the imaging device 200, and calculates the coefficients of the infrared emissivity function of all scanning mirrors on the same optical path. Here, the coefficients refer to the internal parameters of the infrared emissivity function. The technique disclosed in Reference 1 or Reference 2 is used as a method for determining these internal parameters.

[Reference 1]: Adapting Arbitrary Mutation Distributions in Evolution Strategies: The Covariance Matrix Adaptation (Proceedings of IEEE International Conference on Evolutionary Computation, 1996, pp. 312-317)
[Reference 2]: Evolution strategies; Handbook of Computational Intelligence, J. Kacprzyk and W. Pedrycz, editors, pp. 871-898, Springer (2015)

More specifically, the coefficient calculation unit 104 calculates the coefficient 170 of the infrared emissivity function in the following manner.
In the following, the description will proceed on the assumption that the imaging device 200 includes a scanning mirror β, which is a primary scanning mirror, and a scanning mirror α, which is a secondary scanning mirror, as shown in FIG.

In this embodiment, the infrared emissivity function of the scanning mirror α is defined as ε _α (θ _α )=a ₀ +a ₁ θ _α +a ₂ θ _α ² .
Here, _θα represents the angle of incidence of light incident on the scanning mirror α. _θα depends on the rotation angle x of the scanning mirror α, and is therefore a function of x ( _θα (x)). The internal parameters of _θα (x) depend on the hardware.
Similarly, define the infrared emissivity function of scanning mirror β as ε _β (θ _β )=b ₀ +b ₁ θ _β +b ₂ θ _β ² .
Here, θ _β represents the angle of incidence of light incident on the scanning mirror β. θ _β depends on the rotation angle y of the scanning mirror β, and is therefore a function of y (θ _β (y)). The internal parameters of θ _β (y) depend on the hardware.

Also, let C be the digital output of the line sensor (detection element).
The observed luminance of the line sensor (detection element) is expressed as R _d =qC ² +mC+b.
where q, m, and b are the infrared calibration coefficients of the line sensor. q, m, and b are included in the device characteristic data 160.

The temperature of scanning mirror α is represented as T _α , and the temperature of scanning mirror β is represented as T _β .
The infrared radiance of the object to be observed is represented as R.
The blackbody radiance of scanning mirror α is represented as R _α (T _α ), _which is a function of _{T α} _and is calculated using the infrared wavelength values included in device characteristic data 160 .
The blackbody radiance of scanning mirror β is denoted as R _β (T _β ), where R _β (T _β ) is a function of T _β and the value is calculated using the infrared wavelength values included in device characteristic data 160 .

From the above, in the imaging device 200 equipped with two scanning mirrors (scanning mirror α, scanning mirror β) and a line sensor on the same optical path, the infrared radiance (hereinafter also referred to as observed radiance) is expressed by the physical model M of the following equations (1), (2), and (3).
R _d =M Formula (1)
_Rd = ^qC2 + mC + b Equation (2)
M = (1 - _εα (θα ₎ ) (1 - εβ( _θβ )) R + _εα (θ) _Rα ( _Tα ) + _εβ ( _θβ ) (1 - _εα ( _θ )) _Rβ ( _Tβ ) Equation (3)
As previously mentioned, scanning mirror α is the secondary scanning mirror, and scanning mirror β is the primary scanning mirror.
The optical path inside the imaging device 200 is configured as follows.
Subject (light source) → Primary scanning mirror β → Secondary scanning mirror α → Line sensor

Next, a pair of imaging regions imaged by the imaging device 200 will be considered.
The rotation angle of the scanning mirror α when capturing an image i (infrared radiance: R _i ) is denoted as x _i , the temperature of the scanning mirror α is denoted as T _α,i , and the rotation angle of the scanning mirror β when capturing the image i is denoted as y _i , the temperature of the scanning mirror β is denoted as T _β,i .
Furthermore, _θα,i indicates the angle of incidence of light incident on the scanning mirror α when the captured image i is captured. _θα,i depends on the rotation angle x _i of the scanning mirror α, and is therefore a function of x _i ( _θα,i (x _i )). Furthermore, θβ _,i indicates the angle of incidence of light incident on the scanning mirror β when the captured image i is captured. _θβ,i depends on the rotation angle y _i of the scanning mirror β, and is therefore a function of y _i (θβ _,i (y _i )).
Similarly, the rotation angle of scanning mirror α when capturing image j (infrared radiance: R _j ) is denoted as x _j , the temperature of scanning mirror α is denoted as T _α,j , and the rotation angle of scanning mirror β when capturing image j is denoted as y _j , the temperature of scanning mirror β is denoted as T _β,j .
Furthermore, _θα,j indicates the angle of incidence of light incident on scanning mirror α when captured image j is captured. _θα,j depends on the rotation angle _xj of scanning mirror α, and is therefore a function of _xj ( _θα,j ( _xj )). Furthermore, θβ _,j indicates the angle of incidence of light incident on scanning mirror β when captured image j is captured. _θβ,j depends on the rotation angle yj of scanning mirror _β , and is therefore a function of _yj (θβ _,j ( _yj )).

In this case, the relational expressions of the infrared radiance for the pair (i, j) of imaging regions can be expressed by the following expressions (4) to (9).
R _d,i = M _i Equation (4)
R _d,i = qC _i ² + mC _i + b Equation (5)
M _i = (1 - ε _α (θ _α,i ) (1 - ε _β (θ _β,i )) R _i + ε _α (θ _α,i ) R _α (T _α,i ) + ε _β (θ _β,i ) (1 - ε _α (θ _α,i ) R _β (T _β,i ) Equation (6)
R _d,j = M _j Equation (7)
_Rd,j = _qCj2 + _mCj + ^b Equation (8)
(M _j = (1 - ε _α (θ _α,j )) (1 - ε _β (θ _β,j ) R ₂ + ε _α (θ _α,j ) R _α (T _α,j ) + ε _β (θ _β,j ) (1 - ε _α (θ _α,j )) R _β (T _β,j ) Equation (9)
Here, the subscripts i and j at the bottom right of θ, T, R, and M are numbers corresponding to the imaging region.

Next, the difference ΔR _k in observed radiance between imaging region i and imaging region j and the difference ΔM _k in input radiance to the detector are expressed by equations (10) and (11), respectively. The difference between equations (10) and (11) is expressed by equation (12).
ΔR _k =R _d,i −R _d,j Equation (10)
ΔM _k =M _i -M _j Equation (11)
E _k =ΔR _k -ΔM _k Equation (12)
Here, the subscript k at the bottom right of ΔR _k , ΔM _k , and E _k is the region set number, and the region set number k is uniquely determined for each region set (i, j).
The value of _Ek for each region set k indicates the difference between the theoretical value of the observed radiance calculated by the physical model M and the value calculated from the rotation angle and temperature observed by the monitor device 500. The smaller the magnitude of this difference _Ek , the smaller the difference in infrared radiance between the imaging region i and the imaging region j of the region set k.
The coefficient calculation unit 104 calculates coefficients _a0 , _a1 , _a2 , _b0 , b1, and _b2 of the infrared emissivity functions _εα ( _θα ) and _εβ ( _θβ ) that minimize the magnitude of this difference _Ek . In other words, the coefficient calculation unit 104 calculates coefficients ₁₇₀ of the infrared emissivity function that minimizes the magnitude of the difference in infrared radiance between the image capture regions i and j.

The coefficient calculation unit 104 can use an optimization algorithm to calculate the coefficient 170 of the infrared emissivity function. The coefficient calculation unit 104 calculates the coefficient 170 of the infrared emissivity function using an optimization algorithm as follows.
The coefficient calculation unit 104 uses the following formula (13) to calculate the coefficient 170 of the infrared emissivity function.

F = F ({E _k |1 ≦ k ≦ N}) Equation (13)

Here, N is the total number of region sets, F is a function arbitrarily designed by the designer, and k is the region set number. A natural number from 1 to N is assigned to k.
Consider finding the coefficients _a0 , _a1 , _a2 , _b0 , b1, and b2 of the infrared emissivity functions _εα ( _θα ), _εβ ( _θβ ) for minimizing _F defined in equation ( ₁₃ ).
Here, the objective function f is defined as F, as in equation (14).
f = F Equation (14)
The parameter x to be optimized is as shown in equation (15).

The coefficient calculation unit 104 obtains an optimal solution to equation (15) by applying equation (14) and equation (15) to "CMA-ES" in the above-mentioned reference 1 or "(μ/μ _w , λ)-CMA-ES" in reference 2. Furthermore, the coefficient calculation unit 104 may obtain an optimal solution to equation (15) by applying equation (14) and equation (15) to an algorithm including a part or all of "CMA-ES" or "(μ/μ _w , λ)-CMA-ES".
The optimized value of parameter x in equation (15) is coefficient 170 of the infrared emissivity function.

The "f" in "an evolutionary computation method that performs a multi-point search using normal distribution for the optimization of a function f: X→R defined on a subset ^X⊆Rd of a real vector space (continuous optimization)" in Chapter 1 of Reference 3 below corresponds to the above formula (14). This "f" is also referred to in "STEP.2 Target value f(x _i ) of a solution candidate" in Section 2.1 of Reference 3.
Furthermore, the above equation (15) may be optimized by the algorithm described in Section 2.1 of Reference 3.

[Reference 3]: Continuous Optimization by Evolution Strategies - CMA-ES Design Principles and Theoretical Foundations, Yohei Akimoto, System/Control/Information, Vol. 60, No. 7, pp. 292-297, 2016

In addition, if one of the two scanning mirrors is a reflecting mirror that does not have a rotation axis, the coefficient calculation unit 104 treats the value of the rotation angle (x or y) of the scanning mirror corresponding to that reflecting mirror on the physical model M as a constant and performs calculations.
Furthermore, if one of the two scanning mirrors does not exist, the coefficient calculation unit 104 sets the values of the emissivity function and blackbody radiance of the non-existent scanning mirror to 0 and performs calculations.

***Description of Effects of the Embodiment***
In this embodiment, the emissivity calibration device 100 calculates the coefficients of the infrared emissivity function that minimizes the magnitude of the difference in infrared radiance between the imaging region i and the imaging region j of the region set k. The designer can calibrate the infrared emissivity using the calculated coefficients of the infrared emissivity function. Therefore, according to this embodiment, it is possible to reduce errors in the infrared radiance even in images captured by an imaging device after launch.

Furthermore, in this embodiment, in addition to the error in infrared radiance due to the change in infrared emissivity of the scanning mirror caused by the change in the environment before and after launch, it is also possible to reduce the error in infrared radiance due to the change in infrared emissivity of the scanning mirror caused by aging. In other words, due to aging, the infrared emissivity of the satellite at the time of launch may not match the infrared emissivity a long time after launch. According to this embodiment, it is also possible to reduce the error in infrared radiance caused by such a mismatch in infrared emissivity due to aging.

Furthermore, in the imaging device 200, the scanning mirror and the line sensor are on the same optical path. In order to treat all the scanning mirrors and line sensors on the same optical path as one system and calculate the coefficients of the infrared emissivity function for each scanning mirror for one system using a physical model, the coefficients of the infrared emissivity function for all the scanning mirrors need to be calculated from the same physical model.
In this embodiment, the emissivity calibration device 100 calculates the infrared emissivity coefficients of all scanning mirrors by applying the same physical model M. Therefore, according to this embodiment, the infrared emissivities of all scanning mirrors can be calibrated for one system.

Additional hardware configuration information
Finally, a supplementary explanation of the hardware configuration of the emissivity calibration device 100 will be given.
The processor 901 shown in FIG. 4 is an integrated circuit (IC) that performs processing.
The processor 901 is a central processing unit (CPU), a digital signal processor (DSP), or the like.
The main storage device 902 shown in FIG. 4 is a RAM (Random Access Memory).
The auxiliary storage device 903 shown in FIG. 4 is a read only memory (ROM), a flash memory, a hard disk drive (HDD), or the like.
The communication device 904 shown in FIG. 4 is an electronic circuit that performs data communication processing.
The communication device 904 is, for example, a communication chip or a NIC (Network Interface Card).
The input/output device 905 shown in FIG. 4 is, for example, a mouse, a keyboard, and a display.

The auxiliary storage device 903 also stores an OS (Operating System).
At least a part of the OS is executed by the processor 901 .
The processor 901 executes at least a part of the OS, and executes programs that realize the functions of the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105.
The processor 901 executes the OS, thereby performing task management, memory management, file management, communication control, and the like.
In addition, at least one of information, data, signal values, and variable values indicating the results of processing by the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105 is stored in at least one of the main memory device 902, the auxiliary memory device 903, the register within the processor 901, and the cache memory.
Furthermore, the programs for realizing the functions of the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105 may be stored in a portable recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, a DVD, etc. Then, the portable recording medium storing the programs for realizing the functions of the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105 may be distributed.

In addition, the "part" of at least any of the imaging area designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105 may be read as a "circuit" or a "process" or a "procedure" or a "processing" or a "circuitry".
Furthermore, the emissivity calibration apparatus 100 may be realized by a processing circuit. The processing circuit is, for example, a logic IC (Integrated Circuit), a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).
In this case, the imaging region designation unit 101, the image acquisition unit 102, the monitor data acquisition unit 103, the coefficient calculation unit 104, and the coefficient output unit 105 are each realized as part of a processing circuit.
In this specification, the higher-level concept of a processor and a processing circuit is called "processing circuitry."
That is, a processor and a processing circuit are each specific examples of "processing circuitry."

The above-described embodiments are essentially preferred examples, and are not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, or the scope of use of the present disclosure. The above-described embodiments can be modified in various ways as necessary.

100: Emissivity calibration device, 101: Imaging area designation unit, 102: Image acquisition unit, 103: Monitor data acquisition unit, 104: Coefficient calculation unit, 105: Coefficient output unit, 106: Imaging area database, 107: Device characteristics database, 150: Imaging area data, 160: Device characteristics data, 170: Coefficient of infrared emissivity function, 200: Imaging device, 250: Captured image, 300: Satellite, 400: Imaging control device, 500: Monitor device, 550: Monitor data, 901: Processor, 902: Main memory device, 903: Auxiliary memory device, 904: Communication device, 905: Input/output device.

Claims

an imaging area designation unit that designates two imaging areas to be imaged by an imaging device in outer space in which one or more scanning mirrors and a line sensor are provided on the same optical path, the imaging areas being estimated to have a uniform radiance temperature in the outer space, having known infrared radiance values, and having different rotation angles for each imaging of at least one of the one or more scanning mirrors;
an image acquisition unit that acquires two captured images obtained by the imaging device capturing images of the two imaging regions;
a monitor data acquisition unit that acquires monitor data including angle data indicating a rotation angle of each scanning mirror at each time of imaging the two imaging areas;
a coefficient calculation unit that calculates a coefficient of an infrared emissivity function for each of the one or more scanning mirrors using the two captured images and the monitor data.
The coefficient calculation unit
2. The data processing device according to claim 1, wherein coefficients of the infrared emissivity function are calculated using a rotation angle of each scanning mirror at each time of photographing the two imaging regions and an infrared radiance between the two captured images.
The coefficient calculation unit
The data processing apparatus according to claim 1 , further comprising: a coefficient calculating section for calculating a coefficient that minimizes a difference in infrared radiance between the two captured images.
The imaging area designation unit is
The data processing apparatus according to claim 1 , wherein two deep space regions are designated as the two imaging regions.
The imaging area designation unit is
2. The data processing apparatus according to claim 1, wherein two areas located symmetrically with respect to an arbitrary area in the outer space are designated as the two imaging areas.
The imaging device includes two or more scanning mirrors and a line sensor on the same optical path,
The coefficient calculation unit
2. The data processing apparatus of claim 1, further comprising: applying a same physical model to the two or more scanning mirrors to calculate coefficients of the infrared emissivity function for each of the two or more scanning mirrors.
The coefficient calculation unit
2. The data processing apparatus of claim 1, wherein the coefficients of the infrared emissivity function are calculated using an optimization algorithm.
The monitor data acquisition unit
2. The data processing apparatus according to claim 1, wherein the monitor data includes temperature data indicating temperatures at each time of photographing in the imaging of the two imaging regions of each scanning mirror.
The coefficient calculation unit
2. The data processing device according to claim 1, further comprising: a device characteristic data indicating an infrared wavelength and an infrared calibration coefficient for each channel of the line sensor, the ... calibration coefficient for each channel of the line sensor, the device characteristic data indicating an infrared radiation coefficient for each channel of the line sensor, the device characteristic data indicating an infrared radiation coefficient for each channel of the
The image acquisition unit includes:
2. The data processing apparatus according to claim 1, wherein the two captured images are obtained from an imaging device provided with one or more scanning mirrors that are driven in one axis.
a computer specifies, as two imaging regions to be imaged by an imaging device in outer space in which one or more scanning mirrors and a line sensor are provided on the same optical path, two regions which are estimated to have a uniform brightness temperature in the outer space, have known infrared radiance values, and have different rotation angles for each imaging of at least one of the one or more scanning mirrors;
the computer acquires two captured images obtained by the imaging device capturing images of the two imaging regions;
the computer acquires monitor data including angle data indicating a rotation angle of each scanning mirror at each image capture of the two imaging areas;
A data processing method in which the computer calculates coefficients of an infrared emissivity function for each of the one or more scanning mirrors using the two captured images and the monitor data.
an imaging area designation process for designating two imaging areas to be imaged by an imaging device in outer space in which one or more scanning mirrors and a line sensor are provided on the same optical path, the imaging areas being estimated to have a uniform radiance temperature in the outer space, having known infrared radiance values, and having different rotation angles for each imaging of at least one of the one or more scanning mirrors;
an image acquisition process in which the imaging device acquires two captured images obtained by capturing images of the two imaging regions;
a monitor data acquisition process for acquiring monitor data including angle data indicating a rotation angle of each scanning mirror at each time of imaging the two imaging areas;
a coefficient calculation process for calculating a coefficient of an infrared emissivity function for each of the one or more scanning mirrors using the two captured images and the monitor data.