JP2022104279A - Sensing system, information processing device, sensing method and program - Google Patents

Abstract

To provide a sensing system, information processing device, sensing method and program which can properly perform observation even when mirror reflection occurs.SOLUTION: A sensing system comprises: a first sensor which generates a first signal according to the electromagnetic wave in a first wavelength region incident from a first direction; a second sensor which generates a second signal according to the electromagnetic wave in a second wavelength region incident from the first direction; a third sensor which generates a third signal according to the electromagnetic wave in the first wavelength region incident from the second direction; determination means which determines whether or not the electromagnetic wave corresponding to the first signal is caused by mirror reflection on the basis of the second signal; and generation means which sets a pixel value based on the first signal when the electromagnetic wave corresponding to the first signal is not caused by the mirror reflection, sets a pixel value on the basis of the third signal generated according to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal when the electromagnetic wave corresponding to the first signal is caused by the mirror reflection, and generates an output image.SELECTED DRAWING: Figure 2

Description

The present invention relates to a sensing system, an information processing apparatus, a sensing method and a program.

Conventionally, a sensing system for observing the vicinity of the ground surface from an aircraft or the like using electromagnetic waves such as visible light and near infrared rays has been known. The sensing system using visible light observes the vicinity of the ground surface by acquiring the intensity of light such as sunlight reflected on the ground surface by an optical sensor mounted on the flying object. For example, Patent Document 1 discloses a sensing device that detects an electromagnetic wave from a relatively moving observation object and observes the observation object.

International Publication No. 2015/068395

Normally, sunlight is diffused in various directions on the surface of the earth, so the sensing system detects a part of the diffused light by an optical sensor. On the other hand, when observing the water surface of rivers, lakes, the sea, etc., sunlight may be specularly reflected by the water surface. Since the specularly reflected light on the water surface has an intensity far exceeding the intensity of the diffused light on the riverbed or the sea floor, the sensing system may not be able to perform correct observation when the specularly reflected light is generated. Further, even when observing the ground, the same problem may occur when the road is frozen or when an artificial object such as a solar panel is installed.

The present invention has been made to solve the above-mentioned problems, and provides a sensing system, an information processing apparatus, a sensing method and a program capable of appropriately observing even when specular reflection occurs. With the goal.

The sensing system according to the present invention is a sensing system installed in a flying object, and detects an electromagnetic wave in a first wavelength region incident from a first direction forming a first pitch angle downward with respect to the traveling direction of the flying object. A first sensor that is capable of sequentially generating a first signal according to the detected electromagnetic wave and an electromagnetic wave in a second wavelength range different from the first wavelength range incident from the first direction are detectably provided and detected. A second sensor that sequentially generates a second signal according to the electromagnetic wave, and an electromagnetic wave in the first wavelength region incident from the second direction forming a second pitch angle different from the first pitch angle downward with respect to the traveling direction of the projectile. The electromagnetic wave corresponding to the first signal based on the third sensor which is provided so as to be able to detect and sequentially generates the third signal corresponding to the detected electromagnetic wave and the second signal corresponding to the first signal sequentially generated. Is a determination means for determining whether or not is due to mirror reflection, and when it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection, the first signal is used. When the pixel value of the pixel corresponding to one signal is set and it is determined that the electromagnetic wave corresponding to the first signal sequentially generated is due to mirror reflection, the ground position is the same as the electromagnetic wave corresponding to the first signal. It has a generation means for generating and outputting an output image by setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to an electromagnetic wave from.

Further, in the sensing system according to the present invention, among the first signal and the third signal sequentially generated based on the difference in the angle between the first direction and the second direction, they are generated according to the electromagnetic wave from the same ground position. It further has an associating means for associating the first signal and the third signal, and the generating means converts the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal based on the associating result by the associating means. It is preferable to specify the third signal generated accordingly.

Further, in the sensing system according to the present invention, there is further an acquisition means for acquiring information on the position and attitude of the flying object, and the associating means further based on the position and attitude of the flying object, the first signal and the third. It is preferable to associate signals.

Further, in the sensing system according to the present invention, it is preferable that the first wavelength region is the visible wavelength region and the second wavelength region is the near infrared wavelength region.

Further, in the sensing system according to the present invention, there is further a calculation means for calculating the reflectance of the electromagnetic wave corresponding to the first signal in the observation target based on the second signal corresponding to the first signal sequentially generated. However, it is preferable that the determination means determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the reflectance.

Further, in the sensing system according to the present invention, the second sensor is further provided so as to be able to detect the electromagnetic wave incident from above the flying object, and the calculation means is the second signal corresponding to the electromagnetic wave incident from above the flying object. Based on this, it is preferable to correct the reflectance at the observation target.

The information processing apparatus according to the present invention is installed in a flying object and is provided so as to be capable of detecting an electromagnetic wave in a first wavelength region incident from a first direction forming a predetermined pitch angle downward with respect to the traveling direction of the flying object. A first sensor that sequentially generates a first signal according to the detected electromagnetic wave and an electromagnetic wave in a second wavelength range different from the first wavelength range incident from the first direction are provided so as to be able to detect and respond to the detected electromagnetic wave. A second sensor that sequentially generates a second signal and an electromagnetic wave in the first wavelength range incident from the second direction having a pitch angle different from a predetermined pitch angle downward with respect to the traveling direction of the flying object can be detected. A third sensor that sequentially generates a third signal according to the detected electromagnetic wave, an acquisition means that acquires a first signal, a second signal, and a third signal from a sensing device, and a first that is sequentially generated. Based on the second signal corresponding to the signal, the determination means for determining whether or not the electromagnetic wave corresponding to the first signal is due to mirror reflection, and the electromagnetic wave corresponding to the first signal sequentially generated is mirror reflection. If it is determined that the signal is not due to, the pixel value of the pixel corresponding to the first signal is set based on the first signal, and the electromagnetic wave corresponding to the sequentially generated first signal is due to mirror reflection. If it is determined that, the pixel value of the pixel corresponding to the first signal is set based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. It has a generation means for generating and outputting an output image.

In the sensing method according to the present invention, the computer can detect electromagnetic waves in the first wavelength region that are installed in the flying object and are incident from the first direction that forms a predetermined pitch angle downward with respect to the traveling direction of the flying object. The first sensor that sequentially generates the first signal according to the detected electromagnetic wave and the second sensor that can detect the electromagnetic wave in the second wavelength range different from the first wavelength range incident from the first direction are provided and the detected electromagnetic wave. It is possible to detect electromagnetic waves in the first wavelength range incident from the second sensor that sequentially generates the second signal according to the above and the second direction that forms a pitch angle different from the predetermined pitch angle downward with respect to the traveling direction of the flying object. The second sensor, which is provided in the above and sequentially generates a third signal according to the detected electromagnetic wave, acquires the first signal, the second signal, and the third signal, and corresponds to the first signal sequentially generated. Based on the signal, it is determined whether or not the electromagnetic wave corresponding to the first signal is due to mirror reflection, and when it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection. When the pixel value of the pixel corresponding to the first signal is set based on the first signal and it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is due to mirror reflection, the first signal is used. An output image is generated and output by setting the pixel value of the pixel corresponding to the first signal based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the signal. Including doing.

The program according to the present invention is a program for controlling a computer, and is installed in a flying object, and is incident from a first direction having a predetermined pitch angle downward with respect to the traveling direction of the flying object. A first sensor that can detect electromagnetic waves and sequentially generate a first signal according to the detected electromagnetic waves, and a second sensor that can detect electromagnetic waves in a second wavelength range different from the first wavelength range incident from the first direction. A second sensor that sequentially generates a second signal according to the detected electromagnetic wave, and a first wavelength region that is incident from the second direction that forms a pitch angle different from a predetermined pitch angle downward with respect to the traveling direction of the flying object. The first signal is sequentially generated by acquiring the first signal, the second signal, and the third signal from the third sensor which is provided so as to be able to detect the electromagnetic wave of the above and sequentially generates the third signal according to the detected electromagnetic wave. Based on the second signal corresponding to, it is determined whether or not the electromagnetic wave corresponding to the first signal is due to mirror reflection, and the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection. When it was determined, the pixel value of the pixel corresponding to the first signal was set based on the first signal, and it was determined that the electromagnetic wave corresponding to the sequentially generated first signal was due to mirror reflection. In this case, the output image is obtained by setting the pixel value of the pixel corresponding to the first signal based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. Causes the computer to generate and output.

The sensing system, information processing apparatus, sensing method and program according to the present invention can appropriately observe even when specular reflection occurs in the observation target.

It is a figure which shows an example of the schematic structure of the sensing system 1. It is a schematic diagram for demonstrating the structure inside the sensing apparatus 2. It is a schematic diagram for demonstrating the light incident on the sensing apparatus 2. It is a block diagram which shows the schematic structure of the sensing device 2. It is a figure which shows an example of the data structure of the sensor data D. It is a block diagram which shows the schematic structure of the analysis apparatus 3. It is a flow chart which shows an example of the flow of analysis processing. (A) and (b) are schematic diagrams for explaining the correspondence.

Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments but extends to the inventions described in the claims and their equivalents.

FIG. 1 is a diagram showing an example of a schematic configuration of a sensing system 1 according to the present invention. The sensing system 1 has a sensing device 2 and an analysis device 3. The sensing device 2 is a platform composed of a plurality of sensors such as an optical sensor for observing an observation target and a housing containing each sensor. The sensing device 2 is mounted and installed on a flying object 4 such as a UAV (Unmanned Aerial Vehicle) or a helicopter. The sensing device 2 detects electromagnetic waves such as visible light or near infrared rays from the observation target below the flying object 4 while the flying object 4 is flying according to a linear flight path at a preset altitude. Generate sensor data. In this embodiment, the sensing device 2 detects electromagnetic waves in the visible wavelength region and the near infrared wavelength region. In the present embodiment, the visible wavelength region and the near infrared wavelength region refer to wavelength bands of 340 nm to 850 nm and 700 nm to 2500 nm, respectively. The visible wavelength region and the near infrared wavelength region are examples of the first wavelength region and the second wavelength region, respectively.

The analysis device 3 is an information processing device such as a server or a PC (Personal Computer). The analysis device 3 receives and displays the output image generated by the sensing device 2 so that the user can use it.

FIG. 2 is a schematic diagram for explaining the internal configuration of the sensing device 2. The sensing device 2 includes a first lens 201, a second lens 202, a diffuser 203, an optical fiber bundle 204, a third lens 205, a swing mirror 206, a first spectroscope 207, a second spectroscope 208, and a first sensor 209. It has a second sensor 210, a third sensor 211, and the like. In FIG. 2, arrows A1, A2, and A3 indicate the traveling (front-back) direction, the left-right direction, and the up-down direction of the projectile 4, respectively.

The first lens 201 and the second lens 202 are condensing lenses, and condense electromagnetic waves (light) incident from below in the vertical direction A3 toward the optical fiber bundle 204. The first lens 201 and the second lens 202 are telephoto lenses. The first lens 201 and the second lens 202 are provided so as to be able to collect electromagnetic waves from positions different from each other in the left-right direction A2. As the first lens 201 and the second lens 202, a cylindrical lens extending in the left-right direction A2 or a plurality of lenses arranged along the left-right direction A2 so that the optical axes are parallel is used. The first lens 201 is provided so as to be able to collect electromagnetic waves incident from the first direction forming the first pitch angle downward with respect to the traveling direction A1 of the flying object 4. The second lens 202 is provided so as to be able to collect electromagnetic waves incident from a second direction having a second pitch angle different from the first pitch angle downward with respect to the traveling direction A1 of the flying object 4.

For example, the first pitch angle is set to 90 ° and the second pitch angle is set to the range of 50 ° to 85 ° or 95 ° to 130 °. In this case, the first direction is a substantially vertical direction, and the second direction is a direction inclined forward or backward with respect to the vertical direction. If the difference between the first pitch angle and the second pitch angle is too small, specularly reflected light may be incident on both the first lens 201 and the second lens 202, and the loss of the observation result may not be corrected. Further, if the difference between the first pitch angle and the second pitch angle is too large, the appearance of the observation target differs between the first lens 201 and the second lens 202, and the electromagnetic waves incident from each lens may not correspond appropriately. There is. Therefore, it is preferable that the difference between the first pitch angle and the second pitch angle is set in the range of 5 ° to 40 °.

The diffuser 203 is a cosine collector and collects electromagnetic waves incident from a viewing angle of 180 ° centered on a direction 90 ° upward with respect to the traveling direction A1 of the flying object 4, particularly above the flying object 4. , Emit to the optical fiber bundle 204. As the diffuser 203, for example, a cosine collector such as CCSA1 (manufactured by Sorabo) is used.

The optical fiber bundle 204 is a multi-core fiber and includes a plurality of optical fibers 204a to t. One end of each optical fiber 204a to t is arranged so as to face the lens surface of the third lens 205. One end of the optical fibers 204a to j and one end of the optical fibers 204k to t are arranged side by side along the left-right direction A2, respectively. One end of each optical fiber 204k to t is arranged side by side under each optical fiber 204a to j in the vertical direction A3.

The other ends of the optical fibers 204a to d and f to i are arranged side by side along the left-right direction A2 so as to face the lens surface of the first lens 201. The optical fibers 204a to d and f to i guide electromagnetic waves incident from the first lens 201 and emit them toward the third lens 205. Similarly, the other ends of the optical fibers 204k to n and p to s are arranged side by side along the left-right direction A2 so as to face the lens surface of the second lens 202. The optical fibers 204k to n and ps to guide the electromagnetic wave incident from the second lens 202 and emit it toward the third lens 205. The number of optical fibers that emit electromagnetic waves incident from the first lens 201 toward the third lens 205, and the number of optical fibers that emit electromagnetic waves incident from the second lens 202 toward the third lens 205 are , 8 is not limited, and any number may be used. By increasing the number of optical fibers in the left-right direction A2, it is possible to increase the measurement resolution. Further, by arranging the optical fibers in an array and increasing the number of optical fibers, it is possible to increase the measurement range in one measurement in the traveling direction A1 and improve the measurement efficiency. The mirror may be made of a metal reflector or a foamed resin reflector (for example, an ultrafine foamed light reflector (MCPET) manufactured by Furukawa Electric Co., Ltd.). Instead of the swing mirror, a polygon mirror having a regular polygonal column shape (in the embodiment where the number of optical fibers is eight, a regular octagonal prism shape) can also be used.

The other ends of the optical fibers 204j and t are arranged so as to face the lens surface of the diffuser 203. The optical fibers 204j and t guide the electromagnetic wave incident from the diffuser 203 and emit it toward the third lens 205. The other end of the optical fiber 204e is arranged so as to face the first spectroscope 207. The optical fiber 204e guides the electromagnetic wave incident from the third lens 205 and emits it toward the first spectroscope 207. The other end of the optical fiber 204o is arranged so as to face the second spectroscope 208. The optical fiber 204o guides the electromagnetic wave incident from the third lens 205 and emits it toward the second spectroscope 208.

The third lens 205 is a condensing lens, and condenses electromagnetic waves incident from one end side of the optical fibers 204a to d, f to n, and pt toward the swing mirror 206. The third lens 205 is a plano-convex lens.

The swing mirror 206 is arranged so as to face the optical fiber bundle 204 via the third lens 205. The swing mirror 206 is provided by a motor described later so as to be able to swing (rotate) within a predetermined range with the vertical direction A3 as a swing axis (rotation axis). In the swing mirror 206, from the first position where the reflecting surface reflects the electromagnetic waves from one end of the optical fibers 204a and k arranged at one end of the optical fiber bundle 204 toward the one end of the optical fibers 204e and o. It is provided so as to be swingable up to a second position where an electromagnetic wave from one end of the optical fibers 204j and t arranged at the other end of the optical fiber bundle 204 is reflected toward one end of the optical fibers 204e and o. The swing mirror 206 alternately repeats swinging (rotation) from the first position to the second position and swinging (rotation) from the second position to the first position at regular intervals.

The first spectroscope 207 splits the incident electromagnetic wave into an electromagnetic wave in the visible wavelength region and an electromagnetic wave in the near-infrared wavelength region, and emits the incident electromagnetic wave toward the first sensor 209 and the second sensor 210, respectively. The second spectroscope 208 splits the incident electromagnetic wave into an electromagnetic wave in the visible wavelength region and an electromagnetic wave in the near infrared wavelength region, and emits the electromagnetic wave in the visible wavelength region toward the third sensor 211. That is, the electromagnetic waves in each wavelength range emitted from the first spectroscope 207 and the second spectroscope 208 include electromagnetic waves incident from below the flying object 4 and electromagnetic waves incident from above the flying object 4.

The first sensor 209 is provided so as to be able to detect an electromagnetic wave in the visible wavelength range emitted from the first spectroscope 207, and sequentially generates and outputs a first signal corresponding to the detected electromagnetic wave at regular intervals. The first sensor 209 generates the first signal so that the signal value increases as the intensity of the detected electromagnetic wave increases. The second sensor 210 is provided so as to be able to detect an electromagnetic wave in the near-infrared wavelength region emitted from the first spectroscope 207, and sequentially generates and outputs a second signal corresponding to the detected electromagnetic wave at regular intervals. .. The second sensor 210 generates a second signal so that the signal value increases as the intensity of the detected electromagnetic wave increases. The third sensor 211 is provided so as to be able to detect electromagnetic waves in the visible wavelength range emitted from the second spectroscope 208, and sequentially generates and outputs a third signal corresponding to the detected light at regular intervals. The third sensor 211 generates a third signal so that the signal value increases as the intensity of the detected electromagnetic wave increases.

As the first sensor 209 and the third sensor 211, for example, C12880MA (manufactured by Hamamatsu Photonics Co., Ltd.), which is a spectral luminance meter corresponding to the visible light region, is used. As the second sensor 210, for example, C14486GA (manufactured by Hamamatsu Photonics Co., Ltd.), which is a spectral luminance meter corresponding to the near infrared region, is used.

FIG. 3 is a schematic diagram for explaining the light incident on the sensing device 2. On the observation target F, the sunlight reflected in the region S1 extending in the left-right direction A2, which is located on the straight line extending in the first direction with respect to the sensing device 2, is incident on the first lens 201. The sunlight reflected in the corresponding regions S11 to S18 in the region S1 is condensed and incident on the optical fibers 204a to d and f to i facing the first lens 201, respectively. Similarly, on the observation target F, the sunlight reflected in the region S2 extended in the left-right direction A2, which is located on the straight line extending in the second direction with respect to the sensing device 2, is incident on the second lens 202. To. The sunlight reflected in the corresponding regions S21 to S28 in the region S2 is condensed and incident on the optical fibers 204k to n and p to s facing the second lens 202, respectively.

The size of each region S1 and S2 is determined based on the distance from the lens to the observation target F and the focal length of the lens. When the first direction is the vertical direction, the second direction is the direction inclined with respect to the vertical direction, and the observation target F is a plane, the distance from the second lens 202 to the region S2 is the first lens. It is larger than the distance from 201 to the area S1. As will be described later, the electromagnetic wave focused from the second lens 202 is used to replace the first signal generated in response to the electromagnetic wave focused from the first lens 201. Therefore, the size of the region S2 needs to be determined to be the same as the size of the region S1. In order to make the size of the region S2 the same as the size of the region S1, the focal length of the second lens 202 is set to be cos θ times the focal length of the first lens 201. Here, θ is the difference between the first pitch angle θ1 and the second pitch angle θ2, that is, the size of the angle formed by the first direction and the second direction.

On the other hand, as shown in FIG. 2, sunlight is incident on the diffuser 203, and sunlight collected by the diffuser 203 is incident on the optical fibers 204j and t.

The electromagnetic waves emitted from the optical fibers 204a to d, f to j, k to n, and pt are focused toward the swing mirror 206 by the third lens 205. As described above, one end of each optical fiber 204k to t is arranged below each optical fiber 204a to j in the vertical direction A3, and the swing mirror 206 swings with the vertical direction A3 as a swing axis. With the swing of the swing mirror 206, electromagnetic waves from any pair of the optical fibers 204a to d and f to j arranged in the vertical direction A3 and the optical fibers 204k to n and pt are selected. It is reflected toward the optical fiber 204e and the optical fiber 204o.

The electromagnetic waves emitted from the optical fiber 204e and the optical fiber 204o are incident on the first spectroscope 207 and the second spectroscope 208, respectively, and are separated into an electromagnetic wave in the visible wavelength region and an electromagnetic wave in the near infrared wavelength region. The electromagnetic waves in the visible wavelength region and the electromagnetic waves in the near-infrared wavelength region separated by the first spectroscope 207 are incident on the first sensor 209 and the second sensor 210, respectively, and are separated by the second spectroscope 208 in the visible wavelength region. The electromagnetic wave of is incident on the third sensor 211. As a result, the first sensor 209 and the second sensor 210 have the visible wavelength range and the near red from the region S11, the region S12, the region S13, the region S14, the region S15, the region S16, the region S17, the region S18, or the diffuser 203. Electromagnetic waves in the outer wavelength range are selectively and sequentially incident. Further, electromagnetic waves in the region S2 corresponding to the region corresponding to the electromagnetic wave incident on the first sensor 209 or the electromagnetic wave in the visible wavelength region from the diffuser 203 are selectively and sequentially incident on the third sensor 211. Electromagnetic waves from each region and the diffuser 203 are sequentially incident on the first sensor 209, the second sensor 210, and the third sensor 211 while the swing mirror 206 swings between the first position and the second position. Each time the swing mirror 206 swings between the first position and the second position, electromagnetic waves from each region and the diffuser 203 are repeatedly incident.

That is, the first sensor 209 is provided so as to be able to detect electromagnetic waves in the visible wavelength region incident from the first direction. The second sensor 210 is provided so as to be able to detect electromagnetic waves in the near-infrared wavelength region incident from the first direction. The third sensor 211 is provided so as to be able to detect electromagnetic waves in the visible wavelength range incident from the second direction.

FIG. 4 is a block diagram showing a schematic configuration of the sensing device 2. In addition to the above-described configuration, the sensing device 2 further includes a motor 221, a GPS (Global Positioning System) sensor 222, a 6-axis sensor 223, a first communication unit 224, a first storage unit 225, a first processing unit 230, and the like. ..

The motor 221 swings the swing mirror 206 according to the control signal from the first processing unit 230.

The GPS sensor 222 includes an antenna and a receiving circuit for receiving radio waves from GPS satellites. The GPS sensor 222 receives radio waves from GPS satellites received via the antenna in the receiving circuit to generate position information indicating the position of the sensing device 2 and time information indicating the absolute time, and the first processing unit 230. Supply to.

The 6-axis sensor 223 includes an acceleration sensor and a gyro sensor corresponding to the triaxial directions orthogonal to each other. The 6-axis sensor 223 acquires the acceleration and angular velocity of the sensing device 2 in the 3-axis direction at a predetermined cycle, generates posture information (posture change amount) and movement vector (relative position) regarding the posture of the sensing device 2. It is supplied to the first processing unit 230.

The first communication unit 224 includes a communication interface circuit such as a wired LAN (Local Area Network) or USB (Universal Serial Bus) in order to enable the sensing device 2 to communicate with the analysis device 3. The first communication unit 224 transmits the data supplied from the first processing unit 230 to the analysis device 3, and supplies the data received from the analysis device 3 to the first processing unit 230.

The first storage unit 225 includes a semiconductor memory such as, for example, an SDRAM (Synchronous Dynamic Random Access Memory). The first storage unit 225 stores computer programs, data, and the like used for processing by the first processing unit 230. The computer program may be installed in the storage device 140 from a computer-readable portable recording medium using a known setup program or the like. The portable recording medium is, for example, a CD-ROM (compact disc read only memory), a DVD-ROM (digital versatile disc read only memory), or the like. The first storage unit 225 stores sensor data and the like as data. The details of the sensor data will be described later.

The first processing unit 230 includes, for example, a CPU (Central Processing Unit). The first processing unit 230 may include an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and the like. The first processing unit 230 is connected to the first sensor 209, the second sensor 210, the third sensor 211, the motor 221, the GPS sensor 222, the 6-axis sensor 223, the first communication unit 224, the first storage unit 225, and the like. Control each of these parts. The first processing unit 230 reads the program stored in the first storage unit 225 and operates according to the read program, whereby the acquisition means 231, the associating means 232, the calculation means 233, the determination means 234, the output means 235, and the like. It functions as a generation means 236.

FIG. 5 is a diagram showing an example of the data structure of the sensor data D stored in the first storage unit 225. The sensor data D stores the time, position, posture, first signal value, second signal value, third signal value, and the like in association with each other. In the sensor data D, the first processing unit 230 correlates each information supplied from each sensor to the first processing unit 230 at a predetermined cycle while the flying object 4 equipped with the sensing device 2 is flying. It is generated by storing it in the first storage unit 225.

The time is the time when each information shown in the sensor data D is acquired, and is calculated based on the time information generated by the GPS sensor 222. The position is data indicating the position of the sensing device 2 at each time calculated based on the position information generated by the GPS sensor 222 and the movement vector generated by the 6-axis sensor 223, and includes, for example, latitude and longitude. .. The attitude is data indicating the attitude of the sensing device 2 at each time calculated based on the attitude information generated by the 6-axis sensor 223, and includes acceleration and angular velocity in the 3-axis direction. The first signal value is a set of signal values of each first signal generated by the first sensor 209 in response to electromagnetic waves in the visible wavelength region from each optical fiber 204a to d and f to j at each time. .. The second signal value is a set of signal values of each second signal generated by the second sensor 210 in response to electromagnetic waves in the near-infrared wavelength region from each optical fiber 204a to d and f to j at each time. Is. The third signal value is a set of signal values of each third signal generated by the third sensor 211 by the third sensor 211 according to the electromagnetic wave in the visible wavelength range from each optical fiber 204 kn, pt. ..

FIG. 6 is a block diagram showing a schematic configuration of the analysis device 3. The analysis device 3 includes a second communication unit 324, a second storage unit 325, a display unit 326, a second processing unit 330, and the like.

The second communication unit 324 includes a communication interface circuit such as a wired LAN or USB in order to enable the analysis device 3 to communicate with the sensing device 2. The second communication unit 324 transmits the data supplied from the second processing unit 330 to the sensing device 2, and supplies the data received from the sensing device 2 to the second processing unit 330.

The second storage unit 325 includes, for example, a semiconductor memory. The second storage unit 325 stores computer programs, data, and the like used for processing by the second processing unit 330. It may be installed in the storage device 140 from a portable recording medium such as a computer-readable CD-ROM or DVD-ROM using a known setup program or the like.

The display unit 326 has a display composed of a liquid crystal display, organic EL (Electro-Luminescence), and the like, and an interface circuit for outputting image data to the display. The display unit 326 displays various information on the display according to the instruction from the second processing unit 330.

The second processing unit 330 includes, for example, a CPU. The second processing unit 330 may include an LSI, an ASIC, a DSP, an FPGA, and the like. The second processing unit 330 is connected to the second communication unit 324, the second storage unit 325, the display unit 326, and the like, and controls each of these units.

FIG. 7 is a flow chart showing an example of the flow of the observation process executed by the sensing device 2. The observation process is periodically executed while the sensing device 2 is mounted on the projectile 4 and is in flight.

First, the acquisition means 231 drives the motor 221 to swing the swing mirror 206, and is generated from the first sensor 209, the second sensor 210, and the third sensor 211 in response to electromagnetic waves from each optical fiber. The first signal, the second signal, and the third signal are acquired, respectively. Further, the acquisition means 231 acquires position information, time information, posture information, and movement vector from the GPS sensor 222 and the 6-axis sensor 223, respectively. The acquisition means 231 associates the signal value of each acquired signal with the current time, the current position, and the current posture calculated from the position information, the time information, the posture information, and the movement vector, and stores the sensor data D in the first storage unit 225. Remember (S101).

The time for the swing mirror 206 to swing between the first position and the second position is shorter than the cycle in which the GPS sensor 222 generates the position information and the time information. The acquisition means 231 synchronizes the swing timing of the swing mirror 206 with the timing at which the GPS sensor 222 generates the position information and the time information each time the GPS sensor 222 generates the position information. The acquisition means 231 acquires the time information at the time indicated by the latest time information acquired from the GPS sensor 222, and then swings the swing mirror 206 the number of times and the swing mirror 206 is in the first position and the second position. The added value obtained by adding the multiplication value with the time fluctuating between the two is calculated as the current time.

Further, the cycle in which the 6-axis sensor 223 generates the attitude information and the movement vector is shorter than the cycle in which the GPS sensor 222 generates the position information and the time information. The acquisition means 231 synchronizes the timing at which the 6-axis sensor 223 generates the posture information and the movement vector with the timing at which the GPS sensor 222 generates the position information and the time information each time the GPS sensor 222 generates the position information. The acquisition means 231 calculates a position moved by the movement vector acquired from the 6-axis sensor 223 after acquiring the time information from the position indicated by the latest position information acquired from the GPS sensor 222. Further, the acquisition means 231 considers that the sensing device 2 is moving at a constant velocity from the time when the 6-axis sensor 223 is last generated to the next generation. The acquisition means 231 estimates the current position moved by the sensing device 2 from the calculated position to the current time by linear interpolation of the movement vector generated before and after the current time or interpolation by a Kalman filter or the like.

Further, the sensing device 2 sets the initial value of the attitude of the sensing device 2 at the time of takeoff, and sequentially adds the amount of attitude change shown in the attitude information generated by the 6-axis sensor 223 up to now to the initial value to obtain the attitude. calculate. Further, the acquisition means 231 considers that the sensing device 2 is performing constant velocity motion from the time when the 6-axis sensor 223 is last generated to the next generation of the posture information. The acquisition means 231 estimates the current posture changed from the calculated posture to the current time by linear interpolation of the posture information generated before and after the current time or interpolation by a Kalman filter or the like.

Next, the associating means 232 associates the first signal and the third signal stored in the first storage unit 225 (step S102). The associating means 232 is the first signal generated in response to an electromagnetic wave from the same ground position among the first signal and the third signal sequentially generated based on the difference in the angle between the first direction and the second direction. Correspond the signal and the third signal. When the second direction is a direction inclined forward with respect to the first direction, the associating means 232 associates the newly acquired first signal with the previously acquired third signal. On the other hand, when the second direction is a direction inclined backward with respect to the first direction, the associating means 232 associates the newly acquired third signal with the previously acquired first signal.

8 (a) and 8 (b) are schematic views for explaining the correspondence. FIG. 8A shows an example in which the second direction is tilted backward with respect to the first direction, and the projectile 4 (sensing device 2) is flying in parallel with the observation target F at a predetermined altitude h. .. In this case, the predetermined position P on the observation target F is located in the first direction with respect to the sensing device 2 at the time t1, and is positioned in the second direction with respect to the sensing device 2 at the time t2 after the predetermined time. is doing. Therefore, the first signal generated by the first sensor 209 at time t1 and the third signal generated by the third sensor 211 at time t2 are generated in response to electromagnetic waves from the same ground position. As shown in FIG. 8A, the distance between the position of the flying object 4 at the time t1 and the position of the flying object 4 at the time t2 is h · tan θ. The associating means 232 refers to the position information, extracts the first signal and the third signal for which the above relationship holds from the sensor data D, and associates them with each other. In this way, the associating means 232 associates the first signal and the third signal based on the position of the flying object 4. As a result, the mapping means 232 can appropriately map the first signal and the third signal even when the flight position (height) of the flying object 4 changes during flight.

The mapping means 232 may map the first signal and the third signal based on the posture information. FIG. 8B shows an example in which the second direction is tilted backward with respect to the first direction, and the projectile 4 (sensing device 2) is tilted with respect to the observation target F at a predetermined altitude h. Is shown. In this case, the distance between the position of the flying object 4 at the time t1 and the position of the flying object 4 at the time t2 is h · (tan (θ + φ2) −tan φ1). Here, φ1 is a pitch angle based on the first direction at time t1, and φ2 is a pitch angle based on the first direction at time t2. The associating means 232 refers to the posture information and calculates the pitch angles φ1 and φ2 for each sensor data D. The associating means 232 refers to the position information, extracts the first signal and the third signal for which the above relationship holds from the sensor data D, and associates them with each other. In this way, the associating means 232 associates the first signal and the third signal based on the attitude of the flying object 4. As a result, the associating means 232 can appropriately associate the first signal and the third signal even when the attitude of the flying object 4 changes during flight.

Further, the associating means 232 may associate the first signal and the third signal by using the image processing technique. For example, the associating means 232 includes an image including pixels having the first signal value of the plurality of first signals as pixel values and an image including pixels having the third signal values of the plurality of third signals as pixel values. The pattern matching process is executed, and the first signal and the third signal corresponding to the mutually matching pixels are associated with each other. It is preferable that the association using such an image processing technique is used for observation on the ground where the observation target includes feature points such as features.

Next, the calculation means 233 calculates the reflectance of the electromagnetic wave corresponding to the first signal in the observation target based on the second signal corresponding to the first signal sequentially generated (S103). The calculation means 233 calculates the reflectance of the first signal newly associated with the third signal. The calculation means 233 calculates the reflectance by normalizing the second signal value stored in the first storage unit 225. The reflectance R is calculated by the following equation (1).
R = (S-Sd) / (Sw-Sd) (1)
Here, S is the second signal value, Sd is the dark current value, and Sw is the second signal value (hereinafter, reference value) when the electromagnetic wave is reflected by the standard white plate installed on the observation target F. It may be called).

As the dark current value Sd, for example, the second lens detected in advance in a state where light does not enter from the first lens 201 and the second lens 202 (for example, a state where the cap is attached to the first lens 201 and the second lens 202). The signal value is used. Further, as the reference value Sw, the second signal value measured by installing a standard white plate on the observation target F in advance at the start of flight of the projectile 4 or the like is used.

The reference value Sw may be corrected based on the intensity of light from above the flying flying object 4. The reference value Sw changes according to the intensity of the incident light (mainly sunlight from above) on the standard white plate. Therefore, it is preferable that the reference value Sw is measured at the same time as the second signal value S of the observation target F. However, for that purpose, it is necessary to install a standard white plate over a wide range of the observation target F along the flight path of the projectile 4, which is difficult to realize. Therefore, the calculation means 233 uses the reference value Sw measured in advance in response to the electromagnetic wave from above the projectile 4 incident from the diffuser 203, which is measured at the same time as the electromagnetic wave from the observation target F. Correct based on the signal value.

The reference value Sw is corrected by the following equation (2).
Sw = Sw0 × (Sz / Sz0) (2)
Here, Sw0 is a reference value measured in advance, Sz0 is a second signal value measured in advance at the same time as Sw0 according to the electromagnetic wave from above the projectile 4, and Sz is from the observation target F. It is the second signal value measured at the same time as the electromagnetic wave of. That is, the calculation means 233 corrects the reflectance at the observation target based on the second signal value corresponding to the electromagnetic wave from above the flying object 4. As a result, the calculation means 233 accurately determines whether or not the electromagnetic wave is due to specular reflection even when the weather or the like changes during the flight of the projectile 4 and the intensity of sunlight changes. be able to.

Next, the determination means 234 determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the reflectance calculated by the calculation means 233 (S104). The determination means 234 determines that the electromagnetic wave corresponding to the first signal is due to specular reflection when the reflectance is equal to or higher than the predetermined threshold value, and when the reflectance is less than the predetermined threshold value, the electromagnetic wave corresponding to the first signal. Is not due to specular reflection. Normally, the reflectance of diffused light (rather than mirror reflection) in the observation target F is much smaller than the reflectance of the standard white plate, whereas the reflectance of the mirror reflected light is the reflection of the standard white plate. May exceed the rate. Therefore, the predetermined threshold value is set to, for example, 1.

Electromagnetic waves in the near-infrared wavelength range are more easily absorbed by water than electromagnetic waves in the visible wavelength range. Therefore, by using the wavelength band in the near infrared region for the determination of specular reflection, the sensing device 2 can make a more accurate determination without being affected by diffusion in water or reflection on the riverbed or seabed. .. In particular, the sensing device 2 can more accurately determine whether or not the electromagnetic wave is due to specular reflection by using an electromagnetic wave in the wavelength range of 700 nm to 800 nm or 1500 nm to 1600 nm as the electromagnetic wave in the near infrared region. ..

When it is determined that the electromagnetic wave corresponding to the first signal is not due to specular reflection (S104-No), the output means 235 outputs the first signal (S105). The output means 235 outputs, for example, by storing the first signal in the first storage unit 225 in association with the position information corresponding to the first signal.

On the other hand, when it is determined that the electromagnetic wave corresponding to the first signal is due to specular reflection (S104-No), the output means 235 responds to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. The generated third signal is output (S106). The output means 235 identifies a third signal generated in response to an electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal, based on the mapping result by the mapping means 232 in step S102. The output means 235 outputs the third signal by storing it in the first storage unit 225 in association with the position information corresponding to the first signal.

Next, the first processing unit 230 determines whether or not processing has been executed for a predetermined number of first signals (S107). When the processing has not been executed for the predetermined number of first signals (S107-No), the first processing unit 230 repeats the processing of S101 to S107.

On the other hand, when processing is executed for a predetermined number of first signals (S107-Yes), the generation means 236 generates an output image based on the first signal and the third signal output by the output means 235 (S107-Yes). S108), the series of steps is completed. The generation means 236 generates an output image including a plurality of pixels corresponding to the position information corresponding to each first signal. When it is determined that the electromagnetic wave corresponding to the first signal sequentially generated is not due to specular reflection, the generation means 236 sets the pixel value of the pixel corresponding to the first signal based on the first signal. .. On the other hand, when it is determined that the electromagnetic wave corresponding to the first signal sequentially generated is due to specular reflection, the generation means 236 responds to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. Based on the generated third signal, the pixel value of the pixel corresponding to the first signal is set.

For example, the generation means 236 sets a value obtained by normalizing (converting) the first signal value or the third signal value so as to be within a predetermined gradation range as the pixel value of each pixel. In addition, the generation means 236 corrected the first signal value or the third signal value based on the dark current value, the reference value, and the signal value corresponding to the electromagnetic wave from above the projectile 4, in the same manner as the processing of S103. In the above, a value normalized (converted) so as to be within a predetermined gradation range may be set as a pixel value of each pixel.

When the flight of the projectile 4 is completed, the generation means 236 outputs each output image generated by transmitting the generated output image to the analysis device 3 via the first communication unit 224. The analysis device 3 receives an output image from the sensing device 2 via the second communication unit 324, displays the received output image on the display unit 326, or transmits the received output image to another device via the second communication unit 324. Output by doing.

Note that the processing of S103 is omitted, and in S104, the determination means 234 determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on whether or not the second signal value exceeds a predetermined threshold value. May be determined. In that case, the determination means 234 may set a predetermined threshold value based on the second signal value generated in response to the electromagnetic wave from above the flying object 4. For example, the determination means 234 sets a predetermined threshold value so that the intensity of the electromagnetic wave from above increases as the intensity increases. As a result, the sensing device 2 can accurately determine whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection without calculating the reflectance.

Further, in S104 and S105, the output means 235 may output the first signal and the third signal by transmitting the first signal and the third signal to the external device via the first communication unit 224. In that case, the external device generates an output image based on the output first signal and the third signal.

In the above description, it is assumed that the sensing device 2 executes the analysis process, but the analysis process may be executed by the analysis device 3. In that case, the second processing unit 330 of the analysis device 3 has each unit of the first processing unit 230 of the sensing device 2. The first processing unit 230 of the sensing device 2 executes the processing of S101 in FIG. 7 during the flight by the flying object 4, and when the flight by the flying object 4 is completed, the time, position, attitude, first signal, and second. The sensor data D including the signal and the third signal is transmitted to the analysis device 3 via the first communication unit 224. On the other hand, in S101 of FIG. 7, the acquisition means 231 is the time, position, attitude, first signal, second signal, and third from the sensing device 2 in which the flight by the flying object 4 is completed via the second communication unit 324. The sensor data D including the signal is acquired and stored in the second storage unit 325. The second processing unit 330 sequentially executes the processing of S102 to S108 for each acquired sensor data D in order from the sensor data D having the oldest time. The generation means 236 displays the generated output image on the display unit 326 or outputs it by transmitting it to another device via the second communication unit 324.

Further, the analysis device 3 is connected to the sensing device 2 mounted on the projectile 4 so as to be capable of wireless communication, and while the projectile 4 is in flight, sensor data is sequentially acquired, and the acquired data is observed. The process may be executed. Further, the analysis device 3 is mounted on the projectile 4 in a state of being communicably connected to the sensing device 2, and while the projectile 4 is in flight, sensor data is sequentially acquired and observation is performed on the acquired data. The process may be executed.

Further, in the above description, an example is shown in which one set of optical systems for detecting electromagnetic waves in the visible light wavelength region from the second direction is used, and a third signal related to one direction is acquired. However, the sensing device 2 may set the second direction to a plurality of directions different from each other, include as many sets of optical systems as the number of the directions, and acquire a third signal including a signal related to each direction. In that case, the sensing device 2 uses four sets of optical systems in which the second direction is set to four directions of front, back, left, and right and the lens is directed in each direction so as to function as an oblique camera, for example. You may get it. As a result, the sensing device 2 can more reliably replace the first signal.

Further, in the above description, the GPS sensor 222 that uses GPS as a positioning means for acquiring the position together with the time is exemplified, but another GNSS such as GLONASS (GLONASS (GLObal'naya NAvigatsionnaya Sputnikovaya Sistema)) is used instead of the GPS222 sensor. The positioning means may be configured by a sensor, or the positioning means may be configured by a sensor that uses two or more types of GNSS in combination.

As described above, in the sensing system 1, when the incident electromagnetic wave is due to specular reflection, the signal generated according to the electromagnetic wave is generated according to the electromagnetic wave incident from a direction different from the electromagnetic wave. To generate an output image by replacing with. As a result, the sensing system 1 can generate an appropriate output image even when specular reflection occurs in the observation target.

That is, when the specularly reflected light is incident on the first sensor 209, the electromagnetic wave diffused in the observation target is buried in the specularly reflected light, so that the first signal cannot be used as the observation result, and the observation result is lost. Occurs. In this case, the sensing system 1 can prevent the observation result from being lost by using the third signal corresponding to the same point on the observation target instead of the first signal.

In aerial photography, it is assumed that specular reflection of sunlight can occur only from vertically below when the observation target is horizontal. Therefore, even if the electromagnetic wave incident from vertically below is due to specular reflection, it is highly possible that the electromagnetic wave incident from a direction different from the electromagnetic wave is not due to specular reflection. Therefore, the sensing system 1 can generate an appropriate output image even when specular reflection occurs in the observation target.

Although it is possible to estimate the specular reflection component from the first signal and subtract the specular reflection component, in that case, the state of water vapor and aerosol in the atmosphere fluctuates, so the specular reflection component is estimated with high accuracy. That is difficult. On the other hand, since the sensing system 1 replaces with the observation result that does not actually include the specularly reflected light, it is possible to generate a correct output image with high accuracy.

Further, the first sensor 209 and the second sensor 210 detect electromagnetic waves incident on the first lens 201 and the second lens 202, respectively. Since the first lens 201 and the second lens 202 are provided so that their optical axes are tilted from each other, specularly reflected light does not normally be incident on both the first lens 201 and the second lens 202. Therefore, the sensing system 1 can more reliably prevent the observation result from being lost.

It will be appreciated by those skilled in the art that various changes, substitutions and modifications can be made to this without departing from the spirit and scope of the invention. For example, the processes of the above-mentioned parts may be executed in different orders as appropriate within the scope of the present invention. Further, the above-described embodiments and modifications may be carried out in appropriate combinations within the scope of the present invention.

2 Sensing device 209 1st sensor 210 2nd sensor 211 3rd sensor 231 Acquisition means 232 Correspondence means 233 Calculation means 234 Judgment means 235 Output means 236 Generation means 3 Analysis device

Claims (9)

A sensing system installed on a flying object,
A second signal that is provided so as to be able to detect electromagnetic waves in the first wavelength region incident from the first direction forming the first pitch angle downward with respect to the traveling direction of the flying object, and sequentially generates a first signal corresponding to the detected electromagnetic waves. 1 sensor and
A second sensor that is provided so as to be able to detect an electromagnetic wave in a second wavelength region different from the first wavelength region incident from the first direction and sequentially generates a second signal corresponding to the detected electromagnetic wave.
An electromagnetic wave in the first wavelength region incident from a second direction having a second pitch angle different from the first pitch angle is provided downward with respect to the traveling direction of the flying object so as to be detectable, and the detected electromagnetic wave is responsive to the detected electromagnetic wave. The third sensor that sequentially generates the third signal and
A determination means for determining whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the second signal corresponding to the sequentially generated first signal.
When it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection, the pixel value of the pixel corresponding to the first signal is set based on the first signal, and the sequentially generated electromagnetic wave is set. When it is determined that the electromagnetic wave corresponding to the first signal is due to mirror reflection, the electromagnetic wave corresponding to the first signal is based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. , A generation means for generating and outputting an output image by setting the pixel value of the pixel corresponding to the first signal.
A sensing system characterized by having. Of the first and third signals sequentially generated based on the difference in angle between the first direction and the second direction, the first signal and the third signal generated in response to electromagnetic waves from the same ground position. It also has an associating means for associating three signals.
The generating means identifies a third signal generated in response to an electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal, based on the mapping result by the mapping means.
The sensing system according to claim 1. Further having an acquisition means for acquiring information on the position and attitude of the projectile,
The associating means associates the first signal with the third signal based further on the position and attitude of the flying object.
The sensing system according to claim 2. The first wavelength region is a visible wavelength region, and is
The second wavelength region is a near infrared wavelength region.
The sensing system according to any one of claims 1 to 3. Further, it has a calculation means for calculating the reflectance of the electromagnetic wave corresponding to the first signal in the observation target based on the second signal corresponding to the sequentially generated first signal.
The determination means determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the reflectance.
The sensing system according to any one of claims 1 to 4. The second sensor is further provided so as to be able to detect an electromagnetic wave incident from above the flying object.
The calculation means corrects the reflectance of the observation target based on the second signal corresponding to the electromagnetic wave incident from above the flying object.
The sensing system according to claim 5. It is installed on the flying object and is provided so as to be able to detect electromagnetic waves in the first wavelength range incident from the first direction forming a predetermined pitch angle downward with respect to the traveling direction of the flying object, and responds to the detected electromagnetic waves. A first sensor that sequentially generates a first signal and an electromagnetic wave in a second wavelength range different from the first wavelength range incident from the first direction are provided so as to be able to detect, and a second signal corresponding to the detected electromagnetic wave is sequentially generated. The generated second sensor and the electromagnetic wave in the first wavelength region incident from the second direction having a pitch angle different from the predetermined pitch angle below the traveling direction of the flying object are detectable and detected. An acquisition means for acquiring the first signal, the second signal, and the third signal from a sensing device having a third sensor that sequentially generates a third signal corresponding to the generated electromagnetic wave.
A determination means for determining whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the second signal corresponding to the sequentially generated first signal.
When it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection, the pixel value of the pixel corresponding to the first signal is set based on the first signal, and the sequentially generated electromagnetic wave is set. When it is determined that the electromagnetic wave corresponding to the first signal is due to mirror reflection, the electromagnetic wave corresponding to the first signal is based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. , A generation means for generating and outputting an output image by setting the pixel value of the pixel corresponding to the first signal.
An information processing device characterized by having. The computer
It is installed on the flying object and is provided so as to be able to detect electromagnetic waves in the first wavelength range incident from the first direction forming a predetermined pitch angle downward with respect to the traveling direction of the flying object, and responds to the detected electromagnetic waves. A first sensor that sequentially generates a first signal and an electromagnetic wave in a second wavelength range different from the first wavelength range incident from the first direction are provided so as to be able to detect, and a second signal corresponding to the detected electromagnetic wave is sequentially generated. The generated second sensor and the electromagnetic wave in the first wavelength region incident from the second direction having a pitch angle different from the predetermined pitch angle below the traveling direction of the flying object are detectable and detected. The first signal, the second signal, and the third signal are acquired from the third sensor that sequentially generates the third signal corresponding to the generated electromagnetic wave.
Based on the second signal corresponding to the sequentially generated first signal, it is determined whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection.
When it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection, the pixel value of the pixel corresponding to the first signal is set based on the first signal, and the sequentially generated electromagnetic wave is set. When it is determined that the electromagnetic wave corresponding to the first signal is due to mirror reflection, it is based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. By setting the pixel value of the pixel corresponding to the first signal, an output image is generated and output.
A sensing method characterized by including. A program that controls a computer
It is installed on the flying object and is provided so as to be able to detect electromagnetic waves in the first wavelength range incident from the first direction forming a predetermined pitch angle downward with respect to the traveling direction of the flying object, and responds to the detected electromagnetic waves. A first sensor that sequentially generates a first signal and an electromagnetic wave in a second wavelength range different from the first wavelength range incident from the first direction are provided so as to be able to detect, and a second signal corresponding to the detected electromagnetic wave is sequentially generated. The generated second sensor and the electromagnetic wave in the first wavelength region incident from the second direction having a pitch angle different from the predetermined pitch angle below the traveling direction of the flying object are detectable and detected. The first signal, the second signal, and the third signal are acquired from the third sensor that sequentially generates the third signal corresponding to the generated electromagnetic wave.
Based on the second signal corresponding to the sequentially generated first signal, it is determined whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection.
When it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to mirror reflection, the pixel value of the pixel corresponding to the first signal is set based on the first signal, and the sequentially generated electromagnetic wave is set. When it is determined that the electromagnetic wave corresponding to the first signal is due to mirror reflection, it is based on the third signal generated in response to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal. By setting the pixel value of the pixel corresponding to the first signal, an output image is generated and output.
A program that causes the computer to do this.

Images