WO2020044628A1

WO2020044628A1 - Cultivated land imaging system and cultivated land imaging method

Info

Publication number: WO2020044628A1
Application number: PCT/JP2019/010050
Authority: WO
Inventors: 片桐　哲也; 康男小柳
Original assignee: コニカミノルタ株式会社
Priority date: 2018-08-29
Filing date: 2019-03-12
Publication date: 2020-03-05
Also published as: JPWO2020044628A1

Abstract

This cultivated land imaging system comprises a flying vehicle, a plurality of imaging units, a chart, and a parallax correction unit (11c). The plurality of imaging units are retained in the flying vehicle, and, during flight of the flying vehicle, acquire cultivated land images of differing wavelength bands by performing imaging from the air of cultivated land where crops are grown. The parallax correction unit corrects positional offset, resulting from parallax, of pixels indicating the same location among the cultivated land images, the correction carried out on the basis of chart images obtained by the plurality of imaging units that image the chart from the air during flight of the flying body. The chart is located inside of or within the vicinity of the cultivated land.

Description

Field photographing system and field photographing method

The present invention relates to a field photographing system and a field photographing method for photographing a field from above.

システム Conventionally, systems for measuring plant growth indices have been proposed. For example, in the system of Patent Document 1, the reflected light measurement unit measures the light intensity of the reflected light of the measurement target having a plurality of leaves at the first wavelength and the second wavelength, and the sun angle acquisition unit measures the sunlight. The angle of incidence on the reflected light is acquired as the sun angle, and the direction of the sun with respect to the measurement direction of the reflected light measurement unit is acquired as the sun direction by the sun direction acquisition unit. Then, the growth index calculating section obtains each light intensity of the reflected light at each of the first and second wavelengths measured by the reflected light measuring section, the sun angle obtained by the sun angle obtaining section, and the sun direction obtained by the sun direction obtaining section. A growth index representing the degree of growth in the measurement target is obtained based on the obtained sun direction. In Patent Document 1, for example, NDVI (NormalizedmalDifference Vegetation Index; normalized vegetation index) is obtained as the above growth index.

Here, the reflected light measurement unit is a so-called multi-spectral camera including a visible imaging unit that generates an image of visible light and an infrared imaging unit that generates an image of infrared light. Each of the visible imaging unit and the infrared imaging unit includes a bandpass filter, a lens (imaging optical system), and a sensor.

International Publication WO2016 / 181743 (See claim 5, paragraphs [0021] to [0077], FIG. 5 and the like)

In recent years, a multispectral camera is mounted on an unmanned aerial vehicle (also called a drone), and a drone is flown to photograph a field from above, thereby acquiring a visible image and an infrared image, and measuring NDVI. ing. The above-mentioned field refers to a paddy field or a field where a crop is grown. However, when the visible image and the infrared image are acquired by mounting the multispectral camera on the drone, the NDVI value fluctuates due to the following causes.

(A) Change in Internal State of Camera Due to vibration and temperature changes during flight of the drone, for example, the relative directions and focal lengths of the respective optical axes of the visible imaging unit and the infrared imaging unit change. In this case, a shift (also called parallax) occurs at a position indicating the same point in the visible image and the infrared image. Therefore, when the NDVI value is calculated using the pixel values at the same coordinate position in the visible image and the infrared image, the NDVI value is calculated using the pixel values corresponding to different points in the field. However, it is not possible to obtain an accurate NDVI value for an arbitrary point in the field.

(B) Changes in the external state of the camera The mounting directions of the visible imaging unit and the infrared imaging unit are deviated due to vibrations during flight of the drone and variations in gimbal control. Also in this case, since a parallax occurs between the visible image and the infrared image, it is impossible to obtain an accurate NDVI value for an arbitrary point in the field, as in (A).

In particular, if each optical system included in the visible imaging unit and the infrared imaging unit is contaminated by, for example, dust, the pixel values of the visible image and the infrared image acquired by the visible imaging unit and the infrared imaging unit are affected. Includes noise. For this reason, the fluctuation of the NDVI value caused by the above (A) and (B) is further increased.

Therefore, when flying a multispectral camera mounted on a drone, the influence of parallax generated between the visible image and the infrared image is reduced so as to prevent the fluctuation of the NDVI value due to the above-described cause. It is desirable to perform parallax correction (calibration) for performing the correction. In this regard, the above-mentioned Patent Document 1 does not consider parallax correction when the multispectral camera is mounted on a drone and is flown. Even when performing parallax correction, it is desirable to perform parallax correction at the same time as imaging of a field from the viewpoint of processing efficiency, and it is desired to construct a system that can realize such parallax correction.

The present invention has been made in order to solve the above-described problems, and its object is to mount a plurality of imaging units for acquiring images with different wavelength bands on a flying object, in a system for photographing a field from above, A field photographing system and a field that can reduce the influence of parallax between images acquired by a plurality of imaging units and can perform parallax correction for reducing the effect of the parallax simultaneously with shooting of a field It is to provide a shooting method.

A field imaging system according to one aspect of the present invention obtains a field object having a different wavelength band by shooting a flying object and a field held by the flying object and growing a crop from above, while the aircraft is flying. A plurality of imaging units, and a chart captured by the plurality of imaging units, and based on each chart image obtained by capturing the chart from above by the plurality of imaging units during the flight of the flying object. A parallax correction unit that corrects a positional shift of a pixel indicating the same point caused by parallax between the respective field images, wherein the chart is located in the field or around the field.

A field imaging method according to another aspect of the present invention is directed to a field imaging method in which a plurality of imaging units held by a flying object captures, from the sky, a field where a crop grows during the flight of the flying object, and a field image having a different wavelength band. A field image obtaining step of obtaining, and a chart located in or around the field, during the flight of the flying object, from the sky by the plurality of imaging units to obtain each chart image The method includes a chart image acquiring step and a parallax correcting step of correcting a positional shift of a pixel indicating the same point caused by parallax between the respective field images based on the respective chart images.

According to the above-described field imaging system and the field imaging method, a plurality of imaging units held by the flying object image the field from above, so that field images having different wavelength bands are obtained. The parallax correction unit corrects a positional shift of a pixel indicating the same point caused by parallax between each field image based on each chart image obtained by photographing a chart with a plurality of imaging units. Thereby, even if parallax is generated between the field images due to the vibration of the flying object or the like, it is possible to obtain the field images in which the positional deviation of the pixels due to the parallax is corrected, and reduce the influence of the parallax. It becomes possible. In addition, since the above-described chart is located in or around the field, the plurality of imaging units can shoot the chart from above at the same time as shooting the field from above. Accordingly, the parallax correction unit can perform the parallax correction for reducing the influence of the parallax at the same time as the imaging of the field, and can improve the processing efficiency.

FIG. 1 is an explanatory diagram schematically showing an overall configuration of a field photographing system according to an embodiment of the present invention. It is explanatory drawing which shows typically the detailed structure of the imaging device contained in the said field imaging system. FIG. 3 is a block diagram illustrating a schematic configuration of a terminal device included in the field photographing system. It is a top view of a chart contained in the above-mentioned field photography system. It is a top view showing other composition of the above-mentioned chart. It is explanatory drawing which shows the state which attached the chart to the transport vehicle. It is explanatory drawing which shows the state which attached the chart to the helmet. It is explanatory drawing which substitutes as a chart and shows typically the stock missing part in a field. It is explanatory drawing which substitutes as a chart and shows the reflection spot of sunlight reflected on the water surface in a field typically. FIG. 3 is a block diagram illustrating a detailed configuration of a flying object included in the field imaging system. It is a flowchart which shows the flow of the process by the field imaging method implemented by the said field imaging system. It is a flow chart which shows the flow of processing of parallax correction in the above-mentioned field photography system. FIG. 9 is an explanatory diagram schematically showing a chart image extracted from a plurality of captured images. It is explanatory drawing which shows projective transformation typically.

The following is a description of embodiments of the present invention, with reference to the accompanying drawings.

[Configuration of field photographing system]
FIG. 1 is an explanatory diagram schematically showing the overall configuration of the field photographing system 1 of the present embodiment. The field photographing system 1 is a system for photographing a field FD that grows a crop PL, and includes a flying object 10, an imaging device 20, a terminal device 30, and a chart C. The flying object 10 and the terminal device 30 are communicably connected via a communication line NW. The communication line NW is configured by a network (whether wired or wireless) such as a LAN (Local Area Network) or an Internet line. The communication function of the flying object 10 may be provided to the imaging device 20, and the imaging device 20 and the terminal device 30 may be communicably connected via the communication line NW.

(Imaging device)
The imaging device 20 is configured by, for example, a multispectral camera, and is held by the flying object 10. FIG. 2 schematically shows a detailed configuration of the imaging device 20. The imaging device 20 includes a first imaging unit 21 and a second imaging unit 22. The first imaging unit 21 is a visible imaging unit that captures an image of the wavelength band of visible light (visible image) by capturing an image of the field FD. The second imaging unit 22 is a near-infrared imaging unit that captures an image of the field FD and acquires an image in the wavelength band of near-infrared light (near-infrared image). Since the imaging device 20 includes the first imaging unit 21 and the second imaging unit 22, the field FD is photographed from above while the flying object 10 is flying, and field images having different wavelength bands are acquired. Can be.

The first imaging unit 21 includes a first bandpass filter 21a, a first imaging optical system 21b, a first image sensor (optical sensor) 21c, a first digital signal processor (not shown), and the like. Have. The first bandpass filter 21a is an optical filter that transmits light in a relatively narrow band having a center wavelength of, for example, 650 nm. The first imaging optical system 21b includes at least one lens, and forms a first optical image of visible light of a measurement target (crop PL or field FD) transmitted through the first bandpass filter 21a. Form an image on the surface. The first image sensor 21c is composed of, for example, a VGA (640 pixels × 480 pixels) CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and the light receiving surface is a first imaging surface. Detects a relatively narrow band of light centered on a wavelength of 650 nm contained in sunlight reflected by the object to be measured, and converts the optical image of the visible light of the object to be converted into an electrical signal. I do. The first digital signal processor performs image processing on the output of the first image sensor 21c to form a visible image. The image data Rv of the visible image is output to the control unit 11 of the flying object 10 (see FIG. 10). Although the transmission wavelength band of the first bandpass filter 21a is the red (R) wavelength band in the present embodiment, it may be another wavelength band such as blue (B) or green (G). Good.

The second imaging unit 22 includes a second bandpass filter 22a, a second imaging optical system 22b, a second image sensor (optical sensor) 22c, a second digital signal processor (not shown), and the like. Have. The second bandpass filter 22a is an optical filter that transmits light in a relatively narrow band having a center wavelength of a predetermined wavelength of 750 nm or more (for example, a wavelength of 800 nm). The second imaging optical system 22b includes at least one lens, and forms an optical image of near-infrared light of the measurement target transmitted through the second bandpass filter 22a on the second imaging surface. I do. The second image sensor 22c is formed of, for example, a VGA type CCD sensor or a CMOS sensor, is arranged such that the light receiving surface coincides with the second imaging surface, and is included in sunlight reflected by the measurement target. A relatively narrow band light having a center wavelength of 800 nm is detected, and an optical image of near-infrared light to be measured is converted into an electric signal. The second digital signal processor performs image processing on the output of the second image sensor 22c to form a near-infrared image. The image data Ri of the near-infrared image is output to the control unit 11 of the flying object 10.

(Terminal device)
FIG. 3 is a block diagram illustrating a schematic configuration of the terminal device 30. The terminal device 30 is configured by, for example, a PC (personal computer), and includes an input unit 31, a display unit 32, a storage unit 33, a communication unit 34, and a control unit 35. Note that the terminal device 30 may be configured by a multifunctional portable terminal (for example, a smartphone or a tablet terminal).

The input unit 31 is operated by the user, and is provided for receiving input of information by the user. Such an input unit 31 is specifically configured by an operation unit such as a keyboard, a mouse, and a touchpad. The display unit 32 is a display that displays various types of information, and is configured by, for example, a liquid crystal display device.

The storage unit 33 is constituted by, for example, a hard disk, and stores various information in addition to a program for operating the control unit 35. For example, a calculation formula (distortion correction information) for correcting the distortion of the visible image and the near-infrared image acquired by the imaging device 20 (the first imaging unit 21 and the second imaging unit 22) is as follows. Is determined in advance according to the design of the first imaging optical system 21b and the second imaging optical system 22b. Such distortion correction information is stored in the storage unit 33 in advance, and is provided to the flying object 10 via the communication line NW when performing distortion correction processing inside the flying object 10. Note that the above-described distortion correction and parallax correction described below may be performed by the control unit 35 of the terminal device 30.

The communication unit 34 is an interface for performing communication with the flying object 10, and includes a transmission circuit, a reception circuit, a modulation circuit, a demodulation circuit, an antenna, and the like, and is communicably connected to the communication line NW. ing. The control unit 35 is constituted by, for example, a CPU (Central Processing Unit), operates according to the operation program stored in the storage unit 33, controls the operation of each unit of the terminal device 30, and necessitates Computation processing is performed accordingly.

(chart)
The chart C is located in the field FD or around the field FD, and is used when performing parallax correction described later. For example, the chart C is attached to an object OB located in the field FD or around the field FD, and is thereby located in the field FD or around the field FD. FIG. 1 shows an example in which the take-off and landing platform 40 located around the field FD is used as the object OB, and the chart C is attached to the take-off and landing platform 40. The take-off and landing table 40 is a sheet (table) installed on the ground in order to prevent the surrounding sand from rolling up when the flying object 10 takes off and land. Such a chart C is photographed from above by the first imaging unit 21 and the second imaging unit 22 during the flight of the flying object 10.

FIG. 4 is a plan view of the chart C. The chart C has a pattern including at least one circular portion C ₁ (for example, a black circle). Pattern includes a pattern (not intended generated) naturally present in the periphery of the field FD or in the field FD, to distinguish the chart C for parallax correction, chart C is a plurality of circular portions C ₁ it is desirably, three or more and is more preferably a pattern including a circular portion C _1, reflectance from this, in addition to the circular part C ₁ different circular portion C ₂ (for example, black and sheets of background It is desirable that the pattern includes a pattern (a circular portion colored with a color different from the color). Further, as shown in FIG. 5, the chart C may have a lattice pattern. The overall shape of the chart C is preferably a square, but may be another shape such as a rectangle.

Here, the object OB to which the chart C is attached is not limited to the take-off and landing platform 40 described above. As shown in FIG. 6, as the object OB, a transport vehicle 41 that transports the flying object 10 may be used, and the chart C may be attached to the transport vehicle 41 (for example, a ceiling or a hood). Further, as shown in FIG. 7, a helmet 42 worn by a pilot who remotely controls the flight of the flying object 10 is used as the object OB, and a chart C (for example, one circular portion C ₁ ) is placed on top of the helmet 42. May be attached.

In addition, as the object OB, for example, an incident light sensor installed on the ground (around the field FD) may be used, and the chart C may be attached to this incident light sensor. The incident light sensor is provided to correct the image data Rv of the visible image and the image data Ri of the near-infrared image acquired by the imaging device 20 according to the weather (clear, cloudy). And corresponds to the sunlight measuring unit of Patent Document 1. Further, a generator (located around the field FD) for charging the battery 17 (see FIG. 10) mounted on the flying object 10, a field server such as a thermometer and a water level gauge installed in the field FD, and an RTK -GPS (Real Time Kinematic-Global Positioning System) base stations (located in or around the field FD), agricultural machines such as tractors and combiners (located around the field FD), irrigation equipment such as floodgates (field FD) Inside or around the field FD) may be used as the object OB, and the chart C may be attached to these objects OB.

Further, as shown in FIG. 8, the lacking portion N of the crop PL located in the field FD may be used as the chart C. Here, the lacking portion N of the crop PL refers to a portion (region) where the crop PL does not grow from the beginning in the field FD. For example, when the crop PL is rice, the rice is not planted from the beginning. Refers to the part. In FIG. 8, a hatched portion indicates a region where the crop PL is growing in the field FD, and a white portion without hatching corresponds to the missing portion N. Further, as shown in FIG. 9, the reflection spot S of sunlight reflected on the water surface when the field FD is immersed in water may be used as the chart C.

(Flying object)
The flying object 10 shown in FIG. 1 is configured by, for example, an unmanned aerial vehicle (drone) capable of autonomous flight. The flying object 10 may be configured by a balloon, an airship, an airplane, a helicopter, or the like, in addition to the unmanned aircraft described above. The flight of the flying object 10 is remotely controlled by a ground pilot (operator) operating the operation unit 50 (see FIG. 10). By flying the flying object 10 along the field FD, a plurality of regions T constituting the field FD are photographed from the sky by the imaging device 20, and an image (for example, a visible image or an infrared image) is formed for each region T. Can be obtained. Therefore, an image of the entire field FD can be finally obtained by bonding the images of the respective regions T. In addition, by raising the altitude of the flying object 10 and photographing the entire field FD collectively, it is also possible to acquire one image of the entire field FD without pasting the images of the respective regions T.

FIG. 10 is a block diagram showing a detailed configuration of the flying object 10. As shown in FIG. The flying object 10 includes a control unit 11, a communication unit 12, a warning unit 13, a position information acquisition unit 14, an image storage unit 15, a flight drive unit 16, and a battery 17. The flight drive unit 16 has, for example, a propeller, a motor, and the like, and causes the flying object 10 to fly by driving the motor to rotate the propeller. The flight control of the flying object 10 is performed by the control unit 11. At this time, the control unit 11 controls the flight drive unit 16 based on the operation of the operation unit 50 by the pilot on the ground. The battery 17 is a battery that supplies driving power to each part of the flying object 10.

The control unit 11 includes, for example, a CPU, and includes a general control unit 11a, a distortion correction unit 11b, a parallax correction unit 11c, a growth index calculation unit 11d, and an image synthesis unit 11e. That is, the CPU functions as the overall control unit 11a, the distortion correction unit 11b, the parallax correction unit 11c, the growth index calculation unit 11d, and the image synthesis unit 11e. The overall control unit 11a controls the operation of each unit of the flying object 10 according to an operation program stored in a program storage unit (not shown). The details of the distortion correcting unit 11b, the parallax correcting unit 11c, the growth index calculating unit 11d, and the image synthesizing unit 11e will be described later in the operation description.

The communication unit 12 is an interface for performing communication with the terminal device 30 and includes a transmission circuit, a reception circuit, a modulation circuit, a demodulation circuit, an antenna, and the like, and is communicably connected to the communication line NW. ing.

The warning unit 13 performs control when the amount of change in pixel value positional deviation due to parallax in a predetermined period (for example, one day or one week) calculated by the parallax correction unit 11c of the control unit 11 exceeds a threshold. A warning is initially issued under the control of the unit 11 (for example, the overall control unit 11a). Such a warning unit 13 includes a display unit 13a (for example, a liquid crystal display device or a simple light source) for performing a warning display, a sound output unit 13b (for example, a speaker) that emits a warning sound, and transmits warning information to the outside. It can be configured with at least one of the communication units (for example, the communication unit 12).

The position information acquiring unit 14 is configured by, for example, GPS, and acquires position information (latitude X, longitude Y, height Z) of the flying object 10 (the imaging device 20). Thereby, the control unit 11 (for example, the overall control unit 11a) controls the flight driving unit 16 based on the position information, and moves the flying object 10 to a predetermined height position above a predetermined area T of the field FD. Can fly.

The image storage unit 15 is configured by, for example, a nonvolatile memory, and stores each image data of a visible image and a near-infrared image acquired by the first imaging unit 21 and the second imaging unit 22 of the imaging device 20. At the same time, the growth index (particularly, the image data of the NDVI image obtained based on the pixel value after parallax correction) obtained by the growth index calculation unit 11d is stored.

[About processing in the field imaging system]
Next, a field photographing method by the field photographing system 1 having the above configuration will be described. FIG. 11 is a flowchart illustrating a flow of processing according to the field photographing method of the present embodiment. Here, it is assumed that the chart C is attached to the take-off and landing platform 40 located around the field FD.

{Circle around (1)} First, the flying object 10 is caused to fly to photograph the field FD and the chart C (S1; photographing step). More specifically, the pilot takes off the flying object 10 from the take-off and landing platform 40 by operating the operation unit 50. Then, during the flight of the flying object 10, the first imaging unit 21 and the second imaging unit 22 held by the flying object 10 photograph the field FD from above, and acquire field images having different wavelength bands (S1). -1; field image acquisition step). That is, the first imaging unit 21 acquires a visible image of the field FD as one field image, and the second imaging unit 22 acquires a near-infrared image of the field FD as the other field image. The acquired image data Rv and the near-infrared image data Ri of the visible image are input from the first imaging unit 21 and the second imaging unit 22 to the control unit 11 of the flying object 10, respectively.

Further, during the flight of the flying object 10, the chart C attached to the take-off and landing platform 40 is photographed from the sky by the first imaging unit 21 and the second imaging unit 22 to acquire each chart image (S1-2). A chart image obtaining step). That is, the first imaging unit 21 acquires a visible image of the chart C as one chart image, and the second imaging unit 22 acquires a near-infrared image of the chart C as the other chart image. Here, during the above-mentioned “during flight of the flying object 10”, the flying object 10 is rising until the flying object 10 takes off from the take-off and landing platform 40 and reaches a predetermined altitude above the field, and It is assumed that the descent of the flying object 10 from landing at a predetermined altitude in the sky to landing on the take-off and landing platform 40 is included. Image data of each chart image acquired by the first imaging unit 21 and the second imaging unit 22 is input to the control unit 11 of the flying object 10, respectively.

In the present embodiment, the first imaging unit 21 and the second imaging unit 22 simultaneously perform the field image acquiring step of S1-1 and the chart image acquiring step of S1-2. That is, the first imaging unit 21 and the second imaging unit 22 simultaneously capture the chart C while capturing the field FD from above. As a result, a field image showing the chart C (field image also serving as the chart image) is obtained for each of visible and near-infrared.

Next, the distortion correction unit 11b of the control unit 11 performs the first imaging unit 21 and the first imaging unit 21 based on the distortion correction information (the calculation formula for correcting the distortion) provided from the terminal device 30 via the communication unit 12. The image data of each image (each field image, each chart image) output from the second imaging unit 22 is respectively corrected. Thereby, the distortion such as the barrel shape and the pincushion type in each image is corrected, and the next parallax correction can be appropriately performed using each image in a state where the distortion has been corrected.

Subsequently, the parallax correction unit 11c of the control unit 11 determines between the field images based on each chart image obtained by photographing the chart C from above by the first imaging unit 21 and the second imaging unit 22. Then, parallax correction (calibration) for correcting a positional shift of pixels indicating the same point caused by parallax is performed (S3; parallax correction step). The details of the parallax correction in S3 will be described later.

After the parallax correction in S3, the overall control unit 11a determines whether or not the amount of change in the positional deviation during a predetermined period exceeds a threshold (S4). For example, when the visible image and the near-infrared image are superimposed on each other, the overall control unit 11a elapses a predetermined period (for example, one day or one week) of the length of a straight line connecting positions indicating the same point. The difference between before and after the change is referred to as the “change amount of the positional deviation” described above, and it is determined whether the change amount exceeds a threshold. When the change amount exceeds the threshold, the warning unit 13 issues a warning under the control of the overall control unit 11a (S5; warning step). In this warning step, at least one of displaying a warning on the display unit 13a, outputting a warning sound by the audio output unit 13b, and transmitting warning information to the outside by the communication unit 12 is performed. On the other hand, if the amount of change is equal to or smaller than the threshold, the process directly proceeds to S6.

Next, the growth index calculation unit 11d of the control unit 11 calculates the growth index of the crop PL based on each field image after the positional shift has been corrected by the parallax correction unit 11c (S6; growth index calculation step). . In the present embodiment, NDVI is used as the growth index. NDVI is an index indicating the distribution status and activity of vegetation, and the image data Rv of one field image (visible image) and the other field image (visual image) in which the positional shift due to distortion and parallax has been corrected as described above. NDVI = {(Ri−Rv) / (Ri + Rv)} using image data Ri of a near infrared image). NDVI indicates a numerical value normalized between −1 and 1, and a larger positive number indicates a denser vegetation. When the NDVI is calculated for each pixel of each field image, an NDVI image showing the distribution of each pixel of the NDVI is obtained.

Note that the growth index calculation unit 11d may calculate the following value instead of NDVI as the growth index. As an index other than NDVI, for example, RVI (Ratio Vegetation Index; specific vegetation index, RVI = Ri / Rv), DVI (Difference Vegetation Index; difference vegetation index, DVI = Ri-Rv), TVI (Transformed Vegetation Index, TVI) = NDVI ^0.5 +0.5) or IPVI (Infrared Percentage Vegetation Index, IPVI = Ri / (Ri + Rv) = (NDVI + 1) / 2).

生 Furthermore, the growth index calculating unit 11d may calculate the planting rate as a growth index instead of NDVI. The vegetation cover rate indicates a rate at which the crop PL covers the ground surface of the field FD. For example, the growth index calculation unit 11d performs a binarization process on the near-infrared image after correcting the positional shift due to the distortion and the parallax, to form a binary image of white and black, and By calculating the ratio of the white portion in the image, the planting rate can be calculated. In the binarized image, a white portion corresponds to the crop PL, and a black portion corresponds to the soil.

Next, when a plurality of NDVI images are obtained for each field FD, or when an NDVI image is obtained for each region T included in each field FD, the image combining unit 11e of the control unit 11 The NDVI images are combined to generate one NDVI image as a whole (S7; image combining step). Note that “combining” here refers to connecting a plurality of NDVI images side by side. If the number of obtained NDVI images is originally one, image synthesis is unnecessary, and thus the process of S7 is skipped and the process ends.

[Details of parallax correction]
Next, the details of the parallax correction in S3 will be described. FIG. 12 is a flowchart illustrating the flow of the parallax correction process. First, the parallax correction unit 11c uses the first imaging unit 21 and the second imaging unit 22 to extract the chart C from a plurality of captured images acquired at different timings when the flying object 10 is moving over the sky. An image including the image is extracted as the above-described chart image (S11). For example, the disparity compensation unit 11c, from the plurality of captured images, at least one image corresponding to the circular portion C ₁ of the chart C in the image of (preferably four) image present, include an image of the chart C Extracted as a chart image. Then, the extraction of such a chart image is performed on a plurality of captured images obtained by the first imaging unit 21 and the second imaging unit 22, respectively.

FIG. 13 schematically illustrates a chart image extracted by the parallax correction unit 11c from the plurality of captured images. Here, the chart image CR-1 is acquired at time t1, the chart image CR-2 is acquired at time t2, the chart image CR-3 is acquired at time t3, and the second image is acquired by the first imaging unit 21. When the imaging unit 22 obtains the chart image Ci-1 at time t1, obtains the chart image Ci-2 at time t2, and obtains the chart image Ci-3 at time t3, they are arranged in chronological order. Is shown. Note that CR is a chart image in a visible wavelength band, and Ci is a chart image in a near-infrared wavelength band. In FIG. 13, in each chart image, an image corresponding to the circular portion C ₁ of the chart C, indicated by C ₁ '.

Next, the parallax correction unit 11c extracts the area CA of the chart C from each of the extracted chart images (S12). For example, the parallax correction unit 11c calculates the position (coordinates) of the image C ₁ ′ corresponding to the circular portion C ₁ in the extracted chart image, and calculates the number of the images C ₁ ′. Then, the image C ₁ 'if there four are parallax correction unit 11c extracted image C ₁ adjacent in the chart image' the distance between the image C ₁ 'is within a predetermined range, and, When the shape surrounding the four images C ₁ ′ is a square, the area surrounding the four images C ₁ ′ is extracted as the area CA of the chart C. Further, for example, when there is one image C ₁ ′ in the extracted chart image, a square area of a predetermined size surrounding the image C ₁ ′ is extracted as the area CA of the chart C.

Next, the parallax correction unit 11c is included in one chart image (for example, a visible image) based on each chart image captured and acquired by the first imaging unit 21 and the second imaging unit 22 at the same time. A projection matrix for converting the position coordinates of the reference point P to the position coordinates of the corresponding point Q (indicating the same point in the field FD) corresponding to the other chart image (for example, a near-infrared image) is calculated (S13). Here, the reference point P and the corresponding point Q are, for example, the center positions of the area CA of the chart C obtained in S12 in each chart image. For example, when there are four images C ₁ ′ in the chart image, the center position is the average value of the positions of the four images C ₁ ′ (the coordinate position obtained by adding the x coordinate and the y coordinate and dividing by 4). When there is _one image C ₁ ′ in the chart image, the position of the image C ₁ ′ can be set.

Here, a method of calculating the projection matrix will be described. In general, projecting a certain plane onto another plane using a projection matrix is called homography (projection transformation). As one of the projective transformations, an affine transformation represented by the following equation is generally known (for example, see http://zellij.hatenablog.com/entry/20120523/p1). Hereinafter, it is assumed that the projective transformation is an affine transformation.

As shown in FIG. 14, the coordinates of an arbitrary point O on the xy plane are set to (x, y), and the point O is defined by a projection matrix (corresponding to a matrix of 3 rows and 3 columns in Formula 1) on the x′y ′ plane. The coordinates of the point O ′ projected above are (x ′, y ′). The projective transformation is a transformation that combines linear transformation and translation. The above-described linear transformation includes scaling (differential scaling), shearing (skew), and rotation (rotation). In the above projection matrix, the coefficient t _x and t _y, refers to coefficients for parallel movement (translation), the remaining coefficients a, b, c and d refer to coefficients for linear transformation.

And calculating the projection matrix is to calculate the coefficient of the projection matrix a, b, c, d, six parameters t _x and t _y. Since there are six unknown parameters (coefficients), if six equations can be established, all six unknown parameters can be obtained (that is, a projection matrix can be calculated). At this time, if the position coordinates of one point before and after projective transformation are known, two equations can be established for x and y by substituting the coordinates into Equation 1. Therefore, by substituting the position coordinates before and after the projective transformation for the three points into Equation 1, six equations can be established, and all six parameters can be obtained by solving them simultaneously.

Therefore, in the present embodiment, the parallax correction unit 11c inputs the position coordinates of the three points into Expression 1 and calculates the projection matrix as follows. First, the correspondence (x1, y1) of the reference point P on the xy plane of the chart image CR-1 acquired at the time t1 to the correspondence on the x'y 'plane of the chart image Ci-1 acquired at the same time. The coordinates (x1 ′, y1 ′) of the point Q are input into Equation 1. Similarly, the coordinates (x2, y2) of the reference point P on the xy plane of the chart image CR-2 acquired at the time t2 and the x'y 'plane of the chart image Ci-2 acquired at the same time The coordinates (x2 ′, y2 ′) of the corresponding point Q are input into Equation 1. Further, the correspondence (x3, y3) of the reference point P on the xy plane of the chart image CR-2 obtained at the time t3 to the correspondence on the x'y 'plane of the chart image Ci-3 obtained at the same time. The coordinates (x3 ′, y3 ′) of the point Q are input into Equation 1. By simultaneously solving the six equations obtained by these methods, a projection matrix can be calculated. Since the flying object 10 is moving in the sky, the coordinates (x1, y1) of the reference point P at time t1, the coordinates (x2, y2) of the reference point P at time t2, and the reference point at time t3. The coordinates (x3, y3) of P are different from each other. Similarly, the coordinates (x1 ′, y1 ′) of the corresponding point Q at time t1, the coordinates (x2 ′, y2 ′) of the corresponding point Q at time t2, and the coordinates (x3 ′, y3) of the corresponding point Q at time t3. ') And mutually different.

Here, the projection matrix is calculated by setting six equations by substituting the position coordinates of the three reference points P and the corresponding points Q into Equation 1, but for example, linear transformation is not necessary. In the case of a simple projection transformation (in the case of a projection transformation with only translation), the coefficients a, b, c, and d of the projection matrix are all set to 0, and the position coordinates of one reference point P and corresponding point Q By substituting into the equation (1), it is possible to determine the coefficients t _x and t _y to determine the projection matrix.

Alternatively, eight or more equations can be established by substituting the position coordinates of four or more reference points P and corresponding points Q into Equation 1. In this case, a plurality of groups in which six equations are arbitrarily selected from eight or more equations are formed, coefficients of the projection matrix are obtained for each group, and the coefficients obtained in each group are finally determined by the least squares method. May be determined. In this case, each coefficient, that is, a projection matrix can be obtained with high accuracy.

After calculating the projection matrix as described above, the parallax correction unit 11c converts the position coordinates of all points included in one field image (for example, a visible image) by the projection matrix obtained in S13 (S14). Thereby, the positional deviation of the pixels indicating the same point, which is caused by the parallax between one field image and the other field image (for example, a near-infrared image), is corrected. That is, each pixel of one field image is associated with a pixel representing the same point in the field FD in the other field image.

〔effect〕
As described above, in the present embodiment, in the field image acquiring step, the field FD is photographed from the sky by the first imaging unit 21 and the second imaging unit 22 during the flight of the flying object 10, and the wavelength band of the field FD is adjusted. A different field image (visible image, near-infrared image) is acquired (S1-1). In the chart image acquisition step, the chart C is photographed from above by the first imaging unit 21 and the second imaging unit 22 during the flight of the flying object 10 to acquire each chart image (S1-2). Then, in the parallax correction step, the parallax correction unit 11c corrects a positional shift of a pixel indicating the same point caused by parallax between each field image based on each chart image (S3). Thereby, even if parallax is generated between the field images due to the vibration of the flying object 10 or the like, it is possible to obtain each field image in which the positional shift of the pixel due to the parallax is corrected, and to reduce the influence of the parallax. It becomes possible to reduce.

Further, since the chart C is located in the field FD or around the field FD, the first imaging unit 21 and the second imaging unit 22 perform the chart C simultaneously with the field FD during the flight of the flying object 10. Can be taken. Thus, the parallax correction unit 11c can perform parallax correction based on the chart image at the same time as the image of the field FD. Therefore, for example, the efficiency of processing can be improved as compared with a case where parallax correction is performed at a timing different from the imaging of the field FD. That is, the first image capturing unit 21 and the second image capturing unit 22 simultaneously perform the field image obtaining step (S1-1) and the chart image obtaining step (S1-2), thereby improving the processing efficiency. be able to. Further, since the field image and the chart image can be obtained simultaneously by one photographing (since one photographed image can serve as the field image and the chart image), an increase in the number of photographing steps can be avoided.

中 In addition, the above-mentioned flight of the flying object 10 includes the rising and falling of the flying object 10. Therefore, not only while the flying object 10 is flying over the field FD at a predetermined altitude, but also while the flying object 10 is rising or descending, the first imaging unit 21 and the second imaging unit 22 Since a chart image can be obtained, the parallax can be corrected by the parallax correction unit 11c even while the flying object 10 is rising or falling.

{Circle around (2)} When the object OB is located in or around the field FD, the chart C can be positioned using the object OB by attaching the chart C to the object OB. That is, in this case, the object OB can be effectively used in setting the chart C.

In particular, the take-off and landing platform 40 of the flying object 10, the transport vehicle 41 for transporting the flying object 10, and the helmet 42 worn by a pilot who remotely controls the flight of the flying object 10 are used when the field FD is photographed using the flying object 10. Is always required. Therefore, by attaching the chart C to any one of these objects OB (the take-off and landing platform 40, the transport vehicle 41, and the helmet 42), it is not necessary to newly search for an installation place of the chart C or separately provide the chart C. It is possible to quickly install the object on the object OB and quickly start photographing the field.

The chart C may be directly mounted on the head of the pilot who remotely controls the flight of the flying object 10 without using the helmet 42. From this, it can be said that the chart C may be worn on the head of the pilot who remotely controls the flight of the flying object 10 regardless of whether the pilot wears the helmet 42 or not.

The above-described incident light sensor, generator, and RTK-GPS base station may be required when photographing the field FD using the flying object 10 (for example, when the weather fluctuates drastically, the It is very large, and the battery 17 needs to be charged during photography, or the location information acquisition unit 14 needs to communicate with the base station to acquire location information. Further, the field server, the agricultural machine, and the irrigation equipment described above are indispensable for management of the field FD, and are always present in the field FD or around the field FD. Therefore, the chart C may be attached to these objects OB. Even in this case, the chart C is quickly installed on the object OB while effectively utilizing the object OB, and the photographing of the field FD is started quickly. Can be.

Also, the chart C is, a pattern including at least one circular portion C ₁ or when it has a grid pattern, taking the chart C from above of the field FD by the first imaging unit 21 and the second imaging unit 22, Then, when each chart image is obtained, the image of the chart C (including the image C ₁ ′) clearly appears in each chart image. Accordingly, the parallax correction unit 11c can reliably obtain the position of the image of the chart C based on the pixel value, and can acquire the position information accurately and reliably. It can be performed with high accuracy.

In addition, the lacking portion N of the crop PL located in the field FD and the reflection spot S of the sunlight reflected on the water surface in the field FD simultaneously cause the field FD to be scanned by the first imaging unit 21 and the second imaging unit 22. When a photograph is taken, it appears in each photographed image as a position indicating the same point in the field FD. For this reason, even if the lacking portion N or the reflected spot S of sunlight is used as the chart C, the first imaging unit 21 and the second imaging unit 22 acquire each chart image and the parallax correction unit 11c. Can perform parallax correction based on each chart image. Therefore, when performing parallax correction, it is not necessary to attach the chart C to the object OB in or around the field FD. Further, even when such an object OB does not exist, the parallax correction can be performed by utilizing (effectively utilizing) the above-described missing portion N or the reflected spot S of sunlight as the chart C.

In the parallax correction step, the parallax correction unit 11c is included in one chart image based on each chart image captured and acquired by the first imaging unit 21 and the second imaging unit 22 at the same time. By calculating a projection matrix for converting the position coordinates of the reference point P to the position coordinates of the corresponding corresponding point Q in the other chart image, and converting the position coordinates of all points included in one field image by the projection matrix Then, the parallax is corrected (S13, S14). Accordingly, even if the internal state or the external state of the first imaging unit 21 and the second imaging unit 22 is changed due to vibration of the flying object 10 during flight (for example, optical axis deviation, mounting direction). ), It is possible to associate each pixel of one field image (visible image) with a pixel representing the same point in the field FD in the other field image (near infrared image).

In addition, the parallax correction unit 11c calculates a chart from a plurality of captured images acquired at different timings when the flying object 10 is moving over the sky in each of the first imaging unit 21 and the second imaging unit 22. After extracting an image including the image of C as a chart image, the center position of the image of the chart C is obtained from the extracted chart image, and the center position of the image of the chart C in one of the chart images photographed at the same time is used as a reference. A projection matrix is calculated as a point P, and the center position of the image of the chart C in the other chart image captured at the same time is set as the corresponding point Q (S11 to S13). As described above, in each chart image captured at the same time, the projection matrix required for parallax correction is appropriately calculated by calculating the projection matrix using the center position of the image of the chart C as the reference point P and the corresponding point Q, respectively. Can be obtained.

In the growth index calculation step, the growth index calculation unit 11d calculates a growth index (for example, an NDVI value) of the crop PL based on each field image after the positional displacement has been corrected by the parallax correction unit 11c (S6). ). As in the present embodiment, even when the first imaging unit 21 and the second imaging unit 22 are mounted on the flying object 10 to fly for the purpose of acquiring the NDVI value, the parallax correction unit 11c described above performs parallax correction. Since the influence can be reduced, it is possible to effectively prevent fluctuation of the NDVI value due to parallax. Therefore, the growth index calculation unit 11d can acquire an accurate NDVI value at an arbitrary point in the field FD, and can avoid deterioration in the image quality of the NDVI image. Further, even when each optical system included in the first imaging unit 21 and the second imaging unit 22 is contaminated by the attachment of dust and the pixel value of each field image includes noise, as in the present embodiment, By correcting the positional shift of the pixel due to the parallax, it is possible to suppress the fluctuation of the NDVI value due to the noise at an arbitrary point in the field FD as compared with a case where such correction is not performed. .

{Circle around (5)} In the warning step, the warning unit 13 issues a warning when the amount of change in the positional deviation during a predetermined period exceeds a threshold value (S5). In S4, when the amount of change in the position shift exceeds the threshold, the change in the position shift is caused not by vibration of the flying object 10 but by the first image pickup unit 21 and the second image pickup unit 22. Most likely due to failure. Therefore, the warning in S5 promptly prompts the user to take necessary measures such as inspection and repair, and enables the user to quickly deal with a failure. In particular, the warning unit 13 includes at least one of the display unit 13a, the audio output unit 13b, and the communication unit 12, and in S5, at least one of the warning display, the output of the warning sound, and the transmission of the warning information is performed. It is possible to reliably notify the user that an abnormality such as a failure has occurred, and to prompt the user to take necessary measures such as inspection.

[Others]
In the parallax correction (calibration), it is automatically determined whether or not there are three or more images with different chart positions. If the number of images with different chart positions is less than three, a warning is issued. If there are three or more cards, it may be notified that they are appropriate.

If an image for calibration is not found at the end of shooting on a single day, a warning may be issued.

The field imaging system has a calibration mode, and in the calibration mode, the flying object 10 may be made to automatically fly along a predetermined route, or may be provided with a program for executing such an automatic flight.

The field photographing system and the field photographing method of the present embodiment described above can be expressed as follows.

The field imaging system of the present embodiment includes a plurality of flying objects, which are held by the flying objects, and acquire a field image having a different wavelength band by photographing a field where a crop is grown from above in the air while the flying object is growing. An imaging unit, a chart captured by the plurality of imaging units, and, during the flight of the flying object, based on each chart image obtained by capturing the chart from above by the plurality of imaging units, A parallax correction unit configured to correct a positional shift of a pixel indicating the same point caused by parallax between field images, wherein the chart is located in the field or around the field.

In the above-mentioned field imaging system, the chart may be attached to an object located in the field or around the field.

In the above-mentioned field imaging system, the object may be a take-off and landing platform of the flying object.

In the above-mentioned field imaging system, the object may be a transport vehicle that transports the flying object.

In the above field imaging system, the chart may be mounted on a head of a pilot who remotely controls the flight of the flying object.

In the above-described field photographing system, the chart may have a pattern including at least one circular portion or a lattice pattern.

In the above-mentioned field photographing system, the chart may be a missing part of the crop located in the field, or a reflection spot of sunlight reflected on the water surface in the field.

In the above-described field photographing system, the parallax correction unit determines the position coordinates of a reference point included in one of the chart images based on the respective chart images acquired and photographed at the same time by the plurality of imaging units. By calculating a projection matrix to be converted into the position coordinates of the corresponding point in the chart image, by correcting the position coordinates of all points included in one field image by the projection matrix, the position shift is corrected. Is also good.

In the above-described field photographing system, the parallax correction unit may include, in each of the plurality of imaging units, an image of the chart from a plurality of photographed images acquired at different timings when the flying object is moving in the sky. Is extracted as the chart image, the center position of the image of the chart is obtained from the extracted chart image, and the center position of the one chart image taken at the same time as the reference point, The projection matrix may be calculated using the center position of the other chart image captured at the same time as the corresponding point.

The above-mentioned field photographing system may further include a growth index calculation unit that calculates a growth index of the crop based on each field image after the position shift has been corrected by the parallax correction unit.

The field imaging system described above may further include a warning unit that issues a warning when the amount of change in the positional deviation during a predetermined period exceeds a threshold.

In the above-described field photographing system, the warning unit may include at least one of a display unit for displaying a warning, a sound output unit for generating a warning sound, and a communication unit for transmitting warning information to the outside.

The field photographing method according to the present embodiment includes a plurality of image pickup units that are held by a flying object, and that during the flight of the flying object, a field where crops are grown is photographed from above to acquire field images having different wavelength bands. An image obtaining step, and a chart image obtaining step of, during the flight of the flying object, capturing a chart located in the field or around the field from above by the plurality of imaging units to obtain each chart image. And a parallax correction step of correcting, based on each of the chart images, a positional shift of a pixel indicating the same point caused by parallax between each of the field images.

In the above-described field photographing method, it is preferable that the plurality of imaging units simultaneously perform the field image obtaining step and the chart image obtaining step.

In the above-described field photographing method, the flying of the flying object may include rising and falling of the flying object.

In the above-described field photographing method, in the parallax correction step, the position coordinates of a reference point included in one of the chart images is determined based on the respective chart images acquired and photographed at the same time by the plurality of imaging units. By calculating a projection matrix to be converted to the position coordinates of the corresponding point in the chart image, and by converting the position coordinates of all the points included in one field image by the projection matrix, the position shift can be corrected. Good.

In the above-described field photographing method, in the parallax correction step, in each of the plurality of imaging units, an image of the chart is obtained from a plurality of photographed images acquired at different timings when the flying object is moving in the sky. Is extracted as the chart image, the center position of the image of the chart is obtained from the extracted chart image, and the center position of the one chart image taken at the same time as the reference point, The projection matrix may be calculated using the center position of the other chart image captured at the same time as the corresponding point.

The above-mentioned field photographing method may further include a growth index calculating step of calculating a growth index of the crop based on each field image after the positional displacement has been corrected by the parallax correction step.

The above-mentioned field photographing method may further include a warning step of issuing a warning when a change amount of the positional deviation during a predetermined period exceeds a threshold value.

In the field photographing method described above, the warning step may include at least one of displaying a warning, outputting a warning sound, and transmitting warning information to the outside.

Although the embodiments of the present invention have been described above, the scope of the present invention is not limited to the embodiments, and can be extended or modified without departing from the gist of the invention.

INDUSTRIAL APPLICABILITY The present invention can be used for a system that acquires a field image having a different wavelength band by photographing a field from above with a plurality of imaging units held by a flying object, and calculates a growth index based on each acquired field image. is there.

DESCRIPTION OF SYMBOLS 1 Field imaging system 10 Flying object 11c Parallax correction part 11d Growth index calculation part 12 Communication part (warning part)
13 warning unit 13a display portion 13b audio output unit 21 first imaging unit 22 and the second imaging section 40 landing platform 41 transportation vehicles 42 of the helmet C Chart C ₁ circular part C ₂ circular portion FD field P reference point Q corresponding point OB Object N Missing part S Reflection spot

Claims

Flying objects,
A plurality of imaging units that are held by the flying object, and during the flight of the flying object, acquire a field image having a different wavelength band by photographing a field where a crop grows from above,
A chart taken by the plurality of imaging units;
During the flight of the flying object, based on each chart image obtained by photographing the chart from above by the plurality of imaging units, a positional shift of a pixel indicating the same point caused by parallax between the respective field images. And a parallax correction unit that corrects
The field photographing system, wherein the chart is located in the field or around the field.
The field imaging system according to claim 1, wherein the chart is attached to an object located in or around the field.
The field imaging system according to claim 2, wherein the object is a take-off and landing platform of the flying object.
The field imaging system according to claim 2, wherein the object is a transport vehicle that transports the flying object.
The field imaging system according to claim 1, wherein the chart is mounted on a head of a pilot who remotely controls the flight of the flying object.
The field imaging system according to any one of claims 1 to 5, wherein the chart has a pattern including at least one circular portion or a lattice pattern.
2. The field photographing system according to claim 1, wherein the chart is a missing portion of the crop located in the field or a reflection spot of sunlight reflected on a water surface in the field. 3.
The parallax correction unit is configured to correspond to position coordinates of a reference point included in one of the chart images in the other chart image based on each of the chart images captured and acquired at the same time by the plurality of imaging units. 8. The method according to claim 1, further comprising: calculating a projection matrix to be converted into position coordinates of a point; and correcting the position shift by converting the position coordinates of all points included in one field image by the projection matrix. 9. A field imaging system according to any of the claims.
The parallax correction unit, in each of the plurality of imaging units, from a plurality of captured images obtained at different timings when the flying object is moving in the sky, the chart including the image of the chart image is included in the chart After being extracted as an image, the center position of the chart image is obtained from the extracted chart image, and the center position of the one chart image taken at the same time as the reference point is used as the reference point. The field imaging system according to claim 8, wherein the projection matrix is calculated using the center position of the other chart image as the corresponding point.
The field imaging system according to any one of claims 1 to 9, further comprising a growth index calculation unit that calculates a growth index of the crop based on each field image after the positional shift has been corrected by the parallax correction unit. .
11. The field photographing system according to claim 1, further comprising: a warning unit that issues a warning when the amount of change in the position shift during a predetermined period exceeds a threshold.
12. The field photographing system according to claim 11, wherein the warning unit includes at least one of a display unit that displays a warning, a voice output unit that emits a warning sound, and a communication unit that transmits warning information to the outside.
By a plurality of imaging units held by the flying body, during the flight of the flying body, a field image acquisition step of capturing a field where a crop grows from above and acquiring a field image with a different wavelength band,
A chart located in the field or around the field, during the flight of the flying object, a chart image obtaining step of obtaining each chart image by shooting from the sky by the plurality of imaging units,
A parallax correction step of correcting a positional shift of a pixel indicating the same point caused by parallax between the respective field images based on the respective chart images.
The field imaging method according to claim 13, wherein the field image obtaining step and the chart image obtaining step are performed simultaneously by the plurality of imaging units.
The field imaging method according to claim 13 or 14, wherein the flying of the flying object includes ascending and descending of the flying object.
In the parallax correction step, based on each of the chart images captured and acquired at the same time by the plurality of imaging units, a point corresponding to a position coordinate of a reference point included in one of the chart images in the other chart image. The position deviation is corrected by calculating a projection matrix to be converted to the position coordinates of the field image, and converting the position coordinates of all the points included in one field image by the projection matrix. 2. The field photographing method according to item 1.
In the parallax correction step, in each of the plurality of imaging units, from the plurality of captured images obtained at different timings when the flying object is moving in the sky, the image including the image of the chart, the chart After being extracted as an image, the center position of the chart image is obtained from the extracted chart image, and the center position of the one chart image taken at the same time as the reference point is used as the reference point. 17. The field photographing method according to claim 16, wherein the projection matrix is calculated using the center position of the other chart image as the corresponding point.
The field photographing method according to any one of claims 13 to 17, further comprising a growth index calculating step of calculating a growth index of the crop based on each field image after correcting the position shift by the parallax correction step. .
19. The field photographing method according to claim 13, further comprising a warning step of issuing a warning when the amount of change in the positional deviation during a predetermined period exceeds a threshold value.
20. The field photographing method according to claim 19, wherein in the warning step, at least one of displaying a warning, outputting a warning sound, and transmitting warning information to the outside is performed.