JP6887142B2 - Field image analysis method - Google Patents

Description

The present invention relates to a processing method and a program for analyzing the growth condition of a field image, particularly a field image taken by a drone.

As an important field of crop growth analysis, a method of measuring nitrogen absorption into leaves is known in order to measure the growth rate of crops such as rice and wheat. Nitrogen absorption has a great effect on crop yield and taste. If the amount of nitrogen absorbed exceeds the appropriate amount, it causes deterioration of taste and environmental pollution, and if it is less than the appropriate amount, it causes a decrease in yield. large.

Utilizing the fact that the density of leaf chlorophyll (chlorophyll a, chlorophyll b, carotenoid, etc.) changes depending on the amount of nitrogen absorbed, the density of chlorophyll is estimated by analyzing the characteristics of transmitted light in the leaves, and nitrogen to the leaves is estimated. A method of estimating the absorption amount and measuring the growth degree of the crop as a result is known (for example, Patent Document 1). However, this method has a problem that it is inefficient especially when a large number of data samples are obtained because it is necessary to actually go to the field and read the leaves of the crop into a measuring instrument for measurement (typically). It took about 1 hour per field).

In order to improve the efficiency of the above measurements, the amount of nitrogen absorbed is determined by photographing the crops in the field from artificial satellites, drones, or cameras installed in the sky, and especially by analyzing the spectrum in the near infrared region. A method for estimating has also been proposed (for example, Patent Document 2), but there is a problem in terms of accuracy. Reasons for low accuracy include a mixture of water surface images and crop images, a mixture of transmitted light and reflected light, a mixture of primary reflected light and secondary and tertiary reflected light, and a mixture of surface reflection and internal reflection.

Patent Publication Japanese Patent Application Laid-Open No. 2011-38879 Patent Publication Japanese Patent Application Laid-Open No. 2003-339238

Provided are field image analysis methods and programs that enable highly accurate growth analysis of crops.

The present invention corrects the image data according to the acquisition step of acquiring the image data of the field photographed by the camera mounted on the drone and the relative position of the sun and the camera or the state of sunlight at the time of photographing by the camera. The above problem is solved by providing a field image analysis method including a correction step to be performed and an analysis step for analyzing the growth state of crops in the field based on the corrected image data.

The present invention also provides the field image analysis method according to paragraph 0007, wherein in the correction step, the amount of light of the image data is corrected according to the relative angle between the irradiation direction of sunlight and the shooting direction of the camera. This solves the above problem.

Further, according to the present invention, in the correction step, the image data is acquired by shooting the camera toward the front light side with respect to the sun, or by shooting the image data toward the back light side. The above problem is solved by providing the field image analysis method according to paragraph 0007, which changes the correction amount of the image data used for the analysis in the analysis step based on the backlit side image data.

Further, according to the present invention, in the correction step, the forward light side image data or the back light obtained by shooting the camera toward the forward light side with respect to the sun from the image data used for the analysis in the analysis step. The above problem is solved by providing the field image analysis method according to paragraph 0007, which excludes the backlit side image data acquired by being photographed toward the side.

In the correction step, it is determined in paragraph 0008 whether the image data is the forward light side image data or the back light side image data based on the position and time of the drone when the image data is taken. The above problem is solved by providing the described field image analysis method.

Further, according to the present invention, the field image analysis method according to paragraph 0007, wherein in the correction step, the correction content of the image data is changed according to the ratio of the direct light directly incident on the camera directly from the sun to the total incident light. The above problem is solved by providing.

Further, the present invention solves the above problem by providing the field image analysis method described in paragraph 0012, which estimates the ratio of the direct light according to the amount of light of the image data in the correction step.

The present invention also provides the field image analysis method according to paragraph 0012, wherein in the correction step, the ratio of direct light to the incident light is estimated based on the magnitude of the contrast of the shadow of the object in the image data. By doing so, the above problem is solved.

Further, in the correction step of the present invention, the field image according to paragraph 0012, in which the ratio of the direct light to the total incident light incident on the camera at the time of shooting is estimated based on the image of the camera that shoots the sky direction. The above problem is solved by providing an analysis method.

Further, according to the present invention, in the correction step, the altitude of the sun at the time of photographing is estimated based on the position of the shadow of the drone in the field image, and the image data is corrected according to the estimated altitude of the sun. The above problem is solved by providing the field image analysis method according to 0007.

Further, the present invention solves the above problem by providing the field image analysis method according to paragraph 0016, which measures the position of the shadow of the drone at the time of turning in the correction step.

The present invention also solves the above problem by providing the field image analysis method according to paragraph 0007, which includes a drug amount adjusting step for adjusting the amount of the drug sprayed on the drone based on the analysis result of the image data. Solve.

The invention of the present application also comprises the field image analysis method according to paragraph 0018, which includes a position adjustment step of adjusting the position or direction of the camera or the position or direction of the chemical spray nozzle according to the flight degree of the drone. By providing it, the above problems will be solved.

The present invention also provides the field image analysis method according to paragraphs 0007 to 0019, wherein the analysis step includes a step of excluding a region in which the reflectance of near infrared rays is equal to or higher than a predetermined value from the image data. This solves the above problem.

Further, the present invention relates to an image data acquisition unit that acquires image data of a field photographed by a camera mounted on a drone, and the sun and the relative position of the camera or the state of sunlight at the time of photographing by the camera. The above problem is solved by providing a field image analysis system having an image data correction unit for correcting image data and an image analysis unit for analyzing the growth state of crops in the field based on the corrected image data. ..

Further, according to the present invention, the field image analysis system according to paragraph 0021, wherein the image data correction unit corrects the amount of light of the image data according to a relative angle between the irradiation direction of sunlight and the shooting direction of the camera. By providing it, the above problems will be solved.

Further, in the present invention, the image data correction unit uses the acquired forward light side image data in which the image data is taken with the camera facing the sun, or the back light side in which the image data is taken toward the back light side. The above problem is solved by providing the field image analysis system according to paragraph 0021, which changes the correction amount of the image data used for the analysis in the image analysis unit based on the image data.

Further, in the present invention, the image data correction unit obtains forward light side image data obtained by shooting the camera toward the forward light side with respect to the sun from the image data used for analysis by the image analysis unit. Alternatively, the above problem is solved by providing the field image analysis system according to paragraph 0021, which excludes the backlit side image data acquired by being photographed toward the backlit side.

The image data correction unit determines whether the image data is the forward light side image data or the back light side image data based on the position and time of the drone when the image data is taken. The above problem is solved by providing the field image analysis system according to 0023.

The field image analysis system according to claim 15, wherein the image data correction unit changes the correction content of the image data according to the ratio of the direct light directly incident on the camera directly from the sun to the total incident light. This solves the above problem.

The field image according to claim 20, wherein the image data correction unit estimates the ratio of direct light to total incident light based on the magnitude of the contrast of the shadow of an object in the image data. The above problem is solved by providing an analysis system.

The field according to claim 20, wherein the image data correction unit estimates the ratio of the direct light of the image taken by the camera at the time of shooting based on the image of the camera that shoots the sky direction. The above problem is solved by providing an image analysis system.

Further, in the present invention, the image data correction unit estimates the altitude of the sun at the time of shooting based on the position of the shadow of the drone in the field image, and corrects the image data according to the estimated altitude of the sun. The above problem is solved by providing the field image analysis system according to claim 15.

The present invention also solves the above problem by providing the field image analysis system according to claim 23, wherein the image data correction unit measures the position of the shadow of the drone at the time of turning.

The present invention also provides the field image analysis system according to claim 15, which includes a drug amount adjusting unit for adjusting the amount of the drug sprayed on the drone based on the analysis result of the image data. To solve.

The field image analysis according to claim 25, wherein the present invention includes a nozzle position adjusting unit that adjusts the position or direction of the camera or the position or direction of the chemical spray nozzle according to the flight degree of the drone. The above problem is solved by providing the system.

Further, in the present invention, the image analysis unit includes a group of instructions for excluding a region in which the reflectance of near infrared rays is equal to or higher than a predetermined value from the image data, according to any one of claims 15 to 26. The above problem is solved by providing the described field image analysis system.

The invention of the present application also claims 15 to 26, wherein the image data correction unit and the image analysis unit are implemented in the drone, a pilot that inputs flight information to the drone, or another external server. The above problem is solved by providing the field image analysis system according to any one of the above.

Field image analysis methods and programs that enable highly accurate crop growth analysis are provided.

It is a top view of the Example of the drone for field photography which concerns on this invention. It is a front view of the Example of the drone for field photography which concerns on this invention. It is a right side view, the back view, and the perspective view of the Example of the drone for field photography which concerns on this invention. This is an example of an overall conceptual diagram of a field photography system using an example of a field photography drone according to the present invention. It is a schematic diagram which showed the control function of the Example of the drone for field photography which concerns on this invention. This is an example of an outline flowchart of field image analysis according to the present invention. It is a schematic diagram which showed the relationship between the reflected light and the transmitted light in a field photograph. It is a schematic diagram which showed the example of the brightness correction based on the direction of the sun in the field photography.

FIG. 1 is a plan view of an embodiment of the drone (100) according to the present invention, FIG. 2 is a bottom view thereof, FIG. 3-a is a front view thereof (viewed from the traveling direction side), and FIG. 3-b is a top view thereof. A perspective view is shown in the right side view and FIG. 3-c. In the specification of the present application, the drone may be a power means (electric power, motor, etc.) and a maneuvering method (wireless or wired, autonomous flight type, manual maneuvering type, etc.). However, it shall refer to all aircraft having multiple rotors or means of flight.

Rotors (101-1a, 101-1b, 101-2a, 101-2b, 101-3a, 101-3b, 101-4a, 101-4b) (also known as rotors) are used to fly the drone (100). It is desirable to have eight aircraft (four sets of two-stage rotor blades) in consideration of the balance between flight stability, aircraft size, and battery consumption.

Motors (102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 102-4a, 102-4b) have rotary blades (101-1a, 101-1b, 101-2a). , 101-2b, 101-3a, 101-3b, 101-4a, 101-4b), which is a means (typically an electric motor, but may be a motor, etc.) for one rotary blade. It is desirable that one machine is installed. The upper and lower rotors (eg, 101-1a and 101-1b) in one set, and their corresponding motors (eg, 102-1a and 102-1b), are used for drone flight stability, etc. It is desirable that the axes are on the same straight line and rotate in opposite directions.

The drug nozzles (103-1, 103-2, 103-3, 103-4) are means for spraying the drug downward, and it is desirable that four drug nozzles are provided. The medicine tank (104) is a tank for storing the medicine to be sprayed, and it is desirable that the medicine tank (104) is provided at a position close to the center of gravity of the drone (100) and at a position lower than the center of gravity from the viewpoint of weight balance. The drug hose (105-1, 105-2, 1053, 105-4) connects the drug tank (104) and each drug nozzle (103-1, 103-2, 103-3, 103-4). It may be made of a hard material and also serve to support the drug nozzle. The pump (106) is a means for discharging the drug from the nozzle.

The field photographing camera (107) is a camera for photographing a field (agricultural land) from a drone (100), and it is desirable that the field photographing camera (107) is installed so as to photograph a field behind the traveling direction of the aircraft. Further, it is desirable that the angle between the field photographing camera (107) and the machine body is variable by means such as a stepping motor.

FIG. 4 shows an overall conceptual diagram of an embodiment of a field photography system using the drone (100) according to the present invention. This figure is a schematic view, and the scale is not accurate. The pilot (401) sends a command to the drone (100) by the operation of the user (402), and the information received from the drone (100) (for example, position, amount of drug, remaining battery level, field photography camera). It is a means for displaying an image, etc.), and may be realized by a portable information device such as a general tablet terminal that runs a computer program. The drone (100) according to the present invention is preferably controlled to perform autonomous flight, but it is desirable that it can be manually operated during basic operations such as takeoff and return, and in an emergency. It is desirable that the pilot (401) and drone (100) perform wireless communication via Wi-Fi or the like.

The field (403) is a rice field, a field, or the like to be photographed by the drone (100). In reality, the topography of the field (403) is complicated, and the topographic map may not be available in advance, or the topographic map and the situation at the site may be inconsistent. Normally, the field (403) is adjacent to houses, hospitals, schools, other crop fields, roads, railroads, etc. In addition, obstacles such as buildings and electric wires may exist in the field (403).

The base station (404) is a device that provides a master unit function for Wi-Fi communication, and it is desirable that it also functions as an RTK-GPS base station so that it can provide the accurate position of the drone (100). (The base unit function of Wi-Fi communication and the RTK-GPS base station may be independent devices). The farming cloud (405) is typically a group of computers and related software operated on a cloud service, and it is desirable that the farming cloud (405) is wirelessly connected to the controller (401) by a mobile phone line or the like. The farming cloud (405) may analyze the image of the field (403) taken by the drone (100), grasp the growing condition of the crop, and perform a process for determining the flight route. In addition, the topographical information of the stored field (403) may be provided to the drone (100). In addition, the history of the flight and captured images of the drone (100) may be accumulated and various analysis processes may be performed.

Normally, the drone (100) takes off from the departure / arrival point (406) outside the field (403), sprays the chemicals on the field (403), or when it becomes necessary to replenish or charge the chemicals. Return to (406). The flight route (invasion route) from the departure / arrival point (406) to the target field (403) may be stored in advance in a farming cloud (405) or the like, or before the user (402) starts takeoff. You may enter in.

FIG. 5 shows a schematic diagram showing a control function of an embodiment of the drug spraying drone according to the present invention. The flight controller (501) is a component that controls the entire drone, and may be an embedded computer including a CPU, memory, related software, and the like. The flight controller (501) is based on the input information received from the pilot (401) and the input information obtained from various sensors described later, via a control means such as ESC (Electronic Speed Control), and the motor (102-102-). The flight of the drone (100) is controlled by controlling the number of revolutions of 1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 104-a, 104-b). The actual rotation speed of the motors (102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 104-a, 104-b) is fed back to the flight controller (501) and is normal. It is configured so that it can monitor whether or not the rotation is being performed. Alternatively, the rotary blade (101) may be provided with an optical sensor or the like so that the rotation of the rotary blade (101) is fed back to the flight controller (501).

It is desirable that the software used by the flight controller (501) be rewritable through a storage medium or the like for function expansion / change, problem correction, etc., or through communication means such as Wi-Fi communication or USB. Also, even if some of the calculations used by the flight controller (501) for control are performed by another computer located on the pilot (401), on the farming support cloud (405), or elsewhere. Good. Due to the high importance of the flight controller (501), some or all of its components may be duplicated.

The battery (502) is a means of powering the flight controller (501) and other components of the drone, eg, rechargeable. The battery (502) is connected to the flight controller (501) via a fuse or a power supply unit including a circuit breaker or the like. The battery (502) may be a smart battery having a function of transmitting its internal state (storage amount, total usage time, etc.) to the flight controller (501) in addition to the power supply function.

The flight controller (501) communicates with the pilot (401) via the Wi-Fi slave unit function (503) and further via the base station (404), and issues necessary commands from the pilot (401). At the same time as receiving, the necessary information can be transmitted to the controller (401). The base station (404) may also have an RTK-GPS base station function in addition to the Wi-Fi communication function. By combining the signal from the RTK base station and the signal from the GPS positioning satellite, the GPS module (504) can measure the absolute position of the drone (100) with an accuracy of several centimeters. Since the GPS module (504) is so important, it is duplicated and multiplexed, and each redundant GPS module (504) uses a different satellite to cope with the failure of a specific GPS satellite. Is controlled.

The 6-axis gyro sensor (505) is a means for measuring the acceleration of the drone body in three directions orthogonal to each other (further, a means for calculating the velocity by integrating the acceleration). Further, the 6-axis gyro sensor (505) is a means for measuring the change in the attitude angle of the drone aircraft in the above-mentioned three directions, that is, the angular velocity. The geomagnetic sensor (506) is a means for measuring the direction of the drone body by measuring the geomagnetism. The barometric pressure sensor (507) is a means for measuring barometric pressure, and can also indirectly measure the altitude of the drone. The laser sensor (508) is a means for measuring the distance between the drone body and the ground surface by utilizing the reflection of the laser light, and for example, an IR (infrared) laser may be used. Sonar (509) is a means for measuring the distance between a drone aircraft and the surface of the earth by utilizing the reflection of sound waves such as ultrasonic waves. These sensors may be selected according to the cost target and performance requirements of the drone. In addition, a gyro sensor (angular velocity sensor) for measuring the inclination of the aircraft, a wind sensor for measuring wind power, and the like may be added. Further, it is desirable that these sensors are duplicated or multiplexed.

The flow rate sensor (510) is a means for measuring the flow rate of the drug, and is provided at a plurality of locations on the path from the drug tank (104) to the drug nozzle (103). The liquid drainage sensor (511) is a sensor that detects that the amount of the drug has fallen below a predetermined amount. The field photographing camera (107) is a means for photographing the field (403) and acquiring data for performing image analysis, and is preferably a multispectral camera. The obstacle detection camera (513) is a camera for detecting drone obstacles, and since the image characteristics and the orientation of the lens are different from the field photography camera (107), it is a device different from the field photography camera (106). It is desirable to have. The switch (514) is a means for the user (402) of the drone (100) to make various settings. The obstacle contact sensor (515) detects that the drone (100), in particular its rotor or propeller guard, has come into contact with an obstacle such as an electric wire, building, human body, standing tree, bird, or other drone. It is a sensor of. The cover sensor (516) is a sensor that detects that the operation panel of the drone (100) and the cover for internal maintenance are in the open state. The drug inlet sensor (517) is a sensor that detects that the inlet of the drug tank (104) is in an open state. These sensors may be selected according to the cost target and performance requirements of the drone, and may be duplicated or multiplexed.

The flight controller (501) transmits a control signal to the pump (106) to adjust the drug discharge amount and stop the drug discharge. It is desirable that the current status of the pump (106) (for example, the number of revolutions) is fed back to the flight controller (501).

The LED (517-2) is a display means for informing the drone operator of the state of the drone. Display means such as a liquid crystal display may be used in place of or in addition to the LED. The buzzer (518) is an output means for notifying the state of the drone (particularly the error state) by an audio signal. The Wi-Fi slave unit function (519) is an optional component for communicating with an external computer or the like for transferring software, for example, apart from the control unit (401). Instead of or in addition to the Wi-Fi slave function, other wireless communication means such as infrared communication, Bluetooth (registered trademark), ZigBee (registered trademark), NFC, or wired communication means such as USB connection You may use it. The speaker (520) is an output means for notifying the state of the drone (particularly the error state) by means of recorded human voice, synthetic voice, or the like. Depending on the weather conditions, it may be difficult to see the visual display of the drone (100) in flight. In such cases, voice communication of the situation is effective. The warning light (521) is a display means such as a strobe light for notifying the state of the drone (particularly the error state). The nozzle position adjusting mechanism (522) is a means for changing the position of the chemical nozzles (103-1, 103-2, 103-3, 103-4), and may be realized by a stepping motor. These input / output means may be selected according to the cost target and performance requirements of the drone, and may be duplicated or multiplexed.

For analysis of crop growth based on field images, NDVI (Normalized Difference Vegetation Index) is calculated by acquiring images of reflected light of red light (wavelength about 680 nm) and near infrared light (wavelength about 780 nm). It is effective to do. Generally, NDVI is calculated by the formula (IR − R) / (IR + R) (where IR is the reflectance of near-infrared light and R is the reflectance of red light). IR and R can be obtained by analyzing the image of the field for each frequency band, but since the amount of light obtained by the camera is greatly affected by the ambient light at that time, accurate analysis is performed using only the amount of light data. It is not possible. It is necessary to make appropriate corrections according to the situation.

FIG. 6 shows an outline flowchart of the field image analysis according to the present invention. The field image analysis program takes as input an image of the crop taken by the field camera (107) of the drone (100) (S601). The image input may be performed by collecting the image data of the entire field that has been photographed (in this case, the image is recorded as moving image data, and the information on sunlight described later is recorded with the same time code as the moving image. However, a part of the image currently being photographed may be input by a streaming method (quasi-real-time method). Next, the correction parameters to be used are obtained according to the shooting conditions (S602). The correction information may be selected from a set of parameters obtained by a prior experiment or the like according to the environment at the time of photographing such as the position of the sun. Next, the selected parameters are used to correct the data to be analyzed. Typically, correction may be performed by multiplying a predetermined coefficient for each section (particularly red light and near-infrared light) in which the wavelength band is divided. For example, the calculation formula of NDVI is (α * IR − β). * R) / (α * IR + β * R), and correction may be performed by substituting predetermined values for α and β. As for the specific values to be substituted for α and β, the correction values suitable for various sunlight conditions are obtained by prior experiments, and the appropriate values are selected according to the sunlight conditions at the time of shooting obtained by the method described later. Although it is desirable to select it, other correction methods may be used, such as performing calculations at any time according to the sunlight conditions at the time of shooting, instead of selecting a set of correction parameters prepared in advance. Finally, analysis processing is performed on the corrected data to be analyzed (pre-processing and post-processing may be performed in addition to the above NDVI calculation) (S604). The information of the analysis processing result may be temporarily saved and used for the subsequent analysis, but the drone (100) may be able to refer to it in real time during the flight.

The above-mentioned correction parameter determination process (S602), image data correction process (S603), and analysis image target data analysis process (S604) may be performed on the farming cloud (405) or on another external server. , It may be executed by a computer mounted on the drone (100) or a computer mounted on the pilot (401) that inputs a control command to the drone via wireless communication. Computers in multiple locations may cooperate to perform processing. The analysis process may be performed ex post facto, or may be performed in real time during the flight of the drone (100). Real-time analysis has the advantage that the chemical application can be dynamically adjusted according to the growing conditions of the crop.

(Transmitted light / reflected light correction)
FIG. 7 shows a schematic diagram showing the basic concept of separation of reflected light and transmitted light in the nitrogen content measuring method according to the present invention. FIG. 7-a is a view seen from the horizontal direction, and FIG. 7-b is a view seen from the sky. The drone (100), crop (701), and sun (702) are schematic representations, and their positions and sizes do not represent the actual scale. The image of the crop (701) in the field (403) taken by the camera (103) of the drone (100) is the crop (701-1) on the sun (702) side in the horizontal direction when viewed from the current position of the drone (100). ) (Backlight side) has strong transmitted light, and crops (701-2) (forward light side) on the opposite side of the sun (701) in the horizontal direction when viewed from the current position of the drone (100) have reflected light. strong.

Here, the position of the sun may be grasped by an optical sensor provided in the drone (100), or may be grasped from the time and date at the time of shooting obtained by the clock provided in the drone (100) or its control device. In addition, in the field image, there is a discontinuous difference in the brightness of the image on the sun side and the opposite side of the sun with the straight line corresponding to 703 in FIG. 7-b as the boundary. It may be grasped by processing.

It is preferable that the grasped information on the direction of the sun (702) is stored in association with the moving image of the crop (701) taken by the camera (107) and used for analysis processing. In addition, it is preferable to save the position of the drone (100) at the time of shooting measured by means such as GPS and the information of the characteristics of sunlight measured by the optical sensor in association with the moving image of the crop (701). .. The saved moving image is sent to the computer for analysis by a method such as temporarily saving it in the memory in the drone (100) and transferring it later, or transmitting it through the wireless communication function, and it is the object of analysis. Become. It is possible to measure the amount of nitrogen in a crop by filtering each still image frame in a moving image by wavelength with a computer image processing program and measuring the amount of near infrared rays. Since sufficient accuracy cannot be obtained only by measuring the amount of near infrared rays, it is possible to perform highly accurate measurement by applying the correction process.

By excluding the image of the crop (701-1) on the side of the sun (702) from the current position of the drone (100) from the target of nitrogen content measurement, the influence of transmitted light is eliminated and the reflected light center. You can take an image of the mold. The drone (100) can fly over the field (403) by human control or automatic control, resulting in the reflected light center of all crops (701) in the field (403). It is possible to obtain an image of. In the analysis of the image centered on the reflected light, it is desirable to use the correction parameters corresponding to it.

In the image acquired by this method, the light first reflected from the crop (primary reflected light) is the center, and the secondary reflected light generated by the reflected light further reflected on the crop, and the tertiary reflected light. Since the influence and the influence of the transmitted light reflected on the crop can be relatively reduced, it is possible to measure the nitrogen content with high accuracy. In addition, in the case of secondary or higher reflected light (multiple reflected light), it has been clarified by the inventor's experiment that the attenuation rate of the reflected light of red light with respect to the incident light is 50% or less. More accurate measurement of nitrogen content can be performed by measuring near-infrared light after removing the region where the red light is stronger than the threshold value from the image obtained by extracting only the light.

Conversely, by excluding the image of the crop (701-2) on the opposite side of the sun (702) from the current position of the drone (100) from the nitrogen content measurement, the effect of reflected light is eliminated. , It is possible to take a transmitted light center type image. Similarly, by flying the drone (100) over the field (403), it is possible to obtain an image of the center of transmitted light of the entire field. When analyzing an image centered on transmitted light, it is desirable to use correction parameters that match it. Depending on the type of crop and the sunlight conditions, it may be preferable to analyze the image of the transmitted light center, so the computer program for the analysis supports both the reflected light center analysis and the transmitted light center analysis. It is preferable to make it as possible.

The image acquired when shooting the crop on the backlit side has a lower brightness than the image acquired when shooting the crop on the forward light side, so the direction of sunlight and the shooting direction of the camera on the drone By adjusting the brightness of the image data according to the angle formed, both the image on the backlit side and the image on the forward light side can be used for image analysis of growth diagnosis. For example, when the angle θ formed by the shooting direction (801) of the camera on the drone and the direction of sunlight (802) is defined as shown in FIG. 8-a, the angle θ is as shown in FIG. 8-b. Corrected to increase the brightness of the image data acquired in the situation of 0 degrees and 360 degrees, and does not correct the brightness of the image data acquired at an angle θ of 180 degrees, or more than the images of 0 degrees and 360 degrees. The amount of brightness correction may be reduced. In the present embodiment, the direction of the sun can be estimated from the time and date information, and the information on the position and direction of the camera can be obtained from the flight controller (501).

In addition to the above-mentioned method for separating reflected light and transmitted light, or by performing the following image analysis method in place of the method, it is possible to measure the nitrogen absorption amount with higher accuracy.

(Spectroscopic characteristic correction)
In order to measure the nitrogen content more accurately, it is necessary to understand the characteristics of the sunlight illuminating the crop (701) at the time of shooting the field (403) with the drone (100) and calculate the correction accordingly. Is desirable. This is because the reflected and transmitted light of the crop (201) is also affected by the characteristics of sunlight, which is influenced by the weather. In particular, it is preferable to measure the ratio of direct light to sky light (light diffused by water vapor or dust in the atmosphere or reflected from clouds and reaching the ground). The characteristics of sunlight may be measured by an optical sensor provided in the drone (100) or a camera, and stored in association with an image of the field for analysis. Further, the characteristics of sunlight may be indirectly estimated by the following method and corrected accordingly. In particular, it is desirable to make corrections based on the ratio of direct light to total incident light.

The amount of direct light or the ratio of direct light to the incident light is estimated by measuring the contrast of shadows of objects containing crops in the field image (shadow density compared to other parts of the image). You may. The ratio of direct light to sky light may be estimated from the reflectance at all wavelengths of the image of the crop on the backlit side (701-1) and the image of the crop on the forward light side (701-2) divided by the above method. .. This is because it has been clarified that the difference in reflectance between the backlight side and the forward light side is large when the direct light is strong. Similarly, the characteristics of sunlight may be estimated from the difference between the spectral characteristics of the image of the crop on the backlit side (701-1) and the spectral characteristics of the image of the crop on the forward light side (701-2). The ratio of direct light may be estimated from the amount of light in the entire image.

The ratio of direct light may be estimated from the ratio of the solar altitude and the total amount of light that can be calculated from the time of day, and correction may be made accordingly. For example, if the sun is high but the total amount of light is low, it means that the clouds are thick and the proportion of direct light is low.

(Solar zenith correction)
Since the spectral characteristics (histogram of each wavelength band) are greatly affected by the direction and altitude of the sun, information on the direction and altitude of the sun grasped from the time of shooting is recorded together with the image of the field, and the correction parameters are adjusted accordingly. It is desirable to incorporate procedures such as calculating or changing the calculation formula itself into the analysis process.

As an alternative method, the position of the sun may be calculated from the altitude and nose direction of the drone (100) and the position of the shadow of the aircraft in the image taken by the field camera (107). Since the altitude of the drone (100) can be grasped by GPS (504) and various sensors, the position of the sun can be grasped by the principle of triangulation. It is desirable to measure the position of the shadow by hovering the aircraft and rotating it horizontally, preferably 360 degrees, so that the object on the field is not mistaken for the shadow of the aircraft. Rotating 360 degrees also contributes to the calibration of the geomagnetic sensor (506). As an alternative method, the position of the shadow of the drone (100) may be measured when the drone (100) changes direction. In order for the drone (100) to fly all over the field, it is necessary to change direction anyway, so adopting this method has the advantage of not wasting the flight time of the drone (100). ..

Since the position of the sun changes during the flight of the drone (100), it is desirable to measure the position of the shadow above regularly during the flight of the drone (100). The measurement may be performed at predetermined time intervals, and the shadow position may be measured at the time of hovering, which is necessary at the time of landing and at the time of takeoff.

In addition to the correction according to the sunlight condition described above, the following additional processing may be performed in the image analysis processing. The soil and water surfaces have the characteristic that the attenuation rate of the reflected light with respect to the incident light in the near infrared region is 10% or less. If the region where the near infrared rays are above the threshold value is extracted from the image of the center of reflected light obtained by the above method, it is highly likely that it corresponds to the soil and water surfaces. , The accuracy of measurement can be improved.

Further, it is possible to separate the surface reflection and the internal reflection by the threshold value of the reflected light of the red light. Since the surface reflection is not so affected by the chlorophyll content of the leaf, the chlorophyll content can be measured more accurately by removing the surface reflection portion from the image and then measuring the near-infrared ray amount.

Similarly, it is possible to separate leaf reflection and ear reflection by the threshold of reflected light of near-infrared light and red light. Since it is necessary to measure chlorophyll mainly on the leaves, it is possible to measure the chlorophyll content more accurately by removing the ear portion from the image and then measuring the amount of near infrared rays.

Similarly, the red light reflected light threshold can separate the upper leaf reflections from the lower leaf reflections of the crop, which allows the nitrogen content of the upper leaves and the nitrogen of the lower leaves. The content can be measured independently. Based on this measured value, the status of nitrogen translocation in crops can be estimated.

(Camera angle for simultaneous spraying)
With the drone (100) according to the present invention, the field is photographed by the field photographing camera (107), the photographed image as described above is analyzed, and an appropriate amount of chemicals such as fertilizer is sprayed once based on the analysis result. It may be done collectively by the flight of. As a result, the utilization rate of the drone per field area is improved, and an economic merit can be obtained. Even when performing real-time analysis, it takes some time for the analysis process, so point the field camera (107) toward the front of the drone (100) to adjust the chemical spraying or decide whether or not to apply it. It is desirable to secure the analysis time of.

In this case, the analysis time to be secured changes depending on the flight speed of the drone (100), so the angle of the field photography camera (107) can be adjusted by a stepping motor or the like and automatically adjusted according to the flight speed. desirable. That is, when the flight speed is high, the field camera (107) is oriented to shoot the front, and when the flight speed is slow, the field camera (107) is oriented to shoot the rear (a position close to directly under the aircraft). It is desirable to do. It is necessary for analysis by changing the angle adjustment of the field photography camera (107), or in addition, the position of the chemical spray nozzle (103) can be adjusted by a stepping motor or the like and automatically adjusted according to the flight speed. You may try to secure enough time. That is, when the flight speed is high, the chemical spray nozzle (103) may be positioned further rearward in the flight direction.

Conversely, the calculation time required for the analysis may be estimated, the flight speed may be determined accordingly, and the amount of drug sprayed may be adjusted accordingly.

(Technically remarkable effect of the present invention)
The nitrogen content, which is important for evaluating the growth of crops such as rice, can be measured efficiently and with high accuracy by a simple means called a drone. The efficiency is higher than when the measuring device is carried to the field and a large number of samples are measured. In addition, the accuracy is much higher than the measurement method using satellite photographs or simple images taken from the sky. In addition, image analysis takes into account environmental conditions such as the position and angle of the sun and the weather, enabling more accurate analysis.

Claims (13)

The acquisition step to acquire the image data of the field taken by the camera mounted on the drone,
A correction step of correcting the image data according to the relative position of the sun and the camera at the time of shooting by the camera or the state of sunlight, and a correction step.
Including an analysis step of analyzing the growth status of crops in the field based on the corrected image data.
In the correction step, the ratio of direct light to the incident light is estimated based on the magnitude of the contrast of the shadow of the object in the image data, and the correction content of the image data is changed according to the ratio. > Field image analysis method. The acquisition step to acquire the image data of the field taken by the camera mounted on the drone,
A correction step for correcting the image data according to the relative position of the sun and the camera at the time of shooting by the camera or the state of the sunlight, and a correction step.
Includes an analysis step to analyze the growth of crops in the field based on the corrected image data.
In the correction step, the altitude of the sun at the time of shooting is estimated based on the position of the shadow of the drone in the field image, and the image data is corrected according to the estimated altitude of the sun. Method. In the correction step, the position of the shadow of the drone is measured at the time of turning.
The field image analysis method according to claim 2. A drug amount adjustment step for adjusting the amount of drug spraying of the drone based on the analysis result of the image data is further included.
The field image analysis method according to any one of claims 1 to 3. Includes a position adjustment step that adjusts the position or orientation of the camera or the position or orientation of the drug spray nozzle according to the flight altitude of the drone.
The field image analysis method according to any one of claims 1 to 4. The analysis step includes a step of excluding a region where the near-infrared reflectance is equal to or higher than a predetermined value from the image data.
The field image analysis method according to claim 1 to 5. An image data acquisition unit that acquires image data of the field taken by the camera mounted on the drone,
The ratio of direct light to total incident light estimated based on the relative position or state of sunlight between the sun and the camera at the time of shooting by the camera, and the magnitude of the contrast of the shadow of an object in the image data. an image data correction unit which corrects the image data according to,
A field image analysis system including an image analysis unit that analyzes the growth state of crops in the field based on the corrected image data. An image data acquisition unit that acquires image data of the field taken by the camera mounted on the drone, and
The image data is obtained according to the relative position of the sun and the camera at the time of shooting by the camera or the state of sunlight, and the altitude of the sun at the time of shooting estimated based on the position of the shadow of the drone in the field image. Image data correction unit to be corrected and
A field image analysis system including an image analysis unit that analyzes the growth state of crops in the field based on the corrected image data. The field image analysis system according to claim 8, wherein the image data correction unit measures the position of the shadow of the drone at the time of turning. The field image analysis system according to any one of claims 7 to 9, further comprising a drug amount adjusting unit that adjusts the amount of the drug sprayed on the drone based on the analysis result of the image data. The aspect according to any one of claims 7 to 10 , which includes a nozzle position adjusting unit that adjusts the position or direction of the camera or the position or direction of the chemical spray nozzle according to the flight altitude of the drone. Field image analysis system. The field image analysis system according to any one of claims 7 to 11, wherein the image analysis unit executes a process of excluding a region in which the reflectance of near infrared rays is equal to or higher than a predetermined value from the image data. .. The image data correction unit and the image analysis unit are mounted on the drone, a pilot that inputs flight information to the drone, or another external server, according to any one of claims 7 to 12. The field image analysis system described.

Images