JP2022087038A - Vegetative state detection method - Google Patents

Abstract

To provide a vegetative state detection method that can detect a vegetative state under a protective net or an emergency net, by optical means, from above the protective net or the emergency net.SOLUTION: According to a vegetative state detection method, a multi-spectral camera 2 mounted on a UAV (unmanned aircraft) 1 is used to capture images in the infrared band and plural red bands from the sky over a farm, a field, a fruit tree or a mountain 102 covered with a protective net or an emergency net 101, a group of captured images is combined, and the NDVI (normalized difference vegetation index) is mapped onto the image.SELECTED DRAWING: Figure 1

Description

The present invention relates to a vegetation state detection method capable of detecting a vegetation state under a protective net or a disaster prevention net by an optical means from above the protective net or a disaster prevention net.

Conventionally, a vegetation state detection method by optical means (camera) and remote sensing has been proposed (Patent Document 1). Further, in recent years, in fields such as surveying and terrain grasping, an automatic creation method for obtaining three-dimensional point cloud data from aerial photograph groups by UAV (Unmanned Aerial Vehicle) has been proposed.

With such advances in UAV technology, it has become possible to appropriately monitor large-area farms and fields, and by detecting the vegetation state, it is possible to handle harvesting and spraying pesticides in the right place. It has become.

Re-table 2019-017095

In the conventional vegetation state detection method, when a protective net (net) or a disaster prevention net (net) is applied to a farm or a field, the vegetation state cannot be detected with sufficient accuracy. This does not mean that the grid part of the net is not visible as a shield (shadow) and only the open part is visible, but that the reflection and scattering of light in the net reduces the proportion of transmitted light information. It is believed that there is.

The present invention has been made in view of such conventional circumstances, and a vegetation state detection method capable of detecting a vegetation state under a protective net or a disaster prevention net by an optical means from above the protective net or a disaster prevention net. The challenge is to provide.

Other problems of the present invention will be clarified by the following description.

The above problem is solved by the following invention.

(Claim 1)
A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures images of multiple bands of infrared and red from above farms, fields, fruit trees or mountains covered with protective or disaster prevention nets.
A vegetation state detection method characterized by integrating the photographed images and mapping an NDVI (normalized difference vegetation index) onto the images.
(Claim 2)
A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures images of multiple bands of infrared and blue from above farms, fields, fruit trees or mountains covered with protective or disaster prevention nets.
A method for detecting a vegetation state, which comprises integrating the photographed images and mapping HB-NDVI (blue vegetation index) onto the images.
(Claim 3)
The vegetation according to claim 1 or 2, wherein the protective net or the disaster prevention net is a net made of synthetic resin fiber stranded wire having a thickness of 1 mm to 5 mm, and the pitch of the net is 2 mm to 50 mm. State detection method.
(Claim 4)
The UAV is characterized in that it detects its own position by GNSS positioning and executes automatic flight and automatic shooting based on its own position, map information prepared in advance, a planned flight area, and a planned shooting position. The vegetation state detection method according to claim 1, 2 or 3.
(Claim 5)
A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures multiple bands of infrared and red from the side of a farm, field, fruit tree or mountain field covered with protective or disaster prevention nets.
A vegetation state detection method characterized in that a 3D shape is constructed from the captured image group, the image group is integrated, and an NDVI (normalized difference vegetation index) is mapped on the image.
(Claim 6)
A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures multiple bands of infrared and blue from the side of a farm, field, fruit tree or mountain field covered with protective or disaster prevention nets.
A vegetation state detection method characterized in that a 3D shape is constructed from the captured image group, the image group is integrated, and HB-NDVI (blue vegetation index) is mapped on the image.
(Claim 7)
The UAV controls the route to avoid fruit trees by image processing using AI that deeply learns the shape of fruit trees, etc., and automatically based on the map information prepared in advance, the planned flight area, and the planned shooting position. The vegetation state detection method according to claim 5 or 6, wherein flight and automatic imaging are performed.
(Claim 8)
The vegetation state detection method according to any one of claims 1 to 7, wherein an image color-coded according to the numerical value of the NDVI or the HB-NDVI is presented.

The present invention can provide a vegetation state detection method capable of detecting a vegetation state under a protective net or a disaster prevention net by an optical means from above the protective net or a disaster prevention net.

Side view which shows the vegetation state detection method which concerns on 1st Embodiment of this invention. Front view showing protection net or disaster prevention net Front view showing bleached protection net or disaster prevention net Front view showing a multispectral camera used in the first embodiment of the present invention. Side view which shows the vegetation state detection method which concerns on 2nd Embodiment of this invention. A 3D (three-dimensional) shape seen from above constructed in the second embodiment of the present invention. 3D (three-dimensional) shape seen from the side constructed in the second embodiment of the present invention. Visible light image showing the shooting location (left) and NDVI processed image of the shot image (right) Ortho image of a field with a protective net (left) and its NDVI image (right) Graph showing absorption spectrum of chlorophyll Diagram illustrating Rayleigh scattering and Mie scattering Graph showing wavelength and particle radius where Rayleigh scattering and Mie scattering occur Plan view (a) and vertical cross-sectional view (b) showing the reflection of sunlight by the net. Plan view (a) and vertical cross-sectional view (b) showing the scattering of sunlight by the net.

Examples of the present invention will be described in detail with reference to the drawings below.

[First Embodiment]
FIG. 1 is a side view showing a vegetation state detection method according to the first embodiment of the present invention.

As shown in FIG. 1, the vegetation state detection method according to the embodiment of the present invention is covered with a protective net or a disaster prevention net 101 by a multispectral camera 2 mounted on a UAV (unmanned aerial vehicle, for example, a drone) 1. Taken from the sky above the farm, field, fruit tree or mountain field 102. It is preferable to shoot from a low altitude (about 10 m) as much as possible.

In the photographing, high-definition images of a plurality of bands of infrared and red are photographed, and the captured high-definition image group is integrated (ortholithic). Next, it is presented as a target weighted (colored) image by an image technique that maps NDVI (normalized difference vegetation index). Here, the presentation of an image includes any of generation and output of image data, transmission, reproduction of the output image data on a display (display of an image), and printing of an image based on the output image data.

Alternatively, high-definition images in a plurality of bands of infrared and blue are taken, and the taken high-definition images are integrated (orthogonalized). Next, it is presented as a target weighted (colored) image by an image technique that maps HB-NDVI (blue vegetation index).

Conventionally, the state of a farm, a field, a fruit tree or a mountain field under a protective net or a disaster prevention net could not be observed with a visible image, but in the present invention, NDVI image processing using NIR (near infrared: ~ 840 nm) information is used. Alternatively, by using HB-NDVI image processing, the state of the farm, field, etc. under the net can be observed. This enables, for example, agriculture of the right person and the right place, and realizes so-called smart agriculture.

In vegetation state detection under a protective net or disaster prevention net, it is thought that the proportion of transmitted light information decreases due to the reflection and scattering of light by the net, and the object scattering of light strongly depends on its wavelength and is of short wavelength. Is easy to scatter, and long wavelength ones are hard to be scattered, so analysis using infrared light is effective.

[About the vegetation index]
In this embodiment, NDVI (Normalized Difference Vegetation Index) or HB-NDVI (Half Blue Normalized Difference Vegetation Index) is used for indexing.

NDVI is used to discriminate general plant growth and nutrients, and as shown in Equation 1 below, the reflectance of the near-infrared band and red with respect to the sum of the reflectance values of the near-infrared band and red. The ratio of the difference between the values. It is a commonly used indicator of vegetation chlorophyll content and provides information on vegetation growth and nutrients. This indicator can be used to monitor vegetation growth and vegetation coverage.
[Equation 1] NDVI = (NIR-Red (red: 650 nm)) / (NIR + Red)

Further, in the present invention, as shown in Equation 2 below, HB-NDVI, which is the ratio of the difference between the near-infrared band and the blue reflectance value to the total of the near-infrared band and the blue reflectance value, can be used. can. This indicator can also be used to monitor vegetation growth and vegetation coverage.
[Equation 2] HB-NDVI = (NIR-0.5 × Blue (blue: 450 nm)) / (NIR + 0.5 × Blue)

The NDVI uses symmetrical points (bands) of NIR (~ 840 nm) with high reflection and Red (650 nm) with low reflection (high absorption). As shown in FIG. 10, the absorption of plants is absorbed in two bands at 450 nm and 650 nm. From this viewpoint, Red (650 nm) having low reflection (high absorption) is changed to blue (450 nm). The same evaluation can be made even if they are replaced. In this case, the existing index is BNDVI shown in the following formula 3, which shows the same tendency as NDVI.
[Equation 3] BNDVI = (NIR-Blue (blue: 450 nm)) / (NIR + Blue)
Since the blue absorption peak is about 1.5 to 3 times the red absorption peak, in the present invention, the Blue value of BNDVI is halved to obtain the HB-NDVI represented by the above formula 2. Use.

[About protection nets or disaster prevention nets]
FIG. 2 is a front view showing a protection net or a disaster prevention net.
FIG. 3 is a front view showing a bleached protection net or a disaster prevention net.

As shown in FIGS. 2 and 3, the protective net or the disaster prevention net 101 is typically a net made of stranded synthetic resin fibers (for example, polyamide resin) having a thickness of 1 mm to 5 mm. The pitch is 2 mm to 50 mm, but is not limited thereto.

As shown in FIG. 3, the net becomes white due to aging and adhesion of fine algae, and becomes a color close to green.

[About UAV (Unmanned Aerial Vehicle)]
The UAV detects its position by GNSS positioning (eg GPS). Position detection is improved by RTK (Real Time Kinematic) position correction technology (technology that realizes high-precision positioning by position information data from "reference station" installed on the ground), and cm (centimeter). Has order accuracy.

The UAV can perform automatic flight and automatic shooting based on its own position and the map information, planned flight area, and planned shooting position prepared in advance. That is, the UAV can automatically take a picture of the scheduled location and return within the scheduled flight area while passing through the scheduled air route.

[Second Embodiment]
In the above-mentioned UAV1, if the area under the protection net or the disaster prevention net 101 is flown, there is a risk of collision with the branches of 102 such as fruit trees. When the UAV1 collides with a branch of a fruit tree or the like 102, it becomes impossible to take an image sufficient to construct a three-dimensional shape of the whole fruit tree or the like 102.

The UAV used in this embodiment is configured to avoid obstacles by image processing using AI (Artificial Intelligence) without using GNSS positioning (for example, GPS). AI performs deep learning on the shape of fruit trees, etc., and based on this learning result, determines the image of fruit trees, etc. taken during flight, and determines that it is a fruit tree, etc. While making a judgment, the approach speed and approach direction to the fruit tree, etc. are calculated. Further, the AI calculates a route for avoiding the approaching fruit trees and controls the UAV route to avoid the fruit trees and the like.

The UAV can execute automatic flight and automatic photography based on the map information, the planned flight area, and the planned shooting position prepared in advance while avoiding obstacles such as fruit trees. That is, the UAV can automatically take a picture of the planned location and return within the planned flight area without colliding with obstacles such as fruit trees, while passing through the planned air route. ..

FIG. 5 is a side view showing a vegetation state detection method according to a second embodiment of the present invention.

In this embodiment, as shown in FIG. 5, a UAV whose route is controlled by AI is used to photograph a fruit tree or the like 102 under a protective net or a disaster prevention net 101 from the side.

FIG. 6 is a 3D (three-dimensional) shape seen from above constructed in the second embodiment of the present invention.
FIG. 7 is a 3D (three-dimensional) shape seen from the side constructed in the second embodiment of the present invention.

As shown in FIG. 6, a 3D (three-dimensional) shape seen from above can be constructed based on the image taken by the UAV. Further, as shown in FIG. 7, it is possible to construct a 3D shape viewed from the side. Construction of the 3D shape can be performed by, for example, SfM software (such as "MetaShape"). Information from the lower side of the protection net or the disaster prevention net 101 can be constructed by the constructed 3D shape.

For imaging, as in the first embodiment described above, high-definition images in a plurality of bands of infrared and red are captured, and the captured high-definition image group is integrated (orthogonalized). Next, it is presented as a target weighted (colored) image by an image technique that maps NDVI (normalized difference vegetation index). Here, the presentation of an image includes any of generation and output of image data, transmission, reproduction of the output image data on a display (display of an image), and printing of an image based on the output image data.

Alternatively, high-definition images in a plurality of bands of infrared and blue are taken, and the taken high-definition images are integrated (orthogonalized). Next, it is presented as a target weighted (colored) image by an image technique that maps HB-NDVI (blue vegetation index).

If the above-mentioned integration (orthoration) is performed on the constructed 3D shape data and a high-precision positioning technique (for example, RTK technique) is used in combination, the position coordinates of the fruit in the fruit tree or the like 102 can be specified.

[Third Embodiment]
As in the first embodiment described above, trees and the like are photographed from above the protection net or the disaster prevention net, and as in the second embodiment described above, the trees and the like are photographed from the side, and these photographs are taken. By superimposing the acquired information, 3D information (including fruits) of trees and the like, which have many obstacles and are difficult to be three-dimensionalized, can be obtained with high accuracy and high definition.

Based on this 3D information (including fruits), it is possible to perform fruit growth prediction (yield amount prediction) and growth management in the field. Further, by giving 3D information (including fruits) to other harvesting robots and automatic harvesters, automatic harvesting can be performed.

[Example 1]
Hereinafter, examples of the present invention will be described.
FIG. 1 is a side view showing a vegetation state detection method according to the first embodiment of the present invention.
FIG. 4 is a front view showing a multispectral camera used in the first embodiment of the present invention.

In this embodiment, as shown in FIG. 1, a UAV 1 equipped with a multispectral camera 2 was used. As shown in FIG. 4, the multispectral camera 2 has visible light (Vis-RGB), blue (B-450 nm), green (G-560 nm), red (R-650 nm), and an infrared end (RE-730 nm). Has six cameras with a sensitivity band in the near infrared (NIR-840 nm). For UAV1, "Phantom 4 Multispectral (trade name)" manufactured by DJI JAPAN Co., Ltd. was used. This UAV1 can utilize the position detection by GPS and the position correction signal of RTK.

Shooting with UAV1 was performed as automatic shooting by setting the target resolution and overlap rate as a mission flight that automatically flies and shoots in a pre-designated area at the lowest possible altitude (about 10 m). .. NDVI (normalized difference vegetation index) mapping was performed based on the obtained image group (6 images at each shooting point).

FIG. 8 shows a visible light image (left) showing a shooting location and an NDVI-processed image (right) obtained by orthosing the shot image.

In this NDVI, as shown in the left vertical axis of the right figure of FIG. 8, the situation where the growth is good is mapped in red, and the situation where the growth is bad or the soil or the inorganic substance is mapped in gray. In FIG. 8, it is a monochrome display.

FIG. 9 is an ortho image (left) and an NDVI image (right) of the field covered with a protective net.

In trees such as fruit gardens, a net 101 is often applied from fruit set to harvest for the purpose of insect repellent, bird proofing, hail proofing, and wind proofing, as shown in the left figure of FIG. In this case, in the visible image, it is difficult to observe the growth of the fruit trees and the like 102 below, which is obstructed by the net 101. On the other hand, by mapping the NDVI using the near-infrared component and the red component, as shown in the right figure of FIG. 9, the growing state of the fruit trees and the like 102 could be clearly discriminated.

Here, the reason why the vegetation under the protection net or the disaster prevention net is difficult to detect with the image of visible light will be examined.

FIG. 11 is a diagram illustrating Rayleigh scattering and Mie scattering.
FIG. 12 is a graph showing the wavelength and particle radius at which Rayleigh scattering and Mie scattering occur.

Rayleigh scattering and Mie scattering are well-known as light scattering, as shown in FIG. 11, but the stranded wires constituting the protective network and the disaster prevention network are such that these scatterings occur as shown in FIG. It is not a fine thickness.

FIG. 13 is a plan view (a) and a vertical sectional view (b) showing the reflection of sunlight by the net.
FIG. 14 is a plan view (a) and a vertical sectional view (b) showing the scattering of sunlight by the net.

As shown in FIGS. 13 and 14, when the protective net or the disaster prevention net 101 is irradiated with sunlight, the scattering of light by the stranded wire (shielding portion) constituting the net depends on the wavelength. Long-wavelength light (red to infrared) has weak scattering as shown in FIG. 13, but short-wavelength light (blue) has strong scattering as shown in FIG.

When the farm, field, etc. under the protection net or disaster prevention net 101 are photographed from above, it is considered that the vegetation is difficult to detect with visible light due to the influence of such scattered light. However, light of any wavelength passes through the opening of the net. Therefore, vegetation could be detected in the NDVI image or the HB-NDVI image.
Further, the index mapping (indexing) is a normalized vegetation index, which is obtained by dividing the component of each pixel by the sum of the used pixel components. That is, each pixel is corrected for the amount of light by the normalization process. This means that, in principle, shadows are less likely to appear.
For example, in the case of NDVI, the bands used are NIR and Red, and the difference (NIR-Red) is divided by the sum of the reflected light amounts (NIR + Red) to standardize. Other indices described later, GNDVI, LCI, and OSAVI, can be considered in almost the same manner. That is, the component of each pixel is divided by the sum of the used pixel components.
As a result, the shadows of the part of the net are almost eliminated by the light amount correction by the normalization process included in these equations, and the shadows are corrected to be close to the image reflecting normal illumination.
As described above, the effects of the present invention could be illustrated by the present examples.

[Example 2]
In this embodiment, a UAV equipped with a multi-spectral camera and an AI that deeply learned the shape of a fruit tree or the like was used. The multispectral camera is the same as that in Example 1 described above. Based on the learning result, AI determines an image of a fruit tree, etc. taken during flight, determines that it is a fruit tree, etc., calculates the approach speed and approach direction to the fruit tree, etc., and approaches. The route to avoid fruit trees, etc. is calculated, and the route of UAV is controlled to avoid fruit trees, etc.

Shooting with UAV is set so that the target resolution and overlap rate can be obtained as a mission flight that automatically flies and shoots from the side while avoiding obstacles such as fruit trees in the area specified in advance. , I went as an automatic shooting. NDVI (normalized difference vegetation index) mapping was performed based on the obtained image group (6 images at each shooting point).

FIG. 6 is a 3D (three-dimensional) shape seen from above constructed in the second embodiment.

Trees such as fruit gardens are often netted for the purpose of insect repellent, bird repellent, hail repellent, and wind repellent from fruit set to harvest. In this case, in the visible image, it is difficult to observe the growth of the fruit trees and the like 102 below because of the obstruction by the net. On the other hand, by mapping the NDVI using the near-infrared component and the red component, as shown in FIG. 6, the growing state of the fruit trees and the like 102 could be clearly discriminated.

FIG. 7 is a 3D (three-dimensional) shape seen from the side constructed in the second embodiment.

In this NDVI, since the images are taken from the side, as shown in FIG. 7, the growing state of the fruit trees and the like 102 can be clearly discriminated without being disturbed by the net. In FIG. 7, the situation where the growth is good is mapped in red, and the situation where the growth is poor or the soil or inorganic substances are mapped in gray. In FIG. 7, it is a monochrome display.

[Effects of the present invention, etc.]
According to the present invention, the selection of breeding, management and harvesting, which has been performed by humans from the ground near farms and fields based on their experience and intuition, can be changed to tens of minutes of flight (photographing) and tens of minutes. Image processing enables quantitative and continuous analysis and management.

The present invention is mainly effective in cases where protective (disaster prevention) nets such as fruits are applied to farms and fields, and smart agriculture-related cases, but does not exclude cases such as coastal areas and fisheries. Moreover, the various specifications shown in the above-described embodiments are examples and do not limit the present invention in any way. The present invention is to be construed as not limited to these.

In the above-described embodiment, NDVI (normalized difference vegetation index), which is famous for applying remote sensing (image processing with satellite images) to vegetation, is used. However, if an index different depending on the object is used, for example, Floating vinyl, plastic, etc. on the coast or sea may be able to detect their scattering and deterioration state by appropriate image processing. The different indicators are as follows.

GNDVI: Green Normalized Difference Vegetation Index
It is an NDVI that does not use red color, and discriminates chlorophyll. GNDVI uses a green band instead of the red band used in NDVI. This green band represents the surface GNDVI. This is an indicator of the coverage of the green plant canopy. GNDVI indicates whether the vegetation is deficient in water, deficient in nutrients, or suffering from the post-maturity Bromas recession.
[Equation 4] GNDVI = (NIR-Green) / (NIR + Green)

NDRE: Normalized Difference Red Edge Normalized Difference Red Edge Index NDRE uses a red edge band instead of the red edge band used in NDVI. This red edge band is the spectral region of the transition zone from the red spectrum to the near infrared spectrum.
NDRE can be used to manage vegetation based on variables such as chlorophyll and sugar.
[Equation 5] NDRE = (NIR-Red Edge) / (NIR + Red Edge)

LCI: Leaf Chlorophyll Index
It is an index specialized for the content of chlorophyll (chlorophyll) in plants. The chlorophyll content of leaves is an important indicator for assessing vegetation growth and yield. It is also an indicator for assessing plant nutritional stress, disease, growth and aging.
[Number 6] LCI = (NIR-Red Edge) / (NIR + Red)

OSAVI: Optimized Soil-adjusted Vegetation Index
It is an index to judge the content of fertilizer absorbed by plants. The soil-adjusted vegetation index is based on NDVI and related observational data. This indicator is used to remove the effects of soil conditions on vegetation.
[Equation 7] OSAVI = (L + 0.16) * (NIR-Red (red)) / (NIR + Red + 0.16)
However, L is a vegetation covering density and takes a value of 0 to 1.0.

1 UAV (unmanned aerial vehicle)
2 Multispectral camera 101 Protective net or disaster prevention net 102 Farm, field, fruit tree or mountain field (fruit tree, etc.)

Claims (8)

A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures images of multiple bands of infrared and red from above farms, fields, fruit trees or mountains covered with protective or disaster prevention nets.
A vegetation state detection method characterized by integrating the photographed images and mapping an NDVI (normalized difference vegetation index) onto the images. A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures images of multiple bands of infrared and blue from above farms, fields, fruit trees or mountains covered with protective or disaster prevention nets.
A method for detecting a vegetation state, which comprises integrating the photographed images and mapping HB-NDVI (blue vegetation index) onto the images. The vegetation according to claim 1 or 2, wherein the protective net or the disaster prevention net is a net made of synthetic resin fiber stranded wire having a thickness of 1 mm to 5 mm, and the pitch of the net is 2 mm to 50 mm. State detection method. The UAV is characterized in that it detects its own position by GNSS positioning and executes automatic flight and automatic shooting based on its own position, map information prepared in advance, a planned flight area, and a planned shooting position. The vegetation state detection method according to claim 1, 2 or 3. A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures multiple bands of infrared and red from the side of a farm, field, fruit tree or mountain field covered with protective or disaster prevention nets.
A vegetation state detection method characterized in that a 3D shape is constructed from the captured image group, the image group is integrated, and an NDVI (normalized difference vegetation index) is mapped on the image. A multi-spectral camera mounted on a UAV (unmanned aerial vehicle) captures multiple bands of infrared and blue from the side of a farm, field, fruit tree or mountain field covered with protective or disaster prevention nets.
A vegetation state detection method characterized in that a 3D shape is constructed from the captured image group, the image group is integrated, and HB-NDVI (blue vegetation index) is mapped on the image. The UAV controls the route to avoid fruit trees by image processing using AI that deeply learns the shape of fruit trees, etc., and automatically based on the map information prepared in advance, the planned flight area, and the planned shooting position. The vegetation state detection method according to claim 5 or 6, wherein flight and automatic imaging are performed. The vegetation state detection method according to any one of claims 1 to 7, wherein an image color-coded according to the numerical value of the NDVI or the HB-NDVI is presented.

Images