JP7069609B2 - Crop cultivation support device - Google Patents

Description

The present invention relates to a crop cultivation support device including an imaging unit that captures an image of a cultivation area.

The conventional crop cultivation support device is disclosed in Patent Document 1. This crop cultivation support device includes a video camera (imaging unit), a position detection unit, and a personal computer attached to an agricultural vehicle. The position detection unit has a GPS antenna and a GPS receiver. A video camera and a position detector are connected to the personal computer.

In addition, a support column extending to the rear of the agricultural vehicle is provided on the upper part of the agricultural vehicle. The video camera is fixed to the rear end of the column so that it faces vertically downward. A GPS antenna 4 is attached directly above the video camera 1.

In the crop cultivation support device having the above configuration, when the agricultural vehicle travels on the field (cultivation area), the video camera captures the field and forms a plurality of captured images. The personal computer derives the mutual positional relationship of a plurality of captured images based on the position information from the GPS. Next, a plurality of captured images are combined on a personal computer to form a composite image of the entire field.

At the time of compositing, it is possible to select whether to preferentially synthesize the captured image captured earlier or to preferentially combine the captured image captured later for the overlapping portion of the captured image. As a result, even if a part of the agricultural vehicle is reflected in the captured image due to the relationship between the mounting position of the agricultural vehicle of the video camera and the field of view of the video camera, the part of the agricultural vehicle can be erased by selecting the captured image. Can be done.

Japanese Unexamined Patent Publication No. 2001-120042 (Pages 3, 4, 4, Figure 1, Figure 2, Figure 4)

However, according to the above-mentioned conventional crop cultivation support device, a part of the agricultural vehicle is reflected in the first captured image and the sun reflected on the water surface is reflected in the later captured image for the overlapping portion of the captured image. In some cases. In this case, a defect occurs in the captured image used for compositing, and the quality of the composite image of the entire field is deteriorated. Therefore, there is a problem that it is not possible to provide an image useful for determining the growth state of the crop in the cultivated area.

An object of the present invention is to provide a crop cultivation support device capable of providing an image useful for determining the growth state of a crop in a cultivation area.

In order to achieve the above object, the present invention has an image pickup unit that moves over a cultivation area where a crop is cultivated and images the cultivation area, a position detection unit that detects the position of the image pickup unit, and the cultivation area. In a crop cultivation support device provided with a compositing unit that forms a point image of the same point based on a plurality of captured images captured by the imaging unit and synthesizes a plurality of the point images.
A defect detection unit for detecting a defect area having a defect on the captured image is provided.
It is characterized in that the synthesis unit forms the point image excluding the defect region detected by the defect detection unit.

Further, in the present invention, in the crop cultivation support device having the above configuration, it is preferable that the defect detection unit detects the defect region by relative comparison of predetermined characteristic values of a plurality of regions divided in the captured image. ..

Further, in the present invention, in the crop cultivation support device having the above configuration, it is preferable that the defect detection unit detects the defect region by relative comparison of predetermined characteristic values of a plurality of regions with respect to the same point of the plurality of captured images. ..

Further, in the present invention, in the crop cultivation support device having the above configuration, the defect detection unit detects the defect region by comparing the characteristic values of a plurality of regions divided in one captured image with a predetermined threshold value. Is preferable.

Further, in the present invention, in the crop cultivation support device having the above configuration, it is preferable that the defect detecting unit detects the peripheral portion of each captured image as the defect region.

Further, according to the present invention, in the crop cultivation support device having the above configuration, the imaging unit includes a visible imaging unit that captures a visible image and a near-infrared imaging unit that captures a near-infrared image. It is preferable to detect a region where the amount of deviation between the pixels of the visible image and the pixels of the near-infrared image is larger than a predetermined amount as the defective region.

Further, in the present invention, it is preferable to derive the position of the point image based on the position, the angle of view and the number of pixels of the image pickup unit in the crop cultivation support device having the above configuration.

Further, in the present invention, in the crop cultivation support device having the above configuration, it is preferable that a storage unit for storing a plurality of the captured images and a plurality of point images is provided on a predetermined network.

According to the present invention, there is a compositing unit that forms a point image of the same point in the cultivation area based on a plurality of captured images captured by the imaging unit and synthesizes a plurality of point images, and a defective area having a defect on the captured image. It is equipped with a defect detection unit that detects. Then, the point image is formed by excluding the defect region detected by the defect detection unit by the composition unit. As a result, it is possible to combine a plurality of point images from which defective areas have been removed, and it is possible to improve the quality of the combined image. Therefore, it is possible to provide an image useful for determining the growth state of the crop in the cultivation area.

Schematic block diagram which shows the crop cultivation support apparatus of one Embodiment of this invention. A block diagram showing the configuration of a crop cultivation support device according to an embodiment of the present invention. The figure which shows the formation process of the synthetic image of the crop cultivation support apparatus of one Embodiment of this invention. The plan view which shows the image pickup process of the crop cultivation support apparatus of one Embodiment of this invention. Side view which shows the imaging process of the crop cultivation support apparatus of one Embodiment of this invention. The figure for demonstrating the defect detection process of the crop cultivation support apparatus of one Embodiment of this invention. The figure which shows the image synthesis process of the crop cultivation support apparatus of one Embodiment of this invention.

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a crop cultivation support device of the present embodiment. The crop cultivation support device 1 includes an image pickup unit 2 and an information terminal 3, and forms an entire image of the field FD (cultivation area) in which the crop PL is cultivated. The field FD is, for example, a paddy field or a field having a substantially rectangular shape in a plan view and having ridges on all four sides and surrounded by the ridges.

The image pickup unit 2 is composed of, for example, a multispectral camera, and is attached to the flying object 4. The image pickup unit 2 has a visible image pickup unit 20 (see FIG. 2) and a near infrared image pickup unit 21 (see FIG. 2). The visible image pickup unit 20 and the near infrared image pickup unit 21 are arranged at predetermined intervals in a plane parallel to the field FD.

The visible imaging unit 20 forms an image of visible light (visible image). The visible imaging unit 20 includes a first bandpass filter, a first imaging optical system, a first image sensor, a first digital signal processor, and the like (all not shown). The first bandpass filter transmits light in a relatively narrow band having a wavelength of 650 nm as a central wavelength, for example. The first imaging optical system forms an optical image of visible light to be measured that has passed through the first bandpass filter on a predetermined first imaging plane. The first image sensor is arranged so that the light receiving surface is aligned with the first image plane, and converts an optical image of visible light to be measured into an electrical signal. The first digital signal processor performs image processing on the output of the first image sensor to form a visible image.

The near-infrared imaging unit 21 forms an image of near-infrared light (near-infrared image). The near-infrared imaging unit 20 includes a second bandpass filter, a second imaging optical system, a second image sensor, and a second digital signal processor (all not shown). The second bandpass filter transmits light in a relatively narrow band having a predetermined wavelength of 750 nm or more (for example, a wavelength of 800 nm) as a central wavelength. The second imaging optical system forms an optical image of the near-infrared light to be measured that has passed through the second bandpass filter on a predetermined second imaging plane. The second image sensor is arranged so that the light receiving surface is aligned with the second image plane, and converts the optical image of the near infrared light to be measured into an electrical signal. The second digital signal processor performs image processing on the output of the second image sensor to form a near-infrared image. As the first image sensor and the second image sensor, for example, a VGA type (640 pixels × 480 pixels) image sensor can be used.

The imaging unit 2 may omit the near-infrared imaging unit 21 and may be configured by the visible imaging unit 20. In this case, the visible imaging unit 20 includes a first imaging optical system, a first image sensor, and R / G / B / Ir or W / Y / R / Ir arranged on the first image sensor. (See, for example, Japanese Patent No. 5168353). The above-mentioned "R", "G", and "B" are filters that mainly transmit red light, green light, and blue light, respectively. The above-mentioned "Ir" is a filter that mainly transmits near-infrared light. The "W" is a filter that mainly transmits white light, and the "Y" is a filter that mainly transmits yellow light.

The aircraft 4 is composed of an unmanned aerial vehicle (drone) capable of autonomous flight, and flies over the field FD. The air vehicle 4 has a housing 41 provided with a plurality of (for example, eight) horizontal rotary blades 42. The image pickup unit 2 is arranged in the housing 25 attached to the lower surface of the housing 41. The housing 25 is configured to be movable in a direction in which the lower surface having an opening (not shown) faces vertically downward and in a direction toward the front by a moving mechanism (not shown). As a result, the visible imaging unit 20 and the near-infrared imaging unit 21 can move in the direction facing vertically downward and the direction facing the front through the opening. Further, a plurality of leg portions 46 are projected downward from the lower surface of the housing 41. When the aircraft 4 lands, the legs 46 come into contact with the ground.

When the flight path, flight altitude, etc. are set for the aircraft 4 by prior programming, the aircraft 4 rotates the horizontal rotary wing 42 without the user maneuvering using a radio controller (not shown) or the like. Can fly autonomously. When the flying object 4 flies, the visible imaging unit 20 and the near-infrared imaging unit 21 face vertically downward. As a result, the visible imaging unit 20 and the near-infrared imaging unit 21 can move over the field FD to image the field FD. When the aircraft 4 lands, the visible imaging unit 20 and the near-infrared imaging unit 21 face the front direction. This makes it possible to prevent damage to the lenses (not shown) of the visible imaging unit 20 and the near-infrared imaging unit 21 due to collision with the ground. The flight body 4 may be configured to enable radio-controlled flight (guided flight) by the user.

Further, the flying object 4 may be, for example, a balloon, an airship, an airplane, a helicopter, or the like. Further, instead of the flying object 4, a lifting device (not shown) such as a crane that lifts the housing 25 from the ground may be used. At this time, the suspended housing 25 is moved in the horizontal direction.

The information terminal 3 is composed of, for example, a personal computer. The information terminal 3 is configured to be communicable with the flying object 4 and the image pickup unit 2 via the connection units 35 and 45 (see FIG. 2), and has a display unit 31 and an operation unit 32. The display unit 31 is composed of, for example, a liquid crystal panel or the like, and displays an operation menu, a communication status with the flying object 4, a composite image CI described later, and the like. The operation unit 32 has a keyboard 32a and a mouse 32b, and receives and outputs various data input operations. The information terminal 3 may be configured by a mobile phone such as a smartphone or a tablet PC.

FIG. 2 is a block diagram showing the configuration of the crop cultivation support device 1. The information terminal 3 and the flying object 4 each have control units 39 and 49 including CPUs that control each unit. The control unit 39 and the control unit 49 are wirelessly connected via the connection units 35 and 45. A display unit 31, an operation unit 32, a storage unit 33, a synthesis unit 34, a connection unit 35, a defect detection unit 36, and a growth index derivation unit 37 are connected to the control unit 39.

The storage unit 33 stores various programs and various data. Various programs include programs that control the overall operation of the information terminal 3. Various data include a visible image, a near-infrared image, a captured image FI, a point image PI, a composite image CI, and the like. The captured image FI is an image formed based on a visible image and a near-infrared image in the present embodiment. The point image PI is an image of a point on the captured image FI. The point image PI is composed of one or a plurality of pixels, and the size of one point image PI is smaller than the size of one captured image FI. The composite image CI is an image formed by synthesizing a plurality of point image PIs, and in the present embodiment, for example, is an entire image of one field FD. Further, in the present embodiment, the captured image FI, the point image PI, and the composite image CI are formed by the NDVI image described later.

The synthesizing unit 34 forms the point image PIs based on a plurality of (six images in this embodiment) captured image FIs. Further, the synthesizing unit 34 synthesizes a plurality of point image PIs. As a result, the composite unit 34 forms the composite image CI. The details of the point image PI will be described later.

Antennas (not shown) are provided at the connection portions 35 and 45. The connection units 35 and 45 transmit and receive communication data by wireless radio waves via an antenna. The communication data includes a visible image and a near-infrared image captured by the visible imaging unit 20 and the near-infrared imaging unit 21, respectively.

The defect detection unit 36 detects a defect region DR (see FIG. 6) having a defect described later on the captured image FI. The synthesizing unit 34 forms a point image PI excluding the defective region DR as described later.

The growth index derivation unit 37 derives a growth index indicating the growth state of the crop PL in the field FD based on the visible image captured by the visible imaging unit 20 and the near-infrared image captured by the near-infrared imaging unit 21. .. In this embodiment, NDVI (Normalized Difference Vegetation Index, Normalized Difference Vegetation Index) is used as a growth index. An NDVI image, which is an image displayed by NDVI, is formed based on a visible image and a near-infrared image. When the pixel value of the visible image is Rv and the pixel value of the near-infrared image is Ri, the pixel value of the NDVI image corresponding to the pixel value of the visible image and the near-infrared image corresponds to NDVI, and NDVI = (Ri-). It is expressed as Rv) / (Ri + Rv). The larger the NDVI, the thicker the vegetation.

For example, the pixel value at the pixel position (10, 15) of the NDVI image is derived from the pixel value at the pixel position (10, 15) of the visible image and the pixel value at the pixel position (10, 15) of the near-infrared image. Considering the difference between the visible image pickup unit 20 and the near-infrared image pickup unit 21, at least one of the pixel position of the visible image and the pixel position of the near-infrared image is shifted so as to correct the difference. Then, NDVI may be derived. Further, the first imaging optical system of the visible imaging unit 20 and the second imaging optical system of the near infrared imaging unit 21 have the same optical characteristics such as the angle of view and distortion.

In addition, as a growth index, RVI (Ratio Vegetation Index, ratio vegetation index, RVI = Ri / Rv), DVI (Difference Vegetation Index, differential vegetation index, DVI = Ri-Rv), TVI (Transformed Definition0. 5) ^0.5 ), IPVI (Infrared Percentage Vegetation Index, IPVI = Ri / (Ri + Rv) = (NDVI + 1) / 2) or the like may be used.

Further, as a growth index, a vegetation coverage ratio indicating the ratio of the crop PL covering the ground surface of the field FD may be used. For example, the growth index derivation unit 37 performs binarization processing based on the near-infrared image of the field FD imaged by the near-infrared imaging unit 21 to form a white and black binarized image. At this time, the white part corresponds to the crop PL and the black part corresponds to the soil. Then, the growth index derivation unit 37 derives the vegetation coverage ratio indicating the proportion of the white portion in the binarized image.

A storage unit 43, a connection unit 45, a horizontal rotor 42, an azimuth measurement unit 47, and a position detection unit 48 are connected to the control unit 49 of the aircraft body 4. Further, a visible imaging unit 20 and a near infrared imaging unit 21 are connected to the control unit 49. The storage unit 43 stores the control program of the flying object 4 (including the autonomous flight program), the control program of the imaging unit 2, and various data. The various data stored in the storage unit 43 include flight data such as the flight path and altitude of the flying object 4.

The position detection unit 48 includes, for example, GPS (Global Positioning System). The GPS may be a GPS having a correction function for correcting an error such as DGPS (Differential GPS). The position detection unit 48 detects the position (latitude X, longitude Y, altitude Z) of the image pickup unit 2 (visible image pickup unit 20 and near infrared image pickup unit 21). The azimuth measuring unit 47 is composed of, for example, a 3-axis azimuth meter (3-axis geomagnetic sensor), and measures the azimuth of the imaging unit 2 on the earth. Further, the control unit 39 derives the position of the point image PI from the position, the angle of view, and the number of pixels of the image pickup unit 2.

FIG. 3 is a diagram showing a process of forming a composite image CI. The step of forming the composite image CI includes a photographing step, an NDVI image forming step, a defect detection step, a point image forming step, and an image compositing step. In the present embodiment, the composite image CI will be described using an image of one field FD as an example.

FIG. 4 is a plan view showing an imaging process. In the crop cultivation support device 1 having the above configuration, the flying object 4 autonomously flies by the rotation of the horizontal rotary blade 42 and reaches the sky above the field FD. The flying object 4 flies over the field FD at a set altitude H (30 m in this embodiment) and a set speed (15 km / h in this embodiment) in a zigzag manner as shown by the arrow FC. Specifically, the flying object 4 flies from one end in the longitudinal direction toward the other end along the longitudinal direction of the field FD, and when it reaches the other end, it reaches a predetermined distance along the lateral direction of the field FD. After flying, it repeats flying along the longitudinal direction toward one end in the longitudinal direction. While the flying object 4 is flying over the field FD, the visible imaging unit 20 and the near-infrared imaging unit 21 image the field FD.

FIG. 5 shows a side view of the imaging process. In FIG. 5, the illustration of the flying object 4 is omitted, and the arrow FC indicates the flight direction of the flying object 4 (moving direction of the imaging unit 2). In the imaging step, assuming that the angle of view α of the visible imaging unit 20 and the near infrared imaging unit 21 is 45 °, the imaging range D is a region of about 24.85 m × about 33.13 m. Then, when the imaging unit 2 captures one visible image and one near-infrared image per second, the same point SP on the field FD is imaged six times each. The visible image and the near-infrared image are transmitted to the control unit 39 via the connection units 45 and 35 and stored in the storage unit 33. At this time, each visible image and each near-infrared image are stored in the storage unit 33 in association with the position of the image pickup unit 2. After the imaging step, the process shifts to the NDVI image forming step.

In the NDVI image forming step, the NDVI is derived from the pixels of the visible image and the pixels of the near-infrared image by the growth index derivation unit 37. As a result, a captured image FI which is an NDVI image is formed. The NDVI image is color-coded by R, G, and B according to the NDVI. For example, in an NDVI image, the NDVI decreases in the order of R, G, and B. After the NDVI image forming step, the process shifts to the defect detection step.

FIG. 6 is a diagram showing a plurality of (six images in this embodiment) captured image FI in which the same point SP is captured during the defect detection step. The same point SP is imaged in the captured images FI of "A" to "F". The imaging timings of the captured images FI of "A" to "F" are different from each other, and they are arranged in chronological order in the order of "A" to "F". For example, when the field FD is a paddy field, the sun reflected on the water surface during the photographing process may be reflected in the visible image and the near infrared image. For this reason, a defective region DR may occur on the captured image FI, which has a defect due to the reflection of the sun.

Further, in the imaging process, for example, the orientation of the housing 25 accommodating the imaging unit 2 may be temporarily changed to the front direction (direction other than the field FD) due to a malfunction of the moving mechanism or the like. In this case, the region when the orientation of the image pickup unit 2 is changed on the captured image FI becomes the defective region DR.

Further, in the imaging process, for example, the appearance of the crop PL lying down due to wind or the like may be reflected in the visible image and the near infrared image. In this case, the region where the crop PL has fallen on the captured image FI is the defective region DR.

In addition, a shading error may occur in which the NDVI (characteristic value) of the peripheral portion Ph of the captured image FI is lower than that of the NDVI of the central portion CP. In this case, the region of the peripheral portion Ph of the captured image FI becomes the defective region DR.

Further, as described above, the visible imaging unit 20 and the near infrared imaging unit 21 are arranged at a predetermined distance. Therefore, an error may occur due to the deviation between the visible pixel and the near-infrared pixel due to the parallax between the visible image pickup unit 20 and the near-infrared image pickup unit 21. In this case, the region where the deviation amount is larger than the predetermined value is the defective region DR.

The NDVI of the defective region DR in the above example shows a lower value than the NDVI of another region (region that is not the defective region DR) without defects. Therefore, if a plurality of point image PIs are combined using the NDVI of the defective region DR, the quality of the combined image CI deteriorates. Therefore, the defect detection unit 36 detects the defect region DR on the captured image FI, and the synthesis unit 34 forms the point image PI excluding the defect region DR detected by the defect detection unit 36. In the present embodiment, the defect detection unit 36 detects the defect region DR by relatively comparing the NDVIs of a plurality of regions divided within one captured image FI.

For example, when the sun reflected on the water surface of the field FD is reflected in the captured images FI of "A" to "F" obtained by imaging the same point SP shown in FIG. 6, the defect detection unit 36 is the region where the sun is reflected. Is detected as a defective area DR. At this time, the region image Pr of a predetermined region including the same point SP on the captured image FI of "C" overlaps with the defective region DR. The size (range) of the region image Pr is smaller than the size (range) of the captured image FI, and is the same size as the point image PI.

Next, the synthesis unit 34 uses the remaining five region image Prs for the plurality of region image Prs including the same spot SP without using the region image Pr of "C", and the region image including the same spot SP. The average value of NDVI of each pixel in Pr is derived. As a result, a point image PI (see FIG. 7) of the same point SP from which the defective region DR is removed is formed. Similarly, for the other area image Pr of the same point SP, the point image PI from which the defective area DR is removed is formed. As a result, a plurality of point image _PINs (see FIG. 7) from which the defective region DR is removed are formed. Note that "N" is a serial number assigned to each of the plurality of point image PIs, and is an integer from 1 to K when the total number of point image PIs is K.

In the present embodiment, when the defective region DR is not detected for the six region image Prs including the same spot SP, the synthesis unit 34 uses all the six region image Prs and the region image Pr including the same spot SP. The average value of the NDVI of each pixel in the is derived. If the NDVI of the defective region DR can be corrected by the image correction of the region image Pr, the corrected region image Pr may be used.

Next, an image synthesizing step of synthesizing a plurality of point image PIs to form a composite image CI will be described. The synthesizing unit 34 derives an image transformation matrix for affine transformation based on the latitude X _N , the longitude Y _N , the altitude Z _N , and the azimuth θ _N in each of the plurality of point image _PINs from which the defective region DR has been removed. .. Each of the plurality of point image PINs is the latitude X _N of the visible image pickup unit 20 and the near infrared image pickup unit 21 _at the time of imaging each of the visible image and the near infrared image used in forming the point image _PIN . , Longitude Y _N , altitude Z _N , and azimuth θ _N. The synthesizing unit 34 performs an affine transformation on each of the plurality of point image P _INs based on the latitude X _N , the longitude Y _N , the altitude Z _N , and the direction θ _N of the point image _{P IN} . .. As a result, the pixel position of the point image PI is converted into the pixel position in the composite image CI.

FIG. 7 shows a diagram for explaining the image composition process. The upper part of FIG. 7 shows a plurality of point image _PINs , and the lower part of FIG. 7 shows the coordinate system of the composite image CI. As is known, an affine transformation is a transformation that combines a linear transformation and a translation (translation), and is expressed by Equation 1 (for example, "What is an affine transformation?", [Online], [May 15, 2017]. Day search], Internet (URL: http://d.hatena.ne.jp/Zellij/20120523/p1)).

The column vector (x, y) on the right side in Equation 1 is the pixel position in the point image PI as shown in the upper part of FIG. 7 (x = x _N , y = y _N in the _Nth point image PIN). The column vector (x', y') indicates the pixel position in the coordinate system of the composite image CI as shown in the lower part of FIG. 7. The 2-by-2 matrix consisting of the components a, b, c, and d of 1-by-1 column, 1-by-2 column, 2-by-1 column, and 2-by-2 column in the first term on the right side of Equation 1 is a conversion matrix of rotation. It represents R (θ), and the column vector (t _x , _ty ) in the second term on the right side of Equation 1 represents a translation (parallel movement) transformation matrix.

Each component a, b, c, d in the transformation matrix R (θ) of rotation is given by the mathematical formula 2 from the value θ of the direction measuring unit 47 (θ = θ _N in the _Nth point image PIN). If the actual altitude of the imaging unit 2 during the imaging process is different from the reference altitude (set altitude H, see FIG. 5), the actual altitude of the imaging unit 2 during the imaging process is shown in the rotation matrix R (θ). The scaling factor according to the actual altitude at the time of the imaging process is multiplied so as to be equivalent to the reference altitude. For example, the component a is a value obtained by multiplying cos θ by a scaling coefficient.

Then, the components t _x and _ty in the translation (translation) transformation matrix are given by the formulas 3 and 4. In Equations 3 and 4, the angle φ [rad] between the distance d [m] from the origin (0,0) in the coordinate system of the composite image CI and the horizontal direction (x-axis) is derived from the latitude X and the longitude Y. And convert it to the number of pixels.

The coefficient k in the formulas 3 and 4 is a coefficient for converting a meter into the number of pixels. For example, when the number of pixels of the first image sensor and the second image sensor is 480 pixels, the coefficient k = 24.85 [m] / 480 [pixels] ≈0.052 [m / pixel]. Further, methods for calculating the distance d [m] and the angle φ [rad] are known and are given by the formulas 5 and 6 (for example, "distance and azimuth between two points", [online], [2017 5]. Search on 22nd of March], Internet (URL: http://keisan.casio.jp/exec/system/125767079).

Note that X1 and Y1 in Equations 5 and 6 are the latitude and longitude of point A, respectively, and X2 and Y2 in Equations 5 and 6 are the latitude and longitude of point B, respectively. R in Equation 5 is the equatorial radius (= 6378.137 km) when the earth is regarded as a sphere. Here, the point A or the point B is the origin (0,0) in the coordinate system of the composite image CI.

Further, the rotation matrix of Equation 1 and the transformation matrix of translation (translation) can be combined into a single matrix M (matrix of 3 rows and 3 columns) and expressed as Equation 7.

When actually obtaining the NDVI of the pixel at the position (x', y') after conversion, first derive M ^-1 , which is the inverse matrix of the matrix M, and then (x', y') after conversion. Corresponding position (x, y) of is derived by the formula 8.

Then, the NDVI that is interpolated (linear interpolation, etc.) by the NDVI of the pixel at the position (x, y) before conversion obtained by Equation 8 or the NDVI of a plurality of pixels existing in the vicinity of the position is converted to the position (x') after conversion. , Y') is determined as the NDVI of the pixel.

According to the crop cultivation support device 1 of the present embodiment, the image pickup unit 2 that moves over the field FD (cultivation area) in which the crop PL is cultivated to image the field FD, and the position detection unit that detects the position of the image pickup unit 2. It includes 48 and a synthesis unit 34 that forms a point image PI of the same point SP of the field FD based on a plurality of captured image FIs imaged by the image pickup unit 2 and synthesizes a plurality of point image PIs. Further, a defect detection unit 36 for detecting a defect region DR having a defect on the captured image FI is provided, and a point image PI is formed by excluding the defect region DR detected by the defect detection unit 36 by the synthesis unit 34.

As a result, a point image PI from which the defective region DR is removed is formed, and a plurality of point image PIs are combined. Therefore, the quality of the composite image of the entire field FD is improved, and it is possible to provide an image useful for determining the growth state of the crop PL.

Further, the defect detection unit 36 detects the defect region DR by relatively comparing the NDVIs (characteristic values) of a plurality of regions divided within one captured image FI. As a result, the defect detection unit 36 can easily detect the defect region DR.

The defect detection unit 36 may detect the defect region DR by relatively comparing the NDVIs (characteristic values) of the plurality of regions with respect to the same point SP of the plurality of captured image FIs. Even in this case, the defect detection unit 36 can easily detect the defect region DR.

Further, the defect detection unit 36 may detect the defect region DR by comparing the NDVI (characteristic value) of a plurality of regions divided in one captured image FI with a predetermined threshold value. Even in this case, the defect detection unit 36 can easily detect the defect region DR.

Further, the defect detection unit 36 may determine the peripheral portion Ph of each captured image FI as the defect region DR. As a result, it is possible to easily prevent the quality deterioration of the composite image CI due to the shading error in which the NDVI of the peripheral portion Ph of the captured image FI is lower than the NDVI of the central portion CP.

Further, the imaging unit 2 has a visible imaging unit 20 for capturing a visible image and a near-infrared imaging unit 21 for capturing a near-infrared image, and the defect detection unit 36 includes a pixel of the visible image and a near-infrared image. A region in which the amount of deviation from the pixel is larger than a predetermined amount may be determined as a defective region DR. As a result, it is possible to easily prevent the quality deterioration of the composite image CI due to the error due to the deviation between the visible pixel and the near-infrared pixel due to the parallax between the visible image pickup unit 20 and the near-infrared image pickup unit 21.

Further, the control unit 39 derives the position of the point image PI based on the position of the image pickup unit 2, the angle of view α, and the number of pixels. As a result, the position of the point image PI can be easily derived.

The information terminal 3 may be connected to a predetermined network such as the Internet, and a storage unit for storing the captured image FI, the point image PI, and the composite image CI may be provided on the network. As a result, the capacity of the storage unit 33 can be reduced.

In the present embodiment, the captured image FI, the point image PI, and the composite image CI are NDVI images, but the captured image FI, the point image PI, and the composite image CI are not limited to the NDVI image. For example, the near-infrared imaging unit 21 may be omitted from the imaging unit 2, and the captured image FI, the point image PI, and the composite image CI may be a color image of RGB values captured by the visible imaging unit 20. Further, the captured image FI, the point image PI and the composite image CI may be images represented by the vegetation coverage ratio.

INDUSTRIAL APPLICABILITY The present invention can be used for a crop cultivation support device including an image pickup unit that captures an image of a cultivation area.

1 Crop cultivation support device 2 Imaging unit 3 Information terminal 4 Flying object 20 Visible imaging unit 21 Near infrared imaging unit 25 Housing 31 Display unit 32 Operation unit 33 Storage unit 34 Synthesis unit 35 Connection unit 36 Defect detection unit 37 Growth index derivation Part 43 Storage part 45 Connection part 47 Direction measurement part 48 Position detection part DR Defect area FD Field FI Captured image PI Point image PL Crop Pr Area image SP Same point

Claims (6)

An imaging unit that moves over the cultivation area where crops are cultivated and images the cultivation area,
A position detection unit that detects the position of the image pickup unit and
In a crop cultivation support device equipped with a synthesis unit,
A defect detection unit is provided to detect a defect area having a defect on the captured image captured by the imaging unit.
The synthesis unit generates a plurality of region images which are predetermined regions including the same spot in each of the captured images from a plurality of captured images in which the same spot in the cultivated region is imaged and the imaging timings are different from each other. Then, among the plurality of the region images, the region image including the defect region detected by the defect detection unit is excluded, and a point image is generated using the region image not including the defect region, and a plurality of images different from each other are generated. The point image is combined corresponding to the position to form a composite image, and the composite image is formed .
The imaging unit has a visible imaging unit that captures a visible image and a near-infrared imaging unit that captures a near-infrared image.
The defect detection unit is a crop cultivation support device characterized in that a region in which the amount of deviation between the pixels of the visible image and the pixels of the near infrared image is larger than a predetermined amount is detected as the defect region . An imaging unit that moves over the cultivation area where crops are cultivated and images the cultivation area,
A position detection unit that detects the position of the image pickup unit and
In a crop cultivation support device provided with a compositing unit that forms a point image of the same point in the cultivation area based on a plurality of captured images captured by the imaging unit and synthesizes a plurality of the point images.
A defect detection unit for detecting a defect area having a defect on the captured image is provided.
The point image is formed by excluding the defect region detected by the defect detection unit by the synthesis unit.
The imaging unit has a visible imaging unit that captures a visible image and a near-infrared imaging unit that captures a near-infrared image.
The defect detection unit is a crop cultivation support device characterized in that a region in which the amount of deviation between the pixels of the visible image and the pixels of the near infrared image is larger than a predetermined amount is detected as the defect region. The crop cultivation support device according to claim 1 or 2 , wherein the position of the point image is derived based on the position, the angle of view, and the number of pixels of the image pickup unit. The crop cultivation support device according to any one of claims 1 to 3 , wherein a storage unit for storing a plurality of the captured images and a plurality of the point images is provided on a predetermined network. On the computer
A step of moving over a cultivation area where crops are cultivated and receiving an image captured by an image pickup unit that images the cultivation area in association with the position of the image pickup unit.
A region where the amount of deviation between the pixels of the visible image captured by the imaging unit and the pixels of the near-infrared image captured by the imaging unit is larger than a predetermined amount is excluded as a defect region having a defect on the captured image. Then, a step of forming a point image of the same point in the cultivation area based on a plurality of captured images captured by the imaging unit, and
A step of synthesizing a plurality of the above-mentioned point images and a step of synthesizing them.
A program to execute. The program according to claim 5 , wherein the defective area is a region having a defect caused by the reflection of the sun.

Images