WO2019235240A1

WO2019235240A1 - Information processing device

Info

Publication number: WO2019235240A1
Application number: PCT/JP2019/020508
Authority: WO
Inventors: 中川　宏; 山田　和宏; 陽平大野
Original assignee: 株式会社Ｎｔｔドコモ
Priority date: 2018-06-06
Filing date: 2019-05-23
Publication date: 2019-12-12
Also published as: JPWO2019235240A1; JP7218365B2

Abstract

The objective of the present invention is to support appropriate assessment of a growth status in an agricultural field with mixed shade. An agricultural field image generating unit (101) acquires an image of an agricultural field imaged by means of a drone (20). A sunlit region determining unit (102) determines a sunlit region of the agricultural field included in the acquired image of the agricultural field. An index calculating unit (103) calculates an index representing a growth status of produce appearing in the image, from the acquired image of the agricultural field. A growth information generating unit (104) acquires, as a sunlit index, an NDVI in a position included in the determined sunlit region, from among calculated NDVIs. The growth information generating unit (104) acquires, as a shaded index, an index in a position not determined to be a sunlit region. The growth information generating unit (104) uses the acquired sunlit index and shaded index to generate, for each sunlit region and shaded region, a region unit NDVI map representing the growth status of produce in each of a plurality of regions into which the agricultural field A1 is divided.

Description

Information processing device

The present invention relates to a technology that supports the determination of the work content related to crops.

Technology that supports the determination of the content of work related to crops is known. Patent Document 1 discloses a technique for supporting fertilization management including fertilization management such as determination of the amount of fertilizer and other farm work based on observed data such as converted leaf color values calculated from image data obtained by photographing crops. Is disclosed.

JP 2017-125705 A

An index (for example, NDVI) indicating a growth state is obtained by using an output of a sensor (an image sensor or the like) that measures an amount of light with respect to a crop in a farm field, and is used as a guide for a work time. In that case, since the amount of light differs between the sun and the shade, the value of the index obtained even in the same breeding situation changes.
Then, an object of this invention is to support the appropriate judgment of the growth condition in the farm field where shade is mixed.

In order to achieve the above object, the present invention provides a determination unit that determines a sunny area of a field included in a captured field image, and an index that indicates a growth status of the crop in the determined sunny area An information processing apparatus including an index acquisition unit acquired as an output unit and an output unit that outputs the acquired sunny index as an index of the sunny area is provided.

According to the present invention, it is possible to support an appropriate determination of the growth situation in a field with shade.

The figure showing the whole agriculture support system composition concerning an example Diagram showing hardware configuration of server device Diagram showing drone hardware configuration The figure showing the functional composition which an agricultural support system realizes The figure showing an example of the imaging method of a farm field The figure showing an example of the judgment result of the Hinata area The figure showing an example of the NDVI map of a pixel unit The figure showing an example of the NDVI map of an area unit The figure showing an example of the NDVI map of an area unit The figure showing an example of the search screen of growth information The figure showing an example of the search screen of growth information The figure showing an example of the search screen of growth information The figure showing an example of the search screen of growth information The figure showing an example of the operation | movement procedure of each apparatus in a recording process The figure showing the functional composition realized by a modification The figure showing an example of the image for index correction The figure showing the functional composition realized by a modification A figure showing an example of an input screen for shooting conditions The figure showing the functional composition realized by a modification The figure showing an example of the imaging | photography range of the imaging | photography means installed in a modification The figure showing the functional composition realized by a modification A figure showing an example of the correlation of index ratios The figure showing an example of a sunny pixel and a shade pixel

1 Example FIG. 1: represents the whole structure of the agricultural assistance system 1 which concerns on an Example. The agricultural support system 1 is a system that supports a person who performs work in a farm (a place where crops such as rice, vegetables, and fruits are grown) by using an index that represents the growth status of the crop. The index that represents the growth status is an index that represents one or both of the progress of the growing stage of the crop (for example, whether or not it is suitable for harvesting) and the status (also called activity) such as the presence or absence of disease. It is.

In this embodiment, NDVI (Normalized Difference Vegetation 後述 Index), which will be described later, is used, and an index representing the growth status of the crop in the field is calculated using an image of the field taken from above by the flying object. Is done. The flying body may be anything as long as it can photograph the field, and a drone is used in this embodiment. The agricultural support system 1 includes a network 2, a server device 10, a drone 20, and a user terminal 30.

The network 2 is a communication system including a mobile communication network and the Internet, and relays data exchange between devices accessing the own system. The server device 10 is accessing the network 2 by wired communication (may be wireless communication), and the drone 20 and the user terminal 30 are accessing by wireless communication (the user terminal 30 may be wired communication).

The user terminal 30 is a terminal used by a user of the system (for example, a worker who performs work in a farm), and is, for example, a smartphone, a laptop computer, or a tablet terminal. In this embodiment, the drone 20 is a rotorcraft type flying body that includes one or more rotor blades and flies by rotating the rotor blades. The drone 20 includes a photographing unit that photographs a farm field from above while flying. The drone 20 is carried to the field by a farm worker who is a user of the agricultural support system 1, for example, and performs flight and shooting by performing an operation of starting shooting flight.

The server device 10 is an information processing device that performs processing related to worker support. The server device 10 performs, for example, a process of calculating the above-described NDVI from the field image captured by the drone 20. NDVI uses the property that the green leaf of the spot absorbs a lot of red visible light and reflects a lot of light in the near-infrared region (0.7 μm to 2.5 μm). Represented by The worker can determine the timing of watering, fertilizer application, pesticide application, etc. on the crops in the field where he / she works with reference to the growth status represented by the calculation result of NDVI displayed on the user terminal 30. .

FIG. 2 shows the hardware configuration of the server device 10 and the user terminal 30. Each of the server device 10 and the user terminal 30 is a computer including each device such as a processor 11, a memory 12, a storage 13, a communication device 14, an input device 15, an output device 16, and a bus 17. Note that the term “apparatus” here can be read as a circuit, a device, a unit, or the like. Each device may include one or a plurality of devices, or some of the devices may not be included.

The processor 11 controls the entire computer by operating an operating system, for example. The processor 11 may include a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, a register, and the like. Further, the processor 11 reads programs (program codes), software modules, data, and the like from the storage 13 and / or the communication device 14 to the memory 12, and executes various processes according to these.

The number of processors 11 that execute various processes may be one, two or more, and the two or more processors 11 may execute various processes simultaneously or sequentially. Further, the processor 11 may be implemented by one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 12 is a computer-readable recording medium, and includes, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. May be. The memory 12 may be called a register, a cache, a main memory (main storage device), or the like. The memory 12 can store the above-described program (program code), software module, data, and the like.

The storage 13 is a computer-readable recording medium such as an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (for example, a compact disk, a digital versatile disk, a Blu- ray (registered trademark) disk, smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, or the like.

The storage 13 may be called an auxiliary storage device. The above-described storage medium may be, for example, a database including the memory 12 and / or the storage 13, a server, or other suitable medium. The communication device 14 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, or a communication module.

The input device 15 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts an input from the outside. The output device 16 is an output device (for example, a display, a speaker, or the like) that performs output to the outside. Note that the input device 15 and the output device 16 may have an integrated configuration (for example, a touch screen). The devices such as the processor 11 and the memory 12 are accessible to each other via a bus 17 for communicating information. The bus 17 may be composed of a single bus or may be composed of different buses between devices.

FIG. 3 shows the hardware configuration of the drone 20. The drone 20 is a computer that includes a processor 21, a memory 22, a storage 23, a communication device 24, a flying device 25, a sensor device 26, a photographing device 27, and a bus 28. Note that the term “apparatus” here can be read as a circuit, a device, a unit, or the like. Each device may include one or a plurality of devices, or some of the devices may not be included.

The processor 21, the memory 22, the storage 23, the communication device 24, and the bus 28 are the same type of hardware as the device of the same name shown in FIG. 2 (performance and specifications may be different). The communication device 24 can also perform wireless communication between drones in addition to wireless communication with the network 2. The flying device 25 is a device that includes a motor, a rotor, and the like and causes the aircraft to fly. The flying device 25 can move the aircraft in all directions in the air, or can stop (hover) the aircraft.

The sensor device 26 is a device having a sensor group that acquires information necessary for flight control. The sensor device 26 is a position sensor that measures the position (latitude and longitude) of the own device, and the direction in which the own device is facing (the drone has a front direction defined, and the front direction is directed. Direction sensor that measures the altitude of the aircraft, a velocity sensor that measures the velocity of the aircraft, and an inertial measurement sensor (IMU (Inertial) that measures triaxial angular velocity and acceleration in three directions. Measurement Unit)).

The photographing device 27 is a so-called digital camera that has a lens, an image sensor, and the like and records an image photographed by the image sensor as digital data. This image sensor has sensitivity not only to visible light but also to light having a wavelength in the near infrared region necessary for calculating NDVI. The photographing device 27 is attached to the lower part of the casing of the own device (drone 20), has a fixed photographing direction, and photographs a vertically lower part during the flight of the own device. The photographing device 27 has an autofocus function, and can automatically focus and photograph even if the flight altitude changes.

The server device 10 and the drone 20 include a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). It may be configured including hardware, and a part or all of each functional block may be realized by the hardware. For example, the processor 11 may be implemented by at least one of these hardware.

The server device 10 and the drone 20 included in the agricultural support system 1 store programs provided by this system, and the function group described below is executed by the processor of each device executing the program and controlling each unit. Realized.
FIG. 4 shows a functional configuration realized by the agricultural support system 1. The server device 10 includes an agricultural field image generation unit 101, a sunny area determination unit 102, an index calculation unit 103, a growth information generation unit 104, and a growth information recording unit 105.

The drone 20 includes a flight control unit 201, a flight unit 202, a sensor measurement unit 203, and an imaging unit 204. The flight control unit 201 controls the flight of the own aircraft when photographing a farm field. The flight control unit 201 stores, for example, field range information (for example, latitude and longitude information indicating the outer edge of the field) indicating the geographical range of the field registered in advance by a farmer who is a user, and the field range. Based on the information, control is performed to fly the aircraft in a flight path that flies across the entire field at a constant altitude.

The flight path in this case is, for example, a path in which a rectangular farm field flies in a wavy locus from one side of the farm field to the other side facing the side. In addition, it may be a route that flies along the outer edge of the field, and after a week, shifts the route to the inside and draws a spiral trajectory. Any route is acceptable. The flying unit 202 has a function of flying the aircraft. In this embodiment, the flying device 202 is operated by operating a motor, a rotor, and the like included in the flying device 25.

The sensor measurement unit 203 performs measurement by each sensor (position sensor, direction sensor, altitude sensor, speed sensor, inertial measurement sensor) included in the sensor device 26, and calculates the position, direction, altitude, speed, angular velocity, and acceleration of the own device. Measure repeatedly at predetermined time intervals. The sensor measurement unit 203 supplies sensor information indicating the measured position, direction, altitude, speed, angular velocity, and acceleration to the flight control unit 201. The flight control unit 201 controls the flight unit 202 based on the supplied sensor information and causes the aircraft to fly along the above-described flight path.

The sensor measurement unit 203 supplies sensor information indicating the measured position, direction, altitude, and speed to the imaging unit 204. The photographing unit 204 has a function of photographing a subject using the photographing device 27 and is an example of the “photographing unit” in the present invention. When the flight control unit 201 performs control to fly over the field as described above, the imaging unit 204 captures the field as a subject. The imaging unit 204 captures an image of a field, and also captures an area where the crop is growing in the field (a crop area).

Since the image sensor of the imaging device 27 has sensitivity to light having a wavelength in the near-infrared region as described above, each pixel forming a still image captured by the imaging unit 204 shows visible light red, green, and blue. Along with pixel values (R, G, B), it is represented by a pixel value (IR) indicating light having a wavelength in the near infrared region. The imaging unit 204 captures a plurality of still images based on the supplied sensor information so that all areas in the field are included.

FIG. 5 shows an example of a method for photographing a farm field. FIG. 5 shows a path B1 when the drone 20 flies over the field A1 with a wavy locus. The imaging unit 204 calculates the imaging range (the field range included in the angle of view) of the field (0 m above the ground) from the altitude indicated by the sensor information and the angle of view of the imaging device 27. Then, the photographing unit 204 represents the ratio of the area where the current photographing range and the previous photographing range overlap from the speed and direction indicated by the sensor information (for example, expressed as a percentage of the overlapping area when the area of the photographing range is 100%). ) Is less than the threshold value, the next shooting is performed.

In the example of FIG. 5, the photographing unit 204 first photographs the photographing region C1, and then photographs the photographing region C2 that slightly overlaps the photographing region C1. In addition, the photographing unit 204 notifies the flight control unit 201 of the calculated size of the photographing range when the own device (drone 20) is turned back. The flight control unit 201 folds the route by shifting the distance by which the imaging range of the notified size overlaps, for example, the imaging areas C4 and C5 in FIG.

The imaging unit 204 captures still images obtained by imaging the imaging regions C1 to C32 shown in FIG. 5, that is, a plurality of still images whose imaging ranges are slightly overlapped, by repeating imaging using this method. In the example of FIG. 5, the field A <b> 1 has a size and shape that can accommodate a plurality of imaging ranges, but this need not be the case. In that case, all the areas in the field are included in any one of the still images by widening the overlapping part of the shooting ranges or by shooting including the outside of the field.

Note that the photographing method by the photographing unit 204 is not limited to this. For example, if the flight speed and flight altitude at the time of shooting are determined, the overlapping time interval is calculated in advance as shown in FIG. 5, so that shooting may be performed at that time interval. Further, if the map of the farm field and the shooting position are determined in advance, the shooting unit 204 may shoot when flying in the determined position. Further, the photographing unit 204 may photograph a moving image in which the data becomes large as long as the storage capacity or communication speed of the own device is not insufficient.

In addition to these, a known method for photographing the ground using a drone may be used. Operation | movement of each part with which drone 20 is provided is started by operation of the flight start by the farm worker mentioned above. When the operation of each unit is started, the drone 20 flies over the set flight path over the field, and the imaging unit 204 repeatedly performs imaging as described above. When the photographing unit 204 performs photographing, image data indicating the photographed still image and photographing information related to photographing (information indicating the position, orientation, altitude, and time when photographing, and the angle of view of the photographing device 27). Is transmitted to the server device 10.

The farm field image generation unit 101 of the server device 10 receives the transmitted image data, and acquires a still image indicated by the image data as a farm field image captured by the drone 20. The farm field image generation unit 101 is an example of the “first image acquisition unit” in the present invention. When the field image generation unit 101 also acquires the shooting information indicated by the received image data, the field image generation unit 101 generates an image of the entire field from the image for each shooting region.

The farm field image generation unit 101 calculates an overlapping part of each image using the position, orientation, altitude, and angle of view indicated by the acquired imaging information, and for the overlapping part, for example, adopts a pixel of one image to make the entire field Generate an image of When the field image generation unit 101 assigns a pixel ID to each pixel of the generated entire field image, the field image generation unit 101 supplies the entire field image data indicating the pixel ID and the entire field image to the sunny area determination unit 102 and the index calculation unit 103. To do.

The sunny area determination unit 102 determines the sunny area of the farm field included in the photographed field image. The sunny area determination unit 102 is an example of the “determination unit” in the present invention. In this embodiment, the sunny area determination unit 102 determines the sunny area included in the field image captured by the imaging unit 204 of the drone 20 as described above. The sunny area determination unit 102 determines the sunny area based on the pixel value of each pixel of the entire field image indicated by the supplied entire field image data.

The sunny area determination unit 102 calculates, for example, values in the HSV color space (hue H: Hue, saturation S: Saturation, brightness V: Value) of each pixel from the R, G, and B values of the pixel values. . The sunny area determination unit 102 determines, for example, an area of an object of the same color (for example, a crop area and a soil area) that has a difference between the hue H and the saturation S within a predetermined range among the calculated HSV values of each pixel. Judge. The sunny area determination unit 102 extracts pixels whose lightness value V changes by a difference threshold value or more in the same color area determined in this way.

The pixel extracted in this way may indicate the boundary between the sun and the shade. When the average value of the values of brightness V of the region with the larger value of brightness V among the regions divided by the extracted pixels is equal to or greater than the brightness threshold, the value of brightness V is It is determined that it represents a pixel, and the area is determined as a sunny area. Further, the sunny area determination unit 102 determines an area that has not been determined as the sunny area as a shaded area.

It should be noted that depending on the color of the area, the brightness V of the sun and the shade tends to be different, and therefore the difference between the brightness V of the sun and the shade tends to be different. For example, the brightness of the sunny pixel in the green leaf region tends to be larger than the brightness of the sunny pixel in the soil region. Therefore, the sunny area determination unit 102 uses, for example, a difference threshold value and a lightness threshold value corresponding to the hue H and saturation S values of each area (in the leaf area, both threshold values are increased compared to the soil area) The shaded area may be determined.

As shown in FIG. 5, it is assumed that the forest R1 exists along the two-thirds of the length in the east-west direction from the west end of the field A1, on the south side of the field A1.
FIG. 6 shows an example of the determination result of the sunny area. In FIG. 6, in the entire field image E1 of the field A1, as a determination result, the shaded area G1 due to the forest R1 is shown on the southwest side, and the other areas are shown as the sunny area F1.

When the sunny area determination unit 102 performs the determination as described above, the sunny area information indicating the pixel ID of the pixel included in the area determined to be sunny and the shade information indicating the pixel ID of the pixel included in the area determined to be shaded. Is generated. The sunny area determination unit 102 supplies the generated sunny information and shade information to the growth information generation unit 104 together with the entire field image data.

The index calculation unit 103 calculates an index representing the growth status of the crop shown in the image from the field image acquired by the field image generation unit 101. The index calculation unit 103 is an example of the “calculation unit” in the present invention. The index calculation unit 103 calculates the above-described NDVI as an index indicating the growth status. For example, the index calculation unit 103 calculates the above-described red pixel value (R) and pixel value (IR) of light having a wavelength in the near-infrared region for each pixel of the entire field image indicated by the entire field image data NDVI = NDVI is calculated by substituting into the equation (IR-R) / (IR + R).

In the present embodiment, the image capturing unit 204 of the drone 20 captures an image of the farm field, and the index calculation unit 103 calculates NDVI as described above, thereby measuring the state of crop cultivation in the field. The growth information generation unit 104 generates an NDVI map in pixel units representing NDVI at a position on the field corresponding to each pixel.
FIG. 7 shows an example of an NDVI map in pixel units. In the example of FIG. 7, the NDVI map M1 in pixel units of the field A1 shown in FIG. 5 is represented.

The NDVI map M1 includes an upper left corner pixel D1 (NDVI = 0.3), a lower left corner pixel D2 (NDVI = -0.5), and an upper right corner pixel D3 (NDVI) in the entire field image E1 shown in FIG. = 0.3) and a rectangular map having a pixel D4 (NDVI = 0.2) at the lower right corner at the corner. The index calculation unit 103 supplies the NDVI map M1 representing the calculated NDVI to the growth information generation unit 104 together with the entire field image data as index information representing the calculated crop growth status.

The growth information generation unit 104 generates growth information indicating the growth status of the crop in the field using the supplied sunny information, shade information, index information (for example, the NDVI map M1) and the entire field image data. The growth information generation unit 104 generates, in particular, sunny growth information indicating the growth status of the crop in the sun and shade growth information indicating the growth status of the crop in the shade. The growth information generation unit 104 generates such information as follows, for example.

The growth information generation unit 104 includes the NDVI in the sunny area determined by the sunny area determination unit 102 among the NDVIs of the pixels indicated by the NDVI map M1 supplied from the index calculation unit 103 (NDVI at each position included in the sunny area). ) As a Hinata index. Here, the position includes a range from a position represented by one pixel of the entire field image to a position represented by a plurality of pixels, and represents a region having a certain extent in any case. The Hinata index refers to an index representing the growth status of a crop measured from an image of a crop in the sunny area.

Specifically, the growth information generation unit 104 acquires an NDVI associated with the pixel ID indicated by the sun direction information as the sun direction indicator among the NDVIs indicated by the index information. In addition, the growth information generation unit 104 acquires a shade area index that is not determined to be a sunny area (an index of a position that is not determined to be a sunny area) as a shade index. The shade index refers to an index representing the growth status of a crop measured from a crop image in the shaded area. The growth information generation unit 104 acquires, as a shade index, the NDVI associated with the pixel ID indicated by the shade information among the NDVIs indicated by the index information. The growth information generation unit 104 is an example of the “index acquisition unit” in the present invention.

In the present embodiment, as described above, the growth information generating unit 104 includes the NDVI calculated for the sunny area from the NDVI calculated by the index calculating unit 103 for the entire field image as included in the sunny index (the sunny area of the field). As NDVI at position). Further, the growth information generation unit 104 acquires the NDVI calculated for the shaded area from the NDVI calculated by the index calculation unit 103 for the entire field image as a shaded index (NDVI at a position included in the shaded area of the field). .

The growth information generation unit 104 generates an NDVI map for each area representing the growth status of the crop for each of a plurality of areas that divide the field A1, for each of the sunny area and the shaded area, using the acquired sunny index and shade index. To do.
8A and 8B show examples of NDVI maps in units of areas. 8A shows an NDVI map Mf1 in the area unit in the sunny area F1 shown in FIG. 6, and FIG. 8B shows an NDVI map Mg1 in the area unit in the shaded area G1 shown in FIG.

In the example of FIGS. 8A and 8B, each segmented area is represented by an eight-level pattern (Lv1 is the smallest and Lv8 is the largest) according to the average value of NDVI. The NDVI map Mf1 indicates that the NDVI is larger and the growth state is better as it approaches the northeast side of the field A1. Further, the NDVI map Mg1 also shows that the NDVI is larger and the growth state is better as it approaches the northeast side of the field A1. However, in the NDVI map Mg1, the NDVI is small because it is shaded even in the divided area representing the same shooting area as the NDVI map Mf1.

For example, the NDVI of the segmented area Hc14 representing the imaging area C14 is Lv6 (third from the largest) in the NDVI map Mf1, but Lv4 (fifth from the largest) in the NDVI map Mg1. Further, the NDVI of the segmented area Hc19 representing the imaging area C19 is Lv7 (second from the largest) in the NDVI map Mf1, but is Lv5 (fourth from the largest) in the NDVI map Mg1.

The growth information generation unit 104 includes a symbol (a sun identification symbol) indicating that the generated NDVI map Mf1 is generated from the acquired hinata index, shooting information of the entire field image as a base, and information associated with the shooting date and time. It is generated as the above-mentioned sunny growth information. Further, the growth information generation unit 104 associates the generated NDVI map Mg1 with a symbol (shade identification symbol) indicating that the generated NDVI map Mg1 is generated from the acquired shade index, the shooting information and the shooting date and time of the original whole field image. Information is generated as the above-mentioned shade growth information. The growth information generation unit 104 supplies the generated growth direction information and shade growth information to the growth information recording unit 105.

The growth information recording unit 105 records the growth information of the crop in the field (information indicating the growth status of the crop) together with the sunny growth information and the shade growth information generated by the growth information generation unit 104. The growth information recording unit 105 is in a state where the recorded growth information can be browsed by the user (farmer) (for example, on a web page accessible by a URL (Uniform （Resource Locator) transmitted to the user). Hold.

When the user terminal 30 accesses the URL, the growth information recording unit 105 displays a screen for searching for growth information that is held, for example.
9A to 9D show an example of a growth information search screen. In FIG. 9A, the user terminal 30 displays a growth information search screen including input fields for time, field name, and sunshine conditions. In this example, the search condition “Hyuga” of “Agricultural field A1” of “2017/5/15” is input.

When the user performs an operation of pressing the search button H1, the user terminal 30 sends request data for requesting the sunny growth information generated from the photographed image of the field A1 having “2017/5/15” as the photographing date and time. Send to. The growth information recording unit 105 reads the requested sunny growth information from the recorded growth information, and transmits the read sunny growth information to the user terminal 30. The user terminal 30 displays the NDVI map Mf1 indicated by the transmitted sunny growth information as shown in FIG. 9B.

Further, when the user changes only “sunshine condition” to “shade” as shown in FIG. 9C and presses the search button H1, the user terminal 30 takes a photographed image of the field A1 with “2017/5/15” as the photographing date and time. Request data for requesting the shade growth information generated from is transmitted to the server device 10. The growth information recording unit 105 reads the requested shade growth information from the recorded growth information, and transmits the read shade growth information to the user terminal 30. The user terminal 30 displays the NDVI map Mg1 indicated by the transmitted shade growth information as shown in FIG. 9D.

The sunny growth information is information in which the NDVI map Mf1 representing the sunny index (NDVI of the sunny area) acquired by the growth information generating unit 104 is associated with the sunny identification symbol indicating that it is an index of the sunny area. The growth information recording unit 105 outputs (transmits) the sunny growth information to the user terminal 30 to output the associated sunny index as the sunny area index (the sunny area index). Output so that they can be identified.

The shade growth information is information in which the NDVI map Mg1 representing the shade index (NDVI of the shaded area) acquired by the growth information generation unit 104 is associated with the shade identification symbol indicating the shade area index. . The growth information recording unit 105 outputs (transmits) the shade growth information to the user terminal 30 so that the obtained shade index is associated with the shade area index as an index (shade area index). Output so that they can be identified. The growth information recording unit 105 is an example of the “output unit” in the present invention.

Each device provided in the agricultural support system 1 performs a recording process for recording crop growth information based on the above configuration.
FIG. 10 shows an example of the operation procedure of each apparatus in the recording process. This operation procedure is started when a farmer who is a user takes the drone 20 to the field and performs an operation for starting a shooting flight. First, the drone 20 (flight control unit 201, flight unit 202, and sensor measurement unit 203) starts flying over the field based on the stored field range information (step S11).

Next, the drone 20 (the photographing unit 204) starts photographing each photographing region from above the field (step S12), and every time photographing is performed, the photographed still image and photographing information (position when photographing, Image data indicating the azimuth and altitude) is generated and transmitted to the server device 10 (step S13). The server device 10 (farm field image generation unit 101) acquires an image of the farm field indicated by the transmitted image data (step S14).

Next, the server device 10 (the farm field image generation unit 101) generates an image of the entire farm field by combining the acquired farm field images (step S21). Subsequently, the server device 10 (the sunny area determination unit 102) determines the sunny area and the shade area of the field included in the generated image of the entire field (Step S22). Next, the server device 10 (index calculation unit 103) calculates an index (NDVI) indicating the growth status of the crop reflected in the image from the generated image of the entire field (step S23).

Note that the operations in steps S22 and S23 may be performed in the reverse order or in parallel. Then, the server apparatus 10 (growth information generation part 104) produces | generates the sunny growth information which shows the parameter | index etc. of the sunny area | region determined by step S22 among farm fields (step S24). Next, the server apparatus 10 (growth information generation part 104) produces | generates the shade growth information which shows the parameter | index etc. of the shade area | region determined by step S22 among agricultural fields (step S25). Note that the operations in steps S24 and S25 may be performed in the reverse order or in parallel.

Subsequently, the server device 10 (growth information recording unit 105) records the growth information of the sun and the shade growth information generated in steps S24 and S25 as the growth information of the crop in the field (step S26). Then, when there is an access (request) from the user terminal 30, the server device 10 (growth information recording unit 105) outputs the growth information recorded in step S26 to the user terminal 30 (step S31).

In the shaded area, the reflected light becomes weaker and the pixel value of the pixel becomes smaller than in the sunny area. Therefore, even if the pixel value error is the same, the NDVI error is larger. Therefore, the accuracy of NDVI tends to be lower in the shaded area than in the sunny area. In the present embodiment, as described above, by acquiring the NDVI only for the sunny area, it is possible to support an appropriate determination of the growth situation in the field where the shade is mixed as compared with the case where the sunny area and the shade are not distinguished.

In this embodiment, the NDVI only for the shaded area is acquired and the NDVI map for the shaded area is also output. Compared to the NDVI in the shaded area compared to the NDVI in the shaded area, it tends to be judged that the growth situation is bad, but by comparing the NDVI only in the shaded area, the place where the growth situation is good and bad in the shaded area (For example, in the case of the NDVI map Mg1 in FIG. 8B, it can be said that the northeast side has a better growth situation).

2 Modification Examples described above are merely examples of the present invention, and may be modified as follows. Moreover, you may combine an Example and each modification as needed. In that case, you may carry out by giving priority to each modification (it ranks which decides which should be given priority when a competing event occurs if each modification is carried out).

Further, as a specific combination method, for example, a modification using different parameters for obtaining a common value (for example, NDVI) may be combined, and a common value or the like may be obtained using these parameters together. Moreover, you may obtain | require one value etc. by adding together the value etc. which were calculated | required separately according to some rule. Moreover, you may give different weighting for every parameter used in those cases.

2-1 Acquisition of only Hinata Index In the embodiment, the growth information generation unit 104 acquires both the sunflower index and the shade index. However, the present invention is not limited to this. For example, only the sunflower index may be acquired. In that case, the growth information generation unit 104 generates only the NDVI map Mf1 representing the acquired sunny indicator, and outputs the sunny growth information indicating the NDVI map Mf1 to the user terminal 30. Even in this case, since only NDVI in the sunny area can be compared, it is possible to support appropriate judgment of the growing condition of the sunny crop in the field where the shade is mixed.

2-2 Index Acquisition Method In the embodiment, NDVI is calculated for each pixel of the entire field, but the present invention is not limited to this. For example, when only the sunny growth information is generated as in the above modification, NDVI only needs to be calculated for the sunny area. In this case, the sunny area determination unit 102 determines only the sunny area and supplies the generated sunny information to the index calculation unit 103 together with the entire field image data.

The index calculation unit 103 calculates NDVI only for the pixels in the sunny area indicated by the supplied sunny information, and supplies a map representing the calculated NDVI of the sunny area together with the entire field image data to the growth information generation unit 104 as index information. . The growth information generation unit 104 acquires the NDVI supplied in this way, that is, the NDVI calculated only for the sunny area in the field image as the sunny index. Thereby, the load of the process (NDVI calculation process) of the server device 10 can be reduced as compared with the case where the NDVI of the shaded area is also calculated.

2-3 Segmented Area The growth information generation unit 104 generates an NDVI map in units of areas using the area corresponding to the imaging range as the segmented area in the embodiment, but the segmented area is not limited to this. For example, a plurality of shooting ranges may be used as one segmented region, or a region corresponding to a divided region obtained by dividing one shooting region into a plurality of segments may be used as the segmented region. Further, the shape and size of each segmented region may be unified or not uniform.

2-4 Aircraft In the example, a rotary wing aircraft was used as a vehicle for autonomous flight, but this is not a limitation. For example, it may be an airplane type aircraft or a helicopter type aircraft. In addition, the function of autonomous flight is not essential, and if it is possible to fly the assigned flight airspace in the assigned flight permission period, for example, a radio control type (wireless control type) operated by a pilot from a remote location. May be used.

2-5 Original Information for Determination of Hyuga Area and Index Calculation In the embodiment, the determination of the sun area and the calculation of NDVI are performed based on the image taken by the drone 20 during the flight, but the present invention is not limited to this. These determinations and calculations may be performed based on, for example, an image manually captured by an operator using a digital camera, an image captured by a fixed digital camera installed on a farm field, or an image captured from a satellite.

In the embodiment, the NDVI is calculated using the measured value of the image sensor of the imaging device 27 of the drone 20, but the present invention is not limited to this. For example, the NDVI is measured using the measured value of an infrared sensor of a handy type NDVI measuring instrument. May be calculated. Of course, it is less time-consuming to use the image taken with the drone 20 or the like in order to know the growth situation of the entire field, but the handy type can calculate NDVI from the reflected light of the crop only. Accuracy tends to be high. Which method is to be used may be determined in consideration of labor, cost, and required accuracy.

2-6 Index indicating growth status In the examples, NDVI was used as an index indicating the growth status, but the present invention is not limited to this. For example, a leaf color value (value indicating the color of a leaf), a planting rate (occupation rate per unit area of a planting region), SPAD (chlorophyll content), plant height, number of stems, or the like may be used. In short, any value may be used as an index representing the growth status as long as it represents the growth status of the crop and can be calculated from the captured crop region image.

2-7 Correction of indices The intensity of reflected light from the crops that reach the image sensor of the photographing device 27 is not only the sun and shade, but also the intensity of sunlight in each season, the amount of clouds, the atmospheric conditions, etc. It can change depending on the surrounding environment. Moreover, it may change also with the state of the lens of the imaging device 27 influenced by temperature, humidity, etc. Correction for eliminating the change in the index (NDVI) caused by the change in the intensity of the reflected light reaching the image sensor due to those factors may be performed.

FIG. 11 shows a functional configuration realized in this modification. In FIG. 11, a server device 10 a including a corrected image acquisition unit 106 in addition to the units illustrated in FIG. 4 is illustrated. The corrected image acquisition unit 106 acquires an image for index correction taken by the drone 20. The corrected image acquisition unit 106 is an example of the “correction acquisition unit” in the present invention. The index correction image is, for example, an image obtained by photographing a panel having a plurality of regions in which the reflectance of light having a specific wavelength is known in advance.

FIG. 12 shows an example of an index correction image. In FIG. 12, an image obtained by photographing the panel J1 in which the specific reflectance regions J11, J12, J13, and J14 are represented on the surface is represented as an index correction image. Each specific reflectance region is, for example, a region on the panel where the reflectance of red light and infrared rays changes stepwise (for example, the reflectance of red light and infrared rays of J11, J12, J13, and J14 are both 20). %, 40%, 60%, 80%).

In this modification, the farmer causes the drone 20 to photograph the panel J1 before the operation for starting the photographing flight or after the completion of the photographing flight. At the time of photographing the panel J1, the drone 20 may be flying, or the user may lift and photograph it. The imaging unit 204 of the drone 20 transmits image data indicating the captured image of the panel J1 to the server device 10a. The corrected image acquisition unit 106 acquires the image of the panel J1 indicated by the image data thus transmitted as an index correction image.

The corrected image acquisition unit 106 supplies the acquired index correction image to the index calculation unit 103. The index calculation unit 103 calculates a corrected index based on the supplied image, that is, the index correction image acquired by the corrected image acquisition unit 106. The index calculation unit 103 uses the red pixel values (r11, r12, r13, r14) of the specific reflectance regions J11, J12, J13, and J14 included in the acquired index correction image and the wavelengths in the near infrared region. The pixel values of light (ir11, ir12, ir13, ir14) are read out as measurement values.

For example, the server device 10a captures red pixel values (R11, R12, R13, R14) and the near-infrared region when the specific reflectance regions J11, J12, J13, and J14 are photographed in an environment where NDVI can be measured satisfactorily. The pixel values (IR11, IR12, IR13, IR14) of light with wavelengths are stored in the server device 10a as reference values. If the measured value is different from the reference value, the index calculating unit 103 determines a correction formula for correcting the measured value to the reference value.

The index calculation unit 103 determines, for example, a correction formula (for example, measurement value × 1.1 = reference value) by which each measurement value is multiplied by a specific coefficient when the measurement value becomes a reference value. Moreover, the index calculation part 103 should just determine a coefficient for every specific reflectance area | region, when all cannot be made into a reference value with the same coefficient about four specific reflectance area | regions (for example, r11 * 1.1 =). R11, r12 × 1.05 = R12, r13 × 1.1 = R13, r14 × 1.15 = R14).

In this case, if the pixel value is between r11 and r12, a number between 1.1 and 1.05, which is their correction coefficient, is used as the correction coefficient (for example, if the pixel value is the middle of r11 and r12) (1.1 + 1.05) /2=1.075 is used). When the index calculation unit 103 determines correction formulas for R and IR as described above, the index calculation unit 103 performs correction using the correction formula that determines the pixel value of each pixel.

The index calculation unit 103 uses the corrected R and IR pixel values of each pixel to calculate NDVI (corrected NDVI in this modification) as in the embodiment. By correcting the index in this way, changes in NDVI due to changes in the surrounding environment and the state of the lens can be reduced, and the accuracy of NDVI can be increased, compared to the case where correction is not performed. The correction of the pixel value may be performed by a function other than the index calculation unit 103. For example, the pixel value of each pixel may be corrected when the agricultural field image generation unit 101 generates an entire agricultural field image. Further, the NDVI correction method is not limited to the above-described method, and other known methods may be used.

2-8 Hinata and Shade When performing the above correction, the correction target may be narrowed down depending on whether panel J1 is in the sun or in the shade.
FIG. 13 shows a functional configuration realized in this modification. In FIG. 13, a server device 10 b including a sunshine condition specifying unit 107 in addition to the units illustrated in FIG. 11 is illustrated.

The sunshine condition specifying unit 107 specifies whether the shooting condition of the index correction image is sunny or shaded. The sunshine condition specifying unit 107 is an example of the “condition specifying unit” in the present invention. In this modification, for example, a screen for inputting shooting conditions is displayed on the user terminal 30.
FIG. 14 shows an example of an imaging condition input screen. In FIG. 14, the user terminal 30 displays an agricultural support system screen including input fields for a shooting date, a field name, and a sunshine condition.

In this example, the photographing condition “shade” of “field A1” of “2018/5/15” is input. When the user performs an operation of pressing the send button H2, the user terminal 30 captures the photographing condition data indicating that the photographing date and time is “2018/5/15”, the photographing place is “farm field A1”, and the sunshine condition is “shade”. Is transmitted to the server apparatus 10b. The sunshine condition specifying unit 107 sets the sunshine condition (“shade” in the example of FIG. 14) indicated by the transmitted imaging condition data, the date and time indicated by the imaging condition data, and the index correction image captured in the field. Specify as shooting conditions.

The sunshine condition specifying unit 107 notifies the index calculation unit 103 of the specified shooting conditions together with the shooting date and time and the farm field. The index calculation unit 103, based on the index correction image acquired by the correction image acquisition unit 106, is used as an index in a portion determined to be a sunny area in the field image when the shooting conditions for the sun are identified. The corrected index is calculated (the index of the portion determined to be a shaded area is not corrected).

In this modification, the sunny area determination unit 102 supplies the generated sunny information and shade information to the index calculation unit 103 together with the entire field image data. The index calculation unit 103 corrects the pixel value of the pixel in the sunny area indicated by the supplied sunny information among the pixels indicated by the entire field image data as described in the description of FIG. The index calculation unit 103 calculates the NDVI in the same manner as in the embodiment, using the pixel value of each pixel indicating the corrected sunny area.

In addition, when the shadow photographing condition is specified, the index calculation unit 103 uses the corrected image acquisition unit as an index in a portion of the field image that is not determined to be a sunny region (a portion determined to be a shaded region). A corrected index is calculated based on the index correction image acquired in 106 (the index of the portion determined to be the sunny area is not corrected). In this case, the index calculation unit 103 calculates the NDVI by correcting the pixel values of the pixels in the shaded area indicated by the supplied shade information among the pixels indicated by the entire field image data in the same manner as described above.

As described above, in this modification, the NDVI in the sunlit area is corrected when the panel J1 is in the sun, and the NDVI in the shaded area is corrected when the panel J1 is in the shade. Thereby, the accuracy of correction using the image of panel J1 (index correction image) can be improved as compared with the case where correction is performed for all pixels without considering the sunshine conditions of panel J1. In this modification, the NDVI in the area common to the sunshine condition that has not been specified is not corrected, but another correction may be performed for the area. Another correction will be described later.

2-9 Shooting in the shaded area The area determined as the shaded area may also be a sunny area depending on the shooting time zone. Therefore, the shaded area may be photographed by changing the time zone.
FIG. 15 shows a functional configuration realized in this modification. In FIG. 15, a server device 10 c including a flight instruction unit 108 in addition to the units illustrated in FIG. 4 is illustrated.

The flight instruction unit 108 instructs the drone 20 on the shooting method for the position that is not determined to be the sunny area, that is, the position that is determined to be the shaded area. The flight instruction unit 108 is an example of the “instruction unit” in the present invention. The flight instruction unit 108 instructs the drone 20 to re-photograph the position determined to be the shaded area, for example, when the shaded area is determined from the field image captured after the shooting flight is completed.

In this modification, for example, it is assumed that the scheduled end time of farm work is input by the user terminal 30, and the server device 10c stores the scheduled end time. In this case, the flight instruction unit 108 instructs re-imaging so that the photographing is started at the timing when the photographing flight of the shaded area is completed before the scheduled end time. The flight instruction unit 108 calculates the time (shooting time) required for shooting the shaded area based on the area of the shaded area and the distance from the shooting start position, and is more than the time that is back from the estimated shooting time by the calculated shooting time. Instruct to start re-shooting at the previous time.

As a result, some of the positions that were the shaded areas at the time of re-shooting are positions that are the sunny areas. Therefore, the NDVI calculated from the pixels in the sunny area is increased as compared with the case where the re-shooting instruction is not given. be able to. When the work time is short or when the shooting flight is performed at the end of the farm work, it may not be possible to take time for re-photographing on the same day. In that case, the flight instructing unit 108 may instruct the drone 20 to shoot at a different shooting time on another day.

Specifically, the flight instruction unit 108, when a shaded area is determined from the field image captured after the shooting flight is completed, the imaging time, the field ID (information identifying the field), and the device ID (drone 20). Is stored). In this modification, the farmer who is the user takes the drone 20 to the field, but does not perform the shooting flight start operation himself, for example, performs a flight standby operation. The drone 20 transmits to the server device 10c state data indicating that it is in the flight standby state, the field ID (stored in advance by the farm worker), and the device ID.

When the flight instruction unit 108 receives the state data, the drone 20 instructs the flight instruction unit to instruct the shooting start time to be a time different from the shooting time stored in association with the field ID and the device ID indicated by the state data. Send to. As the imaging start time, for example, a time separated from a past imaging time by a predetermined time (a time when the shaded area sufficiently changes) is used. For example, if the past shooting time is 10:00 am and the time determined at 5 am is 5 hours, the flight instruction unit 108 instructs the shooting start time at 3 pm.

Also, if the past shooting time is 4 pm and the time determined at 5 pm is 5 hours, the flight instruction unit 108 gives an instruction with 11:00 am as the shooting start time. The flight control unit 201 of the drone 20 starts shooting flight when the shooting start time indicated by the received instruction data comes. Also in this case, since a part of the position that was the shaded area at the time of re-shooting in the past can be taken as the sunny area, NDVI calculated from the pixels in the sunny area is increased as compared with the case where the shooting time change instruction is not given. be able to.

2-10 Index Correction by Fixed Image The index calculation unit 103 corrects an index calculated from a pixel in a shaded area (an index in the shaded area) to an index that is expected to be calculated when the pixel is a sunny area. May be. The index calculation unit 103 performs this correction by comparing, for example, the pixel value of the image taken when the same crop is in the sunny area and the pixel value of the image taken in the shaded area.

In this modification, when there is an area that is both a sunny area and a shaded area in the field, an imaging unit that captures the area is installed.
FIG. 16 shows an example of the photographing range of the photographing means installed in this modification. In FIG. 16, an entire field image E1 of the field A1 is shown. The entire farm field image E1 is an image taken at an early time in the afternoon in which the shadow (shade region G1) of the forest R1 shown in FIG. 5 extends north.

When this field A1 is photographed early in the morning when the sun is east, the shadow of the forest R1 becomes shorter than the shaded area G1 as in the shaded area G2 shown in the figure. In the present modification, the fixed camera K1 is installed at a position where the photographing region C41 that is shaded in the shaded region G1 but hardly includes the shaded region G2 is photographed. The fixed camera K1 has a predetermined interval in the morning (time when the shooting region C41 becomes the sunny region) and a predetermined time during the day (time when the shooting region C41 becomes the shaded region) (may be every day). Repeat every week).

In this way, the fixed camera K1 captures a fixed area where the sunlit area and the shaded area are switched in the field. The fixed camera K1 is an example of the “photographing means” in the present invention.
FIG. 17 shows a functional configuration realized in this modification. In FIG. 17, a server device 10 d that includes an image acquisition unit 109 in addition to the units illustrated in FIG. 4 is illustrated. The fixed camera K1 has a communication function, and transmits image data indicating a captured image to the server device 10d.

The image acquisition unit 109 acquires an image indicated by the transmitted image data, that is, an image of a fixed area (a fixed area in which the sunny area and the shaded area in the field are switched) captured by the fixed camera K1. The image acquisition unit 109 is an example of the “second image acquisition unit” in the present invention. The image acquisition unit 109 supplies the acquired image of the fixed area to the index calculation unit 103.

The index calculation unit 103 is a part of the field image that is not determined to be the sunny area based on the correlation between the index of the sunny area and the index of the shaded area obtained from the image of the fixed area acquired by the image acquisition unit 109. An index corrected for (a portion determined to be a shaded area) is calculated. For example, the index calculation unit 103 calculates the ratio (index ratio) between the NDVI calculated for the pixel in which the specific part of the crop in the field is captured in the sunny area and the NDVI calculated for the pixel in the shaded area. calculate.

Since NDVI is calculated from a value of −1.0 to +1.0, the index calculation unit 103 converts the index value, for example, from 0.0 to 2.0, and then converts the index ratio (from 0 to 1.0). Value). For example, the index calculation unit 103 obtains an expression representing the correlation between the pixel value in the shaded area and the index ratio.
FIG. 18 shows an example of the correlation of the index ratio. FIG. 18 shows a graph in which the pixel value in the shaded area is shown on the horizontal axis and the index ratio is shown on the vertical axis.

In this example, a correlation is shown in which the index ratio decreases as the pixel value increases. The index calculation unit 103 obtains an approximate expression indicating this correlation using a known method. In the example of FIG. 18, the correlation is represented linearly, but may be represented by a quadratic curve or may be represented by a cubic or higher curve. The index calculation unit 103 calculates the corrected NDVI for each pixel in the shaded area, using an expression representing the correlation thus obtained.

Specifically, the index calculation unit 103 is expected to calculate when the pixel is in the sunny area by dividing the pixel value by the index ratio indicated by this expression with respect to the pixel value of each pixel. To the correct index. By calculating the corrected index of the shaded area as described above, even if there is an area that hardly becomes the sunny area throughout the day, such as the shaded area G2 shown in FIG. Thus, it can be shown with higher accuracy than in the case of not performing the correction of the present modification.

2-11 Index Correction by Boundary Image The index calculation unit 103 may correct the index of the shaded area by a method different from the above modification. In this modification, the index calculation unit 103 includes a portion determined to be a sunny region and a portion not determined to be a sunny region (a portion determined to be a shaded region) from the field image acquired by the field image generation unit 101. Correction is performed focusing on the boundary portion.

Based on the correlation between the index of the sunny area and the index of the shade area within the predetermined range of the boundary portion, the index calculation unit 103 determines a portion (determined as a shade area) of the field image that is not determined as the sunny area. Calculated index) is calculated. Specifically, for example, the index calculation unit 103 identifies, as a boundary pixel, a pixel that is a boundary line between the sunny area and the shaded area determined by the sunny area determination unit 102, and is adjacent to the sunny area side of the boundary pixel. NDVI is compared between a sunny pixel (an example of a sunny region within a predetermined range) and a shaded pixel adjacent to the shaded region side of the boundary pixel (an example of a shaded region within a predetermined range).

FIG. 19 shows an example of a sunny pixel and a shaded pixel. In FIG. 19, in the field whole image E1 of the field A1, the sunny pixel Df11 on the sunny area F1 side and the shaded pixel Dg11 on the shaded area G1 side are represented. A plurality of sunny pixels and shaded pixels are represented along the boundary line between the sunny region F1 and the shaded region G1. In the figure, for the sake of easy understanding, not all the sunny pixels and shaded pixels are shown, but sunny pixels and shaded pixels are included between these sunny pixels and shaded pixels.

The index calculation unit 103 calculates NDVI for each of these sunlit pixels and shaded pixels, and obtains an expression indicating the correlation between the index ratio and the pixel value of the shaded pixel, as in the example of FIG. After that, the index calculation unit 103 calculates the corrected NDVI of the shaded area in the same manner as in the modified example. In the example of FIG. 19, the pixels adjacent to the boundary pixels are used as the sunny area and the shaded area (pixels) within a predetermined range of the boundary pixels, but the present invention is not limited to this. Pixels located at a distance between one or more pixels from the boundary pixel may be used as the sunny area and the shade area. In short, a range in which the growth situation seems to be substantially uniform in the entire field image may be used as the predetermined range.

As described above, if the pixel is adjacent to the boundary line between the sunny area and the shaded area, there is a high possibility that the crop growth situation is similar. For this reason, the NDVI expected to be calculated when both the sunny pixel and the shaded pixel are in the sunny region is likely to be a close value. Therefore, similarly to the above-described modification, the index of the pixel in the shaded area can be corrected to an index that is expected to be calculated when the pixel is the sunny area. Moreover, in this modification, since only the field image which the drone 20 image | photographs is used, the said correction | amendment can be performed even if it does not install the fixed imaging | photography means as shown in FIG.

2-12 Imaging Device The imaging device provided in the drone 20 is not limited to the above. For example, it may have a zoom function (the resolution can be increased to improve the accuracy of NDVI), or it may have sensitivity specialized for red and infrared. In that case, for example, when the sunny area determination unit 102 determines the sunny area, the pixel values of light (blue, green) of other wavelengths may be restored by correcting the spectrum.

2-13 Output of Hinge Index and Shade Index The growth information recording unit 105 may output the sunny index and the shade index by a method different from the embodiment. The growth information recording unit 105 outputs, for example, a sunny indicator for a folder or database prepared for a sunny indicator, and outputs a shaded indicator for a folder or database prepared for a shade indicator, for example. Good.

Further, the growth information recording unit 105 not only outputs the growth information to the user terminal 30, but also, for example, a storage device that accumulates a sunny index and a shade index, an analysis of a growth situation and a future prediction from the sunny index and the shade index, etc. May be output to an analysis apparatus that performs the above and a visualization apparatus that performs a visualization process (such as generation of a graph and a map) that facilitates comparison of the growth status. In short, the growth information recording unit 105 may output the growth information to any output destination as long as it leads to supporting the person who performs the work in the field.

2-14 Apparatus for Implementing Each Unit The apparatus for implementing each function shown in FIG. 4 and the like may be different from those shown in FIG. For example, the drone may have all or some of the functions of the server device. In that case, the drone processor is an example of the “information processing apparatus” of the present invention. Further, the user terminal 30 may realize the function of the server device. In that case, the user terminal 30 is an example of the “information processing apparatus” of the present invention.

Also, the operation performed by each function may be performed by another function or may be performed by a new function. For example, the growth information generation unit 104 may perform the operation performed by the index calculation unit 103 (index calculation operation). Moreover, the output part which newly provided the output of the sunny index and shade index which the growth information recording part 105 performs may perform. Two or more devices may realize each function provided in the server device. In short, as long as these functions are realized as the entire agricultural support system, the agricultural support system may include any number of devices.

2-15 Category of Invention In addition to the information processing apparatus such as the server apparatus and the user terminal described above and a flying object such as a drone (the drone may also serve as the information processing apparatus), the present invention It can also be understood as an information processing system such as an agricultural support system equipped with a flying object. In addition, the present invention can be understood as an information processing method for realizing processing performed by each device, or as a program for causing a computer that controls each device to function. This program may be provided in the form of a recording medium such as an optical disk in which it is stored, or may be provided in the form of being downloaded to a computer via a network such as the Internet, installed and made available for use. May be.

2-16 Processing Procedures, etc. The processing procedures, sequences, flowcharts, etc. of the embodiments described in this specification may be switched in order as long as there is no contradiction. For example, the methods described herein present the elements of the various steps in an exemplary order and are not limited to the specific order presented.

2-17 Handling of Input / Output Information, etc. Input / output information, etc. may be stored in a specific location (for example, a memory) or managed by a management table. Input / output information and the like can be overwritten, updated, or additionally written. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

2-18 Software Software, whether it is called software, firmware, middleware, microcode, hardware description language, or another name, is an instruction, instruction set, code, code segment, program code, program, sub, It should be interpreted broadly to mean a program, a software module, an application, a software application, a software package, a routine, a subroutine, an object, an executable file, an execution thread, a procedure, a function, and the like.

Further, software, instructions, etc. may be transmitted / received via a transmission medium. For example, software may use websites, servers, or other devices using wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission media.

2-19 Information, Signals The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, commands, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these May be represented by a combination of

2-20 System, Network As used herein, the terms “system” and “network” are used interchangeably.

2-21 Meaning of “Based on” As used herein, the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” means both “based only on” and “based at least on.”

2-22 “and”, “or”
In the present specification, for a configuration that can be implemented by either “A and B” or “A or B”, a configuration described in one expression may be used as a configuration described in the other expression. For example, when “A and B” is described, it may be used as “A or B” as long as it is feasible without inconsistency with other descriptions.

2-23 Variations of Embodiments Each embodiment described in the present specification may be used alone, in combination, or may be used by switching according to execution. In addition, notification of predetermined information (for example, notification of being “X”) is not limited to explicitly performed, but is performed implicitly (for example, notification of the predetermined information is not performed). Also good.

Although the present invention has been described in detail above, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the scope of claims. Therefore, the description of the present specification is for illustrative purposes and does not have any limiting meaning to the present invention.

DESCRIPTION OF SYMBOLS 1 ... Agriculture support system, 10 ... Server apparatus, 20 ... Drone, 30 ... User terminal, 101 ... Agricultural field image generation part, 102 ... Hyuga area determination part, 103 ... Index calculation part, 104 ... Growth information generation part, 105 ... Growth Information recording unit 106 ... corrected image acquisition unit 107 ... sunshine condition specifying unit 108 ... flight instruction unit 109 ... image acquisition unit 201 ... flight control unit 202 ... flight unit 203 ... sensor measurement unit 204 ... photographing Department.

Claims

A determination unit for determining a sunny area of the field included in the captured image of the field;
An index acquisition unit that acquires an index indicating a growth status of the crop in the determined hyuga area as a sunny index;
An information processing apparatus comprising: an output unit that outputs the acquired Hinata index as an index of the Hinata area.
The index acquisition unit acquires, as a shade index, an index indicating the growth status of the crop in a shade area that is not determined to be the sunny area,
The information processing apparatus according to claim 1, wherein the output unit outputs the acquired shade index as an index of the shade region.
A first image acquisition unit for acquiring an image of a field imaged by a flying object including an imaging unit;
A calculation unit that calculates an index indicating the growth status of the crop from the acquired image of the field,
The index obtaining unit obtains, as the sunny index, a value calculated for the sunny area from the indices calculated for the entire image of the field, or calculates only the sunny area of the farm image. The information processing apparatus according to claim 1, wherein the obtained index is acquired as the Hinata index.
A correction acquisition unit that acquires an image for index correction captured by the flying object;
A condition specifying unit that specifies whether the shooting condition of the image for index correction is sunny or shaded. The calculation unit, when the shooting condition for the sunny day is specified, the image of the field. The information processing apparatus according to claim 3, wherein the index corrected based on the acquired index correction image is calculated as the index in the portion determined to be the sunny area.
The calculation unit is based on the acquired index correction image as the index in the portion of the image that is not determined to be the sunny area in the image when the shaded shooting condition is specified. The information processing apparatus according to claim 4, wherein the corrected index is calculated.
The information processing apparatus according to any one of claims 3 to 5, further comprising an instruction unit that instructs the flying object to re-photograph a shaded area that is not determined to be the sunny area or to change the shooting time.
A second image acquisition unit that acquires an image of the fixed area captured by an image capturing unit that captures a fixed area where the sunny area and the shaded area of the field are switched;
The calculation unit is a part of the image of the field that is not determined to be the sunny area based on a correlation between the index of the sunny area and the index of the shaded area obtained from the acquired image of the fixed area The information processing apparatus according to claim 3, wherein the index corrected for is calculated.
The calculation unit is an index of the sunny area that is within a predetermined range of a boundary between a portion determined as the sunny area from the acquired image of the field and a shaded area not determined as the sunny area. The information processing apparatus according to any one of claims 3 to 7, wherein the index corrected for the portion of the shade area in the image of the field is calculated based on a correlation between the index of the shade area and the shade area. .