WO2021255940A1 - Drone for diagnosing crop growth, and camera system for same - Google Patents

Abstract

Provided is a drone which, in accordance with the flying state of the drone, performs appropriate management relating to pre-processing of captured images or transmission of pre-processed data to the outside, and a drone provided with an appropriate means for retaking an image if an invalid image is extracted. This drone is provided with a main body, a plurality of rotating blades, a camera, a transmitting unit for transmitting data generated from an image acquired from the camera to the outside, and a storage device, wherein, if the flying state of the drone satisfies a predetermined condition, the drone performs at least one of: flagging the image as an invalid image; stopping storage of the image to the storage device, stopping pre-processing of the image, or stopping transfer of the data to the outside; and deleting the image from the storage device.

Description

A drone that diagnoses the growth of crops and its camera system

The present invention relates to a drone for diagnosing the growth of crops and a camera system thereof.

As a background technology in this technical field, there is International Publication No. 2017/221641 (Patent Document 1). In this publication, "a plant growth index measuring device for generating a growth index indicating the degree of growth in a plant, a plant growth index measuring device for generating an image of an image of a field to be measured from above, a method thereof, and a predetermined program thereof. An image is generated at time intervals, and it is determined whether or not to store the generated image in the image storage unit so that only a significant image is stored in the image storage unit in order to obtain the growth index, and the result is , The generated image is stored in the image storage unit only when it is determined to be stored in the image storage unit (see summary).

International Publication No. 2017/221641

The patent document 1 describes a drone that images a field. However, in the prior art, the captured image is stored when the flight conditions such as the position, altitude, speed, and attitude of the aircraft (including the drone) satisfy preset conditions, but the captured image is stored only under the above conditions. There was a risk that it would not be possible to fully judge the pros and cons of. In addition, although an image suitable for the growth diagnosis of crops in the field was selected and stored, there was a possibility that sufficient measures were not taken except for excluding it from the preservation target. Further, when an invalid image is extracted, there is a possibility that the means for retrieving the image is not sufficiently provided.
Therefore, the present invention provides a drone and a camera system thereof that appropriately manages the pre-processing of the captured image or the transmission of the processed data to the outside according to the flight state of the drone. Further, when an invalid image is extracted, a drone having an appropriate means for retrieving the image and a camera system thereof are provided.

In order to solve the above problems, for example, the configuration described in the claims is adopted.
The present application includes a plurality of means for solving the above problems, and to give an example thereof, a main body, a plurality of rotary blades, a camera, and data generated from an image acquired from the camera are transmitted to the outside. A drone including a transmission unit and a storage device, in which the image is flagged as an invalid image when the flight state of the drone satisfies a predetermined condition, and the storage of the image in the storage device is performed. It is characterized in that at least one of stopping, stopping the preprocessing for the image, stopping the transmission of the data to the outside, and deleting the image from the storage device is performed.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a drone and a camera system thereof that appropriately manages the preprocessing of the captured image or the transmission of the data after the preprocessing to the outside according to the flight state of the drone. .. Further, if an invalid image is extracted, a drone having an appropriate means for retrieving the image and a camera system thereof can be provided.
Issues, configurations and effects other than those described above will be clarified by the following description of the embodiments.

This is an example of a plan view of a drone. This is an example of a front view of a drone. This is an example of the right side view of the drone. This is an example of the rear view of the drone. This is an example of a perspective view of a drone. This is an example of a block diagram showing the control function of the drone. This is an example of a connection configuration diagram of the entire drone management system 700. This is an example of the field information display screen 800 displayed on the mobile terminal 701. This is an example of the drone operation screen 900 displayed on the mobile terminal 701. This is an example of the hardware configuration of the mobile terminal 701. This is an example of the hardware configuration of the management server 702. This is an example of the hardware configuration of the management terminal 703. This is an example of field management information 1300. This is an example of device management information 1400. This is an example of user management information 1500. This is an example of drug management information 1600. This is an example of energy management information 1700. This is an example of flight path management information 1800. This is an example of schedule management information 1900. This is an example showing the growth diagnosis of crops in a field using a drone separately for A and B. This is an example of a dedicated camera system equipped for drones. This is an example of a drone's flight path for diagnosing the growth of crops in the field. This is an example of the invalid image judgment processing flow of the drone. This is an example in which an invalid image was extracted during the flight of a drone that diagnoses the growth of crops in the field. This is an example of retrieving an invalid image in the field during the second drone flight. This is an example of retrieving an invalid image in the field during the second drone flight. This is an example of retrieving an invalid image in the field during the second drone flight. This is an example of retrieving an invalid image in the field during the second drone flight. This is an example of the flow of the drone invalid image determination process and the second flight route calculation process.

Hereinafter, examples will be described with reference to the drawings.
Drones are an example of agricultural machinery. In the present specification, the drone is regardless of the power means (electric power, prime mover, etc.) and the maneuvering method (wireless or wired, autonomous flight type, manual maneuvering type, etc.). It refers to all aircraft with multiple rotor blades.

FIG. 1 is an example of a plan view of a drone.
FIG. 2 is an example of a front view of the drone.
FIG. 3 is an example of a right side view of the drone.
FIG. 4 is an example of a rear view of the drone.
FIG. 5 is an example of a perspective view of the drone.

Rotors 101-1a, 101-1b, 101-2a, 101-2b, 101-3a, 101-3b, 101-4a, 101-4b (also called rotors) are means for flying the drone 100. Eight aircraft (four sets of two-stage rotor blades) are provided in consideration of the balance of flight stability, aircraft size, and power consumption. Each rotary blade 101 is arranged on four sides of the main body 110 by an arm protruding from the main body 110 of the drone 100. That is, the rotors 101-1a and 101-1b are left rearward in the traveling direction, the rotary blades 101-2a and 101-2b are forward left, the rotary blades 101-3a and 101-3b are rearward right, and the rotary blades 101- are forward right. 4a and 101-4b are arranged respectively. In addition, the drone 100 has the traveling direction facing downward on the paper in FIG. Rod-shaped legs 107-1, 107-2, 107-3, and 107-4 extend downward from the rotation axis of the rotary blade 101, respectively.

The motors 102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 102-4a, 102-4b are the rotary blades 101-1a, 101-1b, 101-2a, 101-. It is a means for rotating 2b, 101-3a, 101-3b, 101-4a, 101-4b (typically an electric motor, but may be a motor or the like), and is 1 for one rotary blade. The machine is provided. The motor 102 is an example of a propulsion device. The upper and lower rotor blades (for example, 101-1a and 101-1b) in one set and the corresponding motors (for example, 102-1a and 102-1b) have axes for the stability of drone flight and the like. They are on the same straight line and rotate in opposite directions.

As shown in FIGS. 2 and 3, the radial member for supporting the propeller guard provided so that the rotor does not interfere with foreign matter has a wobble-like structure rather than a horizontal structure. This is to encourage the member to buckle to the outside of the rotor blade in the event of a collision and prevent it from interfering with the rotor.

The drug nozzles 103-1, 103-2, 103-3 are means for spraying the drug downward, and are provided with four machines. In addition, in this specification, a drug is a liquid, powder or fine particles sprayed in a field such as a pesticide, a herbicide, a liquid fertilizer, an insecticide, a seed, and water.

The medicine tank 104 is a tank for storing the medicine to be sprayed, and is provided at a position close to the center of gravity of the drone 100 and at a position lower than the center of gravity from the viewpoint of weight balance. The medicine hoses 105-1, 105-2, 105-3 connect the medicine tank 104 and the medicine nozzles 103-1, 103-2, 103-3. The drug hose is made of a hard material and may also serve to support the drug nozzle. The pump 106 is a means for discharging the drug from the nozzle.

FIG. 6 is an example of a block diagram showing the control function of the drone.
The flight controller 501 is a component that controls the entire drone, and may be an embedded computer including a CPU, a memory, related software, and the like. The flight controller 501 is based on the input information received from the mobile terminal 701 and the input information obtained from various sensors described later, via a control means such as an ESC (Electronic Speed Control), and the motors 102-1a and 102-1b. , 102-2a, 102-2b, 102-3a, 102-3b, 104-a, 104-b to control the flight of the drone 100. The actual rotation speeds of the motors 102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 104-a, 104-b are fed back to the flight controller 501, and normal rotation is performed. It is configured so that it can be monitored. Alternatively, the rotary blade 101 may be provided with an optical sensor or the like so that the rotation of the rotary blade 101 is fed back to the flight controller 501.

The software used by the flight controller 501 can be rewritten through a storage medium or the like for function expansion / change, problem correction, etc., or through communication means such as Wi-Fi communication or USB. In this case, protection is performed by encryption, checksum, digital signature, virus check software, etc. so that rewriting by unauthorized software is not performed. Further, a part of the calculation process used by the flight controller 501 for control may be executed by another computer located on the mobile terminal 701, the management server 702, or somewhere else. Due to the high importance of the flight controller 501, some or all of its components may be duplicated.

The flight controller 501 communicates with the mobile terminal 701 via the Wi-Fi slave unit function 503 and further via the base station 710, receives necessary commands from the mobile terminal 701, and receives necessary information from the mobile terminal. Can be sent to 701. In this case, the communication may be encrypted to prevent fraudulent acts such as interception, spoofing, and device hijacking. The base station 710 also has a function of an RTK-GPS base station in addition to a communication function by Wi-Fi. By combining the signal from the RTK base station and the signal from the GPS positioning satellite, the flight controller 501 can measure the absolute position of the drone 100 with an accuracy of about several centimeters. Since the flight controller 501 is so important, it may be duplicated / multiplexed, and each redundant flight controller 501 should use a different satellite in order to cope with the failure of a specific GPS satellite. It may be controlled. Communication between the flight controller 501, the base station 710, and the mobile terminal 701 may use a mobile network such as LTE instead of Wi-Fi.

The 6-axis gyro sensor 505 measures the acceleration of the drone aircraft in three directions orthogonal to each other. Furthermore, the velocity is calculated by integrating the acceleration. The 6-axis gyro sensor 505 measures the change in the attitude angle of the drone aircraft in the above-mentioned three directions, that is, the angular velocity. The geomagnetic sensor 506 measures the direction of the drone body by measuring the geomagnetism. The barometric pressure sensor 507 can also measure barometric pressure and indirectly measure the altitude of the drone. The laser sensor 508 measures the distance between the drone body and the ground surface by utilizing the reflection of the laser light, and may be an IR (infrared) laser.

Sonar 509 measures the distance between the drone aircraft and the ground surface using the reflection of sound waves such as ultrasonic waves. These sensors may be selected according to the cost target and performance requirements of the drone. Further, a gyro sensor (angular velocity sensor) for measuring the inclination of the airframe, a wind power sensor for measuring wind power, and the like may be added. Further, these sensors may be duplicated or multiplexed. If there are multiple sensors for the same purpose, the flight controller 501 may use only one of them, and if it fails, it may switch to an alternative sensor for use. Alternatively, a plurality of sensors may be used at the same time, and if the measurement results do not match, it may be considered that a failure has occurred.

The flow rate sensor 510 measures the flow rate of the drug, and is provided at a plurality of locations on the route from the drug tank 104 to the drug nozzle 103. The liquid drain sensor 511 is a sensor that detects that the amount of the drug is equal to or less than a predetermined amount. The multispectral camera 512 is a means of photographing the field 720 and acquiring data for image analysis. The obstacle detection camera 513 is a camera for detecting an obstacle, and is a device different from the multispectral camera 512 because the image characteristics and the lens orientation are different from those of the multispectral camera 512.

Switch 514 is a means for the user of the drone 100 to make various settings. The obstacle contact sensor 515 is a sensor for detecting that the drone 100, particularly its rotor or propeller guard portion, has come into contact with an intruder such as an electric wire, a building, a human body, a standing tree, a bird, or another drone. .. The obstacle contact sensor 515 may be replaced with a 6-axis gyro sensor 505. The cover sensor 516 is a sensor that detects that the operation panel of the drone 100 and the cover for internal maintenance are in an open state.

The drug injection port sensor 517 is a sensor that detects that the injection port of the drug tank 104 is in an open state. These sensors may be selected according to the cost target and performance requirements of the drone, or may be duplicated / multiplexed. Further, a sensor may be provided at a base station 710, a mobile terminal 701, or another place outside the drone 100, and the read information may be transmitted to the drone 100. For example, a wind power sensor may be provided in the base station 710 to transmit information on the wind power and the wind direction to the drone 100 via Wi-Fi communication.

The flight controller 501 transmits a control signal to the pump 106 to adjust the drug discharge amount and stop the drug discharge. The current state of the pump 106 (for example, the number of revolutions) is fed back to the flight controller 501.

The LED 107 is a display means for notifying the drone operator of the state of the drone. Display means such as a liquid crystal display may be used in place of or in addition to the LED. The buzzer 518 is an output means for notifying the state of the drone (particularly the error state) by an audio signal. The Wi-Fi slave unit function 519 is an optional component for communicating with an external computer or the like for transferring software, for example, in addition to the mobile terminal 701. Instead of or in addition to the Wi-Fi handset function, other wireless communication means such as infrared communication, Bluetooth®, ZigBee®, NFC, or wired communication means such as USB connection. You may use it. Further, communication between each device of the flight controller 501, the mobile terminal 701, and the base station 710 can be communicated with each other by a mobile communication system such as 3G, 4G, and LTE instead of the Wi-Fi slave unit function. May be good.

The speaker 520 is an output means for notifying the state of the drone (particularly the error state) by means of recorded human voice, synthetic voice, or the like. Depending on the weather conditions, it may be difficult to see the visual display of the drone 100 in flight. In such a case, it is effective to convey the situation by voice. The warning light 521 is a display means such as a strobe light for notifying the state of the drone (particularly the error state). These input / output means may be selected according to the cost target and performance requirements of the drone, and may be duplicated / multiplexed.

FIG. 7 is an example of a connection configuration diagram of the entire drone management system 700.
The drone management system 700 includes a drone 100, a mobile terminal 701, a management terminal 703, and a base station 710, each of which is connected to the management server 702 via a network. The network may be wired or wireless, and each terminal can send and receive information via the network.
The drone 100 and the mobile terminal 701 can communicate with each other in the field 720 via the base station 710, and the drone 100 performs a drug spraying flight.

The network may be a network that communicates according to one communication standard, or may be a network that is a combination of a plurality of communication standard networks. For example, the drone 100 and the mobile terminal 701 may be network-connected by Wi-Fi provided by the base station 710, respectively, or the drone 100 and the mobile terminal 701 may be network-connected by a mobile communication network such as LTE, respectively. Further, the drone 100 may be connected by Wi-Fi provided by the base station 710, and the base station 710 and the mobile terminal 701 may be connected by a mobile communication network.

The mobile terminal 701 sends a command to the drone 100 by the operation of the user, and also displays information received from the drone 100 (for example, position, drug amount, remaining battery level, camera image, etc.). For example, it is realized by a mobile information device such as a tablet terminal or a smartphone. The drone 100 performs autonomous flight according to instructions from the management server 702, but the mobile terminal 701 can perform manual operations during basic operations such as takeoff and return, and in an emergency. The mobile terminal 701 is connected to the base station 710, and can communicate with the management terminal 703 via the base station 710 or directly.

The management server 702 is, for example, a server arranged on the cloud, calculates the spray flight route of the drone 100 based on the field management information 1300, and controls the independent flight of the drone 100. In addition, it is possible to collect information acquired from a camera mounted on the drone 100 and various sensors, and perform various analyzes such as the state of fields and crops.
The management terminal 703 is a terminal that operates the management server 702, and makes various settings for the management server 702. It is also possible to control the drone 100 and the mobile terminal 701.

The base station 710 is a device installed in the field 720 that provides a master unit function for Wi-Fi communication, and also functions as an RTK-GPS base station, so that the accurate position of the drone 100 can be provided. (The base unit function of Wi-Fi communication and the RTK-GPS base station may be independent devices). Further, the base station 710 can communicate with the management server 702 using a mobile communication network such as 3G, 4G, and LTE.

Each terminal and management server 702 of the drone management system 700 may be, for example, a mobile terminal (mobile terminal) such as a smartphone, a tablet, a mobile phone, or a personal digital assistant (PDA), or may be a glasses type, a wristwatch type, a clothing type, or the like. It may be a wearable terminal of. It may also be a stationary or portable computer, or a server located in the cloud or on a network. Further, the function may be a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or an MR (Mixed Reality) terminal. Alternatively, it may be a combination of these plurality of terminals. For example, a combination of one smartphone and one wearable terminal can logically function as one terminal. Further, it may be an information processing terminal other than these.

Each terminal and management server 702 of the drone management system 700 has a processor (control unit) that executes an operating system, an application, a program, etc., a main storage device such as a RAM (RandomAccessMemory), and an IC card or a hard disk drive. , SSD (Solid State Drive), auxiliary storage device such as flash memory, communication control unit such as network card, wireless communication module, mobile communication module, touch panel, keyboard, mouse, voice input, motion detection by imaging of camera unit It is equipped with an input device such as an input device and an output device such as a monitor and a display. The output device may be a device or a terminal for transmitting information for output to an external monitor, display, printer, device, or the like.

Various programs and applications (modules) are stored in the main memory, and each functional element of the entire system is realized by the processor executing these programs and applications. In addition, each of these modules may be implemented by hardware by integrating them. Further, each module may be an independent program or application, but may be implemented in the form of a part of a subprogram or a function in one integrated program or application.

In this specification, each module is described as a subject (subject) that performs processing, but in reality, a processor that processes various programs, applications, etc. (module) executes processing.
Various databases (DBs) are stored in the auxiliary storage device. A "database" is a functional element (storage unit) that stores a data set so that it can handle arbitrary data operations (for example, extraction, addition, deletion, overwriting, etc.) from a processor or an external computer. The method of implementing the database is not limited, and may be, for example, a database management system, spreadsheet software, or a text file such as XML or JSON.
The mobile terminal 701 may be referred to as an information processing device, and the management server 702 may be referred to as an information processing device.

FIG. 8 is an example of the field information display screen 800 displayed on the mobile terminal 701.
The screen display module 1011 of the mobile terminal 701 acquires the map information 1200 and the field management information 1300 stored in the mobile terminal 701, generates the field information display screen 800, and outputs the field information display screen 800 to the output device 1005 such as a screen.
The screen display module 1011 may be configured to acquire the map information 1200 or 1200 stored in the management server 702 and the field management information 1300 via the network to generate the field information display screen 800.

A map 801 is displayed on the back surface of the field information display screen 800, indicating that the information is registered in the fields 802, 803, and 804 in which the field information is stored in the field management information 1300. Anchor 805 is displayed.
The field is a rice field, a field, or the like that is the target of chemical spraying by the drone 100. In reality, the terrain of the field is complicated, and the topographic map may not be available in advance, or the topographic map and the situation at the site may be inconsistent. Fields are usually adjacent to houses, hospitals, schools, other crop fields, roads, railroads, etc. In addition, there may be intruders such as buildings and electric wires in the field. The field is an example of a target area for chemical spraying.

When the screen display module 1011 receives the selection of the field 802 from the user via the input device 1004 by tapping the screen or the like, the screen display module 1011 acquires the information corresponding to the field 802 from the field management information 1300 and displays it in the field information display area 810. do. Further, the screen display module 1011 displays a highlight indicating that the field 802 is selected, such as changing the periphery of the selected field 802 to a thick line of a bright color.

In the field information display area 810, information acquired from the field management information 1300, such as the field name 811, the address 812, the area 813, and the planted crop name 814, is displayed.
Information related to the spraying of the drug is displayed in the spraying information display area 820. The drug to be sprayed changes depending on the crop name 814 and the spraying time, and the drug information to be sprayed in the near future is acquired from the drug management information 1600 and displayed.
In the spraying information display area 820, information related to the spraying of the drug acquired or calculated by the spraying-related information management module 1114 of the management server 702, for example, the drug name, spraying amount, dilution amount, and energy amount required for the spraying flight in the field. Etc. are displayed.
In the state 830, information such as "surveyed" or "with flight path" is displayed as the current state for the selected field 802.
Information on the latest spray flight date and time is displayed in the latest flight date and time 840.
The flight status display field 850 displays the current status of the drone's spray flight.

Compass 861 indicates the orientation displayed by map 801.
When the field-wide display button 862 is selected, the screen display module 1011 changes the scale of the display so that the selected field fills the screen.
When the current location movement button 863 is selected, the screen display module 1011 changes the display so that the current location acquired by the GPS of the mobile terminal 701 becomes the center of the screen.
When the schedule display button 870 is selected, the screen display module 1011 displays the drug spraying schedule for the day.

FIG. 9 is an example of the drone operation screen 900 displayed on the mobile terminal 701.
The drone battery display 901 displays the current remaining battery level of the drone.
At the drone position 902, the current position information of the drone 100 is displayed.
The spray flight progress information 912 displays the progress information of the current spray flight. For example, the progress of the flight route of the spray flight, the remaining amount of the sprayed drug, the remaining battery level, etc. are displayed.
In the flight status display field 921, the current status of the spray flight of the drone 100 is displayed.
In the message display field 922, a message indicating the communication content with the drone 100, the flight status, and the like is displayed.

The altitude change buttons 923 and 924 are buttons for changing the flight altitude of the drone 100. Press the minus to lower the altitude, and press the plus to raise the altitude.
The emergency stop button 925 is a button for urgently stopping the flying drone 100, and in addition to a temporary stop for hovering on the spot, an option for returning to the flight start point, an option for urgently stopping the motor on the spot, etc. Can also be displayed.
In the example of the drone operation screen 900, the field 930 to be sprayed with the chemical is displayed on the map, and the flight route 931 of the spraying flight on the field 930 is displayed. The drone 100 sequentially flies at the designated flight coordinates according to the flight route management information 1800 stored in the mobile terminal 701 or the management server 702.

Upon receiving operations that require operations on the drone 100, such as the altitude change buttons 923 and 924 and the emergency stop button 925, the drone operation module 1012 sends information such as commands corresponding to these operations to the drone 100. The drone 100 can be operated.
The next spraying schedule display button 940 is a button for displaying the schedule of the next spraying flight of the currently executed spraying flight. When this button is pressed, information about the next spray flight obtained from Schedule Management Information 1900 is displayed.

FIG. 10 is an example of the hardware configuration of the mobile terminal 701.
The mobile terminal 701 is, for example, a terminal such as a tablet, a smartphone, or a head-mounted display.
Programs and applications such as a screen display module 1011 and a drone operation module 1012 and a schedule management module 1013 are stored in the main storage device 1001, and each of the mobile terminals 701 is executed by the processor 1003 by executing these programs and applications. Functional elements are realized.
The screen display module 1011 displays the field information display screen 800 and the drone operation screen 900 on an output device 1005 such as a display panel.

When the drone operation module 1012 receives operations such as the altitude change buttons 923 and 924 and the emergency stop button 925 by the user, the drone operation module 1012 transmits information such as commands corresponding to these operations to the drone 100, and makes a drone flight. Manipulate.
The schedule management module 1013 manages the schedule of each spray flight when the spray flights are continuously performed in a plurality of fields.
The auxiliary storage device 1002 stores various information such as map information 1200, field management information 1300, device management information 1400, user management information 1500, drug management information 1600, energy management information 1700, flight route management information 1800, and schedule management information 1900. Remember.

FIG. 11 is an example of the hardware configuration of the management server 702.
The management server 702 is composed of, for example, a server arranged on the cloud.
The main storage device 1101 stores a screen output module 1111, a flight management module 1112, a user / equipment management module 1113, a spray-related information management module 1114, a flight route management module 1115, and a schedule management module 1116. Each functional element of the management server 702 is realized by executing the application or the application by the processor 1103.
The screen output module 1111 extracts and generates information for displaying the field information display screen 800 and the drone operation screen 900, and transmits the information to the mobile terminal 701. The screen information itself may be generated and displayed on the mobile terminal 701 or the like.
The flight management module 1112 manages the spray flight of the drone 100 based on the information such as the field management information 1300 and the flight route management information 1800.

The user / device management module 1113 registers and manages information about a user who uses the drone 100 in the user management information 1500.
The spraying-related information management module 1114 manages the amount of chemicals required for the spraying flight, the amount of chemicals, the amount of dilution, the amount of water required for dilution, the amount of energy such as the number of batteries, and the like.
The flight path management module 1115 calculates the flight path of the spray flight of the drone 100 based on the field management information 1300.
The schedule management module 1116 generates and manages a schedule of spray flights over a plurality of fields and a plurality of days. The generated drug spraying schedule is stored in the schedule management information 1900.

The auxiliary storage device 1102 stores various information such as map information 1200, field management information 1300, device management information 1400, user management information 1500, drug management information 1600, energy management information 1700, flight route management information 1800, and schedule management information 1900. Remember.
The same information is stored in the mobile terminal 701 and the management server 702, but the respective information may be synchronized with each other, or either information may be simply copied. Further, some or all of the information may be stored in the management server 702, and the information may be downloaded from the management server 702 from the mobile terminal 701 as needed.

FIG. 12 is an example of the hardware configuration of the management terminal 703.
The management terminal 703 is, for example, a terminal such as a desktop PC, a notebook PC, or a tablet.
Programs and applications such as the drone setting module 1211 and the management server setting module 1212 are stored in the main storage device 1201, and each functional element of the management terminal 703 is realized by executing these programs and applications by the processor 1203. Will be done.
The drone setting module 1211 performs various operations and settings such as spray flight setting and initial setting of the drone 100.
The management server setting module 1212 makes various settings such as initial settings of the management server 702.
The auxiliary storage device 1202 stores various information such as drone setting information 1221 and management server setting information 1222.

FIG. 13 is an example of field management information 1300.
The field management information 1300 stores various information about the field to which the chemicals are sprayed, and stores information such as the field ID, the field name, the field position, the field peripheral coordinates, the field area, and the crops cultivated. The field management information 1300 may be simply referred to as field information.
The field ID is identification information that uniquely identifies the field.
The field position 1311 indicates the position coordinates of the field, and has, for example, information on the latitude and longitude of the center of the field.
The field peripheral coordinates 1312 indicate the coordinates around the field, and are, for example, the position coordinates of the four corners in the case of a quadrangular field. The sample value GC007 indicates a link to information in which the position coordinates are continuously stored separated by commas or the like.
The field area 1313 is the total area of the field corresponding to the field ID.
The cultivated crop 1314 stores information for identifying the crop or the like cultivated in the field.

FIG. 14 is an example of the device management information 1400.
The device management information 1400 stores information for managing the drone 100, and stores information such as a device ID, a device name, a model number, specifications, a user, energy, and flight time.
The device ID is identification information that uniquely identifies the drone 100.
The user is information on the user who is currently using the drone 100, and stores the user ID of the user management information 1500.
The energy 1411 is information about energy that can be mounted on the drone 100, and stores the energy ID of the energy management information 1700.
The flightable time 1412 indicates the flightable time due to the energy that can be mounted on the drone 100. For example, information such as being able to fly for 15 minutes with a set of two batteries is stored.

FIG. 15 is an example of user management information 1500.
The user management information 1500 stores information on the user who operates the drone 100, and stores information such as a user ID, a user display ID, a name, an e-mail address, a date of birth, and a gender.
The user ID is identification information that uniquely identifies the user.
The user display ID is user information displayed on the mobile terminal 701 or the like, and is, for example, a nickname registered by the user.

FIG. 16 is an example of drug management information 1600.
The drug management information 1600 stores information on the drug to be sprayed, and stores the drug ID, drug name, product number, specifications, dilution rate, spray amount, and the like.
The drug ID is identification information that uniquely identifies the drug.
The drug name 1602 indicates the name of a product such as a liquid, powder or fine particle to be sprayed in a field such as a pesticide, a herbicide, a liquid fertilizer, an insecticide, or a seed.

The specification 1603 stores information such as a method of using the drug, a method of diluting the drug, a target crop, and a method of spraying, and the drug is diluted or sprayed according to the contents described in the specification 1603.
The dilution ratio 1604 stores the ratio of diluting the drug, for example, the ratio of the drug to water, the amount of the drug and water used for dilution, and the like.
The spraying amount 1605 stores the sprayed amount of the diluted post-diluted drug (spraying drug). For example, it has been shown to spray 10 L of spraying agent per ha.

FIG. 17 is an example of energy management information 1700.
The energy management information 1700 stores information on energy such as a battery required for the flight of the drone 100, and stores information such as an energy ID, an energy name, a model number, a type, and specifications.
The energy ID is identification information that uniquely identifies the energy.
The type indicates the type of energy, and for example, a battery, gasoline, jet fuel, or the like is stored.

FIG. 18 is an example of flight path management information 1800.
The flight route management information 1800 stores information indicating the flight route of the drone 100, and stores the route ID, the target ID, the route coordinates, the total route distance, and the like.
The route ID is identification information that uniquely identifies the flight route.
The target ID is information for specifying the field for which the flight route is calculated, the movement route between the fields, and the like. For example, farm003 indicates that the subject is in the field, and route002 indicates that the subject is a movement route outside the field.
The route coordinate 1811 is a link to information indicating the route coordinate of the flight, and the route coordinate of the flight is represented by, for example, a combination of a plurality of continuous position coordinates. As the position coordinates, a combination of latitude and longitude, a combination of latitude, longitude and altitude, etc. can be considered.
The total route distance 1812 indicates the total distance of the route when the entire flight route from the start of the flight to the schedule is flown.

FIG. 19 is an example of schedule management information 1900.
The schedule management information 1900 is information that defines a schedule when a plurality of fields are sprayed, and stores information such as a schedule ID, a schedule name, a date and time, a start place, and a schedule.
The schedule 1901 stores information for specifying the field where the spray flight is performed, the movement route between the fields, and the like. For example, in the example of the sample value, after flying two fields specified by farm006 and farm005, after flying the movement route indicated by route001, flying the field specified by farm003, and others specified by other001. It is a schedule to fly the field specified by farm002 after the event (for example, lunch time) has passed.

The spraying-related information 1902 stores the total drug spraying amount, dilution amount, energy amount, etc. of the entire schedule. The amount of chemicals sprayed, the amount of dilution, the amount of energy, etc. for each field may be stored.
The method for defining the schedule is an example, and other schedule management methods may be used.

In the embodiment of the present invention, a drone 100 equipped with a camera (photographing device) is used to diagnose the growth of crops in the field.
To give a specific example of crop growth diagnosis, we may focus on the bright reaction of photosynthesis by chlorophyll (chlorophyll) contained in crops. The wavelength range in which chlorophyll can be used for photosynthesis is approximately 400 to 700 nm. Among them, the contribution rate to photosynthesis differs depending on the wavelength range, and the wavelength absorption spectrum (distribution) of chlorophyll has relatively high short wavelength range (blue wavelength range) and long wavelength range (red wavelength range), and medium wavelength range (distribution). The green wavelength range) tends to be relatively low. For example, when focusing on the long wavelength region, the reflectance in the long wavelength region is low because the crops in which photosynthesis is actively performed have a high absorption rate of chlorophyll. On the other hand, crops with inactive photosynthesis have low chlorophyll absorption and therefore high reflectance in the long wavelength range. Therefore, the growth of the crop can be diagnosed by detecting the amount of reflected light from the crop for each wavelength and the ratio thereof based on the image of the crop in the field.

FIG. 20A is an example of a conceptual diagram in which a crop growth diagnosis is performed based on the amount of reflected light (green, red, etc.) for each wavelength from the crop by the camera 512 of the flying drone 100.
The drone 100 photographs crops on the ground from a position at a predetermined altitude with a camera 512 oriented at a predetermined angle (not limited to directly below). Since the shooting range of the camera that shoots at one time is limited, it is configured to shoot all the crops in the field by continuously shooting the fields of view B1 and B2 in a predetermined range.
For example, in region A1, images with relatively high reflectance in a relatively long wavelength region of two types of crops 11 and 12 can be taken, while in adjacent region A2, comparisons of two different types of crops 13 and 14 are made. It is possible to take an image having a low reflectance in the long wavelength region. The captured images of the regions A1 and A2 may be combined in the post-treatment, enabling the growth diagnosis of the crop in the entire field.

The crop growth diagnosis according to the embodiment of the present invention is not limited to the above contents, and the number of cameras 512 used is not limited to one.
For example, in crop growth diagnosis, in addition to the amount of reflected light for each wavelength from the crop, the growth height of the crop (Z-axis direction) and / or the growth range of the crop (X-axis, Y-axis direction), the stem and leaves of the crop. , Fruit, size and / or number of paddy, etc. may be targeted. In this case, using a plurality of cameras 512, two-dimensional or three-dimensional information is acquired based on the images of the crops in the field, and an appropriate growth diagnosis is performed on the growth height, growth range, etc. of the crops. May be good. In addition, growth diagnosis may be performed depending on whether the crop is sick or not.

The pretreatment carried out with the drone 100 in the growth diagnosis of the crop is
-Processing to label the captured information on the captured image-Process to detect the intensity of the light reflected from the crop for each wavelength based on the captured image-Determine the degree of growth of the crop based on the captured image Processing-Based on the captured image, processing such as associating the acquired position information with the growth degree of the crop can be considered.

FIG. 20B shows a crop growth diagnosis using the camera 512 of the flying drone 100 based on the crop growth height (Z-axis direction) and / or the crop growth range (X-axis, Y-axis direction). This is an example of a conceptual diagram to be performed.
The drone 100 photographs crops on the ground from a position at a predetermined altitude with a camera 512 at a predetermined angle (not limited to directly below). Since the shooting range of the camera that shoots at one time is limited, it is configured to shoot all the crops in the field by continuously shooting the fields of view B3 and B4 in a predetermined range.
For example, in region A3, crops 15 and 16 having a relatively low height Z1 but a relatively wide spread L1 can be photographed, while in the adjacent region A4, a height Z2 is relatively high but spread. It is possible to photograph a crop 17 having a relatively narrow L2. The captured images of the regions A3 and A4 may be combined in the post-treatment, enabling the growth diagnosis of the crop in the entire field.

The crop growth diagnosis is not limited to the contents exemplified in FIGS. 20A and 20B. Furthermore, any kind of camera suitable for the growth diagnosis of various crops can be used.
For example, as the camera 512, not only a digital camera for still image shooting but also a video camera for moving image shooting or a camera for both moving image and still image shooting can be used.
It is also possible to use a panoramic camera for panoramic photography that captures an extremely wide width (angle of view) in one frame.
It is also possible to use a stereo camera that can record information in the depth direction by simultaneously photographing an object from a plurality of different directions.
Further, if necessary, an infrared camera, an ultraviolet camera, an X-ray camera, or the like can be used alone or in combination.

Hereinafter, as a specific example, a case where a still image of a crop in a field is taken using a single or a plurality of digital cameras and a growth diagnosis is performed based on the color of the crop will be described, but the scope of the present invention is this. Please understand that it is not limited to the content.

Usually, in crop growth diagnosis, a crop in a field is photographed using a digital camera for taking one or more images. At this time, the following operations may be included.
-Camera photography of field crops,
-Flagging invalid or valid images on captured images,
-Memory (save) of captured images,
-Preprocessing for captured images,
-Flagging invalid or valid images on preprocessed data,
-Memory of data after preprocessing,
-Transmission of preprocessed data to the outside, etc.

In order to properly perform growth diagnosis, it is required to acquire high-quality images. Therefore, the flight condition of the drone 100 becomes a problem in particular. For example, if the shooting angle and shooting distance of the drone 100 during flight are poor, or if the drone 100 flies in a time zone or weather conditions that are not suitable for growth diagnosis, the image quality of the shot image deteriorates. In that case, it may be difficult to perform a growth diagnosis with the required accuracy. Therefore, when the growth diagnosis of crops in the field is performed using the drone 100, the flight state of the drone 100 at the time of imaging of each image is distinguished according to a predetermined classification, and the appropriate image is taken when the flight condition is good. It is preferable to extract and use only the image.

In addition, the camera mounted on the drone 100 has a problem that the data capacity at the time of shooting is relatively large. Therefore, the capacity of the storage medium for storing the captured image, the processing speed of the control unit for preprocessing the captured image, and the speed of the transmission unit for transmitting the preprocessed data to the outside are relatively burdensome. Is high. When an image unsuitable for growth diagnosis is stored, preprocessed or transmitted, an unnecessary burden is applied to a storage medium or the like, resulting in waste. Therefore, when the growth diagnosis of the crop in the field is performed using the drone 100, it is preferable to automatically exclude the images inappropriate for the growth diagnosis from the storage target and the like.

It is preferable to reduce the data capacity by preprocessing the captured image and storing or transmitting the preprocessed data instead of the raw data. Pretreatment refers to various image processing related to crop growth diagnosis. For example, it may include detection of the proportion of light such as green or red reflected by a plant, detection of the amount of light, and the like. Alternatively, it may include analysis of plant distribution in two or three dimensions. Preferably, in the captured image, a predetermined image analysis is performed for each pixel (pixel) or for each group of pixels.
In addition, the preprocessing may include a process of labeling the image with shooting information, that is, a process of associating the shooting information. This shooting information may include, for example, the situation of the field at the time of shooting, the time, the temperature, the humidity, the weather, the situation of the camera at the time of shooting, the flight situation of the drone, the situation of the shot image, and the like.

As image storage control, it is conceivable to store captured images when the flight conditions such as the position, altitude, speed, and attitude of the aircraft (including the drone) satisfy preset conditions. However, in the case of a drone that performs autonomous flight flying on a preset flight path, there is a possibility that the propriety of storing the captured image cannot be sufficiently determined only by the above conditions. Further, as a memory control of other images, it is conceivable to select and store an image suitable for the growth diagnosis of crops in the field. However, there is a possibility that sufficient measures have not been taken except for excluding it from the storage target.

Hereinafter, the camera system of the drone 100 used for the growth diagnosis of crops in the field will be described in detail.
For example, as illustrated in FIG. 6, the drone 100 can include a multispectral camera 512. The multispectral camera 512 includes a lens and an image pickup unit, and can photograph crops in a field from a predetermined height.
Further, the drone 100 includes a flight controller 501 that controls the entire drone, and the output from the multispectral camera 512 (FIG. 6) can be transmitted to the flight controller 501.
The flight controller 501 functions as a processor (processing device) that controls the flight of the drone, and can also preprocess the image acquired by the multispectral camera 512.
As illustrated in FIG. 7, the preprocessed data and the like can be stored on the drone 100 and transmitted to an external management server 702, a mobile terminal 701, and the like.

FIG. 21 shows an example in which a dedicated camera system 20 is externally equipped for the drone 100. Again, the camera system 20 has a camera 21 with a lens and an image pickup unit.
Further, the camera system 20 has a camera controller (camera control unit) 22 that controls the overall control of the camera 21, and can include a camera capture 23 that gives an instruction for shooting to the camera.
Further, the camera system 20 has an image processing unit (image processor) 24 that preprocesses the image taken by the camera.
Further, the camera system 20 has a storage device (a non-volatile memory card such as an SD card, an SSD (Solid State Drive), a storage device such as a hard disk) 25, and captures captured images or data of images after preprocessing. Can be remembered.

The camera system 20 illustrated in FIG. 21 further has a transmission device 26, and can transmit raw data of captured images, data of preprocessed images, and the like to an external management server or the like.
For example, it is possible to connect to a cloud device 28 such as a management server 702 by using application software 27 or the like that synchronizes files and directories between remote locations.
Further, the camera system 20 includes a connection portion for connecting to the drone 100 including the main body and a plurality of rotary wings. The camera system 20 may be connected at the same time as the drone 100 is manufactured, or may be retrofitted to the drone 100.

The camera system 20 illustrated in FIG. 21 can be further connected to a flight controller 501 (see FIG. 6). Therefore, the camera system 20 can transmit data such as captured images to the flight controller 501.
Further, the camera system 20 can receive information such as the position, speed, attitude, time, flight status (mission status), and altitude above ground level of the drone 100 from the flight controller 501.
Hereinafter, as the camera of the drone 100 used for the growth diagnosis of the crop in the field, both the multispectral camera 512 exemplified in FIG. 6 and the camera system 20 exemplified in FIG. 21 may be included. The latter includes (1) a camera, (2) a control unit that preprocesses images acquired from the camera, (3) a transmission unit that transmits preprocessed data to the outside, and (4) a main body and a plurality of units. It is housed in one housing and assembled with respect to the drone 100, including a connection portion connecting to the drone with a rotor.

The flight state of the drone 100 may be transmitted from the flight controller 501 of the drone to the camera control unit 22 via the connection unit, or the camera system 20 itself or an external module different from the flight controller-501 may be used. It may have a plurality of sensors and may be configured to determine the flight state of the drone 100. Further, the flight state information may be received from the management server 702 or the mobile terminal 701 via the transmission device 26 included in the camera system 20.

As described above, various data regarding the drone 100 are sent to the flight controller 501.
For example, the flight controller 501 is fed back with the actual rotation speeds of the motors 102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 104-a, 104-b. (See FIG. 6).
Further, the flight controller 501 can measure the absolute position of the drone 100 with an accuracy of about several centimeters by combining the signal of the RTK base station and the signal from the GPS positioning satellite (504-1 in FIG. 6). See 504-2 and 504-3).

Further, the flight controller 501 can calculate the acceleration and velocity of the drone aircraft in three directions orthogonal to each other by the 6-axis gyro sensor 505, and can measure the change in the attitude angle of the drone aircraft in the three directions, that is, the angular velocity (FIG. 6). reference).
Further, the flight controller 501 can measure the direction of the drone aircraft by measuring the geomagnetism by the geomagnetic sensor 506 (see FIG. 6).
In addition, the barometric pressure sensor 507 can measure the barometric pressure and indirectly measure the altitude of the drone (see FIG. 6).
Further, the flight controller 501 can measure the distance between the drone aircraft and the ground surface by using the reflection of sound waves such as ultrasonic waves by the laser sensor 508 and sonar 509 (see FIG. 6).

Therefore, information such as the position, altitude, speed, and attitude of the drone 100 is given to the flight controller 501 based on the signal from the drone sensor. Based on this information, the control unit, that is, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 can determine whether or not the drone 100 is in a predetermined flight state. ..

Further, the drone 100 is wirelessly connected to an external mobile terminal 701 or a management server 702 (see FIG. 7), and the flight route management information 1800 (see FIG. 7) stored in the mobile terminal 701 or the management server 702 (see FIG. 7). 10. Refer to FIGS. 11 and 18), the flight coordinates are specified in order.
At this time, the flight management module 1112 of the management server 702 manages the sprayed flight of the drone 100 based on the information such as the field management information 1300 and the flight route management information 1800 (see FIG. 11).

Further, the flight path management module 1115 of the management server 702 calculates the flight path of the spray flight of the drone 100 based on the field management information 1300 (see FIG. 11).
As described above, the field management information 1300 stores various information about the field to which the chemicals are sprayed, and stores information such as the field ID, the field name, the field position, the coordinates around the field, the field area, and the crops planted. Remember (see FIG. 13).
Further, the flight route management information 1800 stores information indicating the flight route of the drone 100, and stores the route ID, the target ID, the route coordinates, the total route distance, and the like (see FIG. 18).
Therefore, according to the instruction from the management server 702, the drone 100 can perform autonomous flight, and the flight controller 501 can determine the flight state based on the signal.

Further, apart from the instruction from the management server 702, the mobile terminal 701 (see FIG. 7) allows the drone 100 to perform basic operations such as takeoff and temporary return. Further, in an emergency, the mobile terminal 701 can manually operate the drone 100.
For example, when the drone 100 performs a hovering pause, it can be obtained from the emergency stop button 925 (FIG. 9) of the drone operation screen 900 displayed on the mobile terminal 701. The same applies to the option for the drone 100 to return to the flight start point (temporary return, etc.) and the option to stop the motor on the spot (landing, etc.).

Depending on the operation of the drone 100 such as the altitude change buttons 923 and 924 by the user and the emergency stop button 925, the drone operation module 1012 (see FIG. 10) transfers information such as commands corresponding to these operations to the drone 100. Send and operate the drone 100.
Therefore, according to the instruction from the mobile terminal 701, the drone 100 can be separated from the autonomous flight, and the flight controller 501 can determine the state based on the signal.

FIG. 22 is a conceptual diagram of the flight path of the drone 100 for diagnosing the growth of crops in the field.
For ease of understanding, the fields are shown schematically as squares of equal length on all four sides, with the horizontal axis equally divided from X0 to X10 and the vertical axis equally divided from Y0 to Y10. .. It is assumed that an image is taken for each square defined by the vertical axis and the horizontal axis. However, at the time of actual shooting, adjacent shot images may be partially overlapped.
For example, starting from the starting point (S) in the lower left region (X0, X1, Y0, Y1), the drone 100 first flies outward along the four sides of the field, as shown by the outer arrow.
Next, after making almost one round along the edge of the field and reaching the area (X0, X1, Y1, Y2) in front of the starting point (S), the drone 100 excluding the outer area that has already flown. It flies in a zigzag manner in the area inside it, so as to capture all the crops in the field.
When the goal point (G) in the last area (X8, X9, Y1, Y2) in the lower right is reached, the drone 100 completes the shooting work and leaves the field.
It should be noted that the shape of the field in which the drone 100 actually flies is not limited to that illustrated in FIG. 22. Further, the photographed image (square) is not limited to a square.

Here, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 can determine the flight state of the drone 100 at the time of shooting (it can be determined based on the position, altitude, speed, attitude, autonomous flight, etc.). In particular, can be subdivided into the following AF.
A. When the drone 100 is flying in the field (see FIG. 22)
B. When the drone 100 is flying in and out of the field C. When the drone 100 is temporarily returning D. When the drone 100 is landing or taking off E. When the drone 100 is in an emergency stop (hovering) F. When the position signal or speed measurement of the drone 100 cannot be measured well due to the disturbance of the input signal, etc., and the flight state of the drone 100 cannot be properly determined.

Further, the flight controller 501 or the camera controller 22 of the camera system 20 can be classified into the following two types regarding the flight state of AF. That is, in the case of A, the flight controller 501 determines that the captured image is a valid image. On the other hand, in the case of BF, since it is difficult to properly photograph the crops in the field, the flight controller 501 or the camera controller 22 of the camera system 20 determines that the captured image is an invalid image. If valid and invalid images are extracted at this stage, in each case, the flight controller 501 or the camera controller 22 of the camera system 20 will use each image together with position (latitude, longitude, altitude, etc.) and time information. Is flagged (disabled / enabled).

In particular, it is preferable that the flight controller 501 or the camera controller 22 of the camera system 20 determines that the image taken in the following cases is an invalid image.
B. When the drone 100 is flying in and out of the field, that is, when the drone 100 has not reached the field or has left the field, the crops in the field cannot be photographed properly. Therefore, it is determined that the flight state of the drone 100 does not satisfy the predetermined condition, that is, the captured image is an invalid image. The entry / exit flight to and from the field is an entry flight from the takeoff point of the drone to the shooting target area, or an exit flight from the shooting target area to the landing point.

At the time of photographing a normal field, the drone 100 flies in a zigzag or meandering manner. Moves to the next area in order (see FIG. 22). On the other hand, when entering and exiting the field, the drone 100 can fly diagonally to the field without being restricted by such movement. Therefore, if the flight state is tracked together with the position information of the drone 100 and a zigzag flight or the like is detected, it may be determined as a valid image. On the other hand, if an oblique flight that deviates from a predetermined route flight (zigza flight) is detected, it may be determined as an invalid image.

C. When the drone 100 is temporarily returning, that is, when the drone 100 leaves the field due to some inconvenience or the like and returns to the flight start point or the location of the mobile terminal 701, the crops in the field cannot be photographed properly. Therefore, it is determined that the flight state of the drone 100 does not satisfy the predetermined condition, that is, the captured image is an invalid image. In the case of temporary return, it is judged that the image taken in the temporary return flight in which the drone temporarily returns from the work interruption point or the work restart flight from the return point to the work interruption point is an invalid image.
Also in this case, the flight state is tracked together with the position information of the drone 100, and if a zigzag flight or the like is detected, it is judged as a valid image, and if an oblique flight or the like is detected, it is determined. It may be determined as an invalid image.

D. When the drone 100 is landing or taking off, that is, when the camera cannot photograph the crop in the field from a predetermined altitude, the crop in the field cannot be photographed properly. Therefore, it is determined that the flight state of the drone 100 does not satisfy the predetermined condition, that is, the captured image is an invalid image.
In this case, the valid image and the invalid image may be distinguished by tracking the instruction from the mobile terminal 701, the speed and altitude of the drone 100, and the like together with the position information of the drone 100.

E. When the drone 100 is in an emergency stop (hovering), that is, when the camera does not move the crops in the field from a predetermined position and altitude, it is not possible to properly photograph all the crops in the field. Therefore, it is determined that the flight state of the drone 100 does not satisfy the predetermined condition, that is, the captured image is an invalid image.
In this case, the valid image and the invalid image may be distinguished by tracking the instruction from the mobile terminal 701, the speed and altitude of the drone 100, and the like together with the position information of the drone 100.

F. If the position signal or speed measurement cannot be measured well due to the disturbance of the input signal, etc., and the flight condition of the drone 100 cannot be properly determined, it is possible to properly photograph the crops in the field. Can not. Therefore, it is determined that the flight state of the drone 100 does not satisfy the predetermined condition, that is, the captured image is an invalid image.
Examples of the input signal include various sensors of the drone 100, signals from the mobile terminal 701 and the management server 702, satellite signals, and the like. In this case, the effective signal and the invalid signal may be distinguished by tracking the values of these signals and comparing and determining them with a predetermined threshold value or the like.

Therefore, the flight controller 501 or the camera controller 22 of the camera system 20 may determine that the image is invalid when the position coordinates are outside the target field based on the position information of the drone 100.
Further, in addition to the above BF, it may be determined as an invalid image based on the following conditions.
For example, G. If the altitude above ground level of the drone 100 is out of the predetermined range (too high or too low), it will be difficult to take an appropriate picture, so it may be determined as an invalid image.
Furthermore, H. If the posture angle of the drone 100 is out of the predetermined range, it is difficult to take an appropriate image, so it may be determined that the image is invalid.

In addition, I. When the flight controller 501 is abnormal (temporary abnormality, etc.) or malfunctions (non-temporary permanent equipment failure, etc.), it may be determined as an invalid image.
In addition, J. When the camera system 20 is abnormal (for example, a temporary abnormality of the camera controller 22) or fails (for example, the lens of the camera 21 is dirty), it may be determined as an invalid image.
In addition, K. If the drone 100 flies overlapping with the position where the shooting flight has already been taken, it may be determined as an invalid image.
Furthermore, L. When the flight controller 501 detects an emergency intervention command, an abnormal state detection, or the like, it may be determined as an invalid image.

Furthermore, M. For the purpose of diagnosing the growth of crops in the field, if the growth diagnosis (for example, judgment of the color of the crop such as green or red) cannot be performed well in the image of the crop in the field, the image is invalidated. You may judge that. In this case, in particular, whether or not an appropriate amount of light is insufficient in the captured image becomes a problem.
For example, an image obtained during the day can make a good image judgment, but an image obtained at night when the surrounding environment becomes dark and good color identification becomes difficult cannot make an appropriate image judgment. Therefore, if the shooting time is out of the predetermined range (other than daytime), it may be determined as an invalid image.

Furthermore, N. For example, if the drone 100 is operated in the evening and the surrounding environment gradually darkens with sunset, the degree of illumination of the surroundings (light intensity, etc.) obtained from the sensor (light sensor, etc.) or the image of the camera. If is out of the predetermined range, it can be determined that the image is invalid. That is, instead of judging by the binary value of day or night, the change in the degree of light irradiation from day to night is assumed in advance in a multi-step manner, and when it deviates from the predetermined range, it is regarded as an invalid image. You may judge.

Furthermore, O. For example, even when the drone 100 is operated in fine weather (daytime), if the surrounding environment deteriorates due to the influence of climate change (wind, insects, etc.), the captured image quality may deteriorate. In this case, the image obtained at that time may be determined as an invalid image based on the weather information, the information obtained from the sensor, and the like. That is, even in the daytime, if the captured image is stained or disturbed by wind, insects, or the like and appropriate image quality cannot be obtained, it may be determined as an invalid image.

Furthermore, P. For example, even if the drone 100 is operated in fine weather, if the surrounding environment becomes dark even during the daytime due to the influence of climate change (rain, snow, etc.), the captured image quality may deteriorate. In this case, the image obtained at that time may be determined as an invalid image based on the weather information, the information obtained from the sensor, and the like. That is, even during the daytime, if a sufficient amount of light is insufficient due to weather conditions (rain, snow, etc.) and appropriate image quality cannot be obtained, it may be determined as an invalid image.

When an invalid image is extracted by satisfying any of the above BP conditions, particularly when an invalid image is extracted by satisfying any of the above BF conditions, sound or light is extracted from the drone 100. You may output a warning with, or you may output a warning to a mobile terminal 701 or the like.
For example, when an invalid image is extracted, an optical signal (particularly an error state) is given to the drone operator by lighting such as LED 107 and warning light 521 (see FIG. 6) provided in the drone 100. You may be notified by lighting, blinking, etc.). Further, the display may be performed on the display unit provided in the drone.

Further, when an invalid image is extracted, the buzzer 518 and the speaker 520 (see FIG. 6) provided in the drone 100 are used to notify the drone operator of the state (particularly the error state) by an audio signal. May be good.
Further, when an invalid image is extracted, a warning may be issued on the mobile terminal 701 (see FIG. 7) used by the drone operator, and information for notifying the state (particularly an error state) may be output. .. For example, it is conceivable to output a text message or an optical signal to the display unit (display or display light) of the mobile terminal 701, or to output an audio signal (warning sound or the like) from the speaker.

Therefore, the user can immediately identify whether the drone 100 is acquiring the valid image or the invalid image by the signal emitted from the drone 100 main body or the mobile terminal 701. Therefore, the user can quickly understand whether or not the invalid image is mixed in the valid image during the operation.

In order to avoid imposing an excessive load on the storage device and the processing device of the camera system 20, it is preferable to exclude the invalid image from the target of the growth diagnosis when the captured image contains an invalid image unsuitable for the growth diagnosis.
Therefore, in the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20, the acquired flight information (position, attitude, speed, altitude, flight state, etc.) is a predetermined condition (the above BF, etc.). ), Especially when it is determined that the image is invalid, the captured image is flagged as an invalid image, the storage of the image is stopped, the preprocessing for the image is stopped, or the image or after the preprocessing. It is preferable to stop the storage of the data of the above, stop the transmission of the image or the preprocessed data to the outside, and delete the image or the preprocessed data from the storage device.

Depending on the embodiment, any one of the following may be performed, or an appropriate combination may be adopted from the following plurality of combinations.
-Flag on / off invalid
-Record / not record the original image (raw data)
-Pre-process the image / not
-Record / do not record preprocessed data
-Transmit / not transmit the original image or preprocessed data, etc.
For example, these combinations include the case where the original image is not recorded in the first place, the case where the original image is recorded as it is but the invalid flag is added, and the case where the preprocessing is performed but the invalid flag is added.

That is, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 not only specifies whether or not to store the captured image, but also preprocesses or does not preprocess the captured image. The data after processing is stored / not stored, and the data after preprocessing is sent / not sent to the management server.
Preferably, the camera controller stores the captured raw data in the storage unit 25 of the camera system 20 in the drone 100, while prohibiting preprocessing by the image processor 24, or after preprocessing by the transmission unit 26. Prohibit the transmission of the data to the outside. As a result, it is possible to avoid performing preprocessing based on poor quality captured images and reduce the burden of unnecessary data transmission. By finely dividing the drone as described above, it is possible to control the drone with the optimum contents according to various embodiments.

On the other hand, in the simple method, when an invalid image is extracted, a condition is set in which the image is simply taken / not taken, or the taken image is stored / not taken. On the other hand, in the present embodiment, even if it is an invalid image, by storing it, it is possible to leave an option to be used in subsequent analysis or the like. For example, even if the conditions for invalid images are met, there may be cases where a part or all of them can be used for image processing, or a part or all of them have material value. In such a case, by avoiding pre-processing or data transmission on the drone 100 side, it is possible to suppress unnecessary processing load and leave an option for image processing later if necessary.

FIG. 23 is an example of an invalid image determination processing flow of the drone 100.
In the invalid image determination processing flow, when the flight instruction of the drone 100 is received in step S01, the flight controller 501 receives the flight state of the drone 100 from various sensors (FIG. 6) or an external server (FIG. 7) in step S02. To get.
Next, in step S03, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) determines whether or not the flight state acquired in step S02 satisfies a predetermined invalid image condition. For example, when any of the conditions such as BF is satisfied, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) flags the captured image as an invalid image in step S04. After that, in step S05, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) performs preprocessing of the invalid image, storage of the preprocessed data, transmission of the preprocessed data to the outside, and the like. Stop.

On the other hand, in step S03, for example, when the condition of A is satisfied, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) flags the captured image as an effective image in step S06. After that, in step S07, preprocessing of the effective image, storage of the data after the preprocessing, transmission of the data after the preprocessing to the outside, and the like are executed.
The above processing may be performed by the flight controller 501 (FIG. 6) in cooperation with the camera controller 22 (FIG. 21), or the camera controller 22 (FIG. 21) instead of the flight controller 501 (FIG. 6). May be carried out.

After the invalid image is extracted, the management server 702 can recalculate the flight path flying in the area where the invalid image is acquired.
The control unit, that is, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 stores the position information of each invalid image and transmits the position information to the external management server 702. do. The management server 702 may recalculate the flight path connecting each invalid image based on the received data and send an instruction to the flight controller 501 to regain the area of each invalid image.

For example, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 flags the area corresponding to the invalid image and obtains the position (latitude and longitude, etc.). Then, the management server 702 calculates the flight path of the economical drone 100 based on the position information and the like for each flagged image, or selects an appropriate flight path from the predetermined flight paths. Then, the drone 100 is re-flyed according to the route.

FIG. 24 is a conceptual diagram showing a case where an invalid image is extracted during the flight of the drone 100 for diagnosing the growth of crops in the field illustrated in FIG. 22.
In this example, in the three colored regions 31 (X1, X2, Y5, Y6), region 32 (X3, X4, Y8, Y9), and region 33 (X5, X6, Y8, Y9), the invalid image It is flagged.
The management server 702 can take, for example, the following means when calculating a new flight path for re-shooting the areas 31, 32, 33 from which these invalid images have been extracted.

FIG. 25 shows that the same flight path is adopted again in the flight path including the areas 31, 32, and 33 of the three invalid images exemplified in FIG. 24, and the flight is continued until all the invalid images are retaken. An example is shown.
That is, the management server 702 can also calculate that, in the newly obtained flight path, the image is taken only at the point corresponding to the invalid image according to the flight path already adopted, in whole or in part. be.
In this case, the calculation of the flight path can be simplified. However, if the number of invalid images that need to be retaken is small, the flight path tends to be wasted. On the other hand, since there is no change in the flight path between the first and second flight, there is an advantage that it is easy to standardize the direction and angle of the camera at the time of shooting. This method is effective when the number of invalid images that need to be retaken is large. In addition, the flight around the large circumference of the field may be omitted.

FIG. 26 shows flight so as to connect the invalid images at the shortest distance in the flight path including the regions 31, 32, 33 of the three invalid images exemplified in FIG. 24 without adopting the same flight path again. Here is an example of how to do it.
In this case, the flight path is calculated by connecting a plurality of positions corresponding to a plurality of invalid images with a single stroke and minimizing the length of the entire flight path.
For example, the management server 702 connects the areas 31, 32, and 33 of each flagged image with a single stroke in the newly obtained flight path, and calculates to minimize the overall flight distance. May be good.
In this case, the time required to fly the drone 100 can be shortened to avoid waste in the flight path, but a peculiar problem may occur due to the departure from the zigzag flight path (FIG. 22).

That is, as illustrated in FIG. 22, in the zigzag flight path, the drone 100 flies almost parallel to the vertical axis and the horizontal axis, whereas as illustrated in FIG. 26, the invalid image is the shortest. When connecting by distance, the drone 100 tends to fly at an angle with respect to the vertical axis and the horizontal axis. As a result, as illustrated at the right end of FIG. 26, the orientation of the invalid image 31 taken in the first flight path (zigzag flight path) and the second flight path (angled flight path) are taken. A deviation (angle α) is likely to occur between the direction and the direction of the image 31'. Therefore, when the first invalid image 31 is replaced with the subsequent valid image 31', the orientations of the corresponding images are different. Therefore, when merging (combining) a plurality of images into one, the position of the image shifts at that location. (Especially when the captured image 31 is not square).

Therefore, in the embodiment illustrated in FIG. 26, due to the nature of taking each invalid image away from the first flight path (zigzag flight path), the direction at the time of the first shooting and the direction at the time of the second shooting are taken. It is preferable to store the image in correspondence with the direction and the like. Later, when the first effective image and the second effective image correspond to each other, the flight controller 501 or the camera controller 22 corrects the deviation (angle α) of the image in which the retake occurs (photographed image). (Rotation, etc.) is preferable. This method is effective when the number of invalid images that need to be retaken is small.

FIG. 27 shows the case of the first flight path while connecting the three invalid images at the shortest possible distance without adopting the same flight path again in the flight path including the three invalid images exemplified in FIG. 24. It shows an example of flying to shoot in the same camera orientation as. That is, the length of the entire flight path is minimized while matching the direction in which the invalid image is retaken with the case of the first flight path.
In this means, during the second flight, the section that flies toward each invalid image at the shortest possible distance (the section that flies diagonally with respect to the first flight path) 41 and the first for each invalid image. It includes a section (a section that flies in the same manner as the first flight path) 42 that corrects the direction of the drone 100 so that the image is taken with the same camera orientation as during flight. The ratio of the lengths of the sections 41 and 42 can be changed according to the embodiment, but the length of the sections 42 may be as short as possible.

In this case, unlike the case illustrated in FIG. 26, the orientation of the image taken in the first flight path (zigzag flight path) and the second flight path (angled flight path) retaken are used. There is no deviation (angle α in FIG. 26) from the orientation of the captured image. Therefore, when the first invalid image is replaced with the subsequent valid image, the orientations of the images are almost the same, so that the possibility of causing misalignment when merging a plurality of images into one can be minimized.
However, in order to match the orientation of the image taken in the first flight path, multiple times (three times or), especially when flying from the area 32 of the second invalid image to the area 33 of the third invalid image. (Twice) It may cause waste of changing the shooting direction (reference numeral 43).

In this case, the flight path calculates a flight path that is partially or wholly the same as the first flight path so as to include a plurality of positions corresponding to the plurality of invalid images.
Further, when reacquiring an image at a plurality of positions corresponding to a plurality of invalid images, the direction of the drone is made the same as the direction in which the invalid image was acquired, and then the image is acquired again.
As another method, when re-acquiring an image at a plurality of positions corresponding to a plurality of invalid images, the re-acquired image is changed to the same composition as the corresponding invalid image (image processing). You may take the method of reacquiring the image of the same composition as the first flight.

FIG. 28 shows that in the flight path including the regions 31, 32, 33 of the three invalid images exemplified in FIG. 24, each invalid image is connected at the shortest possible distance without adopting the same flight path again. An example of flying in the same direction as in the case of the first flight path or in a direction of 90 degrees or 180 degrees is shown.
Comparing the example of FIG. 28 with the example of FIG. 27, in the former, especially when flying from the area 32 of the second invalid image to the area 33 of the third invalid image, the image taken in the first flight path The shooting direction is changed only once in order to correspond to the direction (reference numeral 43). Therefore, the example of FIG. 28 is more economical because the flight path is shortened as compared with the example of FIG. 27. However, the image after shooting is symmetrical vertically or horizontally.

Normally, when the captured image is rectangular, the vertical and horizontal lengths are the same, so by changing the orientation (rotation) by 180 degrees, the retaken valid image can be replaced with an invalid image. can.
Further, when the captured image is a square, the length of each side is the same, so that the retaken valid image can be replaced with the invalid image by changing the orientation (rotation) by 90 degrees.
However, the shot image is rotated in consideration of the shooting angle of the camera at the time of shooting.

Therefore, the management server 702 can adopt an appropriate means among the following means when newly obtaining a flight path.
-In the case illustrated in FIG. 25 (when calculating a new flight path so that the first flight path and a part or all of the same flight path are used so as to include an invalid image).
-In the case illustrated in Fig. 26 (when connecting invalid images with a single stroke and calculating a new flight path so as to minimize the length of the entire flight path / giving priority to connecting invalid images at the shortest distance If the shooting direction of the invalid image does not match the case of the first flight path)
-In the case illustrated in Fig. 27 (when connecting invalid images at the shortest distance and calculating a new flight path so that the shooting direction of the invalid image matches the case of the first flight path)
-In the case illustrated in FIG. 28 (a new flight so as to connect invalid images at the shortest distance and to match the shooting direction of the invalid image with the case of the first flight path or to change the direction by 90 degrees or 180 degrees. When calculating the route)
In addition, the management server 702 may prepare a plurality of flight paths in the database in advance, select an appropriate flight path including the position of the invalid image from the database, and obtain a new flight path.

FIG. 29 is an example of the invalid image determination process and the second flight route calculation process flow of the drone 100.
In the invalid image determination processing flow, when the flight instruction of the drone 100 is received in step S11, the flight controller 501 (FIG. 6) is transmitted from various sensors (FIG. 6) or an external server (FIG. 7) in step S12. Acquire the flight status of the drone 100.
Next, in step S13, it is determined whether or not the flight state acquired by the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) satisfies the predetermined invalid image condition in step S12. For example, when any of the above conditions such as BF is satisfied, the captured image is flagged as an invalid image in step S14. After that, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) processes invalid images (image pre-processing, pre-processing data storage, pre-processing data transmission, etc.) in step S15. ) Is stopped.
On the other hand, in step 3, for example, when the condition of A is satisfied, the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) flags the captured image as an effective image in step S16. After that, in step S17, processing of the effective image (pre-processing of the image, storage of data after pre-processing, transmission of data after processing, etc.) is executed.

Further, in the second flight route calculation processing flow, the invalid image information (FIG. 24, invalid image flag, position information, etc.) is transmitted to the external management server 702 in step S18 (FIG. 7).
Next, in step S19, the external management server 702 side recalculates the new flight path connecting the invalid images (FIGS. 25-28).
Next, in step S20, the recalculated flight path is transmitted from the external management server 702 to the drone 100 (FIG. 7). After that, the drone 100 is re-flyed to retake the invalid image.

The flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 will flag each retaken image as newly valid or invalid on the second flight when the invalid image is taken again. conduct. As a result, there is a one-to-one correspondence between the image determined to be an invalid image during the first flight and the image retaken as a valid image during the second flight. At that time or thereafter, each invalid image during the first flight may be replaced with each corresponding valid image.
If the flight controller 501 (FIG. 6) or the camera controller 22 (FIG. 21) of the camera system 20 causes an invalid image again during the second flight, the flight controller 501 (FIG. 6) takes the same measures as during the first flight. If necessary, the photograph may be taken again during the third flight.

Therefore, the present invention provides a drone 100 equipped with a camera 512 or a camera system 20 attached to the drone 100 for diagnosing the growth of crops in a field.
When an invalid image is extracted from the captured image by the drone 100 or the camera system 20, the transmission of the preprocessed or preprocessed data to the external server is stopped. By doing so, waste of image processing is eliminated and work efficiency is optimized.
Further, when an invalid image is extracted from the captured image by the drone 100 or the camera system 20, an external server is made to calculate a new flight route so as to newly capture the invalid image. This makes the optimized flight path available promptly for retrieving invalid images.

The present invention is not limited to the above-described embodiment, but includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

Further, each of the above configurations, functions, processing units, processing means, etc. may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be stored in a memory, a storage device such as a hard disk or SSD (Solid State Drive), or a storage medium such as an IC card, SD card, or DVD.

In addition, the control lines and information lines indicate those that are considered necessary for explanation, and do not necessarily indicate all the control lines and information lines in the product. In practice, it can be considered that almost all configurations are interconnected.
It should be noted that the above-mentioned embodiment discloses at least the configuration described in the claims.

21 ... camera, 22 ... control unit (camera controller), 100 ... drone, 501 ... control unit (flight controller), 512 ... camera (multispectral camera), 701 ... mobile terminal, 702 ... management server, 703 ... management terminal, 710 ... Base station

Claims (21)

1. With the main body
   With multiple rotors,
   With the camera
   A transmission unit that transmits data generated from the image acquired from the camera to the outside,
   With storage
   It is a drone equipped with
   When the flight state of the drone satisfies a predetermined condition, the image is flagged as an invalid image, the storage of the image in the storage device is stopped, the preprocessing for the image is stopped, or the data. A drone that stops transmission to the outside of the image, deletes the image from the storage device, and performs at least one of the following.
2. The drone according to claim 1, wherein the predetermined condition is an entry flight from the takeoff point of the drone to the target area, or an exit flight from the target area to the landing point.
3. The drone according to claim 1, wherein the predetermined condition is a temporary return flight in which the drone temporarily returns from the work interruption point, or a work restart flight from the return point to the work interruption point.
4. The drone according to claim 1, wherein the predetermined condition is landing or takeoff of the drone.
5. The drone according to claim 1, wherein the predetermined condition is that the position measurement or speed measurement of the drone cannot be performed.
6. The drone according to any one of claims 1 to 5, wherein the pre-processing is a process of associating shooting information with the image.
7. The drone according to any one of claims 1 to 5, wherein the pretreatment of the image is to detect the proportion or amount of green or red light reflected by the plant based on the image.
8. The drone according to any one of claims 1 to 5, wherein the pretreatment of the image is to determine the degree of growth of the crop based on the image.
9. The drone according to any one of claims 1 to 5, wherein the preprocessing of the image is to associate the position information obtained with the image with the growth degree of the crop based on the image.
10. The drone according to any one of claims 1 to 9, which determines whether or not the flight state of the drone satisfies a predetermined condition based on the detection information of the sensor mounted on the drone.
11. The drone according to any one of claims 1 to 9, which determines whether or not the flight state of the drone satisfies a predetermined condition based on information received from at least either an external management server or a mobile terminal. ..
12. Claimed to perform at least one of outputting light from the lighting mounted on the drone, outputting sound from the speaker, or displaying on the display unit when it is determined that the predetermined condition is satisfied. The drone according to any one of 1 to 11.
13. The drone according to any one of claims 1 to 12, which transmits information for displaying a warning to a mobile terminal when it is determined that the predetermined conditions are satisfied.
14. With the main body
    With multiple rotors,
    With the camera
    A control unit that preprocesses the image acquired from the camera, and
    A transmission unit that transmits the preprocessed data to the outside,
    It is a drone equipped with
    The control unit
    When the flight state of the drone satisfies a predetermined condition, the information of the scheduled re-shooting position in which the predetermined condition is satisfied is stored.
    The stored information on the scheduled re-shooting position is transmitted to the outside.
    Drone.
15. The drone according to claim 14, wherein a flight path for flying at the planned re-shooting position is calculated as a flight path for the drone to fly.
16. The drone according to claim 14, wherein a flight path that is partially or wholly the same as the first flight path is calculated so as to include the planned re-shooting position as the flight path that the drone flies.
17. The drone according to claim 15 or 16, wherein the control unit acquires the image again after making the direction of the drone the same as that of the first flight when the image of the scheduled re-shooting position is acquired again.
18. The control unit changes the re-acquired image to the same composition as the corresponding image acquired in the first flight when the image of the scheduled re-shooting position is acquired again, according to claim 15 or 16. The drone listed.
19. With the camera
    A transmission unit that transmits data generated from the image acquired from the camera to the outside,
    A connection that connects to a drone with a body and multiple rotors,
    With storage
    It is a camera system equipped with
    When the flight state of the drone satisfies a predetermined condition, the image is flagged as an invalid image, the storage of the image in the storage device is stopped, the preprocessing for the image is stopped, or the data. To stop transmission to the outside of the image, to delete the image from the storage device, and to perform at least one of the following.
    Camera system.
20. With the camera
    A control unit that preprocesses the image acquired from the camera, and
    A transmission unit that transmits the preprocessed data to the outside,
    A connection that connects to a drone with a body and multiple rotors,
    It is a camera system equipped with
    The control unit
    When the flight state of the drone satisfies a predetermined condition, the information of the scheduled re-shooting position in which the predetermined condition is satisfied is stored.
    The stored information on the scheduled re-shooting position is transmitted to the outside.
    Camera system.
21. A server that receives at least one of the images taken by the camera mounted on the drone or the data generated from the images.
    The image taken when the flight state of the drone satisfies a predetermined condition is flagged as an invalid image, the recording of the invalid image in the recording device is stopped, the processing for the invalid image is stopped, and the invalid image is stopped. A server that performs at least one of deleting images from the recording device.
    
    

Images