JP2024061767A - Drone-work support system and drone-work support method - Google Patents

Abstract

To enable an operator to be guided to a suitable position to operate a working drone.SOLUTION: In a drone-work support system 1 for doing one or more work using one or more working drones 7, a work plan creation part 51 of a control server 5 is configured so that: a moving route of the working drone 7 is determined based on the location information of an actual work area necessary for actual work in a predetermined field; a moving route along which the operator should move is determined according to the moving route of the working drone 7; and a work support screen including the moving route of the working drone 7 and the moving route of the operator is displayed on an operation terminal 6. The work plan creation part 51 of the control server 5 may determine the moving route of the operator so that the distance between the operator and the working drone 7 moving along the moving route of the working drone 7 becomes close in range.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system for supporting work such as surveying, inspecting, maintaining, processing, destroying, or constructing man-made or natural objects such as facilities, buildings, land, fields, and forests, and, for example, a drone work support system that supports work using drones. In this specification, the term "drone" is used to refer to a mobile object that can be operated without a crew member (for example, an aircraft, running machine, surface ship, submarine vehicle, etc. that can be remotely controlled by radio or autonomously controlled).

Drones are being used for a variety of purposes, such as spraying pesticides and fertilizers in the agricultural sector.

Traditionally, when spraying pesticides or fertilizers, a set amount per unit area was typically sprayed evenly over the entire field. However, as farmland becomes more consolidated and consolidated, problems have become apparent, including the increasing cost of spraying the entire amount, and a shortage of human resources to carry out agricultural work as the area under management increases. Drones are beginning to be used to overcome this situation.

When used for purposes such as spraying pesticides or fertilizer, a farmer with expert experience and skills will first check the condition of the field and determine the necessary agricultural work. The method and extent of the agricultural work, the type of pesticide to be used, and the amount of pesticide to be used will then be determined by a human. The drone will then be manually operated to carry out the work in accordance with the decisions made.

Manually piloting a work drone requires skilled techniques and is not easy for many people. For this reason, there is a demand for technology that can automatically move work drones along a specified route.

In order to automatically move a work drone along a specified route, a map with sufficient positioning accuracy is required when determining the route the work drone will take. However, it is not always easy to obtain a map with sufficient positioning accuracy. For example, publicly available maps cannot be used as is because their positioning accuracy is not guaranteed.

For example, patent document 1 and patent document 2 disclose a technology that uses an airborne marker equipped with a GNSS receiver, which can be used to improve the positional accuracy of a map when setting a travel route.

JP 2018-146546 A International Patent Publication No. 2017/024358

When using a work drone to spray pesticides or fertilizer, the operator operates the work drone from the ground, and is therefore unable to check the condition of the inside of the field while operating the work drone. As a result, even if the area that needs work is only part of the field, it is difficult to maneuver the drone to a position suitable for operation, and the reality is that work is being done uniformly over the entire field.

For example, when a work drone is moved automatically, a situation may arise in which control of the work drone must be switched to manual control by an operator. In addition, there may be cases where it is necessary to give the work drone various instructions other than control (e.g., start moving, stop moving, start working, end working, etc.).

Work drones are often used in relatively large areas, and in some cases, the work drone may move far away from the operator. In such cases, the operator may be unable to visually check the situation around the work drone, making it difficult to operate it properly.

In addition, a work drone may finish a task, move far away from the operator's location, stop, and take a long time to start another task.

The present invention has been made in consideration of the above circumstances, and aims to achieve at least one of the following first to third objectives. The first objective is to provide technology that can guide an operator to a position suitable for operating a work drone. The second objective is to provide technology that can easily and appropriately improve the positional accuracy of a work drone. The third objective is to provide technology that can improve the work efficiency of a work drone.

To achieve the first objective, the drone work support system according to the first aspect is a drone work support system for performing one or more tasks using one or more drones, and includes a drone movement path determination means for determining a movement path for the drone based on position information of an actual work area in a specified target area where actual work is required, an operator movement path determination means for determining a movement path for the operator to move along with the drone movement path, and a display control means for displaying a work support screen including the drone movement path and the operator movement path on an operation terminal.

With this drone operation support system, the operator can easily and clearly understand how they should move when the drone is moving.

In addition, in the drone operation support system, the operator movement path determination means may determine a movement path for the operator that brings the operator close to the drone, which moves along the drone movement path. With this drone operation support system, the operator can move along a path that brings the operator close to the drone, and the state of the drone can be properly grasped, allowing appropriate response when, for example, operation of the drone is required.

In addition, in the above drone work support system, the drone movement path determination means determines a movement path to a position where energy can be replenished, passing through one or more actual work areas within a range where the drone can move using the energy possessed by the drone, the operator movement path determination means determines the operator's movement path to a movement path to the position where energy can be replenished, and the display control means may display, on the work support screen, the position where energy can be replenished and information indicating energy replenishment.

With this drone operation support system, operators can know in advance when energy replenishment is required and where energy can be replenished.

In addition, in the drone operation support system, the display control means may display information indicating the position of the operation terminal on the operation support screen based on the position information of the operation terminal. With this drone operation support system, the operator can properly grasp the positional relationship between the drone and the operator's own position (operation terminal).

In addition, in the above drone operation support system, the display control means may display a line indicating a range of a predetermined distance centered on the position of the operation terminal or the position of the drone on the operation support screen.

This drone operation support system allows the operator to properly determine whether or not he or she is in an appropriate position relative to the drone. In addition, if the operator is far away from the drone, he or she can easily estimate where to move to.

To achieve the first objective, the drone operation support method according to the second aspect is a drone operation support method using a drone operation support system for performing one or more tasks using one or more drones, which determines a movement path for the drone based on position information of an actual work area in a specified target area where actual work is required, determines a movement path for the operator to move in accordance with the movement path of the drone, and displays a work support screen including the movement path of the drone and the movement path of the operator on the operation terminal.

This drone operation support method allows the operator to easily and clearly understand how to move when the drone is moving.

In addition, in order to achieve the second object, the drone operation support system according to the third aspect is a drone operation support system for performing one or more tasks using one or more drones, and includes: a map information storage means for storing a high-precision map including highly accurate positional information of multiple positions that define the shape of a predetermined target area; a target area identification means for identifying the target area from an aerial image including an image of the target area; a position information matching means for matching the position of the target area in the aerial image with the position of the target area on the high-precision map based on the shape of the target area in the aerial image and the shape of the target area on the high-precision map; an actual work area identification means for identifying an actual work area in the target area where actual work is required based on the aerial image; a position information identification means for identifying highly accurate positional information of the actual work area based on the correspondence between the position of the target area in the aerial image identified by the actual work area identification means and the position of the target area on the high-precision map; and a drone movement path determination means for determining a movement path of the drone based on the positional information of the actual work area.

With this drone operation support system, even if the positional accuracy of the aerial image containing an image of the target area is low, the position of the target area in the aerial image can be identified with high accuracy based on a high-precision map.

In the above drone operation support system, the actual work area identification means may perform edge extraction on the aerial image to identify the shape of the target area, the position information association means may deform the aerial image so that the shape of the target area in the aerial image matches the shape of the target area based on the high-precision map, and associate the corresponding positions of the target area in the deformed aerial image and the target area based on the high-precision map as being the same position, and the actual work area identification means may identify the actual work area based on the deformed aerial image.

This drone operation support system can map a deformed aerial image containing an image of the target area to a high-precision map, making it possible to grasp a specific position in the deformed image with high accuracy.

In addition, in order to achieve the second objective, the drone operation support method according to the fourth aspect is a drone operation support method using a drone operation support system for performing one or more tasks using one or more drones, which stores a high-precision map including highly accurate positional information of multiple positions that define the shape of a specified target area, identifies the target area from an aerial image including an image of the target area, matches the position of the target area in the aerial image with the position of the target area on the high-precision map based on the shape of the target area in the aerial image and the shape of the target area on the high-precision map, identifies an actual work area in the target area where actual work is required based on the aerial image, identifies highly accurate positional information of the actual work area based on the correspondence between the position of the target area in the identified aerial image and the position of the target area on the high-precision map, and determines a movement route for the drone based on the positional information of the actual work area.

With this drone operation support method, even if the positional accuracy of the aerial image containing an image of the target area is low, the position of the target area in the aerial image can be identified with high accuracy based on a high-precision map.

In addition, in order to achieve the third objective, the drone work support system according to the fifth aspect is a drone work support system for performing one or more tasks using one or more drones, and includes: actual work area identification means for identifying one or more actual work areas in a target area where actual work is required based on an aerial image including an image of the target area; work plan creation means for creating a work plan including a movement path for moving linearly from the end point of the actual work for one actual work area to the start point of the actual work for the next actual work area when there are multiple actual work areas, and the actual work content for performing the actual work between the start point of the actual work in each actual work area and the end point of the actual work; and control information transmission means for transmitting control information for executing the work plan to the drone.

This drone operation support system can reduce the travel time of the drone when performing work, improving processing efficiency.

In addition, in order to achieve the third objective, the drone work support method according to the sixth aspect is a drone work support method using a drone work support system for performing one or more tasks using one or more drones, which identifies one or more actual work areas in a target area where actual work is required based on an aerial image including an image of the target area, and when there are multiple actual work areas, creates a work plan including a movement path for moving in a straight line from the end point of the actual work in one actual work area to the start point of the actual work in the next actual work area and work content for performing the actual work between the start point of the actual work in each actual work area and the end point of the actual work, and transmits control information for executing the work plan to the drone.

This drone operation support method can reduce the drone's travel time when performing work, improving processing efficiency.

FIG. 1 is an overall configuration diagram of a drone operation support system according to one embodiment. FIG. 2 is a diagram illustrating the configuration and processing of a GNSS airborne marker according to an embodiment. FIG. 3 is a configuration diagram of a survey drone according to one embodiment. FIG. 4 is a configuration diagram of a work drone according to one embodiment. FIG. 5 is a flowchart of processing by a drone operation support system according to one embodiment. FIG. 6 is a diagram showing a task support screen according to an embodiment. FIG. 7 is a flowchart of processing by a drone operation support system according to a modified example. FIG. 8 is a first diagram illustrating the high-precision map generation process and the farm field condition analysis process according to the modified example. FIG. 9 is a second diagram illustrating the high-precision map generation process and the farm field condition analysis process according to the modified example. FIG. 10 is a diagram illustrating the work plan creation process.

The following embodiments are described with reference to the drawings. Note that the embodiments described below do not limit the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

In the following explanation, an embodiment of the present invention will be described using as an example an application in which an aerial drone configured as an unmanned aerial vehicle is used for inspecting and maintaining a farm field (target area) (e.g., identifying areas where disease or physiological disorders have occurred and spraying pesticides or fertilizers thereon). However, this application is merely an example for the purpose of explanation, and is not intended to limit the application of the embodiment to other applications.

Figure 1 is an overall configuration diagram of a drone operation support system according to one embodiment.

The drone operation support system 1 includes a GNSS (Global Navigation Satellite System) airborne beacon 2, a survey drone 3, an image analysis and map generation server 4, a control server 5, an operation terminal 6, and a work drone 7.

The GNSS airborne marker 2 is an airborne marker that can be photographed from the sky, and receives signals (GNSS signals) from GNSS satellites and transmits them to the image analysis and map generation server 4. Details of the GNSS airborne marker 2 will be described later.

The survey drone 3 photographs an area including a farm field from above and outputs the photographed images. In this embodiment, when photographing an area including a farm field, it is necessary to place GNSS airborne markers 2 in advance, for example, at the four corners and in the center of the area including the farm field, and after installing the GNSS airborne markers 2, they must be left in place for several tens of minutes to several hours, during which time GNSS data must be continuously collected. Details of the survey drone 3 will be described later.

The image analysis and map generation server 4 is configured, for example, by a computer (e.g., a PC (Personal Computer), a server computer) equipped with a processor, memory, and an interface. The image analysis and map generation server 4 includes an interface 41, a data storage and search unit 42, a map generation unit 43, a field condition analysis unit 44, and an interface 45.

The interface 41 communicates data via the network with the GNSS airborne beacon 2, the survey drone 3, the weather data distribution service, the ground sensor observation data distribution service, the GNSS correction data distribution service, etc. Specifically, the interface 41 receives GNSS data from the GNSS airborne beacon 2, receives images including the farm field from the survey drone 3, receives weather observation data from the weather data distribution service, receives ground sensor observation data from the ground sensor observation data distribution service, and receives GNSS correction data used for correction to improve the accuracy of the GNSS data from the GNSS correction data distribution service.

The data storage and search unit 42 stores and manages data received via the interface 41. Specifically, the data storage and search unit 42 stores and manages GNSS data from the GNSS airborne beacon 2, images including the farm field from the survey drone 3, meteorological observation data from a meteorological data distribution service, observation data by ground sensors from a ground sensor observation data distribution service, and GNSS correction data used for correction to improve the accuracy of the GNSS data from a GNSS correction data distribution service.

The map generation unit 43 generates map data (high-precision map data) by identifying the position information (e.g., latitude, longitude) of each position in the image based on the image received from the survey drone 3 stored in the data storage and search unit 42 and the GNSS data received from the GNSS anti-aircraft sign 2, and associating it with each position. Specifically, the map generation unit 43 identifies the anti-aircraft signs of multiple GNSS anti-aircraft signs 2 from the image, identifies the position information (e.g., latitude, longitude) of each GNSS anti-aircraft sign 2 based on the GNSS data acquired from each GNSS anti-aircraft sign 2 in the image, and associates the identified position coordinates with the position of the GNSS anti-aircraft sign 2 in the image. Note that positions other than the positions corresponding to the GNSS anti-aircraft sign 2 in the image can be identified based on the position information of multiple GNSS anti-aircraft signs 2. Identifying the position information using this GNSS anti-aircraft sign 2 can achieve high position accuracy, for example, on the order of cm to 10 cm.

The field condition analysis unit 44 analyzes the image of a specific field in the image captured by the survey drone 3 to identify the area where actual work is required (actual work area). For example, if the actual work is spraying pesticides in the field, the field condition analysis unit 44 identifies the area where disease is assumed to have occurred as the actual work area based on the color and shape of the crops and leaves in the image of the target field. The area where disease is assumed to have occurred from the image of the field can be identified, for example, by using a neural network model that has previously learned about images of disease. The field condition analysis unit 44 may also identify the content of the actual work to be performed in the actual work area (for example, the pesticide to be sprayed and the amount to be sprayed, etc.).

The interface 45 communicates data with the control server 5. The interface 45 transmits to the control server 5 the high-precision map data generated by the map generation unit 43 and the analysis results (information specifying the actual work area and the actual work content) identified by the field condition analysis unit 44.

The control server 5 is configured, for example, by a computer (e.g., a PC (Personal Computer), a server computer) equipped with a processor, memory, and an interface.

The control server 5 has a work plan creation unit 51 that creates a work plan for the work drone 7. The work plan creation unit 51 is an example of a drone movement path determination means, an operator movement path determination means, and a display control means. The work plan creation unit 51 is a functional unit configured by a processor executing a program. The work plan creation unit 51 receives high-precision map data and analysis results from the image analysis/map generation server 4. The work plan creation unit 51 determines a work plan for the work drone 7 that satisfies preset conditions based on the high-precision map data and the analysis results. Here, the work plan includes a flight path (an example of a movement path) from a takeoff position to a landing position via one or more actual work areas, and work content for performing actual work on the flight path. The takeoff position and landing position may be selected from preset candidate positions. The preset condition may be a work plan that can be realized within the range that the work drone 7 can fly with the battery capacity of the work drone 7. Therefore, if the battery capacity of the work drone 7 is not enough to pass through all of the actual work areas, the landing position will be selected from positions where the battery can be replaced (refillable positions). The battery capacity of the work drone 7 may be the standard battery capacity of the battery mounted on the work drone 7, or, if the actual battery capacity of the work drone 7 can be known, the actual battery capacity may be used.

The work plan may include information on the flight speed of the work drone 7. For example, if the work mechanism 72 of the work drone 7 always sprays pesticides or the like at a constant speed, the flight speed may be determined according to the amount sprayed in the actual work area. In addition, a condition may be that the flight path does not cross a specified road (e.g., a public road). In addition, the work plan creation unit 51 transmits the created work plan (specifically, control information for the work plan) to the work drone 7.

The work plan creation unit 51 also determines the operator's movement route from the takeoff position to the landing position of the flight route in the created work plan. Here, the operator's movement route may be subject to conditions such as passing through a road that the operator can move along and being a road that is close to the flight route of the work drone 7 (for example, a road that is a predetermined distance that allows the operation of the work drone 7 to be performed appropriately, for example, within a maximum of about 100 to 150 m). Note that if the road that the operator can move along is not within a predetermined distance from the flight route of the work drone 7, the road that is closest to the flight route may be used. Note that as for the road that the operator can move along, information on the road that the operator can move along may be associated with map data in advance, or it may be possible to recognize whether or not the road is a road that can be moved along from a map or image.

The work plan creation unit 51 also generates a work support screen 100 (see FIG. 6) based on the created work plan and the created movement route of the operator, and transmits it to the operation terminal 6. Details of the work support screen 100 will be described later.

The operation terminal 6 is a terminal for using the work support system 1, and is carried and used by, for example, an operator of the work drone 7. The operation terminal 6 may be, for example, a general-purpose portable information processing terminal (for example, a so-called mobile phone, a smartphone, a tablet terminal, a mobile PC, or a notebook personal computer). The operation terminal 6 has, for example, a function for performing data communication with the control server 5 via a communication network such as the Internet, a function for displaying various information, and a function for receiving signals from GPS satellites. In this embodiment, the operation terminal 6 displays a work support screen 100 (see FIG. 6) transmitted from the control server 5. The work support screen 100 may be displayed, for example, by a web browser of the operation terminal 6. The operation terminal 6 may have a function for performing data communication wirelessly with the work drone 7.

The work drone 7 performs certain tasks that need to be performed in the field (hereinafter referred to as actual tasks), such as selectively spraying pesticides at areas where disease has occurred in a vast field (actual task areas). Details of the work drone 7 will be described later.

Figure 2 is a diagram explaining the configuration and processing of a GNSS airborne marker according to one embodiment.

The GNSS airborne beacon 2 receives GNSS data transmitted from multiple GNSS satellites 9.

The GNSS satellite 9 includes a data generation unit 91 and a GNSS transmitter 92. The data generation unit 91 generates the GNSS data to be transmitted. The GNSS transmitter 92 transmits the GNSS data generated by the data generation unit 91.

The GNSS airborne beacon 2 includes a GNSS receiver 21, a data storage unit 22, and a transmitter 23. The GNSS receiver 21 receives GNSS data transmitted from a plurality of GNSS satellites 9. The data storage unit 22 stores the GNSS data received by the GNSS receiver 21. The transmitter 23 transmits the GNSS data stored in the data storage unit 22 to the image analysis/map generation server 4. The transmitter 23 may transmit data by communication in accordance with 5G (fifth generation mobile communication system).

Figure 3 is a configuration diagram of a survey drone according to one embodiment.

The survey drone 3 includes an ESC (Electronic Speed Controller) 31, a flight mechanism 32, a controller 33, a transmitter 34, a camera 35, and a GPS receiver 36.

The flight mechanism 32 is a mechanism for flying the survey drone 3, and has, for example, multiple sets of motors and propellers. The ESC 31 controls the rotation speed of the motor of the flight mechanism 32 according to instructions from the controller 33. The GPS receiver 36 receives GPS data from GPS satellites, measures the three-dimensional position (latitude, longitude, altitude) of the survey drone 3, and passes it to the control unit 33. The camera 35 takes images from the sky. The transmitter 34 transmits the images taken by the camera 35 to the image analysis and map creation server 4.

The controller 33 controls the ESC 31, the transmitter 34, the camera 35, and the GPS receiver 36. Specifically, the controller 33 controls the flight according to the input flight plan. The flight plan is information indicating the flight route to photograph the field where the work is to be performed. The controller 33 also adds the three-dimensional position of the survey drone 3 at the time the image was taken to the image taken by the camera 35, and passes the image to the transmitter 34.

The survey drone 3 is equipped with a radio controller (known as a radio transmitter) (not shown) that allows an operator to wirelessly control it remotely. The radio controller can wirelessly communicate with the controller 33, and transmits various control commands to the controller 33 in response to various user operations on the radio controller.

Figure 4 is a diagram showing the configuration of a work drone according to one embodiment.

The work drone 7 includes an ESC 31, a flight mechanism 32, a controller 71, a work mechanism 72, a transmitter 34, a camera 35, and a high-precision GNSS receiver 73. Note that functional parts similar to those of the survey drone 3 are given the same reference numerals and will not be described.

The high-performance GNSS receiver 73 receives GNSS data and measures the three-dimensional position of the work drone 7 with higher accuracy than the GPS receiver 36. As the GNSS receiver 73, for example, a receiver that measures the three-dimensional position using technologies such as DGPS (Differential GPS), D-RTK (Differential Real Time Kinematic), and dual-frequency GNSS can be adopted. The work mechanism 72 is a mechanism that performs work on the actual work area of the field. The work mechanism 72 sprays, for example, liquid pesticides and fertilizers, or powder pesticides and fertilizers. The work mechanism 72 may have a fixed amount of pesticides and fertilizers sprayed per unit time, or may be able to adjust the amount sprayed per unit time. The controller 71 controls the ESC 31, the work mechanism 72, the transmitter 34, the camera 35, and the high-precision GNSS receiver 73. The controller 71 performs control to fly and work according to the received work plan. Specifically, the controller 71 controls the ESC 31 according to the flight path of the work plan, and controls the work mechanism 72 according to the work content of the work plan.

The work drone 7 also comes with a radio controller (so-called radio control) (not shown) that allows the user to wirelessly control it remotely. The radio controller can wirelessly communicate with the controller 71, and transmits various control commands to the controller 71 in response to various user operations on the radio controller.

Next, we will explain the processing operations performed by the drone operation support system 1.

Figure 5 is a flowchart of processing by a drone operation support system according to one embodiment.

The survey drone 3 takes an image from above an area including the field, adds the three-dimensional position measured by the GPS receiver 36 to the image (step S1), and transmits the image to the image analysis/map generation server 4 (step S2).

Meanwhile, multiple GNSS airborne markers 2 placed in the field receive the GNSS data (step S3) and transmit the received GNSS data to the image analysis and map generation server 4 (step S4). The GNSS airborne markers 2 execute the processes of steps S3 and S4 for a period of time that is longer than that required for high-precision position analysis.

The image analysis and map generation server 4 analyzes the highly accurate position of each GNSS airborne marker 2 based on the GNSS data transmitted from the multiple GNSS airborne markers 2 (step S5). Next, the image analysis and map generation server 4 extracts the GNSS airborne markers 2 from the image of the farm field transmitted from the survey drone 3 (step S6). Next, the image analysis and map generation server 4 generates a highly accurate map by associating the positions of the GNSS airborne markers 2 in the image with the positions of the corresponding GNSS airborne markers 2 obtained by analysis in step S5 (step S7).

Next, the image analysis and map generation server 4 analyzes the condition of the field from the image of the field transmitted from the survey drone 3 (step S8). Specifically, the image analysis and map generation server 4 identifies the actual work area where work such as spraying pesticides or fertilizer will be performed. Next, the image analysis and map generation server 4 transmits the high-precision map generated in step S7 and the analysis result of step S8 (information indicating the identified actual work area: actual work area information) to the control server 5 (step S9).

The control server 5 receives the high-precision map and actual work area information sent from the image analysis/map generation server 4 (step S10), and creates a work plan based on the high-precision map and actual work area information, and creates a work support screen 100 including the work plan and the operator route (step S11). Next, the control server 5 sends the work plan (control information for executing the work plan) to the work drone 7, and sends the work support screen 100 to the operation terminal 6 (step S12).

Then, the work drone 7 moves and performs work according to the work plan received from the control server 5 (step S13). Meanwhile, the operation terminal 6 displays the work support screen 100 received from the control server 5 on the display unit (such as a liquid crystal panel) (step S14).

Figure 6 shows a work support screen according to one embodiment.

The work support screen 100 displays an image 101 including a farm field. In the example of FIG. 6, the image 101 includes two farm fields 104 and a road (e.g., a public road) 103. The farm field 104 may be made up of multiple small farm fields.

The work support screen 100 displays the flight path 110 of the work drone 7 and the movement path 120 of the operator on the image 101.

The flight path 110 includes a start point 111 indicating the takeoff location of the work drone 7, an end point 112 indicating the landing location, and one or more actual work areas 113, with a line connecting the start point 111 to the end point 112 via one or more actual work areas 113. Note that the work support screen 100 in FIG. 6 shows an example of a flight path 110 where the condition is that the work drone 7 does not fly across the road 103, so the actual work area 113 of the field 104 beyond the road 103 is not included in the same flight path 110.

In addition, in this embodiment, a drone mark 115 indicating the actual position of the work drone 7 is displayed on the flight path 110. Furthermore, at the end point 112, a warning message 114 is displayed as information indicating that the battery of the work drone 7 needs to be replaced. This warning message 114 allows the operator to know in advance that the battery needs to be replaced at the end point 112, so a replacement battery can be arranged in advance for the end point 112.

The operator's movement route 120 is a route that passes through roads on which the operator can move from the start point 111 to the end point 112.

In addition, the work support screen 100 displays an operator mark 121 indicating the current position of the operator (strictly speaking, the operation terminal 6), and an appropriate range 122 indicating a range of a predetermined distance (for example, a distance in the range of 100 to 150 m) suitable for operating the work drone 7 centered on the operator mark 121 is displayed. Depending on whether the drone mark 115 is present in this appropriate range 122, the operator can easily determine whether the work drone 7 is at an appropriate distance for operation. In the example of FIG. 6, the appropriate range is displayed centered on the position of the operator (the position of the operator mark 121 in FIG. 6), but for example, the appropriate range may be a predetermined distance range centered on the position of the work drone 7 (the position of the drone mark 115 in FIG. 6). In this case, depending on whether the operator is present in the appropriate range, it is easy to determine whether the distance to the work drone 7 is suitable for operation.

Here, we consider how to ensure that the work drone 7 can work in an appropriate position.

To ensure that the work drone 7 can work in the appropriate position, the position of the work drone 7 needs to be controlled with high precision, and the positional accuracy of the work drone 7 is key.

The position error Δ occurring in the work drone 7 is
Δ can be expressed as Δ=Δrobot + Δbasemap + Δsesing_data.

Here, Δrobot indicates the error caused by the position accuracy of the work drone 7 itself, and indicates the degree of accuracy with which the work drone 7 can move to a specified location when instructed to move to that location.

Δbasemap indicates the error caused by the positional accuracy of the map used when issuing work instructions to the work drone 7, and indicates the degree of accuracy with which the locations of features on the map are managed. This map is used, for example, when working the entirety of a field, when moving from a parking area to a field, when moving between fields, etc.

Δsensing_data indicates the error resulting from the data acquired to know the latest field conditions. Aerial photographs are a typical example of such data. Many aerial photographs store GNSS data at the time of shooting, but often contain an error of about ±10 m due to the positional accuracy of the survey drone. Also, in many cases, the orientation (direction) of the survey drone 3 at the time of shooting is not stored, making it difficult to properly overlay the aerial photograph on a map. Also, because there is a large error in altitude, the size and distance of features are often not displayed correctly when the image is overlaid on a map.

Technologies such as DGPS, D-RTK, and dual-frequency GNSS exist as ways to improve Δrobot, making it possible to achieve positioning accuracy on the order of centimeters.

There are techniques for improving the Δbasemap, such as ortho-generation processing using GNSS airborne markers, which can achieve positional accuracy on the order of centimeters to 10 cm.

One way to improve Δsensing_data is to implement technologies such as DGPS, D-RTK, and dual-frequency GNSS in survey drones, just like the Δrobot. However, these technologies are not implemented in survey drones that are in general use, and developing a dedicated drone that can use this technology would be costly, so the number of users of dedicated drones is limited.

In response to this, there is a method of using GNSS air beacons 2 (as with Δbasemap) as shown in the example above when acquiring sensing data (e.g., aerial images). With this method, it is necessary to keep the GNSS air beacons 2 in place at the site for several tens of minutes to several hours after installation to acquire GNSS data, and it is also necessary to install the GNSS air beacons 2, for example, at the four corners and the center of the shooting area, so it may be difficult to use when sensing a wide area.

In response to this, the positional accuracy of the work drone can be improved by using the drone operation support system according to the modified example shown below. For convenience, the drone operation support system according to the modified example will be explained with reference to FIG. 1.

In the drone operation support system according to the modified example, when polygon information (e.g., latitude and longitude information of the vertices of the outline of the field) that can identify the position and shape of the field with high accuracy can be prepared in advance, the position information of a specific position in the captured image can be determined with high accuracy based on this polygon information and the image captured by the survey drone 3.

In the drone operation support system, polygon information about farm fields is stored in the data storage and search unit 42 of the image analysis and map generation server 4. The polygon information about farm fields can be obtained, for example, from a farm field information service. The data storage and search unit 42 is an example of a map information storage means.

Figure 7 is a flowchart of processing by a drone operation support system according to a modified example.

The survey drone 3 takes an image from above the area including the field, adds the three-dimensional position measured by the GPS receiver 36 to the image (step S15), and transmits the image to the image analysis/map generation server 4 (step S16).

The image analysis and map generation server 4 generates a high-precision map based on the polygon information stored in the data storage and search unit 42 and the image of the field transmitted from the survey drone 3 (step S17: high-precision map generation process). Details of the high-precision map generation process will be described later.

Next, the image analysis and map generation server 4 analyzes the image of the field to identify the actual work area where work such as spraying pesticides or fertilizer will be performed, and extracts the position information of this actual work area based on the high-precision map (step S18: field condition analysis process). Next, the image analysis and map generation server 4 transmits the high-precision map generated in step S17 and the position information of the actual work area identified in step S18 (actual work area information) to the control server 5 (step S19).

The control server 5 receives the high-precision map and the actual work area information sent from the image analysis/map generation server 4 (step S20), and creates a work plan based on the high-precision map and the actual work area information, and creates a work support screen 100 including the work plan and the operator route (step S21). The control server 5 then transmits the work plan to the work drone 7, and transmits the work support screen 100 to the operation terminal 6 (step S22).

After this, the work drone 7 moves and performs work according to the work plan received from the control server 5 (step S23). Meanwhile, the operation terminal 6 displays the work support screen 100 received from the control server 5 on the display unit (step S24).

Figure 8 is a first diagram illustrating the high-precision map generation process (step S17 in Figure 7) and the field condition analysis process (step S18 in Figure 7) according to the modified example, and Figure 9 is a second diagram illustrating the high-precision map generation process and the field condition analysis process according to the modified example.

First, the data storage and search unit 42 of the image analysis and map generation server 4 registers the image of the field transmitted from the survey drone 3 (Step A).

Next, the map generation unit 43 (an example of a target area identification means and a location information association means) of the image analysis and map generation server 4 executes an edge detection process on the registered image 200 to extract an edge (field edge) 201 that is assumed to be a field (step B). Next, the map generation unit 43 identifies polygon information corresponding to the field edge 201 from the polygon information about the field stored in the data storage and search unit 42, and superimposes a polygon 210 based on the polygon information on the field edge 201 on the field edge 201 (step C). The polygon information corresponding to the field edge 201 is polygon information having position information close to the position information attached to the image 200 from which the field edge 201 was extracted, and is polygon information having a shape similar to that of the field edge 201 (for example, being the same or similar shape).

Next, the map generation unit 43 adjusts the angle of the field edge 201, i.e., rotates the field edge 201, so that each side of the field edge 201 is parallel to each side of the polygon 210 (step D).

Next, the map generation unit 43 adjusts the magnification ratio of the field edge 201 so that each side of the field edge 201 coincides with each side of the polygon 201 (step E).

Then, the map generation unit 43 rotates the image 200 of the field transmitted from the survey drone 3 by the adjustment angle in step C for the field edge 201, adjusts it by the magnification in step D, and matches the position of the polygon information to the adjusted image 202 (step F). This makes it possible to generate a highly accurate map in which the polygon 210 of the field is superimposed and matched to the image 202 based on the image 200 of the field transmitted from the survey drone 3. This makes it possible to identify the position (latitude, longitude) of a specific point in the image 202 with high accuracy based on the position information of the polygon information.

Next, the field condition analysis unit 44 (an example of an actual work area identification means and a position information identification means) analyzes the adjusted image 202 to extract the actual work area 220 where, for example, work such as spraying pesticides or fertilizers is performed (step G), identifies polygon information (for example, the latitude (lat) and longitude (lon) of each vertex indicating the position of the actual work area 220) corresponding to the extracted actual work area 220 (for example, 220A, 220B), and outputs the polygon information (position information) of the actual work area 220 together with the adjusted image 202 and the associated polygon information to the control server 5 (step H). Here, in FIG. 9, the polygon information of the actual work area 220A is shown as Polygon 1, and the polygon information of the actual work area 220B is shown as Polygon 2. This process makes it possible to output highly accurate position information about the actual work area 220 to the control server 5.

The drone operation support system according to the modified example can create a highly accurate map and obtain highly accurate position information of the actual work area even if the survey drone 3 is not equipped with a highly accurate GNSS receiver, and even if GNSS aerial beacons 2 are not placed to take photographs.

Next, a specific example of the work plan creation process will be described. This process corresponds to step S11 in FIG. 5 and step S22 in FIG. 7.

Figure 10 is a diagram explaining the work plan creation process.

When work is to be performed uniformly over the entire field, the work plan creation unit 51 (an example of a work plan creation means and a control information transmission means) creates a work plan as shown in FIG. 10(a). Specifically, the work plan creation unit 51 continues work while moving in a first direction (horizontal direction in the drawing), and when it reaches the end of the first direction, it moves in a second direction (vertical direction in the drawing) that intersects with the first direction without performing work, and after the movement, it moves in the first direction and performs work. Here, the movement path shown in FIG. 10(a) is called the entire field work path.

In addition, when the work plan creation unit 51 receives polygon information corresponding to the actual work area 220 (e.g., the latitude and longitude of each vertex of the actual work area 220) from the control server 5, it creates a work plan using one of methods 1 (Figure 10(c)) to 3 (Figure 10(e)).

The work plan according to method 1 is a work plan in which the work drone 7 moves along the same route as the overall work route and performs work only within the range of the actual work area 220.

The work plan according to method 2 is a work plan in which the work drone 7 moves along the overall work path, omitting the path in the first direction in which the actual work area 220 does not exist, and the work is performed only within the range of the actual work area 220.

The work plan according to method 3 is a movement plan in which the work drone 7 is moved linearly to a position (work start point) in the actual work area 220 where the work is to begin, and the actual work is performed, and after the work in that actual work area 220 is completed, the drone 7 is moved linearly to the work start point of the next actual work area 220 repeatedly. Note that the order in which the work in the actual work area 220 is performed may be the order that results in the shortest movement path.

According to the above-mentioned work plan creation process, regardless of which method is used to create the work plan, processing efficiency can be improved compared to when work is performed uniformly over the entire field. In particular, the work plan created by method 3 can shorten the travel distance of the work drone 7, resulting in the highest processing efficiency.

The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

For example, in the high-precision map generation process and field condition analysis process of Figures 8 and 9 described above, the actual work area 220 is extracted from the adjusted image 202, but the present invention is not limited to this. For example, the actual work area may be extracted from the unadjusted image 200, adjustments may be made to the actual work area in steps C and D, and the position information of the adjusted actual work area may be identified based on the polygon information of the field.

In addition, in the above embodiment and modified example, the control server 5 and the image analysis/map generation server 4 are configured as separate server computers, but the present invention is not limited to this, and they may be configured as a single processing server configured by the same server computer, and multiple functional means realized by the image analysis/map generation server 4 may be executed by more server computers.

In addition, in the above embodiment, a work drone that moves using battery power as energy has been shown as an example, but the present invention is not limited to this, and a work drone that moves using energy based on fuel such as gasoline or diesel may also be used. In this case, the battery replacement point described above may be read as a fuel replenishment point.

1...Drone operation support system, 2...GNSS aerial beacon, 3...Survey drone, 4...Image analysis and map generation server, 5...Control server, 6...Operation terminal, 7...Work drone

Claims (11)

A drone operation support system for performing one or more operations using one or more drones,
A drone movement path determination means for determining a movement path of the drone based on position information of an actual work area in which actual work is required in a predetermined target area;
An operator movement path determination means for determining a movement path along which the operator should move in accordance with the movement path of the drone;
A drone work support system including a display control means for displaying a work support screen on the operation terminal, the work support screen including the movement path of the drone and the movement path of the operator. The operator movement route determination means
The drone operation support system of claim 1 , wherein the movement route of the operator is determined so that the distance between the operator and the drone moving along the movement route of the drone is short. The drone movement route determination means includes:
Determine a movement path to a position where the energy can be replenished, via one or more actual work areas within a range where the drone can move using the energy possessed by the drone;
The operator movement route determination means
A movement path of the operator is determined to be a movement path leading to a position where the energy can be replenished;
The drone work support system of claim 1 or claim 2, wherein the display control means displays the energy replenishment location on the work support screen and also displays information indicating the energy replenishment. The display control means
The drone work support system according to any one of claims 1 to 3, wherein the work support screen displays information indicating the position of the operation terminal based on position information of the operation terminal. The display control means
The drone work support system of claim 4, wherein the work support screen displays a line indicating a range of a predetermined distance centered on the position of the operation terminal or the position of the drone. A drone operation support system for performing one or more operations using one or more drones,
A map information storage means for storing a high-precision map including high-precision position information of a plurality of positions that define the shape of a predetermined target area;
A target area identification means for identifying the target area from an aerial image including an image of the target area;
a position information matching means for matching a position of the target area in the aerial image with a position of the target area on the high precision map based on a shape of the target area in the aerial image and a shape of the target area on the high precision map;
an actual work area specifying means for specifying an actual work area in the target area where actual work is required based on the aerial photograph;
a position information specifying means for specifying highly accurate position information of the actual work area based on a correspondence between a position of the target area in the aerial image specified by the actual work area specifying means and a position of the target area on the high-precision map;
A drone movement path determination means for determining a movement path of the drone based on position information of the actual work area;
A drone work support system equipped with The actual work area specifying means
performing edge extraction on the aerial image to identify the shape of a target area;
The position information associating means
deforming the aerial image so that a shape of the target area in the aerial image coincides with a shape of the target area based on the high-precision map, and associating a corresponding position of the target area in the deformed aerial image with a corresponding position of the target area based on the high-precision map as being the same position;
The actual work area specifying means
The drone work support system of claim 6, wherein the actual work area is identified based on the deformed aerial image. A drone operation support system for performing one or more operations using one or more drones,
an actual work area specifying means for specifying one or more actual work areas in the target area where actual work is required based on an aerial image including an image of the target area;
a work plan creation means for creating a work plan including a moving path for moving linearly from an end point of an actual work in one actual work area to a start point of the actual work in the next actual work area, and work content for executing the actual work between the start point of the actual work in each actual work area and the end point of the actual work;
A drone work support system comprising: a control information transmission means for transmitting control information to the drone to execute the work plan. A drone operation support method by a drone operation support system for performing one or more tasks using one or more drones,
determining a movement path of the drone based on position information of an actual work area in which actual work is required in a predetermined target area;
Determine a route along which the operator should move in accordance with the route of movement of the drone;
A drone work support method that displays a work support screen on the operation terminal, the work support screen including the movement path of the drone and the movement path of the operator. A drone operation support method by a drone operation support system for performing one or more tasks using one or more drones,
storing a high-precision map including high-precision position information for a plurality of positions defining a shape of a predetermined target area;
Identifying the target area from an aerial image including an image of the target area;
Correlating a position of the target area in the aerial image with a position of the target area on the high-precision map based on a shape of the target area in the aerial image and a shape of the target area on the high-precision map;
Identifying an actual work area in the target area where actual work is required based on the aerial image;
Identifying highly accurate position information of the actual work area based on the correspondence between the identified position of the target area in the aerial image and the position of the target area on the high-precision map;
A drone work support method that determines the movement path of the drone based on position information of the actual work area. A drone operation support method by a drone operation support system for performing one or more tasks using one or more drones,
Identifying one or more actual work areas in the target area where actual work is required based on an aerial image including an image of the target area;
When there are a plurality of actual work areas, a work plan is created including a movement path for moving linearly from an end point of an actual work in one actual work area to a start point of the actual work in the next actual work area, and work content for executing the actual work between the start point of the actual work in each actual work area and the end point of the actual work;
A drone work support method for transmitting control information to the drone to execute the work plan.