WO2021029065A1

WO2021029065A1 - Management server and management system for flying object

Info

Publication number: WO2021029065A1
Application number: PCT/JP2019/032055
Authority: WO
Inventors: 智史岡村; 兼太郎深見; 高橋　和也
Original assignee: 株式会社センシンロボティクス
Priority date: 2019-08-15
Filing date: 2019-08-15
Publication date: 2021-02-18
Also published as: JPWO2021029065A1; JP6678983B1

Abstract

[Problem] To provide a management server and a management system that do not require, in operation of creating a flying route of a flying object, setting a waypoint by a hand or the like to at least a part of the flying route. [Solution] A management server according to the present invention is a management server that is connected to a user terminal and a flying object via a network and manages a flying route of the flying object, the management server comprising: a first flying route acquisition unit that acquires, from outside the management server, a first flying route from a flying start position to a target object, the first flying route being created on the basis of an overhead view image photographed from a satellite or the flying object or a figure showing the overhead view image on a plane; a second flying route creation unit that creates a second flying route for acquiring information about the target object; and a third flying route creation unit that combines the first flying route with the second flying route to create a third flying route.

Description

Aircraft management server and management system

The present invention relates to an air vehicle management server and a management system.

In recent years, flying objects (hereinafter collectively referred to as "aircraft") such as drones and unmanned aerial vehicles (UAVs) have begun to be used in industry. Under these circumstances, Patent Document 1 discloses a system in which an air vehicle sequentially shoots an imaged object at a plurality of waypoints set in advance.

Japanese Unexamined Patent Publication No. 2014-089160

However, in the disclosed technology of Patent Document 1, it is necessary to manually set and memorize all waypoints in order to create a flight route of an air vehicle. Therefore, when the flight route becomes long, it takes time and effort to set waypoints for the entire flight route, and it lacks speed, especially in situations where the work time is short and limited. It was.

The present invention has been made in view of such a background, and in particular, in the work of creating a flight route of an air vehicle, it is not necessary to manually set waypoints for at least a part of the flight route. The purpose is to provide a management server and a management system.

The main invention of the present invention for solving the above problems is a management server that manages the flight route of the flying object, which is connected to the user terminal and the flying object via a network, and is imaged from a satellite or the flying object. A first flight route acquisition unit that acquires a first flight route from a flight start position to an object from outside the management server, which is generated based on a bird's-eye view image or a diagram showing the bird's-eye view image on a plane, and the target. A third flight route that combines a second flight route generator that generates a second flight route for acquiring information about an object, the first flight route, and the second flight route to generate a third flight route. It is a management server characterized by including a generation unit.

According to the present invention, it is possible to provide a management server and a management system that do not require manual input or the like to set waypoints for at least a part of the flight route, particularly in the work of creating a flight route of an air vehicle. it can.

It is a figure which shows the structure of the management system which concerns on embodiment of this invention. It is a block diagram which shows the hardware configuration of the management server of FIG. It is a block diagram which shows the hardware configuration of the user terminal of FIG. It is a block diagram which shows the hardware composition of the flying object of FIG. It is a block diagram which shows the function of the management server of FIG. It is a block diagram which shows the structure of the storage part of FIG. This is an example of a display screen according to the embodiment of the present invention. This is an example of a display screen according to the embodiment of the present invention. It is another example of the display screen which concerns on embodiment of this invention. It is another example of the display screen which concerns on embodiment of this invention.

The contents of the embodiments of the present invention will be described in a list. The flight management server and flight management system according to the embodiment of the present invention have the following configurations.
[Item 1]
A management server that manages the flight route of the aircraft, which is connected to the user terminal and the aircraft via a network.
The first flight to acquire the first flight route from the flight start position to the object, which is generated based on the bird's-eye view image taken from the satellite or the flying object or the figure showing the bird's-eye view image on a plane, from outside the management server. Route acquisition department and
A second flight route generator that generates a second flight route for acquiring information about the object,
A third flight route generator that combines the first flight route and the second flight route to generate a third flight route,
Management server with.
[Item 2]
The first flight route is a movement route along a path through which animals including humans and moving objects having a different form from the flying object can move.
The management server according to item 1, characterized in that.
[Item 3]
The object is the last object to reach among the plurality of objects.
The first flight route is a route that passes through other objects other than the last object among the plurality of objects as transit points.
The management server
Further provided with a flight route dividing unit that sequentially divides the first flight route from the flight start position to each waypoint.
The third flight route generation unit sequentially connects each of the divided first flight routes and the second flight route in each of the plurality of objects from the flight start position, and the third flight route. To generate,
The management server according to

item

1 or 2, characterized in that.
[Item 4]
The third flight route generation unit generates a plurality of flight routes as the third flight route, and generates a plurality of flight routes.
The management server
It further includes a candidate flight route setting unit that sets the plurality of flight routes as a plurality of candidate flight routes so as to be selectable.
The management server according to items 1 to 3, characterized in that.
[Item 5]
The candidate flight route setting unit sets information related to each candidate flight route so that the plurality of candidate flight routes can be displayed on the user terminal in a distinctive manner.
The management server according to item 4, characterized in that.
[Item 6]
The candidate flight route related information is color information.
The management server according to item 5, characterized in that.
[Item 7]
A route editorial unit for changing a part of the third flight route to another route is further provided.
The management server according to items 1 to 6, characterized in that.
[Item 8]
Further provided with a return route generation unit that generates a return route from the flight end point of the second flight route in the object to the flight start position or a flight end position different from the flight start position.
The third flight route further includes the return route.
The management server according to items 1 to 7, characterized in that.
[Item 9]
The management server generates a third flight route based on the flight start position and the position of the object selected by the user terminal.
The management server according to items 1 to 8, characterized in that.
[Item 10]
The second flight route is a route that turns around the center coordinates of the object.
The management server according to items 1 to 9, wherein the management server is characterized by the above.
[Item 11]
A past flight route storage unit that stores the third flight route as a past flight route,
The management server according to items 1 to 10, wherein the management server further includes a past flight route calling unit that sets a past flight route stored in the past flight route storage unit as the third flight route. ..
[Item 12]
The information is a still image or a moving image.
The management server according to items 1 to 11, characterized in that.
[Item 13]
An air vehicle management system that includes a management server that manages the flight route of the air vehicle, which is connected to the user terminal and the air vehicle via a network.
The management server is;
The first flight route from the flight start position to the object, which is generated based on the bird's-eye view image taken from the satellite or the aircraft or the bird's-eye view image on a plane, is acquired from outside the management server;
Generate a second flight route to obtain information about the object;
The first flight route and the second flight route are combined to generate a third flight route;
Management system.

<Details of the embodiment>
Hereinafter, the management server and the management system of the unmanned aircraft according to the embodiment of the present invention will be described in particular, the embodiment of the management system (hereinafter referred to as “the present system”). In the accompanying drawings, the same or similar elements are given the same or similar reference numerals and names, and duplicate description of the same or similar elements may be omitted in the description of each embodiment. In addition, the features shown in each embodiment can be applied to other embodiments as long as they do not contradict each other.

<Composition>
As shown in FIG. 1, this system includes a management server 1, a plurality of

user terminals

2 and 3, one or more flying objects 4, and one or more flying object storage devices 5. The management server 1, the

user terminals

2, 3 and the flying object 4 and the flying object storage device 5 are connected to each other so as to be able to communicate with each other via a network. The illustrated configuration is an example, and is not limited to this. For example, a configuration may be carried by a user without having the flying object storage device 5.

<Management server 1>
FIG. 2 is a diagram showing a hardware configuration of the management server 1. The illustrated configuration is an example, and may have other configurations.

As shown in the figure, the management server 1 is connected to a plurality of

user terminals

2 and 3, an air vehicle 4, and an air vehicle storage device 5 to form a part of this system. The management server 1 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

The management server 1 includes at least a processor 10, a memory 11, a storage 12, a transmission / reception unit 13, an input / output unit 14, and the like, and these are electrically connected to each other through a bus 15.

The processor 10 is an arithmetic unit that controls the operation of the entire management server 1, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing. For example, the processor 10 is a CPU (Central Processing Unit), and executes each information processing by executing a program or the like for the system stored in the storage 12 and expanded in the memory 11.

The memory 11 includes a main memory composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary memory composed of a non-volatile storage device such as a flash memory or an HDD (Hard Disk Drive). .. The memory 11 is used as a work area of the processor 10, and also stores a BIOS (Basic Input / Output System) executed when the management server 1 is started, various setting information, and the like.

The storage 12 stores various programs such as application programs. A database storing data used for each process may be built in the storage 12.

The transmission / reception unit 13 connects the management server 1 to the network and the blockchain network. The transmission / reception unit 13 may be provided with a short-range communication interface of Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

The bus 15 is commonly connected to each of the above elements and transmits, for example, an address signal, a data signal, and various control signals.

<

User terminals

2, 3>
The

user terminals

2 and 3 shown in FIG. 3 also include a processor 20, a memory 21, a storage 22, a transmission / reception unit 23, an input / output unit 24, and the like, which are electrically connected to each other through a bus 25. Since the functions of each element can be configured in the same manner as the management server 1 described above, detailed description of each element will be omitted.

<Flight 4>
FIG. 4 is a block diagram showing a hardware configuration of the flying object 4. The flight controller 41 can have one or more processors such as a programmable processor (eg, a central processing unit (CPU)).

Further, the flight controller 41 has a memory 411 and can access the memory. Memory 411 stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps. Further, the flight controller 41 may include sensors 412 such as an inertial sensor (accelerometer, gyro sensor), GPS sensor, proximity sensor (for example, rider) and the like.

The memory 411 may include, for example, a separable medium such as an SD card or a random access memory (RAM) or an external storage device. The data acquired from the cameras / sensors 42 may be directly transmitted and stored in the memory 411. For example, still image / moving image data taken by a camera or the like may be recorded in the internal memory or an external memory, but the present invention is not limited to this, and at least the management server 1 or the management server 1 or the internal memory may be recorded from the camera / sensor 42 or the internal memory via the network NW. It may be recorded in one of the

user terminals

2, 3 and the air vehicle storage device 5. The camera 42 is installed on the aircraft 4 via the gimbal 43.

The flight controller 41 includes a control module (not shown) configured to control the state of the flying object. For example, the control module adjusts the spatial placement, velocity, and / or acceleration of an air vehicle with six degrees of freedom (translational motion x, y and z, and rotational motion θ _x , θ _y and θ _z ). , ESC44 (Electric Speed Controller) to control the propulsion mechanism (motor 45, etc.) of the flying object. The propeller 46 is rotated by the motor 45 supplied from the battery 48 to generate lift of the flying object. The control module can control one or more of the states of the mounting unit and the sensors.

The flight controller 41 is configured to transmit and / or receive data from one or more external devices (eg, transmitter / receiver (propo) 49, terminal, display device, or other remote control). It is possible to communicate with the unit 47. The transmitter / receiver 49 can use any suitable communication means such as wired communication or wireless communication.

For example, the transmission / reception unit 47 uses one or more of a local area network (LAN), a wide area network (WAN), infrared rays, wireless, WiFi, a point-to-point (P2P) network, a telecommunications network, and cloud communication. can do.

The transmission / reception unit 47 transmits and / or receives one or more of the data acquired by the sensors 42, the processing result generated by the flight controller 41, the predetermined control data, the user command from the terminal or the remote controller, and the like. be able to.

Sensors 42 according to this embodiment may include an inertial sensor (accelerometer, gyro sensor), GPS sensor, proximity sensor (eg, rider), or vision / image sensor (eg, camera).

<Management server function>
FIG. 5 is a block diagram illustrating the functions implemented in the management server 1. In the present embodiment, the management server 1 includes a communication unit 110, a flight mission generation unit 120, a first flight route acquisition unit 130, a flight route division unit 140, a past flight route calling unit 150, a storage unit 160, and a report generation unit. It has 170. The flight mission generation unit 120 includes a second flight route generation unit 121, a third flight route generation unit 123, a candidate flight route setting unit 125, a route editing unit 127, and a return route generation unit 129. Further, the storage unit 160 includes various databases of the flight route storage unit 162, the flight log storage unit 164, and the interface information storage unit 166. Further, as shown in FIG. 6, the flight route storage unit 162 includes a first flight route storage unit 1621, a second flight route storage unit 1623, a third flight route storage unit 1625, a return route storage unit 1627, and a past flight route storage unit. Includes part 1629.

The communication unit 110 communicates with the

user terminals

2 and 3, the flying object 4, and the flying object storage device 5. The communication unit 110 also functions as a reception unit that receives flight requests from the

user terminals

2 and 3. The flight request may include any of the flight location, flight purpose, and number of flying objects.

Flight mission generation unit 120 generates flight missions. Flight missions include at least flight routes. The flight route is generated as a third flight route by the third flight route generation unit 123 with reference to the first flight route storage unit 1621 and the second flight route storage unit 1623. The first flight route will be described later. The second flight route is generated by the second flight route generation unit 121, and is automatically generated by setting the target turning radius, the number of waypoints (WP) during turning, and the like, as shown in FIG. 7, for example. It may be a generated configuration, it may be a configuration that calls a preset route and / or waypoint, or it may manually set a waypoint according to the shape of the object or the route length and working time of the second flight route. You may do so. Although the second flight route is a turning route as a specific example of the present embodiment, the route is not limited to this and may be an arbitrarily generated route.

The third flight route generation unit 123 generates a third flight route by combining the first flight route and the second flight route. As one of the specific generation examples, each flight route is composed of a plurality of waypoints (including position coordinate information), and the closest waypoints (position coordinates) of the first flight route and the second flight route. ) Can be used as a connection point to generate a third flight route so as to fly continuously on both flight routes, but the present invention is not limited to this.

When there are a plurality of first flight routes searched or generated, the candidate flight route setting unit 125 generates a third flight route corresponding to each of the first flight routes, so that the plurality of third flight routes can be selected. Each can be selected and set as multiple candidate flight routes. As a more specific example, for example, as shown in FIG. 10, the flight time of each candidate flight route is displayed, and the candidate flight route related information (for example, color information) so that each route can be selected is displayed. Etc.) are set, and the user can select one of the flight routes.

The route editing unit 127 changes a part of the route to a desired route according to the user's instruction after the third flight route is generated and selected by the user as described above. As a result, when the first flight route searched or generated by the method described later does not satisfy all the route conditions, or when you want to create a new flight route based on the third flight route used in the past as described later. Even in (for example, increasing or decreasing the number of objects based on the previous flight result), it is possible to generate a flight route modified by the user as the final flight route.

The return route generation unit 129 generates a return route from the flight end point of the second flight route in the object to the flight start position or the flight end position different from the flight start position, and stores it in the return route storage unit 1627. .. This return route is combined with the first flight route and the second flight route when the third flight route is generated by the third flight route generation unit 123. The return route generation unit 129 does not necessarily have to be provided. For example, the flight start position may be automatically set as the flight end position, or the flight may be acquired from outside the management server 1 as in the first flight route. It may be configured.

The flight route may be, for example, a configuration in which the position where the aircraft is carried by the user is set as the flight start position or the user collects the aircraft at the flight end position without having the flight object storage device 5. Then, based on the information of the flight object storage device 5 managed by the management server 1 (for example, position information, storage state information, storage device information, etc.), the flight object storage device selected as the flight start position or flight end position. The configuration may be generated as a flight route including the position of 5.

The first flight route acquisition unit 130 acquires the first flight route generated outside the management server 1 via the communication unit 110 and stores it in the first flight route storage unit 1621. The first flight route is based on, for example, a bird's-eye view image (for example, a satellite photograph or an aerial photograph) taken from a satellite or an air vehicle or a map (for example, a map or a route map) showing the bird's-eye view image on a plane. It was generated. As a more specific example, the first flight route is a movement route (for example, a roadway, a railroad track, a route, a sidewalk, a mountain road) along a route in which an animal including a moving body or an animal including a human can move. , Beast trail, etc.). For this movement route, for example, the flight start position, the waypoint, the position of the object, etc. are determined based on the above-mentioned images and figures by using API (Application Programming Network) or application software on the

user terminals

2 and 3. If specified, the location information is transmitted directly or via the management server 1 to an external server or the like (not shown) via the network NW, and the moving body based on the location information searches for a movable route. The result is transmitted to the management server 1. The management server 1 can acquire the route of the search result as the first flight route and store it in the first flight route storage unit 1621. This eliminates the need to manually set all waypoints for the movement route to the object, reducing the work load and work time. Further, since the flight is performed on the movement route, it is possible to acquire not only the information on the object but also the information on the movement route to the object. Therefore, it can be used in various situations such as confirmation of traffic congestion on the road to the object, confirmation of the route to the object in the event of a disaster, and search for a mountain road when searching for a victim. The acquired information on the movement route at this time may be a moving image, or may be an image captured at an appropriate number of waypoints automatically generated.

Further, in the above-mentioned specific example, it is possible to reduce the load of the management server 1 by performing a route search outside the management server 1, but if the storage capacity and processing capacity of the management server 1 allow, it is managed. It is also possible to perform a route search within the server 1. Further, in the above-mentioned specific example, the first flight route is defined as a movement route along a path in which a moving body different from the flying body or an animal including a human can move, but the first flight route is not limited to this. It may be a route obtained by image recognition (for example, a ridge such as a mountain range, a fence, a roof, an electric wire, etc.), a general search method (for example, Dijkstra's algorithm, best-first search, A algorithm, etc.), AI, etc. It is possible to target all routes that can be searched using.

When there are a plurality of objects, the flight route dividing unit 140 sets a flight start position on a first flight route that passes through a first flight route as a transit point, except for the object that reaches the end of the plurality of objects. Divide sequentially for each waypoint from. In line with this, in the third flight route generation unit, each of the divided first flight routes and the second flight route in each of the plurality of objects are sequentially combined from the flight start position to form the third flight. Generate a route.

The past flight route calling unit 150 calls the past flight route as the third flight route with reference to the past flight route storage unit 1629. The third flight route generated in the past is manually or automatically stored in the past flight route storage unit 1629. A part of the past flight route can be changed by the route editorial unit 127 before or after the call.

The interface information storage unit 166 stores various control information for display on the display units (displays and the like) of the

user terminals

2 and 3.

The report generation unit 170 generates report information to be transmitted to the

user terminals

2 and 3 based on the flight log storage unit 164. In the present embodiment, for example, it is acquired by the aircraft 4 on the flight route (on the first flight route and / or on the second flight route) and / or on the return route set in the flight mission. Information (still image, moving image, sound and other information) is stored in the flight log storage unit 164.

A specific display example of this system is shown with reference to FIGS. 7 and 8. In FIGS. 7 and 8, it is assumed that there is one target. First, as shown in FIG. 7, on the left side of the display screen, parameters such as the flight speed and altitude of the flying object, the turning radius of the object, and the number of waypoints (WP) during turning may be set, and the right side of the display screen may be set. On the image of, the flight start position (illustration of the drone) and the object position (number 1) may be selectable by a pointer, a touch operation, or the like. And on the lower left side of the display screen, even if a route creation button for starting route creation based on the selected flight start position and object position, a route call button for calling a route created in the past, etc. are arranged. Good.

Next, in FIG. 8, the acquired first flight route is the part shown by the solid line, the generated second flight route is the part shown by the dotted line, and the generated return route is one point. Although it is the part shown by the chain line, on the actual display screen, the route display does not have to be divided in this way, and it is continuously displayed as the third flight route with a solid line of a predetermined color or the like. You may. As shown in FIG. 8, the route edit button for changing a part of the generated third flight route, the route save button for saving the third flight route in the past flight route storage unit 1629, and the aircraft are A route execution button or the like for starting the flight of the third flight route may be arranged.

A specific display example of this system is shown with reference to FIGS. 9 and 10. In FIGS. 9 and 10, it is assumed that there are two targets and two first flight routes. First, as shown in FIG. 9, on the left side of the display screen, parameters such as the speed and altitude of the flying object, the turning radius of the object, and the number of waypoints (WP) during turning may be set, and the right side of the display screen may be set. On the image, the flight start position (illustration of the drone), the object position (numbers 1 and 2), and the flight end position (illustration of the flying object storage device) may be selectable by a pointer or a touch operation. And on the lower left side of the display screen, even if a route creation button for starting route creation based on the selected flight start position and object position, a route call button for calling a route created in the past, etc. are arranged. Good.

Next, in FIG. 10, the acquired first flight route is the part shown by the solid line, the generated second flight route is the part shown by the dotted line, and the generated return route is one point. Although it is the part shown by the chain line, on the actual display screen, the route display does not have to be divided in this way, and it is continuously displayed as the third flight route with a solid line of a predetermined color or the like. You may. Further, as shown in FIG. 10, when there are a plurality of third flight routes, they are displayed as

candidate flight routes

1 and 2 together with each flight time, and each route is displayed in a different color and can be selected. May be good. Further, as shown in FIG. 10, a route edit button for changing a part of the generated third flight route and a route save button for saving the third flight route in the past flight route storage unit 1629, A route execution button or the like for the aircraft to start flying on the third flight route may be arranged.

The air vehicle of the present invention can be used in an airplane-related industry such as a multicopter drone, and further, the present invention can be suitably used as an air vehicle for aerial photography equipped with a camera or the like. It can also be used in various industries such as security, agriculture, infrastructure monitoring, surveying, sports venue inspections such as golf courses and tennis courts, roof inspections of buildings such as factories and warehouses, disaster response, and disaster response.

The above-described embodiment is merely an example for facilitating the understanding of the present invention, and is not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes an equivalent thereof.

1 Management server 2 User terminal 4 Aircraft

Claims

A management server that manages the flight route of the aircraft, which is connected to the user terminal and the aircraft via a network.
The first flight to acquire the first flight route from the flight start position to the object, which is generated based on the bird's-eye view image taken from the satellite or the flying object or the view showing the bird's-eye view image on a plane, from outside the management server. Route acquisition department and
A second flight route generator that generates a second flight route for acquiring information about the object,
A third flight route generator that combines the first flight route and the second flight route to generate a third flight route,
Management server with.
The first flight route is a movement route along a path through which animals including humans and moving objects having a different form from the flying object can move.
The management server according to claim 1.
The object is the last object to reach among the plurality of objects.
The first flight route is a route that passes through other objects other than the last object among the plurality of objects as transit points.
The management server
Further provided with a flight route dividing unit that sequentially divides the first flight route from the flight start position to each waypoint.
The third flight route generation unit sequentially connects each of the divided first flight routes and the second flight route in each of the plurality of objects from the flight start position, and the third flight route. To generate,
The management server according to claim 1 or 2.
The third flight route generation unit generates a plurality of flight routes as the third flight route, and generates a plurality of flight routes.
The management server
It further includes a candidate flight route setting unit that sets the plurality of flight routes as a plurality of candidate flight routes so as to be selectable.
The management server according to claim 1 to 3.
The candidate flight route setting unit sets information related to each candidate flight route so that the plurality of candidate flight routes can be displayed on the user terminal in a distinctive manner.
The management server according to claim 4.
The candidate flight route related information is color information.
The management server according to claim 5.
A route editorial unit for changing a part of the third flight route to another route is further provided.
The management server according to claim 1 to 6.
Further provided with a return route generation unit that generates a return route from the flight end point of the second flight route in the object to the flight start position or a flight end position different from the flight start position.
The third flight route further includes the return route.
The management server according to claim 1 to 7.
The management server generates a third flight route based on the flight start position and the position of the object selected by the user terminal.
The management server according to claim 1 to 8.
The second flight route is a route that turns around the center coordinates of the object.
The management server according to claim 1 to 9.
A past flight route storage unit that stores the third flight route as a past flight route,
The management according to claim 1 to 10, wherein the management server further includes a past flight route calling unit that sets a past flight route stored in the past flight route storage unit as the third flight route. server.
The information is a still image or a moving image.
The management server according to claim 1 to 11.
An air vehicle management system that includes a management server that manages the flight route of the air vehicle, which is connected to the user terminal and the air vehicle via a network.
The management server is;
The first flight route from the flight start position to the object, which is generated based on the bird's-eye view image taken from the satellite or the aircraft or the bird's-eye view image on a plane, is acquired from outside the management server;
Generate a second flight route to obtain information about the object;
The first flight route and the second flight route are combined to generate a third flight route;
Management system.