JP7067897B2 - Information processing equipment, flight control instruction method, program, and recording medium - Google Patents

Description

The present disclosure relates to an information processing device for instructing flight control of a plurality of flying objects, a flight control instruction method, a program, and a recording medium.

In recent years, it has been known that a plurality of unmanned aerial vehicles fly in cooperation in one area. In order to fly a plurality of unmanned aerial vehicles in cooperation with each other, for example, by executing a preset flight program, a plurality of unmanned aerial vehicles can fly in cooperation with each other (see Patent Document 1). In Patent Document 1, a plurality of projectiles as a plurality of unmanned aerial vehicles stop moving to a designated position in the air by a command from a ground station and emit light. As a result, the observer can observe the constellations and the like in a pseudo manner.

Japanese Unexamined Patent Publication No. 2016-206443

The flying object described in Patent Document 1 can fly according to a preset flight route and flight position, but it is difficult to fly in consideration of a flight route and a flight position not set in advance. .. For example, the system described in Patent Document 1 cannot specify the flight shape formed by a plurality of flying objects in real time, and the degree of freedom during flight of an unmanned aerial vehicle is low. In addition, the work for setting the flight route and flight position is complicated and difficult.

In addition, when the flight of an unmanned aerial vehicle is imaged using an operating device (propo), the flight route and flight position can be instructed to the unmanned aerial vehicle in real time, reflecting the intention of the operator. However, in order to operate a plurality of unmanned aerial vehicles, a plurality of operating devices are required, and it is difficult to operate a plurality of unmanned aerial vehicles in cooperation with each other.

In one aspect, the information processing device is an information processing device that instructs the control of the flight of a plurality of flying objects, and includes a processing unit, and the processing unit is one or more for controlling the flight of the plurality of flying objects. Acquires information about the control node of, associates one or more control nodes with multiple flying objects, and based on the movement of one or more control nodes, one or more flying objects associated with the moved control node. To move.

The processing unit may associate an arbitrary flying object among a plurality of flying objects with a control node having the shortest distance to the arbitrary flying object and a control node having the second shortest distance to the arbitrary flying object. ..

The processing unit acquires a first operation information for associating one or more control nodes with a plurality of flying objects, and based on the first operation information, one or more control nodes and a plurality of flying objects. May be associated.

The information of the control node may include the position information of the control node. The processing unit acquires the position information of a plurality of flying objects, and based on the position information of one or more control nodes and the position information of the plurality of flying objects, the processing unit obtains the position information of the one or more control nodes and the plurality of flying objects. The value of the association may be calculated.

The processing unit may acquire the second operation information for moving one or more control nodes, and may move one or more control nodes based on the second operation information.

The processing unit may hold movement restriction information for restricting the movement of one or more control nodes, and may restrict the movement of one or more control nodes based on the movement restriction information.

The processing unit calculates the positions of a plurality of flying objects when the flying objects associated with the moved control node are moved, and when the distance between the calculated positions of the plurality of flying objects is equal to or greater than the threshold value, the processing unit calculates the positions of the plurality of flying objects. The flying object associated with the moved control node is moved, and when the distance is less than the threshold value, the flying object associated with the moved control node is not moved.

The processing unit may set the positional relationship of a plurality of control nodes.

The information processing device may be a terminal. The processing unit may generate movement control information for moving the flying object associated with the moved control node, and may transmit the movement control information to a plurality of flying objects.

The information processing device may be any flying object among a plurality of flying objects. The processing unit generates movement control information for moving the flying object associated with the moved control node.
You may move based on the movement control information.

In one aspect, the flight control instruction method is a flight control instruction method in an information processing device that instructs the control of the flight of a plurality of flying objects, and is one or more control nodes for controlling the flight of the plurality of flying objects. A step to get information about, a step to associate one or more control nodes with multiple air vehicles, and one or more flights associated with a moved control node based on the movement of one or more control nodes. It has a step to move the body.

The step of associating one or more control nodes with multiple flying objects is the distance between any flying object of the plurality of flying objects, the control node having the shortest distance to any flying object, and any flying object. May include a step, which associates with the second shortest control node.

The flight control instruction method may further include a step of acquiring a first operation information for associating one or more control nodes with a plurality of flying objects. The step of associating one or more control nodes with the plurality of flying objects may include associating the one or more control nodes with the plurality of flying objects based on the first operation information.

The information of the control node may include the position information of the control node. The flight control instruction method may further include a step of acquiring position information of a plurality of flying objects. The step of associating one or more control nodes with multiple air vehicles is based on the position information of one or more control nodes and the position information of multiple air vehicles. It may include a step of calculating the value of the association with.

The flight control instruction method includes a step of acquiring a second operation information for moving one or more control nodes and a step of moving one or more control nodes based on the second operation information. Further may be included.

The flight control instruction method includes a step of holding movement restriction information for restricting the movement of one or more control nodes, and a step of restricting the movement of one or more control nodes based on the movement restriction information. Further may be included.

The step of moving the flying object is the distance between the step of calculating the positions of the plurality of flying objects when the flying objects associated with the moved control node are moved and the calculated positions of the plurality of flying objects. Is greater than or equal to the threshold, the step of moving the flying object associated with the moved control node, and if the distance is less than the threshold, the moving object associated with the moved control node is not moved. And may include.

The flight control instruction method may further include a step of setting the positional relationship of a plurality of control nodes.

The information processing device may be a terminal. The step of moving the flying object includes a step of generating movement control information for moving the flying object associated with the moved control node and a step of transmitting the movement control information to a plurality of flying objects. May include.

The information processing device may be any flying object among a plurality of flying objects. The step of moving the flying object may include a step of generating movement control information for moving the flying object associated with the moved control node. The flight control instruction method may further include a step of moving based on the movement control information.

In one aspect, the program is a step of acquiring information from one or more control nodes for controlling the flight of a plurality of aircraft to an information processing device that directs the control of the flight of the plurality of aircraft. Perform a step of associating the above control node with a plurality of flying objects and a step of moving one or more flying objects associated with the moved control node based on the movement of the one or more control nodes. It is a program for.

In one embodiment, the recording medium is a step of acquiring information of one or more control nodes for controlling the flight of a plurality of flying objects from an information processing device instructing the control of the flight of the plurality of flying objects, and one. Performs a step of associating one or more control nodes with multiple flying objects and a step of moving one or more flying objects associated with the moved control node based on the movement of the one or more control nodes. It is a computer-readable recording medium on which a program for making a flight is recorded.

The outline of the above invention does not list all the features of the present disclosure. A subcombination of these feature groups can also be an invention.

Schematic diagram showing a first configuration example of the flight body group control system in the first embodiment Schematic diagram showing a second configuration example of the flight body group control system in the first embodiment A diagram showing an example of the concrete appearance of an unmanned aerial vehicle Block diagram showing an example of the hardware configuration of an unmanned aerial vehicle Block diagram showing an example of terminal hardware configuration Diagram explaining the positional relationship between the control node and the unmanned aerial vehicle group Diagram explaining the movement operation of a group of unmanned aerial vehicles Flow chart showing the first example of the operation procedure of the terminal and the unmanned aerial vehicle Flow chart showing the second example of the operation procedure of the terminal and the unmanned aerial vehicle

Hereinafter, the present disclosure will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Not all combinations of features described in the embodiments are essential to the solution of the invention.

The claims, description, drawings, and abstracts include matters subject to copyright protection. The copyright holder will not object to any reproduction of these documents by any person, as shown in the Patent Office files or records. However, in other cases, all copyrights are reserved.

In the following embodiment, an unmanned aerial vehicle (UAV) is exemplified as an air vehicle. Unmanned aerial vehicles include aircraft that move in the air. In the drawings attached to this specification, the unmanned aerial vehicle is referred to as "UAV". Further, as the information processing device, an unmanned aerial vehicle, a terminal, and a PC (Personal Computer) are exemplified. The information processing device may be an unmanned aerial vehicle, a device other than a terminal or a PC, for example, a transmitter (Propotional Controller), or other device. The flight control instruction method defines the operation of the information processing device. Further, the recording medium is one in which a program (for example, a program for causing an information processing apparatus to execute various processes) is recorded.

(First Embodiment)
FIG. 1 is a schematic diagram showing a first configuration example of the flight body group control system 10 according to the first embodiment. The flight group control system 10 includes an unmanned aerial vehicle 100 and a terminal 80. The unmanned aerial vehicle 100 and the terminal 80 can communicate with each other by wired communication or wireless communication (for example, wireless LAN (Local Area Network)). FIG. 1 illustrates that the terminal 80 is a terminal (for example, a smartphone or a tablet terminal). The terminal 80 is an example of an information processing device.

The flight body group control system 10 may be configured to include an unmanned aerial vehicle 100, a transmitter, and a terminal 80. If a transmitter is provided, the user can instruct control of the flight of the unmanned aerial vehicle using the left and right control rods located in front of the transmitter. Further, in this case, the unmanned aerial vehicle 100, the transmitter, and the terminal 80 can communicate with each other by wired communication or wireless communication.

FIG. 2 is a schematic diagram showing a second configuration example of the flight body group control system 10 according to the first embodiment. FIG. 2 illustrates that the terminal 80 is a PC. In any of FIGS. 1 and 2, the function of the terminal 80 may be the same.

FIG. 3 is a diagram showing an example of a specific appearance of the unmanned aerial vehicle 100. FIG. 3 shows a perspective view of the unmanned aerial vehicle 100 flying in the moving direction STV0. The unmanned aerial vehicle 100 is an example of an air vehicle.

As shown in FIG. 3, it is assumed that the roll axis (see x-axis) is defined in the direction parallel to the ground and along the moving direction STV0. In this case, the pitch axis (see y-axis) is defined in the direction parallel to the ground and perpendicular to the roll axis, and further, the yaw axis (z-axis) in the direction perpendicular to the ground and perpendicular to the roll axis and the pitch axis. See) is defined.

The unmanned aerial vehicle 100 includes a UAV main body 102, a gimbal 200, an image pickup unit 220, and a plurality of image pickup units 230.

The UAV main body 102 includes a plurality of rotary wings (propellers). The UAV main body 102 flies the unmanned aerial vehicle 100 by controlling the rotation of a plurality of rotor blades. The UAV body 102 flies the unmanned aerial vehicle 100 using, for example, four rotor blades. The number of rotor blades is not limited to four. Further, the unmanned aerial vehicle 100 may be a fixed-wing aircraft having no rotor blades.

The image pickup unit 220 is a camera for taking an image of a subject included in a desired imaging range (for example, a state of the sky to be aerial photographed, a landscape such as a mountain or a river, a building on the ground).

The plurality of image pickup units 230 are sensing cameras that image the surroundings of the unmanned aerial vehicle 100 in order to control the flight of the unmanned aerial vehicle 100. Two image pickup units 230 may be provided in front of the nose of the unmanned aerial vehicle 100. Further, two other imaging units 230 may be provided on the bottom surface of the unmanned aerial vehicle 100. The two image pickup units 230 on the front side are paired and may function as a so-called stereo camera. The two image pickup units 230 on the bottom surface side may also be paired and function as a stereo camera. Three-dimensional spatial data around the unmanned aerial vehicle 100 may be generated based on the images captured by the plurality of imaging units 230. The number of image pickup units 230 included in the unmanned aerial vehicle 100 is not limited to four. The unmanned aerial vehicle 100 may include at least one image pickup unit 230. The unmanned aerial vehicle 100 may include at least one image pickup unit 230 on each of the nose, tail, side surface, bottom surface, and ceiling surface of the unmanned aerial vehicle 100. The angle of view that can be set by the image pickup unit 230 may be wider than the angle of view that can be set by the image pickup unit 220. The image pickup unit 230 may have a single focus lens or a fisheye lens.

FIG. 4 is a block diagram showing an example of the hardware configuration of the unmanned aerial vehicle 100. The unmanned aircraft 100 includes a UAV control unit 110, a communication interface 150, a memory 160, a storage 170, a gimbal 200, a rotary wing mechanism 210, an image pickup unit 220, an image pickup unit 230, a GPS receiver 240, and the like. The configuration includes an inertial measurement unit (IMU) 250, a magnetic compass 260, a barometric altimeter 270, an ultrasonic sensor 280, and a laser measuring instrument 290.

The UAV control unit 110 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). The UAV control unit 110 performs signal processing for controlling the operation of each unit of the unmanned aerial vehicle 100, data input / output processing with and from other units, data calculation processing, and data storage processing.

The UAV control unit 110 controls the flight of the unmanned aerial vehicle 100 according to the program stored in the memory 160. The UAV control unit 110 may control the flight according to the instruction of the flight control by the transmitter or the terminal 80. The UAV control unit 110 may have the image pickup unit 220 or the image pickup unit 230 take an aerial image.

The UAV control unit 110 acquires position information indicating the position of the unmanned aerial vehicle 100. The UAV control unit 110 may acquire position information indicating the latitude, longitude, and altitude in which the unmanned aerial vehicle 100 exists from the GPS receiver 240. The UAV control unit 110 acquires latitude / longitude information indicating the latitude and longitude of the unmanned aerial vehicle 100 from the GPS receiver 240 and altitude information indicating the altitude at which the unmanned aerial vehicle 100 exists from the barometric altitude meter 270 as position information. good. The UAV control unit 110 may acquire the distance between the emission point of the ultrasonic wave and the reflection point of the ultrasonic wave by the ultrasonic sensor 280 as altitude information.

The UAV control unit 110 may acquire orientation information indicating the orientation of the unmanned aerial vehicle 100 from the magnetic compass 260. The orientation information may be shown, for example, in the orientation corresponding to the orientation of the nose of the unmanned aerial vehicle 100.

The UAV control unit 110 may acquire position information indicating a position where the unmanned aerial vehicle 100 should exist when the image pickup unit 220 captures an image pickup range to be imaged. The UAV control unit 110 may acquire position information indicating the position where the unmanned aerial vehicle 100 should exist from the memory 160. The UAV control unit 110 may acquire position information indicating the position where the unmanned aerial vehicle 100 should exist from another device via the communication interface 150. The UAV control unit 110 may refer to the three-dimensional map database to specify a position where the unmanned aerial vehicle 100 can exist, and acquire the position as position information indicating the position where the unmanned aerial vehicle 100 should exist.

The UAV control unit 110 may acquire image pickup range information indicating the respective image pickup ranges of the image pickup unit 220 and the image pickup unit 230. The UAV control unit 110 may acquire image angle information indicating the angle of view of the image pickup unit 220 and the image pickup unit 230 from the image pickup unit 220 and the image pickup unit 230 as a parameter for specifying the image pickup range. The UAV control unit 110 may acquire information indicating the image pickup direction of the image pickup unit 220 and the image pickup unit 230 as a parameter for specifying the image pickup range. The UAV control unit 110 may acquire posture information indicating the posture state of the imaging unit 220 from the gimbal 200, for example, as information indicating the imaging direction of the imaging unit 220. The posture information of the image pickup unit 220 may indicate the rotation angle of the gimbal 200 from the reference rotation angle of the pitch axis and the yaw axis.

The UAV control unit 110 may acquire position information indicating the position where the unmanned aerial vehicle 100 exists as a parameter for specifying the imaging range. The UAV control unit 110 defines an imaging range indicating a geographical range imaged by the imaging unit 220 based on the angle of view and the imaging direction of the imaging unit 220 and the imaging unit 230, and the position where the unmanned aerial vehicle 100 exists. The imaging range information may be acquired by generating the imaging range information.

The UAV control unit 110 may acquire image pickup range information from the memory 160. The UAV control unit 110 may acquire image pickup range information via the communication interface 150.

The UAV control unit 110 controls the gimbal 200, the rotor blade mechanism 210, the image pickup unit 220, and the image pickup section 230. The UAV control unit 110 may control the image pickup range of the image pickup unit 220 by changing the image pickup direction or the angle of view of the image pickup unit 220. The UAV control unit 110 may control the imaging range of the imaging unit 220 supported by the gimbal 200 by controlling the rotation mechanism of the gimbal 200.

The image pickup range refers to a geographical range imaged by the image pickup unit 220 or the image pickup unit 230. The imaging range is defined by latitude, longitude, and altitude. The imaging range may be a range in 3D spatial data defined by latitude, longitude, and altitude. The imaging range may be a range in two-dimensional spatial data defined by latitude and longitude. The imaging range may be specified based on the angle of view and imaging direction of the imaging unit 220 or the imaging unit 230, and the position where the unmanned aerial vehicle 100 is present. The imaging direction of the image pickup unit 220 and the image pickup unit 230 may be defined from the direction and depression angle at which the front surface of the image pickup unit 220 and the image pickup unit 230 provided with the image pickup lens faces. The imaging direction of the imaging unit 220 may be a direction specified from the orientation of the nose of the unmanned aerial vehicle 100 and the posture of the imaging unit 220 with respect to the gimbal 200. The imaging direction of the imaging unit 230 may be a direction specified from the orientation of the nose of the unmanned aerial vehicle 100 and the position where the imaging unit 230 is provided.

The UAV control unit 110 may identify the environment around the unmanned aerial vehicle 100 by analyzing a plurality of images captured by the plurality of image pickup units 230. The UAV control unit 110 may control the flight, for example, avoiding obstacles, based on the environment around the unmanned aerial vehicle 100.

The UAV control unit 110 may acquire three-dimensional information (three-dimensional information) indicating the three-dimensional shape (three-dimensional shape) of an object existing around the unmanned aerial vehicle 100. The object may be, for example, a part of a landscape such as a building, a road, a car, a tree, or the like. The three-dimensional information is, for example, three-dimensional spatial data. The UAV control unit 110 may acquire the three-dimensional information by generating the three-dimensional information indicating the three-dimensional shape of the object existing around the unmanned aerial vehicle 100 from each image obtained from the plurality of image pickup units 230. The UAV control unit 110 may acquire three-dimensional information indicating the three-dimensional shape of an object existing around the unmanned aerial vehicle 100 by referring to the three-dimensional map database stored in the memory 160 or the storage 170. The UAV control unit 110 may acquire three-dimensional information regarding the three-dimensional shape of an object existing around the unmanned aerial vehicle 100 by referring to a three-dimensional map database managed by a server existing on the network.

The UAV control unit 110 controls the flight of the unmanned aerial vehicle 100 by controlling the rotary wing mechanism 210. That is, the UAV control unit 110 controls the position including the latitude, longitude, and altitude of the unmanned aerial vehicle 100 by controlling the rotary wing mechanism 210. The UAV control unit 110 may control the image pickup range of the image pickup unit 220 by controlling the flight of the unmanned aerial vehicle 100. The UAV control unit 110 may control the angle of view of the image pickup unit 220 by controlling the zoom lens included in the image pickup unit 220. The UAV control unit 110 may control the angle of view of the image pickup unit 220 by digital zoom using the digital zoom function of the image pickup unit 220.

When the image pickup unit 220 is fixed to the unmanned aerial vehicle 100 and the image pickup unit 220 cannot be moved, the UAV control unit 110 moves the unmanned aerial vehicle 100 to a specific position at a specific date and time to obtain a desired image in a desired environment. The range may be imaged by the image pickup unit 220. Alternatively, even if the imaging unit 220 does not have a zoom function and the angle of view of the imaging unit 220 cannot be changed, the UAV control unit 110 can move the unmanned aerial vehicle 100 to a specific position at a specified date and time, thereby making it desirable. The image pickup unit 220 may image a desired imaging range under the above environment.

The communication interface 150 communicates with the terminal 80. The communication interface 150 may perform wireless communication by any wireless communication method. The communication interface 150 may perform wired communication by any wired communication method. The communication interface 150 may transmit an aerial image or additional information (metadata) related to the aerial image to the terminal 80.

The UAV control unit 110 has a gimbal 200, a rotary wing mechanism 210, an image pickup unit 220, an image pickup unit 230, a GPS receiver 240, an inertial measurement unit 250, a magnetic compass 260, a barometric altimeter 270, an ultrasonic sensor 280, and a laser. Stores programs and the like required to control the measuring instrument 290. The memory 160 may be a computer-readable recording medium, and may be a SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), an EPROM (Erasable Programmable Read Only Memory), an EPROM (Electrically Erasable Programmable Read-Only Memory), and an EPROM (Electrically Erasable Programmable Read-Only Memory). It may include at least one flash memory such as a USB (Universal Serial Bus) memory. The memory 160 may be removable from the unmanned aerial vehicle 100. The memory 160 may operate as a working memory.

The storage 170 may include at least one of an HDD (Hard Disk Drive), an SSD (Solid State Drive), an SD card, a USB memory, and other storage. The storage 170 may hold various information and various data. The storage 170 may be removable from the unmanned aerial vehicle 100. The storage 170 may record aerial images.

The gimbal 200 may rotatably support the imaging unit 220 about the yaw axis, the pitch axis, and the roll axis. The gimbal 200 may change the imaging direction of the imaging unit 220 by rotating the imaging unit 220 around at least one of the yaw axis, the pitch axis, and the roll axis.

The rotary blade mechanism 210 includes a plurality of rotary blades and a plurality of drive motors for rotating the plurality of rotary blades. The rotary wing mechanism 210 flies the unmanned aerial vehicle 100 by controlling the rotation by the UAV control unit 110. The number of rotary blades 211 may be, for example, four or any other number. Further, the unmanned aerial vehicle 100 may be a fixed-wing aircraft having no rotor blades.

The image pickup unit 220 captures a subject in a desired imaging range and generates data of the captured image. The image data (for example, an aerial image) obtained by the image pickup of the image pickup unit 220 may be stored in the memory of the image pickup unit 220 or the storage 170.

The image pickup unit 230 captures the surroundings of the unmanned aerial vehicle 100 and generates data of the captured image. The image data of the image pickup unit 230 may be stored in the storage 170.

The GPS receiver 240 receives a plurality of signals indicating the time and the position (coordinates) of each GPS satellite transmitted from the plurality of navigation satellites (that is, GPS satellites). The GPS receiver 240 calculates the position of the GPS receiver 240 (that is, the position of the unmanned aerial vehicle 100) based on the plurality of received signals. The GPS receiver 240 outputs the position information of the unmanned aerial vehicle 100 to the UAV control unit 110. The position information of the GPS receiver 240 may be calculated by the UAV control unit 110 instead of the GPS receiver 240. In this case, information indicating the time included in the plurality of signals received by the GPS receiver 240 and the position of each GPS satellite is input to the UAV control unit 110.

The inertial measurement unit 250 detects the attitude of the unmanned aerial vehicle 100 and outputs the detection result to the UAV control unit 110. The inertial measurement unit 250 detects, as the posture of the unmanned aerial vehicle 100, the acceleration in the three axial directions of the front-back, left-right, and up-down of the unmanned aerial vehicle 100, and the angular velocity in the three-axis directions of the pitch axis, the roll axis, and the yaw axis. It's okay.

The magnetic compass 260 detects the direction of the nose of the unmanned aerial vehicle 100 and outputs the detection result to the UAV control unit 110.

The barometric altimeter 270 detects the altitude at which the unmanned aerial vehicle 100 flies, and outputs the detection result to the UAV control unit 110.

The ultrasonic sensor 280 emits ultrasonic waves, detects ultrasonic waves reflected by the ground or an object, and outputs the detection result to the UAV control unit 110. The detection result may indicate the distance or altitude from the unmanned aerial vehicle 100 to the ground. The detection result may indicate the distance from the unmanned aerial vehicle 100 to the object (subject).

The laser measuring instrument 290 irradiates an object with laser light, receives the reflected light reflected by the object, and measures the distance between the unmanned aircraft 100 and the object (subject) by the reflected light. As an example, the time-of-flight method may be used as the method for measuring the distance by the laser beam.

FIG. 5 is a block diagram showing an example of the hardware configuration of the terminal 80. The terminal 80 includes a terminal control unit 81, an operation unit 83, a communication unit 85, a memory 87, a display unit 88, and a storage 89. The terminal 80 may be possessed by a user who desires flight control instructions for a plurality of unmanned aerial vehicles 100.

The terminal control unit 81 is configured by using, for example, a CPU, an MPU, or a DSP. The terminal control unit 81 performs signal processing for controlling the operation of each unit of the terminal 80 in a unified manner, data input / output processing with each other unit, data calculation processing, and data storage processing.

The terminal control unit 81 conveys data (for example, various measurement data, aerial image data) and information (for example, position information of the unmanned aerial vehicle 100, and unmanned aerial vehicles collide with each other) from the unmanned aerial vehicle 100 via the communication unit 85. Information) may be obtained. The terminal control unit 81 may acquire data and information (for example, various parameters) input via the operation unit 83. The terminal control unit 81 may acquire data or information held in the memory 87. Even if the terminal control unit 81 transmits data or information (for example, the position of the unmanned aerial vehicle after movement, information on the amount of movement, information on the flight route for movement) to the unmanned aerial vehicle 100 via the communication unit 85. good. The terminal control unit 81 may send data or information to the display unit 88 and display the display information based on the data or information on the display unit 88.

The terminal control unit 81 may execute an application for instructing flight control of a plurality of unmanned aerial vehicles 100 (also referred to as unmanned aerial vehicle group 100G). The terminal control unit 81 may generate various data used in the application.

The operation unit 83 receives and acquires data and information input by the user of the terminal 80. The operation unit 83 may include an input device such as a button, a key, a touch screen, a microphone, and the like. Here, it is illustrated that the operation unit 83 and the display unit 88 are mainly composed of a touch panel. In this case, the operation unit 83 can receive a touch operation, a tap operation, a drag operation, and the like. The operation unit 83 may receive information on various parameters. The information input by the operation unit 83 may be transmitted to the unmanned aerial vehicle 100. The various parameters may include parameters related to flight control (for example, information regarding the position of the control node, the movement of the control node, the weighting of the control node, etc., which will be described later).

The communication unit 85 wirelessly communicates with the unmanned aerial vehicle 100 by various wireless communication methods. The wireless communication method of this wireless communication may include communication via, for example, a wireless LAN, Bluetooth®, or a public wireless line. The communication unit 85 may perform wired communication by any wired communication method.

The memory 87 has, for example, a ROM in which data of a program or set value that defines the operation of the terminal 80 is stored, and a RAM in which various information and data used during processing of the terminal control unit 81 are temporarily stored. You may. The memory 87 may include a memory other than the ROM and the RAM. The memory 87 may be provided inside the terminal 80. The memory 87 may be provided so as to be removable from the terminal 80. The program may include an application program.
The display unit 88 is configured by using, for example, an LCD (Liquid Crystal Display), and displays various information and data output from the terminal control unit 81. The display unit 88 may display various data and information related to the execution of the application.

The storage 89 stores and holds various data and information. The storage 89 may be an HDD, SSD, SD card, USB memory, or the like. The storage 89 may be provided inside the terminal 80. The storage 89 may be provided so as to be removable from the terminal 80. The storage 89 may hold an aerial image or additional information acquired from the unmanned aerial vehicle 100. The additional information may be held in the memory 87.

Next, the function related to the flight control instruction of the unmanned aerial vehicle group 100G including the plurality of unmanned aerial vehicles 100 will be described. Here, it is mainly described that the terminal control unit 81 of the terminal 80 has a function related to the flight control instruction of the unmanned aerial vehicle group 100G, but the unmanned aerial vehicle 100 has a function related to the flight control instruction of the unmanned aerial vehicle group 100G. It's okay. The terminal control unit 81 is an example of a processing unit. The terminal control unit 81 performs processing related to the flight control instruction of the unmanned aerial vehicle group 100G.

The unmanned aerial vehicle group 100G to be the target of flight control may be a plurality of unmanned aerial vehicles 100 that fly in cooperation with each other, or may be a plurality of unmanned aerial vehicles 100 that fly in a group in a certain space without cooperation. Well, not particularly limited.

The terminal control unit 81 acquires information on one or more control nodes C for controlling the flight of the unmanned aerial vehicle group 100G. The control node C is a virtual node for controlling the flight of the unmanned aerial vehicle group 100G. The information of the control node C may include identification information for identifying the control node C, position information of the control node C, information on the positional relationship of a plurality of control nodes C, and the like. The information of the control node C may be held in the memory 160 or the storage 170, or may be acquired from an external device via the communication interface 150. Further, the terminal control unit 81 may generate information on the control node C by receiving a user operation via the operation unit 83. For example, the terminal control unit 81 may input the position of the control node C and the number of control nodes C via the operation unit 83.

The position information of the control node C may be the position information in the three-dimensional space. The information on the positional relationship of the plurality of control nodes C may be a shape (flight shape) formed by the plurality of unmanned aerial vehicles 100 in the unmanned aerial vehicle group 100G during flight. The flight shape may be a predetermined shape pattern (for example, a polygonal prism shape, a polygonal pyramid shape, a spherical shape, etc.), or a shape pattern generated by a user operation via the operation unit 83.

The terminal control unit 81 acquires the position information of the plurality of unmanned aerial vehicles 100 included in the unmanned aerial vehicle group 100G, for example, via the communication unit 85. The position information of the unmanned aerial vehicle 100 may be, for example, the information acquired by the GPS receiver 240 of the unmanned aerial vehicle 100, or the information acquired by the ultrasonic sensor 280.

FIG. 6 is a diagram illustrating the positional relationship between the control node C and the unmanned aerial vehicle group 100G. In the terminal 80, the terminal control unit 81 receives the position information (for example, GPS position information) of the unmanned aerial vehicle 100 from each unmanned aerial vehicle 100 via the communication unit 85, and the display unit 88 is based on the position information of the unmanned aerial vehicle 100. Display the unmanned aerial vehicle group 100G. The terminal control unit 81 may arrange the control node C at a position designated by the user with respect to the unmanned aerial vehicle group 100G displayed on the display unit 88 via the operation unit 83. The terminal control unit 81 may arrange the control node C at an arbitrary position without depending on the position of the unmanned aerial vehicle 100 at the time of determining the arrangement of the control node C. When the terminal control unit 81 determines the arrangement of the control node C, the terminal control unit 81 may arrange the control node C based on the position of the unmanned aerial vehicle group 100G. The terminal control unit 81 may set the positional relationship of the plurality of control nodes C by, for example, arranging one or more control nodes at desired positions via the operation unit 83.

In FIG. 6, the four control nodes C1, C2, C3, and C4 are arranged at positions p1, p2, p3, and p4, respectively, according to a radial shape pattern. In other words, the control nodes C2, C3, and C4 are arranged around the control node C1 with the control node C1 as the center or the center of gravity. The flight form of the unmanned aerial vehicle group 100G is formed by approximating the shape of this shape pattern. When the four control nodes C1, C2, C3, and C4 are arranged at the positions designated via the operation unit 83, the terminal control unit 81 generates three bones (skeletons) b12, b13, and b14 connecting them. do. The terminal control unit 81 may change the flight mode of the unmanned aerial vehicle group 100G by receiving a change in the positions of the control nodes C1, C2, C3, and C4 via a user operation to the operation unit 83. The terminal control unit 81 may move the control node C directly via the operation unit 83, or may move the control node C by moving the bones b12, b13, and b14.

The positions p1, p2, p3, and p4 of the control nodes C1, C2, C3, and C4 may be arranged on the same plane (two-dimensional plane) or on different planes (three-dimensional space). .. The shape pattern of the control node C may be predetermined. In this case, the shape pattern information may be held in the memory 87 or the storage 89, and the terminal control unit 81 may acquire the shape pattern information. The shape pattern of the control node C may be newly generated independently. In this case, the terminal control unit 81 may receive an instruction to arrange the control nodes C1 to C4 via the operation unit 83 and generate a shape pattern formed by the control nodes C1 to C4. The generated shape pattern may be held in the memory 87 or the storage 89.

In this way, the terminal 80 can determine the shape pattern of the plurality of control nodes C into the shape desired by the user. Therefore, the terminal 80 can adjust the flight form of the unmanned aerial vehicle group 100G determined by the positional relationship of the control node C to the form desired by the user.

The terminal control unit 81 may associate one or more control nodes C with a plurality of unmanned aerial vehicles 100. The terminal control unit 81 may receive a user operation via the operation unit 83 and select at least one of the unmanned aerial vehicle 100 to be associated and the one or more control nodes C. The terminal control unit 81 may make an association between, for example, a predetermined unmanned aerial vehicle 100 and a predetermined control node C without going through the operation unit 83. In the association, weighting is performed to indicate the strength of the association between the unmanned aerial vehicle 100 and the control node C. The degree of association may be indicated by the magnitude of the weight w. The larger the weight w, the higher the relationship between the unmanned aerial vehicle 100 and the control node C. For example, the unmanned aerial vehicle 100 is greatly affected when the control node C moves, and the unmanned aerial vehicle 100 moves greatly according to the movement of the control node C. The smaller the weight w, the lower the relationship between the unmanned aerial vehicle 100 and the control node C. For example, the unmanned aerial vehicle 100 is less affected when the control node C moves, and the unmanned aerial vehicle 100 moves smaller according to the movement of the control node C. Do not move.

The terminal control unit 81 sets the weights w of a plurality of control nodes C (for example, two in the order of the shortest distance from the unmanned aerial vehicle 100 (also referred to as two in the vicinity)) for each unmanned aerial vehicle 100. The weight w is an example of a value that associates the control node C with the unmanned aerial vehicle 100, and may be a coefficient set in the range of the value 0 to the value 1. The closer the weight w is to the value 1, the stronger the association with the corresponding control node C. The closer the weight w is to the value 0, the weaker the association with the corresponding control node C. That is, the closer the weight w is to the value 1, the stronger the influence of the control node C, and the closer the weight w is to the value 1, the less the influence of the control node C. Further, the two control nodes C located in the vicinity of the unmanned aerial vehicle 100 may be a control node C having the shortest distance to the unmanned aerial vehicle 100 and a control node C having the second shortest distance. The number of control nodes C to be weighted may be other than two, at least one, and may be three or more.

The first to third examples of the weighting method are shown below.

In the first example, the terminal control unit 81 inputs a value of an arbitrary weight w by the user via the operation unit 83. For example, the terminal control unit 81 selects two control nodes C1 and C2 (see FIG. 6) in the vicinity of the unmanned aerial vehicle 100 via the operation unit 83. The terminal control unit 81 sets the weight w of the control node C, which is closer to the unmanned aerial vehicle 100, to be larger among the two selected control nodes C1 and C2 via the operation unit 83. In FIG. 6, since the control node C2 is the closest to the unmanned aerial vehicle 100 located at the position p, the weight w2 of the control node C2 is set to have a large value. The control node C2 is an example of a control node having the shortest distance from any unmanned aerial vehicle 100. The control node C1 is an example of a control node having the second shortest distance to any unmanned aerial vehicle 100.

In this way, the user intuitively sets the weight w for the terminal 80 via the operation unit 83 so that the control node C and the unmanned aerial vehicle 100 desired by the user can be intuitively moved with respect to the movement of the unmanned aerial vehicle 100. Can be associated. Further, since the association can be easily performed by the user operation, the terminal 80 can reduce the processing load of the terminal 80 when making the association.

In the second example, the terminal control unit 81 calculates the weight w based on the distance between the unmanned aerial vehicle 100 and the control node C. For example, when determining the weights w of the two control nodes C1 and C2 in the vicinity of the unmanned aerial vehicle 100, the terminal control unit 81 determines the weights w1 of the control node C1 and the control node C2 for the unmanned aerial vehicle 100 according to the equation (1). Weight w2 may be calculated. Note that d1 is the distance between the unmanned aerial vehicle 100 and the control node C1. d2 is the distance between the unmanned aerial vehicle 100 and the control node C2.
w1 = d2 / (d1 + d2)
w2 = d1 / (d1 + d2) …… (1)

In the second example, a threshold value for the weight w may be provided. In this case, the terminal control unit 81 may set the weight w to the value 1 when the weight w is the threshold value th1 or more, and set the weight w to the value 0 when the weight w is less than the threshold value th2. good. The threshold values th1 and th2 may be the same value or different values.

In the third example, the terminal control unit 81 calculates the weight w of the two control nodes C with respect to the unmanned aerial vehicle 100 based on the positional relationship between the unmanned aerial vehicle 100 and the control node C. The component dli along the direction connecting the two control nodes C at the distance between the unmanned aerial vehicle 100 and the control node C of any one of the two is determined by the equation (2).

Here, pi indicates the position of the control node Ci. pj indicates the position of the control node Cj. p indicates the position of the unmanned aerial vehicle 100. The arrow attached to the upper part in the equation (2) represents a vector. The vector pip represents a vector from pi to p. The vector pipj represents a vector from pi to pj. Vector ip * Vector pipj represents the cosine of vector pip, that is, the inner product of vector pip and vector pipj in the direction of vector pipj.

In FIG. 6, referring to the equation (2), the component dl1 along the direction connecting the control nodes C1 and C2 at the distance between the unmanned aerial vehicle 100 and the control node C1 has an internal product of the vector p1p and the vector p1p2 having a value of 0 or more. Therefore, it is the value of this inner product. Further, the component dl2 along the direction connecting the control nodes C1 and C2 at the distance between the unmanned aerial vehicle 100 and the control node C2 has a value of 0 because the inner product of the vector p2p and the vector p2p1 is negative.

Then, the terminal control unit 81 has dl1 as d1 of the equation (1) and dl1 as d2 of the equation (1), and the weights w1 and w2 of the two control nodes C1 and C2 for the unmanned aerial vehicle 100 according to the equation (1). Is calculated.

That is, in the third example, when the unmanned aerial vehicle 100 located at the position p in FIG. 6 is located between the two control nodes C1 and C2, the control of each distance from the two control nodes C1 and C2 to the unmanned aerial vehicle 100 is performed. The components dl1 and dl2 in the direction connecting the nodes C1 and C2 are the values of the inner product according to the equation (2), respectively. On the other hand, when the unmanned aerial vehicle 100 is not located between the two control nodes C1 and C2, the component d2 related to the distance from the nearest control node C2 to the unmanned aerial vehicle 100 among the two control nodes C1 and C2 has a value of 0. The component d1 related to the distance from the second closest control node C1 to the unmanned aerial vehicle 100 is the value of the above-mentioned inner product. In this case, since the weight w1 has a value of 0 and the weight w2 has a value of 1, the movement of the unmanned aerial vehicle 100 located at the position p is controlled by the movement of the nearest control node C2.

Here, the weights w1 and w2 are calculated for the two control nodes C1 and C2 located in the vicinity of the unmanned aerial vehicle 100, but when the unmanned aerial vehicle 100 is associated with three or more control nodes C, Three or more weights w (w1, w2, w3, ...) Are calculated.

In the equation (2), when the value of the inner product of the vector ip * vector pipj is a negative value, it is calculated as a value 0, but it may remain a negative value. In this case, the unmanned aerial vehicle 100 can move, taking into consideration the influence of the control node C (control node C1 in FIG. 6) which is farther from the two control nodes C.

As described above, in the second example and the third example, the terminal control unit 81 acquires the position information of the plurality of control nodes C, the position information of the control node C and the position information of the plurality of unmanned aerial vehicles 100 (for example, GPS position). The value (for example, the weight w) of the association between the control node C and the plurality of unmanned aerial vehicles 100 may be calculated based on the information). As a result, the terminal 80 can be automatically associated with the control node C based on the positional relationship between the unmanned aerial vehicle 100 and the control node C. In addition, the terminal 80 can eliminate the need for user operations when associating, and can reduce the complexity.

Further, the terminal 80 can be associated only with the control node C close to the unmanned aerial vehicle 100 (for example, two control nodes C1 and C2 in the vicinity). Therefore, the unmanned aerial vehicle 100 is affected by the movement of the control node C when the control node C close to the own aircraft moves, but the control node C moves when the control node C far from the unmanned aerial vehicle 100 moves. Not affected by. Therefore, the terminal 80 can reduce the movement of the unmanned aerial vehicle 100 due to the movement of the control node C, and can suppress the movement of many unmanned aerial vehicles 100. Further, the unmanned aerial vehicle 100 around the moved control node C will mainly move, and the interlocking between the movement of the control node C and the movement of the unmanned aerial vehicle 100 will be intuitively easy for the user to understand. By making the number of control nodes C associated with one unmanned aerial vehicle 100 variable, the terminal 80 can adjust how the unmanned aerial vehicle 100 moves according to the movement of the control node C, and the movement of the unmanned aerial vehicle 100 can be adjusted. You can increase the variation of.

Further, when the terminal 80 is associated with many control nodes C, the movement of many control nodes C affects the movement of the unmanned aerial vehicle 100. Therefore, by associating the terminal 80 with many control nodes C, the movement of the unmanned aerial vehicle 100 can be complicated, and the unmanned aerial vehicle 100 can be made to perform fine flight control.

The terminal control unit 81 moves one or more unmanned aerial vehicles 100 associated with the moved control node C based on the movement of the one or more control nodes C. The terminal control unit 81 may receive a user operation via the operation unit 83 and move the control node C based on the user operation. The terminal control unit 81 may generate movement control information for moving one or more unmanned aerial vehicles 100 associated with the moved control node C based on the movement of the control node C. The terminal control unit 81 may transmit movement control information to each unmanned aerial vehicle 100 via the communication unit 85.

For example, the terminal control unit 81 calculates the movement amount of each unmanned aerial vehicle 100 based on the movement amount of the two control nodes C located in the vicinity of each unmanned aerial vehicle 100. The movement amount of the control node C may be expressed as a vector value in which the movement direction and the movement distance of the control node C are taken into consideration. The movement amount of the unmanned aerial vehicle 100 may be expressed as a vector value in which the movement direction and the movement distance of the unmanned aerial vehicle 100 are taken into consideration.

FIG. 7 is a diagram illustrating a movement operation of the unmanned aerial vehicle group 100G. When the terminal control unit 81 receives a user operation via the operation unit 83 and moves the control node C, the unmanned aerial vehicle 100 associated with the moved control node C in the unmanned aerial vehicle group 100G moves.

The movement amount of the control node Ci is the vector mi, the movement amount of the unmanned aerial vehicle 100 is the vector dnj, and the weight of the unmanned aerial vehicle 100 with respect to the control node Ci is the weight wi. The movement amount dnj of the unmanned aerial vehicle 100 accompanying the movement of the control node Ci is expressed by the equation (3).

That is, the terminal control unit 81 may calculate the movement amount dnj of the unmanned aerial vehicle 100 based on the weight wi for the control node Ci of the unmanned aerial vehicle 100 and the movement amount mi of the control node Ci according to the equation (3). .. The terminal control unit 81 may move the unmanned aerial vehicle 100 based on the movement amount dnj of the unmanned aerial vehicle 100. In this case, the terminal control unit 81 may transmit the movement control information including the movement amount dnj of the unmanned aerial vehicle 100 to the terminal 80 via the communication unit 85.

FIG. 7 shows a case where the user performs an operation of moving the control node C1 and the control node C4 to the four control nodes C1 to C4 displayed on the display unit 88 as the touch panel by using the finger fg. ing. For the plurality of unmanned aerial vehicles 100 that move with the movement of the control nodes C1 and C4, the positions of the unmanned aerial vehicles 100 before the movement are represented by u1 to u7, and the positions after the movement are represented by u11 to u17, respectively. ..

In FIG. 7, the terminal control unit 81 receives a user operation via the operation unit 83 as a touch panel, moves the control node C1 slightly to the lower left as shown by the vector m1, and the control node C4 is shown by the vector m4. It is moved downward as much as possible. Along with this, the bones b12 and b14 expand and contract and move. The terminal control unit 81 may move the control nodes C1 and C4 by moving the bones b14 and b12 instead of moving the control nodes C1 and C4 via the operation unit 83.

Due to the movement of the control nodes C1 and C4, the unmanned aerial vehicle 100 located at the positions u1 to u7 before the movement moves to the positions u11 to u17 after the movement. For example, due to the movement of the control node C1, the unmanned aerial vehicle 100 that was in the position u1 before the movement moves to the position u11 after the movement according to the equation (3). Due to the movement of the control node C4, the unmanned aerial vehicle 100 located at the position u3 before the movement largely moves to the position u13 after the movement according to the movement amount dn3 derived by the equation (3).

In this way, the user can move the control node C desired and intuitively by performing the movement operation of the control node C via the operation unit 83. Therefore, the user can indirectly move the unmanned aerial vehicle 100 associated with the control node C by the movement operation of the control node C.

The terminal control unit 81 may move the control node C without receiving the movement operation of the control node C via the operation unit 83. For example, the terminal control unit 81 acquires movement information (predetermined movement information) for moving one or more predetermined control nodes C from the memory 160 or the like, and corresponds to the movement information based on the movement information. The control node C may be moved. The terminal control unit 81 may move the corresponding control node C based on the predetermined movement information, for example, when the predetermined time is reached based on the time information. The terminal control unit 81 may move the corresponding control node C based on the predetermined movement information, for example, when the unmanned aerial vehicle 100 reaches a predetermined place based on the position information.

The terminal control unit 81 may limit the movement of the unmanned aerial vehicles 100 when the unmanned aerial vehicles 100 are moved or when the unmanned aerial vehicles 100 collide with each other according to the calculated movement amount dnj of the unmanned aerial vehicles 100. The determination of whether or not the unmanned aerial vehicles 100 collide with each other may be performed, for example, by calculating the position of each unmanned aerial vehicle 100 after movement and checking whether or not the distance between the unmanned aerial vehicles 100 is equal to or greater than the threshold value th3. If it is less than the threshold value th3, the terminal control unit 81 may determine that a collision between the unmanned aerial vehicles 100 will occur. On the other hand, when it is equal to or more than the threshold value, the terminal control unit 81 may determine that the unmanned aerial vehicles 100 do not collide with each other. The threshold value th3 may be, for example, a safe distance (distance for ensuring safety) between the unmanned aerial vehicles 100.

Even if there is a period in which the movement of the control node C and the movement of the unmanned aerial vehicle 100 are not linked to avoid collision, there is a possibility of collision of a plurality of unmanned aerial vehicles 100 in calculation due to the further movement of the control node C. When becomes low, the unmanned aerial vehicle 100 moves with the movement of the control node C, so that the movement of the control node C and the flight of the unmanned aerial vehicle 100 can be restored to a linked state again.

When the unmanned aerial vehicles 100 collide with each other, the terminal control unit 81 may limit the movement of the unmanned aerial vehicle 100 only to the unmanned aerial vehicle 100 which is determined to collide and is associated with the moved control node C. When the unmanned aerial vehicles 100 collide with each other, the terminal control unit 81 may limit the movement of all the unmanned aerial vehicles 100 associated with the moved control node C regardless of whether or not it is determined that they collide with each other.

When the control node C1 moves, the terminal control unit 81 may limit the movement of the unmanned aerial vehicle 100 by not moving the unmanned aerial vehicle 100. When the control node C1 moves, the terminal control unit 81 may limit the movement of the unmanned aerial vehicle 100 by making the unmanned aerial vehicle 100 movable within a range in which the unmanned aerial vehicles 100 do not collide with each other.

The terminal control unit 81 does not determine whether or not to collide based on the position of each unmanned aerial vehicle 100 after movement, but collides based on the flight path for the unmanned aerial vehicle 100 to move before and after the movement. You may judge whether or not. For example, the terminal control unit 81 may determine that the unmanned aerial vehicles 100 collide with each other when a plurality of flight paths for the plurality of unmanned aerial vehicles 100 to move collide before and after the movement. Collision of a plurality of flight paths may include the intersection of a plurality of flight paths in a two-dimensional plane (latitude / longitude) or a three-dimensional space (latitude / longitude / altitude).

As described above, the terminal 80 calculates that when the unmanned aerial vehicle 100 moves with the movement of the control node C, the unmanned aerial vehicle 100 is at the same position after the movement and the possibility of contact is low. Can be instructed to move. In other words, the terminal 80 can limit the movement of the unmanned aerial vehicle 100 in real space when, in calculation, a plurality of unmanned aerial vehicles 100 are present at the same position or at a position where there is a possibility of contact after the movement. In this case, the terminal 80 can suppress the collision of a plurality of unmanned aerial vehicles 100 in the real space.

The terminal control unit 81 may limit the movement of the control node C based on the movement restriction information for restricting the movement of the control node C. The movement restriction information may be held in the memory 160 or the storage 170. The movement restriction information may be preset information or information determined based on user operations via the operation unit 83. The movement restriction information may be fixed information or variable information.

The movement restriction information may be defined for each control node C, or may be defined for a plurality of control nodes C in common. The movement restriction information may be, for example, information for restricting the movement of the control node C in a specific direction. This specific direction may be, for example, the Z-axis direction (gravitational direction in real space). The control restriction information may be, for example, information for restricting the movement of the control node C at a distance equal to or greater than the threshold value th4.

In this way, the terminal 80 limits the degree of freedom of movement of the control node C, so that the movement of the control node C is reduced, so that the movement of the unmanned aerial vehicle 100 can also be indirectly restricted. Therefore, it is possible to suppress the possibility that the positions of the plurality of unmanned aerial vehicles 100 move to the same position or a position where there is a possibility of contact due to the movement of the plurality of unmanned aerial vehicles 100. That is, the terminal 80 can suppress the collision of a plurality of unmanned aerial vehicles 100 by limiting the degree of freedom of movement of the control node.

The terminal control unit 81 may limit the movement of the unmanned aerial vehicle 100 and the movement of the control node C as described above, or both may be performed.

Next, an operation example of the flight body group control system 10 will be described.

FIG. 8 is a flowchart showing a first example of the operation procedure of the terminal 80 and each unmanned aerial vehicle 100. For example, during the flight of the unmanned aerial vehicle 100, the UAV control unit 110 repeatedly transmits the GPS position information (latitude, longitude, altitude) obtained by the GPS receiver 240 to the terminal 80 at a fixed timing via the communication interface 150. good.

At the terminal 80, the terminal control unit 81 receives the position information of each unmanned aerial vehicle 100 via the communication unit 85 (S1). The terminal control unit 81 displays the position of each unmanned aerial vehicle 100 on the display unit 88 based on the position information of each unmanned aerial vehicle 100.

The terminal control unit 81 acquires the information of one or more control nodes C according to the user's operation via the operation unit 83 (S2). The terminal control unit 81 may receive a user operation via the operation unit 83, and may acquire information on the shape patterns of the plurality of control nodes C prepared in advance from the memory 87 or the like. The terminal control unit 81 receives a user operation via the operation unit 83, specifies the positions and positional relationships of the plurality of control nodes C, generates a plurality of control nodes C, and acquires information on the control node C. You can do it. The terminal control unit 81 causes the display unit 88 to display one or more control nodes C.

The terminal control unit 81 associates each unmanned aerial vehicle 100 with the control node C (S3). The terminal control unit 81 specifies, via the operation unit 83, which of the plurality of unmanned aerial vehicles 100 the unmanned aerial vehicle 100 is associated with which control node C of the one or more control nodes C. good. The user operation for associating the control node C with the unmanned aerial vehicle 100 is an example of the first operation information. The terminal control unit 81 calculates the weight w in the association between the unmanned aerial vehicle 100 and the control node C.

The processing of S1 to S3 may be performed during the flight of the unmanned aerial vehicle 100, or may be performed before the flight of the unmanned aerial vehicle 100.

The terminal control unit 81 may receive a drag operation via the operation unit 83 to move the position of the control node C. The user operation (for example, drag operation) for moving the position of the control node C is an example of the second operation information. The terminal control unit 81 determines whether or not the movement operation of the control node C has been performed (S4). When the move operation is not performed, the terminal control unit 81 repeats the process of S4. On the other hand, when the movement operation is performed, the terminal control unit 81 calculates the movement amount dnj of the unmanned aerial vehicle 100 (S5). The calculation of the movement amount dnj of the unmanned aerial vehicle 100 is performed for each unmanned aerial vehicle 100 included in the unmanned aerial vehicle group 100G.

The terminal control unit 81 determines whether or not the unmanned aerial vehicles 100 collide with each other when the unmanned aerial vehicle group 100G is moved according to the calculated movement amount dnj of the unmanned aerial vehicle 100 (S6).

When it is determined that a collision between the unmanned aerial vehicles 100 will occur, the terminal control unit 81 causes the display unit 88 to display warning information that the unmanned aerial vehicles 100 may collide with each other (S7), and processes S4. Return to. In this case, in order to avoid a collision, the terminal control unit 81 limits the movement of the unmanned aerial vehicle 100. In addition to displaying the warning information, or instead of displaying the warning information, the terminal control unit 81 may indicate that there is a possibility of collision between the unmanned aerial vehicles 100 by voice, vibration, or the like.

On the other hand, when the unmanned aerial vehicles 100 do not collide with each other in S6, the terminal control unit 81 transmits the movement control information including the movement amount to each unmanned aerial vehicle 100 via the communication unit 85 (S8).

The terminal control unit 81 updates the display position of each unmanned aerial vehicle 100 displayed on the display unit 88 based on the position information of each unmanned aerial vehicle 100 after movement (S9). In this case, the terminal control unit 81 may update the display position of the unmanned aerial vehicle 100 after the movement based on the movement amount of each unmanned aerial vehicle 100 calculated in S5. After that, the terminal control unit 81 returns to the process of S4.

In the unmanned aerial vehicle 100, the UAV control unit 110 receives the movement control information of the own aircraft from the terminal 80 (S11). The flight is controlled so as to move to a position corresponding to the movement amount included in the movement control information (S12). After this, the UAV control unit 110 returns to the process of S11.

As described above, in the flight body group control system 10, the terminal 80 is provided with the control node C even when the flight route and the flight position of each unmanned aerial vehicle 100 are not set in advance, and the control node C and each unmanned aerial vehicle 100 are provided. By associating with, each unmanned aerial vehicle 100 can be moved according to the movement of the control node C. Therefore, the terminal 80 can instruct flight control to a plurality of unmanned aerial vehicles 100 in real time even during flight of the unmanned aerial vehicle 100, and can increase the degree of freedom in flight of the unmanned aerial vehicle 100. Further, the user does not need to individually set the flight route and the flight position for the unmanned aerial vehicle 100, and can easily instruct the flight control. Since the terminal 80 can move a plurality of unmanned aerial vehicles 100 by moving with respect to the control node C, it is not necessary to prepare a plurality of operating devices for operating the plurality of unmanned aerial vehicles 100, and a plurality of flying objects. Can be easily linked.

Further, the terminal 80 can collectively generate movement control information for moving the plurality of unmanned aerial vehicles 100, and can instruct the plurality of unmanned aerial vehicles 100 to move the plurality of unmanned aerial vehicles 100 accompanying the movement of the control node C. Therefore, the unmanned aerial vehicle 100 can acquire the movement control information without generating the movement control information corresponding to the movement of the control node C by the own aircraft. Therefore, the terminal 80 can reduce the processing load of each unmanned aerial vehicle 100 related to the control of the movement of the plurality of unmanned aerial vehicles 100 accompanying the movement of the control node C.

The flight control instruction of this embodiment may be carried out by the unmanned aerial vehicle 100. In this case, the UAV control unit 110 of the unmanned aerial vehicle 100 has the same function as the flight control instruction function of the terminal control unit 81 of the terminal 80. The UAV control unit 110 is an example of a processing unit. The UAV control unit 110 performs processing related to flight control instructions. In the process related to the flight control instruction by the UAV control unit 110, the same process as the process related to the flight control instruction performed by the terminal control unit 81 will be omitted or simplified.

As for the flight control instruction, one unmanned aerial vehicle 100 may instruct the flight control of all unmanned aerial vehicles, or each unmanned aerial vehicle 100 may instruct the flight control of its own aircraft. The unmanned aerial vehicle 100 that directs flight control is also referred to as a specific unmanned aerial vehicle 100. The specific unmanned aerial vehicle 100 is an example of an information processing device.

FIG. 9 is a flowchart showing a second example of the operating procedure of the terminal 80 and the unmanned aerial vehicle 100. Similar to the first example of the operation procedure, for example, during the flight of the unmanned aerial vehicle 100, the UAV control unit 110 constants the GPS position information (latitude, longitude, altitude) obtained by the GPS receiver 240 via the communication interface 150. It is repeatedly transmitted to the specific unmanned aerial vehicle 100 at the timing.

In the specific unmanned aerial vehicle 100, the UAV control unit 101 receives the position information of each unmanned aerial vehicle 100 via the communication interface 150 (S31). The UAV control unit 101 also acquires the position information of its own device via the GPS receiver 240 or the like (S31).

The UAV control unit 101 acquires information on one or more control nodes C according to the user's operation via the operation unit 83 (S32). In this case, the UAV control unit 101 may acquire information on the shape patterns of the plurality of control nodes C prepared in advance from the memory 160 or the like via the communication interface 150. The UAV control unit 101 may acquire the position information of the control node C based on the positional relationship of the plurality of control nodes C generated by the terminal 80 via the communication interface 150.

Similarly, in the terminal 80, the terminal control unit 81 acquires the position information of each unmanned aerial vehicle 100 and the information of one or more control nodes C1. The terminal control unit 81 causes the display unit 88 to display the positions of the control node C1 and the unmanned aerial vehicle 100 (S21).

The UAV control unit 101 associates each unmanned aerial vehicle 100 with the control node C (S33). In this case, in the terminal 80, the terminal control unit 81 determines which unmanned aerial vehicle 100 among the plurality of unmanned aerial vehicles 100 and which control node C among one or more control nodes C, via the communication interface 150. May be specified via the operation unit 83. The terminal control unit 81 transmits the operation information related to this association to the specific unmanned aerial vehicle 100 via the communication unit 85 (S22). The UAV control unit 101 may acquire operation information related to the association via the communication interface 150. The operation information related to the association is an example of the first operation information. The UAV control unit 101 calculates the weight w in the association between the unmanned aerial vehicle 100 and the control node C.

The processing of S31 to S33 may be performed during the flight of the unmanned aerial vehicle 100, or may be performed before the flight of the unmanned aerial vehicle 100.

The UAV control unit 101 acquires the movement information of the control node C (S34). In this case, in the terminal 80, the terminal control unit 81 may receive a drag operation via the operation unit 83 to move the position of the control node C. The terminal control unit 81 transmits the movement information of the control node C to the specific unmanned aerial vehicle 100 via the communication unit 85 (S23). The UAV control unit 101 may acquire the operation information related to the movement of the control node C as the movement information of the control node C via the communication interface 150. The operation information related to the movement of the control node C is an example of the second operation information.

The UAV control unit 101 calculates the movement amount dnj of the unmanned aerial vehicle 100 based on the operation information related to the movement of the control node C (S35). In this case, the UAV control unit 101 stores, for example, the correspondence information (for example, the information of the equation (3)) corresponding to one-to-one between the operation amount related to the movement of the control node C and the movement amount dnj of the unmanned aerial vehicle 100. You may keep it at 160 etc. and refer to it. The calculation of the movement amount dnj of the unmanned aerial vehicle 100 is performed for each unmanned aerial vehicle 100 included in the unmanned aerial vehicle group 100G.

The UAV control unit 101 determines whether or not the unmanned aerial vehicles 100 collide with each other when the unmanned aerial vehicle group 100G is moved according to the calculated movement amount dnj of the unmanned aerial vehicle 100 (S36).

When it is determined that a collision between the unmanned aerial vehicles 100 will occur, the UAV control unit 101 may transmit warning information to the effect that the unmanned aerial vehicles 100 may collide with each other via the communication interface 150 (S37). .. After S37, the UAV control unit 110 returns to the process of S31. In the terminal 80, the terminal control unit 81 may acquire the warning information via the communication unit 85 and display the warning information on the display unit 88 (S24). In addition to displaying the warning information, or instead of displaying the warning information, the terminal control unit 81 may indicate that there is a possibility of collision between the unmanned aerial vehicles 100 by voice, vibration, or the like.

On the other hand, when the unmanned aerial vehicles 100 do not collide with each other in S36, the UAV control unit 101 connects the unmanned aerial vehicle 100 to another unmanned aerial vehicle 100 (an unmanned aerial vehicle 100 other than the specific unmanned aerial vehicle 100) via the communication interface 150. The movement control information including the movement amount is transmitted (S38).

The UAV control unit 101 may transmit the position information of each unmanned aerial vehicle 100 after movement to the terminal 80 via the communication interface 150. In the terminal 80, the terminal control unit 81 updates the display position of each unmanned aerial vehicle 100 displayed on the display unit 88 based on the position information of each unmanned aerial vehicle 100 after movement (S25). After that, the terminal control unit 81 returns to the process of S23.

The UAV control unit 110 controls the flight of the own aircraft (S39) so that the own aircraft moves to a position corresponding to the movement amount of the own aircraft (specific unmanned aerial vehicle 100). After this, the UAV control unit 110 returns to the process of S31.

As described above, in the flight body group control system 10, the unmanned aerial vehicle 100 is provided with the control node C even when the flight route and the flight position of each unmanned aerial vehicle 100 are not set in advance, and the control node C and each unmanned aerial vehicle are provided. By associating with 100, each unmanned aerial vehicle 100 can be moved according to the movement of the control node C. Therefore, the unmanned aerial vehicle 100 can instruct flight control to a plurality of unmanned aerial vehicles 100 in real time, and can increase the degree of freedom in flight of the unmanned aerial vehicle 100. Further, the user does not need to individually set the flight route and the flight position for the unmanned aerial vehicle 100, and can easily instruct the flight control. Since the unmanned aerial vehicle 100 can move a plurality of unmanned aerial vehicles 100 by moving with respect to the control node C, it is not necessary to prepare a plurality of operating devices for operating the plurality of unmanned aerial vehicles 100, and a plurality of flights are performed. It can facilitate the coordination of the body.

Further, the unmanned aerial vehicle 100 can collectively perform from the generation of the movement control information of the own aircraft to the flight control based on the movement control information. Therefore, the unmanned aerial vehicle 100 can reduce the processing load of the terminal 80 related to the control of the movement of the plurality of unmanned aerial vehicles 100 accompanying the movement of the control node C.

When the flight body group control system 10 includes a transmitter (propo), the process executed by the terminal 80 may be executed by the transmitter. Since the transmitter has the same components as the terminal 80, detailed description thereof will be omitted. The transmitter has a control unit, an operation unit, a communication unit, a memory, and the like. The operation unit may be, for example, a control stick for instructing the control of the flight of the unmanned aerial vehicle 100. For example, the control stick may accept a user's operation to move the control node C. The transmitter may have a display unit.

Although the present disclosure has been described above using the embodiments, the technical scope of the present disclosure is not limited to the scope described in the above-described embodiments. It will be apparent to those skilled in the art to make various changes or improvements to the embodiments described above. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present disclosure.

The order of execution of each process such as operation, procedure, step, and step in the apparatus, system, program, and method shown in the claims, the specification, and the drawing is particularly "before" and "prior to". , Etc., and can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. is not.

10 Aircraft group control system 80 Terminal 81 Terminal control unit 83 Operation unit 85 Communication unit 87 Memory 88 Display unit 89 Storage 100 Unmanned aerial vehicle 100G Unmanned aerial vehicle group 110 UAV control unit 150 Communication interface 160 Memory 170 Storage 200 Gimbal 210 Rotating wing mechanism 220 , 230 Imaging unit 240 GPS receiver 250 Inertial measurement unit 260 Magnetic compass 270 Atmospheric pressure meter 280 Ultrasonic sensor 290 Laser measuring instrument b12, b13, b14 Bone C1, C2, C3, C4 Control node p1, p2, p3, p4 Position STV0 Movement direction u1 to u7 Position before movement u11 to u17 Position after movement

Claims (20)

An information processing device that directs flight control of multiple aircraft.
Equipped with a processing unit
The processing unit
Acquire information on one or more control nodes that are virtual control nodes different from the flying object and control the flight of a plurality of the flying objects.
Associating one or more of the control nodes with the plurality of the flying objects,
Acquire the second operation information for moving one or more of the control nodes input to the operation unit by the user, and obtain the second operation information.
Based on the second operation information, one or more of the control nodes are moved.
In conjunction with the movement of one or more control nodes, one or more of the airframes associated with the moved control node are moved.
Information processing equipment. The processing unit
An association of any of a plurality of flying objects with a control node having the shortest distance to the arbitrary flying object and a control node having the second shortest distance to the arbitrary flying object.
The information processing apparatus according to claim 1. The processing unit
Acquire the first operation information that associates one or more of the control nodes with the plurality of the flying objects input to the operation unit by the user, and obtains the first operation information.
Associating one or more control nodes with a plurality of aircraft based on the first operational information.
The information processing apparatus according to claim 1 or 2. The information of the control node includes the position information of the control node.
The processing unit
Acquire the position information of a plurality of the above-mentioned flying objects, and
Based on the position information of one or more control nodes and the position information of a plurality of the flying objects, the value of the association between the one or more control nodes and the plurality of the flying objects is calculated.
The information processing apparatus according to claim 1 or 2. The processing unit
Holds movement restriction information that restricts the movement of one or more control nodes,
Restrict the movement of one or more control nodes based on the movement restriction information.
The information processing apparatus according to any one of claims 1 to 4 . The processing unit
The positions of a plurality of the flying objects when the moving objects associated with the moved control nodes are moved are calculated, and the positions of the plurality of the flying objects are calculated.
When the calculated distance between the positions of the plurality of aircraft is equal to or greater than the threshold value, the aircraft associated with the moved control node is moved.
When the distance is less than the threshold value, the flying object associated with the moved control node is not moved.
The information processing apparatus according to any one of claims 1 to 4 . The processing unit sets the positional relationship of the plurality of control nodes.
The information processing apparatus according to any one of claims 1 to 6 . The information processing device is a terminal.
The processing unit
Generates movement control information to move the flying object associated with the moved control node.
The movement control information is transmitted to a plurality of the flying objects.
The information processing apparatus according to any one of claims 1 to 7 . The information processing device is an arbitrary flying object among the plurality of the flying objects.
The processing unit
Generates movement control information to move the flying object associated with the moved control node.
Move based on the movement control information,
The information processing apparatus according to any one of claims 1 to 7 . It is a flight control instruction method in an information processing device that instructs flight control of a plurality of aircraft.
A step of acquiring information of one or more control nodes that are virtual control nodes different from the flying object and control the flight of the plurality of the flying objects.
A step of associating one or more control nodes with a plurality of the flying objects.
A step of acquiring a second operation information for moving one or more control nodes input to the operation unit by the user, and a step of acquiring the second operation information.
A step of moving one or more control nodes based on the second operation information,
A step of moving one or more of the flying objects associated with the moved control node in conjunction with the movement of the one or more control nodes.
Flight control instruction method with. The step of associating one or more control nodes with the plurality of flying objects is a control node having the shortest distance between any flying object among the plurality of flying objects and the arbitrary flying object and the arbitrary flight. Includes a step to associate with the control node, which has the second shortest distance to the body,
The flight control instruction method according to claim 10 . Further including a step of acquiring a first operation information associating the one or more control nodes and the plurality of the flying objects input to the operation unit by the user.
The step of associating the one or more control nodes with the plurality of flying objects includes a step of associating the one or more control nodes with the plurality of the flying objects based on the first operation information.
The flight control instruction method according to claim 10 or 11 . The information of the control node includes the position information of the control node.
Further including the step of acquiring the position information of the plurality of said aircraft.
The step of associating the one or more control nodes with the plurality of flying objects is based on the position information of the one or more control nodes and the position information of the plurality of the control nodes. And the step of calculating the value of the association with the plurality of said aircraft, including,
The flight control instruction method according to claim 10 or 11 . A step of holding movement restriction information that restricts the movement of one or more control nodes, and
Further comprising limiting the movement of one or more control nodes based on the movement restriction information.
The flight control instruction method according to any one of claims 10 to 13 . The step of moving the flying object is
A step of calculating the positions of a plurality of the flying objects when the moving objects associated with the moved control nodes are moved, and a step of calculating the positions of the plurality of the flying objects.
When the calculated distance between the positions of the flying objects is equal to or greater than the threshold value, the step of moving the flying objects associated with the moved control node and the step of moving the flying objects.
A step of not moving the air vehicle associated with the moved control node when the distance is less than the threshold is included.
The flight control instruction method according to any one of claims 10 to 13 . Further including a step of setting the positional relationship of the plurality of control nodes.
The flight control instruction method according to any one of claims 10 to 15 . The information processing device is a terminal.
The step of moving the flying object is
A step of generating movement control information for moving the flying object associated with the moved control node, and
A step of transmitting the movement control information to the plurality of the flying objects, and the like.
The flight control instruction method according to any one of claims 10 to 16 . The information processing device is an arbitrary flying object among the plurality of the flying objects.
The step of moving the flying object includes a step of generating movement control information for moving the flying object associated with the moved control node.
Further including a step of moving based on the movement control information.
The flight control instruction method according to any one of claims 10 to 16 . For information processing devices that instruct the control of the flight of multiple aircraft,
A step of acquiring information of one or more control nodes that are virtual control nodes different from the flying object and control the flight of the plurality of the flying objects.
A step of associating one or more control nodes with a plurality of the flying objects.
A step of acquiring a second operation information for moving one or more control nodes input to the operation unit by the user, and a step of acquiring the second operation information.
A step of moving one or more control nodes based on the second operation information,
A step of moving one or more of the flying objects associated with the moved control node in conjunction with the movement of the one or more control nodes.
A program to execute. For information processing devices that instruct the control of the flight of multiple aircraft,
A step of acquiring information of one or more control nodes that are virtual control nodes different from the flying object and control the flight of the plurality of the flying objects.
A step of associating one or more control nodes with a plurality of the flying objects.
A step of acquiring a second operation information for moving one or more control nodes input to the operation unit by the user, and a step of acquiring the second operation information.
A step of moving one or more control nodes based on the second operation information,
A step of moving one or more of the flying objects associated with the moved control node in conjunction with the movement of the one or more control nodes.
A computer-readable recording medium that contains a program for executing the program.

Images