JP7082351B2 - Flight control system - Google Patents

Description

The present invention relates to a flight control system including an operation information creating device for creating operation information for operating an unmanned flight device.

Conventionally, a technique for operating an unmanned flight device with an operating device is known. Patent Document 1 discloses a technique for moving an unmanned flight device based on a change in the three-dimensional position of the operating device.

Japanese Unexamined Patent Publication No. 2019-142290

In the conventional technique, it is necessary for the operator to operate the unmanned flight device by moving the operating device. When operating an unmanned flight device using an operating device, the range in which the unmanned flight device can be moved is limited to the operable range of the operating device. Therefore, there has been a problem that it is difficult to move the unmanned flight device as the operator desires.

Therefore, the present invention has been made in view of these points, and an object thereof is to improve the operability of the unmanned flight apparatus.

The operation information creating device of the first aspect of the present invention is an operation information creating device for creating operation information used for operating an unmanned flight device, and is around a position where the unmanned flight device is in flight. In the three-dimensional space of the operator's finger or hand of the operation information creating device, an information acquisition unit that acquires peripheral image information indicating a state, a display control unit that displays a peripheral image based on the peripheral image information on the display unit, and a display unit. An operation for operating the unmanned flight device based on the movement specifying unit for specifying the movement and the movement of the finger or hand detected by the movement specifying unit while the peripheral image is displayed on the display unit. It has an operation information creation unit that creates information.

When the unmanned flight device is moved based on the movement of the finger or hand and the unmanned flight device performs an unacceptable operation, the operation information creating unit performs an operation different from the operation corresponding to the movement of the finger or hand. The operation information for making the unmanned flight device may be created.

When the operation information creating unit determines that the unmanned flight device collides with an obstacle when the unmanned flight device is moved based on the movement of the finger or the hand, the unmanned flight so as not to collide with the obstacle. The operation information for operating the device may be created.

When the operation information creating unit moves the unmanned flight device based on the movement of the finger or the hand, the speed or acceleration of the unmanned flight device becomes an unacceptable range, and the speed or acceleration of the unmanned flight device is increased. The operation information for causing the unmanned flight device to perform an operation corrected for the operation corresponding to the movement of the finger or the hand so as to be within an allowable range may be created.

The operation information creating unit is based on the movement of the finger or hand detected by the movement specifying unit, starting from the position of the unmanned flight device at the time when the state of the operator's hand is in a predetermined state. The operation information for moving the unmanned flight device may be created.

The display control unit may superimpose the virtual image of the unmanned flight device that changes the position in the peripheral image based on the movement of the finger or the hand on the peripheral image and display it on the display unit.

The display control unit may display the locus line corresponding to the movement of the finger or the hand on the peripheral image on the display unit.

The flight control system according to the second aspect of the present invention is an operation device for operating the unmanned flight device and an operation for creating operation information used for operating the unmanned flight device via the operation device. It is equipped with an information creation device. The operation information creating device includes an information acquisition unit that acquires peripheral image information indicating a state around the position where the unmanned flight device is in flight, and a display control unit that displays a peripheral image based on the peripheral image information on the display unit. And the motion specifying unit that specifies the movement of the operator's finger or hand in the three-dimensional space of the operation information creating device, and the motion specifying unit that the motion specifying unit detects while the peripheral image is displayed on the display unit. It has an operation information creating unit that creates operation information for operating the unmanned flight device based on the movement of a finger or a hand, and an operation information transmitting unit that transmits the operation information to the operation device. The operation device receives the operation information receiving unit for receiving the operation information from the operation information creating device, and the unmanned operation device for operating the unmanned flight device based on the operation information received by the operation information receiving unit. It has an instruction data transmission unit for transmitting to the flight device.

The operation device further includes a display control unit for displaying on the display unit that the operation information has been received, and an operation reception unit for receiving the operation of the user of the operation device, and the instruction data transmission unit includes the instruction data transmission unit. When the operation receiving unit accepts an operation that permits the unmanned flight apparatus to operate based on the operation information, the instruction data based on the operation information may be transmitted to the unmanned flight apparatus.

According to the present invention, there is an effect that the operability of the unmanned flight apparatus is improved.

It is a figure which shows the outline of a flight control system. It is a figure which shows the structure of the operation device. It is a figure which shows the structure of the operation information creation apparatus. It is a figure which shows typically the image around the unmanned flight apparatus displayed on the display part. It is a figure for demonstrating the movement of a finger. It is a figure which shows an example of the screen which is displayed on the display part when an operator moves a finger. It is a figure which shows the operation which an unmanned flight apparatus avoids an obstacle. It is a figure which shows the relationship between the speed of a finger of an operator and the speed of an unmanned flight apparatus. It is a figure which shows the movement of a finger and the movement of an unmanned flight device when a finger vibrates finely. It is a figure which shows the operation of the unmanned flight apparatus when an operator moves a finger sequentially in a 1st direction and a 2nd direction. It is a figure which shows the processing sequence in a flight control system.

[Overview of flight control system S]
FIG. 1 is a diagram showing an outline of the flight control system S. The flight control system S is a system for flying an unmanned flight device 1, and includes an unmanned flight device 1, an operation device 2, and an operation information creation device 3.

The unmanned aerial vehicle 1 is an unmanned aerial vehicle that flies in the air, for example, a drone. The unmanned flight device 1 flies in the direction indicated by the instruction data received from the operation device 2. The unmanned flight device 1 is provided with a camera, and can photograph the surroundings of the unmanned flight device 1 during flight.

The operation device 2 is a device for operating the unmanned flight device 1, for example, a smartphone. The operation device 2 is used by the operator U1 who operates the unmanned flight device 1 while visually recognizing the unmanned flight device 1. The operating device 2 transmits instruction data for operating the unmanned flight device 1 to the unmanned flight device 1 via, for example, the wireless communication line W1 of Wi-Fi (registered trademark).

The operation information creation device 3 creates operation information used for operating the unmanned flight device 1. The operation information creating device 3 is, for example, a device used by an operator U2 who supports the operation of the operator U1 who uses the operation device 2, and is a goggle type device which the operator U2 wears on the head and uses. be. The operation information creating device 3 may include a display 31 which is a display unit that can be visually recognized by the operator U2 while the operator U2 is attached to the head, and a finger or a hand of the operator U2 (hereinafter, referred to as a “finger”). ) Is provided with a detection device 32 for detecting. The detection device 32 is, for example, a camera.

The display 31 of the operation information creating device 3 displays an image showing the state around the unmanned flight device 1. The image displayed on the operation information creating device 3 is, for example, a three-dimensional image or a two-dimensional image captured by the unmanned flight device 1, or a computer graphics image created in advance. The operation information creating device 3 identifies the movement of the finger of the operator U in the three-dimensional space while displaying the image, and moves the unmanned flight device 1 based on the detected finger movement. Create operation information.

As an example, the operation information creating device 3 transmits the created operation information to the operating device 2 via a wireless communication line W2 (for example, LTE or Wi-Fi) different from the wireless communication line W1. When the predetermined condition is satisfied, the operation device 2 transmits the operation information received from the operation information creation device 3 to the unmanned flight device 1 as instruction data for operating the unmanned flight device 1. The predetermined conditions are, for example, that the operation device 2 is set to a mode in which the operation by the operation information creation device 3 is permitted, and that the operator U1 operates the operation device 2 to move the unmanned flight device 1. There is no such thing. The operation information creating device 3 may directly transmit the created operation information to the unmanned flight device 1.

As described above, the operation information creating device 3 identifies and detects the movement of the finger of the operator U2 performed while presenting the image showing the state around the unmanned flight device 1 to the operator U2. Create operation information based on. By operating the operation information creating device 3 in this way, the operator U2 can move the unmanned flight device 1 by a desired route while feeling as if he / she is the unmanned flight device 1.

[Configuration of operating device 2]
FIG. 2 is a diagram showing the configuration of the operating device 2. The operation device 2 includes a display 21, an operation unit 22, a communication unit 23, a storage unit 24, and a control unit 25. The control unit 25 includes an information acquisition unit 251, a display control unit 252, an operation reception unit 253, and an instruction data creation unit 254.

The display 21 is a display unit that displays various information to be notified to the operator U1.
The operation unit 22 is a device for receiving the operation of the operator U1, and is, for example, a touch panel provided on the display 21.

The communication unit 23 is a communication interface for transmitting and receiving various data between the unmanned flight device 1 and the operation device 2, and has, for example, a wireless communication controller. The communication unit 23 transmits instruction data for operating the unmanned flight device 1 to the unmanned flight device 1 via the wireless communication line W1, and the state data indicating the state of the unmanned flight device 1 and the unmanned flight device 1 are transmitted from the unmanned flight device 1 to the unmanned flight device 1. The flight device 1 receives the captured image data of the unmanned flight device 1 generated by photographing the surroundings. Further, the communication unit 23 receives the operation information transmitted from the operation information creating device 3 via the wireless communication line W2.

The storage unit 24 has a storage medium such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 24 stores a program executed by the control unit 25. Further, the storage unit 24 temporarily stores the image around the unmanned flight device 1 received by the communication unit 23.

The control unit 25 is, for example, a CPU (Central Processing Unit). The control unit 25 functions as an information acquisition unit 251, a display control unit 252, an operation reception unit 253, and an instruction data creation unit 254 by executing the program stored in the storage unit 24.

The information acquisition unit 251 functions as an operation information receiving unit that acquires operation information transmitted from the operation information creating device 3 via the communication unit 23. The information acquisition unit 251 notifies the display control unit 252 that the operation information has been acquired, and inputs the acquired operation information to the instruction data creation unit 254. Further, the information acquisition unit 251 acquires the state data and the captured image data transmitted from the unmanned flight apparatus 1, and inputs the state data and the captured image data to the display control unit 252.

The display control unit 252 displays various information on the display 21. The display control unit 252 displays, for example, the state data and the captured image data input from the information acquisition unit 251 on the display 21. Further, when the display control unit 252 receives the notification from the information acquisition unit 251 that the operation information has been acquired from the operation information creation device 3, the display 21 displays that the operation information has been acquired. The display control unit 252 further causes the display 21 to display a screen for accepting an operation permitting the unmanned flight apparatus 1 to operate based on the operation information transmitted from the operation information creating apparatus 3.

The operation reception unit 253 receives the operation input by the operator U1 in the operation unit 22. The operation receiving unit 253 identifies the position touched by the user based on the coordinate data received from the operation unit 22, for example, and accepts the operation corresponding to the specified position. The operation reception unit 253, for example, accepts an operation for controlling the position and flight speed of the unmanned flight device 1, and notifies the instruction data creation unit 254 of the content of the accepted operation. Further, when the operation reception unit 253 receives an operation for permitting the operation of the unmanned flight apparatus based on the operation information, the operation reception unit 253 notifies the instruction data creation unit 254 that the operation has been accepted.

The instruction data creation unit 254 creates instruction data for operating the unmanned flight device 1 based on the content of the operation notified from the operation reception unit 253. The instruction data creation unit 254 functions as an instruction data transmission unit that transmits the created instruction data to the unmanned flight device 1 via the communication unit 23. The instruction data creation unit 254 receives instruction data based on the operation information transmitted from the operation information creation device 3 when the operation reception unit 253 receives an operation for permitting the unmanned flight device 1 to operate based on the operation information. Is created, and the created instruction data is transmitted to the unmanned flight device 1.

The instruction data creation unit 254 is an operation for controlling the unmanned flight device 1 by the operator U1 even when the operation reception unit 253 accepts an operation for permitting the operation of the unmanned flight device 1 based on the operation information. When the above is performed, the instruction data based on the operation of the operator U1 may be preferentially transmitted to the unmanned flight apparatus 1. By operating the instruction data creation unit 254 in this way, when the operator U1 who is visually recognizing the unmanned flight device 1 recognizes a dangerous situation, he / she can perform an operation to avoid the danger.

On the other hand, the instruction data creation unit 254 operates an operation that permits the unmanned flight device 1 to be operated based on the operation information, for example, the skill of the operator U1 stored in the storage unit 24 is less than the threshold level. When the reception unit 253 accepts, even if the operation to control the unmanned flight device 1 by the operator U1 is performed, the instruction data based on the operation information transmitted from the operation information creation device 3 is given priority to the unmanned flight device 1. May be sent to. By operating the instruction data creation unit 254 in this way, when the operator U1 with a low skill makes an erroneous operation, the danger can be avoided by the operation of the operator U2 with a high skill.

[Configuration of operation information creation device 3]
FIG. 3 is a diagram showing the configuration of the operation information creating device 3. The operation information creating device 3 has a communication unit 33, a storage unit 34, and a control unit 35 in addition to the display 31 and the detection device 32 shown in FIG. The control unit 35 includes an information acquisition unit 351, a display control unit 352, a motion specifying unit 353, and an operation information creating unit 354.

The display 31 is a display that displays an image and other information around the unmanned flight apparatus 1. FIG. 4 is a diagram schematically showing an image of the periphery of the unmanned flight apparatus 1 displayed on the display 31. In FIG. 4, the current position (★) of the unmanned flight device 1 and the position (■) of the destination to move the unmanned flight device 1 are superimposed on the image of the periphery of the unmanned flight device 1 taken by the unmanned flight device 1. The image shown is displayed.

The detection device 32 detects the finger of the operator U2. The detection device 32 has, for example, a stereo camera having a plurality of image pickup elements. The detection device 32 creates distance data indicating the distance between the subject and the stereo camera corresponding to each pixel of the plurality of captured images, based on, for example, a plurality of captured images created by each of the plurality of image pickup elements. The detection device 32 inputs a plurality of captured images and the created distance data to the motion specifying unit 353.

The communication unit 33 is a communication interface for receiving an image of the periphery of the unmanned flight device 1 and transmitting operation information, and has, for example, a wireless communication controller. The communication unit 33 stores the received image in the storage unit 34. The communication unit 33 transmits the operation information created by the operation information creation unit 354 to the wireless communication line W2.

The storage unit 34 has a storage medium such as a ROM and a RAM. The storage unit 34 stores a program executed by the control unit 35. Further, the storage unit 34 temporarily stores the image around the unmanned flight device 1 received by the communication unit 33.

The control unit 35 is, for example, a CPU. The control unit 35 functions as an information acquisition unit 351, a display control unit 352, a motion specifying unit 353, and an operation information creation unit 354 by executing a program stored in the storage unit 34.

The information acquisition unit 351 acquires peripheral image information indicating a state around the position where the unmanned flight device 1 is in flight via the communication unit 33. The information acquisition unit 351 inputs the acquired peripheral image information to the display control unit 352. The information acquisition unit 351 may further acquire obstacle information for identifying an obstacle in flight of the unmanned flight apparatus 1 in association with peripheral image information.

The display control unit 352 causes the display 31 to display a peripheral image based on the peripheral image information. The display control unit 352, for example, superimposes a virtual image of the unmanned flight device 1 that changes the position in the peripheral image based on the movement of the finger of the operator U2 on the peripheral image and displays it on the display unit. As an example, as shown in FIG. 4, the display control unit 352 causes the display 31 to display an image showing the current position and the position of the destination of the unmanned flight apparatus 1 together with the peripheral image.

The motion specifying unit 353 identifies the motion of the finger of the operator U2 of the operation information creating device 3 in the three-dimensional space based on the captured image and the distance data input from the detection device 32. FIG. 5 is a diagram for explaining the movement of the finger.

FIG. 5 shows an image of the hand of the operator U2 in a three-dimensional space represented by the X-axis, Y-axis, and Z-axis. FIG. 5A shows a state before the operator U2 starts the operation of moving the finger. FIG. 5B shows a state in which the operator U2 starts the operation of moving the finger, and the three-dimensional coordinates of the position of the tip of the finger are (x0, y0, z0). FIG. 5C shows a state after the operator U2 moves the finger, and the three-dimensional coordinates of the position of the tip of the finger are (x1, y1, z1).

The motion specifying unit 353 identifies the region of the operator U2's hand in the captured image by comparing the reference hand image with the captured image, and identifies the position of the tip of the finger based on the identified region. .. Subsequently, the motion specifying unit 353 specifies the distance between the detection device 32 and the position of the tip of the finger by referring to the distance data.

The motion specifying unit 353 specifies the position of the tip of the finger in the three-dimensional space based on the position of the tip of the finger in the captured image and the specified distance. The movement specifying unit 353 specifies the position of the tip of the finger at a predetermined time interval (for example, 100 millisecond interval), and specifies the movement of the finger by the change of the specified position. The movement specifying unit 353 notifies the operation information creating unit 354 of the specified finger movement.

The movement specifying unit 353 may specify the state of the finger of the operator U2. The motion specifying unit 353 is shown in, for example, a state in which the finger of the operator U2 is not extended (for example, a state in which the hand is held) as shown in FIG. 5 (a), and FIGS. 5 (b) and 5 (c). The operator U2 is identified as having one finger extended. The movement specifying unit 353 notifies the operation information creating unit 354 of the state of the specified finger.

The movement specifying unit 353 starts from the time when the state of the hand of the operator U2 changes from the first state (for example, the state of holding the hand) to the second state (for example, the state where one finger is extended). Identify the movement. The motion specifying unit 353 notifies the operation information creating unit 354 of the three-dimensional coordinates of the position of the tip of the finger at predetermined time intervals after the state of the hand of the operator U2 is changed to the second state. The motion specifying unit 353 may notify the operation information creating unit 354 of vector data indicating the movement of the position of the tip of the finger.

The operation information creating unit 354 creates operation information for operating the unmanned flight device 1 based on the movement of the finger detected by the movement specifying unit 353 while the peripheral image is displayed on the display 31. The operation information creation unit 354, for example, creates operation information corresponding to information (for example, three-dimensional coordinates or vector data) indicating the movement of the finger notified from the movement identification unit 353. As an example, the operation information creation unit 354 creates information in which the time information and the three-dimensional coordinates corresponding to the position of the finger are associated with each other as the operation information. The operation information creation unit 354 transmits operation information including one or more three-dimensional coordinates to the operation device 2 at predetermined time intervals via the communication unit 33.

The operation information creation unit 354 may convert the information indicating the movement of the finger into the operation information indicating the direction in which the unmanned flight apparatus 1 flies. For example, the operation information creation unit 354 moves the movement component of the finger in the X-axis direction in the three-dimensional space in the left-right direction of the unmanned flight device 1 and the movement component of the finger in the Y-axis direction in the vertical direction of the unmanned flight device 1. The component and the movement component of the finger in the Z-axis direction are converted into the movement component in the front-back direction of the unmanned flight device 1. The operation information creation unit 354 creates operation information indicating a vector obtained by synthesizing each of the converted moving components.

The operation information creation unit 354 is an unmanned flight device 1 at a time when the state of the hand of the operator U2 changes from the first state (for example, the state of holding the hand) to the second state (for example, the state of extending one finger). Based on the movement of the finger detected by the movement specifying unit 353, the operation information for moving the unmanned flight device 1 is created from the position of. The operation information creation unit 354 is said to have changed from, for example, a state in which the operator U2 is holding a hand (state in FIG. 5A) to a state in which one finger is extended (state in FIG. 5B). Starting from the time when the notification is received from the movement specific unit 353, the operation information corresponding to the subsequent finger movement is created.

Specifically, the operation information creation unit 354 creates operation information corresponding to the movement of the finger from the position of the finger shown in FIG. 5 (b) to the position of the finger shown in FIG. 5 (c). The operation information creation unit 354 continues to create operation information corresponding to the movement of the finger until the state of the hand of the operator U2 changes from the second state to the first state. In this way, the operation information creation unit 354 determines the period for creating the operation information according to the state of the hand of the operator U2, so that the finger while the operator U2 does not have the intention to move the unmanned flight device 1. It is possible to prevent the operation information from being created based on the movement of.

FIG. 6 is a diagram showing an example of a screen displayed on the display 31 when the operator U2 moves his / her finger. As shown in FIG. 6, the display control unit 352 may display the locus line corresponding to the movement of the finger of the operator U2 on the peripheral image on the display 31. The locus line of the unmanned flight apparatus 1 shown by the broken line in FIG. 6 corresponds to the locus line of the finger shown by the broken line in FIG. 5 (c). By displaying such a locus line on the display 31 by the display control unit 352, the operator U2 can move the finger while checking the movement locus of the unmanned flight device 1, so that the operator U2 can move the finger. It is easier to understand whether it is more appropriate.

The display control unit 352 may display the trajectory line on the display 31 before the unmanned flight device 1 actually moves when the operator U2 moves the finger. By displaying the locus line on the display 31 in this way, the display control unit 352 can perform the operation again when the route indicated by the locus line is not the route desired by the operator U2.

The operation information creation unit 354 creates the operation information creation unit 354 when it is specified that the hand of the operator U2 is in the third state (for example, the hand is open) so that the operator U2 can perform the operation again. You may cancel the completed operation information. In this case, the operation information creating unit 354 moves the unmanned flight device 1 based on the movement of the finger detected by the movement specifying unit 353 from the time when the hand of the operator U2 enters the second state through the first state. Create operation information for. The operation information creation unit 354 switches the created operation information to new operation information when the hand of the operator U2 is in the second state. By operating the operation information creating unit 354 in this way, the unmanned flight apparatus 1 can be continuously moved even when the operation information is switched.

When the unmanned flight device 1 is moved based on the movement of the finger and the unmanned flight device 1 performs an unacceptable operation, the operation information creation unit 354 causes the unmanned flight device 1 to perform an operation different from the operation corresponding to the movement of the finger. You may create the operation information to make it. For example, the operation information creation unit 354 identifies an obstacle included in an image showing a state around the unmanned flight device 1, and when the unmanned flight device 1 is moved based on the movement of a finger, the operation information creation unit 354 collides with the obstacle. When it is determined, the operation information for operating the unmanned flight apparatus 1 so as not to collide with an obstacle is created. The operation information creation unit 354 may identify obstacles in the vicinity of the unmanned flight apparatus 1 based on the obstacle information acquired by the information acquisition unit 351 in association with the peripheral image.

FIG. 7 is a diagram showing the operation of the unmanned flight device 1 to avoid obstacles. Even when the operator U2 moves his / her finger as shown by the broken line arrow in FIG. 7 (a), the operation information creating unit 354 is the unmanned flight device 1 indicated by ★ as shown in FIG. 7 (b). Create operational information to fly to avoid obstacles.

The operation information creating unit 354 may create operation information indicating the moving speed of the unmanned flight device 1 based on the speed at which the finger moves detected by the movement specifying unit 353. The operation information creation unit 354 converts the speed at which the finger moves into the speed at which the unmanned flight device 1 moves, for example, by referring to the conversion table stored in the storage unit 34 in advance.

However, the operation information creation unit 354 allows the speed or acceleration of the unmanned flight device 1 when the speed or acceleration of the unmanned flight device 1 is within the allowable range when the unmanned flight device 1 is moved based on the movement of the finger. The operation information for making the unmanned flight device 1 perform the motion corrected for the motion corresponding to the movement of the finger may be created so as to be within the range. For example, an upper limit is set for the speed or acceleration at which the unmanned flight device 1 moves, and the operation information creation unit 354 limits the speed or acceleration at which the unmanned flight device 1 moves when the speed or acceleration at which the finger moves is equal to or greater than the threshold value. It may be a value.

FIG. 8 is a diagram showing the relationship between the speed of the finger of the operator U2 and the speed of the unmanned flight device 1. As shown in FIG. 8, the operation information creation unit 354 creates operation information in which the speed of the unmanned flight device 1 increases as the speed of the finger increases while the speed of the finger is equal to or less than the threshold value. The operation information creation unit 354 creates operation information in which the speed of the unmanned flight device 1 does not increase even if the speed of the finger increases when the speed of the finger becomes larger than the threshold value. By operating the operation information creating unit 354 in this way, it is possible to prevent the unmanned flight apparatus 1 from flying at a dangerous speed or acceleration.

The operation information creation unit 354 may create operation information so that the unmanned flight apparatus 1 moves more smoothly than the trajectory line of the finger movement detected by the movement identification unit 353. The operation information creation unit 354 removes the high frequency component contained in the locus line by, for example, passing the locus line of the finger movement through the low frequency passing filter, and operates based on the locus line after the high frequency component is removed. Create information.

FIG. 9 is a diagram showing the movement of the finger and the movement of the unmanned flight device 1 when the finger vibrates finely. In FIG. 9A, the finger of the operator U2 is displaced to the left and right with respect to the moving direction, but in FIG. 9B, the unmanned flight device 1 indicated by ★ moves in a substantially straight line. There is. By operating the operation information creating unit 354 in this way, it is possible to prevent the unmanned flight apparatus 1 from flying while vibrating finely when the finger vibrates finely.

Further, when the finger of the operator U2 moves at a speed equal to or higher than the threshold value, the operation information creation unit 354 may create operation information that does not include the movement operation. For example, after the finger moves in the first direction, the operation information creating unit 354 moves in the second direction, which is almost opposite to the first direction, at a speed faster than the moving speed in the first direction. In this case, operation information that does not include the movement operation in the second direction is created.

FIG. 10 is a diagram showing the operation of the unmanned flight apparatus 1 when the operator U2 sequentially moves his / her finger in the first direction and the second direction. The solid line in FIG. 10A shows the first direction, and the broken line shows the second direction. Even when the operator U2 moves his / her finger in this way, as shown in FIG. 10B, the unmanned flight device 1 continues to fly in the first direction. By operating the operation information creating unit 354 in this way, the operator U2 can continuously move the unmanned flight device 1 without switching the state of the hand between the first state and the second state. ..

[Sequence Diagram]
FIG. 11 is a diagram showing a processing sequence in the flight control system S. The sequence diagram shown in FIG. 11 starts from a state in which the unmanned flight device 1 is flying based on the first instruction data transmitted from the operation device 2 by the operation of the operator U1.

When the operation reception unit 253 accepts the operation for permitting the remote operation by the operator U2 (S1), the operation reception unit 253 notifies the operation information creating device 3 that the remote operation is permitted via the communication unit 23. When the operation information creating device 3 receives the notification that the remote control is permitted, the display control unit 352 displays the notification on the display 31 (S2). After that, when the movement specifying unit 353 specifies the movement of the finger of the operator U2 (S3), the operation information creating unit 354 creates the operation information based on the movement of the finger (S4), and the created operation information is used as the operation device. Send to 2.

In the operation device 2, when the information acquisition unit 251 receives the operation information from the operation information creation device 3, the instruction data creation unit 254 creates the second instruction data based on the operation information (S5), and the created second instruction. The data is transmitted to the unmanned flight device 1. The instruction data creation unit 254 creates the second instruction data based on the operation information until the operation reception unit 253 accepts a predetermined operation (NO in step S6). The instruction data creation unit 254 creates instruction data based on the operation content received by the operation reception unit 253 after the operation reception unit 253 receives a predetermined operation (YES in step S6) (S7).

[First modification]
In the above description, the flight control system S includes the operation device 2 and the operation information creation device 3, the operation device 2 receives the operation information created by the operation information creation device 3, and the operation device 2 receives the operation information. Although the case where the instruction data created based on the above is transmitted to the unmanned flight device 1 is exemplified, the flight control system S may not have the operation device 2. In this case, the operation information creating device 3 creates instruction data based on the operation information, and directly transmits the created instruction data to the unmanned flight device 1.

[Second modification]
In the above description, the case where the operation information creating device 3 creates the operation information based only on the movement of the finger or the hand of the operator U2 in the three-dimensional space is illustrated, but the operation information creating device 3 is the operator U2. The operation information may be created based on the operation contents performed by the controller (operation unit) (not shown) included in the operation information creation device 3 together with the movement of the finger or the hand.

In this case, the operation information creation unit 354 creates, for example, operation information regarding the direction in which the unmanned flight device 1 is moved based on the movement of the finger or hand of the operator U2, and the operation information regarding the speed at which the unmanned flight device 1 is moved. Is created based on the operation performed by the controller. By operating the operation information creating unit 354 in this way, the operator U2 can easily set the speed with the controller in advance, or set the speed to a constant value by operating the controller, for example. Therefore, the operator U2 does not have to worry about the speed at which the finger or the hand is moved, and only needs to pay attention to the direction in which the finger or the hand is moved, so that the operability is further improved.

[Effect of operation information creation device 3]
As described above, in the operation information creating device 3, the movement specifying unit 353 that specifies the movement of the finger or hand of the operator U2 of the operation information creating device 3 in the three-dimensional space, and the peripheral image are displayed on the display 31. The operation information for operating the unmanned flight apparatus 1 is created based on the movement of the finger or the hand detected by the movement specifying unit 353 while the movement is being performed. With the operation information creating device 3 configured in this way, the operator U2 can move the unmanned flight device 1 in a desired direction by moving a finger or a hand in the direction in which the unmanned flight device 1 is desired to be moved. Since it can be done, operability is improved.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, all or part of the device can be functionally or physically distributed / integrated in any unit. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

1 Unmanned flight device 2 Operation device 3 Operation information creation device 21 Display 22 Operation unit 23 Communication unit 24 Storage unit 25 Control unit 31 Display 32 Detection device 33 Communication unit 34 Storage unit 35 Control unit 251 Information acquisition unit 252 Display control unit 253 Operation Reception unit 254 Instruction data creation unit 351 Information acquisition unit 352 Display control unit 353 Motion identification unit 354 Operation information creation unit

Claims (8)

An operation device for operating the unmanned flight device and an operation information creating device for creating operation information used for operating the unmanned flight device via the operation device are provided.
The operation information creating device is
An information acquisition unit that acquires peripheral image information indicating the state around the position where the unmanned flight device is in flight, and an information acquisition unit.
A display control unit that displays a peripheral image based on the peripheral image information on the display unit,
A movement specifying unit that specifies the movement of the finger or hand of the operator of the operation information creating device in the three-dimensional space.
An operation information creating unit that creates operation information for operating the unmanned flight device based on the movement of the finger or hand detected by the movement specifying unit while the peripheral image is displayed on the display unit. ,
An operation information transmission unit that transmits the operation information to the operation device,
Have,
The operating device is
An operation information receiving unit that receives the operation information from the operation information creating device, and
An instruction data transmitting unit that transmits instruction data for operating the unmanned flight device to the unmanned flight device based on the operation information received by the operation information receiving unit.
Flight control system with. The operation device includes a display control unit that displays on the display unit that the operation information has been received.
An operation reception unit that accepts operations by the user of the operation device,
Have more
When the operation receiving unit receives an operation that permits the unmanned flight apparatus to operate based on the operation information, the instruction data transmitting unit transmits the instruction data based on the operation information to the unmanned flight apparatus. Send,
The flight control system according to claim 1. When the unmanned flight device is moved based on the movement of the finger or hand and the unmanned flight device performs an unacceptable operation, the operation information creating unit performs an operation different from the operation corresponding to the movement of the finger or hand. To create the operation information for making the unmanned flight device
The flight control system according to claim 1 or 2. When the operation information creating unit determines that the unmanned flight device collides with an obstacle when the unmanned flight device is moved based on the movement of the finger or the hand, the unmanned flight so as not to collide with the obstacle. Creating the above-mentioned operation information for operating the device,
The flight control system according to claim 3. When the operation information creating unit moves the unmanned flight device based on the movement of the finger or the hand, the speed or acceleration of the unmanned flight device becomes an unacceptable range, and the speed or acceleration of the unmanned flight device is increased. The operation information for making the unmanned flight device perform the motion corrected for the motion corresponding to the motion of the finger or the hand so as to be within the permissible range is created.
The flight control system according to any one of claims 1 to 4. The operation information creating unit is based on the movement of the finger or hand detected by the movement specifying unit, starting from the position of the unmanned flight device at the time when the state of the operator's hand is in a predetermined state. Creating the operation information for moving the unmanned flight device,
The flight control system according to any one of claims 1 to 5. The display control unit of the operation information creating device superimposes a virtual image of the unmanned flight device that changes a position in the peripheral image based on the movement of the finger or a hand on the peripheral image, and the operation information creating device. Display on the display
The flight control system according to any one of claims 1 to 6. The display control unit of the operation information creating device superimposes a locus line corresponding to the movement of the finger on the peripheral image and displays it on the display unit of the operation information creating device .
The flight control system according to any one of claims 1 to 7.

Images