JP7029565B2 - Maneuvering equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to a control device for maneuvering a flight device, an information processing method executed by the control device, and a program executed by the control device.

In recent years, drones, which are flight devices that can be operated remotely, have become widespread. Patent Document 1 discloses a technique in which a camera mounted on a flight device captures the surroundings and the captured image data is transmitted to a terminal used by a user to operate the flight device.

Japanese Unexamined Patent Publication No. 2016-174360

When the flight device is flying out of the user's field of view, the user relies on the image taken by the camera mounted on the flight device to steer the flight device. However, in the image taken by the camera mounted on the flight device, the flight device itself is often not included in the subject, or even if it is included, it is only a part thereof. For this reason, even if the image taken by the camera mounted on the flight device is viewed, it is difficult for the user to grasp the positional relationship between the flight device and other objects, the sense of distance, etc., and it is difficult to operate the flight device. was there.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to provide a technique for improving the operability of a flight device that is out of the user's field of view.

The first aspect of the present invention is a control device for maneuvering a flight device via a wireless network. This device has a transmission unit that transmits an operation command indicating the flight direction of the flight device to the flight device via the radio network, and position information indicating the position of the flight device via the radio network. A virtual viewpoint at a position different from the existing position of the flight device, the position information receiving section receiving from the flight device, the real image receiving section for receiving the real image taken by the flight device via the wireless network, and the position of the flight device. A viewpoint setting unit for setting the above, and a display control unit for displaying a flight image showing an area in which the flight device is in flight when the flight device is viewed from the virtual viewpoint on the display unit of the control device. Be prepared. Here, the display control unit stops displaying the flight image when a predetermined condition is satisfied.

The display control unit may display the actual image captured by the flight device and the flight image on the display unit, and may stop displaying the flight image if the predetermined conditions are met.

The display control unit may stop displaying the flight image when the predetermined condition regarding the subject in the actual image is satisfied.

The second aspect of the present invention is an information processing method. In this method, a processor included in a flight device for maneuvering a flight device via a wireless network transmits an operation command indicating the flight direction of the flight device to the flight device via the radio network, and the above-mentioned step. A step of receiving position information indicating the location of the flight device from the flight device via a wireless network, a step of receiving an actual image taken by the flight device via a wireless network, and a step of the flight device. A display unit of the control device displays a step of setting a virtual viewpoint at a position different from the existing position and a flight image showing a flight area of the flight device when the flight device is viewed from the virtual viewpoint. The step of displaying the flight image and the step of stopping the display of the flight image when a predetermined condition is met are executed.

A third aspect of the present invention is a program. This program has a function of transmitting an operation command indicating the flight direction of the flight device to the flight device via the radio network to a computer provided in the control device for maneuvering the flight device via the wireless network. A function of receiving position information indicating the location of the flight device from the flight device via the wireless network, a function of receiving an actual image taken by the flight device via the wireless network, and the flight device. A function to set a virtual viewpoint at a position different from the existing position of the flight device and a flight image showing a flight area of the flight device when the flight device is viewed from the virtual viewpoint are displayed on the control device. The function of displaying the flight image on the unit and the function of stopping the display of the flight image when a predetermined condition is met are realized.

According to the present invention, it is possible to improve the operability of a flight device that is out of the user's field of view.

It is a figure which shows typically the whole structure of the communication system which concerns on embodiment. It is a figure which shows typically the functional structure of the control device which concerns on embodiment. It is a figure which shows an example of the operation screen which is displayed on the display part of the control device which concerns on embodiment. It is a figure which shows another example of the operation screen which is displayed on the display part of the control device which concerns on embodiment. It is a figure which shows an example of the display screen which displayed the flight path displayed on the display part of the control device which concerns on embodiment. It is a figure which shows an example of the radio wave intensity map displayed on the display part. It is a figure which shows typically the display screen which is displayed on the display part when the radio wave strength around the flight apparatus becomes less than a predetermined threshold value. It is a flowchart for demonstrating the flow of information processing executed by the control device which concerns on embodiment.

<Outline of the embodiment>
FIG. 1 is a diagram schematically showing an overall configuration of a communication system S according to an embodiment. The communication system S includes a control device 1 and a flight device 2. The control device 1 and the flight device 2 are paired with each other and can communicate with each other via the wireless network N. The control device 1 is a device for controlling the flight device 2 via the wireless network N, and the user can control the flight device 2 by using the control device 1. Therefore, the flight device 2 includes a receiving unit that receives an operation command from the control device 1 and a flight control unit that controls the flight of the flight device 2 based on the operation command.

The control device 1 is a communication terminal such as a smartphone and a tablet. The control device 1 may also be a dedicated radio for maneuvering the flight device 2. When the user operates the control device 1, the control device 1 transmits control information according to the operation content to the flight device 2. The flight device 2 transmits flight information necessary for maneuvering, such as flight position and flight speed, to the flight device 1. Further, the flight device 2 has a camera, and the image data captured by the camera and generated is transmitted to the control device 1.

The control device 1 and the flight device 2 can transmit and receive control information, flight information, and image data using a plurality of wireless communication lines. The control device 1 and the flight device 2 communicate using a first wireless communication line W1 that can transmit and receive information without going through a base station 3 of a mobile phone network. The control device 1 and the flight device 2 can also communicate using the second wireless communication line W2 capable of transmitting and receiving information via the base station 3 of the mobile phone network. In particular, when the control device 1 and the flight device 2 are far apart from each other, the control device 1 and the flight device 2 communicate with each other using the second radio communication line W2. The first wireless communication line W1 is, for example, a Wi-Fi (registered trademark) line. The second wireless communication line W2 is a 4G (4th Generation) line including, for example, LTE (Long Term Evolution).

Here, when the user and the flight device 2 are far apart from each other, it is difficult for the user to maneuver while directly visually observing the flight device 2. Even if the user and the flight device 2 are not far apart, if there is a shield such as a structure between the user and the flight device 2, the user cannot maneuver while visually observing the flight device 2. Therefore, the control device 1 is provided with a display unit for displaying the image data received from the flight device 2. Even if the user cannot directly see the flight device 2, if the image data captured by the flight device 2 can be seen, it is possible to avoid colliding the flight device 2 with a structure or the like.

Here, the flight device 2 takes an image with a camera provided in the own aircraft. Therefore, in the image taken by the flight device 2, the flight device 2 itself is often not included in the subject, or even if it is included, it is only a part thereof. Therefore, even when looking at the image data captured by the flight device 2, it may be difficult for the user to grasp the positional relationship between the flight device 2 and other objects, a sense of distance, and the like, and it may be difficult to operate the flight device.

Therefore, the flight device 2 according to the embodiment has a position acquisition unit that acquires position information indicating the existence position of the flight device 2, and position information (latitude, longitude, altitude) of the flight device 2 via the wireless network N. It also has a transmitter that transmits various information such as speed, direction, attitude (tilt), wind speed, communication quality, and information on the remaining battery level to the control device 1. The control device 1 receives position information from the flight device 2, and displays image data indicating an area in which the flight device 2 is in flight when the flight device 2 is viewed from a position different from the existing position of the flight device 2. To display. Therefore, the image data displayed on the display unit is an image when the flight device 2 is viewed from behind or from the side, and is an image including the flight device 2. This makes it easy for the user to grasp the positional relationship between the flight device 2 and other objects, a sense of distance, and the like. As a result, the control device 1 can improve the operability of the flight device 2 which is out of the user's field of view.
Hereinafter, the control device 1 according to the embodiment will be described in more detail.

<Functional configuration of control device 1>
FIG. 2 is a diagram schematically showing a functional configuration of the control device 1 according to the embodiment. The control device 1 includes a communication unit 10, a storage unit 11, a display unit 12, an operation unit 13, a quality information acquisition unit 14, and a control unit 15.

The communication unit 10 is a communication interface for the control device 1 to send and receive information to and from the flight device 2. The communication unit 10 includes a transmission unit 100 and a reception unit 101. Further, the receiving unit 101 includes a position information receiving unit 1010, an actual video receiving unit 1011 and a quality information receiving unit 1012. The communication unit 10 is realized by a known communication module such as a Wi-Fi module or a mobile phone communication module.

The storage unit 11 is a ROM (Read Only Memory) for storing an OS (Operating System), an application program, etc., a RAM (Random Access Memory) for a work area of the control device 1, and an SSD (Solid State) for storing various other information. It is a large-capacity storage device such as Drive).

The display unit 12 is an interface for visually notifying the user of the information generated by the control device 1. The display unit 12 may be a touch screen in which a display area and a touch panel are integrated, and in this case, the display unit 12 also functions as an input interface.

The operation unit 13 is a device for performing an operation for the user to operate the flight device 2. The operation unit 13 includes, for example, an operation stick or a volume for controlling the flight direction and the flight speed. When the display unit 12 is a touch screen, the operation unit 13 may be fully or partially mounted using a GUI (Graphical User Interface) displayed on the display unit 12.

The quality information acquisition unit 14 acquires quality information indicating the communication quality of the wireless network N at the place where the control device 1 exists. Here, the "communication quality of the wireless network N" includes not only the radio wave strength of the wireless network N but also the radio wave interference degree. For example, when the radio wave strength of the wireless network N is weakened, the communication quality is deteriorated. Further, when the radio wave interference degree of the wireless network N is large, the communication quality of the wireless network N may deteriorate even when the radio wave strength of the wireless network N is strong.

The control unit 15 is a processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the control device 1, and by executing a program stored in the storage unit 11, the operation reception unit 151 and the viewpoint setting unit It functions as 152, a display control unit 153, a route reception unit 154, and a notification unit 155.

The operation reception unit 151 converts the input of the operation unit 13 into an operation command of the flight device 2. The transmission unit 100 transmits an operation command indicating the flight direction of the flight device 2 to the flight device 2 via the wireless network N. The position information receiving unit 1010 receives the position information indicating the existence position of the flight device 2 from the flight device 2 via the wireless network N.

The viewpoint setting unit 152 sets a virtual viewpoint at a position different from the existing position of the flight device 2. The display control unit 153 causes the display unit 12 of the control device 1 to display a flight image showing the area in which the flight device 2 is in flight when the flight device 2 is viewed from a virtual viewpoint set by the viewpoint setting unit 152.

Here, the "virtual viewpoint" is a viewpoint that does not necessarily exist, unlike the image pickup unit such as a camera possessed by the flight device 2. The viewpoint setting unit 152 is set, for example, 10 meters behind the traveling direction of the flight device 2 and 2 m above the relative altitude. The user can change the place where the virtual viewpoint is set by the viewpoint setting unit 152 by operating the operation stick of the control device 1. Further, the viewpoint setting unit 152 may change the position where the virtual viewpoint is set according to the flight speed of the flight device 2. Specifically, when the flight speed of the flight device 2 is high, the viewpoint setting unit 152 may set a virtual viewpoint at a position farther from the flight device 2 than when the flight speed is slow.

The display control unit 153 is, for example, a bird's-eye view when the flight device 2 is viewed from a set virtual viewpoint based on image data created in advance by a technique such as CG (Computer Graphics) based on known topographical information. An image is generated and displayed on the display unit 12. If the control device 1 captures and holds an image of the airspace in which the flight device 2 flies in advance, the display control unit 153 may read the image and display it on the display unit 12. Further, when the control device 1 and the flight device 2 store the same three-dimensional terrain information, the display control unit 153 takes a bird's-eye view based on the position information and the terrain information received from the flight device 2. An image may be generated and displayed on the display unit 12.

The actual image receiving unit 1011 receives the actual image captured by the flight device 2 from the flight device 2 via the wireless network N. The display control unit 153 causes the display unit 12 to display the actual image captured by the flight device 2. As a result, the user can confirm not only the actual image captured by the flight device 2 but also the bird's-eye view image when the flight device 2 is viewed from a distant position on the display unit 12. When the flight device 2 approaches another object, the user can observe an image in which both the flight device 2 and the other object are shown. Therefore, the user can intuitively grasp the positional relationship between the flight device 2 and another object and the sense of distance. As a result, even if the flight device 2 is out of the user's field of view, the operability of the flight device 2 is improved.

When the flight device 2 includes a range-finding sensor such as an infrared sensor (not shown), the flight device 2 can measure the distance to an object around the aircraft. When the distance between the aircraft and surrounding objects is within a predetermined distance, the flight device 2 notifies the control device 1 to that effect. In the control device 1, when the flight device 2 approaches a surrounding object within a predetermined distance, the notification unit 155 of the control device 1 causes a speaker (not shown) included in the control device 1 to play a warning sound or vibrate the vibrator. The user may be notified that the flight device 2 has approached another object by causing the flight device 2 to approach another object.

Further, at least one of the control device 1 and the flight device 2 has the flight device 2 approaching another object based on the stored three-dimensional topographical information and the position information of the flight device 2. It may be determined whether or not it is present. When the distance between the flight device 2 and another object is within a predetermined distance, the notification unit 155 of the control device 1 causes a speaker (not shown) included in the control device 1 to play a warning sound or vibrate the vibrator. By doing so, the user may be notified that the flight device 2 has approached another object.

Therefore, the "predetermined distance" is the "approach notification determination threshold distance" referred to by the notification unit 155 in order to determine whether or not to notify the user that the flight device 2 has approached another object. .. The approach notification determination threshold distance is stored in the storage unit 11. The specific value of the threshold distance for approach notification determination is a distance at which the flight device 2 can avoid other objects by taking an evasive action, and may be determined by an experiment in consideration of the flight performance of the flight device 2. However, for example, it is 20 meters. When the flight device 2 determines the approach to another object, the flight device 2 may change the approach notification determination threshold distance in consideration of the wind speed at that time.

As described above, the bird's-eye view image displayed on the display unit 12 by the display control unit 153 is CG. For this reason, it is not always possible to guarantee that the bird's-eye view image can faithfully reproduce the actual situation. Therefore, when the display control unit 153 displays the bird's-eye view image on the display unit 12, the display unit 153 may always display the actual image on the display unit 12. As a result, the user can always check the actual situation around the flight device 2.

FIG. 3 is a diagram schematically showing an example of an operation screen displayed on the display unit 12 of the control device 1 according to the embodiment. In the example shown in FIG. 3, the operation screen includes a quality information indicator 301, a battery level indicator 302, an information panel 303, a bird's-eye view display area 304, a flight device icon 305, and a live image display area 306.

The quality information indicator 301 is an icon indicating the quality information acquired by the quality information acquisition unit 14, that is, the communication quality of the wireless network N in the place where the control device 1 exists. The battery level indicator 302 is an icon indicating the remaining battery level of the control device 1.

The information panel 303 shows information about the flight of the flight device 2. The information panel 303 displays the distance between the control device 1 and the flight device 2, the direction of travel of the flight device 2, the flight speed of the flight device 2, and the altitude of the flight device 2.

The bird's-eye view display area 304 is a display area for a bird's-eye view image displayed on the display unit 12 by the display control unit 153. The bird's-eye view display area 304 in FIG. 3 shows an example of a bird's-eye view image displayed on the display unit 12 by the display control unit 153 when the viewpoint setting unit 152 sets a virtual viewpoint behind the flight device 2. Therefore, the bird's-eye view display area 304 includes the flight device icon 305 indicating the flight device 2. Further, the live image display area 306 is a display area of the actual image captured by the flight device 2.

As shown in FIG. 3, the flight device icon 305 is shown in the center of the bird's-eye view display area 304. The display control unit 153 changes the display area of the bird's-eye view display area 304 so that the flight device icon 305 becomes the center of the bird's-eye view display area 304 in accordance with the movement of the flight device 2. Further, the display control unit 153 may change the scale of the bird's-eye view display area 304 according to the flight speed of the flight device 2. Specifically, the display control unit 153 may display the bird's-eye view display area 304 showing a wide range of images on the display unit 12 when the flight speed of the flight device 2 is high, as compared with the case where the flight speed is slow.

The camera included in the flight device 2 according to the embodiment captures a landscape in front of the flight device 2 in the traveling direction. Since the bird's-eye view display area 304 in FIG. 3 is a bird's-eye view image when the viewpoint setting unit 152 sets a virtual viewpoint behind the flight device 2, the direction from the virtual viewpoint toward the flight device icon 305 is the flight device 2. It becomes the traveling direction of. Therefore, as shown in FIG. 3, when the viewpoint setting unit 152 sets a virtual viewpoint behind the flight device 2, the image displayed in the bird's-eye view display area 304 and the image displayed in the live image display area 306 are different. It will be similar.

By the way, the user executes the operation of the flight device 2 by tilting the operation stick (not shown) provided in the control device 1 or operating the operation GUI (not shown) displayed on the display unit 12. .. For example, when the user tilts the operation stick to the "right", the flight device 2 turns to the "right".

Here, "right" and "left" are concepts of relative direction, and some standard is required to determine the absolute direction. Therefore, in the control device 1 according to the embodiment, the operation reception unit 151 determines an operation command indicating the flight direction of the flight device 2 with reference to the virtual viewpoint set by the viewpoint setting unit 152. The transmission unit 100 transmits the operation command determined by the operation reception unit 151 to the flight device 2. For example, when a virtual viewpoint is set behind the flight device 2, when the user tilts the stick "right", the flight device 2 turns to the right with respect to the traveling direction. In this case, for example, assuming that the flight device 2 is moving toward the user, when the user tilts the stick "right", the flight device 2 appears to turn to the left when viewed from the user.

Therefore, when the virtual viewpoint is set behind the flight device 2 with respect to the flight direction of the flight device 2, the system direction of the stick by the user and the movement direction of the flight device icon 305 in the bird's-eye view display area 304 match. Will be done. Therefore, the viewpoint setting unit 152 may set a virtual viewpoint behind the flight device 2 with respect to the flight direction of the flight device 2. As a result, the operation by the user and the moving direction of the flight device 2 match, so that the user can intuitively operate the flight device 2.

When the flight device 2 is far from the control device 1, the viewpoint setting unit 152 sets a virtual viewpoint between the control device 1 and the flight device 2. More specifically, in the viewpoint setting unit 152, the foot of the perpendicular line of the virtual viewpoint in the straight line passing through the existence position of the control device 1 and the existence position of the flight device 2 is the existence position of the control device 1 and the flight device 2. Set a virtual viewpoint so that it exists between the existing position.

FIG. 4 is a diagram schematically showing another example of the operation screen displayed on the display unit 12 of the control device 1 according to the embodiment, and is a diagram showing the positional relationship between the virtual viewpoint and the flight device 2. be. In the example shown in FIG. 4, the operation screen has a quality information indicator 301, a battery level indicator 302, an information panel 303, a flight device icon 305, a positional relationship display area 307, a virtual viewpoint icon 308, a user icon 309, and a route map 310. include.

The positional relationship display area 307 is an area showing the positional relationship between the control device 1, the flight device 2, and the virtual viewpoint. In the positional relationship display area 307, a flight device icon 305 indicating the position of the flight device 2, a virtual viewpoint icon 308 indicating a virtual viewpoint, and a user icon 309 indicating the position of the user (that is, the control device 1) are arranged. .. Further, in the positional relationship display area 307, a three-dimensional coordinate system is set with an axis indicating the east-west direction, an axis indicating the north-south direction, and an axis indicating the height direction with the flight device icon 305 as the origin.

In the example shown in FIG. 4, the user is located in the southwest of the flight device 2 at a position lower than the flight device 2, and the virtual viewpoint is set to the same height as the flight device 2 in the south of the flight device 2. It is shown that. In this case, when the user tilts the stick "to the right", the flight device 2 turns to the right side, that is, to the east direction when viewed from the virtual viewpoint icon 308. By confirming the positional relationship display area 307, the user can grasp the positional relationship between the flight device 2 and the virtual viewpoint.

Since the flight device 2 generally flies over a distance from the user, the distance between the control device 1 and the flight device 2 is sufficiently longer than the distance between the virtual viewpoint set by the viewpoint setting unit 152 and the flight device 2. I can say. Therefore, the positional relationship display area 307 is information indicating the direction in which the user (that is, the control device 1) exists when the flight device 2 is the starting point, that is, the user exists when the virtual viewpoint is the starting point. It can also be said to be information indicating the direction of flight. By displaying the positional relationship display area 307 on the display unit 12 by the display control unit 153, the user can easily grasp the positional relationship between himself / herself and the virtual viewpoint.

The display control unit 153 also causes the display unit 12 to display a route map 310, which is a two-dimensional map including the start point of the flight device 2 and the arrival target point of the flight device 2. In the example of the route map 310 shown in FIG. 4, the start point of the flight device 2 is indicated by a black circle, and the destination point of the flight device 2 is indicated by a white star shape. Further, the current position of the flight device 2 in the route map 310 is shown by a black triangle, and the route traced from the start point of the flight device 2 to the current position of the flight device 2 is also shown.

The route map 310 can be said to be information indicating the flight direction of the flight device 2 when the control device 1 is the starting point. By displaying the route map 310 on the display unit 12, the display control unit 153 allows the user to display the start point of the flight device 2 (that is, the point where the user exists), the current position of the flight device 2, and the arrival of the flight device 2. It is possible to grasp the relationship with the target point.

Returning to the description of FIG. The route reception unit 154 receives input from the user of the control device 1 at one or a plurality of points existing on the route on which the flight device 2 is flown. Specifically, the user operates the operation unit 13 while looking at the map (not shown) displayed on the display unit 12 to input the route of the flight device 2. The route reception unit 154 converts the operation of the operation unit 13 by the user into route information and outputs it to the display control unit 153. The display control unit 153 causes the display unit 12 to display the area of the flight image corresponding to one or a plurality of points as a mode different from the other areas.

The communication system S according to the embodiment further includes a server (not shown) connected to the control device 1 by the second wireless communication line W2, and the server stores one or a plurality of route information. If so, the control device 1 may receive route information from the server. By doing so, the control device 1 does not need to increase the storage capacity of the storage unit 11 for storing a large amount of route information.

FIG. 5 is a diagram schematically showing an example of a display screen on which a flight path displayed on the display unit 12 of the control device 1 according to the embodiment is displayed. An example of a display screen displaying a flight path shown in FIG. 5 includes a quality information indicator 301, a battery level indicator 302, an information panel 303, a bird's-eye view display area 304, a flight device icon 305, and a route indicator 311.

In FIG. 5, the arrow indicated by reference numeral 311 in the bird's-eye view display area 304 is a route indicator 311 indicating a flight route input by the user. Although all the arrows are not designated to prevent complication, the arrow having the same shape as the arrow with the reference numeral 311 in FIG. 5 is the route indicator 311. As shown in FIG. 5, the display control unit 153 distinguishes the display unit 12 from other flight images by displaying the route indicator 311 which is a three-dimensional arrow at the location corresponding to the flight path input by the user. It is displayed in. By looking at the flight device icon 305 and the route indicator 311, the user can at a glance grasp whether or not the flight device 2 is flying along the flight path set.

Here, the notification unit 155 notifies the user of the control device 1 that the flight device 2 has deviated from the path through which the flight device 2 is flown. When the flight device 2 deviates from the flight path, the notification unit 155 notifies the user by displaying a message indicating that fact on the display unit 12 via the display control unit 153. In place of or in addition to this, the notification unit 155 causes the flight device 2 to deviate from the flight path by causing a speaker (not shown) provided in the control device 1 to play a warning sound or vibrate the vibrator. May be notified to the user. As a result, the user can quickly recognize that the flight device 2 has deviated from the set flight path.

Further, when the flight device 2 deviates from the flight path of the flight device 2, the viewpoint setting unit 152 includes the route indicator 311 indicating the flight path and the flight device icon 305 indicating the flight device 2 in the flight image. Set a virtual viewpoint. As a result, the bird's-eye view display area 304 displayed on the display unit 12 by the display control unit 153 includes the route indicator 311 indicating the flight path and the flight device icon 305 indicating the flight device 2. By looking at the bird's-eye view display area 304, the user can grasp the direction in which the flight device 2 should be moved in order to return the flight device 2 to the flight path.

The above is based on the premise that the control device 1 can receive information such as the position information of the flight device 2 and the image captured by the flight device 2 from the flight device 2 via the wireless network N. Here, if the control device 1 is far away from the flight device 2, communication using Wi-Fi becomes difficult, so that the control device 1 can use the position information of the flight device 2 via the second wireless communication line W2. Information such as an image captured by the flight device 2 is received from the flight device 2. Therefore, the communication quality of the second radio communication line W2 in the area where the flight device 2 is flying is important when exchanging information between the control device 1 and the flight device 2.

Therefore, the quality information receiving unit 1012 receives quality information indicating the communication quality of the second wireless communication line W2 at the place where the flight device 2 exists from the flight device 2 via the second wireless communication line W2. The display control unit 153 superimposes the quality information of the second wireless communication line W2 acquired from the flight device 2 on the flight image and displays it on the display unit 12.

FIG. 6 is a diagram showing an example of the radio wave intensity map 312 displayed on the display unit 12. In FIG. 6, distribution information showing the distribution of the communication quality of the second wireless communication line W2 for each area is displayed as a radio wave intensity map 312. As shown in the hatching label 313, the radio wave intensity map 312 divides the regions of the second wireless communication line having different radio wave intensities with different hatches.

The radio wave intensity map 312 includes three-dimensional distribution information of communication quality. Therefore, the display control unit 153 displays the radio field intensity map 312 in association with the altitude of the flight device 2 from the ground. On this display screen, a radio field intensity map 312 at an altitude of around 35 m is shown. Each time the user selects the up icon 314, the display control unit 153 displays a radio field intensity map whose altitude is increased by 5 m. Further, each time the user selects the down icon 315, the display control unit 153 displays a radio field intensity map with the altitude reduced by 5 m.

The display control unit 153 displays the flight device icon 305 in the center of the radio field intensity map 312. This display screen shows an example in which the flight device 2 is located at an altitude of 34.2 m from the ground. In this way, the display control unit 153 superimposes the place where the flight device 2 exists on the radio wave intensity map 312 including the place where the flight device 2 exists, and causes the display unit 12 to display the place. As a result, the user can select an area with good communication quality and fly the flight device 2. Instead of displaying the quality information received from the flight device 2 on the display unit 12, the display control unit 153 reads out the radio wave intensity map 312 stored in advance in the storage unit 11 and displays it on the display unit 12. You may.

When the radio wave intensity map 312 of the place where the flight device 2 exists cannot be acquired, the display control unit 153 may display the area map on the display unit 12 instead of the radio wave intensity map 312. Here, the "area map" is a communicable area map showing whether or not communication of the second wireless communication line W2 is possible, and is a known map created from the communication status of an existing smartphone.

Here, when the communication quality of the second wireless communication line W2 included in the quality information becomes lower than the predetermined quality in the place where the flight device 2 exists, the display control unit 153, for example, the radio wave of the second wireless communication line W2. When the intensity is equal to or less than a predetermined threshold, the display unit 12 displays information indicating the direction in which there is a region where the radio wave intensity is larger than the predetermined threshold (that is, a region where the communication quality is improved from the predetermined quality). To display. The "predetermined threshold value" is the radio wave strength referred to by the display control unit 153 in order to determine whether or not the communication quality that can withstand the transmission of the operation information from the control device 1 to the flight device 2 can be maintained. It is a discrimination standard threshold value of. The specific value of the discrimination reference threshold value may be determined by an experiment in consideration of the performance of the antenna provided in the control device 1 and the flight device 2.

FIG. 7 is a diagram schematically showing a display screen displayed on the display unit 12 when the radio wave intensity around the flight device 2 becomes equal to or less than a predetermined threshold value. As shown in FIG. 7, when the radio wave intensity around the flight device 2 becomes equal to or less than a predetermined threshold value, the display unit 12 displays the radio wave intensity decrease message 316. As a result, the user can move the flight device 2 to an environment with good communication quality before the communication with the flight device 2 is interrupted.

Here, it is useful for the user if it is possible to know in which direction the flight device 2 should be moved to improve the communication environment. Therefore, the display control unit 153 causes the display unit 12 to display the direction in which the communication quality of the second wireless communication line W2 improves when the flight device 2 moves.

In FIG. 7, the direction indicator 317 indicates the direction in which the communication quality of the wireless network N improves when the flight device 2 moves, and the user icon 309 indicates the direction in which the user exists when the position of the flight device 2 is used as a reference. Shows. In the example shown in FIG. 7, it is shown that the communication quality of the wireless network N is improved if the user operates the flight device 2 so as to raise it in the northwest direction.

The direction indicator 317 shown in FIG. 7 indicates the direction in which the flight device 2 can actually move. The display control unit 153 does not display the direction indicator 317 on the display unit 12 in the direction in which the flight device 2 cannot move due to an obstacle or the like, even if the communication quality of the second wireless communication line W2 is improved. Or, display an icon (not shown) indicating that it cannot be moved.

Here, when the radio wave intensity around the flight device 2 becomes equal to or less than a predetermined threshold value, the flight device 2 is stopped in the air (hovering) on the spot until new operation information is transmitted from the control device 1. You may do it. This makes it possible to prevent the flight device 2 from moving to a region where the radio wave strength is further low.

<Processing flow of information processing executed by the control device 1 according to the embodiment>
FIG. 8 is a flowchart for explaining the flow of information processing executed by the control device 1 according to the embodiment. The process in this flowchart starts, for example, when the power of the control device 1 is turned on.

The position information receiving unit 1010 receives the position information indicating the existence position of the flight device 2 from the flight device 2 via the wireless network N (S2). The viewpoint setting unit 152 sets a virtual viewpoint at a position different from the existing position of the flight device 2 (S4). The display control unit 153 causes the display unit 12 of the control device 1 to display a flight image showing the area in which the flight device 2 is in flight when the flight device 2 is viewed from a virtual viewpoint set by the viewpoint setting unit 152. S6). The display control unit 153 causes the display unit 12 to display the actual image captured by the flight device 2 acquired via the wireless network N (S8). When the display control unit 153 displays the actual image on the display unit 12, the process in this flowchart ends.

<Effects of the control device 1 according to the embodiment>
As described above, according to the control device 1 according to the embodiment, it is possible to improve the operability of the flight device 2 which is out of the user's field of view.
In particular, since a virtual viewpoint is set at a position different from that of the flight device 2 and an image showing the flight area of the flight device 2 when the flight device 2 is viewed from that viewpoint is displayed on the display unit 12, the user can fly. The positional relationship and the distance relationship between the device 2 and other objects can be easily grasped.

Further, when the user operates the operation unit of the control device 1, the flight direction of the flight device 2 is changed based on the virtual viewpoint, so that the user intuitively operates the flight device 2 while looking at the display unit 12. be able to. Further, since the actual image captured by the flight device 2 is also displayed in real time on the display unit 12 of the control device 1, the user can confirm the current situation around the flight device 2.

Since the control device 1 displays the flight path to which the flight device 2 is scheduled to fly, which has been input in advance, on the display unit 12, the user can operate the flight device 2 while confirming the flight path. When the flight device 2 deviates from this flight path, the control device 1 displays a flight image including the flight path and the flight device 2 on the display unit 12. As a result, the user can easily return the flight device 2 to the flight path.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. In particular, the specific embodiments of the distribution / integration of the devices are not limited to those shown above, and all or a part thereof may be in arbitrary units according to various additions or the like, or according to the functional load. It can be functionally or physically distributed and integrated. An example of such a modification will be described below.

<First modification>
In the above, the case where the flight device 2 captures the surrounding situation by using an image pickup device such as a camera has been described. Here, the flight device 2 may include an image recognition unit (not shown) that recognizes the subject of the image captured by the image pickup device. When the image recognition unit recognizes another flying object in the image captured by the image pickup device, the image recognition unit may notify the control device 1 possessed by the user and hover on the spot. When the display control unit 153 of the control device 1 receives a notification from the flight device 2 that another flying object has been recognized, the display unit 12 may display only the actual image. As a result, it is possible to prevent the flight device 2 from colliding with another flying object.

The image recognition unit can be realized by using a detector generated by using a known machine learning technique such as a neural network, SVM (Support Vector Machine), or boosting.

<Second modification>
In the above, the route receiving unit 154 has described the case where the user of the control device 1 accepts the input of one or a plurality of points existing on the path on which the flight device 2 is flown. Alternatively or additionally, the route reception unit 154 may download a three-dimensional map including a plurality of recommended routes from an external server (not shown). The display control unit 153 may receive a switching instruction from the user via the operation unit 13 and switch the recommended route to be displayed on the display unit 12. As a result, the user can select the route to actually fly the flight device 2 from the plurality of recommended routes.

<Third modification example>
In the above, the case where the operation reception unit 151 determines the operation command indicating the flight direction of the flight device 2 with reference to the virtual viewpoint set by the viewpoint setting unit 152 has been described. This is useful when the bird's-eye view display area 304 is displayed on the display unit 12. However, when the bird's-eye view display area 304 is not displayed on the display unit 12 and the positional relationship display area 307 or the route map 310 is displayed instead, the operation reception unit 151 may display the positional relationship display area 307 or the route. It is preferable to determine an operation command indicating the flight direction of the flight device 2 based on the positional relationship between the control device 1 and the flight device 2 on the map 310 (that is, the positional relationship on the map displayed on the display unit 12).

Therefore, the control device 1 according to the third modification includes a changeover switch realized by a GUI or a physical switch (not shown). By operating the changeover switch, the user determines an operation command indicating the flight direction of the flight device 2 based on the virtual viewpoint set by the viewpoint setting unit 152, or displays the operation command on the display unit 12. It is possible to switch whether to determine the operation command indicating the flight direction based on the positional relationship on the map. As a result, the control device 1 according to the third modification can provide the user with an operation user interface suitable for the display content of the display unit 12.

In addition, a new modification generated by arbitrarily combining the above modifications is also included in the embodiment of the present invention. The effect of the modified example newly generated by the combination has the effect of each original modified example together.

1 ... Control device 2 ... Flight device 3 ... Base station 10 ... Communication unit 11 ... Storage unit 12 ... Display unit 13 ... Operation unit 14 ... Quality information acquisition unit 15 ... Control unit 100 ... Transmission unit 101 ... Reception unit 151 ... Operation reception unit 152 ... Viewpoint setting unit 153 ... Display control unit 154 ... Route reception unit 155 ... Notification unit 1010 ... Position information receiving unit 1011 ... Actual video receiving unit 1012 ... Quality information receiving unit N ... Wireless network S ... Communication system

Claims (5)

A maneuvering device for maneuvering a flight device over a wireless network.
A transmission unit that transmits an operation command indicating the flight direction of the flight device to the flight device via the wireless network.
A position information receiving unit that receives position information indicating the existence position of the flight device from the flight device via the wireless network.
A real video receiver that receives real images taken by the flight device via a wireless network,
A viewpoint setting unit that sets a virtual viewpoint at a position different from the existing position of the flight device,
A display control unit that displays a flight image showing an area in which the flight device is in flight when the flight device is viewed from the virtual viewpoint on the display unit of the control device.
Equipped with
The display control unit stops displaying the flight image when a predetermined condition is met.
Maneuvering device. The display control unit displays the actual image captured by the flight device and the flight image on the display unit, and stops displaying the flight image when the predetermined conditions are met.
The control device according to claim 1. The display control unit stops displaying the flight image when the predetermined condition for the subject in the actual image is satisfied.
The control device according to claim 1 or 2. The processor of the maneuver for maneuvering the flight device over the wireless network
A step of transmitting an operation command indicating the flight direction of the flight device to the flight device via the wireless network, and a step of transmitting the operation command to the flight device.
A step of receiving position information indicating the location of the flight device from the flight device via the wireless network, and
The step of receiving the actual image taken by the flight device via the wireless network,
A step of setting a virtual viewpoint at a position different from the position where the flight device exists,
A step of displaying a flight image showing an area in which the flight device is in flight when the flight device is viewed from the virtual viewpoint on the display unit of the control device.
When the predetermined conditions are met, the step of stopping the display of the flight image and
Information processing method to execute. To the computer equipped with the control device for maneuvering the flight device over the wireless network,
A function of transmitting an operation command indicating the flight direction of the flight device to the flight device via the wireless network, and
A function of receiving position information indicating the location of the flight device from the flight device via the wireless network, and
The function of receiving the actual image taken by the flight device via the wireless network,
A function to set a virtual viewpoint at a position different from the existing position of the flight device,
A function of displaying a flight image showing an area in which the flight device is in flight when the flight device is viewed from the virtual viewpoint on the display unit of the control device.
A function to stop the display of the flight image when certain conditions are met, and
A program that realizes.

Images