JP2018110352A - Steering device, information processing method, program, and flight system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve operationality of a flight device that is deviated from user's field of view.SOLUTION: In a steering device 1 for steering a flight device via a wireless network, a transmission unit 100 transmits an operation instruction, indicating the flight direction of the flight device, to the flight device via the wireless network. A positional information reception unit 1010 receives positional information, indicating the current position of the flight device, from the flight device via the wireless network. A view point setting unit 152 sets a virtual view point at a position different from the existence position of the flight device. A display control unit 153 controls the display 12 of the steering device 1 to display a flight image indicating the region of the flight device during flight, when viewing the flight device from the virtual view point.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a control device for controlling a flight device, an information processing method executed by the control device, a program executed by the control device, and a flight system including the flight device and the control device.

  In recent years, drones, which are flying devices that can be remotely controlled, have become widespread. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique in which the surroundings are photographed by a camera mounted on a flying device, and data of the photographed image is transmitted to a terminal used by a user to operate the flying device.

Japanese Patent Laid-Open No. 2006-174360

  When the flying device is flying in a place that is out of the user's field of view, the user operates the flying device relying on an image taken by a camera mounted on the flying device. However, in many cases, an image captured by a camera mounted on a flying device does not include the flying device itself as a subject or, if included, only a part thereof. For this reason, even when looking at an image taken by a camera mounted on a flying device, it is difficult for the user to grasp the positional relationship between the flying device and other objects, a sense of distance, etc., and it is difficult to operate the flying device. was there.

  Therefore, the present invention has been made in view of these points, and an object thereof is to provide a technique for improving the operability of a flying device that is out of the user's field of view.

  A first aspect of the present invention is a control device for controlling a flying device via a wireless network. The device includes a transmission unit that transmits an operation command indicating a flight direction of the flying device to the flying device via the wireless network, and position information that indicates the location of the flying device via the wireless network. The position information receiving unit that receives from the flying device, the viewpoint setting unit that sets a virtual viewpoint at a position different from the position where the flying device exists, and the flight when the flying device is viewed from the virtual viewpoint A display control unit that displays a flight image indicating a region in which the device is in flight on the display unit of the control device.

  The transmission unit may transmit the operation command indicating a flight direction of the flying device determined based on the virtual viewpoint.

  The control device may further include an actual video receiving unit that receives an actual image captured by the flying device from the flying device via the wireless network, and the display control unit is captured by the flying device. An actual image may be displayed on the display unit.

  The control device may further include a quality information receiving unit that receives quality information indicating communication quality of the wireless network in a place where the flying device exists via the wireless network, and the display control unit includes: The quality information may be superimposed on the flight image and displayed on the display unit.

  When the communication quality of the wireless network included in the quality information is lower than a predetermined quality in the place where the flying device exists, the display control unit has an area where the communication quality is improved from the predetermined quality. Information indicating the direction to perform may be displayed on the display unit.

  The control device may further include a route reception unit that receives an input of one or a plurality of points existing on a route on which the flying device is caused to fly from a user of the control device. Or you may display the area | region of the said flight image corresponding to a some point on the said display part as a different aspect from another area | region.

  The piloting device may further include a notification unit that notifies a user of the piloting device that the flying device has deviated from a route for flying the flying device.

  The viewpoint setting unit may set the virtual viewpoint so that the route and the flying device are included in the flight image when the flying device deviates from a route for flying the flying device.

  The display control unit may cause the display unit to display information indicating a flight direction of the flying device when the control device is a starting point.

  The display control unit may cause the display unit to display a direction in which the control device exists when the virtual viewpoint is set as a starting point.

  The viewpoint setting unit may set the virtual viewpoint behind the flying device with respect to the flight direction of the flying device.

  The second aspect of the present invention is an information processing method. In this method, a processor included in a control device for controlling a flying device via a wireless network transmits an operation command indicating a flight direction of the flying device to the flying device via the wireless network; Receiving position information indicating the position of the flying device via the wireless network from the flying device; setting a virtual viewpoint at a position different from the position of the flying device; and the virtual Displaying a flight image showing a region in which the flying device is flying when the flying device is viewed from the viewpoint of the control device on the display unit of the control device.

  The third aspect of the present invention is a program. The program includes a function of transmitting an operation command indicating a flight direction of the flying device to the flying device via the wireless network to a computer for maneuvering the flying device via the wireless network, and the wireless network. Via the function of receiving position information indicating the position of the flying device from the flying device, a function of setting a virtual viewpoint at a position different from the position of the flying device, and from the virtual viewpoint When the flying device is viewed, the flying device displays a flight image indicating a region in flight on the display unit.

  A 4th aspect of this invention is a flight system provided with a flight apparatus and the control apparatus for operating the said flight apparatus via a wireless network. In this system, the control device includes a transmission unit that transmits an operation command indicating a flight direction of the flying device to the flying device via the wireless network, and a location of the flying device via the wireless network. A position information receiving unit that receives position information indicating the position of the flying device, a viewpoint setting unit that sets a virtual viewpoint at a position different from the position where the flying device exists, and the flying device is viewed from the virtual viewpoint. A display control unit that displays a flight image indicating a region in which the flying device is in flight on the display unit of the control device. The flight device acquires a reception unit that receives the operation command from the control device, a flight control unit that controls the flight of the flight device based on the operation command, and position information that indicates the location of the flight device. A position acquisition unit that transmits the position information to the control device via the wireless network.

  ADVANTAGE OF THE INVENTION According to this invention, the operativity of the flying apparatus which has remove | deviated from the user's visual field can be improved.

1 is a diagram schematically showing an overall configuration of a communication system according to an embodiment. It is a figure which shows typically the function structure of the control apparatus which concerns on embodiment. It is a figure which shows typically an example of the operation screen displayed on the display part of the control apparatus which concerns on embodiment. It is a figure which shows typically another example of the operation screen displayed on the display part of the control apparatus which concerns on embodiment. It is a figure which shows typically an example of the display screen on which the flight path displayed on the display part of the control apparatus which concerns on embodiment was displayed. It is a figure which shows an example of the electromagnetic wave intensity map displayed on a display part. It is a figure which shows typically the display screen displayed on a display part, when the electromagnetic field intensity around a flight apparatus becomes below a predetermined threshold value. It is a flowchart for demonstrating the flow of the information processing which the control apparatus which concerns on embodiment performs.

<Outline of the embodiment>
FIG. 1 is a diagram schematically illustrating an overall configuration of a communication system S according to an embodiment. The communication system S includes a control device 1 and a flight device 2. The control device 1 and the flight device 2 are paired with each other and can communicate with each other via the wireless network N. The control device 1 is a device for controlling the flying device 2 via the wireless network N, and the user can control the flying device 2 using the control device 1. Therefore, the flying device 2 includes a receiving unit that receives an operation command from the control device 1 and a flight control unit that controls the flight of the flying device 2 based on the operation command.

  The control device 1 is a communication terminal such as a smartphone or a tablet. The control device 1 may also be a dedicated radio for controlling the flying device 2. When the user operates the control device 1, the control device 1 transmits control information corresponding to the operation content to the flying device 2. The flying device 2 transmits flight information necessary for maneuvering such as the flight position and the flight speed to the maneuvering device 1. The flying device 2 has a camera, and transmits image data generated by imaging with the camera to the control device 1.

  The control device 1 and the flight device 2 can transmit and receive control information, flight information, and image data using a plurality of wireless communication lines. The control device 1 and the flight device 2 communicate using the first wireless communication line W1 that can transmit and receive information without going through the base station 3 of the mobile phone network. The control device 1 and the flight device 2 can also communicate using a second wireless communication line W2 that can transmit and receive information via the base station 3 of the mobile phone network. In particular, when the control device 1 and the flying device 2 are far away from each other, the control device 1 and the flying device 2 communicate using the second wireless communication line W2. The first wireless communication line W1 is, for example, a Wi-Fi (registered trademark) line. The second wireless communication line W2 is a 4G (4th Generation) line including, for example, LTE (Long Term Evolution).

  Here, when the user and the flying device 2 are far apart, it is difficult for the user to steer the flying device 2 while directly viewing it. Even if the user and the flying device 2 are not far apart, if there is a shield such as a structure between the user and the flying device 2, the user cannot steer the flying device 2 while viewing it. Therefore, the control device 1 includes a display unit for displaying the image data received from the flying device 2. Even if the user cannot see the flying device 2 directly, if the user can see the image data captured by the flying device 2, it is possible to avoid causing the flying device 2 to collide with a structure or the like.

  Here, the flying device 2 captures an image with a camera provided in the aircraft. For this reason, it is often the case that the flying device 2 itself is not included in the subject or only a part of the flying device 2 is included in the image captured by the flying device 2. Therefore, even if the image data captured by the flying device 2 is viewed, it may be difficult for the user to grasp the positional relationship between the flying device 2 and other objects, a sense of distance, and the like, and it may be difficult to operate the flying device.

Therefore, the flying device 2 according to the embodiment includes the position information (latitude, longitude, altitude) of the flying device 2 via the position acquisition unit that acquires position information indicating the presence position of the flying device 2 and the wireless network N. A transmission unit that transmits various information such as information on speed, direction, attitude (tilt), wind speed, communication quality, and remaining battery level to the control device 1 is also provided. The control device 1 receives position information from the flying device 2 and displays image data indicating an area in which the flying device 2 is flying when the flying device 2 is viewed from a position different from the position where the flying device 2 exists. To display. For this reason, the image data displayed on the display unit is an image when the flying device 2 is viewed from behind or from the side, and is an image including the flying device 2. This makes it easy for the user to grasp the positional relationship, distance feeling, etc. between the flying device 2 and other objects. As a result, the control device 1 can improve the operability of the flying device 2 that is out of the user's field of view.
Hereinafter, the control device 1 according to the embodiment will be described in more detail.

<Functional configuration of control device 1>
FIG. 2 is a diagram schematically illustrating a functional configuration of the control device 1 according to the embodiment. The control device 1 includes a communication unit 10, a storage unit 11, a display unit 12, an operation unit 13, a quality information acquisition unit 14, and a control unit 15.

  The communication unit 10 is a communication interface for the control device 1 to transmit / receive information to / from the flying device 2. The communication unit 10 includes a transmission unit 100 and a reception unit 101. The receiving unit 101 includes a position information receiving unit 1010, an actual video receiving unit 1011, and a quality information receiving unit 1012. The communication unit 10 is realized by a known communication module such as a Wi-Fi module or a mobile phone communication module.

  The storage unit 11 includes a ROM (Read Only Memory) that stores an OS (Operating System), application programs, and the like, a RAM (Random Access Memory) that is a work area of the control device 1, and an SSD (Solid State) that stores various other information. Drive).

  The display unit 12 is an interface for visually notifying the user of information generated by the control device 1. The display unit 12 may be a touch screen in which a display area and a touch panel are integrated. In this case, the display unit 12 also functions as an input interface.

  The operation unit 13 is a device for a user to perform an operation for maneuvering the flying device 2. The operation unit 13 includes, for example, an operation stick or volume for controlling the flight direction and the flight speed. When the display unit 12 is a touch screen, the operation unit 13 may be entirely or partially implemented using a GUI (Graphical User Interface) displayed on the display unit 12.

  The quality information acquisition unit 14 acquires quality information indicating the communication quality of the wireless network N in the place where the control device 1 exists. Here, the “communication quality of the wireless network N” includes not only the radio wave intensity of the wireless network N but also the radio wave interference. For example, when the radio field intensity of the wireless network N becomes weak, the communication quality is lowered. Further, when the radio network N has a high radio wave interference degree, the communication quality of the radio network N can be reduced even if the radio network N has a high radio field intensity.

  The control unit 15 is a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the control device 1, and executes an operation reception unit 151, a viewpoint setting unit by executing a program stored in the storage unit 11. 152, a display control unit 153, a route reception unit 154, and a notification unit 155.

  The operation reception unit 151 converts the input from the operation unit 13 into an operation command for the flying device 2. The transmission unit 100 transmits an operation command indicating the flight direction of the flying device 2 to the flying device 2 via the wireless network N. The position information receiving unit 1010 receives position information indicating the presence position of the flying device 2 from the flying device 2 via the wireless network N.

  The viewpoint setting unit 152 sets a virtual viewpoint at a position different from the position where the flying device 2 is present. The display control unit 153 causes the display unit 12 of the control device 1 to display a flight image indicating an area in which the flying device 2 is flying when the flying device 2 is viewed from the virtual viewpoint set by the viewpoint setting unit 152.

  Here, the “virtual viewpoint” is a viewpoint that does not necessarily exist unlike an imaging unit such as a camera included in the flying device 2. The viewpoint setting unit 152 sets, for example, 10 meters behind the flight direction of the flying device 2 and 2 m above the relative altitude. The user can change the place where the viewpoint setting unit 152 sets the virtual viewpoint by operating the operation stick of the control device 1. Further, the viewpoint setting unit 152 may change the position where the virtual viewpoint is set according to the flight speed of the flying device 2. Specifically, the viewpoint setting unit 152 may set a virtual viewpoint at a position farther from the flying device 2 when the flying speed of the flying device 2 is faster than when it is slower.

  For example, the display control unit 153 is a bird's-eye view when the flying device 2 is viewed from a set virtual viewpoint based on image data created in advance by a technique such as CG (Computer Graphics) based on known terrain information. An image is generated and displayed on the display unit 12. In the case where the control device 1 previously captures and holds a video of the airspace where the flying device 2 flies, the display control unit 153 may read the video and display it on the display unit 12. When the control device 1 and the flying device 2 store the same three-dimensional terrain information, the display control unit 153 performs a bird's eye view based on the position information and the terrain information received from the flying device 2. An image may be generated and displayed on the display unit 12.

  The real video receiving unit 1011 receives a real image captured by the flying device 2 from the flying device 2 via the wireless network N. The display control unit 153 causes the display unit 12 to display a real image captured by the flying device 2. Thereby, the user can confirm not only the real image captured by the flying device 2 but also the bird's-eye view image when the flying device 2 is viewed from a position away from the display unit 12. When the flying device 2 approaches another object, the user can observe an image showing both the flying device 2 and the other object. For this reason, the user can intuitively grasp the positional relationship and the sense of distance between the flying device 2 and other objects. As a result, even if the flying device 2 is out of the user's field of view, the operability of the flying device 2 is improved.

  In addition, when the flying device 2 includes a distance measuring sensor such as an infrared sensor (not shown), the flying device 2 can measure a distance from an object around the aircraft. When the distance between the aircraft and the surrounding object is within a predetermined distance, the flying device 2 notifies the control device 1 to that effect. When the flying device 2 approaches a surrounding object within a predetermined distance, the control device 1 causes the notification unit 155 of the control device 1 to reproduce a warning sound on a speaker (not shown) provided in the control device 1 or vibrate the vibrator. The user may be notified that the flying device 2 has approached another object.

  In addition, at least one of the control device 1 and the flying device 2 is configured such that the flying device 2 approaches another object based on the stored three-dimensional terrain information and the position information of the flying device 2. It may be determined whether or not. When the distance between the flying device 2 and another object is within a predetermined distance, the notification unit 155 of the control device 1 reproduces a warning sound on a speaker (not shown) included in the control device 1 or vibrates the vibrator. By doing so, the user may be notified that the flying device 2 has approached another object.

  Therefore, the “predetermined distance” is a “threshold distance for approach notification determination” referred to by the notification unit 155 to determine whether or not to notify the user that the flying device 2 has approached another object. . The approach notification determination threshold distance is stored in the storage unit 11. The specific value of the threshold distance for approach notification determination is a distance by which the flying device 2 can avoid other objects by taking an avoidance action, and may be determined by experiment in consideration of the flight performance of the flying device 2. Is, for example, 20 meters. When the flying device 2 determines approach with another object, the flying device 2 may change the approach notification determination threshold distance in consideration of the wind speed at that time.

  As described above, the bird's-eye view image displayed on the display unit 12 by the display control unit 153 is CG. For this reason, it is not necessarily guaranteed that the bird's-eye view image can faithfully reproduce the actual situation. Therefore, when the bird's-eye view image is displayed on the display unit 12, the display control unit 153 may always display the actual video on the display unit 12. Thereby, the user can always confirm the actual situation around the flying device 2.

  FIG. 3 is a diagram schematically illustrating an example of an operation screen displayed on the display unit 12 of the control device 1 according to the embodiment. In the example shown in FIG. 3, the operation screen includes a quality information indicator 301, a battery level indicator 302, an information panel 303, a bird's eye view display area 304, a flying device icon 305, and a live image display area 306.

  The quality information indicator 301 is an icon indicating the quality information acquired by the quality information acquisition unit 14, that is, the communication quality of the wireless network N in the place where the control device 1 is present. The remaining battery level indicator 302 is an icon indicating the remaining battery level of the control device 1.

  The information panel 303 shows information related to the flight of the flying device 2. The information panel 303 displays the distance between the control device 1 and the flying device 2, the traveling direction of the flying device 2, the flight speed of the flying device 2, and the altitude of the flying device 2.

  The bird's-eye view display area 304 is a display area for a bird's-eye image that the display control unit 153 displays on the display unit 12. A bird's-eye view display area 304 in FIG. 3 shows an example of a bird's-eye view image that the display control unit 153 displays on the display unit 12 when the viewpoint setting unit 152 sets a virtual viewpoint behind the flying device 2. For this reason, the bird's eye view display area 304 includes a flying device icon 305 indicating the flying device 2. The live image display area 306 is a display area of a real image captured by the flying device 2.

  As shown in FIG. 3, the flying device icon 305 is shown in the center of the bird's eye view display area 304. Following the movement of the flying device 2, the display control unit 153 changes the display area of the bird's eye view display area 304 so that the flying apparatus icon 305 becomes the center of the bird's eye view display area 304. In addition, the display control unit 153 may change the scale of the bird's eye view display area 304 according to the flight speed of the flying device 2. Specifically, the display control unit 153 may cause the display unit 12 to display a bird's eye view display area 304 showing a wide range of images when the flying speed of the flying device 2 is high, compared to when the flying speed is low.

  The camera included in the flying device 2 according to the embodiment images a landscape in front of the flying device 2 in the traveling direction. The bird's-eye view display area 304 in FIG. 3 is a bird's-eye view image when the viewpoint setting unit 152 sets a virtual viewpoint behind the flying device 2, so that the direction from the virtual viewpoint toward the flying device icon 305 is the flying device 2. Direction of travel. Therefore, as shown in FIG. 3, when the viewpoint setting unit 152 sets a virtual viewpoint behind the flying device 2, the image displayed in the bird's eye view display area 304 and the image displayed in the live image display area 306 are It will be similar.

  By the way, the user executes the operation of the flying device 2 by tilting an operation stick (not shown) provided in the control device 1 or operating an operation GUI (not shown) displayed on the display unit 12. . For example, when the user tilts the operation stick “right”, the flying device 2 turns “right”.

  Here, “right” and “left” are concepts of relative directions, and some kind of standard is required to determine the absolute direction. Therefore, in the control device 1 according to the embodiment, the operation reception unit 151 determines an operation command indicating the flight direction of the flying device 2 on the basis of the virtual viewpoint set by the viewpoint setting unit 152. The transmission unit 100 transmits the operation command determined by the operation reception unit 151 to the flying device 2. For example, when a virtual viewpoint is set behind the flying device 2, when the user tilts the stick “right”, the flying device 2 turns to the right with respect to the traveling direction. In this case, for example, if the flying device 2 is moving toward the user, when the user tilts the stick to the “right”, the flying device 2 appears to turn leftward as viewed from the user.

  Therefore, when the virtual viewpoint is set behind the flying device 2 with respect to the flying direction of the flying device 2, the user's stick system direction matches the moving direction of the flying device icon 305 in the bird's eye view display area 304. Will do. Therefore, the viewpoint setting unit 152 may set a virtual viewpoint behind the flying device 2 with respect to the flight direction of the flying device 2. As a result, the user's operation and the moving direction of the flying device 2 coincide with each other, so that the user can intuitively operate the flying device 2.

  When the flying device 2 is away from the control device 1, the viewpoint setting unit 152 sets a virtual viewpoint between the control device 1 and the flying device 2. More specifically, the viewpoint setting unit 152 is configured so that the vertical foot of the virtual viewpoint on the straight line passing through the position where the control device 1 is present and the position where the flight device 2 is present is the position where the flight device 2 is present. A virtual viewpoint is set so as to exist between the existing positions.

  FIG. 4 is a diagram schematically illustrating another example of the operation screen displayed on the display unit 12 of the control device 1 according to the embodiment, and is a diagram illustrating the positional relationship between the virtual viewpoint and the flying device 2. is there. In the example illustrated in FIG. 4, the operation screen includes a quality information indicator 301, a remaining battery level indicator 302, an information panel 303, a flying device icon 305, a positional relationship display area 307, a virtual viewpoint icon 308, a user icon 309, and a route map 310. Including.

  The positional relationship display area 307 is an area indicating the positional relationship between the control device 1, the flying device 2, and the virtual viewpoint. In the positional relationship display area 307, a flying device icon 305 indicating the position of the flying device 2, a virtual viewpoint icon 308 indicating a virtual viewpoint, and a user icon 309 indicating the position of the user (that is, the control device 1) are arranged. . In the positional relationship display area 307, a three-dimensional coordinate system is set with an axis indicating the east-west direction, an axis indicating the north-south direction, and an axis indicating the elevation direction with the flying device icon 305 as the origin.

  In the example shown in FIG. 4, the user is present at a position lower than the flying device 2 in the southwest of the flying device 2, and the virtual viewpoint is set at the same height as the flying device 2 in the south of the flying device 2. It has been shown. In this case, when the user tilts the stick “right”, the flying device 2 turns to the right, that is, in the east direction as viewed from the virtual viewpoint icon 308. By checking the positional relationship display area 307, the user can grasp the positional relationship between the flying device 2 and the virtual viewpoint.

  Since the flying device 2 generally flies over the distance from the user, the distance between the control device 1 and the flying device 2 is sufficiently longer than the distance between the virtual viewpoint set by the viewpoint setting unit 152 and the flying device 2. I can say that. Therefore, the positional relationship display area 307 is information indicating the direction in which the user (that is, the control device 1) exists when the flying device 2 is the starting point, which is that there is a user when the virtual viewpoint is the starting point. It can also be said to be information indicating the direction to perform. The display control unit 153 causes the display unit 12 to display the positional relationship display area 307 so that the user can easily grasp the positional relationship between the user and the virtual viewpoint.

  The display control unit 153 also causes the display unit 12 to display a route map 310 that is a two-dimensional map including the start point of the flying device 2 and the arrival target point of the flying device 2. In the example of the route map 310 shown in FIG. 4, the start point of the flying device 2 is indicated by a black circle, and the arrival target point of the flying device 2 is indicated by a white star. In addition, the current position of the flying device 2 in the route map 310 is indicated by a black triangle, and the route from the starting point of the flying device 2 to the current position of the flying device 2 is also shown.

  It can be said that the route map 310 is information indicating the flight direction of the flying device 2 when the control device 1 is the starting point. The display control unit 153 displays the route map 310 on the display unit 12, so that the user can start the flying device 2 (ie, the point where the user exists), the current position of the flying device 2, and the arrival of the flying device 2. The relationship with the target point can be grasped.

  Returning to the description of FIG. The route receiving unit 154 receives an input of one or a plurality of points existing on the route on which the flying device 2 flies from the user of the control device 1. Specifically, the user inputs the route of the flying device 2 by operating the operation unit 13 while viewing a map (not shown) displayed on the display unit 12. The route receiving unit 154 converts the operation of the operation unit 13 by the user into route information and outputs the route information to the display control unit 153. The display control unit 153 causes the display unit 12 to display a flight image region corresponding to one or a plurality of points as a mode different from other regions.

  The communication system S according to the embodiment further includes a server (not shown) connected to the control device 1 by the second wireless communication line W2, and the server stores one or more pieces of route information. If it is, the control device 1 may receive route information from the server. By doing in this way, the control device 1 does not need to increase the storage capacity of the storage unit 11 for storing a lot of route information.

  FIG. 5 is a diagram schematically illustrating an example of a display screen on which a flight path displayed on the display unit 12 of the control device 1 according to the embodiment is displayed. 5 includes a quality information indicator 301, a battery level indicator 302, an information panel 303, a bird's eye view display area 304, a flying device icon 305, and a path indicator 311.

  In FIG. 5, an arrow indicated by reference numeral 311 in the bird's eye view display area 304 is a route indicator 311 indicating a flight route input by the user. In order to prevent complication, not all arrows are labeled, but an arrow having the same shape as the arrow denoted by 311 in FIG. As shown in FIG. 5, the display control unit 153 displays a route indicator 311 that is a three-dimensional arrow at a location corresponding to the flight route input by the user, thereby distinguishing it from other flight images. Is displayed. By viewing the flight device icon 305 and the route indicator 311, the user can grasp at a glance whether or not the aircraft is flying along the flight route set by the flight device 2.

  Here, the notification unit 155 notifies the user of the control device 1 that the flying device 2 has deviated from the path for flying the flying device 2. When the flying device 2 deviates from the flight path, the notification unit 155 notifies the user by displaying a message indicating this on the display unit 12 via the display control unit 153. Instead of this, or in addition to this, the notification unit 155 causes the flying device 2 to be out of the flight path by playing a warning sound on a speaker (not shown) provided in the control device 1 or vibrating the vibrator. May be notified to the user. As a result, the user can quickly recognize that the flight device 2 has deviated from the set flight path.

  In addition, when the flying device 2 deviates from the route on which the flying device 2 flies, the viewpoint setting unit 152 includes a route indicator 311 indicating the flying route and a flying device icon 305 indicating the flying device 2 in the flight image. Set a virtual perspective. As a result, the bird's eye view display area 304 displayed on the display unit 12 by the display control unit 153 includes the route indicator 311 indicating the flight route and the flying device icon 305 indicating the flying device 2. By looking at the bird's eye view display area 304, the user can grasp the direction in which the flying device 2 should be moved in order to return the flying device 2 to the flight path.

  The above is based on the premise that the control device 1 can receive information such as the position information of the flying device 2 and the image captured by the flying device 2 from the flying device 2 via the wireless network N. Here, when the control device 1 is far away from the flying device 2, communication using Wi-Fi becomes difficult. Therefore, the control device 1 transmits the position information of the flying device 2 or the like via the second wireless communication line W2. Information such as an image captured by the flying device 2 is received from the flying device 2. For this reason, the communication quality of the second wireless communication line W2 in the region where the flying device 2 is flying becomes important when information is exchanged between the control device 1 and the flying device 2.

  Therefore, the quality information receiving unit 1012 receives the quality information indicating the communication quality of the second wireless communication line W2 in the place where the flying apparatus 2 exists from the flying apparatus 2 via the second wireless communication line W2. The display control unit 153 causes the display unit 12 to display the quality information of the second wireless communication line W2 acquired from the flying device 2 on the flight image.

  FIG. 6 is a diagram illustrating an example of the radio wave intensity map 312 displayed on the display unit 12. In FIG. 6, distribution information indicating the distribution of communication quality of the second wireless communication line W2 for each area is displayed as a radio wave intensity map 312. In the radio wave intensity map 312, as indicated by the hatching label 313, regions having different radio wave intensities of the second wireless communication line are assigned with different hatchings.

  The radio wave intensity map 312 includes three-dimensional distribution information of communication quality. For this reason, the display control unit 153 displays the radio wave intensity map 312 in association with the altitude of the flying device 2 from the ground. In this display screen, a radio wave intensity map 312 at an altitude of about 35 m is shown. Each time the user selects the up icon 314, the display control unit 153 displays a radio wave intensity map with the altitude increased by 5 m. Further, every time the user selects the down icon 315, the display control unit 153 displays a radio wave intensity map with the altitude reduced by 5 m.

  The display control unit 153 displays the flying device icon 305 at the center of the radio wave intensity map 312. This display screen shows an example in which the flying device 2 is located at an altitude of 34.2 m from the ground. As described above, the display control unit 153 causes the display unit 12 to display the place where the flying device 2 exists on the radio wave intensity map 312 including the place where the flying device 2 exists. Thereby, the user can fly the flying device 2 by selecting an area with good communication quality. The display control unit 153 reads out the radio wave intensity map 312 stored in advance in the storage unit 11 and displays it on the display unit 12 instead of displaying the quality information received from the flying device 2 on the display unit 12. May be.

  When the radio wave intensity map 312 of the place where the flying device 2 exists cannot be acquired, the display control unit 153 may display an area map on the display unit 12 instead of the radio wave intensity map 312. Here, the “area map” is a communicable area map indicating whether or not the communication of the second wireless communication line W2 is possible, and is a known map created from the communication status of an existing smartphone.

  Here, when the communication quality of the second wireless communication line W2 included in the quality information becomes lower than a predetermined quality in the place where the flying device 2 exists, the display control unit 153, for example, the radio wave of the second wireless communication line W2 When the strength is equal to or lower than the predetermined threshold, the display unit 12 displays information indicating a direction in which there is a region where the radio wave strength is larger than the predetermined threshold (that is, a region where the communication quality is improved from the predetermined quality). To display. Note that the “predetermined threshold value” refers to the radio wave intensity referred to by the display control unit 153 in order to determine whether or not communication quality that can withstand transmission of operation information from the control device 1 to the flying device 2 can be maintained. This is a discrimination reference threshold value. The specific value of the discrimination reference threshold value may be determined by experiment in consideration of the performance of the antenna provided in the control device 1 and the flying device 2.

  FIG. 7 is a diagram schematically showing a display screen displayed on the display unit 12 when the radio field intensity around the flying device 2 becomes a predetermined threshold value or less. As shown in FIG. 7, when the radio field intensity around the flying device 2 is equal to or lower than a predetermined threshold, the display unit 12 displays a radio field intensity decrease message 316. Thereby, the user can move the flying device 2 to an environment with good communication quality before the communication with the flying device 2 is interrupted.

  Here, it is useful for the user if it is possible to know in which direction the flying device 2 is moved to improve the communication environment. Therefore, the display control unit 153 causes the display unit 12 to display a direction in which the communication quality of the second wireless communication line W2 is improved when the flying device 2 moves.

  In FIG. 7, a direction indicator 317 indicates a direction in which the communication quality of the wireless network N is improved when the flying device 2 moves, and a user icon 309 indicates a user's presence direction when the position of the flying device 2 is used as a reference. Show. In the example shown in FIG. 7, it is shown that the communication quality of the wireless network N is improved if the user operates the flying device 2 so as to rise in the northwest direction.

  Note that the direction indicator 317 shown in FIG. 7 indicates the direction in which the flying device 2 can actually move. The display control unit 153 does not display the direction indicator 317 on the display unit 12 in a direction in which the flying device 2 cannot move due to an obstacle or the like even in a direction in which the communication quality of the second wireless communication line W2 is improved. Alternatively, an icon (not shown) indicating that it cannot be moved is displayed.

  Here, when the radio field intensity around the flying device 2 becomes a predetermined threshold value or less, the flying device 2 stops in the air (hovering) on the spot until new operation information is transmitted from the control device 1. It may be. Thereby, it is possible to prevent the flying device 2 from moving to a region where the radio field intensity is lower.

<Processing Flow of Information Processing Performed by Control Device 1 according to Embodiment>
FIG. 8 is a flowchart for explaining the flow of information processing executed by the control device 1 according to the embodiment. The process in this flowchart starts when the power of the control device 1 is turned on, for example.

  The position information receiving unit 1010 receives position information indicating the presence position of the flying device 2 from the flying device 2 via the wireless network N (S2). The viewpoint setting unit 152 sets a virtual viewpoint at a position different from the position where the flying device 2 is present (S4). The display control unit 153 causes the display unit 12 of the control device 1 to display a flight image indicating a region in which the flying device 2 is flying when the flying device 2 is viewed from the virtual viewpoint set by the viewpoint setting unit 152 ( S6). The display control unit 153 causes the display unit 12 to display the actual image captured by the flying device 2 acquired via the wireless network N (S8). When the display control unit 153 displays the actual image on the display unit 12, the processing in this flowchart ends.

<Effects of the control device 1 according to the embodiment>
As described above, according to the control device 1 according to the embodiment, the operability of the flying device 2 that is out of the user's field of view can be improved.
In particular, since a virtual viewpoint is set at a position different from that of the flying device 2 and the flying region of the flying device 2 when the flying device 2 is viewed from the viewpoint is displayed on the display unit 12, the user can fly It is possible to easily grasp the positional relationship and distance relationship between the device 2 and another object.

  Further, when the user operates the operation unit of the control device 1, the flight direction of the flying device 2 is changed based on the virtual viewpoint, so the user intuitively operates the flying device 2 while viewing the display unit 12. be able to. Further, since the real image captured by the flying device 2 is also displayed on the display unit 12 of the control device 1 in real time, the user can check the current situation around the flying device 2.

  Since the control device 1 displays the flight path planned to fly the flight device 2 input in advance on the display unit 12, the user can operate the flight device 2 while confirming the flight route. When the flying device 2 deviates from this flight path, the control device 1 displays a flight image including the flight path and the flying device 2 on the display unit 12. As a result, the user can easily return the flying device 2 to the flight path.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. In particular, the specific embodiments of the distribution / integration of the devices are not limited to those illustrated above, and all or a part thereof may be added in arbitrary units according to various additions or according to functional loads. It can be configured functionally or physically distributed and integrated. Such modifications will be described below.

<First Modification>
In the above description, a case has been described in which the flying device 2 images the surrounding situation using an imaging device such as a camera. Here, the flying device 2 may include an image recognition unit (not shown) that recognizes a subject of an image captured by the imaging device. When the image recognition unit recognizes another flying object in the image captured by the imaging device, the image recognition unit may notify the control device 1 possessed by the user and hover on the spot. The display control unit 153 of the control device 1 may display only the actual image on the display unit 12 when receiving a notification from the flying device 2 that another flying object has been recognized. Thereby, it can suppress that the flying apparatus 2 collides with another flying body.

  The image recognition unit can be realized by using a detector generated by using a known machine learning technique such as a neural network, SVM (Support Vector Machine), boosting or the like.

<Second Modification>
In the above, the case where the route reception unit 154 receives an input of one or a plurality of points existing on the route on which the flying device 2 flies from the user of the control device 1 has been described. Instead of this, or in addition to this, the route receiving unit 154 may download a three-dimensional map including a plurality of recommended routes from an external server (not shown). The display control unit 153 may receive a switching instruction from the user via the operation unit 13 and switch the recommended route to be displayed on the display unit 12. Thereby, the user can select a route for actually flying the flying device 2 from a plurality of recommended routes.

<Third Modification>
In the above description, the case where the operation reception unit 151 determines the operation command indicating the flight direction of the flying device 2 based on the virtual viewpoint set by the viewpoint setting unit 152 has been described. This is useful when the bird's eye view display area 304 is displayed on the display unit 12. However, when the bird's eye view display area 304 is not displayed on the display unit 12 and the positional relationship display area 307 and the route map 310 are displayed instead, the operation reception unit 151 displays the positional relationship display region 307 and the route. It is preferable to determine an operation command indicating the flight direction of the flying device 2 based on the positional relationship between the control device 1 and the flying device 2 on the map 310 (that is, the positional relationship on the map displayed on the display unit 12).

  Therefore, the control device 1 according to the third modification includes a change-over switch realized by a GUI or a physical switch (not shown). The user operates the changeover switch so that the operation reception unit 151 determines an operation command indicating the flight direction of the flying device 2 with reference to the virtual viewpoint set by the viewpoint setting unit 152 or is displayed on the display unit 12. It is possible to switch whether to determine an operation command indicating the flight direction based on the positional relationship on the map. Thereby, the control device 1 according to the third modification can provide a user interface for operation suitable for the display content of the display unit 12 to the user.

  It should be noted that a new modification example generated by arbitrarily combining the above modification examples is also included in the embodiment of the present invention. The effects of the modified example newly generated by the combination have the effects of the original modified examples.

DESCRIPTION OF SYMBOLS 1 ... Steering device 2 ... Flight apparatus 3 ... Base station 10 ... Communication part 11 ... Memory | storage part 12 ... Display part 13 ... Operation part 14 ... Quality information acquisition part DESCRIPTION OF SYMBOLS 15 ... Control part 100 ... Transmission part 101 ... Reception part 151 ... Operation reception part 152 ... Viewpoint setting part 153 ... Display control part 154 ... Path | route reception part 155 ... Notification unit 1010 ... Position information receiving unit 1011 ... Real video receiving unit 1012 ... Quality information receiving unit N ... Wireless network S ... Communication system

Claims (14)

A control device for controlling a flying device via a wireless network,
A transmission unit that transmits an operation command indicating a flight direction of the flying device to the flying device via the wireless network;
A position information receiving unit that receives position information indicating the presence position of the flying device from the flying device via the wireless network;
A viewpoint setting unit that sets a virtual viewpoint at a position different from the position where the flying device exists;
A display control unit for displaying a flight image indicating a region in which the flying device is in flight when the flying device is viewed from the virtual viewpoint on the display unit of the control device;
A control device comprising: The transmission unit transmits the operation command indicating a flight direction of the flying device determined based on the virtual viewpoint;
The control device according to claim 1. Via the wireless network, further comprising an actual video receiver for receiving the actual image captured by the flying device from the flying device;
The display control unit causes the display unit to display a real image captured by the flying device.
The control device according to claim 1 or 2. A quality information receiving unit for receiving quality information indicating the communication quality of the wireless network in a place where the flying device exists via the wireless network;
The display control unit causes the quality information to be superimposed on the flight image and displayed on the display unit.
The control device according to any one of claims 1 to 3. When the communication quality of the wireless network included in the quality information is lower than a predetermined quality in the place where the flying device exists, the display control unit has an area where the communication quality is improved from the predetermined quality. Information indicating the direction to be displayed is displayed on the display unit,
The control device according to claim 4. A path accepting unit that accepts an input of one or a plurality of points existing on a route for flying the flying device from a user of the control device;
The display control unit causes the display unit to display a region of the flight image corresponding to the one or a plurality of points as a mode different from other regions.
The control device according to any one of claims 1 to 5. A notification unit for notifying a user of the control device that the flying device has deviated from a route for flying the flying device;
The control device according to claim 6. The viewpoint setting unit sets the virtual viewpoint so that the route and the flying device are included in the flight image when the flying device deviates from a route for flying the flying device.
The control device according to claim 6 or 7. The display control unit causes the display unit to display information indicating a flight direction of the flying device when the control device is a starting point.
The control device according to any one of claims 1 to 8. The display control unit causes the display unit to display a direction in which the control device is present when the virtual viewpoint is set as a starting point;
The control device according to any one of claims 1 to 9. The viewpoint setting unit sets the virtual viewpoint behind the flying device with respect to the flight direction of the flying device;
The control device according to any one of claims 1 to 10. A processor included in a control device for controlling a flying device via a wireless network,
Transmitting an operation command indicating a flight direction of the flying device to the flying device via the wireless network;
Receiving position information indicating the location of the flying device from the flying device via the wireless network;
Setting a virtual viewpoint at a position different from the presence position of the flying device;
Displaying a flight image indicating a region in which the flying device is in flight when the flying device is viewed from the virtual viewpoint on the display unit of the control device;
Information processing method to execute. To a computer for maneuvering flight equipment over a wireless network,
A function of transmitting an operation command indicating a flight direction of the flying device to the flying device via the wireless network;
A function of receiving, from the flying device, position information indicating the location of the flying device via the wireless network;
A function of setting a virtual viewpoint at a position different from the position where the flying device exists;
A function of causing the display unit to display a flight image indicating an area in which the flying device is flying when the flying device is viewed from the virtual viewpoint;
A program that realizes A flying device;
A control device for maneuvering the flying device via a wireless network;
A flight system comprising:
The steering device is
A transmission unit that transmits an operation command indicating a flight direction of the flying device to the flying device via the wireless network;
A position information receiving unit that receives position information indicating the presence position of the flying device from the flying device via the wireless network;
A viewpoint setting unit that sets a virtual viewpoint at a position different from the position where the flying device exists;
A display control unit that displays a flight image indicating a region in which the flying device is in flight when the flying device is viewed from the virtual viewpoint on the display unit of the control device;
The flying device is
A receiving unit for receiving the operation command from the control device;
A flight controller that controls the flight of the flying device based on the operation command;
A position acquisition unit for acquiring position information indicating an existing position of the flying device;
A transmission unit that transmits the position information to the control device via the wireless network;
Flight system.

Images