JP6495562B1 - Aerial imaging system, method and program using unmanned air vehicle - Google Patents

Abstract

To provide an aerial imaging system that minimizes the effort required for editing an image by unintentionally shooting an unintended portion by an unmanned air vehicle.
An aerial imaging system according to the present invention includes an unmanned air vehicle including an aerial imaging apparatus, a controller, and a computer. The control unit 14 of the unmanned air vehicle 10 executes the route determination module 142 to determine whether the actual flight of the unmanned air vehicle 10 is away from the predetermined flight planned route previously input by using the computer 30. To do. Then, when the distance is a predetermined distance, the control unit 14 executes the aerial image non-display module 143 to enable the image display unit 34 to hide the aerial image by the camera 17.
[Selection] Figure 1

Description

  The present invention relates to an aerial imaging system, method, and program using an unmanned air vehicle.

  Aerial photography using an unmanned aerial vehicle can reduce costs compared to using a real airplane or helicopter, and can shoot safely even in a narrow space at low altitudes, as well as shooting close to the shooting target. As a result, high-quality photographs and video images can be obtained, and the maintenance status of structures, inspection of high places, ground observation from high places, etc. are being implemented in various fields.

  As an aerial photography method using an unmanned aerial vehicle, it is proposed to measure the distance between the aircraft and the object to be photographed during the flight of the unmanned aerial vehicle, determine the zoom magnification of the camera from the result, and photograph the object to be photographed (For example, refer to Patent Document 1). According to this method, it is possible to accurately capture an image of an object to be photographed with a camera mounted on an unmanned air vehicle and expand it to a sufficient size within a camera frame and record it as high-definition image data.

Japanese Patent Laid-Open No. 2006-027448

  By the way, when an unmanned aerial vehicle is manually operated and aerial photography is performed, an unintended operation by an operator may be performed and an unrelated place may be photographed. For example, in order to avoid an obstacle or when a strong wind blows, an unintended operation is performed and a portion unrelated to the aerial object is captured. In that case, it will be necessary to edit the video in which the unintended portion is shot later, but it is desirable to save the editing work as much as possible.

  The present invention has been made in view of such a demand, and provides an aerial imaging system that minimizes the effort required to edit an image by aerial shooting of an unintended portion by an unmanned air vehicle. Objective.

  The present invention provides the following solutions.

The invention according to the first aspect is an aerial imaging system using an unmanned air vehicle,
Route input receiving means for receiving in advance an input of a flight route of an unmanned air vehicle having an aerial imaging device;
Route determination means for determining whether the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route;
Aerial image non-display that enables the aerial image captured by the aerial image device to be hidden on the display device when it is determined that the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route And an aerial imaging system.

  According to the first aspect of the invention, the route determination means determines whether or not the actual flight of the unmanned air vehicle is a predetermined distance away from the previously input flight route. The aerial image non-display means hides the aerial image from the aerial imaging apparatus on the display device when the route determination means determines that the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route. Make it possible.

  As a result, it is possible to reduce the risk that an unintentional portion of an unmanned air vehicle can be aerial shot that may be caused by a predetermined distance from a previously input flight route. Can save time and effort for editing video.

The invention according to the second feature is the invention according to the first feature,
The flight route includes information on latitude, longitude and altitude where the unmanned air vehicle is located,
The route determination means determines whether the latitude, longitude, and altitude at which the unmanned air vehicle is actually located are a predetermined distance away from the latitude, longitude, and altitude input as the flight route. provide.

  According to the invention relating to the second feature, the route determination means can determine the accuracy of the actual flight route by the unmanned air vehicle from the three-dimensional information of latitude, longitude and altitude. Thereby, the risk that the route determination means makes an incorrect determination can be suppressed. As a result, it is determined that the actual flight of the unmanned air vehicle is away from the flight route by a predetermined distance from the pre-input flight route and the unintended part is aerial, but it is not away from the flight route. The erroneous determination that the aerial image is not displayed on the display device because it is determined that it is a predetermined distance away from the flight route despite the originally planned aerial object can be eliminated.

The invention according to the third feature is the invention according to the first or second feature,
When the aerial image non-display means determines that the actual flight of the unmanned air vehicle is away from the flight route by a predetermined distance, the aerial image non-display unit incorporates a predetermined flag in the aerial image data by the aerial imager,
The imaging system further provides an aerial imaging system further comprising notification means for notifying that a frame of an aerial video at the time when the predetermined flag is incorporated may need to be edited.

  According to the invention relating to the third feature, when the actual flight of the unmanned air vehicle is away from the flight route by a predetermined distance, the aerial imaging device is not immediately stopped, but is temporarily stopped by the aerial imaging device. The aerial shooting is continued, and after the aerial shooting is completed, the editor is informed that it may be necessary to edit the frame of the aerial shooting video at the time when the predetermined flag is incorporated. As a result, the actual flight of the unmanned air vehicle was actually separated from the flight route entered in advance by a predetermined distance, but the originally planned aerial object was reflected, or an unintended part was not reflected. If not, it is possible to eliminate the situation where the aerial video cannot be taken.

  According to the present invention, it is possible to reduce a risk that an unintentional portion of an unmanned air vehicle that is caused by leaving a predetermined distance from a previously input flight route can be aerial shot. This saves the time and effort required to edit the video by taking an aerial shot.

FIG. 1 is a block diagram showing a hardware configuration and software functions of an aerial imaging system 1 using an unmanned air vehicle 10 in the present embodiment. FIG. 2 is a flowchart showing an aerial photography method in the present embodiment. FIG. 3 is an example of a display example on the image display device 34 when a flight route is input.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

<Configuration of aerial imaging system 1 with unmanned air vehicle>
FIG. 1 is a block diagram for explaining the hardware configuration and software functions of an aerial imaging system 1 in the present embodiment. The aerial imaging system 1 includes an unmanned aerial vehicle 10 having an aerial imaging device, a controller 20 that is connected so as to be able to wirelessly communicate with the unmanned aerial vehicle 10, and a flight route of the unmanned aerial vehicle 10. And a computer 30 for inputting and editing a video taken by the unmanned aerial vehicle 10.

[Unmanned aerial vehicle 10]
The unmanned aerial vehicle 10 is not particularly limited as long as it is a device capable of photographing a subject to be photographed from the sky. For example, the unmanned aerial vehicle 10 may be a radio controlled airplane or an unmanned aerial vehicle called a drone. In the following description, it is assumed that the unmanned air vehicle 10 is a drone.

  The unmanned aerial vehicle 10 is powered by a battery 11 that functions as a power source for the unmanned aerial vehicle 10, a motor 12 that operates with electric power supplied from the battery 11, and rotates by the operation of the motor 12. And a rotor 13.

  The unmanned aerial vehicle 10 is in accordance with a control unit 14 that controls the operation of the unmanned aerial vehicle 10, a position detection unit 15 that transmits position information of the unmanned aerial vehicle 10 to the control unit 14, and a control signal from the control unit 14. A driver circuit 16 for driving the motor 12, a camera 17 for taking an aerial image of an object to be photographed in accordance with a control signal from the control unit 14, a control program executed by the microcomputer of the control unit 14, and the like are stored in advance. 17 includes a storage unit 18 for storing an image taken by the camera.

  The unmanned air vehicle 10 includes a wireless communication unit 19 that performs wireless communication with the controller 20.

  These components are mounted on a main body structure (frame or the like) having a predetermined shape. As for a main body structure (frame or the like) having a predetermined shape, a similar one to a known drone may be adopted.

[Battery 11]
The battery 11 is a primary battery or a secondary battery, and supplies power to each component in the unmanned air vehicle 10. The battery 11 may be fixed to the unmanned air vehicle 100 or may be detachable.

[Motor 12, rotor 13]
The motor 12 functions as a drive source for rotating the rotor 13 with electric power supplied from the battery 11. As the rotor 13 rotates, the unmanned air vehicle 10 can be levitated and fly.

[Control unit 14]
The controller 14 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like.

  In addition, the control unit 14 reads a predetermined program, thereby realizing the imaging module 141, the route determination module, and the aerial video non-display module 143.

  The control unit 14 controls the motor 12 to perform flight control (control of ascending, descending, horizontal movement, etc.) of the unmanned air vehicle 10. Further, the control unit 14 controls the attitude of the unmanned air vehicle 10 by controlling the motor 12 using a gyro (not shown) mounted on the unmanned air vehicle 10.

[Position detection unit 15]
The position detection unit 15 includes a LIDAR (Laser Imaging Detection and Ranging) technique and a GPS (Global Positioning System) technique. The LIDAR technology and the GPS technology are used in combination to detect the latitude, longitude, and height of the position where the unmanned air vehicle actually flies.

[Driver circuit 16]
The driver circuit 16 has a function of applying a voltage designated by a control signal from the control unit 14 to the motor 12. As a result, the driver circuit 16 can drive the motor 12 in accordance with the control signal from the control unit 14.

[Camera 17]
The camera 17 functions as an aerial imaging device. The camera 17 converts (images) an optical image captured by the lens into an image signal by an image sensor such as a CCD or a CMOS. The type of the camera 17 may be selected as appropriate according to the image analysis method to be photographed.

[Storage unit 18]
The storage unit 18 is a device that stores data and files, and includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card. The storage unit 18 includes a control program storage area 181 for storing in advance a control program executed by the microcomputer of the control unit 14 and a computer transferred through a portable recording medium such as a USB memory or an SD card. The flight schedule storage area 182 for storing the flight schedule route stored in the storage unit 33 of the 30 and the three-dimensional coordinate data detected by the position detection unit 15 (photographed by the camera 17). And an image data storage area 183 for storing the data of latitude, longitude, and height of a point corresponding to the shooting position of the captured image.

  Note that the image constituting the image data may be a still image or a moving image. The data stored in the image data storage area can be transferred to the computer 30 through a portable recording medium such as a USB memory or an SD card.

[Wireless communication unit 19]
The wireless communication unit 19 is configured to be able to perform wireless communication with the controller 20 and receives a remote control signal from the controller 20.

[Controller 20]
The controller 20 has a function of maneuvering the unmanned air vehicle 10. The controller 20 includes an operation unit 21 used by a user to steer the unmanned air vehicle 10, a control unit 22 that controls the operation of the controller 20, and a control program executed by a microcomputer of the control unit 22 in advance. A storage unit 23 to be stored, a wireless communication unit 24 that wirelessly communicates with the unmanned air vehicle 10, and an image display unit 25 that displays a predetermined image to the user are provided.

  The wireless communication unit 24 is configured to be able to wirelessly communicate with the unmanned air vehicle 10 and receives a remote control signal toward the unmanned air vehicle 10.

  The image display unit 25 may be integrated with a control device that controls the unmanned air vehicle 10, or may be separate from the control device. If integrated with the control device, the number of devices used by the user can be reduced, and convenience is enhanced. When the control device is separate from the control device, examples of the image display unit 25 include portable terminal devices such as smartphones and tablet terminals that can be wirelessly connected to the wireless communication unit 19 of the unmanned air vehicle 10. When the control device is separate from the control device, an existing control device that does not have the image display unit 25 can be applied.

[Computer 30]
The computer 30 has a function of inputting a flight route of the unmanned air vehicle 10 and a function of editing a video imaged by the unmanned air vehicle 10. The computer 30 stores an input unit 31 for a user to input command information, a control unit 32 that controls the operation of the computer 30, a control program executed by the microcomputer of the control unit 32, and the like. Unit 33 and an image display unit 34 for displaying a predetermined image to the user.

  The control part 32 implement | achieves the route input reception module 321 and the alerting | reporting module 322 by reading a predetermined program.

  The storage unit 33 includes a map database 331 in which map information around a place to be taken aerial is stored, a planned flight route storage area 332 for storing information on the flight route of the unmanned air vehicle 10 input by the user, and a USB memory And an image data storage area 333 for storing the image data stored in the storage unit 18 of the unmanned air vehicle 10 transferred through a portable recording medium such as an SD card.

<Flowchart showing aerial photography method using aerial photography system 1>
FIG. 2 is a flowchart showing an aerial imaging method using the aerial imaging system 1. The processing executed by each hardware and the software module described above will be described.

[Step S10: Enter Flight Route]
First, the control unit 32 of the computer 30 of the aerial imaging system 1 executes the route input reception module 321 and receives the input of the flight route of the unmanned air vehicle 10 via the input unit 31.

  The control unit 32 of the computer 30 refers to the map database 331 and causes the image display unit 34 to display a map around the area where the unmanned air vehicle 10 is scheduled to fly. The user inputs the flight route of the unmanned air vehicle 10 on the map displayed on the image display unit 34. The flight route input operation method is not particularly limited, and may be an input operation using a mouse or an input operation using a touch panel.

  It is preferable that the user inputs not only the flight route (latitude and longitude) on the map but also altitude information where the unmanned air vehicle is located. By inputting three-dimensional information such as latitude, longitude, and altitude, it is later determined whether or not the actual flight of the unmanned air vehicle 10 is a predetermined distance away from the previously input flight route (step S13). It becomes possible to judge more accurately.

  FIG. 3 is an example of a display example on the image display device 34 at that time. The image display device 34 displays a map around the area where the unmanned air vehicle 10 is scheduled to fly. On the map, the flight path (latitude and longitude information) input by the user and the flight altitude information at a plurality of points are displayed.

  Information on the planned flight route input by the user is stored in the planned flight route storage area 332. The information stored in the planned flight route storage area 332 is set in the planned flight route storage area 182 of the unmanned air vehicle 10 via a portable recording medium such as a USB memory or an SD card.

[Step S11: Start of flight]
Subsequently, the user operates the operation unit 21 of the controller 20. The control unit 22 of the controller 20 transmits the content of information input to the operation unit 21 to the unmanned air vehicle 10 via wireless communication. The unmanned air vehicle 10 receives information input to the operation unit 21 via the wireless communication unit 19, and the control unit 14 causes the unmanned air vehicle 10 to fly according to the received information.

[Step S12: Start aerial photography]
Subsequently, the control unit 14 of the unmanned air vehicle 10 executes the imaging module 141 and starts aerial imaging using the camera 17. The data of the aerial image captured by the camera 17 is stored in the image data storage area 183.

[Step S13: Determination of Flight Route]
Subsequently, the control unit 14 of the unmanned air vehicle 10 executes the route determination module 142, and determines whether or not the actual flight of the unmanned air vehicle 10 is a predetermined distance away from the planned flight route previously input in step S10. To do.

  The position detector 15 of the unmanned air vehicle 10 detects the latitude, longitude, and altitude at which the unmanned air vehicle 10 is actually located. Then, the control unit 14 of the unmanned air vehicle 10 refers to the data stored in the planned flight route storage area 182, and the point (latitude, longitude, and altitude) detected by the position detection unit 15 is the planned flight route storage area. The distance from the planned flight route stored in 182 is calculated. Then, the control unit 14 determines whether the distance obtained by the calculation exceeds the predetermined distance or is equal to or less than the predetermined distance.

[Step S14: Determination of whether or not a predetermined distance]
Subsequently, the control unit 14 of the unmanned air vehicle 10 determines whether or not the distance obtained by the calculation exceeds a predetermined distance as a result of the process of step S13. When this determination is YES, that is, when the distance obtained by the calculation exceeds the predetermined distance, the control unit 14 moves the process to step S15. When this determination is NO, that is, when the distance obtained by calculation is equal to or less than the predetermined distance, the control unit 14 proceeds to step S16.

[Step S15: Set out-of-route flag in aerial video data]
Subsequently, the control unit 14 of the unmanned air vehicle 10 executes the aerial image non-display module 143 and incorporates an out-of-route flag in the aerial image data captured by the camera 17.

  In the invention described in the present embodiment, in step S14, when it is determined that the actual flight of the unmanned air vehicle 10 is a predetermined distance away from the planned flight route, the aerial image captured by the camera 17 is imaged. It is sufficient to make the display unit 34 non-displayable. That is, in this case, as shown in step S15, it is not essential to incorporate the out-of-route flag into the aerial image data taken by the camera 17, and the aerial shooting by the camera 17 may be stopped.

[Step S16: Did you finish the flight?]
Subsequently, the control unit 14 of the unmanned air vehicle 10 refers to the flight planned route storage area 182 and determines whether or not the flight of the planned flight route has been completed. When this determination is YES, the control unit 14 proceeds to step S17. When this determination is NO, the control unit 14 moves the process to step S13 and repeats the processes of step S13 to step S16.

[Step S17: End of aerial photography]
Subsequently, the control unit 14 of the unmanned air vehicle 10 ends the aerial photography using the camera 17.

[Step S18: End of flight]
Subsequently, the user operates the operation unit 21 of the controller 20 to perform an operation for landing the unmanned air vehicle 10. The control unit 22 of the controller 20 transmits the content of information input to the operation unit 21 to the unmanned air vehicle 10 via wireless communication. The unmanned air vehicle 10 receives information input to the operation unit 21 via the wireless communication unit 19, and the control unit 14 causes the unmanned air vehicle 10 to land according to the received information. The information stored in the image data storage area 183 is set in the image data storage area 333 of the computer 30 via a portable recording medium such as a USB memory or an SD card.

[Step S19: Announce aerial video frames for which the out-of-route flag is set]
Subsequently, the control unit 32 of the computer 30 executes the notification module 332 to notify the user that it may be necessary to edit the frame of the aerial video at the time when the out-of-route flag is incorporated. When the process of step S19 is completed, the aerial imaging system 1 ends a series of processes of the aerial imaging method according to the present embodiment.

  According to the invention described in the present embodiment, the control unit 14 of the unmanned air vehicle 10 executes the route determination module 142, and the actual flight of the unmanned air vehicle 10 is preliminarily input using the computer 30. It is determined whether or not a predetermined distance from the route. Then, when the distance is a predetermined distance, the control unit 14 executes the aerial image non-display module 143 to enable the image display unit 34 to hide the aerial image by the camera 17.

  As a result, it is possible to suppress a risk that an unintentional portion that may occur due to the actual flight of the unmanned air vehicle 10 being away from the flight route input in advance by a predetermined distance is taken aerial. As a result, it is possible to reduce the effort required to edit the video due to aerial shooting of an unintended portion.

  In the invention described in the present embodiment, the control unit 14 preferably determines the accuracy of the actual flight route by the unmanned air vehicle 10 from the three-dimensional information of latitude, longitude, and altitude. Thereby, when the control unit 14 executes the route determination module 142, the risk that the control unit 14 makes an incorrect determination can be suppressed. As a result, it is determined that the actual flight of the unmanned air vehicle 10 is away from the flight route by a predetermined distance from the pre-input flight route and the unintended portion is taken aerial. It is possible to eliminate the erroneous determination that the aerial image is not displayed on the image display unit 34 because it is determined that it is a predetermined distance away from the flight route even though the aerial image object is originally planned. .

  Further, in the invention described in the present embodiment, when the actual flight of the unmanned air vehicle 10 is away from the flight route by a predetermined distance, the aerial shooting by the camera 17 is not immediately stopped, but once by the camera 17. Preferably, the aerial photography is continued, and after the aerial photography is completed, it is preferable to notify the editor that it may be necessary to edit the frame of the aerial video at the time when the out-of-route flag is incorporated. As a result, the actual flight of the unmanned air vehicle 10 was actually a predetermined distance away from the previously input flight route, but the originally planned aerial object was reflected, or an unintended part was reflected. When it is not indented, it is possible to eliminate the situation where it is impossible to shoot an aerial image.

  The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.), for example. In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

DESCRIPTION OF SYMBOLS 1 Aerial imaging | photography system by unmanned air vehicle 10 Unmanned air vehicle 11 Battery 12 Motor 13 Rotor 14 Control part 141 Shooting module 142 Route determination module 143 Aerial imaging | video non-display module 15 Position detection part 16 Driver circuit 17 Camera 18 Memory | storage part 181 Control program Storage area 182 Flight planned route storage area 183 Image data storage area 19 Wireless communication unit 20 Controller 30 Computer 31 Input unit 32 Control unit 321 Route reception module 322 Notification module 33 Storage unit 331 Map database 332 Flight planned route storage area 333 Image data storage Region 34 Image display area

Claims (5)

Route input receiving means for receiving in advance an input of a flight route of an unmanned air vehicle having an aerial imaging device;
Route determination means for determining whether the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route;
Aerial image non-display that enables the aerial image captured by the aerial image device to be hidden on the display device when it is determined that the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route And an aerial imaging system using an unmanned air vehicle. The flight route includes information on latitude, longitude and altitude where the unmanned air vehicle is located,
The route determination means determines whether the latitude, longitude, and altitude at which the unmanned air vehicle is actually located are a predetermined distance away from the latitude, longitude, and altitude input as the flight route. The aerial imaging system with the unmanned air vehicle described. When the aerial image non-display means determines that the actual flight of the unmanned air vehicle is away from the flight route by a predetermined distance, the aerial image non-display unit incorporates a predetermined flag in the aerial image data by the aerial imager,
3. The unmanned system according to claim 1, wherein the control unit of the computer further includes notification means for notifying that a frame of an aerial image at the time when the predetermined flag is incorporated may need to be edited. Aerial imaging system with flying objects. An unmanned aerial photography method executed by an unmanned aerial photography system,
Receiving in advance an input of a flight route of an unmanned air vehicle having an aerial imaging device;
Determining whether the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route;
Enabling the aerial image captured by the aerial imaging device to be hidden on a display device when it is determined that the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route. , Aerial photography method by unmanned air vehicle. For aerial imaging systems using unmanned air vehicles,
Receiving in advance an input of a flight route of an unmanned air vehicle having an aerial imaging device;
Determining whether an actual flight of the unmanned air vehicle is a predetermined distance away from the flight route;
Enabling the aerial image captured by the aerial imaging device to be hidden on a display device when it is determined that the actual flight of the unmanned air vehicle is a predetermined distance away from the flight route;
A computer-readable program for executing the program.

Images