JP5767731B1 - Aerial video distribution system and aerial video distribution method - Google Patents

Abstract

An aerial image data photographed by an unmanned aerial vehicle is distributed in real time regardless of the distance between the unmanned aerial vehicle and a ground station. An aerial video distribution system includes an unmanned aerial vehicle and a ground station, and distributes a video shot from the unmanned aerial vehicle. The unmanned aircraft includes a communication unit that communicates with the ground station via satellite communication, a camera that captures and acquires aerial image data, a transmission unit that transmits the aerial image data to the ground station via the communication unit, Have The ground station communicates with the unmanned aircraft via satellite communication, the receiving unit that receives the aerial image data from the unmanned aircraft by the communication unit, and the video generation that generates the video data including the aerial image data Part. The position information is transmitted from the unmanned aerial vehicle to the ground station, corrected by the ground station, and displayed together with the aerial image. The unmanned aircraft preferably transmits measurement data from various sensors to the ground station in addition to the aerial image, and the ground station also displays the measurement data. [Selection] Figure 1

Description

  The present invention relates to a technique for distributing video shot from an unmanned aerial vehicle.

In recent years, with the increasing resolution of cameras and the spread of wireless communication technology, it has been studied to collect information by taking an image from the sky using an aircraft or performing measurement with a sensor. In particular, information collection using unmanned aerial vehicles (hereinafter also referred to as unmanned aerial vehicles and UAV (Unmanned Aerial Vehicles)) can acquire information safely and efficiently even in disaster areas and accident sites that are difficult to reach with manned aircraft. It has been attracting attention as a means (Patent Document 1). If aerial images and measurement data acquired by the drone are immediately transmitted to the ground station or the like by wireless communication, the ground station can collect information in real time.

JP 2009-223407 A Japanese Patent Laid-Open No. 10-285099

  When data collected by the drone is transmitted to the ground station by wireless communication, the drone cannot transmit the collected data in real time unless the drone is within a range in which radio communication with the ground station is possible. By arranging multiple drones and forming an ad hoc network between these drones and relaying data to the ground station, a wider range of information can be collected. It is necessary to use a machine, and there is a problem that the cost for realizing it is high.

  The present invention has been made in view of the above circumstances, and an object thereof is to distribute aerial image data captured by an unmanned aerial vehicle in real time.

  The present invention is an aerial image distribution system that includes an unmanned aerial vehicle and a ground station and distributes an image captured from the unmanned aircraft. In the present invention, the unmanned aircraft includes a communication unit that communicates with the ground station via satellite communication, a camera that performs shooting to acquire aerial image data, and the aerial image data by the communication unit. And a transmitter for transmitting to the ground station. The ground station includes a communication unit that communicates with the unmanned aircraft via satellite communication, a reception unit that receives the aerial image data from the unmanned aircraft by the communication unit, and the aerial image data. A video generation unit that generates video data.

  According to such a structure, the video image | photographed by the unmanned aircraft can be distributed to the ground station via satellite communication. Note that the video generated in the ground station may be distributed to other devices by the ground station or displayed inside the ground station. Further, since satellite communication can cover the ground over a wide range, it is possible to expand the information collection range by unmanned aerial vehicles simply by mounting a communication device for performing satellite communications on the unmanned aerial vehicle. According to the configuration of the present invention, from the viewpoint of communication range, there is no restriction on the distance between the unmanned aircraft and the ground station in principle.

The unmanned aerial vehicle according to the present invention includes any aircraft as long as it is a vehicle on which a human is not on board. For example, the unmanned aerial vehicle in the present invention includes a fixed wing aircraft, a rotary wing aircraft, an airship, and a balloon. In addition, the form of the satellite communication in the present invention is not particularly limited as long as it is communication via relay by an artificial satellite. For example, the satellite communication in the present invention includes communication using a geostationary satellite, communication using a quasi-zenith satellite, and communication using a non-geostationary satellite (ellipse, middle altitude orbit, low altitude orbit). The ground station in the present invention is a communication facility located on the surface of the earth including the ground and the sea, and includes a mobile facility mounted on a vehicle, a ship, etc. in addition to the fixed facility.

  In the present invention, the unmanned aircraft further includes a position information acquisition unit that acquires position information, and the transmission unit transmits the position information when the aerial image data is acquired together with the aerial image data. It is also preferable that the video generation unit generates video data including the position information together with the aerial image data. The position information acquisition unit can be, for example, a device using a satellite positioning system such as GPS, Galileo, Hokuto. The latitude, longitude, and altitude can be acquired by the satellite positioning system. The altitude may be acquired using an altimeter (including a barometric altimeter and a radio altimeter).

  According to such a configuration, the position of the unmanned aircraft can be constantly monitored. In an unmanned aerial vehicle, when the flight path is set in advance and automatically fly far from the ground station, the position in flight at the ground station is unknown, and it may be difficult to know the acquisition position of the aerial image accurately. is there. Further, if the acquisition position of the aerial image is not known, the aerial image cannot be fully utilized. According to the above configuration, the position information is transmitted via satellite communication, so that the position of the unmanned aircraft, that is, the shooting position of the aerial image can be accurately grasped at the ground station and the distribution destination of the video data. Therefore, it is possible to effectively use the aerial image transmitted from the unmanned aerial vehicle.

  In the present invention, the ground station further includes a geographic information storage unit for storing geographic information, and the video generation unit generates video data representing a position of the unmanned aircraft on a map image. Is preferred. By representing the position of the unmanned aircraft on the map image, the position of the unmanned aircraft can be easily grasped.

  In the present invention, it is also preferable that the video generation unit displays at least one of a linear distance or an altitude difference between the ground station and the unmanned aerial vehicle in video data representing the position of the unmanned aircraft. . In addition, when the flight route or destination of the unmanned aircraft is set, it is also preferable to display the straight line distance and altitude difference between the destination and the unmanned aircraft, the estimated arrival time to the destination, and the like. It is also preferable to display that the unmanned aerial vehicle can return to the ground station (including how much fuel is available) in the position information of the unmanned aircraft and the ground station.

  Further, in the present invention, the ground station is based on a position information acquisition unit that acquires position information, the position information that the ground station has acquired by the position information acquisition unit, and the actual position information of the ground station, It is also preferable to further include a position information correction unit that corrects position information of the unmanned aircraft. The position information may contain an error, but if the position information acquired by the position information acquisition unit includes the same error for both the ground station and the unmanned aircraft, the ground station acquires the position information. The position information of the unmanned aircraft can be corrected based on the difference between the position information acquired by the unit and the actual position information of the ground station. In particular, when the ground station is a fixed station and the actual position information can be accurately grasped, accurate correction can be performed.

Further, in the present invention, the ground station obtains the position information of the unmanned aircraft by comparing the aerial image data and the reference feature with a geographic information storage unit that stores geographic information including an image of the reference feature. It is also preferable to further include a position information correction unit for correction. In the geographic information storage unit, an image when the reference feature is photographed from a specific position is stored, and the reference feature is included in the aerial image data, and the reference feature is located in the aerial image data. The position information of the unmanned aerial vehicle can be accurately estimated based on whether it is included. The reference feature may be any object as long as it is a reference feature (existing on the ground), and includes, for example, a road, a construction, a steel tower, a mountain, a river, and the like.

  In the present invention, the unmanned aircraft further includes a time information acquisition unit that acquires time information, and the transmission unit includes time information when the aerial image data is acquired together with the aerial image data. Preferably, the video generation unit generates video data including the time information together with the aerial image data. When aerial image data or the like is transmitted via satellite communication, there is a difference between the time when the image data is generated at the ground station due to a communication delay and the time when the aerial image that is the source of the image is captured. May occur. By including the time information in the video data, it is possible to grasp when the aerial image is taken. It is also preferable that the video generation unit generates video data including a time difference between the time when the aerial image data is acquired and the video data generation time. Thereby, for example, it becomes possible to grasp how many seconds ago the aerial image was taken. For example, when controlling an unmanned aerial vehicle based on aerial images, information about when the aerial image viewed at the ground station was taken (especially how long before the current time) was taken. Useful information.

  In the present invention, the unmanned aircraft further includes a sensor that acquires measurement data outside the unmanned aircraft, and the transmission unit transmits the measurement data together with the aerial image data to the ground station, It is also preferable that the ground station generates video data including the aerial image data and the measurement data. A sensor is any device that converts the physical and chemical properties of an object into signals and data and outputs them. For example, an infrared sensor, infrared thermography, microwave radar, millimeter wave radar, ultrasonic radar, collection Examples include, but are not limited to, sound devices, radiation sensors, gas sensors, toxic gas detectors, atmospheric measurement devices, magnetic sensors, ion sensors, atmospheric pressure sensors, temperature / humidity sensors, and life exploration devices. Thus, by mounting the sensor and transmitting measurement data obtained from the sensor during flight to the ground station via satellite communication, the measurement data can be observed in real time on the ground. Measurement using an unmanned aerial vehicle is particularly suitable for real-time observation in places where it is difficult to access by manned aircraft, such as places with high radiation concentrations, volcanoes with high toxic gas concentrations, and accident sites.

  In the present invention, the unmanned aerial vehicle determines whether or not the measurement data by the sensor satisfies a predetermined standard, and starts imaging by the camera and transmission by the transmission unit when the standard is satisfied. It is also preferable. In this way, an aerial image can be transmitted when the conditions are met.

  In the present invention, it is preferable that the ground station further includes a video distribution unit that distributes the video data. It is also preferable that the video distribution unit transmits the video data to another information processing apparatus via a data communication network. For example, the communication network may be any data communication network. For example, if video data is distributed using a WAN (Wide Area Network), typically the Internet, the video data is distributed to a long-distance base. Is possible. In addition to the WAN, a data communication network such as a LAN (Local Area Network) or a MAN (Metropolitan Area Network) may be used. A data communication network connected by a dedicated line may be used. However, video data may be distributed by broadcasting without using a data communication network.

  In the present invention, the ground station further includes a camera operation unit that transmits a camera control command for controlling the camera of the unmanned aircraft to the unmanned aircraft by the communication unit. It is also preferable to further include a receiving unit that receives the camera control command from a station and a camera control unit that controls at least one of the shooting direction and the shooting magnification of the camera according to the camera control command.

  Further, in the present invention, the ground station further includes an airframe operation unit that transmits a flight control command for controlling the unmanned aircraft to the unmanned aircraft by the communication unit, and the unmanned aircraft is transmitted from the ground station. It is also preferable to further include a receiving unit that receives the flight control command and an airframe control unit that controls the flight of the drone according to the flight control command.

  In this way, by transmitting camera control commands and flight control commands from the ground station to the unmanned aircraft via satellite communication, real-time operation can be performed from the ground station. Since operation is possible while viewing aerial images transmitted in real time from an unmanned aerial vehicle, it is highly convenient. In particular, when position information and time information are displayed together with an aerial image, more advanced control is possible from the ground station.

  Further, the present invention can be understood as an aerial image distribution system, an unmanned aerial vehicle, or a ground station including at least a part of the above means. The present invention also regards an aerial video distribution method including at least a part of the above processing, a program for causing a computer to execute the method, or a non-temporarily stored computer-readable recording medium. You can also. Each of the above configurations and processes can be combined with each other as long as no technical contradiction occurs.

  According to the present invention, aerial image data photographed by an unmanned aerial vehicle can be distributed in real time regardless of the distance between the unmanned aerial vehicle and a ground station.

FIG. 1 is a diagram showing a system overview of an aerial video distribution system according to the first and second embodiments. FIG. 2 is a diagram illustrating functional blocks of the unmanned aerial vehicle and the ground station according to the first embodiment. FIG. 3 is a flowchart showing the flow of the aerial video distribution process in the first embodiment. FIG. 4 is a diagram illustrating an example of a distribution video in the first embodiment. FIG. 5 is a diagram illustrating functional blocks of the unmanned aircraft and the ground station in the second embodiment. FIG. 6 is a flowchart showing the flow of aerial video distribution processing in the second embodiment. FIG. 7 is a diagram illustrating an example of a distribution video in the second embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be exemplarily described in detail with reference to the drawings. However, the dimensions, materials, shapes, relative arrangements, and the like of the components described in this embodiment are not intended to limit the scope of the present invention only to those unless otherwise specified.

(First embodiment)
<System overview>
The first embodiment of the present invention is an aerial video distribution system that distributes images taken by an unmanned aerial vehicle (hereinafter referred to as UAV: Unmanned Aerial Vehicle or unmanned aerial vehicle) by aerial imaging in real time. FIG. 1 is a diagram showing an overview of an aerial video distribution system according to the present embodiment.

  The aerial video distribution system includes a drone 1, a communication satellite 2, a ground station 3, and an end user facility 4. Images taken aerial by the drone 1 are transmitted to the ground station 3 by satellite communication and distributed from the ground station to the end user equipment 4. Since the aerial image is transmitted via the satellite communication, the aerial image can be transmitted in real time even when the drone 1 is far away from the ground station 3. In addition, in order to perform satellite communication, it is only necessary to install an antenna and a communication device for satellite communication in the unmanned aircraft 1, so real-time distribution of aerial images can be realized at low cost.

  In the aerial video distribution system according to the present embodiment, the drone 1 transmits position information and time information together with the aerial image to the ground station 3 via satellite communication. By transmitting the position information, the position of the drone 1 can be constantly monitored in the ground station 3. Also, when the time information is transmitted together, even when a delay due to satellite communication occurs, the aerial image transmitted from the drone 1 is taken when (for example, how many seconds ago) It becomes possible to grasp whether it is.

  In the aerial video distribution system according to the present embodiment, a flight control instruction or an aerial camera control instruction can be transmitted from the ground station 3 to the drone 1 via satellite communication. As a result, it becomes possible to cope with when it is necessary to avoid emergency or change the flight path, or when it is desired to change the shooting direction or shooting magnification of aerial photography.

<Configuration>
Hereinafter, the configuration of the aerial video distribution system according to the present embodiment will be described in more detail with reference to FIG. FIG. 2 is a diagram showing functional blocks of the drone 1 and the ground station 3.

[Drone 1]
The drone 1 is an arbitrary flying body capable of autonomous flight without a human being boarding. The drone 1 includes a fixed wing aircraft, a rotary wing aircraft, an airship, a balloon, and the like. In addition, as long as a device that provides the functions described below can be mounted, the size and type of the device are not particularly limited. For example, as the drone 1, a fixed-wing small drone equipped with a gasoline engine having a main wing width of about 3 m and about 90 cc as power can be adopted.

  As shown in FIG. 2, the drone 1 includes a satellite communication antenna 101, a satellite communication device 102, a data transmission unit 103, a data reception unit 104, a camera 105, a camera control unit 106, an airframe control unit 107, a measuring instrument 108, A GPS antenna 109, a GPS device 110, and the like are included.

  The satellite communication device 102 is a device that communicates with the communication satellite 2 via the satellite communication antenna 101. The satellite communication device 102 receives transmission data from the data transmission unit 103, performs encoding processing, modulation processing, and the like, and transmits the data to the communication satellite 2 as a radio signal. Further, the satellite communication device 102 receives a radio signal from the communication satellite 2, performs demodulation processing, decoding processing, and the like, and outputs the received data to the data reception unit 104. The communication satellite 2 is a relay station, and the drone 1 performs wireless communication with the ground station 3 via the communication satellite 2.

  The data transmission unit 103 is a functional unit that transmits aerial image data acquired by the camera 105, position information and time information acquired by the GPS device 110, various data acquired by the measuring instrument 108, and the like to the ground station 3. It is. The data transmission unit 103 outputs transmission data to the satellite communication device 102 and transmits the data to the ground station 3 via satellite communication.

  The data receiving unit 104 is a functional unit that receives data transmitted from the ground station 3. The data receiving unit 104 acquires data transmitted from the ground station 3 via satellite communication from the satellite communication device 102. The data transmitted from the ground station 3 includes a flight control command for controlling the drone 1, a camera control command for controlling the camera 105, and the like.

  The camera 105 is an image pickup apparatus that takes an image of the ground surface from the drone 1 (aerial photography). The camera 105 may be a visible light camera that captures visible light, or may be an infrared camera that captures infrared light. The camera 105 may shoot a still image or a moving image. When taking a still image, the image may be taken periodically or irregularly, or only when a specific condition is satisfied. The camera 105 can change its shooting direction, shooting magnification, image quality, and the like. The image data acquired by the camera 105 is transmitted from the data transmission unit 103 to the ground station 3. The image data acquired by the camera 105 is preferably stored in a storage device (not shown) in the drone 1. It is also preferable to perform compression processing on the image data before transmission to the ground station 3 or storage in the storage device.

  The camera control unit 106 controls the execution of shooting by the camera 105 according to a flight plan (flight program) input in advance and a camera control command transmitted from the ground station 3, and the shooting direction and shooting magnification at the time of shooting. , Change settings such as image quality. The flight plan is setting data that defines the flight path of the drone 1 and what kind of aerial image is acquired at which position, and is stored in advance in a storage device (not shown). The camera control unit 106 acquires a flight plan from the storage device and controls the camera 105. Even when the camera control command transmitted from the ground station 3 is received from the data receiving unit 104, the camera 105 is controlled according to the camera control command.

  The body control unit 107 controls the propulsion system and the steering system (both not shown) of the unmanned aerial vehicle 1 according to the flight plan input in advance and the flight control command transmitted from the ground station. For example, the airframe control unit 107 uses the specified altitude / coordinate based on the flight path specified in the flight plan, the position information acquired by the GPS device 110, various airframe information acquired from the measuring instrument 108, and the like. Control the propulsion system and steering system so that you can fly.

  The measuring instrument 108 is a device that acquires the state of the drone 1. The drone 1 has a plurality of types of measuring instruments 108. The measuring instrument 108 includes, for example, an engine speed, an attitude sensor, an altimeter (barometric altimeter or radio altimeter), an acceleration sensor, an airspeed meter, an azimuth meter (magnetic compass or gyrocompass), and the like.

The GPS device 110 is a device that acquires position information and time information using a global positioning system. The GPS device 110 acquires position information (latitude / longitude / altitude) and time information based on radio waves transmitted from GPS satellites via the GPS antenna 109. Therefore, the GPS device 110 serves as a position information acquisition unit and a time information acquisition unit. Note that a satellite positioning system other than GPS (for example, Galileo or Hokuto) may be used as long as position information can be acquired. Further, the position information and the time information may be acquired by different apparatuses. For example, time information may be acquired using a radio clock.

  The drone 1 may have functional units other than those described here. For example, the drone 1 may have a wireless communication device that performs direct wireless communication with the ground station 3. In this wireless communication apparatus, wireless communication can be performed only within the line-of-sight distance. However, when the distance is within the line-of-sight distance, communication with less delay than satellite communication can be realized.

Part or all of the functional units of the drone 1 may be realized by a computer (information processing apparatus) that includes a microprocessor and a storage device and executes a computer program stored in the storage device. Part or all of the functional units of the drone 1 may be realized by a dedicated hardware circuit or the like, or may be realized by a combination of a dedicated hardware circuit and a computer.

[Communication satellite 2]
The communication satellite 2 is a communication device that flies in outer space in order to relay wireless communication. As long as the communication satellite 2 can relay the communication between the drone 1 and the ground station 3, any communication satellite can be adopted. The communication satellite 2 can typically use a geostationary satellite that revolves in a geostationary orbit, but may also use a quasi-zenith satellite, a non-geostationary satellite, or the like.

[Ground station]
The ground station 3 is a facility that is installed on the ground and includes a satellite communication device that performs wireless communication with the communication satellite 2 and a distribution device that distributes aerial image data obtained from the drone 1 to the end user facility 4. The ground station 3 is typically installed on the ground surface, but may be installed at a position away from the ground surface or installed on the sea. The ground station 3 may be a fixed facility that is geographically fixed, or may be a moving facility that is mounted on a vehicle or a ship.

  As shown in FIG. 2, the ground station 3 includes a satellite communication antenna 301, a satellite communication device 302, a data transmission unit 303, a data reception unit 304, a UAV aircraft operation unit 305, a UAV camera operation unit 306, a GPS antenna 307, A GPS device 308, a data correction unit 309, a geographic information storage unit 310, a distribution video generation unit 311, a video distribution unit 312, a display device 313, and the like are included.

  The satellite communication device 302 is a device that performs communication with the communication satellite 2 via the satellite communication antenna 301. The satellite communication device 302 receives transmission data from the data transmission unit 303, performs encoding processing, modulation processing, and the like, and transmits the data to the communication satellite 2 as a radio signal. The satellite communication device 302 receives a radio signal from the communication satellite 2, performs demodulation processing, decoding processing, and the like, and outputs the received data to the data reception unit 304. The communication satellite 2 is a relay station, and the ground station 3 performs wireless communication with the drone 1 via the communication satellite 2.

  The data transmission unit 303 is a functional unit that transmits flight control commands, camera control commands, and the like to the drone 1. The data transmission unit 303 receives a control instruction for the drone 1 from the UAV body operation unit 305 and the UAV camera operation unit 306, outputs transmission data to the satellite communication device 302, and transmits data to the drone 1 via satellite communication. Send.

  The data receiving unit 304 is a functional unit that receives data transmitted from the drone 1. The data receiving unit 304 acquires data transmitted from the drone 1 via satellite communication from the satellite communication device 302. The data transmitted from the drone 1 includes aerial image data captured by the camera 105 of the drone 1, position information, time information, state information, and the like of the drone 1.

  The UAV aircraft operation unit 305 is a functional unit that issues a flight control command to the drone 1. The operator of the ground station 3 can instruct the unmanned aerial vehicle 1 using the UAV body operation unit 305 for, for example, changing the flight route, emergency avoidance, and landing return. Further, the UAV machine body operation unit 305 may issue an instruction to the propulsion system or the steering system of the drone 1 so that the operator can remotely control the drone 1. The control command issued by the UAV machine operation unit 305 is output to the data transmission unit 303 and transmitted to the unmanned aircraft 1 by satellite communication.

  The UAV camera operation unit 306 is a functional unit that issues a camera control command for the camera 105 of the drone 1. The operator of the ground station 3 can use the UAV camera operation unit 306 to instruct the drone 1 to, for example, pan / tilt / zoom in / zoom out of the camera, change the image quality, and the like. The control command issued by the UAV camera operation unit 306 is output to the data transmission unit 303 and transmitted to the drone 1 by satellite communication.

  The GPS device 308 receives a GPS satellite signal from the GPS antenna 307 and acquires position information and time information.

  The data correction unit 309 is a functional unit that corrects position information received from the drone 1. The position information acquired by the drone 1 may include an error. When acquiring position information from GPS satellite signals, errors caused by the GPS system (orbit information errors transmitted from GPS satellites, GPS satellite time synchronization errors), and errors caused by atmospheric propagation (ionosphere) Delay and tropospheric delay) and errors due to the receiver. The error caused by the system and the error caused by atmospheric propagation are included in the position information of the drone 1 and the ground station 3 in common. Since the position of the ground station 3 is known in advance, the above error can be grasped from the position information obtained from the GPS device 308 and the actual position of the ground station 3. By removing this error from the position information transmitted from the drone 1, a more accurate position of the drone 1 can be acquired. Even when the altitude is acquired using the altimeter, the altitude of the drone 1 can be corrected using the altitude acquired from the altimeter in the ground station 3 and the actual altitude of the ground station 3.

  The data correction unit 309 can also correct the position information received from the drone 1 by a method different from the above. That is, a reference feature serving as a checkpoint on the ground is stored in advance in the geographic information storage unit 310 as geographic information, and based on the checkpoint reflected in the aerial image data transmitted from the drone 1, The position information can also be corrected. Here, the check point may be any object as long as it is a feature that can be identified by an image, and includes, for example, a road, a construction, a steel tower, a mountain, a river, and the like. If the position and moving direction (shooting direction) of the drone 1 are known, the check points included in the aerial image and the positions of the check points in the aerial image can be estimated. Conversely, if there is no checkpoint at the estimated position, the position of the drone 1 can be calculated from the shift in the position. Therefore, the data correction unit 309 can correct the position information of the drone 1 from the aerial image data received from the drone 1 and the geographic information stored in the geographic information storage unit 310.

  The data correction unit 309 may employ either of the above two position correction methods or both. When both are employed, the average of the position information obtained by both methods can be used as the position information of the drone 1 after correction. The position information of the drone 1 corrected by the data correction unit 309 is preferably transmitted to the drone 1 via satellite communication.

  The geographic information storage unit 310 stores various types of geographic information (also called geospatial information, spatial information, etc.). The geographic information includes map information and information about features stored in association with the map information. In addition, it is also preferable to store an image of the feature in the geographic information in order to identify the feature that serves as a check point from the aerial image.

The distribution video generation unit 311 is a functional unit that generates video data to be distributed to the end user equipment 4 based on an aerial image transmitted from the drone 1. The distribution video data includes at least an aerial image transmitted from the drone 1, and also includes information associated with the aerial image, information regarding the state of the drone 1, and the like. For example, the distribution video generation unit 311 generates a video including information related to the time when the aerial image is captured. Since the time when the aerial image was captured is transmitted from the drone 1, the distribution video generation unit 311 determines the time when the aerial image was captured and the shooting time from the current time (distribution video data generation time). Video data including information related to the time difference between the shooting time and the current time is generated. The distribution video generation unit 311 also generates a video representing the position and flight direction of the drone 1 on a map image based on the position information, posture information, and the like of the drone 1 and geographic information (map information). Also, the distribution image should include the straight line distance and altitude difference between the drone 1 and the ground station 3, or information such as whether the drone 1 can return to the ground station 3 and the cruising range. Is also preferable. On this map image, it is also preferable to represent the shooting range of the aerial image based on information such as the shooting direction of the camera 105 and the shooting magnification. Examples of distribution images will be described later.

  The video distribution unit 312 is a functional unit that distributes the video data generated by the distribution video generation unit 311 to the end user equipment 4. The distribution method of the video data by the video distribution unit 312 may be arbitrary, and can be distributed using, for example, a data communication network via a WAN such as the Internet. Alternatively, the video distribution unit 312 can distribute video by broadcasting. At the time of video distribution, user authentication processing or billing processing may be performed.

  The display device 313 is a display device that receives and displays the video data generated by the distribution video generation unit 311. Since the display device 313 can check the status information and aerial images of the drone 1, it can be referred to when the drone 1 is operated and the camera is operated.

  Although an example in which all the above information is distributed as video has been described here, data other than images such as aerial images and map images is not included in the distributed video and is transmitted to the end user equipment 4 as separate data. May be. In this case, it is preferable for the end user equipment 4 to acquire and display each data collectively.

  Part or all of the functional units of the ground station 3 may be realized by a computer (information processing apparatus) that includes a microprocessor and a storage device and executes a computer program stored in the storage device. Further, part or all of the functional units of the ground station 3 may be realized by a dedicated hardware circuit or the like, or may be realized by a combination of a dedicated hardware circuit and a computer.

[End user equipment 4]
The end user equipment 4 includes a device for acquiring and displaying video distributed from the ground station 3. In a form in which video is transmitted by data communication, the end user equipment 4 can be configured as a computer (information processing apparatus) including a microprocessor, a storage device, a display device, a communication interface, and the like. Further, in a form in which video is transmitted by broadcasting, the end user equipment 4 can be configured as a video display device that receives and displays broadcast radio waves.

<Processing>
Next, the flow of aerial video distribution processing in the aerial video distribution system according to the present embodiment will be described with reference to the flowchart of FIG.

  First, before the flight of the drone 1 is started, a flight plan of the drone 1 is created in the ground station 3 and stored in the drone 1 (S1 and S2). The flight plan (flight program) designates a flight path by the drone 1. In the present embodiment, the flight plan is one in which a point (waypoint) through which the drone 1 should pass is designated by three-dimensional (latitude / longitude / altitude) coordinates. The flight plan may include a shooting point of an aerial image and a shooting direction. For example, the flight plan includes an aerial shooting start position and an end position. However, it may be set so that aerial photography is always performed during flight.

When the flight plan is stored, the drone 1 performs automatic flight (autonomous flight) according to the flight plan (S3). Here, description will be made assuming that the aerial shooting is performed only at a designated place (may be one point or a route). Until the aerial shooting start point is reached, shooting by the camera 105 or transmission of an aerial image is not performed. However, an operation may be performed in which shooting is performed by the camera 105 and shooting data is recorded, but transmission to the ground station 3 is not performed. Further, even while aerial photography is not being performed, position information, information on the state of the drone 1 obtained from the measuring instrument 108, and the like are transmitted to the ground station 3 via satellite communication.

  The drone 1 knows its own position based on the position information acquired from the GPS device, and performs flight while confirming whether it has reached the aerial shooting start point. When it is determined that the drone 1 has reached the aerial shooting start point (S4), aerial shooting by the camera 105 is started, and aerial image data, position information, time information, and the like are transmitted to the ground station 3 via satellite communication. Transmit (S5). These data are transmitted from the satellite communication device 102 of the drone 1 to the communication satellite 2, and are relayed by the communication satellite 2 and received by the satellite communication device 302 of the ground station 3 (S6). When the ground station 3 receives aerial image data, position information, time information, etc., it first corrects the position information of the drone 1 (S7). In the correction process, the correction based on the difference between the GPS position information of the ground station 3 and the actual position information of the ground station 3 and the comparison between the reference feature (checkpoint) included in the aerial image and the geographic information. Both corrections based on the above are performed, and the position information of the drone 1 is corrected using the average. Next, the ground station 3 generates a distribution video including an aerial image, position information, and time information (S8). In addition to the aerial image, the generated distribution image includes a map image representing the position and traveling direction of the drone 1 and a shooting direction of the aerial image, an acquisition time of the aerial image (absolute time and distribution image generation time) Information), a linear distance between the drone 1 and the ground station 3, an altitude difference, whether the drone 1 can return, and the like. The generated video is transmitted from the ground station 3 to the end user equipment 4 via the network (S9), and the end user can check an aerial image of the drone 1 at an arbitrary base away from the ground station 3. .

  Note that, during the period in which the drone 1 is designated to perform aerial shooting, shooting of aerial images, transmission to the ground station, and generation and distribution of video at the ground station are continuously performed. Moreover, the correction process of the position information of the drone 1 in the ground station 3 may be executed continuously, or may be executed at the timing when the drone 1 passes the check point.

  Although not shown in the flowchart of FIG. 3, it is possible to control the drone 1 and the camera 105 by issuing a flight control command and a camera control command from the ground station 3 to the drone 1.

<Example of distribution video>
With reference to FIG. 4, an example of a distribution video generated by the ground station 3 will be described. The distribution video includes a map image 40, an aerial image 50, and a reference information column 60.

  On the map image 40, the position 41 of the drone 1 and the position 42 of the ground station 3 are superimposed and displayed. Further, the flight direction (traveling direction) 43 of the drone 1 and the aerial shooting range 44 are also displayed superimposed on the map image. The aerial image 50 displays an aerial image transmitted from the drone 1. The aerial image 50 is superimposed with time information 51 representing the difference between the time when this image was taken and the current time (distributed video generation time). In the reference information column 60, various information related to the drone 1, the aerial image, and the like are displayed. For example, in the reference information column 60, the position information (latitude / longitude / altitude) of the drone 1, the linear distance (three-dimensional distance), horizontal distance, altitude difference between the drone 1 and the ground station 3, Information on whether or not the drone 1 can return to the ground station 3, information on the cruising range, the flight route of the drone 1, information on equipment mounted on the drone 1, and the like are included.

  The distribution video shown in FIG. 4 is merely an example, and what kind of video is specifically generated as the information may be arbitrary. For example, the time information 51 may be displayed in the reference information column 60 without being superimposed on the distribution image, or the position coordinates of the drone 1 may be superimposed on the map image 40 and displayed. In addition, the map image and the aerial image may not be distributed as one video, but may be distributed as different videos, or the map image and the aerial image may be selected and distributed according to a request from the end user equipment 4. You may do it.

<Effect of this embodiment>
According to the aerial image distribution system according to the present embodiment, an aerial image can be received from the unmanned aircraft 1 in real time via satellite communication at the ground station 3 and immediately distributed to the end user equipment 4. Here, since satellite communication is used for communication between the drone 1 and the ground station 3, the operation range of the drone 1 can be widened, and the operation range is not limited as far as communication is concerned. Conventionally, when the drone 1 and the ground station 3 communicate directly, the drone 1 can be operated only within the communication range. When the ad hoc network is formed and relayed, the relay drone etc. There was a problem such as becoming necessary. However, according to the present embodiment, the range in which aerial images can be acquired can be expanded simply by installing the satellite communication device in the drone 1.

  In addition, when the drone 1 is caused to fly automatically, the position in flight may be unknown in the ground station 3, but by acquiring position information from the drone 1 via satellite communication, the ground station 3 It becomes possible to grasp the position of the drone 1 in FIG. Moreover, the position information of the drone 1 can be accurately grasped by performing the correction process on the position information of the drone 1.

  In addition, when an aerial image or other information is transmitted from the drone 1 to the ground station 3 via satellite communication, a time lag due to communication delay may occur. Since the ground station 3 can grasp when the received information is information based on the time information transmitted from the drone 1, the information transmitted from the drone 1 is effectively used. The delay in distribution between the ground station 3 and the end user equipment 4 can be ignored.

  Further, since the ground station 3 can acquire the position information, the body information, and the aerial image of the drone 1 in real time and can control the drone 1 from the ground station 3, the operator can control the drone 1 based on the above information. It is also possible to perform appropriate control for the drone 1. At this time, as described above, the information transmitted from the drone 1 is displayed so as to be able to grasp the information at the time, so that the operator can display the position information and the aerial image of the drone 1 being displayed. To make more appropriate decisions.

  In addition, by distributing video from the ground station 3 to the end user equipment 4, the aerial image acquired by the drone 1 can be viewed at an arbitrary point.

(Second Embodiment)
In the first embodiment, an aerial image is transmitted from the drone 1, but in this embodiment, in addition to the aerial image, the measurement data from the sensor is also transmitted from the drone 1. In the following description, in the configuration and processing of the present embodiment, the description of the same parts as in the first embodiment will be omitted, and the parts different from the first embodiment will be mainly described.

  FIG. 5 is a diagram showing a functional configuration of the drone 1 and the ground station 3 according to the present embodiment. The present embodiment is different from the first embodiment in that a sensor 111 is provided in the drone 1. The sensor 111 acquires information related to the environment outside the drone 1. The sensor 111 may be any device as long as it is a device that converts the physical and chemical properties of the object into signals and data and outputs them. For example, as the sensor 111, an infrared sensor, an infrared thermography, a microwave radar, a millimeter wave radar, an ultrasonic radar, a sound collection device, a radiation sensor, a gas sensor, a toxic gas detector, an atmospheric measurement device, a magnetic sensor, an ion sensor, an atmospheric pressure sensor Temperature / humidity sensors, life exploration devices, etc. The measurement data acquired by the sensor 111 is stored in a storage device (not shown) in the drone 1 in association with position information and time information at the time of measurement, and to the ground station 3 via the satellite communication device 302. It can be sent.

In the present embodiment, measurement data obtained by the sensor 111 together with the aerial image is transmitted from the drone 1 to the ground station 3 to enable display on the ground station 3 and distribution to the end user equipment 4. Here, the period during which the sensor 111 performs the measurement can be matched with the period during which the aerial image is performed. For example, an aerial image and measurement data (and other information) may always be acquired and transmitted to the ground station 3 during flight, or a location for acquiring an aerial image and measurement data may be set in advance. You may make it acquire and transmit an aerial image and measurement data in the location. Alternatively, measurement by the sensor 111 is always performed during the flight, and when the measurement data satisfies a predetermined condition, an aerial image is automatically captured and transmitted to the ground station. Examples of the predetermined condition include a case where the detection amount of radiation dose or toxic gas exceeds a predetermined value, or a case where a person is detected by a human life exploration device, but is not limited to this. Conditions can be adopted.

  FIG. 6 is a diagram showing the flow of processing in the aerial video distribution system according to the present embodiment. First, before the flight of the drone 1 is started, a flight plan of the drone 1 is created in the ground station 3 and stored in the drone 1 (S11 and S12). The flight plan specifies a flight path by the unmanned aerial vehicle 1. In the present embodiment, a condition for starting capturing and transmitting an aerial image is also included in the flight plan. As a start condition for capturing and transmitting an aerial image, the measurement data obtained by the sensor 111 satisfies a predetermined condition as described above.

  When the flight plan is stored, the drone 1 automatically flies according to the flight plan (S13). During the automatic flight, the drone 1 performs measurement by the sensor 111. The measurement data by the sensor 111 may be stored in the storage device in the unmanned aircraft 1 together with the position information of the unmanned aircraft 1 and other information, or may be sent to the ground station 3 via satellite communication. When the measurement data obtained by the sensor 111 is acquired, the drone 1 determines whether or not the measurement data satisfies the aerial shooting start condition (S15).

  If the measurement data satisfies the conditions for starting aerial photography, the drone 1 starts aerial photography and transmits aerial image data to the ground station 3 via satellite communication (S16). Processes S16 to S21 after the aerial photography is started are the same as those in the first embodiment except that the measurement data of the sensor 111 is transmitted together and the distribution video includes the measurement data. Is omitted.

  FIG. 7 is an example of a distribution video generated by the ground station 3 in the present embodiment. Here, unlike the first embodiment, an example in which two types of video shown in FIGS. 7A and 7B are switched and delivered will be described. However, these pieces of information are generated as one video. It doesn't matter.

  The distribution video shown in FIG. 7A includes a map image 40 including the position of the drone 1, a sensor measurement data display column 70, and a reference information column 60. The information displayed in the map image 40 and the reference information column 60 is the same as in the first embodiment. In the measurement data display column 70, measurement data measured by the sensor of the drone 1 is displayed. In this example, the measurement data of the sensor is displayed only with the character information, but it may be displayed graphically by superimposing it with the map information.

  The distribution video shown in FIG. 7B includes a map image 40 including the position of the drone 1, an aerial image 50 taken by the drone 1, and a reference information column 60. These pieces of information are the same as those in the first embodiment. 7A and 7B can be switched by an instruction from the ground station 3 or the end user equipment 4. In addition, during the period when the drone 1 performs only the measurement by the sensor and does not perform the aerial shooting, only the image of FIG. 7A is generated, and when the aerial shooting starts, the image is automatically switched to the image of FIG. 7B. Thereafter, the user may arbitrarily switch the display.

According to this embodiment, the drone 1 can transmit measurement data by a sensor over a wide range in real time. Because information is collected by unmanned aircraft, it is possible to collect information even at accident sites and disaster sites where it is difficult for humans to fly.

  By using this embodiment, for example, it can be configured as a survivor search system in an accident site or a disaster site. A rotary wing aircraft (helicopter) is adopted as an unmanned aerial vehicle, and a human life exploration device is installed in addition to an aerial camera. There are various types of human life exploration devices such as an electromagnetic wave type, an acoustic type, and a thermal sensor type. With these sensors, it is possible to search even at night and to search for survivors in the building. The drone starts aerial photography triggered by the detection of a person by the human life exploration device. That is, when a survivor is discovered by the drone, an aerial image is transmitted to the ground station together with measurement data and position information. Along with this, the ground station can know the possibility of survivor discovery. The aerial image is preferably a visible image in the daytime and an infrared image in the nighttime. The ground station generates an image including an aerial image and distributes it to the rescue center. In the rescue center, the aerial image is checked to confirm whether the survivor's discovery by the human life exploration device is a false detection. At this time, the drone and its camera can be controlled as necessary to obtain more detailed information about the site. Thus, after confirming that the survivor discovery by the human life exploration device is not a false detection, it is possible to dispatch a rescue vehicle, a manned aircraft, and the like to the site. With such a system, survivor exploration can be safely carried out even at dangerous sites and at night, and rescue vehicles and the like can be dispatched on site after confirming that the detection by the life exploration device is not an error. In this example, the survivor search system at the accident site or the like has been described as an example. However, by using the same configuration, it is possible to search the escape person from the sky. However, the person search system is only a specific example to which this embodiment can be applied, and this embodiment can be applied to any system.

(Other variations)
The configuration of the above-described embodiment is merely a specific example of the present invention, and is not intended to limit the scope of the present invention. The present invention can take various specific configurations without departing from the technical idea thereof.

  For example, in the above-described embodiment, the video generated in the ground station is distributed to the end user equipment. However, the video may be displayed on the ground station without being distributed to the end user equipment.

  In addition, although the drone is controlled only from the ground station and is not controlled from the end user equipment, the drone may be controlled from the end user equipment. Moreover, it is also preferable to employ | adopt the vehicle carrying the function mentioned above instead of fixed equipment as a ground station.

  Further, although the position information is corrected in the ground station, the position information may be corrected in the unmanned aircraft. That is, the position information measured by the ground station using the GPS device is transmitted to the drone, and the drone acquired by the drone based on the GPS position information of the ground station and the actual position information of the ground station. The position information may be corrected. In addition, the drone may correct the position information of the own machine based on the aerial image and the checkpoint image.

1: Unmanned aerial vehicle (unmanned aircraft, UAV) 2: Communication satellite 3: Ground station 4: End user equipment 101: Antenna for satellite communication 102: Satellite communication device 103: Data transmission unit 104: Data reception unit 105: Camera 106: Camera Control unit 107: Airframe control unit 108: Measuring instrument 109: GPS antenna 110: GPS device 111: Sensor 301: Satellite communication antenna 302: Satellite communication device 303: Data transmission unit 304: Data reception unit 305: UAV body operation unit 306: UAV camera operation unit 307: GPS antenna 308: GPS device 309: Data correction unit 310: Geographic information storage unit 311: Distribution video generation unit 312: Video distribution unit

Claims (13)

An aerial video distribution system that consists of unmanned aerial vehicles and ground stations and distributes images taken from unmanned aircraft,
The unmanned aircraft
A communication unit that communicates with the ground station via satellite communication;
A camera that captures aerial image data by shooting,
A time information acquisition unit for acquiring time information;
A transmitter that transmits the aerial image data and time information when the aerial image data is acquired to the ground station by the communication unit;
Have
The ground station is
A communication unit for communicating with the unmanned aerial vehicle via satellite communication;
A receiver that receives the aerial image data and the time information from the unmanned aerial vehicle through the communication unit;
A video generator for generating a movies image data,
I have a,
The video data includes the aerial image data, a time difference between the time when the aerial image data was acquired and the video data generation time,
This is an aerial video distribution system. The unmanned aerial vehicle further includes a position information acquisition unit that acquires position information,
The transmission unit transmits the position information when the aerial image data is acquired together with the aerial image data,
The video generation unit generates video data including the position information together with the aerial image data.
The aerial video distribution system according to claim 1. The ground station further includes a geographic information storage unit for storing geographic information,
The video generation unit generates video data representing a position of the unmanned aerial vehicle on a map image;
The aerial video distribution system according to claim 2. The video generation unit displays at least one of a linear distance or an altitude difference between the ground station and the unmanned aircraft in video data representing the position of the unmanned aircraft.
The aerial video distribution system according to claim 2 or 3. The ground station is
A location information acquisition unit for acquiring location information;
A position information correction unit that corrects the position information of the unmanned aircraft based on the position information acquired by the position information acquisition unit by the ground station and the actual position information of the ground station;
The aerial video distribution system according to any one of claims 2 to 4, further comprising: The ground station is
A geographic information storage unit for storing geographic information including images of reference features;
A position information correction unit that corrects position information of the unmanned aerial vehicle by comparing the aerial image data and reference features;
The aerial video distribution system according to any one of claims 2 to 5, further comprising: The unmanned aerial vehicle further includes a sensor that acquires measurement data outside the unmanned aerial vehicle,
The transmitter transmits the measurement data together with the aerial image data to the ground station,
The ground station generates video data including the aerial image data and the measurement data;
The aerial video distribution system according to any one of claims 1 to 6 . The unmanned aerial vehicle determines whether or not the measurement data by the sensor satisfies a predetermined criterion, and when the criterion is satisfied, starts photographing by the camera and transmission by the transmission unit.
The aerial video distribution system according to claim 7 . The ground station further includes a video distribution unit that distributes the video data.
The aerial video distribution system according to any one of claims 1 to 8 . The video distribution unit distributes the video data to another information processing apparatus via a data communication network;
The aerial video distribution system according to claim 9 . The ground station further includes a camera operation unit that transmits a camera control command for controlling the camera of the unmanned aircraft to the unmanned aircraft by the communication unit,
The unmanned aircraft
A receiving unit for receiving the camera control command from the ground station;
In accordance with the camera control command, a camera control unit that controls at least one of a shooting direction or a shooting magnification of the camera;
The aerial video distribution system according to any one of claims 1 to 10 , further comprising: The ground station further includes a fuselage operation unit that transmits a flight control command for controlling the unmanned aircraft to the unmanned aircraft by the communication unit,
The unmanned aircraft
A receiver for receiving the flight control command from the ground station;
An airframe control unit for controlling the flight of the unmanned aircraft according to the flight control command;
The aerial video distribution system according to any one of claims 1 to 11 , further comprising: An aerial image distribution method in an aerial image distribution system composed of unmanned aerial vehicles and ground stations,
The unmanned aerial vehicle captures aerial image data using a camera,
A time information acquisition step in which the unmanned aircraft acquires time information; and
The unmanned aircraft transmits the aerial image data and the time information when the aerial image data was acquired to the ground station via satellite communication, and
A receiving step in which the ground station receives the aerial image data and the time information via satellite communication;
The ground station is a video generation step of generating video data, wherein the video data includes the aerial image data, a time difference between the time when the aerial image data was acquired and the time when the video data was generated. Including a video generation step ,
Aerial video delivery method.

Images