JP2018129577A - Imaging system, control method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an appropriate image even when vibration occurs in an unmanned aerial vehicle equipped with an imaging device .SOLUTION: An imaging system having an unmanned aerial vehicle including a first imaging device and a second imaging device different from the first imaging device obtains data of images captured by the first imaging device and the second imaging device. Next, the imaging system receives data on vibration of the unmanned aerial vehicle and performs control for transmitting the data of images captured by the first imaging device and the second imaging device. The control for transmitting the image data is performed such that one of the image data captured by the first imaging device and the second imaging device is transmitted according to the received data on vibration.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an imaging system that processes image data captured by an unmanned aerial vehicle, a control method thereof, and a program.

  2. Description of the Related Art Conventionally, a system that transmits an image captured by an imaging device in real time is known. The image is used, for example, for creating a television broadcast video or map data. In recent years, a relay system capable of providing an image with a higher degree of freedom than conventional systems by photographing with an unmanned aerial vehicle equipped with a photographing device has been studied.

  For example, Patent Document 1 discloses a relay system that uses an image taken from the sky by an unmanned aircraft for news reports such as television news.

JP, 2006-225983, A

  By the way, since the unmanned aerial vehicle performs shooting while flying over the sky, it is susceptible to external influences such as changes in wind and atmospheric pressure. However, Patent Document 1 does not discuss any measures against the external influence. For this reason, the unmanned aerial vehicle may be affected by vibrations, so that the quality of the captured image may be deteriorated. In particular, when an image is provided in real time such as on a television broadcast, there is a possibility that an appropriate video cannot be provided to the viewer.

  Therefore, an object of the present invention is to provide an appropriate image even when vibration occurs in an unmanned aerial vehicle equipped with an imaging device.

  The present invention is an imaging system including an unmanned aircraft including a first imaging device and a second imaging device different from the first imaging device, wherein the first imaging device and the second imaging device An acquisition unit that acquires image data captured by the imaging device, a reception unit that receives data relating to vibrations of the unmanned aircraft, and image data captured by the first imaging device and the second imaging device are transmitted. Control means for performing control for the image data captured by the first photographing apparatus and the second photographing apparatus according to data relating to vibration received by the receiving means. Of these, control is performed to transmit any one of the image data.

  According to the present invention, there is an effect that it is possible to provide an appropriate image even when shaking occurs during shooting of an image using an unmanned aerial vehicle.

It is a figure which shows an example of the system configuration | structure of an imaging | photography system in embodiment of this invention. 2 is a diagram illustrating an example of a hardware configuration of an unmanned aircraft 101. FIG. 2 is a diagram illustrating an example of a hardware configuration of a network camera 103. FIG. 2 is a diagram illustrating an example of a hardware configuration of a control computer 105. FIG. It is a figure which shows an example of a function structure of an imaging | photography system. It is a flowchart which shows an example of the control process of an unmanned aerial vehicle in 1st Embodiment. It is a figure which shows an example of the various data output from the angular velocity sensor. It is a flowchart which shows an example of the control process of an unmanned aerial vehicle in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, the first embodiment will be described. FIG. 1 is a diagram illustrating an example of a system configuration of an imaging system in the present embodiment.

  The imaging system of this embodiment includes an unmanned aircraft 101, an operation terminal 102, a network camera 103, a relay device 104, a control computer 105, and an operation console 106. And they are connected via a network 110 and a wireless LAN (including a mobile communication network) 120 so as to be communicable. Note that the system configuration in FIG. 1 is an example, and there are various configuration examples depending on the application and purpose.

  Unmanned aerial vehicle 101 is an unmanned aircraft that can be remotely controlled by operation terminal 102. The unmanned aerial vehicle 101 flies by operating a plurality of rotor blades in response to an instruction from the operation terminal 102. The unmanned aerial vehicle 101 moves the unmanned aircraft 101 forward, backward, turns, hovers, and the like by increasing or decreasing the rotational speed of the rotor blades. The unmanned aerial vehicle 101 is, for example, a drone.

  The unmanned aerial vehicle 101 can communicate with the operation terminal 102, the control computer 105, and the like in a wireless and wired manner. In this embodiment, the unmanned aerial vehicle 101 may communicate with the operation terminal 102, the control computer 105, and the like by either wired or wireless methods.

  The operation terminal 102 is a transmitter (remote operation terminal) for operating the unmanned aircraft 101. In the present embodiment, a proportional system (proportional control system) called “propo” will be described as an example of an operation terminal capable of maneuvering the unmanned aircraft 101. However, the operation terminal 102 is a portable terminal such as a so-called smartphone or tablet terminal. May be. The propo can control the rotation speed of the rotor blades of the unmanned aerial vehicle 101 in proportion to the amount of movement of the operation unit of the propo by accepting the operation. As a method for controlling the unmanned aerial vehicle 101, instead of manual control by the operation terminal 102, the unmanned aircraft 101 may be controlled automatically by setting a flight route or the like in advance.

  The network camera 103 is a camera that is communicably connected to the control computer 105 via a wireless LAN or a telephone line, and indicates a fixed camera installed at a predetermined location. The network camera 103 here is a camera installed on the roof of a building, for example, and includes a camera used as a weather camera. The network camera 103 includes a surveillance camera installed at the entrance of a building or in the city. The network camera 103 can change the shooting direction by moving the direction of the camera. The movement of the camera direction includes panning to move the camera lens to the left and right, tilt to move the camera lens up and down, and zoom to make the camera lens telephoto and wide-angle. doing. These cameras can be controlled from a remote location (PTZ control). The network camera 103 of the present embodiment will be described by taking a fixed camera as an example. However, the network camera 103 is not limited to this, and may be a camera mounted on another unmanned aircraft different from the unmanned aircraft 101. Good.

  The relay device 104 has a function of supplying power to the network camera 103 and the unmanned aircraft 101 and transmitting a control signal from the console 106.

  The control computer 105 (information processing apparatus) and the console 106 may be installed in a remote place physically separated from the place where the network camera 103 is installed. For example, the control computer 105 (information processing apparatus) and the console 106 may be installed at a short distance that is not so far away from each other, such as within the same site. It is also possible to set a centralized management center that collectively manages a plurality of unmanned aircraft 101 and network cameras 103.

  The control computer 105 is a device that connects a control line of the console 106 for controlling the plurality of unmanned aircraft 101 and the network camera 103.

  The console 106 is a device that receives an instruction for controlling the unmanned aircraft 101 and the network camera 103.

  The network 110 and the wireless LAN 120 are networks that connect each device of the photographing system. Each device of the photographing system can be implemented regardless of whether it is connected via a network, connected via a wireless LAN, or connected via a mobile communication network.

  FIG. 2 is a diagram illustrating a hardware configuration of the unmanned aerial vehicle 101. Note that the hardware configuration of the unmanned aerial vehicle 101 shown in FIG. 2 is an example, and there are various configuration examples depending on the application and purpose.

  The flight controller 200 is a microcontroller for performing flight control of the unmanned aerial vehicle 101, and includes a CPU 201, a ROM 202, a RAM 203, and a peripheral bus interface 204 (hereinafter referred to as a peripheral bus I / F 204).

  The CPU 201 comprehensively controls each device connected to the system bus. The external memory 280 connected to the ROM 202 or the peripheral bus I / F 304 stores a basic input / output system (BIOS) that is a control program of the CPU 201 and an operating system program.

  The external memory 280 (storage means) stores various programs and the like necessary for realizing the functions executed by the unmanned aircraft 101. A RAM 203 (storage means) functions as a main memory, work area, and the like for the CPU 201.

  The CPU 201 implements various operations by loading a program necessary for execution of processing into the RAM 203 and executing the program.

  The peripheral bus I / F 204 is an interface for connecting to various peripheral devices. Connected to the peripheral bus I / F 204 are a PMU 210, a SIM adapter 220, a wireless LAN BB unit 230, a mobile communication BB unit 240, a GPS unit 250, a sensor 260, a GCU 270, and an external memory 280.

  The PMU 210 is a power management unit and can control power supply from the battery included in the unmanned aircraft 101 to the ESC 211. The ESC 211 is an electronic speed controller and can control the rotation speed of the motor 212 connected to the ESC 211. By rotating the motor 212 using the ESC 211, the propeller 213 (rotary blade) connected to the motor 212 is rotated.

  Note that a plurality of sets of ESCs 211, motors 212, and propellers 213 are provided according to the number of propellers 213. For example, in the case of a quadcopter, the number of propellers 213 is four, so four sets are required.

  The SIM adapter 220 is a card adapter for inserting the SIM card 221. The type of the SIM card 221 is not particularly limited. Any SIM card 221 may be used depending on the carrier that provides the mobile communication network.

  The wireless LAN BB unit 230 is a baseband unit for performing communication via a wireless LAN. The wireless LAN BB unit 230 can generate a baseband signal from data or signals to be transmitted and send it to the modem circuit. Furthermore, original data and signals can be obtained from the received baseband signal.

  The wireless LAN RF unit 231 is an RF (Radio Frequency) unit for performing communication via a wireless LAN. The wireless LAN RF unit 231 can modulate the baseband signal transmitted from the wireless LAN BB unit 230 into the frequency band of the wireless LAN and transmit it from the antenna. Furthermore, when a signal in the wireless LAN frequency band is received, it can be demodulated into a baseband signal.

  The mobile communication BB unit 240 is a baseband unit for performing communication via a mobile communication network. The mobile communication BB unit 240 can generate a baseband signal from data or signals to be transmitted and send it to the modem circuit. Furthermore, original data and signals can be obtained from the received baseband signal.

  The mobile communication RF unit 241 is an RF (Radio Frequency) unit for performing communication via a mobile communication network. The mobile communication RF unit 241 can modulate the baseband signal transmitted from the mobile communication BB unit 240 into the frequency band of the mobile communication network and transmit it from the antenna. Further, when a signal in the frequency band of the mobile communication network is received, it can be demodulated into a baseband signal.

  The GPS unit 250 is a receiver that can acquire the current position of the unmanned aerial vehicle 101 using a global positioning system. The GPS unit 250 can receive a signal from a GPS satellite and estimate the current position.

  The sensor 260 is a sensor for measuring the tilt, direction, speed, and surrounding environment of the unmanned aircraft 101. The unmanned aircraft 101 includes a gyro sensor, an acceleration sensor, an atmospheric pressure sensor, a magnetic sensor, an ultrasonic sensor, and the like as the sensor 260. Based on the data acquired from these sensors, the CPU 201 controls the attitude and movement of the unmanned aerial vehicle 101.

  The GCU 270 is a gimbal control unit and is a unit for controlling the operations of the camera 271 and the gimbal 272. The unmanned aerial vehicle 101 will vibrate or become unstable due to the flight of the unmanned aerial vehicle 101. Therefore, the gimbal 272 absorbs the vibration of the unmanned aerial vehicle 101 so that no blurring occurs when the camera 271 takes a picture. Maintain level. Further, the camera 271 can be remotely operated by the gimbal 272.

  Various programs and the like used by the unmanned aerial vehicle 101 of the present invention to execute various processes, which will be described later, are recorded in the external memory 280 and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. . Furthermore, definition files and various information tables used by the program according to the present invention are stored in the external memory 280.

  FIG. 3 is a configuration diagram illustrating a hardware configuration of the network camera 103.

  The CPU 301 comprehensively controls each device and controller connected to the system bus 304.

  Further, the ROM 302 or the external memory 305 stores a basic input / output system (BIOS) that is a control program of the CPU 301 and an operating system program (hereinafter referred to as OS). In addition, various programs described below necessary for realizing the functions executed by the image processing server 108 are stored. A RAM 303 functions as a main memory, work area, and the like for the CPU 301. The CPU 301 implements various operations by loading a program or the like necessary for execution of processing into the RAM 303 and executing the program.

  A memory controller (MC) 306 controls access to a hard disk (HD) that stores a boot program, various applications, font data, user files, editing files, various data, image data, and the like. Further, access to an external memory 305 such as a compact flash (registered trademark) or smart media (registered trademark) connected to the PCMCIA card slot via an adapter is controlled.

  The camera unit 307 is connected to the image processing unit 308 and performs photoelectric conversion on the light obtained through the lens directed toward the monitoring target by a light receiving cell such as a CCD or CMOS, and then outputs an RGB signal. Or a complementary color signal is output to the image processing unit 308.

  The image processing unit 308 performs white balance adjustment, gamma processing, and sharpness processing based on the RGB signal and the color collection signal. Further, YC signal processing is performed to generate a luminance signal Y and a chroma signal (hereinafter referred to as YC signal). The YC signal is then compressed in a predetermined compression format (for example, JPEG format or Motion JPEG format), and the compressed data is temporarily stored in the external memory 305 as image data.

  A communication I / F controller (communication I / FC) 309 connects and communicates with an external device via a network. A communication I / F controller (communication I / FC) 309 executes communication control processing in the network, and image data stored in the external memory 305 is transmitted to an external device by the communication I / F controller 309. .

  FIG. 4 is a diagram illustrating an example of a hardware configuration of the control computer 105 (hereinafter, information processing apparatus 105).

  The CPU 401 comprehensively controls each device and controller connected to the system bus 404.

  The ROM 402 or the external memory 411 (storage means) stores a BIOS (Basic Input / Output System) and an operating system program (hereinafter referred to as OS) which are control programs for the CPU 401. In addition, various programs to be described later necessary for realizing the functions executed by the information processing apparatus 105 are stored. The RAM 403 functions as a main memory, work area, and the like for the CPU 401.

  The CPU 401 implements various operations by loading a program or the like necessary for execution of processing into the RAM 403 and executing the program.

An input controller (input C) 405 controls input from an input device 409 such as a character input means such as a keyboard, a pointing device such as a mouse (not shown), a sound acquisition means such as a microphone, and a video acquisition means such as a camera. To do.

A video controller (VC) 406 controls display on a display device such as the display 410. The type of the display device is assumed to be a CRT or a liquid crystal display, but is not limited thereto.

  A memory controller (MC) 407 controls access to a hard disk (HD) that stores a boot program, browser software, various applications, font data, user files, editing files, various data, and the like. Further, access to an external memory 411 such as a card-type memory connected to a flexible disk (FD) or PCMCIA card slot via an adapter is controlled.

  A communication I / F controller (communication I / FC) 408 connects and communicates with an external device via a network, and executes communication control processing in the network. For example, Internet communication using TCP / IP is possible.

  Note that the CPU 401 enables display on the display 410 by executing outline font rasterization processing on a display information area in the RAM 403, for example.

  In addition, the CPU 401 allows a user instruction with a mouse cursor (not shown) on the display 410.

  Various programs used for the information processing apparatus 105 of the present invention to execute various processes described later are recorded in the external memory 411 and are executed by the CPU 401 by being loaded into the RAM 403 as necessary. is there. Furthermore, definition files and various information tables used by the program according to the present invention are stored in the external memory 411.

  FIG. 5 is a diagram illustrating an example of a functional configuration of the imaging system. Note that the functional configurations of the unmanned aerial vehicle 101 and the network camera 103 shown in FIG. 5 are merely examples, and there are various configuration examples depending on applications and purposes.

  The unmanned aircraft 101 includes a flight control unit 511, a wireless LAN communication control unit 512, a mobile communication control unit 513, a GPS control unit 514, a sensor control unit 515, an imaging control unit 516, and an image data transmission unit 517 as functional units. .

  The flight control unit 511 is a functional unit for controlling the flight of the unmanned aircraft 101. The flight control unit 511 rotates a plurality of rotary wings included in the unmanned aircraft 101 according to instructions from the operation terminal 102, the relay device 104, and the information processing device 105 (control computer 105), and moves forward, backward, turn, Drive control such as hovering is performed.

  The wireless LAN communication control unit 512 is a functional unit for performing communication with the operation terminal 102 via the wireless LAN.

  The mobile communication control unit 513 is a functional unit for performing communication with the operation terminal 102 via the mobile communication network. The mobile communication control unit 513 controls the BB unit 240 for mobile communication and the RF unit 241 for mobile communication, modulates the signal to a frequency band of the mobile communication network, transmits a signal, and also uses the frequency band of the mobile communication network. When the signal is received, it is demodulated.

  The GPS control unit 514 is a functional unit for acquiring the current position of the unmanned aerial vehicle 101. The GPS control unit 514 controls the GPS unit 250 to receive a signal from a GPS satellite, and estimates the current position of the unmanned aircraft 101.

  The sensor control unit 515 is a functional unit for acquiring information detected by the sensor 260. Information detected by various sensors such as a gyro sensor, an acceleration sensor, an atmospheric pressure sensor, a magnetic sensor, and an ultrasonic sensor included in the unmanned aircraft 101 is always acquired and used for flight control of the flight control unit 511. In addition, information indicating the swing of the unmanned aerial vehicle 101 acquired from the gyro sensor is used for switching determination of a camera that captures an image transmitted by broadcasting radio waves.

  The shooting control unit 516 is a functional unit for obtaining image data by causing the camera 271 to perform a shooting operation via the GCU 270. The camera 271 takes a picture in response to an instruction from the operation terminal 102, and the generated image data is stored in a storage medium such as the external memory 280. Alternatively, the generated image data may be transmitted to the operation terminal 102, the relay device 104, or the information processing device 105.

  In addition, the shooting control unit 516 can control the shooting direction of the camera 271 by controlling the operation of the gimbal 272 via the GCU 270 in response to an instruction from the operation terminal 102 or the like.

  The network camera 103 includes a shooting control unit 521, a communication control unit 522, and an image data transmission unit 523 as functional units.

  The shooting control unit 521 is a functional unit for obtaining image data by causing the camera unit 307 to perform a shooting operation using the CPU 301. In response to the instruction, the camera unit 307 performs shooting, and the generated image data is stored in the external memory 305 or the like. Further, the zoom operation of the lens is performed according to the received zoom-in operation and zoom-out operation instructions, and the network camera 103 is rotated to the right or left according to the received pan operation instruction.

  The communication control unit 522 is a functional unit that communicates with each device (the information processing device 105) that is connected to the network in a communicable manner. Alternatively, the image data generated by the function of the imaging control unit 521 may be transmitted to the operation terminal 102, the relay device 104, or the information processing device 105.

  The image data transmission unit 523 is a functional unit that transmits image data using broadcast radio waves used by a television station or the like.

  The information processing apparatus 105 includes a data transmission instruction unit 531, an unmanned aircraft detection unit 532, a network camera control unit 533, an unmanned aircraft control unit 534, and an unmanned aircraft information reception unit 535.

  The data transmission instruction unit 531 is a functional unit for transmitting an instruction for transmitting image data using broadcast radio waves to a device including each photographing apparatus. By transmitting this instruction, switching between using image data obtained by the camera 271 provided in the unmanned aircraft 101 for broadcasting or using image data obtained by the camera unit 307 provided for the network camera 103 for broadcasting is performed. Is possible.

  The unmanned aerial vehicle detection unit 532 has a function of acquiring position information where the unmanned aircraft is flying (or present) by acquiring a position from the unmanned aircraft.

  The network camera control unit 533 is a functional unit that controls the network camera in accordance with a command received from the user.

  The unmanned aerial vehicle control unit 534 is a functional unit that controls the unmanned aerial vehicle 101 according to a command received from a user or according to a preset setting.

  Unmanned aerial vehicle information reception unit 535 is a functional unit that receives information acquired or output by the function of sensor control unit 515 included in unmanned aerial vehicle 101.

  Next, the first embodiment will be described with reference to FIG. In the first embodiment of the present invention, when the information processing apparatus 105 acquires a shake of the unmanned aerial vehicle 101 that is capturing an image and transmitting the broadcast radio wave, the image transmitted by the broadcast radio wave is transmitted to the network camera 103 or the like. This is an invention for switching to an image of a fixed camera in which shaking does not easily occur.

  In step S601, the CPU 401 of the information processing apparatus 105 accepts an image capturing start operation from the user.

  In step S <b> 602, the CPU 301 of the network camera 103 accepts an image capturing start instruction from the user or from the information processing apparatus 105.

  In step S603, the CPU 301 of the network camera 103 starts capturing an image.

  In step S <b> 604, the CPU 201 of the unmanned aircraft 101 determines whether a flight start instruction has been received by the user or the function of the information processing apparatus 105. If it is determined that a flight start instruction has been received, the process proceeds to step S605. Otherwise, the process of step S604 is repeated until a flight start instruction is accepted.

  In step S605, the CPU 201 of the unmanned aerial vehicle 101 receives a flight start instruction in step S604, and starts the flight of the unmanned aircraft 101 by the function of the flight control unit 511. When the flight is started, the position information of the unmanned aircraft 101 is transmitted to the information processing apparatus 105. The position information may be transmitted at predetermined time intervals such as 1 second.

  In step S <b> 606, the CPU 401 of the information processing apparatus 105 receives position information of the unmanned aircraft 101 by the function of the unmanned aircraft detection unit 532.

  In step S607, the CPU 401 of the information processing apparatus 105 determines whether the unmanned aircraft 101 is in flight. Whether or not the aircraft is in flight may be determined by the change in position information received in step S606 or the acquisition of altitude information. If it is determined that the unmanned aircraft 101 is in flight, the process proceeds to step S610. Otherwise, the process of step S607 is repeated until it is determined that the flight is in progress.

  In step S <b> 608, the CPU 201 of the unmanned aircraft 101 accepts an imaging start instruction according to an instruction from the operation terminal 102 or an instruction from the information processing apparatus 105. Note that the imaging start instruction may be received before the flight is started in step S605.

  In step S <b> 609, the CPU 201 of the unmanned aircraft 101 starts shooting with the camera 271 by the function of the shooting control unit 516.

  In step S610, the CPU 401 of the information processing apparatus 105 transmits the position information of the unmanned aerial vehicle 101 received in step S606 to the network camera 103.

  In step S <b> 611, the CPU 301 of the network camera 103 receives position information of the unmanned aircraft 101 from the information processing apparatus 105 using the function of the communication control unit 522.

  In step S612, the CPU 301 of the network camera 103 performs pan / tilt / zoom operations toward the position of the unmanned aircraft based on the position information received in step S611 (PTZ control), and determines the shooting direction.

  In step S <b> 613, the CPU 401 of the information processing apparatus 105 transmits an image data transmission instruction to the unmanned aircraft 101 by the function of the data transmission instruction unit 531.

  In step S <b> 614, the CPU 201 of the unmanned aircraft 101 receives a data transmission instruction from the information processing apparatus 105.

  In step S615, the CPU 201 of the unmanned aerial vehicle 101 transmits the image data captured by the function of the imaging control unit 516 using broadcast radio waves by the function of the image data transmission unit 517. Image data transmitted by broadcasting radio waves is displayed in real time on each home television.

  In step S <b> 616, the CPU 201 of the unmanned aircraft 101 acquires angular velocity information using the gyro sensor by the function of the sensor control unit 515, and transmits the acquired angular velocity information to the information processing apparatus 105.

  In step S617, the CPU 401 of the information processing apparatus 105 receives angular velocity information by the function of the unmanned aircraft information receiving unit 535.

  FIG. 7 is a diagram illustrating an example of output voltage (angular velocity information) output by the gyro sensor. Both the angular velocity information (a) and the angular velocity information (b) in FIG. 7 represent the output voltage detected by the gyro sensor on the vertical axis, and the horizontal axis represents the passage of time when the output voltage was received. The diagrams of the angular velocity information (a) and the angular velocity information (b) show how the output voltage indicating the fluctuation of the unmanned aircraft 101 fluctuates within a predetermined range. In this embodiment, the reference voltage is 2.0 volts, and the predetermined range is a range of plus or minus 0.5 volts (hereinafter referred to as V1), but this value is an example and can be arbitrarily set. The angular velocity information (a) indicates a state in which the amount of voltage fluctuation with respect to the reference voltage is relatively small and the output voltage is in the range of V1. When the output voltage is in the range of V1, the image quality of the captured image data is not substantially affected by the shaking of the unmanned aircraft 101. The angular velocity information (b) in FIG. 7 shows a state in which the amount of voltage fluctuation with respect to the reference voltage is relatively large and the output voltage is outside the range of V1. That is, the image quality of the captured image data is affected by the shake of the unmanned aerial vehicle 101. The range of T1 and T2 in the angular velocity information (b) indicates the time when the output voltage drops and falls below the range of V1 (1.5 volts or less). In the case of T1, transmission of image data captured by the unmanned aerial vehicle 101 is stopped, and transmission of image data captured by the network camera 103 enables transmission of appropriate image data. In the time after T1, that is, when the output voltage returns to the range of V1, the image data to be transmitted may be returned from the image data captured by the network camera 103 to the image data captured by the unmanned aircraft 101. In addition, if the image data is switched many times in a short time, the receiver may have difficulty in viewing the image data. Therefore, in the case of T2 where the output voltage falls below the range of V1 and the time T2 is short, the unmanned aircraft The transmission of the image data captured in 101 may be intentionally continued. In order to accurately detect voltage fluctuations, it is preferable to obtain angular velocity information in a cycle as short as possible. In the present embodiment, the angular velocity information is used to detect the shaking of the unmanned aircraft 101, but other information may be used as long as the information can detect the shaking of the unmanned aircraft 101. As an example of other information, the unmanned aerial vehicle 101 may use information indicating shaking detected by various sensors such as an acceleration sensor, an atmospheric pressure sensor, a magnetic sensor, and an ultrasonic sensor other than the gyro sensor.

  In step S618, the CPU 401 of the information processing apparatus 105 determines whether or not the body of the unmanned aircraft 101 is shaken by a predetermined value or more, based on the angular velocity information received in step S617. If it is determined that the predetermined value is equal to or greater than the predetermined value, the process proceeds to step S619. Otherwise, the process of step S618 is repeated until a shake equal to or greater than the predetermined value is detected. In step S618, it is determined whether or not appropriate image data can be acquired by the unmanned aircraft 101 by detecting the swing of the unmanned aircraft 101. When a shake equal to or greater than a predetermined value is detected, appropriate image data cannot be acquired, and the processing in the next step S619 and subsequent steps is advanced to switch the imaging device that acquires the image data.

  In step S619, the CPU 201 of the unmanned aerial vehicle 101 transmits to the unmanned aircraft 101 an instruction to stop the process of transmitting image data using broadcast radio waves.

  In step S <b> 620, the CPU 201 of the unmanned aircraft 101 receives an image data transmission stop instruction from the information processing apparatus 105.

  In step S621, the CPU 201 of the unmanned aerial vehicle 101 stops the transmission of image data.

  In step S <b> 622, the CPU 401 of the information processing apparatus 105 transmits an image data transmission instruction to the network camera 103 by the function of the data transmission instruction unit 531.

  In step S <b> 623, the CPU 301 of the network camera 103 receives a data transmission instruction from the information processing apparatus 105.

  In step S624, the CPU 301 of the network camera 103 transmits the image data acquired by the camera unit 307 using broadcast radio waves by the function of the image data transmission unit 523.

  According to the present embodiment described above, the image data is acquired by the camera 271 provided in the unmanned aircraft 101 and the network camera 103, respectively, and the information processing apparatus 105 receives data related to vibration from the unmanned aircraft 101. The information processing apparatus 105 controls to transmit either image data acquired by the camera 271 included in the unmanned aircraft 101 or image data acquired by the network camera 103 in accordance with data relating to vibration received from the unmanned aircraft 101. To do. This makes it possible to provide an appropriate image without shaking. Above, description of 1st Embodiment is complete | finished.

  Next, a second embodiment will be described. In the first embodiment, the process of switching the imaging device that transmits image data to an imaging device different from the unmanned aircraft 101 according to the vibration data of the unmanned aircraft 101 has been described. In the second embodiment, a description will be given of a process of not switching an imaging device that transmits image data when each imaging device has not captured the same region or the same range when an instruction to switch the imaging device is received. .

  In step S <b> 801, the CPU 201 of the unmanned aircraft 101 starts shooting with the function of the shooting control unit 516.

  In step S <b> 802, the CPU 301 of the network camera 103 starts shooting using the function of the shooting control unit 521.

  In step S803, the CPU 201 of the unmanned aerial vehicle 101 sets a flight destination according to an instruction from the user via the operation terminal 102 or in advance.

  In step S804, the CPU 201 of the unmanned aerial vehicle 101 flies to the destination set in step S803 by the function of the flight control unit 511.

  In step S805, the CPU 201 of the unmanned aircraft 101 transmits the destination information and position information set in step S803 to the information processing apparatus 105.

  In step S806, the CPU 401 of the information processing apparatus 105 receives the destination information and the position information by using the functions of the unmanned aircraft detection unit 532 and the unmanned aircraft information reception unit 535.

  In step S807, the CPU 401 of the information processing apparatus 105 transmits the destination information and position information received in step S806 to the network camera 103.

  In step S <b> 808, the CPU 301 of the network camera 103 receives destination information and position information by the function of the communication control unit 522.

  In step S809, the CPU 301 of the network camera 103 performs pan / tilt / zoom (PTZ control) and lens focusing operations using the functions of the imaging control unit 521.

  In step S <b> 810, the CPU 401 of the information processing apparatus 105 transmits an image data transmission instruction to the unmanned aircraft 101 by the function of the data transmission instruction unit 531.

  In step S811, the CPU 201 of the unmanned aerial vehicle 101 receives a data transmission instruction.

  In step S812, the CPU 401 of the information processing apparatus 105 determines whether a camera switching instruction has been accepted. The camera switching instruction here indicates a switching instruction of a photographing apparatus that transmits image data using broadcast radio waves. If it is determined that a camera switching instruction has been received, the process proceeds to step S814. Otherwise, the process of step S812 is repeated until a switching instruction is accepted.

  In step S813, the CPU 301 of the network camera 103 transmits, to the information processing apparatus 105, operation information regarding the pan / tilt / zoom and lens focus operations performed in step S809 by the function of the communication control unit 522.

  In step S814, the CPU 401 of the information processing apparatus 105 receives control information about pan / tilt / zoom and lens focus operations from the network camera 103.

  In step S815, the CPU 201 of the unmanned aerial vehicle 101 transmits the image data captured by the data transmission instruction received in step S811 using broadcast radio waves.

  In step S <b> 816, the CPU 201 of the unmanned aircraft 101 transmits the shooting information captured by the shooting control unit 516 to the information processing apparatus 105. The shooting information may be any information as long as it is information that shows where the unmanned aircraft 101 is shooting, and may be position information or the like.

  In step S817, the CPU 401 of the information processing apparatus 105 receives shooting information from the unmanned aircraft 101.

  In step S818, the CPU 401 of the information processing apparatus 105 uses the control information about pan / tilt / zoom and lens focus operations received in step S814 and the shooting information received in step S817. Then, it is determined where each photographing apparatus is photographing. Then, it is determined whether or not the imaging device that has received the information shows the same region, range, position, or subject. If it is determined that the same region, range, position, or subject is being photographed, the process proceeds to step S819. Otherwise, the process of step S818 is repeated until it is determined that the same region, position, or subject is being photographed.

  In step S <b> 819, the CPU 401 of the information processing apparatus 105 transmits an image data transmission instruction to the network camera 103 by the function of the data transmission instruction unit 531.

  In step S820, the CPU 301 of the network camera 103 receives an image data transmission instruction.

  In step S <b> 821, the CPU 301 of the network camera 103 transmits image data using broadcast radio waves by the function of the image data transmission unit 523.

  In step S822, the CPU 401 of the information processing apparatus 105 transmits an instruction to stop transmitting image data to the unmanned aircraft 101.

  In step S823, the CPU 201 of the unmanned aerial vehicle 101 receives an image data transmission stop instruction.

  In step S824, CPU 201 of unmanned aerial vehicle 101 stops the transmission of image data.

  According to the second embodiment, the information processing apparatus 105 acquires position information of the unmanned aircraft 101 and information regarding the shooting direction of the network camera 103. Then, based on the acquired positional relationship between the cameras, it is determined whether or not the image data captured by the unmanned aircraft camera 271 and the network camera 103 are capturing the same region, range, position, and subject. Then, when the image data acquired by each imaging device is the same area, range, position, and subject, the imaging device that transmits the image data is switched. As a result, the problem that the image data to be transmitted is suddenly changed to image data obtained by photographing a different place or subject is reduced, and more appropriate image data can be transmitted. This is the end of the description of the second embodiment.

  Thus, according to the present invention, there is an effect of providing an appropriate image even when shaking occurs during image capturing using an unmanned aerial vehicle.

  The present invention can be implemented as a system, apparatus, method, program, storage medium, or the like, and can be applied to a system including a plurality of devices. You may apply to the apparatus which consists of one apparatus.

  The present invention includes a software program that realizes the functions of the above-described embodiments directly or remotely to a system or apparatus. The present invention also includes a case where the system or the computer of the apparatus is achieved by reading and executing the supplied program code.

  Therefore, the program code itself installed in the computer in order to realize (executable) the functional processing of the present invention by a computer also realizes the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, and CD-RW. In addition, there are magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  Also, key information for encrypting the program of the present invention from a homepage via the Internet to the user who has encrypted the program, stored it in a storage medium such as a CD-ROM, distributed to the user, and cleared predetermined conditions. Download. It is also possible to execute the encrypted program by using the downloaded key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

101 Unmanned Aircraft 102 Operation Terminal 103 Network Camera 104 Relay Device 105 Information Processing Device

Claims (7)

An imaging system having an unmanned aerial vehicle including a first imaging device and a second imaging device different from the first imaging device,
Obtaining means for obtaining image data photographed by the first photographing device and the second photographing device;
Receiving means for receiving data relating to vibrations of the unmanned aircraft;
Control means for performing control for transmitting image data photographed by the first photographing device and the second photographing device;
The control means transmits one of the image data taken by the first photographing device and the second photographing device in accordance with the vibration-related data received by the receiving means. An imaging system characterized by being controlled.   The control means transmits image data captured by the second imaging device when the data related to the vibration of the unmanned aircraft received by the receiving means is equal to or greater than a predetermined value. The imaging system according to claim 1, wherein the imaging system is controlled as follows. The acquisition means further acquires position information on which the unmanned aircraft flies,
The imaging system according to claim 1, wherein the imaging system further includes a determining unit that determines an imaging direction of the second imaging device in accordance with position information of the unmanned aircraft. The acquisition means further acquires position information on which the unmanned aircraft flies and control information on the shooting direction determined by the determination means,
The control unit uses the position information acquired by the acquisition unit and the control information, and when the range captured by the first imaging device is different from the range captured by the second control device, The imaging system according to claim 3, wherein control is performed so as not to change an imaging apparatus that transmits the image data.   5. The imaging system according to claim 1, wherein the second imaging device is installed in another unmanned aerial vehicle different from the unmanned aerial vehicle. A method for controlling an imaging system, comprising: an unmanned aerial vehicle including a first imaging device; and a second imaging device different from the first imaging device,
An acquisition step of acquiring image data captured by the first imaging device and the second imaging device;
Receiving for receiving data relating to vibrations of the unmanned aerial vehicle;
A control step for performing control for transmitting image data photographed by the first photographing device and the second photographing device;
With
The control step transmits one of the image data captured by the first imaging device and the second imaging device according to the vibration-related data received in the reception step. A control method for an imaging system, characterized in that the control is performed. A program for causing an unmanned aircraft including a first imaging device to function as an imaging system having a second imaging device different from the first imaging device,
The imaging system;
Obtaining means for obtaining image data photographed by the first photographing device and the second photographing device;
Receiving means for receiving data relating to vibrations of the unmanned aircraft;
Control means for performing control for transmitting image data photographed by the first photographing device and the second photographing device;
Function as
The control means transmits one of the image data taken by the first photographing device and the second photographing device in accordance with the vibration-related data received by the receiving means. Program to control.

Images