JP6281720B2 - Imaging system - Google Patents

Description

  The present invention relates to an imaging system, an imaging control method, an imaging control system, a moving object, a control method, and a program.

Patent Document 1 discloses a camera system that uses a plurality of camera devices to photograph the same subject from a plurality of different viewpoints. Patent Document 2 discloses a camera control system that accesses an arbitrary camera from an arbitrary camera client via a network to link a plurality of cameras.
Patent Document 1 Japanese Patent Application Laid-Open No. 2006-12832 Patent Document 2 Japanese Patent Application Laid-Open No. 2000-113166

  There is a demand for a system in which a plurality of moving objects equipped with an imaging device are linked to image a desired subject.

  In one aspect, an imaging system includes a first moving body on which a first imaging device is mounted and a second moving body on which a second imaging device is mounted, and the first moving body is a position of the first moving body. A first transmission unit that transmits position information indicating the position of the first imaging device and a second transmission unit that transmits setting information indicating the setting of the first imaging device, and the second mobile unit receives the position information. And a setting for setting the second imaging device based on the setting information, a first control unit that controls the movement of the second moving body, a second receiving unit that receives the setting information, and the position indicated by the position information Part.

  The first moving body may further include a second control unit that controls movement of the first moving body. The first control unit moves the second moving body with respect to the position indicated by the position information, and after the second imaging device starts imaging with the setting indicated by the setting information, the second control unit moves the first moving body. You may move to the direction away from a 2nd mobile body.

  The first moving body may further include a third control unit that controls the first imaging device. After the first control unit moves the second moving body with respect to the position indicated by the position information and the second imaging device starts imaging with the setting indicated by the setting information, the second control unit is controlled by the first imaging device. The imaging operation may be terminated.

  The second moving body issues an imaging start notification when the first control unit moves the second moving body with respect to the position indicated by the position information and the second imaging device starts imaging with the setting indicated by the setting information. You may further have the 3rd transmission part which transmits. The first moving body may further include a third receiving unit that receives an imaging start notification.

  The first moving body may further include a third control unit that controls the first imaging device. The third control unit may end the imaging operation by the first imaging device when the third receiving unit receives the imaging start notification.

  The first imaging device and the second imaging device may record a moving image. The first moving body may further include a third control unit that controls the first imaging device. The second moving body may further include a fourth control unit that controls the second imaging device. The third control unit may record the time when the third reception unit receives the imaging start notification in association with the moving image recorded by the first imaging device. The fourth control unit records the time at which the recording of the moving image is started by the second imaging device with the setting indicated by the setting information, or the time at which the third transmission unit transmits the imaging start notification, by the second imaging device. You may record in relation to.

  After the first control unit moves the second moving body with respect to the position indicated by the position information, the second transmission unit may transmit the setting information. The second receiving unit may receive setting information.

  The setting information may include at least one of the optical conditions of the first imaging device and the imaging direction.

  The first mobile unit may further include a fourth transmission unit that transmits a dispatch request notification for requesting the second mobile unit to be dispatched when a predetermined condition is satisfied.

  The second mobile unit may further include a fourth receiving unit that receives the dispatch request notification. The first control unit may control the movement of the second moving body with the position indicated by the position information as the target position when the fourth receiving unit receives the dispatch request notification.

  The fourth transmission unit issues a dispatch request notification when the remaining capacity of the battery mounted on the first moving body is less than a predetermined amount or when a failure of the first moving body or the first imaging device is detected. May be sent.

  The imaging system may further include a remote control device that remotely controls the first moving body and the first imaging device, and the second moving body and the second imaging device. The remote control device may include a transmission / reception unit that receives position information from the first moving body and transmits the position information to the second moving body.

  The transmission / reception unit may receive the setting information from the first moving body and transmit the setting information to the second moving body.

  The remote operation device may be switched between a state where the first imaging device and the first moving body can be remotely operated and a state where the second imaging device and the second moving body can be remotely operated.

  The remote operation device is in a state in which the first moving body and the first imaging device can be remotely operated before imaging is started by the second imaging device with the setting indicated by the setting information, and the second setting is indicated by the setting information. After the imaging is started by the imaging device, the second moving body and the second imaging device may be in a remotely operable state.

  In one aspect, an imaging control method includes a step in which a first moving body on which a first imaging device is mounted transmits position information indicating a position of the first moving body; A step of transmitting setting information indicating the setting; a step of receiving the position information by the second moving body on which the second imaging device is mounted; and a second movement of the second moving body with respect to the position indicated by the position information. The step of controlling the movement of the body, the step of receiving the setting information by the second moving body, and the step of setting the second imaging device by the second moving body based on the setting information.

  In one aspect, the imaging control system includes a first receiving unit that receives position information indicating a position of the first moving body from the first moving body on which the first imaging device is mounted, and a second imaging device. In order to move the second moving body relative to the position indicated by the position information, a first transmission unit that transmits position information to the second moving body, and setting information that indicates the setting of the first imaging device from the first moving body. And a second transmitter for transmitting the setting information to the second moving body in order to set the second imaging device based on the setting information.

  In one aspect, the imaging control method includes a step of receiving position information indicating a position of the first moving body from a first moving body on which the first imaging device is mounted, and a second movement on which the second imaging device is mounted. Transmitting the position information to the second moving body to move the body relative to the position indicated by the position information; receiving setting information indicating the setting of the first imaging device from the first moving body; Transmitting the setting information to the second moving body to set the second imaging device based on the setting information.

  In one aspect, a program receives a position information indicating a position of the first moving body from a first moving body on which the first imaging device is mounted, and a second moving body on which the second imaging device is mounted. A step of transmitting position information to the second moving body to receive movement of the position information indicated by the position information, a step of receiving setting information indicating the setting of the first imaging device from the first moving body; In order to set the imaging device based on the setting information, the second mobile unit is caused to execute a step of transmitting the setting information.

  In one aspect, the moving body is a moving body on which an imaging device is mounted, and a position information transmitting unit that transmits position information indicating the position of the moving body, and setting information that transmits setting information indicating the setting of the imaging device. When another moving body moves with the transmission unit and the position indicated by the position information as the target position, and another imaging device mounted on the other moving body starts imaging with the setting indicated by the setting information, A movement control unit that moves the moving body in a direction away from the moving body.

  In one aspect, the control method includes a step of transmitting position information indicating a position of a moving body on which the imaging device is mounted, a step of transmitting setting information indicating a setting of the imaging device, and the other moving body has position information. Moving the moving body in the direction away from the other moving body when another imaging apparatus mounted on the other moving body starts imaging with the setting indicated by the setting information With.

  In one aspect, the program transmits the position information indicating the position of the moving body on which the imaging device is mounted, transmits the setting information indicating the setting of the imaging device, and uses the position indicated by the position information as the target position. A step of moving the moving body in a direction away from the other moving body when the other moving body moves and another imaging device mounted on the other moving body starts imaging with the setting indicated by the setting information; Is executed on the computer.

  In one aspect, the moving body is a moving body on which an imaging device is mounted, the first receiving unit receiving position information indicating the position of another moving body, and the moving body with respect to the position indicated by the position information. A control unit for moving the camera, a second receiving unit for receiving setting information indicating settings of another imaging device mounted on another moving body, a setting unit for setting the imaging device based on the setting information, and position information And a transmission unit that transmits an imaging start notification when the control unit moves the moving body and the imaging apparatus starts imaging with the setting indicated by the setting information.

  In one aspect, the control method includes a step of receiving position information indicating the position of another moving body, a step of moving the moving body on which the imaging device is mounted with respect to the position indicated by the position information, and another movement. Receiving setting information indicating the setting of another imaging device mounted on the body, setting the imaging device based on the setting information, moving the moving body relative to the position indicated by the position information, and setting information And a step of transmitting an imaging start notification when the imaging apparatus starts imaging with the setting indicated by.

  In one aspect, the program receives the position information indicating the position of the other moving body, moves the moving body on which the imaging device is mounted with respect to the position indicated by the position information, and the other moving body. Receiving the setting information indicating the setting of the other imaging device mounted on the device, setting the imaging device based on the setting information, moving the moving body with respect to the position indicated by the position information, and setting information When the imaging apparatus starts imaging with the setting shown, the computer is caused to execute a stage of transmitting an imaging start notification.

  The above summary of the present invention does not enumerate all of the features of the present invention. A sub-combination of these feature groups can also be an invention.

It is a figure which shows an example of a structure of an imaging system. It is a figure which shows an example of the functional block of UAV. It is a figure which shows an example of the functional block of a UAV control part. It is a figure which shows an example of the functional block of a remote control device. It is a figure which shows an example of an imaging sequence.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Not all combinations of features described in the embodiments are essential for the solution of the invention.

  The claims, the description, the drawings, and the abstract include matters subject to copyright protection. The copyright owner will not object to any number of copies of these documents as they appear in the JPO file or record. However, in other cases, all copyrights are reserved.

  FIG. 1 shows an example of the configuration of the imaging system 10. The imaging system 10 includes a plurality of unmanned aircraft (UAV) 100 and a remote control device 300. The plurality of UAVs 100 cooperate with each other via the remote control device 300 to continuously capture a desired imaging range 30.

  The UAV 100 includes a UAV main body 102, a gimbal 200, an imaging device 220, and a plurality of imaging devices 230. The UAV 100 is an example of a moving object. The moving body is a concept including, in addition to UAV, other aircraft that moves in the air, vehicles that move on the ground, ships that move on the water, and the like.

  The UAV main body 102 includes a plurality of rotor blades. The UAV main body 102 flies the UAV 100 by controlling the rotation of a plurality of rotor blades. For example, the UAV main body 102 causes the UAV 100 to fly using four rotary wings. The number of rotor blades is not limited to four. Further, the UAV 100 may be a fixed wing aircraft that does not have a rotating wing.

  The imaging device 220 is an imaging camera that images a subject included in a desired imaging range 30. The plurality of imaging devices 230 are sensing cameras that image the surroundings of the UAV 100 in order to control the flight of the UAV 100. Two imaging devices 230 may be provided on the front surface that is the nose of the UAV 100. Two other imaging devices 230 may be provided on the bottom surface of the UAV 100. The two imaging devices 230 on the front side may be paired and function as a so-called stereo camera. The two imaging devices 230 on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the UAV 100 may be generated based on image data captured by the plurality of imaging devices 230. The number of imaging devices 230 included in the UAV 100 is not limited to four. The UAV 100 only needs to include at least one imaging device 230. The UAV 100 may include at least one imaging device 230 on each of the nose, the tail, the side surface, the bottom surface, and the ceiling surface of the UAV 100. The angle of view that can be set by the imaging device 230 may be wider than the angle of view that can be set by the imaging device 220. The imaging device 230 may have a single focus lens or a fisheye lens.

  The remote operation device 300 communicates with the plurality of UAVs 100 to remotely operate the plurality of UAVs 100. The remote control device 300 may communicate with a plurality of UAVs 100 wirelessly. The remote operation device 300 transmits various commands to the plurality of UAVs 100. The remote operation device 300 relays various information transmitted from the UAV 100 to other UAVs 100. The remote operation device 300 is in a state in which the UAV 100 serving as the imaging role and the imaging device 220 included in the UAV 100 serving as the imaging role can be remotely operated, and a state in which the imaging device 220 corresponding to the standby UAV 100 and the standby UAV 100 can be remotely operated. The operation can be switched. The remote operation device 300 is an example of an imaging control system. Station 400 provides a place where UAV 100 waits. The station 400 may have a charging function for charging a battery mounted on the UAV 100.

  FIG. 2 shows an example of functional blocks of the UAV 100. The UAV 100 includes a UAV control unit 110, a battery 140, a communication interface 150, a memory 160, a gimbal 200, a rotary wing mechanism 210, an imaging device 220, an imaging device 230, a GPS receiver 240, an inertial measurement device (IMU) 250, and a magnetic compass 260. , And a barometric altimeter 270.

  The battery 140 supplies power to each unit included in the UAV 100. The communication interface 150 communicates with an external device. The communication interface 150 receives various commands for the UAV control unit 110 from the remote operation device 300. The memory 160 stores programs necessary for the UAV control unit 110 to control the gimbal 200, the rotary blade mechanism 210, the imaging device 220, the imaging device 230, the GPS receiver 240, the IMU 250, the magnetic compass 260, and the barometric altimeter 270. Store. The memory 160 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 160 may be provided in the housing of the UAV 100. The memory 160 may be provided so as to be removable from the housing of the UAV 100.

  The gimbal 200 supports the imaging device 220 rotatably about at least one axis. The gimbal 200 may support the imaging device 220 rotatably about the yaw axis, pitch axis, and roll axis. The gimbal 200 may change the imaging direction of the imaging device 220 by rotating the imaging device 220 about at least one of the yaw axis, the pitch axis, and the roll axis. The rotary blade mechanism 210 includes a plurality of rotary blades and a plurality of drive motors that rotate the plurality of rotary blades.

  The imaging device 220 captures a subject within a desired imaging range and generates image data. The imaging device 220 may capture a subject within a desired imaging range and generate a moving image as image data. The imaging device 220 may store image data in a memory included in the imaging device 220 or the memory 160. The imaging device 220 may record a moving image in the memory included in the imaging device 220 or the memory 160. The imaging device 230 captures the surroundings of the UAV 100 and generates image data. The imaging device 230 captures the surroundings of the UAV 100 and generates image data. The imaging device 230 may store the generated image data in the memory 160. The image data stored in the memory 160 may be transmitted to an external device via the communication interface 150. The memory 160 may be provided in the housing of the UAV 100. It may be provided so as to be removable from the housing of the UAV 100.

  The GPS receiver 240 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 240 calculates the position of the GPS receiver 240, that is, the position of the UAV 100, based on the received signals. The inertial measurement device (IMU) 250 detects the posture of the UAV 100. As the posture of the UAV 100, the IMU 250 detects the acceleration in the three axial directions of the UAV 100 in the front, rear, left, and right directions, and the angular velocity in the three axial directions of pitch, roll, and yaw. The magnetic compass 260 detects the heading of the UAV 100. The barometric altimeter 270 detects the altitude at which the UAV 100 flies.

  The UAV control unit 110 controls the flight and imaging of the UAV 100 according to a program stored in the memory 160. The UAV control unit 110 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The UAV control unit 110 controls the flight of the UAV 100 according to a command received from the remote control device 300 via the communication interface 150. The UAV control unit 110 functions as an imaging system by reading a program for flight control and imaging control from the memory 160 and executing the program.

  The UAV control unit 110 may specify the environment around the UAV 100 by analyzing a plurality of image data captured by the plurality of imaging devices 230. The UAV control unit 110 controls the flight while avoiding obstacles based on the environment around the UAV 100, for example. The UAV control unit 110 may generate three-dimensional spatial data around the UAV 100 based on a plurality of image data captured by the plurality of imaging devices 230, and control the flight based on the three-dimensional spatial data. The UAV control unit 110 and the imaging device 220 are an example of an imaging system. The imaging system may include other configurations such as a gimbal 200.

  FIG. 3 shows an example of functional blocks of the UAV control unit 110. The UAV control unit 110 includes an outgoing request notification transmission unit 112, an outgoing request notification reception unit 114, a location information reception unit 116, a movement control unit 118, a location information transmission unit 120, a setting information transmission unit 122, a setting information reception unit 124, and imaging. A setting unit 126, an imaging start notification receiving unit 128, an imaging control unit 130, and an imaging start notification transmission unit 132 are included. Each unit included in the UAV control unit 110 may be configured by any one of hardware, firmware, and software. The position information transmission unit 120 is an example of a first transmission unit. The setting information transmission unit 122 is an example of a second transmission unit. The imaging start notification transmission unit 132 is an example of a transmission unit and a third transmission unit. The dispatch request notification transmission unit 112 is an example of a fourth transmission unit. The position information receiving unit 116 is an example of a first receiving unit. The setting information receiving unit 124 is an example of a second receiving unit. The imaging start notification receiving unit 128 is an example of a third receiving unit. The dispatch request notification receiving unit 114 is an example of a fourth receiving unit. The movement control unit 118 is an example of a control unit, a first control unit, and a second control unit. The imaging control unit 130 is an example of a third control unit and a fourth control unit. The imaging setting unit 126 is an example of a setting unit.

  The dispatch request notification transmission unit 112 transmits a dispatch request notification for requesting another UAV 100 to be dispatched when a predetermined condition is satisfied. The dispatch request notification transmission unit 112 may transmit a dispatch request notification to the waiting UAV 100 via the remote operation device 300. The dispatch request notification transmission unit 112 may transmit the dispatch request notification to the standby UAV 100 without using the remote operation device 300. The dispatch request notification transmission unit 112 is activated when the remaining capacity of the battery 140 mounted on the UAV 100 serving as the imaging role is smaller than a predetermined amount or when a malfunction of the UAV 100 serving as the imaging role or the UAV 100 serving as the imaging role is detected. A request notification may be sent. The dispatch request notification transmission unit 112 may transmit a dispatch request notification to the standby UAV 100 in response to a command from the remote operation device 300.

  The dispatch request notification receiving unit 114 receives a dispatch request notification from another UAV 100. The dispatch request notification receiving unit 114 may receive a dispatch request notification from the UAV 100 serving as the imaging role via the remote operation device 300. The dispatch request notification receiving unit 114 may receive the dispatch request notification from the UAV 100 serving as the imaging role without using the remote operation device 300.

  The position information receiving unit 116 receives position information indicating the position of the UAV 100. The position information includes information indicating the latitude, longitude, and altitude where the UAV 100 exists. The position information receiving unit 116 receives position information indicating the position of the UAV 100 serving as the imaging role. The position information receiving unit 116 may receive the position information transmitted from the UAV 100 serving as the imaging role toward the standby UAV 100 via the remote operation device 300. The position information receiving unit 116 may receive position information without using the remote operation device 300.

  The position information transmission unit 120 transmits position information indicating the position of the UAV 100. The position information transmission unit 120 acquires information indicating latitude and longitude from the GPS receiver 240. The position information transmission unit 120 acquires information indicating the altitude from the barometric altimeter 270. The position information transmission unit 120 transmits information indicating the acquired latitude, longitude, and altitude as position information. The position information transmission unit 120 may further transmit information indicating the orientation of the UAV 100 serving as the imaging role as position information. The position information transmission unit 120 transmits position information indicating the position of the UAV 100 serving as the imaging role. The position information transmission unit 120 may transmit position information indicating the position of the UAV 100 serving as the imaging role to the UAV 100 that is on standby via the remote operation device 300. The position information transmission unit 120 may transmit position information indicating the position of the UAV 100 serving as the imaging role to the UAV 100 that is on standby without using the remote operation device 300.

  The movement control unit 118 controls the movement of the waiting UAV 100 with the position indicated by the position information as the target position when the dispatch request notification receiving unit 114 receives the dispatch request notification. The movement control unit 118 controls the movement of the waiting UAV 100 so that the waiting UAV 100 is moved to the vicinity of the UAV 100 serving as the imaging role.

  The setting information transmission unit 122 transmits setting information indicating the setting of the imaging device 220 included in the UAV 100 serving as the imaging role. The setting information includes at least one of the optical conditions of the imaging device 220 and the imaging direction. The optical condition of the imaging device 220 may include at least one of the angle of view and the focal length of the imaging device 220. The setting information transmission unit 122 may transmit the setting information of the UAV 100 serving as the imaging role to the waiting UAV 100 via the remote operation device 300. The setting information transmission unit 122 may transmit the setting information of the imaging system 10 serving as the imaging role to the standby UAV 100 without using the remote operation device 300. The setting information transmission unit 122 of the UAV 100 serving as the imaging role may transmit the setting information after the movement control unit 118 moves the UAV 100 that is on standby using the position indicated by the position information as the target position.

  The setting information receiving unit 124 receives setting information. The setting information receiving unit 124 receives the setting information of the UAV 100 of the imaging role transmitted from the UAV 100 of the imaging role to the UAV 100 in standby. The setting information receiving unit 124 may receive setting information via the remote operation device 300. The setting information receiving unit 124 may receive setting information without using the remote operation device 300. The setting information receiving unit 124 of the waiting UAV 100 may receive the setting information after the movement control unit 118 moves the waiting UAV 100 using the position indicated by the position information as the target position.

  The imaging setting unit 126 sets the imaging device 220 of the UAV 100 based on the setting information. Based on the setting information, the imaging setting unit 126 matches the setting of the imaging device 220 of the standby UAV 100 with the setting of the imaging device 220 of the UAV 100 serving as the imaging role. The imaging setting unit 126 controls the imaging device 220 and the gimbal 200 to match the angle of view, the focal length, and the imaging direction of the imaging device 220 of the UAV 100 that is in standby with the settings of the imaging device 220 of the UAV 100 that is the imaging role.

  The imaging start notification receiving unit 128 receives the imaging start notification transmitted from the standby UAV 100 when the standby UAV 100 starts imaging. The imaging start notification receiving unit 128 may receive an imaging start notification via the remote operation device 300. The imaging start notification receiving unit 128 may receive the imaging start notification without using the remote operation device 300.

  The imaging start notification transmission unit 132 moves the UAV 100 that is on standby by the movement control unit 118 using the position indicated by the position information as the target position, and the imaging device 220 of the UAV 100 that is on standby with the setting indicated by the setting information starts imaging. In addition, an imaging start notification is transmitted. The imaging start notification transmission unit 132 may transmit an imaging start notification to the UAV 100 serving as an imaging role via the remote operation device 300. The imaging start notification transmission unit 132 may transmit an imaging start notification to the UAV 100 serving as an imaging role without using the remote operation device 300.

  The imaging control unit 130 controls the imaging device 220. The imaging control unit 130 moves the UAV 100 that is on standby by the movement control unit 118 using the position indicated by the position information as a target position, and the imaging apparatus 220 of the UAV 100 that is on standby with the setting indicated by the setting information starts imaging. The imaging operation by the imaging device 220 of the role UAV 100 is terminated. When the imaging start notification receiving unit 128 receives the imaging start notification, the imaging control unit 130 ends the imaging operation by the imaging device 220 of the UAV 100 serving as the imaging role.

  The imaging control unit 130 of the imaging role UAV 100 records the time when the imaging start notification receiving unit 128 receives the imaging start notification in association with the moving image recorded by the imaging device 220 of the imaging role UAV 100. The imaging control unit 130 of the standby UAV 100 starts recording a moving image by the imaging device 220 of the standby UAV 100 with the setting indicated by the setting information, or the time when the imaging start notification transmission unit 132 transmits the imaging start notification Are recorded in association with the moving image recorded by the imaging device 220 of the UAV 100 in standby.

  FIG. 4 shows an example of functional blocks of the remote operation device 300. The remote operation device 300 includes a remote operation control unit 310, a display unit 330, an operation unit 340, a communication interface 350, and a memory 360. The communication interface 350 transmits various commands to the UAV 100. The communication interface 350 receives various information from the UAV 100. The memory 360 stores programs and the like necessary for the remote operation control unit 310 to control the display unit 330 and the operation unit 340. The memory 360 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 360 may be provided in the housing of the remote operation device 300. The memory 360 may be provided so as to be removable from the casing of the remote control device 300.

  The display unit 330 displays various information regarding the UAV 100. The display unit 330 may display a moving image transmitted from the UAV 100. The operation unit 340 receives a flight control command and an imaging control command for the UAV 100 from the user. The remote operation control unit 310 controls the flight and imaging of the UAV 100 according to the control command received from the operation unit 340 according to the program stored in the memory 360. The remote operation control unit 310 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like.

  The remote operation control unit 310 includes a transmission / reception unit 312, a command generation unit 314, and a communication switching unit 316. The transmission / reception unit 312 transmits information received from the UAV 100 to another UAV 100. The transmission / reception unit 312 may transmit information including identification information of the destination UAV 100. The transmission / reception unit 312 may transmit information including identification information of the source and destination UAVs 100. The UAV 100 may determine whether the information is information addressed to itself based on the identification information included in the information transmitted from the remote operation control unit 310. The transmission / reception unit 312 receives the position information of the UAV 100 serving as the imaging role from the UAV 100 serving as the imaging role, and transmits the UAV 100 to the standby UAV 100. The transmission / reception unit 312 receives the position information from the UAV 100 serving as the imaging role. The transmission / reception unit 312 transmits the position information to the waiting UAV 100 in order to move the waiting UAV 100 using the position indicated by the position information as the target position. The transmission / reception unit 312 receives setting information indicating the setting of the imaging device 220 serving as the imaging role from the UAV 100 serving as the imaging role. The transmission / reception unit 312 transmits the setting information to the waiting UAV 100 in order to set the imaging device 220 of the waiting UAV 100 based on the setting information.

  The instruction generation unit 314 generates a control instruction for controlling the UAV 100. The command generation unit 314 generates a movement control command and an imaging control command. The command generation unit 314 generates a movement control command and an imaging control command based on an instruction received from the user via the operation unit 340.

  The communication switching unit 316 switches between a state in which communication with the UAV 100 serving as an imaging agent is possible and a state in which communication with the UAV 100 in standby is possible. The communication switching unit 316 may switch the communication link with the UAV 100 as appropriate. When the communication switching unit 316 switches the communication link with the UAV 100, the remote operation device 300 can remotely control the imaging device UAV 100 and the imaging device 220 of the imaging role UAV 100, the standby UAV 100, and the standby. The operation is switched between a state in which the imaging device 220 of the UAV 100 can be remotely operated.

  When the remote operation device 300 can simultaneously communicate with a plurality of UAVs 100 via the communication interface 350, the communication switching unit 316 may not appropriately switch the communication link with the UAV 100. That is, the transmission / reception unit 312 may form a plurality of communication links with the plurality of UAVs 100 and communicate with the plurality of UAVs 100 in parallel.

  FIG. 5 is a diagram illustrating an example of an imaging sequence. The first UAV in FIG. 5 is the UAV 100 of the imaging role, and the second UAV is the standby UAV 100.

  The remote operation device 300 transmits a movement control command to the first UAV (S100). When the first UAV moves to a desired position, the remote operation device 300 transmits an imaging control command to the first UAV (S102). The first UAV sets an angle of view, a focal length, and an imaging direction of the imaging device 220 mounted on the first UAV according to an imaging control command. The first UAV transmits the moving image captured by the imaging device 220 to the remote operation device 300 as image data (S104). The first UAV may transmit the image data to an image data collection device other than the remote operation device 300.

  If the state of the first UAV satisfies a predetermined condition while the imaging operation by the first UAV is continuing, the first UAV transmits an outgoing request notification to the remote control device 300 (S106). For example, when the remaining capacity of the battery of the first UAV is less than a predetermined amount, or when a malfunction of the first UAV or the first UAV imaging device 220 is detected, the first UAV sends an outgoing request notification to the remote control device 300. Send. The first UAV further transmits position information indicating the position of the first UAV to the remote operation device 300 (S108). The first UAV may transmit the dispatch request notification and the position information to the remote operation device 300 together.

  The remote operation device 300 transmits the dispatch request notification received from the first UAV to the second UAV (S110). The remote operation device 300 also transmits the position information received from the second UAV to the second UAV (S112). The remote operation device 300 may send the dispatch request notification and the position information together to the second UAV. The remote operation device 300 may temporarily disconnect communication with the first UAV while communicating with the second UAV. While the communication with the first UAV is disconnected, the remote operation device 300 may display a message indicating that the first UAV cannot be temporarily operated on the display unit 330 to notify the user. When communicating with the second UAV, the remote operation device 300 may temporarily switch the communication link from a state where the first UAV can be remotely operated to a state where the second UAV can be remotely operated.

  When the second UAV receives the dispatch request notification and the position information, the second UAV moves using the position indicated by the position information as the target position. When the second UAV moves to the target position, the second UAV transmits a movement completion notification to the remote operation device 300 (S114). The remote operation device 300 transmits the movement completion notification received from the second UAV to the first UAV (S116).

  Upon receiving the movement completion notification, the first UAV transmits setting information indicating the current setting of the first UAV imaging device 220 to the remote operation device 300 (S118). The first UAV transmits detailed position information to the remote operation device 300 in order to more accurately convey the imaging range captured by the imaging device 220 of the first UAV and the position of the first UAV to the second UAV (S120). The detailed position information is information generated based on information collected from various sensors provided in the first UAV, for example, in order to specify the position of the first UAV in more detail. The detailed position information may be information based on image data obtained from the first UAV imaging device 230. The detailed position information may include three-dimensional spatial data generated based on image data obtained from the plurality of imaging devices 230 of the first UAV. The detailed position information may include information indicating the position of the first UAV on the three-dimensional space data generated based on the plurality of image data from the plurality of imaging devices 230 of the first UAV. The detailed position information is information indicating a plurality of image data captured by the plurality of imaging devices 230 of the first UAV, and distances from the first UAV calculated from the plurality of image data to each object existing around the first UAV. And may include.

  The remote operation device 300 transmits the setting information to the second UAV (S122). The remote control device 300 further transmits detailed position information to the second UAV (S124). The second UAV sets the angle of view, the focal length, and the imaging direction of the imaging device 220 based on the setting information. The second UAV further adjusts the position of the second UAV with reference to the detailed position information. For example, the movement control unit 118 of the second UAV, based on the information indicating the position of the first UAV on the three-dimensional spatial data provided from the first UAV, on the three-dimensional spatial data generated by the second UAV Map location. The movement control unit 118 of the second UAV maps the position of the second UAV on the three-dimensional space data generated by the second UAV. The movement control unit 118 of the second UAV moves the position of the second UAV so that the second UAV approaches the first UAV based on the position of the first UAV and the position of the second UAV on the three-dimensional spatial data generated by the second UAV. May be adjusted. The second UAV may fly away from the first UAV by a predetermined distance or more in order to avoid a collision with the first UAV. In this case, the movement control unit 118 of the second UAV may adjust the position of the second UAV so that the distance from the first UAV approaches the predetermined distance.

  The movement control unit 118 of the second UAV calculates a plurality of image data from the plurality of imaging devices 230 of the first UAV, and a distance from the first UAV calculated from the plurality of image data to each object existing around the first UAV. The position of the second UAV may be adjusted based on the information shown. The movement control unit 118 of the second UAV specifies each object corresponding to each object included in the detailed position information from the plurality of image data from the plurality of imaging devices 230 of the second UAV. The movement controller 118 of the second UAV calculates the distance to each identified object based on a plurality of image data from the plurality of imaging devices 230 of the second UAV. The movement control unit 118 of the second UAV is based on the distance to each object included in the detailed position information and the distance to each object calculated based on the plurality of image data from the plurality of imaging devices 230 of the second UAV. Thus, the position of the second UAV may be adjusted so that the second UAV is closer to the first UAV.

  When the adjustment of the imaging range of the imaging device 220 is completed, the second UAV starts imaging by the imaging device 220. The second UAV starts to transmit image data that is a moving image captured by the imaging device 220 to the remote control device 300. The second UAV transmits an imaging start notification to the remote operation device 300 (S126). The remote operation device 300 transmits an imaging start notification to the first UAV (S128). The display unit 330 may display on the display unit 330 that the imaging start notification has been received from the second UAV. The remote operation device 300 may inquire the user whether or not the imaging start notification may be transmitted to the first UAV, and may transmit the imaging start notification to the first UAV when permission is obtained from the user.

  Upon receiving the imaging start notification, the first UAV ends the imaging operation by the first UAV imaging device 220, and the first UAV movement control unit 118 moves the first UAV in a direction away from the second UAV. The movement controller 118 of the first UAV may control the movement of the first UAV so that the first UAV is returned to the station 400.

  After the imaging start notification is transmitted to the first UAV, a communication link is formed between the second UAV and the remote operation device 300, so that the remote operation device 300 can be remotely operated with the second UAV. The remote control device 300 may adjust the position of the second UAV by transmitting a movement control command to the second UAV (S130). The remote operation device 300 transmits an imaging control command to the second UAV and adjusts the imaging range of the second UAV imaging device 220 (S132). After the communication link with the remote operation device 300 is formed, the second UAV transmits the moving image captured by the second UAV imaging device 220 to the remote operation device 300 as image data (S134).

  As described above, according to the imaging system 10, a plurality of UAVs 100 in which the imaging device 220 is mounted can be linked to capture a desired imaging range. For example, when the remaining capacity of the battery 140 of the UAV 100 being imaged decreases, the alternative UAV 100 automatically moves to the vicinity of the UAV 100 being imaged. The alternative UAV 100 automatically sets the imaging range of the imaging device 220 based on the setting information of the UAV 100 being imaged, and starts imaging. When the alternative UAV 100 starts imaging, the target of remote operation by the remote operation device 300 is automatically switched from the UAV 100 being imaged to the alternative UAV 100. Therefore, the user can continue to operate the remote operation device 300 without being aware of the switching of the UAV 100. The user can continue shooting in a desired imaging range without being aware of switching of the UAV 100. The user can continue shooting in a desired imaging range without being aware of the remaining capacity of the battery 140 of the UAV 100.

  At least one of the steps shown in the above embodiments may be implemented by hardware or a program instructing related hardware. The program may be stored in a computer readable recording medium. The recording medium may include at least one of a ROM, a magnetic disk, and an optical disk.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. The output can be realized in an arbitrary order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

10 Imaging system 100 UAV
102 UAV main body 110 UAV control unit 112 Dispatch request notification transmission unit 114 Dispatch request notification reception unit 116 Location information reception unit 118 Movement control unit 120 Location information transmission unit 122 Setting information transmission unit 124 Setting information reception unit 126 Imaging setting unit 128 Imaging start Notification receiving unit 130 Imaging control unit 132 Imaging start notification transmission unit 140 Battery 150 Communication interface 160 Memory 200 Gimbal 210 Rotor blade mechanism 220 Imaging device 230 Imaging device 240 GPS receiver 260 Magnetic compass 270 Barometric altimeter 300 Remote operation device 310 Remote operation control Unit 312 transmission / reception unit 314 command generation unit 316 communication switching unit 330 display unit 340 operation unit 350 communication interface 360 memory 400 station

Claims (13)

A first moving body on which the first imaging device is mounted;
A second moving body equipped with a second imaging device,
The first moving body is:
A first transmitter that transmits position information indicating the position of the first moving body;
A second transmitter that transmits setting information indicating the setting of the first imaging device;
The second moving body is
A first receiver for receiving the position information;
A first control unit that controls movement of the second moving body with respect to the position indicated by the position information;
A second receiving unit for receiving the setting information;
A setting unit configured to set the second imaging device based on the setting information ;
An imaging start notification is transmitted when the first control unit moves the second moving body with respect to the position indicated by the position information and the second imaging device starts imaging with the setting indicated by the setting information. A third transmitter to
Have a <br/> a fourth control unit that controls the second imaging device,
The first moving body is:
A third receiver for receiving the imaging start notification;
A third control unit for controlling the first imaging device;
Further comprising
The first imaging device and the second imaging device record moving images,
The third control unit records the time when the third reception unit receives the imaging start notification in association with the moving image recorded by the first imaging device;
The fourth control unit sets a time at which the recording of the moving image is started by the second imaging device with a setting indicated by the setting information, or a time at which the third transmission unit transmits the imaging start notification. An imaging system for recording in association with the moving image recorded by the imaging device. The first moving body is:
A second control unit for controlling movement of the first moving body;
The second control unit moves the second moving body with respect to the position indicated by the position information, and the second imaging device starts imaging with the setting indicated by the setting information . When the first moving body receives the imaging start notification transmitted from the moving body ,
The imaging system according to claim 1, wherein the second control unit moves the first moving body to a station that provides a place where the first moving body waits . The first moving body is:
A third control unit for controlling the first imaging device;
The second control unit moves the second moving body with respect to the position indicated by the position information, and the second imaging device starts imaging with the setting indicated by the setting information . When the first moving body receives the imaging start notification transmitted from the moving body ,
The imaging system according to claim 1, wherein the third control unit terminates an imaging operation by the first imaging device. The first moving body is:
A third control unit for controlling the first imaging device;
The imaging system according to any one of claims 1 to 3, wherein the third control unit terminates an imaging operation by the first imaging device when the third receiving unit receives the imaging start notification. . The first mobile unit receives a movement completion notification transmitted from the second mobile unit in response to the first control unit moving the second mobile unit relative to the position indicated by the location information. Then
The second transmission unit transmits the setting information,
The second reception unit receives the setting information, imaging system according to any one of claims 1 to 4. The imaging system according to any one of claims 1 to 5 , wherein the setting information includes at least one of an optical condition and an imaging direction of the first imaging device. The first moving body is:
A fourth transmission unit configured to transmit a dispatch request notification for requesting dispatch of the second mobile unit when a predetermined condition is satisfied;
The second moving body is
A fourth receiver for receiving the dispatch request notification;
The first control unit, when the fourth receiver receives the turnout request notification, to the location where the position information indicates, to control the movement of the second moving body, of claims 1-6 The imaging system according to any one of the above. When the remaining capacity of the battery mounted on the first moving body is less than a predetermined amount or when a failure of the first moving body or the first imaging device is detected, the fourth transmitting unit is The imaging system according to claim 7 , wherein the dispatch request notification is transmitted. A remote operation device for remotely operating the first moving body and the first imaging device, and the second moving body and the second imaging device;
The remote control device includes a transceiver for transmitting and receiving the position information from the first movable body in the second mobile imaging system according to any one of claims 1 to 8. The imaging system according to claim 9 , wherein the transmission / reception unit receives the setting information from the first moving body and transmits the setting information to the second moving body. The remote operation device can be switched between a state in which the first imaging device and the first moving body can be remotely operated and a state in which the second imaging device and the second moving body can be remotely operated. Item 11. The imaging system according to Item 9 or 10 . A first moving body on which the first imaging device is mounted;
  A second moving body on which a second imaging device is mounted;
  A remote operation device for remotely operating the first moving body and the first imaging device, and the second moving body and the second imaging device;
With
  The first moving body is:
  A first transmitter that transmits position information indicating the position of the first moving body;
  A second transmitter for transmitting setting information indicating the setting of the first imaging device;
Have
  The second moving body is
  A first receiver for receiving the position information;
  A first control unit that controls movement of the second moving body with respect to the position indicated by the position information;
  A second receiving unit for receiving the setting information;
  A setting unit configured to set the second imaging device based on the setting information;
Have
  The remote control device includes a transmission / reception unit that receives the position information from the first mobile body and transmits the position information to the second mobile body,
  The remote operation device is switched between a state in which the first imaging device and the first moving body can be remotely operated and a state in which the second imaging device and the second moving body can be remotely operated. system. The remote operation device is in a state in which the first moving body and the first imaging device can be remotely operated before the second imaging device starts imaging with the setting indicated by the setting information, and the setting information The imaging system according to claim 11 or 12 , wherein the second moving body and the second imaging device are remotely operable after imaging is started by the second imaging device with the setting indicated by.

Images