JP6528760B2 - INFORMATION PROCESSING APPARATUS, CONTROL METHOD THEREOF, AND PROGRAM - Google Patents

Description

  The present invention relates to an information processing apparatus, a control method therefor, and a program, and in particular, a technology capable of photographing an image exceeding the operation limit of a zoom operation or pan operation of an imaging device mounted on an unmanned airplane by a simple operation. About.

  In recent years, for example, cameras are mounted on unmanned airplanes (unmanned air vehicles) such as drone, and various images are taken.

  For example, Patent Document 1 discloses that when a drone shoots the same person as a person photographed by a surveillance camera, the drone follows the person.

JP 2014-149620

  However, for example, although the camera mounted on the drone can perform zoom-in operation and can shoot the zoomed-in image, for example, when the tele end which is the limit of the zoom-in operation is reached, the video is further zoomed in and photographed I can not do it. Similarly, the camera mounted on the drone can perform zoom-out operation and can shoot the zoomed-out image, but if the wide end which is the limit of the zoom-out operation is reached, the image is further zoomed out Can not shoot. Also, similarly, the camera mounted on the drone can perform pan operation and can shoot the image subjected to pan operation, but if the pan end which is the limit of pan operation is reached, pan operation of the image is further performed. I can not shoot.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a mechanism that enables imaging of an image exceeding the operation limit of a zoom operation or pan operation of an imaging device mounted on an unmanned airplane by a simple operation.

The present invention is an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, the reception means for receiving an operation instruction related to the zoom of the lens of the imaging device, and the operation received by the reception means operation of the lens of the zoom of the imaging apparatus according to instruction, if the exceeds the operating limit of the imaging device, a flight interaction in the direction of the zoom corresponding to the operation instruction, the imaging device according to the operation instruction And an instruction unit for instructing the unmanned aerial vehicle to perform a flight operation at a speed corresponding to the zoom speed of the lens .
Further, the present invention is an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, the reception means for receiving an operation instruction of the panning operation of the imaging device, and the reception means When the panning operation of the imaging device by the operation instruction exceeds the operation limit of the panning operation of the imaging device, it is a flying operation that turns in the pan direction of the operation instruction, and the panning of the imaging device by the operation instruction And an instruction unit that instructs the unmanned aircraft to perform a flight operation at a speed corresponding to the speed.
Further, the present invention is an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, the reception means for receiving the operation instruction of the imaging device, and the operation instruction received by the reception means When the operation of the imaging device exceeds the operation limit of the imaging device, the instruction to cause the unmanned airplane to perform the flight operation corresponding to the operation instruction, and the focus mode of the imaging device are manually adjusted for focus And an instruction unit for instructing the unmanned airplane to change from the manual focus mode to the automatic focus mode for automatically adjusting the focus.

Further, the present invention is a control method in an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, wherein the accepting unit accepts an operation instruction regarding zooming of a lens of the imaging device. If, instruction means, the operation of the zoom of the image pickup apparatus of the lens by the operation instruction received by the reception step, if the exceeds the operating limit of the imaging device, in the direction of the zoom corresponding to the operation instruction And an instruction step of instructing the unmanned aircraft to perform the flying operation at a speed according to the zoom speed of the lens of the imaging device according to the operation instruction. I assume.
Further, the present invention is a control method in an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, the accepting means accepting an operation instruction of the panning operation of the imaging device; And an instruction unit is a flying operation that turns in a pan direction of the operation instruction when the pan operation of the imaging device according to the operation instruction received in the reception step exceeds the operation limit of the pan operation of the imaging device. And an instruction step of instructing the unmanned aerial vehicle to perform a flight operation at a speed according to the pan speed of the imaging device according to the operation instruction.
Further, the present invention is a control method in an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, the accepting means accepting an operation instruction of the imaging device, an instructing means And an instruction to cause the unmanned airplane to perform a flight operation corresponding to the operation instruction when the operation of the imaging device according to the operation instruction received in the receiving step exceeds the operation limit of the imaging device, and the imaging device And d) instructing the unmanned airplane to change the focus mode from the manual focus mode in which the focus is manually adjusted to the automatic focus mode in which the focus is automatically adjusted.

Further, the present invention causes a computer, which is programmed to function as the information processing apparatus.

  According to the present invention, it is possible to capture an image exceeding the operation limit of the zoom operation or the pan operation of the imaging device mounted on the unmanned airplane by a simple operation.

It is a figure showing an example of the system concerning an embodiment of the invention. FIG. 2 is a diagram illustrating an example of a functional configuration diagram of hardware of the information processing apparatus 101, the drone 102, and the imaging device 104 illustrated in FIG. 1; When the zoom-in instruction is given even if the tele end is reached, the drone 102 flies in the imaging direction of the imaging device 104, and when the zoom-out instruction is given even when the wide end is reached, It is an example of the schematic diagram which shows a mode that the drone 102 flies in the reverse direction of an imaging direction. Even when reaching the pan end which is the upper limit of the right rotation operation of the imaging device 104, the drone 104 indicates that the drone 102 rotates and flies in the pan direction (right rotation) when the right rotation pan operation is instructed. It is an example of the top view of. FIG. 6 is a diagram showing an example of a flowchart showing a control processing procedure executed by a CPU (control means) of the information processing apparatus 101.

Hereinafter, the present invention will be described in detail according to a preferred embodiment with reference to the attached drawings.
<Description of FIG. 1>
FIG. 1 is a diagram showing an example of a system according to an embodiment of the present invention.

  As shown in FIG. 1, the system according to the embodiment of the present invention is a system including an information processing apparatus 101 and a drone 102 (also referred to as an unmanned air vehicle or an unmanned airplane) provided with an imaging device (camera) 104. is there.

  The information processing apparatus 101, the drone 102, and the imaging apparatus 104 are communicably connected to each other via the network 105. The network 105 may be a wired network or a wireless network.

  The information processing apparatus 101 is an application example of the information processing apparatus of the present invention, and instructs the unmanned airplane (drone 102) provided with the imaging apparatus 104 to perform a flight operation.

  The information processing apparatus 101 includes a console 103 as an operation unit 202. The console 103 includes an operation unit 106 for operating the drone 102 and an operation unit 107 for operating the imaging device 104 mounted on the drone 102.

  The operation unit 107 is an application example of the reception unit of the present invention, and receives an operation instruction of the imaging device 104.

  The user operates the operation unit 106 to control the flight direction to the drone 102 and the flight speed. Then, when the information processing apparatus 101 receives the operation of the operation unit 106, the information processing apparatus 101 transmits, to the drone 102, an instruction to fly according to the received operation content, and when the drone 102 receives the instruction, the information processing apparatus 101 The propeller drive unit 210 controls the propeller 209 to fly as instructed. Thus, the information processing apparatus 101 transmits the content of the operation instruction operated by the operation unit 106 to the drone 102, and operates the flight of the drone.

  Also, the user operates the operation unit 107 to perform the zoom-in operation, the zoom-out operation, and the pan operation of the imaging device 104. When the information processing apparatus 101 receives an operation of the operation unit 107, the information processing apparatus 101 transmits an instruction to the imaging apparatus 104 to operate according to the received operation content, and when the imaging apparatus 104 receives the instruction, the information processing apparatus 101 As instructed, the unit (the zoom operation unit 213 and the pan operation unit 214) that operates each member such as the lens in the imaging apparatus to operate is operated. As described above, the information processing apparatus 101 transmits the content of the operation instruction operated by the operation unit 107 to the imaging apparatus 104, and operates the zoom operation of the lens of the imaging apparatus 104 and the pan operation in the imaging direction.

In the present embodiment, the drone 102 is already flying by the operation of the operation unit 106, and the imaging apparatus 104 performs the zoom-in operation, the zoom-out operation, and the pan operation according to the operation of the operation unit 107. Further, it is assumed that the imaging device 104 is always photographed while the drone 102 is in flight. Here, the imaging device 104 is a digital camera that can function as a weather camera, a security camera, or the like.
<Description of FIG. 2>

  FIG. 2 is a diagram illustrating an example of a functional configuration diagram of hardware of the information processing apparatus 101, the drone 102, and the imaging device 104 illustrated in FIG.

  As shown in FIG. 2, the information processing apparatus 101 includes a control unit 201, an operation unit 202, a communication unit 203, a display unit 204, and a storage unit 205.

  The control unit 201 is, for example, a CPU, and is a unit that centrally controls each device and controller of the information processing apparatus 101.

  The operation unit 202 is a unit that receives an operation by the user, and is, for example, a keyboard, a mouse, the operation unit 106 of the console 103, and the operation unit 107.

  The communication unit 203 is a unit that communicates with each device (drone 102 and imaging device 104) communicably connected to the network 105.

  The display unit 204 is a display that displays an image (moving image, still image) captured by the imaging device 104. The imaging device 104 constantly transmits the video captured by the imaging device 104 to the information processing device 101, and when the information processing device 101 receives the transmitted video, the imaging device 104 displays the received video on the display unit 204.

  The storage unit 205 is a storage unit such as a RAM, a ROM, or a hard disk. Programs necessary for operating the information processing apparatus 101 (including programs for executing the processing of FIG. 5) and the like are stored.

  Further, as shown in FIG. 2, the drone 102 includes a control unit 206, a communication unit 207, a storage unit 208, a propeller 209, a propeller drive unit 210, and an imaging device 104.

  The control unit 206 is, for example, a CPU that centrally controls the devices and the controller of the drone 102.

  The communication unit 207 is a unit that communicates with each device (the information processing device 101) communicably connected to the network 105.

The storage unit 208 is a storage unit such as a RAM, a ROM, or a hard disk. Programs and the like necessary for operating the drone 102 are stored.
The propeller 209 is a propeller for flying the drone.

  The propeller drive unit 210 is controlled by the control unit 206, and is a drive source such as a motor for driving the propeller 209 to rotate.

  The imaging device 104 further includes a control unit 211, a zoom operation unit 213, a pan operation unit 214, and a communication unit 212.

  The control unit 211 is, for example, a CPU, and is a unit that centrally controls each device and controller of the imaging device 104.

  The communication unit 212 is a unit that communicates with each device (the information processing device 101) communicably connected to the network 105.

  The zoom operation unit 213 is a unit that performs a zoom-in operation of the lens of the imaging device 104 according to a zoom-in operation instruction by a user operation, and the imaging device 104 according to a zoom-out operation instruction by a user operation. Is a unit that performs the zoom out operation of the lens.

The pan operation unit 214 is a unit that performs a pan operation to rotate the imaging device 104 clockwise or counterclockwise in response to a pan operation instruction by a user operation.
<Description of FIG. 3>

  Next, when a zoom-in instruction is issued even when the tele end is reached using FIG. 3, the drone 102 flies in the imaging direction of the imaging device 104 and a zoom-out instruction is issued even when the wide end is reached. The case where the drone 102 flies in the direction opposite to the imaging direction of the imaging device 104 will be described.

  FIG. 3 shows that the drone 102 flies in the imaging direction of the imaging device 104 when the zoom-in instruction is given even when the tele end is reached, and when the zoom-out instruction is given even when the wide end is reached, FIG. 7 is an example of a schematic view showing how the drone 102 flies in the opposite direction of the imaging direction of the imaging device 104.

  Currently, when the process shown in FIG. 5 starts, for example, when the drone 102 stands still while flying to the position of FIG. 3A and the imaging device 104 captures an image of the object 108, the tele end is reached. When the zoom-in instruction is given by the operation unit 107 (S502: YES and S503: YES), the drone flying at the position of (A) shown in FIG. Move to position 302 of B) and fly closer to the object 108.

  As a result, even if the tele end has been reached, it is possible to perform telephoto photography beyond the tele end. Further, since the drone 102 can be operated by the operation unit 107, the operability of the drone 102 including the imaging device 104 can be improved.

  Also, at the start of the process shown in FIG. 5, for example, when the drone 102 stands still while flying to the position of FIG. 3A and the imaging device 104 captures an image of the object 108, When a zoom-out instruction is given from the operation unit 107 in the state where the operation has reached (S513: YES and S514: YES), the drone flying at the position of (A) shown in FIG. Move to the position 302 of (C) and fly away from the object 108.

  This makes it possible to perform wide-angle shooting beyond the wide-angle end even when the wide-angle end is reached. Further, since the drone 102 can be operated by the operation unit 107, the operability of the drone 102 including the imaging device 104 can be improved.

Further, as shown in FIG. 3, the direction in which the drone 102 moves in S506 and S517 is the shooting direction of the imaging device 104 or the opposite direction.
<Description of FIG. 4>

  Next, even when the pan end, which is the upper limit of the right rotation operation of the imaging device 104, is reached using FIG. 4, the drone 102 in the pan direction (right rotation) when a right rotation pan operation is instructed. Explain that the aircraft is flying.

  FIG. 4 shows that the drone 102 rotates and flies in the pan direction (right rotation) when the right rotation pan operation is instructed even when the pan end, which is the upper limit of the right rotation operation of the imaging device 104, is reached. FIG. 12 is an example of a top view of the drone 102 showing

  Currently, when the process shown in FIG. 5 is started, for example, it is assumed that the drone 102 stands still while flying to the position of FIG. 4A and the imaging device 104 is photographing.

  In that case, if the right rotation pan operation is instructed by the operation unit 107 in a state where the right rotation pan end has not been reached (S523: YES, and S524: NO), the imaging device 104 is Pan the rotation. As a result, the result of the panning operation is shown in FIG. 4 (B). Here, for example, it is assumed that the state shown in FIG. 4B is a pan end rotated right. As shown in FIG. 4B, in the state where the right rotation pan end is reached, when the right rotation pan operation is instructed from the operation unit 107 (S524: YES and S525: YES), S527. Then, the drone shown in FIG. 4B performs a flight to rotate to the position 302 in FIG. 4C.

  As a result, even if the right rotation pan end is reached, it is possible to perform shooting in the direction beyond the right rotation pan end. Further, since the drone 102 can be operated by the operation unit 107, the operability of the drone 102 including the imaging device 104 can be improved.

In FIG. 4, the pan operation of right rotation has been described as an example, but the pan operation of left rotation is also the same, and by changing the right rotation described in FIG. Even if it has reached, it is possible to take pictures in the direction of the left rotation pan end or more.
<Description of FIG. 5>

  The control processing procedure executed by the CPU (control means) of the information processing apparatus 101 will be described using FIG.

  FIG. 5 is a diagram showing an example of a flowchart showing a control processing procedure executed by the CPU (control means) of the information processing apparatus 101.

  When the process of FIG. 5 is started, it is assumed that the drone 102 is flying while standing still at a certain position.

  The user operates the operation unit 106 of the operation console 103 for operating the drone 102 to operate the flight direction to the drone 102 and the flight speed. When the information processing apparatus 101 receives an operation of the operation unit 106, the information processing apparatus 101 transmits an instruction to fly to the drone 102 according to the received operation content, and when the drone 102 receives the instruction, the information processing apparatus 101 receives the instruction. In the street, the propeller drive unit 210 controls the propeller 209 to fly.

  Further, the user operates the operation unit 107 of the console 103 for operating the imaging device 104 mounted on the drone 102 to perform the zoom-in operation, the zoom-out operation, and the pan operation of the imaging device 104. When the information processing apparatus 101 receives an operation of the operation unit 107, the information processing apparatus 101 transmits an instruction to the imaging apparatus 104 to operate according to the received operation content, and when the imaging apparatus 104 receives the instruction, the information processing apparatus 101 As instructed, the unit (the zoom operation unit 213 and the pan operation unit 214) that operates each member such as the lens in the imaging apparatus to operate is operated.

  The information processing apparatus 101 determines whether an instruction for the zoom-in operation has been received by the operation unit 107 (S501). If it is determined that the zoom-in operation instruction has not been received (NO), the process proceeds to step S512. If it is determined that the zoom-in operation instruction has been received (YES), the process proceeds to step S502.

  The determination of YES in S501 means that the operation instruction accepted by the acceptance unit in S501 is an operation instruction for the zoom-in operation of the imaging apparatus.

  Then, the information processing apparatus 101 determines whether the information indicating that the tele end has been reached by the zoom-in operation is received from the imaging apparatus 104, whereby the image pickup apparatus 104 reaches the tele end by the zoom-in operation. It is determined whether or not (S502). When the imaging device 104 reaches the tele end by the zoom-in operation, the imaging device 104 transmits, to the information processing apparatus 101, information indicating that the tele end is reached by the zoom-in operation.

  When it is determined that the imaging apparatus 104 has reached the tele end by the zoom-in operation (S502), the information processing apparatus 101 determines whether the zoom-in operation by the user is further received by the operation unit 107 (S502) S503). Then, when it is determined that the zoom-in operation by the user is further received by the operation unit 107 (YES), the information processing apparatus 101 shifts the processing to S504.

  S502 is an application example of the determination unit of the present invention, and determines whether the zoom-in operation of the imaging device 104 according to the operation instruction received by the reception unit exceeds the limit (tele end) of the zoom-in operation of the imaging device 104.

  When it is determined that the imaging device 104 has not reached the tele end in the zoom-in operation (S502: NO), the information processing device 101 transmits an instruction for the zoom-in operation to the imaging device 104 and transmits the instruction to the imaging device 104. The zoom-in operation is performed, and the process returns to step S501.

  If the information processing apparatus 101 determines that the zoom-in operation by the user is not further received by the operation unit 107 (S503: NO), the process returns to S501.

  The information processing apparatus 101 acquires, from the imaging apparatus 104, the current pan position, tilt position, and zoom-in speed immediately before reaching the tele end (S504).

  The zoom-in speed acquired in S504 may be the zoom-in speed at which the instruction for the zoom-in operation is currently received by the operation unit 107, and the zoom-in speed in that case is acquired from the information processing apparatus 101.

  Then, the information processing apparatus 101 identifies the imaging direction in which the imaging device 104 is imaging from the pan position and the tilt position acquired in S504. Then, the information processing apparatus 101 specifies the imaging direction as the movement direction in which the drone 102 is moved (S505).

  The information processing apparatus 101 moves the drone 102 at the zoom-in speed acquired in S504 toward the movement direction identified in S505, and instructs the imaging apparatus 104 to approach the imaging direction. Specifically, the information processing apparatus 101 transmits the information indicating the moving direction and the information (vector information) indicating the zoom-in speed to the drone 102, and when the drone 102 receives the vector information, the vector According to the information, the drone 102 is caused to fly and approach the subject. For example, the drone flying in the position of (A) shown in FIG. 3 moves to the position 302 of (B) and flies to approach the object 108 (S506).

  S506 is an application example of the instruction means of the present invention, and when the determination means of S502 determines that the limit (tele end) of the zoom-in operation of the imaging apparatus 104 is exceeded (S502: YES), the acceptance means accepts As the flight operation corresponding to the operation instruction, the unmanned aircraft is instructed to perform an operation of flying and moving the unmanned airplane (drone 102) in the imaging direction of the imaging device 104.

  Next, the information processing apparatus 101 determines whether the current imaging apparatus 104 is in the manual focus mode in which the focus is manually adjusted or in the automatic focus mode in which the focus is automatically adjusted. Specifically, the information processing apparatus 101 acquires, from the imaging apparatus 104, information indicating whether the current mode is the manual focus mode or the automatic focus mode, and the current imaging is performed according to the information. It is determined whether the apparatus 104 is in the manual focus mode in which the focus is manually adjusted or in the automatic focus mode in which the focus is automatically adjusted (S507).

  When it is determined that the current imaging device 104 is in the manual focus mode (S507: YES), the information processing device 101 transmits, to the imaging device 104, a setting change request to operate in the automatic focus mode. Then (S508), upon receiving the request, the imaging apparatus 104 switches the operation from the manual focus mode to the automatic focus mode (mode of auto focus). In addition, when it is determined that the current imaging device 104 is in the automatic focus mode (S507: NO), the information processing device 101 shifts the processing to S509.

  Further, the information processing apparatus 101 acquires, from the drone 102, position information (including information on longitude, latitude, and altitude) of the flight position at which the drone 102 is currently flying. Then, the information processing apparatus determines whether the flight position of the drone 102 currently in flight has exceeded a predetermined area registered in advance in the information processing apparatus 101 (S509), and the predetermined area is exceeded. If it is determined (YES), an instruction to stop the movement of the drone 102 is transmitted to the drone 102 to stop the movement of the drone 102 (S511). On the other hand, when it is determined that the predetermined area is not exceeded (S 509: NO), the information processing apparatus 101 determines whether the user further accepts the zoom-in operation by the user using the operation unit 107 at present. (S510). When it is determined that the zoom-in operation is not further received (S510: NO), the process proceeds to S511, and the movement of the drone 102 is stopped (S511). On the other hand, when it is determined that the zoom-in operation is further received (S510: YES), the process proceeds to S504, and the drone 102 is subsequently moved closer to the subject 108.

  When the information processing apparatus 101 stops the movement of the drone 102 in S511, the process returns to S501.

  As described above, the camera mounted on the drone zooms in, reaches the tele end which is the limit of the zoom in operation, and the drone approaches the subject and shoots the subject even if the user receives an instruction to zoom in from the user. This makes it possible to acquire an image (zoomed image) taken as if it were zoomed in.

  If it is determined that the instruction for the zoom-in operation has not been received by the operation unit 107 (S501: NO), the information processing apparatus 101 determines whether the instruction for the zoom-out operation has been received by the operation unit 107 (S512) ). Then, when it is determined that the instruction of the zoom out operation has not been received (NO), the process proceeds to S523. If it is determined that the zoom out operation instruction has been received (YES), the process proceeds to step S513.

  Step S512 is an application example of the receiving unit of the present invention, and the operation instruction received by the receiving unit is an operation instruction of the zoom-out operation of the imaging apparatus 104.

  Then, the information processing apparatus 101 determines whether the information indicating that the wide end has been reached by the zoom out operation has been received from the imaging apparatus 104, so that the imaging apparatus 104 moves to the wide end by the zoom out operation. It is determined whether it has reached (S513). When the imaging device 104 reaches the wide end in the zoom out operation, the imaging device 104 transmits information indicating that the wide end is reached in the zoom out operation to the information processing apparatus 101.

  S513 is an application example of the determination unit of the present invention, and it is determined whether the zoom-out operation of the imaging device 104 according to the operation instruction received by the reception unit exceeds the limit of the zoom-out operation of the imaging device 104.

  When it is determined that the imaging device 104 has reached the wide end in the zoom out operation (S513: YES), the information processing apparatus 101 determines whether the user has further received a zoom out operation by the user via the operation unit 107. Is determined (S514). Then, when it is determined that the zoom-out operation by the user is further received by the operation unit 107 (YES), the information processing apparatus 101 shifts the processing to S515.

  When it is determined that the imaging device 104 has not reached the wide end in the zoom-out operation (S513: NO), the information processing device 101 transmits an instruction for the zoom-out operation to the imaging device 104, and the imaging device The zoom out operation is performed by 104, and the process returns to step S501.

  Further, when it is determined that the zoom-out operation by the user is not further received by the operation unit 107 (S514: NO), the information processing apparatus 101 returns the process to S501.

  The information processing apparatus 101 acquires, from the imaging apparatus 104, the current pan position, tilt position, and zoom-out speed immediately before reaching the wide end (S515).

  The zoom-out speed acquired in step S515 may be the zoom-out speed currently accepting a zoom-out operation instruction from the operation unit 107, and the zoom-out speed in that case is acquired from the information processing apparatus 101.

  Then, the information processing apparatus 101 identifies the imaging direction in which the imaging device 104 is imaging from the pan position and the tilt position acquired in S515. Then, the information processing apparatus 101 specifies the direction opposite to the imaging direction as the movement direction in which the drone 102 is moved (S516).

  The information processing apparatus 101 moves the drone 102 at the zoom-out speed acquired in step S515 toward the movement direction specified in step S516, and instructs the imaging apparatus 104 to move away from the subject 108. Specifically, the information processing apparatus 101 transmits the information indicating the moving direction and the information (vector information) indicating the zoom out speed to the drone 102, and when the drone 102 receives the vector information, the information processing apparatus 101 The drone 102 is caused to fly away from the subject according to the vector information. For example, the drone flying in the position of (A) shown in FIG. 3 moves to the position 301 of (C) and flies away from the object 108 (S517).

  S517 is an application example of the instruction unit of the present invention, and when it is determined that the limit (wide end) of the zoom out operation of the imaging apparatus 104 is exceeded by the determination unit of S513, the operation instruction received by the reception unit As the corresponding flight operation, the unmanned airplane is instructed to perform an operation of flying and moving the unmanned airplane (drone 102) in the direction opposite to the imaging direction of the imaging device 104.

  Next, the information processing apparatus 101 determines whether the current imaging apparatus 104 is in the manual focus mode in which the focus is manually adjusted or in the automatic focus mode in which the focus is automatically adjusted. Specifically, the information processing apparatus 101 acquires, from the imaging apparatus 104, information indicating whether the current mode is the manual focus mode or the automatic focus mode, and the current imaging is performed according to the information. It is determined whether the apparatus 104 is in the manual focus mode in which the focus is manually adjusted or in the automatic focus mode in which the focus is automatically adjusted (S518).

  When it is determined that the current imaging device 104 is in the manual focus mode (S518: YES), the information processing device 101 transmits, to the imaging device 104, a setting change request to operate in the automatic focus mode. When the imaging device 104 receives the request (S519), the imaging device 104 switches the operation from the manual focusing mode to the automatic focusing mode (mode for auto focusing). In addition, when it is determined that the current imaging device 104 is in the automatic focus mode (S518: NO), the information processing device 101 shifts the processing to S520.

  Further, the information processing apparatus 101 acquires, from the drone 102, position information (including information on longitude, latitude, and altitude) of the flight position at which the drone 102 is currently flying. Then, the information processing apparatus determines whether or not the flight position of the drone 102 currently in flight has exceeded a predetermined area registered in advance in the information processing apparatus 101 (S520), and the predetermined area is exceeded. If it is determined (YES), an instruction to stop the movement of the drone 102 is transmitted to the drone 102, and the movement of the drone 102 is stopped (S522). On the other hand, when it is determined that the predetermined area is not exceeded (S520: NO), the information processing apparatus 101 determines whether the user further accepts the zoom out operation by the user using the operation unit 107 at present. (S521). Then, if it is determined that the zoom-out operation is not further received (S521: NO), the process proceeds to S522, and the movement of the drone 102 is stopped (S522). On the other hand, when it is determined that the zoom-out operation is further received (S521: YES), the process proceeds to S515, and the drone 102 is continuously moved away from the object 108.

  When the information processing apparatus 101 stops the movement of the drone 102 in S522, the process returns to S501.

  In this way, the camera mounted on the drone zooms out, reaches the wide end which is the limit of the zoom out movement, and the drone moves away from the subject even if the user receives an instruction for the zoom out operation from the user. By photographing the subject, it is possible to acquire an image (image zoomed out) taken as if it were zoomed out.

  When it is determined that the instruction for the zoom out operation has not been received by the operation unit 107 (S512: NO), the information processing apparatus 101 determines whether the instruction for the pan operation has been received by the operation unit 107 (S523) ). Then, if it is determined that the pan operation instruction has not been received (NO), the process returns to S501. If it is determined that the pan operation instruction has been received (YES), the process proceeds to S524.

  A unit that receives an operation instruction of the imaging apparatus in S501, S521, and S523 is an application example of the reception unit of the present invention.

  S523 is an application example of the reception unit of the present invention, and the operation instruction received here is an operation instruction of the pan operation of the imaging apparatus 104.

  Then, the information processing apparatus 101 determines whether the information indicating that the pan end has been reached by the pan operation is received from the imaging apparatus 104, whereby the imaging apparatus 104 has reached the pan end by the pan operation. It is determined whether or not (S524). When the imaging device 104 reaches the pan end by the panning operation, the imaging device 104 transmits information indicating that the panning end is reached by the panning operation to the information processing apparatus 101.

  The determination processing of S502, S513, and S524 is an application example of the determination unit of the present invention, and determines whether the operation of the imaging device 104 according to the operation instruction received by the reception unit exceeds the operation limit of the imaging device 104.

  S524 is an application example of the determination unit of the present invention, and determines whether the pan operation of the imaging device 104 according to the operation instruction received by the reception unit exceeds the limit (pan end) of the pan operation of the imaging device 104.

  When it is determined that the imaging device 104 has reached the pan end by the pan operation (S 524: YES), the information processing apparatus 101 determines whether the operation unit 107 further accepts a pan operation by the user. (S525). Then, when it is determined by the operation unit 107 that the user has further received a pan operation (YES), the information processing apparatus 101 causes the process to proceed to S526.

  When it is determined that the imaging device 104 has not reached the pan end in the panning operation (S 524: NO), the information processing device 101 transmits an instruction for the panning operation to the imaging device 104 and transmits the instruction to the imaging device 104. The pan operation is performed, and the process returns to step S501.

  Further, when it is determined by the operation unit 107 that the user has not further received a pan operation (S525: NO), the information processing apparatus 101 returns the process to S501.

  The information processing apparatus 101 acquires, from the imaging apparatus 104, the rotation direction (left rotation or right rotation) in which the current panning operation is performed and the panning speed immediately before reaching the pan end (S526).

  The pan speed acquired in S526 may be a pan speed currently receiving a pan operation instruction from the operation unit 107, and the pan speed in that case is acquired from the information processing apparatus 101.

  Then, the information processing apparatus 101 instructs the drone 102 to swing (rotate) the drone 102 at the pan speed acquired in S526 in the pan direction acquired in S526 (S527).

  Specifically, the information processing apparatus 101 transmits the information indicating the pan direction and the information (vector information) indicating the pan speed to the drone 102, and when the drone 102 receives the vector information, the vector According to the information, the drone 102 is made to fly at the pan speed in the pan direction. For example, the imaging device 104 of the drone 102 in FIG. 4A pans in the direction of the arrow in FIG. 4A (moves clockwise as viewed from the direction of gravity) to obtain the state shown in FIG. However, since the state shown in FIG. 4B has reached the pan end, the drone 102 flies and moves clockwise (in the direction of the arrow) instead of the imaging device 104 as viewed from the direction of gravity. It will be in the state shown to (C).

  S506, S517, and S527 are application examples of the instruction unit of the present invention, and when the determination unit determines that the operation limit of the imaging device 104 is exceeded, the flight operation corresponding to the operation instruction is performed on the unmanned airplane Give instructions to the unmanned airplane.

  When the determining unit determines that the operation limit of the imaging device 104 is not exceeded, the instruction unit instructs the imaging device 104 to receive the operation instruction received by the receiving unit. When it is determined that the operation limit is exceeded, the unmanned airplane is instructed to cause the unmanned airplane to perform a flight operation corresponding to the operation instruction without issuing the operation instruction accepted by the accepting unit to the imaging device.

  S527 is an application example of the instruction unit of the present invention, and when the determination unit of S524 determines that the limit (pan end) of the panning operation of the imaging apparatus 104 is exceeded, the operation instruction accepted by the acceptance unit is handled. As the flight operation, the unmanned airplane is instructed to cause the unmanned airplane to perform an operation of rotating the unmanned airplane (drone 102) while flying in the same direction as the rotation direction of the pan action received by the accepting means.

  Next, the information processing apparatus 101 determines whether or not the pan unit by the user is further received by the operation unit 107 (S528). Then, if it is determined that the pan operation is not further received (S528: NO), the process proceeds to S529, and the movement of the drone 102 is stopped (S529). On the other hand, when it is determined that the pan operation is further received (S528: YES), the process proceeds to S526, and the drone 102 is subsequently rotated in the pan direction acquired in S526.

  When the information processing apparatus 101 stops the movement of the drone 102 in S529, the process returns to S501.

  As described above, the camera mounted on the drone performs the panning operation, reaches the panning end which is the limit of the panning operation, and the drone performs the panning operation even if an instruction for the panning operation is received from the user. It is possible to turn in the pan direction at the same speed as the pan operation, and to acquire an image captured by the pan operation.

  As described above, according to the present invention, it is possible to capture an image exceeding the operation limit of the zoom operation or the pan operation of the imaging device mounted on the unmanned airplane by a simple operation.

  As mentioned above, although one embodiment of the present invention was explained in full detail, the present invention can take an embodiment as a system, an apparatus, a method, a program executable by an apparatus, a storage medium, etc., for example.

  Further, although the process of FIG. 5 has been described as the process executed by the information processing apparatus 101, for example, the control unit (information processing apparatus) included in the unmanned airplane (drone 102) executes the process of FIG. You can also. In that case, the control unit (information processing apparatus) included in the unmanned airplane (drone 102) is the information processing apparatus of the present invention.

  Further, an object of the present invention is to provide a system or apparatus with a storage medium storing a program code of software for realizing the functions of the above-described embodiments, and a computer (or CPU or MPU) of the system or apparatus serves as a storage medium. Needless to say, this can also be achieved by reading out and executing the program code stored in. That is, the present invention can be applied not only as an imaging device but also as an information processing device such as a personal computer.

  In this case, the program code itself read out from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (a basic system or an operating system running on the computer based on instructions of the program code). It is needless to say that the case where the functions of the above-described embodiment are realized by the processing etc. by performing part or all of the actual processing etc. is realized.

101 information processing apparatus 102 drone (unmanned air vehicle)
103 console 104 imaging device (camera)
105 Network 106 Operation Unit 107 Operation Unit

Claims (15)

An information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, comprising:
Reception means for receiving an operation instruction regarding zoom of the lens of the imaging device;
Operation of the zoom of the image pickup apparatus of the lens by the operation instruction received by said receiving unit, when the exceeds the operating limit of the imaging device, a flight interaction in the direction of the zoom corresponding to the operation instruction, Instruction means for instructing the unmanned aerial vehicle to cause the unmanned aerial vehicle to perform a flight operation at a speed according to the zoom speed of the lens of the imaging device according to the operation instruction;
An information processing apparatus comprising: The operation instruction received by the reception means is an operation instruction for the zoom-in operation of the lens of the imaging device,
A zoom-in speed acquisition unit that acquires a zoom-in speed before reaching the limit of the zoom-in operation of the lens of the imaging device;
Flight said instructing means, zoom operation of the lens of the imaging device by the operation instruction received by said receiving unit, when the exceeds the limit of the zoom operation of the lens of the imaging device, corresponding to the operation instruction In operation, the unmanned airplane is instructed to cause the unmanned airplane to perform an operation of flying and moving the unmanned airplane in the imaging direction of the imaging device at a speed according to the zoom-in speed acquired by the zoom-in speed acquiring means. the information processing apparatus according to claim 1, characterized in that. The operation instruction accepted by the acceptance unit is an operation instruction for zoom out operation of the lens of the imaging device,
A zoom-out speed acquisition unit that acquires a zoom-out speed before reaching the limit of the zoom-out operation of the lens of the imaging device;
Said instructing means, zoom-out operation of the lens of the imaging device by the operation instruction received by the receiving unit, if exceeds the limit of the zoom-out operation of the imaging device, flight operation corresponding to the operation instruction As an instruction to cause the unmanned airplane to perform an operation of flying and moving the unmanned aerial vehicle in the direction opposite to the imaging direction of the imaging device at a speed corresponding to the zoom out speed acquired by the zoom out speed acquiring means The information processing apparatus according to claim 1 , wherein the information processing apparatus is performed on the unmanned airplane. The receiving unit receives an operation instruction of the panning operation of the imaging device .
Said instructing means, the pan operation of the imaging device by the operation instruction received by said receiving unit, when in excess of the operating limit of the pan operation of the imaging device, in flight operation of turning in the pan direction of the operation instruction 4. The unmanned airplane according to claim 1, wherein the unmanned airplane is instructed to perform a flight operation at a speed according to the pan speed of the imaging device according to the operation instruction. The information processing apparatus according to claim 1. Pan speed acquisition means for acquiring the pan speed of the imaging device before reaching the limit of the panning operation of the imaging device,
The instruction means is responsive to the pan speed acquired by the pan speed acquisition means when the pan operation of the imaging device according to the operation instruction received by the reception means exceeds the operation limit of the pan operation of the imaging device. The information processing apparatus according to claim 4, wherein the instruction to cause the unmanned airplane to perform a flight operation that turns in the pan direction of the operation instruction at a fixed speed is issued to the unmanned airplane. The instruction unit manually adjusts the focus mode of the imaging device manually when the operation of the imaging device according to the operation instruction received by the reception unit exceeds the operation limit of the imaging device. The information processing apparatus according to any one of claims 1 to 3, wherein an instruction to change to an automatic focus mode for automatically adjusting the focus is issued. When the operation of the imaging device according to the operation instruction received by the reception unit exceeds the operation limit of the imaging device, and the current focus mode of the imaging device is the manual focus mode, the instruction unit 7. The information processing apparatus according to claim 6, wherein an instruction to change the focus mode of the imaging device from the manual focus mode to the automatic focus mode is issued. The unmanned aerial vehicle receives a command for controlling the unmanned aerial vehicle, and a first communication unit of the imaging device with which the information processing apparatus and the image capturing device communicate to receive an instruction for controlling the image capturing device. And a second communication unit of the unmanned airplane communicating with the unmanned airplane in order to receive the information.
Said instruction means, the operation of the imaging device by the operation instruction received by said receiving unit, when operated not exceeding the limit of the imaging device, the first of the imaging device an operation instruction received by said receiving means performed for one communication unit, also, the operation of the imaging device by the operation instruction received by said receiving unit, when the exceeds the operating limit of the imaging device, the unmanned flight operation corresponding to the operation instruction The information processing apparatus according to any one of claims 1 to 7, wherein an instruction to be issued to an airplane is issued to the second communication unit of the unmanned airplane. Said instruction means, the operation of the imaging device by the operation instruction received by said receiving unit, when it is determined to exceed the operating limits of the imaging device, the imaging device an operation instruction received by said receiving means 9. The information processing apparatus according to claim 8 , wherein the second communication unit of the unmanned aircraft is instructed to cause the unmanned airplane to perform a flight operation corresponding to the operation instruction. An information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, comprising:
Reception means for receiving an operation instruction of the panning operation of the imaging device;
The pan operation of the imaging device by the operation instruction received by the receiving means, if in excess of the operating limit of the pan operation of the imaging device is a flying operation of turning in the pan direction of the operation instruction, the operation An instruction unit that instructs the unmanned airplane to cause the unmanned airplane to perform a flight operation at a speed according to a panning speed of the imaging device according to an instruction;
An information processing apparatus comprising: An information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, comprising:
Reception means for receiving an operation instruction of the imaging device;
Operation of the imaging device by the operation instruction received by said receiving unit, when in excess of the operating limit of the imaging device, an instruction to perform the flight operation corresponding to the operation instruction to the unmanned aircraft, and of the imaging device An instruction unit for instructing the unmanned airplane to change the focus mode from a manual focus mode in which the focus is manually adjusted to an automatic focus mode in which the focus is automatically adjusted ;
An information processing apparatus comprising: A control method in an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, comprising:
A receiving step of receiving an operation instruction regarding a zoom of a lens of the imaging device;
Indicating means, the operation of the zoom of the image pickup apparatus of the lens by the operation instruction received by the reception step, if the exceeds the operating limit of the imaging device, flight operation in the direction of the zoom corresponding to the operation instruction An instruction step of instructing the unmanned aerial vehicle to cause the unmanned aerial vehicle to perform a flight operation at a speed according to the zoom speed of the lens of the imaging device according to the operation instruction;
A control method comprising: A control method in an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, comprising:
A receiving step of receiving an operation instruction of a pan operation of the imaging device;
In the flight operation, the instruction means turns in the pan direction of the operation instruction when the pan operation of the imaging device according to the operation instruction received in the reception step exceeds the operation limit of the pan operation of the imaging device. An instruction step of instructing the unmanned aerial vehicle to cause the unmanned aerial vehicle to perform a flight operation at a speed according to the panning speed of the imaging device according to the operation instruction;
A control method comprising: A control method in an information processing apparatus for instructing a flight operation to an unmanned airplane provided with an imaging device, comprising:
A receiving step of receiving an operation instruction of the image pickup apparatus;
An instruction to cause the unmanned airplane to perform a flight operation corresponding to the operation instruction when the operation of the imaging device according to the operation instruction received in the reception step exceeds the operation limit of the imaging device; Instructing the unmanned airplane to change the focus mode of the imaging device from a manual focus mode in which the focus is manually adjusted to an automatic focus mode in which the focus is automatically adjusted;
A control method comprising: A program for causing a computer to function as an information processing apparatus according to any one of claims 1 to 11.

Images