JP6697340B2 - Photography equipment, mobile photography device, photography moving body, and photography control device for moving body - Google Patents

Description

  The present invention relates to a photographing device, a moving photographing device, a photographing moving body, and a moving body photographing control device which are suitable for a remote-controlled or self-supporting flying object such as a drone.

  In recent years, unmanned aircraft such as drones that fly by remote control or stand alone have been commercialized. For example, a relatively small unmanned aerial vehicle can be used because it can safely and efficiently acquire information in dangerous areas that are difficult to reach with manned aircraft, accident sites and disaster areas that require observation at relatively low altitudes. It is supposed to be done. For such a purpose, some drones and the like have a built-in camera.

  However, as the built-in camera, a relatively small lens or an image pickup element is often used, and a picked-up image with a sufficiently satisfactory image quality is not always obtained. Therefore, even in such an unmanned aerial vehicle, it is conceivable to adopt and attach a high-performance and high-performance imaging device. In particular, by adopting a camera capable of mounting a high-performance lens having a zoom function, an autofocus function, etc., it becomes easy to obtain an image of the image quality desired by the user.

  Note that Patent Document 1 discloses a gimbal system capable of accurately supporting an imaging device on a moving body such as an aircraft.

U.S. Patent Application Publication No. 2016/0014309

  However, unmanned aerial vehicles such as drones are relatively small in size and have a relatively small maximum payload. For this reason, it is better not to have a large-scale device such as a gimbal, and it is better to have a simple device. Furthermore, if a shooting device whose center of gravity changes depending on the shooting method is installed, the aircraft may tilt with the movement of the center of gravity of the shooting device, which may impair flight stability and control. There's a problem.

  INDUSTRIAL APPLICABILITY The present invention can improve the stability of a moving body and the stability of photographing by controlling the state change of the photographing device mounted on the moving body based on the information on the posture of the moving body in the vertical direction. An object of the present invention is to provide an imaging device, a mobile imaging device, a moving object for shooting, and a shooting control device for a moving object.

  An imaging device according to an aspect of the present invention is an imaging device that can be attached to a moving body, and an imaging unit capable of shooting in different imaging states, and whether the posture of the moving body has a predetermined state change permission posture. And a control unit that controls the change of the above-mentioned imaging state accompanied by the change of the center of gravity based on the determination result of whether or not.

  Further, according to an aspect of the present invention, there is provided a mobile image capturing apparatus, a mobile unit, an imaging unit attached to the mobile unit, capable of capturing images in different shooting states, and a posture of the mobile unit having a predetermined state change permission posture. And a control unit for controlling the change of the photographing state accompanied by the change of the center of gravity based on the determination result of whether or not the change has occurred.

  In addition, the photographing moving body according to one aspect of the present invention is a photographing moving body to which a photographing device is attached, and has a posture determining unit that determines a posture in the vertical direction and a posture that the posture determining unit determines. And a control unit that controls the change of the photographing state with the change of the center of gravity of the photographing device based on whether or not the posture is the state change permission posture.

  In addition, the moving body photographing control apparatus according to one aspect of the present invention includes a first communication unit that receives a posture determination result from the moving body and information for controlling a photographing state of a photographing device attached to the moving body. And a change in the center of gravity of the imaging device only when the determination result received by the first communication unit and the second communication unit that transmits And a control unit that determines permission to change the photographing state and controls the photographing device via the second communication unit.

  ADVANTAGE OF THE INVENTION According to this invention, the stability in a mobile body and the stability of imaging | photography are improved by controlling the state change of the imaging device mounted in a mobile body based on the information of the attitude | position with respect to the vertical direction of a mobile body. It has the effect that

The block diagram which shows the mobile imaging device which concerns on the 1st Embodiment of this invention. 3 is a flowchart showing an outline of shooting control of the mobile shooting device 10. The lens sectional view showing an example of the composition of each lens in optical system 11a. 6 is a flowchart showing movement and shooting control. Explanatory drawing which shows an example of the external appearance of the mobile imaging device 10 when the drone 70 is employ | adopted as the mobile body 10a. Explanatory drawing which shows an example of the external appearance of the mobile imaging device 10 when the drone 70 is employ | adopted as the mobile body 10a. Explanatory drawing which shows a mode that the mobile imaging device 10 is remotely operated by the control device 83. Explanatory drawing which shows an example of the operation screen 83a of the control device 83 shown in FIG. 6A. Explanatory drawing which shows an example of the operation screen 83a of the control device 83 shown in FIG. 6A. Explanatory drawing for demonstrating the control for the moving imaging device 10 to move to a target position. Explanatory drawing for demonstrating the control for the moving imaging device 10 to move to a target position. Explanatory drawing for demonstrating the control for the moving imaging device 10 to move to a target position. Explanatory drawing for demonstrating the control for the moving imaging device 10 to move to a target position. Explanatory drawing for demonstrating the control for the moving imaging device 10 to move to a target position. 6 is a flowchart showing the control by the imaging unit and the control by the moving body. 6 is a flowchart showing the control by the imaging unit and the control by the moving body. FIG. 6 is a block diagram showing an imaging device according to a second embodiment of the present invention, the imaging device constituting a part of a moving body imaging control device. The block diagram which shows the moving body for imaging which concerns on 2nd Embodiment, and shows a moving body which comprises some imaging | photography control apparatuses for moving bodies. FIG. 12B is a block diagram showing a photographing remote control device of the photographing device of FIG. 12A. The block diagram which shows the movement remote control apparatus of the mobile body of FIG. 12B. The flowchart which shows camera control. The flowchart which shows a mobile body control. Explanatory drawing which shows a mode at the time of control and an operation screen. Explanatory drawing which shows a mode at the time of control and an operation screen. Explanatory drawing which shows the mode and operation screen at the time of photography. Explanatory drawing which shows the mode and operation screen at the time of photography.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a mobile photographing device according to a first embodiment of the present invention.

  Mobile imaging device 10 according to the present embodiment is composed of moving body 10a and imaging section 10b mounted on moving body 10a. The moving body 10a may be, for example, an unmanned aircraft such as a drone, a radio-controlled traveling vehicle, a radio-controlled vessel, or the like, and is a remote-controlled or self-supporting unmanned moving object. In the present embodiment, the movement of the center of gravity of the imaging unit 10b and the change in shape affect the movement and posture of the moving body 10a. Therefore, it is possible to improve the stability of the moving body by controlling the state change of the imaging unit mounted on the moving body based on the information on the posture of the moving body 10a in the vertical direction. Further, by controlling so as to allow the movement of the center of gravity and the change in shape of the image pickup unit while ensuring the stability of the moving body, it is possible to reliably take an image.

  A moment is generated in the moving body 10a according to the position of the center of gravity of the moving body 10a such as a drone, the position of the center of gravity of the entire imaging unit 10b, the weight of the imaging unit 10b, and the mounting position of the imaging unit 10b on the moving body 10a. However, the line segment connecting the position of the center of gravity of the moving body 10a and the position of the center of gravity of the entire imaging unit 10b is the vertical direction, and the change in the center of gravity due to the change in the state of the imaging unit 10b during zooming or the like only changes in the vertical direction. Even when the state of the imaging unit 10b changes, the moment acting on the moving body 10a remains 0.

  That is, when the moving body 10a has a certain posture with respect to the vertical direction (hereinafter referred to as a state change permission posture), the line segment connecting the center of gravity of the moving body 10a and the center of gravity of the entire imaging unit 10b is in the vertical direction and When the moving body 10a is equipped with the image pickup unit 10b so that the change of the center of gravity due to the change of the state of the image pickup unit 10b such as a zoom operation changes only in the vertical direction, if the moving body 10a maintains the state change permitting posture, the zoom is performed. Even if an operation or the like is performed, the resulting moment remains zero. For example, when the moving body 10a is a flying body, the flight attitude is changed when the pitching angle, which is the angle around the left-right axis, and the bank angle, which is the angle around the front-rear axis, of the moving body 10a are both 0 degrees. It may be set to the permitted posture.

In the present embodiment, whether or not to permit the state change accompanying the movement of the center of gravity of the imaging unit 10b is determined based on whether or not the posture of the moving body 10a is the state change permission posture.
Note that the description below also assumes the center of gravity movement in which the lens groups that make up the optical system move, assuming telephoto shooting using a zoom lens, etc. Needless to say, it is also a technology for dealing with. In addition to the above, it goes without saying that the present invention can also take countermeasures for the center of gravity movement due to the control of the optical filter and the dustproof / dripproof mechanism, and the center of gravity movement due to the mechanism change caused by the irradiation of strobe light or illumination light .

  FIG. 2 is a flow chart showing an outline of shooting control of such a mobile shooting device 10. In the mobile photographing device 10, a photographing target is designated in step S1. For example, the mobile photographing device 10 is instructed about the latitude and longitude information to be photographed based on a user operation. The mobile photographing device 10 performs control for moving to a designated photographing target position. As a result, the moving body 10a starts moving (step S2). When the moving body 10a reaches the photographing target position, the mobile photographing device 10 determines whether or not the posture is the state change permission posture (step S3).

  If the moving image capturing apparatus 10 is not in the state change permission posture, the state change of the image pickup unit 10b is not permitted (step S4). As a result, when the moving body 10a is not in the state change permission posture, for example, when one of the pitching angle and the bank angle is not approximately 0 degrees in a flying body such as a drone, the imaging unit 10b is Does not control zoom or focus. Therefore, a change in the state of the imaging unit 10b does not cause a moment to act on the moving body 10a to make the movement of the moving body 10a unstable.

  Here, as a result of the posture control of the moving body 10a, the posture of the moving body 10a becomes the state change permission posture. Then, the mobile imaging device 10 permits the imaging unit 10b to change the state (step S5). For example, when both the pitching angle and the bank angle of the moving body 10a become approximately 0 degrees, the zoom control and the focus control of the imaging unit 10b are permitted. By such photographing control, the state of the image pickup unit 10b changes and the position of the center of gravity changes. If the change in the center of gravity in this case is a change in the vertical direction with respect to the position of the center of gravity of the moving body 10a, no moment acts on the moving body 10a depending on the change in the center of gravity of the imaging unit 10b. Therefore, the moving body 10a does not become unstable. Further, since the posture of the moving body 10a is maintained stable, the photographing range by the image pickup unit 10b is also stabilized, and it is possible to accurately capture a desired photographing target and perform photographing in a reliable focused state.

  In FIG. 1, the mobile photographing device 10 has a control unit 11. The control unit 11 may be configured by a processor such as a CPU (not shown), or may realize each function according to a program stored in a memory (not shown). The moving body 10a is provided with a propulsion unit 14. The propulsion unit 14 generates a propulsive force for moving the moving body 10a. When the moving body 10a is a drone that moves in the air, it is necessary to generate propulsive force also during hovering, while when the moving body 10a is an airplane or the like, propulsive force may be unnecessary during gliding. is there.

  When the moving body 10a is a drone, for example, a plurality of propulsion units 14 are provided at the end of an arm (not shown), and a control unit 11 is provided at the upper center of the arm. Each propulsion unit 14 includes, for example, a motor and a propeller that is rotationally driven by each motor. In this case, the motors forming the propulsion units 14 are controlled independently of each other so that the moving body 10a can move in a predetermined posture and speed. An attachment member is attached to the lower part of the center of the arm so that, for example, the imaging unit 10b is arranged.

  The control unit 11 is provided with a propulsion control unit 13a, and the propulsion control unit 13a can independently control the propulsive forces of the plurality of propulsion units 14. The control unit 11 is provided with a direction control unit 13b and a posture control unit 13c. The direction control unit 13b and the attitude control unit 13c respectively output control signals for controlling the moving direction of the moving body 10a or the attitude of the moving body 10a to the propulsion control unit 13a.

  The movement determination unit 13d determines the movement direction of the moving body 10a and outputs the determination result to the direction control unit 13b. Further, the posture determination unit 13e determines the posture of the moving body 10a and outputs the determination result to the posture control unit 13c. In order to control the moving direction of the moving body 10a to the designated moving direction, the direction control unit 13b controls the propulsion control unit 13a so that the movement determination result matches the designated moving direction. In addition, the attitude control unit 13c controls the propulsion control unit 13a so that the attitude determination result matches the specified attitude.

  Various sensor information is provided to the control unit 11 in order to determine the moving direction and posture of the moving body 10a. In the example of FIG. 1, the moving body 10a is provided with a spatial information acquisition unit 16a, a position / orientation determination unit 16b, and an altitude / orientation determination unit 16c. The spatial information acquisition unit 16a can be configured by, for example, a radar or a camera, and acquires spatial information such as ground speed. The position / direction determining unit 16b can be configured by, for example, a GPS (Global Positioning System) receiver, and acquires the position and direction information of the moving body 10a. The altitude / posture determination unit 16c can be configured by, for example, an atmospheric pressure sensor, an acceleration sensor, a gyroscope, or the like, and determines the altitude and posture of the moving body 10a. The determination results of the determination units 16a to 16c are given to the movement determination unit 13d and the posture determination unit 13e, and the movement determination unit 13d and the posture determination unit 13e determine the moving direction and the posture of the moving body 10a based on these determination results. It is designed to judge.

  A wind speed detector 15 is provided on the moving body 10a. The wind speed detection unit 15 detects the wind speed relative to the moving body 10a and outputs it to the control unit 11. The control unit 11 can use the wind speed detection result for the direction determination and the attitude determination, and can also use the wind speed detection result for the propulsion control. This may have a propeller or the like to detect the rotation thereof, or a piezo element or a pressure sensor may be diverted. It may be rewritten as wind force detection.

    Further, the mobile body 10a may be provided with a built-in camera 18. The built-in camera 18 is capable of capturing an image in a predetermined direction around the moving image capturing device 10 to obtain a captured image. In particular, the built-in camera 18 is configured to be capable of relatively wide-angle photography so that the surroundings can be easily observed when the mobile photography device 10 is moved. For example, when the moving body 10a is an airplane such as a drone, the built-in camera 18 may be able to take images in front of and below the moving direction as the shooting range while the moving body 10a is flying horizontally. desirable.

  In FIG. 1, the image pickup unit 10b is provided with an image pickup device (not shown) composed of a CCD, a CMOS sensor or the like, and an optical system 21 guides a subject optical image to an image pickup surface of the image pickup device. .. In the optical system 11a, a mechanical unit that drives a focus lens, a zoom lens, a diaphragm, and the like (not shown) are configured, and the drive unit 22 drives and controls these mechanical units. The imaging control unit 12a of the control unit 11 controls the drive unit 22 to drive and control the focus lens, the zoom lens, and the diaphragm formed in the optical system 11a. The image sensor photoelectrically converts the subject optical image to obtain a captured image.

  Although FIG. 1 shows an example in which the optical system 11a is provided in the image pickup unit 10b, an interchangeable lens detachably attached to the housing of the image pickup unit 10b is used as an optical system, and an object from the interchangeable lens is adopted. The optical image may be formed on the image pickup surface of the image pickup device.

  FIG. 3 is a lens cross-sectional view showing an example of the configuration of each lens in the optical system 11a. 3 shows the state A at the wide-angle end when focusing on an object point at infinity, the middle stage shows the intermediate state B, and the lower stage shows the state C at the telephoto end.

  In the example of FIG. 3, the optical system includes, in order from the object side, a first lens group G1 having a positive refractive power, a second lens group G2 having a negative refractive power, an aperture stop S, and a positive refractive power. It has a third lens group G3, a fourth lens group G4 having a negative refractive power, and a fifth lens group G5 having a positive refractive power. In FIG. 3, a parallel plate forming a low-pass filter provided with a wavelength band limiting coat for limiting infrared light is indicated by F, a parallel plate of the cover glass of the electronic image pickup element is indicated by C, and an image plane is indicated by I. A multi-layer film for limiting the wavelength range may be applied to the surface of the cover glass C. Further, the cover glass C may have a low-pass filter function.

  With such a lens configuration, the first lens group G1 moves toward the object side during zooming from the wide-angle end to the telephoto end. The second lens group G2 moves to the image side. The third lens group G3 moves to the object side. The fourth lens group G4 moves to the object side. The fifth lens group G5 moves to the object side and then to the image side. Therefore, regarding the distance between the lens groups, the distance between the first lens group G1 and the second lens group G2 increases, the distance between the second lens group G2 and the third lens group G3 decreases, and the distance between the third lens group G3 and The distance between the fourth lens group G4 and the fourth lens group G4 changes, and the distance between the fourth lens group G4 and the fifth lens group G5 increases. Further, the aperture stop S moves together with the third lens group G3.

  Further, it is desirable that the fifth lens group or the fourth lens group is used for focusing for adjusting the focus. Focusing with this group reduces the load on the motor because the lens weight is light. Focusing may be performed with another lens group. Further, focusing may be performed by moving a plurality of lens groups. Further, the entire lens system may be extended to perform focusing, or a part of the lenses may be extended or extended to perform focusing.

  In FIG. 1, the photographing designation unit 17 can generate various instructions regarding photographing by the image pickup unit 10b. For example, the photographing designation unit 17 can give an instruction to the control unit 11 to perform photographing in a designated direction and a designated angle of view at a designated position based on a user operation or preprogrammed information. The photographing control unit 12a in the control unit 11 is configured to control the photographing of the image pickup unit 10b according to the photographing instruction from the photographing designation unit 17. For example, the photographing control unit 12a can generate a focus signal and a zoom signal and give them to the driving unit 22 according to the instruction of the photographing designating unit 17 to drive each lens of the optical system 11a.

  By the focus control and zoom control by the photographing control unit 12a, each lens of the optical system 11a is driven to perform focusing and zooming. The focus / angle-of-view information acquisition unit 12b acquires information about the focus position and zoom position and information about the shooting angle of view from the imaging unit 10b and outputs the information to the focus / angle-of-view control unit 12c. The focus / angle-of-view control unit 12c is controlled by the imaging control unit 12a to drive the drive unit 22 to control the focusing and zooming of the optical system 11a.

  For example, as shown in FIG. 3, the optical system 11a is composed of a plurality of lens groups arranged in the optical axis direction. The optical system 11a is arranged in a lens barrel (not shown) attached to the front surface of the housing of the image pickup unit 10b. The interchangeable lens includes this lens barrel. During zooming from the wide-angle end side to the telephoto end, as shown in FIG. 3, the distance from the front end to the rear end of the lens group changes, and the arrangement position of each lens group on the optical axis changes. The lens barrel itself may expand and contract according to zooming. The position of the center of gravity of the entire imaging unit 10b changes due to such movement and expansion / contraction of the lens group and the lens barrel.

  However, the line segment connecting the position of the center of gravity of the moving body 10a and the position of the center of gravity of the entire imaging unit 10b is moved in the vertical direction, and the change in the center of gravity due to a change in the state of the imaging unit 10b such as a zoom operation is changed only in the vertical direction. When the image pickup unit 10b is mounted on the body 10a, if the movable body 10a maintains the state change permitting posture, the moment due to the zoom operation or the like remains zero.

  The permission determination unit 13f of the control unit 11 is provided with the information regarding the posture of the moving body 10a from the posture determination unit 13e, and when the posture of the moving body 10a matches or substantially matches the state change permission posture, the imaging control unit 13f. A permission signal for permitting the photographing control accompanied by the change of the state such as zooming is given to the photographing control unit 12a. It should be noted that the permission determination unit 13f generates a photographing permission signal when the stability of the posture can be ensured, for example, when the posture of the moving body 10a matches or substantially matches the state change permission posture for a predetermined time or more. May be. The image capturing control unit 12a is configured to perform image capturing control such as focusing and zooming that causes a state change accompanied by the movement of the center of gravity only when the image capturing permission signal is input.

  The control unit 11 receives the captured image from the image capturing unit 10b, performs predetermined image signal processing, for example, color adjustment processing, matrix conversion processing, noise removal processing, and other various signal processing, and then does not show it. It can be given to the recording unit for recording. The control unit 11 may be configured to be able to transfer the captured image after signal processing to a photographing remote control device (not shown) or the like via a communication unit (not shown).

  Next, the operation of the embodiment configured as described above will be described with reference to FIGS. 4 to 11B. FIG. 4 is a flowchart showing movement and shooting control. 5A and 5B are explanatory diagrams showing an example of the outer appearance of the moving image capturing apparatus 10 when the drone 70 is adopted as the moving body 10a. FIG. 4 shows an example in which the drone 70 shown in FIGS. 5A and 5B is adopted as the moving body 10a of FIG. Further, the mobile photographing device 10 will be described as being remotely controlled by using the control device 83 (see FIGS. 6A to 6c).

  The drone 70, which is the moving body 10a, includes a mounting member 75, and the imaging unit 10b can be mounted on the mounting member 75. The imaging unit 10b is attached to the drone 70 which is the moving body 10a via the attachment member 75, and exchanges information with the control unit 11 built in the drone 70 via the cable 76. There is. The drone 70 has four propulsion units 71 to 74 corresponding to the propulsion unit 14 in FIG. 1, and the propulsion units 71 to 74 maintain a desired posture.

In the example of FIGS. 5A and 5B, the imaging unit 10b has a lens barrel 21a that expands and contracts according to zooming. In the imaging unit 10b, for example, the optical axis is arranged vertically below the center of gravity of the drone 70, and the lens barrel 21a is in the contracted state of FIG. And the stretched state shown in FIG. 5B.
As described above, with the vertical expansion and contraction and the movement of the center of gravity, for example, a flying vehicle that is trying to maintain a floating force in the zenith direction, which is against gravity, does not lose its balance. However, this can be said when the flying vehicle is floating, but it is possible to lose the balance when moving with horizontal component changes, and there is wind because it is easily affected by wind due to expansion and contraction. Sometimes it is safer not to stretch or move the center of gravity. Therefore, for such a flying vehicle, it is safer for the shooting device not to move the center of gravity without permission. Therefore, even if the shooting unit has an imaging unit that moves the center of gravity in different shooting conditions, the shooting device must perform the following control. Is preferred. That is, it is desired that the control unit controls the change of the photographing state accompanied by the change of the center of gravity, based on the determination result of whether or not the posture of the moving body has reached the predetermined state change permission posture.

  FIG. 6A is an explanatory diagram showing a state in which the mobile photographing device 10 is remotely controlled by the control device 83. As shown in FIG. 6A, the user 81 holds the control device 83 with the left hand 82L and operates the control device 83 with the finger 82R of the right hand. The control device 83 is provided with, for example, a touch panel (not shown), and by operating the finger 82R on the touch panel, the mobile photographing device 10 can be remotely controlled.

  In step S31, the control unit 11 of the mobile imaging device 10 determines the current state using the altimeter, radar, built-in camera, GPS, and the like. The control unit 11 is in a standby state for access from the control device 83 to the mobile image capturing device 10 in step S32. Next, the control unit 11 transmits the state information acquired in step S31 to the control device 83 (step S33). Next, the control unit 11 transmits the image acquired by the built-in camera 18 in step S34 to the control device 83.

  6B and 6C are explanatory views showing an example of the operation screen 83a of the control device 83 shown in FIG. 6A. As shown in FIGS. 6B and 6C, an operation screen 83a is provided on the surface of the control device 83. A touch panel (not shown) is provided on the operation screen 83a, and the user can remotely operate the mobile photographing device 10 by touching or sliding on the operation screen 83a. On the operation screen 83a, as shown in FIGS. 6B and 6C, a display area for displaying an image 85 obtained by the built-in camera 18 and an image 86a obtained by the image pickup unit 10b are displayed. A display area for displaying 86b is also provided. Further, on the operation screen 83a, operation buttons 87a and 87b for zooming control of the imaging unit 10b are also provided. The image pickup unit 10b can be zoomed up by touching the operation button 87a, and the image pickup unit 10b can be zoomed down by touching the operation button 87b. In the present embodiment, a display 88 of "zoom operation OK" is displayed on the operation screen 83a. This display 88 indicates that the moving body 10a is in the state change permission posture and is in a stable state. This shows that the zoom control and the focus control can be performed. The control based on the zoom operation by the control device 83 is performed only during the period when the display 88 is displayed.

  In step S35, the control unit 11 determines whether or not the navigation program exists or the mobile control signal is received. When the navigation program does not exist and the movement control signal is not received, the control unit 11 determines whether the flight can be stopped in step S36. When the vehicle is landing on the ground, the control unit 11 determines that the flight can be stopped and stops driving. When it is not determined that the flight can be stopped, the control unit 11 determines whether it is better to return (return) in step S37. The control unit 11 performs feedback control when determining that it is better to return (step S38), and otherwise performs hovering control (step S39).

  When the navigation program is present or the mobile control signal is received, the control unit 11 determines whether or not there is a mobile obstacle in step S40. When there is a movement obstacle, the control unit 11 shifts the processing to step S37, and when there is no movement obstacle, the control portion 11 moves according to the navigation program or the movement control signal (step S41). The control device 83 can confirm the flight status based on the status information transmitted in step S33. The control unit 11 determines in step S42 whether or not the target position has been reached. If the target position has not been reached, the process returns to step S31, and steps S31 to S42 are repeated.

  7 to 9, FIG. 10A, and FIG. 10B are explanatory diagrams for explaining the control for moving the mobile photographing device 10 to the target position.

  FIG. 7 shows an example in which the drone 70, which is the moving body 10a, reaches the target position Pt from the position P1 distant from the target position Pt through the position P2. In the present embodiment, the zoom control of the imaging unit 10b is to perform zoom-up or zoom-down in the vertical direction, for example. Therefore, it is preferable to perform shooting with the drone 70 positioned right above the target position Pt. For example, it is assumed that the tree 71 located at the target position Pt is photographed by the mobile photographing device 10.

In this case, if the position of the tree 71 (target position Pt) is known, by setting the target position Pt, the drone 70 can be moved to the target position Pt using a function such as GPS. The example of FIG. 7 shows a method of moving to the target position Pt when the position of the tree 71 is not known. Now, it is assumed that the drone 70 is located directly above the position P1 and is stopped in a flight attitude in which the pitching angle and the bank angle are both 0 degrees. The control unit 11 of the mobile photographing device 10 acquires the ground clearance H1 according to the determination result of the altitude / attitude determination unit 16c. Further, the control unit 11 obtains a shooting direction based on the horizontal direction of the tree 71 captured by the built-in camera 18, that is, a depression angle θ1 by image analysis of the built-in camera 18. In this case, the horizontal distance from the position P1 to the target position Pt is X1, and the following expression (1) is established. It should be noted that the following equation (1) should be corrected in consideration of the height of the tree 71.
H1 / X1 = tan θ1
X1 = H1 / tan θ1 (1)
Using the GPS function or the like, the control unit 11 moves from the position P1 by a distance X1 and moves directly above the target position Pt. At the position P2 during the movement from the position P1 to the target position Pt, the pitching angle and the bank angle of the drone 70 may not be 0 degrees as shown in FIG. In this case, when the moving distance to the target position Pt is obtained using the image of the built-in camera 18, the depression angle θ1 is corrected according to the posture of the moving body.

  When determining that the target position has been reached, the control unit 11 performs posture control in step S43. The permission determination unit 13f determines whether or not the posture of the moving body 10a is in the state change permission posture based on the determination result of the posture determination unit 13e (step S44).

  The permission determination unit 13f does not permit the photographing control unit 12a to change the state when the moving body 10a is not in the state change permission posture. Accordingly, when the moving body 10a is not in the state change permission posture, for example, when one of the pitching angle and the bank angle is not approximately 0 degrees in a flying body such as a drone, the photographing control unit 12a. Does not perform zoom or focus control on the imaging unit 10b. Therefore, such a change in the state of the imaging unit 10b does not cause a moment to act on the moving body 10a to make the movement of the moving body 10a unstable. In this case, the control unit 11 may perform the movement control so as to further approach (fall) the target object in step S46.

  Here, as a result of the posture control of the moving body 10a by the posture control unit 13c, it is assumed that the posture of the moving body 10a becomes the state change permission posture. When the determination result of the posture determination unit 13e indicates that the moving body 10a is in the state change permitted posture, the permission determination unit 13f permits the photographing control unit 12a to change the state (step S45).

  For example, when both the pitching angle and the bank angle of the moving body 10a become substantially 0 degrees, the shooting control unit 12a is allowed to perform zoom control and focus control of the imaging unit 10b. By such photographing control, the state of the image pickup unit 10b changes and the position of the center of gravity changes. However, the change in the center of gravity in this case is a change in the vertical direction with respect to the position of the center of gravity of the moving body 10a, and the moment does not act on the moving body 10a depending on the change in the center of gravity of the imaging unit 10b. Therefore, in such a hovering state, if there is no strong wind, the moving body 10a does not become unstable due to a change in the vertical center of gravity of the image pickup unit (including the optical system). Further, since the posture of the moving body 10a is maintained stable, the photographing range by the image pickup unit 10b is also stabilized, and it is possible to accurately capture a desired photographing target and perform photographing in a reliable focused state. In other words, this image pickup unit can be attached to a moving body, and when shooting is performed in cooperation with this, the determination result of whether or not the posture of the moving body has reached a predetermined state change permission posture (here, hovering state) is obtained. On the basis of the above, a control unit is provided for controlling the change of the above-mentioned photographing state accompanied by the change of the center of gravity in the vertical direction. This is not guaranteed by the system even if the mobile unit permits, even if the control unit of the imaging unit determines hovering to determine permission, or even if another control unit determines this. Good.

  Now, it is assumed that when the drone 70 reaches the target position Pt, the image capturing unit 10b is capturing images on a relatively wide angle side. In this case, for example, images 85 and 86a shown in FIG. 6B are displayed on the operation screen 83a of the control device 83. The image 85 shows an image of a part of the entire tree 71 obtained by the built-in camera 18. The built-in camera 18 is provided for the flight of the drone 70 and the overall observation of the landscape, and is capable of sufficiently wide-angle shooting. On the other hand, the imaging unit 10b is intended to shoot a specific area with high image quality. When the target position Pt is reached, the image capturing unit 10b is capturing images on a relatively wide-angle side, and the image captured by the image capturing unit 10b is displayed on the operation screen 83a as an image 86a showing some leaves of a part of the tree 71. It is displayed.

Here, it is assumed that the user 81 confirms the display 88 of FIGS. 6B and 6C and operates the operation button 87a to perform shooting on the telephoto side. Then, for example, an image 86b shown in FIG. 6C is displayed on the operation screen 83a of the control device 83. The image 86a shows an image of one leaf of the tree 71. In this way, it is possible to reliably capture the zoomed-in image without affecting the stability of the drone 70. If the zoom operation OK by the display 88 is not displayed, even if the user 81 operates the operation buttons 87a and 87b, the zoom control is not performed by these operations, and the stability of the drone 70 is maintained. Does not affect.
Here, in order to facilitate the description, an example is shown in which the vertical direction, the vertical direction, is enlarged on the optical axis when the lens is set to the telephoto position, but of course, if a bending optical system such as a mirror or prism is adopted, It is possible to bend the optical path at the time of shooting, and it is possible to design the center of gravity to move vertically, but not to enlarge it. Although the shooting is performed in different shooting states such as zooming up, the output result of the wind speed detection unit 15 is not only based on the determination result of whether or not the posture of the moving body is in the predetermined state change permission posture. It goes without saying that you may decide according to. It is characterized by having a control unit that determines such a situation, or receives the result of the determination by communication, and controls the change of the above-mentioned photographing state with the vertical direction or a predetermined change of the center of gravity that can be grasped in advance. It is sufficient to prepare a database or the like in which data is stored for each shooting state in the memory, and refer to the contents to determine what the result will be. Here, although this memory is not specified, it is assumed that a recording unit for recording the photographing result is built in (or connected to) the photographing control unit 12a. In addition, wind speed and wind power should be managed in the database as well. That is, in which shooting state (shooting mode), which wind force and wind speed can be endured may be managed in the database. Further, changes in the wind speed and the wind force are not constant and there may be gusts. Therefore, the changes in the wind speed and the wind force may be monitored and the change in the wind that may occur at the time of photographing may be estimated from the tendency. The judgment may be made by reflecting the weather forecast information and the like. This determination may be performed by the mobile unit or the image capturing unit, and these operating units may cooperate with artificial intelligence on the cloud in cooperation with the Internet or the like.

  In the example of FIG. 7, one drone moves from the position P1 to the target position Pt. On the other hand, FIG. 8 shows an example in which the drone that moves to the target position Pt and the drone that obtains the distance to the target position Pt are different. The drone 77 is a general drone that has the built-in camera 18, and does not include the image pickup unit 10b. The drone 77 obtains the equation (1) by the method described above, and gives the information of the moving distance X1 to the drone 78 which is the moving body 10a to which the imaging unit 10b is attached. The built-in camera 18 of the drone 78 may be omitted. The drone 78 moves to the target position Pt based on the information on the moving distance X1 acquired by the drone 77. Other actions are similar to those of the example of FIG.

  In addition, the example of FIG. 9 shows an example in which the imaging unit 10b is not only capable of shooting on the telephoto side but also shooting on a sufficiently wide angle side. In this case, the built-in camera 18 can be omitted from the drone 79, which is the moving body 10a to which the imaging unit 10b is attached. The imaging unit 10b includes a so-called high-magnification zoom lens capable of shooting in a sufficiently wide focal length range from the wide-angle side to the telephoto side. Even when the drone 79 is located at the position P1 and the pitching angle and the bank angle are both approximately 0 degrees, the imaging unit 10b can capture the image of the tree 71 at the target position Pt.

  FIG. 10A shows a visual field range 91a by the imaging unit 10b in this case. A part of the image 92a of the tree 71 is included in the visual field range 91a. For example, by displaying an image obtained for the visual field range 91a on the operation screen 83a of the control device 83, it is possible to specify the target position Pt by touch operation or the like. For example, the user touches the position corresponding to the image 92a of FIG. 10A in the image displayed on the operation screen 83a to specify that the position of the tree 71 shown in this image 92a is the target position Pt. You can The control unit 11 obtains the moving distance X1 by the same method as the above equation (1). The drone 79 moves to the target position Pt based on the acquired information on the moving distance X1. Other actions are similar to those of the example of FIG. After the drone 79 reaches the target position Pt, it is shown that both the pitching angle and the bank angle are in the state change permitting posture of substantially 0 degrees, and then zoom operation is performed to perform zoom photographing.

  FIG. 10B shows the visual field range 91b by the imaging unit 10b in this case. A part of the image 92b of the tree 71 is included in the visual field range 91b. Other operations are similar to those of the example of FIG. 7.

  As described above, in the present embodiment, when the posture of the moving body is the predetermined state change permission posture, the image capturing unit is configured so that the moment acting on the moving body due to the movement of the center of gravity due to the state change of the image capturing unit becomes zero. Is mounted on a moving body, and it is determined whether or not the posture is allowed to be changed to determine whether or not to permit the change in the state of the image pickup unit. This prevents the moving object from becoming unstable due to the change in the state of the image pickup unit, stabilizes the shooting range by the image pickup unit, and enables shooting in a reliable focused state that accurately captures the desired shooting target. Can be

(Modification)
The mobile photographing device 10 in FIG. 1 includes a moving body 10a and an image capturing section 10b, and includes a control section 11 that controls the moving body 10a and the image capturing section 10b. In addition, the present embodiment employs a moving body and an imaging unit (imaging device) that each have a control unit and operates, and exchange information by communication between the moving body and the imaging unit. It is also possible to perform shooting control similar to that of the mobile shooting device 10 of FIG.

  11A and 11B are flowcharts showing the control by the imaging unit and the control by the moving body in this case, respectively. In FIG. 11A, in step S11, the imaging unit predicts a change in a physical quantity such as a change in the position of the center of gravity due to a state change due to focus control, zoom control, or the like. In step S12, the imaging unit communicates with the moving body to acquire information on whether the moving body is in the state change permission posture. In step S13, the imaging unit determines whether the moving body is in the state change permission posture.

  The imaging unit does not permit the state change such as focus control and zoom control when the moving body is not in the state change permission posture (step S14). As a result, when the moving body is not in the state change permission posture, for example, when one of the pitching angle and the bank angle is not approximately 0 degrees in a flying body such as a drone, the imaging unit zooms or Does not perform focus control. Therefore, such a change in the state of the imaging unit does not cause a moment to act on the moving body to make the movement of the moving body unstable.

  Here, it is assumed that the posture of the moving body becomes the state change permission posture as a result of the posture control of the moving body. When it is indicated that the moving body is in the state change permitting posture, the image capturing unit permits state change such as zooming and focus control (step S15). In this way, the imaging unit changes the state in cooperation with the moving body, thereby performing shooting while preventing an unintended moment from being applied to the moving body.

  On the other hand, in the moving body, as shown in FIG. 11B, the shooting target is designated in step S21. The moving body performs control for moving to the designated shooting target position (step S22). When the moving body reaches the shooting target position, the moving body determines whether or not the posture is the state change permitting posture (step S23).

  The moving body does not permit the state change to the imaging unit when the moving body is not in the state change permission posture (step S24). As a result, when the moving body is not in the state change permission posture, for example, when one of the pitching angle and the bank angle is not approximately 0 degrees in a flying body such as a drone, the imaging unit zooms or Does not perform focus control. Therefore, such a change in the state of the imaging unit does not cause a moment to act on the moving body to make the movement of the moving body unstable.

  Here, it is assumed that the posture of the moving body becomes the state change permission posture as a result of the posture control of the moving body. Then, the moving body permits the image capturing unit to change the state while maintaining the state change permitting posture (step S25). In this way, even when the imaging unit performs zooming and focus control, it is possible to cause the imaging unit to perform imaging while preventing an unintended moment from being applied to the moving body.

(Second embodiment)
FIG. 12A is a block diagram showing an imaging device according to a second embodiment of the present invention, and FIG. 12B is a block diagram showing an imaging mobile unit according to the second embodiment. 13A is a block diagram showing a photographing remote control device of the photographing device of FIG. 12A, and FIG. 13B is a block diagram showing a moving remote control device of a moving body of FIG. 12B.

  The first embodiment has shown the mobile image capturing apparatus in which the single control unit 11 controls the moving body 10a and the image capturing unit 10b. The present embodiment employs a moving body and an image capturing unit (imaging device) that individually have a control unit and operates, and exchanges information by communicating between the moving body and the image capturing unit to move. The state of the photographic equipment is changed while ensuring the stability of the body to enable reliable photography.

  Note that, also in the present embodiment, the state change permission posture of the moving body is determined to determine whether or not the photographing control is possible. However, the state change permission posture is determined, and whether or not the photographing control is possible based on this determination. The determination of and the shooting control based on this determination are not limited to the example of the present embodiment, and any of the shooting device, the shooting moving body, the moving remote control device, and the shooting remote control device may perform the determination. The above devices may be dispersed, and further devices other than these devices may be performed.

  In FIG. 12A, the imaging device 30 is attached to the moving body 40 of FIG. 12B by an attachment member (not shown). The photographing device 30 includes an image pickup unit 31. The image pickup unit 31 is provided with an image pickup device (not shown) composed of a CCD, a CMOS sensor or the like, and an optical system 31a guides a subject optical image to the image pickup surface of the image pickup device. The photographing control unit 32a of the control unit 32 controls a mechanical unit (not shown) of the optical system to drive and control the focus lens, the zoom lens, and the diaphragm formed in the optical system 31a. The image sensor photoelectrically converts the subject optical image to obtain a captured image. The control unit 32 may be configured by a processor such as a CPU (not shown), or may realize each function according to a program stored in a memory (not shown).

  In the present embodiment, a physical information storage unit 31b that holds information regarding the optical system 31a is provided in the image pickup unit 31. Although FIG. 12A shows an example in which the optical system 31a is provided in the image pickup unit 31, a detachable interchangeable lens is adopted in the housing of the photographing device 30 as an optical system, and a subject optical image from the interchangeable lens is adopted. May be formed on the image pickup surface of the image pickup device. The optical system 31a may have the configuration shown in FIG.

  In FIG. 12A, the operation unit 33 outputs an operation signal based on a user operation to the control unit 32. The imaging control unit 32a in the control unit 32 can generate a focus signal and a zoom signal based on an operation signal from the operation unit 33 to drive each lens of the optical system 31a.

  By the focus control and zoom control by the photographing control unit 32a, each lens of the optical system 31a is driven to perform focusing and zooming. The focus / angle-of-view information acquisition unit 32b acquires information about the focus position and zoom position and information about the shooting angle of view from the imaging unit 31 and outputs the information to the angle-of-view control unit 32c. The angle-of-view control unit 32c is controlled by the imaging control unit 32a and controls the trimming control unit 32d to perform electronic zooming by trimming when zooming exceeds the optical zoom range of the optical system 31a.

  For example, as shown in FIG. 3, the optical system 31a is composed of a plurality of lens groups arranged in the optical axis direction. The optical system 31a is arranged in a lens barrel (not shown) attached to the front surface of the housing of the photographing device 30. The interchangeable lens includes this lens barrel. During zooming from the wide-angle end side to the telephoto end, as shown in FIG. 3, the distance from the front end to the rear end of the lens group changes, and the arrangement position of each lens group on the optical axis changes. The lens barrel itself may expand and contract according to zooming. By such movement or expansion / contraction of the lens group or the lens barrel, the position of the center of gravity of the entire imaging device 30 changes.

  A moment may be generated in the moving body 40 depending on the position of the center of gravity of the moving body 40 such as a drone, the position of the center of gravity of the entire photographing device 30, the weight of the photographing device 30, and the mounting position of the photographing device 30 on the moving body 40. is there. However, the line segment connecting the position of the center of gravity of the moving body 40 and the position of the center of gravity of the entire imaging device 30 is in the vertical direction, and if the change in the center of gravity due to the change in the state of the imaging device 30 during zooming or the like changes only in the vertical direction. Even when the state of the photographing device 30 changes, the moment acting on the moving body 40 remains 0.

  That is, when the moving body 40 is in a certain state permitting posture with respect to the vertical direction, the line segment connecting the position of the center of gravity of the moving body 40 and the position of the center of gravity of the entire image taking device 30 is in the vertical direction and the image taking device such as a zoom operation is performed. When the moving body 40 is equipped with the imaging device 30 so that the change of the center of gravity due to the change of the state of 30 changes only in the vertical direction, if the moving body 40 maintains the state change permitting posture, even if a zoom operation or the like is performed. The resulting moment remains zero. For example, when the moving body 40 is a flying body such as a drone, the flight attitude when the pitching angle that is the angle around the left-right axis and the bank angle that is the angle around the front-rear axis of the moving body 40 are both 0 degrees May be set to the state change permission posture.

  The physical information storage unit 32e of the photographing device 30 stores physical quantity information indicating how the position of the center of gravity of the photographing device 30 changes when zoom focusing or zooming is performed. That is, if the moving body 40 maintains the state change permitting posture in the physical information storage unit 32e, the zoom operation is performed in the physical information storage unit 32e by appropriately attaching the moving body 40 to the moving body 40 and setting the movement of the center of gravity depending on the change in the state of the moving body 40. Even if the above is performed, the information indicating that the moment generated in the moving body 40 due to this is still 0 is stored.

  The imaging device 30 has a communication unit 37, and the communication control unit 32f in the control unit 32 controls the communication unit 37 so that information can be transmitted / received to / from the communication unit 47 of the mobile unit 40 described later. To When changing the state in which the center of gravity moves such as a zoom operation, the control unit 32 makes the moving body 40 in the state change permission posture by communicating with the communication unit 37 and the communication unit 47 of the moving body 40 in advance. It is designed to acquire information about whether or not it is. The control unit 32 controls the imaging control unit 32a only when the information indicating that the mobile unit 40 is in the state change permitting posture is provided to change the state accompanied by the movement of the center of gravity such as zooming and focusing. It is supposed to allow it.

  Further, in the present embodiment, as will be described later, the moving body 40 transmits shooting control information for zoom and focus control to the control unit 32 of the shooting device 30 via the communication units 47 and 37. The photographing operation of the photographing device 30 may be controlled. In this case, the moving body 40 may transmit shooting control information for zoom and focus control to the moving body 40 only when it is in the state change permission posture.

  Further, as described above, it is possible to adopt an interchangeable lens as the optical system 31a. In this case, it may be better to store the same information as the information stored in the physical information storage unit 32e in the interchangeable lens. In consideration of this case, FIG. 12A shows an example in which the physical information storage unit 31b is provided in the imaging unit 31. When the zoom operation is performed, the control unit 32 may transmit the information stored in the physical information storage unit 31b of the imaging unit 31 to the moving body 40 via the communication unit 37 in advance. Good.

  The control unit 32 is provided with the captured image from the image capturing unit 31 and performs predetermined image signal processing, for example, color adjustment processing, matrix conversion processing, noise removal processing, and other various signal processing, and then causes the recording unit 36 to perform recording. Can be given and recorded. As the recording unit 36, for example, an IC memory can be adopted. Note that the same information as the information stored in the physical information storage unit 32e may be recorded in the physical information storage area 36a of the recording unit 36. Further, the control unit 32 can transfer the captured image to the photographing remote control device 50 via the communication unit 38.

  The photographing device 30 is also provided with an elevation angle / direction sensor 34. The elevation angle and azimuth sensor 34 detects the posture of the photographing device 30 and outputs the detection result to the control unit 32. The control unit 32 can determine the shooting direction of the imaging unit 31 based on the elevation angle and the detection result of the azimuth sensor 34. The control unit 32 may also transmit information regarding the shooting direction of the image pickup unit 31 to the moving body 40 via the communication unit 37.

  A clock unit 35 is also provided in the photographing device. The clock unit 35 generates time information and outputs it to the control unit 32. In order to maintain the stability of the moving body 40, the control unit 32 uses the time information from the clock unit 35 so that the control amount per unit time in zoom and focus control is within a predetermined control amount. It may be configured to control each unit. Further, the control unit 32 synchronizes the time information of the clock unit 35 with the time information used by the mobile unit 40 in order to perform control in cooperation with the imaging device 30 and the mobile unit 40. Good.

  Further, the control unit 32 is provided with a distance determination unit 32g. The distance determination unit 32g can calculate the distance to the subject. The control unit 32 may be configured to transmit the distance determination result by the distance determination unit 32g to the moving body 40 via the communication unit 37. The moving body 40 can use this distance information for determining whether or not the target position has been reached.

  In FIG. 12B, the moving body 40 has a control unit 42. The control unit 42 may be configured by a processor such as a CPU (not shown), or may realize each function according to a program stored in a memory (not shown). The moving body 40 is provided with a propulsion unit 41 and a power supply 43. The power supply 43 is composed of, for example, a battery and supplies electric power to each unit of the moving body 40. The propulsion unit 41 generates a propulsive force for moving the moving body 40. When the moving body 40 is a drone that moves in the air, it is necessary to generate propulsive force even when hovering, while when the moving body 40 is an airplane or the like, propulsive force may be unnecessary when gliding. is there.

  When the moving body 40 is a drone, for example, a plurality of propulsion units 41 are provided at the end of an arm (not shown), and a control unit 42 is provided at the upper center of the arm. Each propulsion unit 41 includes, for example, a motor and a propeller that is rotationally driven by each motor. In this case, the motors forming the propulsion units 41 are controlled independently of each other so that the moving body 40 can move in a predetermined posture and speed. An attachment member is attached to the lower center part of the arm so that the imaging device 30 of FIG. 12A can be attached.

  The control unit 42 is provided with a propulsion control unit 42a, and the propulsion control unit 42a can independently control the propulsive forces of the plurality of propulsion units 41. The control unit 42 is provided with a direction control unit 42b and a posture control unit 42c. The direction control unit 42b and the posture control unit 42c respectively output control signals for controlling the moving direction of the moving body 40 or the posture of the moving body 40 to the propulsion control unit 42a.

  The movement determination unit 42d determines the movement direction of the moving body 40 and outputs the determination result to the direction control unit 42b. Further, the posture determination unit 42e determines the posture of the moving body 40 and outputs the determination result to the posture control unit 42c. In order to control the movement direction of the moving body 40 to the designated direction, the direction control unit 42b controls the propulsion control unit 42a so that the movement determination result matches the designated movement direction. The attitude control unit 42c also controls the propulsion control unit 42a so that the attitude determination result matches the specified attitude.

  Various sensor information is provided to the control unit 42 in order to determine the moving direction and posture of the moving body 40. In the example of FIG. 12B, the moving body 40 is provided with a spatial information acquisition unit 45a, a position / direction determination unit 45b, and an altitude / posture determination unit 45c. The spatial information acquisition unit 45a can be configured by, for example, a radar or a camera, and acquires spatial information such as ground speed. The position / direction determining unit 45b can be configured by, for example, a GPS (Global Positioning System) receiver, and acquires the position and direction information of the moving body 40. The altitude / posture determination unit 45c can be configured by, for example, an atmospheric pressure sensor, an acceleration sensor, a gyroscope, or the like, and determines the altitude and posture of the moving body 40. The determination results of the determination units 45a to 45c are given to the movement determination unit 42d and the posture determination unit 42e, and the movement determination unit 42d and the posture determination unit 42e determine the moving direction and the posture of the moving body 40 based on these determination results. It is designed to judge.

  The moving body 40 is provided with a wind speed detecting unit 44 similar to the wind speed detecting unit 15 in FIG. The wind speed detector 44 detects the wind speed relative to the moving body 40 and outputs it to the controller 42. The control unit 42 can use the detection result of the wind speed for the direction determination and the posture determination, and can also use the detection result of the wind speed for the propulsion control.

  The control unit 42 is provided with a power supply determination unit 42f. The power supply determination unit 42f determines the remaining battery level of the power supply 43. When the battery remaining amount is less than or equal to a predetermined threshold value, the control unit 42 performs control for stopping movement or moving to a predetermined position. For example, when the moving body 40 is a drone, when the remaining battery capacity becomes smaller than a predetermined threshold value, control such as returning the moving body 40 to a predetermined base station is performed in order to prevent a fall.

  Further, the moving body 40 is provided with a recording unit 46. The recording unit 46 can record information on the moving route of the moving body 40. Further, the recording unit 46 may be capable of recording the image information transmitted from the photographing device 30 via the communication unit 47. A communication unit 48 is provided on the mobile body. The communication unit 48 can communicate with the communication unit 63 of the mobile remote control device 60 described later.

  The mobile unit 40 is provided with a communication unit 47. The communication unit 47 can send and receive information to and from the communication unit 37 of the photographing device 30. The communication control unit 42g controls the communication unit 47 to receive the information from the photographing device 30 and to send the information from the permission determination unit 42h to the communication unit 37 of the photographing device 30.

  Further, the moving body 40 may be provided with a built-in camera 49. The built-in camera 49 can pick up a picked-up image by picking up an image in a predetermined direction around the moving body 40. In particular, the built-in camera 49 is configured to allow relatively wide-angle shooting so that the surroundings can be easily observed when the moving body 40 moves. For example, when the moving body 40 is an airplane such as a drone, the built-in camera 49 may be capable of shooting the front and the lower side in the moving direction as the shooting range while the moving body 40 is flying horizontally. desirable.

  When the permission determination unit 42h of the control unit 42 receives the information requesting zoom or focus control from the imaging device 30, whether or not the posture of the moving body 40 is in the state change permission posture according to the determination result of the posture determination unit 42e. It is determined whether or not it is determined, and the determination result is supplied to the control unit 32 of the imaging device 30 via the communication units 47 and 37. Note that the control unit 32 of the imaging device 30 is configured to perform zooming, focus control, and the like only when a determination result indicating that the posture of the moving body 40 is in the state change permission posture is given. ..

  The permission determination unit 42h permits the control unit 32 of the imaging device 30 via the communication units 47 and 37 to perform zoom and focus control only when the posture of the moving body 40 is in the state change permission posture. When the permission signal is supplied and the state change permission posture is not set, a prohibition signal for prohibiting zoom and focus control is supplied to the control unit 32 of the photographing device 30 via the communication units 47 and 37. May be.

  In addition, when the permission determination unit 42h permits the control unit 32 of the image capturing device 30 to change the state associated with the center of gravity change such as zooming and focus control, the permission determining unit 42h sets the state change permission posture in the period in which the image capturing device 30 changes the state. The propulsion force required to maintain, that is, the propulsion force for ensuring stability (hereinafter referred to as stable propulsion force) is calculated, and in order to obtain this stable propulsion force, the propulsion control unit 42a, the direction control unit 42b, and the Control information for controlling the attitude control unit 42c is generated.

  The permission determination unit 42h determines whether or not a stable propulsion force can be obtained in the propulsion unit 41 based on the detection result of the wind speed detection unit 44 and the like. The permission determination unit 42h may prohibit the state change of the photographing device 30 when there is no possibility that stable propulsive force is obtained during the period when the photographing device 30 changes the state.

  The photographing device 30 can be operated by an operation unit 33 provided in the photographing device 30, and photographing control can be performed by using the photographing remote control device 50. In addition, the moving body 40 can move in a self-supporting manner according to a program recorded in a storage unit (not shown), and movement control can be performed using the movement remote control device 60.

  As shown in FIG. 13A, the imaging remote control device 50 has an operation unit 51 and a control unit 52. The control unit 52 can be configured by a processor such as a CPU (not shown) and controls each unit of the photographing remote control device 50. The imaging remote control device 50 is provided with a storage unit 56, and the storage unit 56 can store various kinds of information and programs used in the control unit 52. The photographing remote control device 50 has a communication unit 53, and the communication unit 53 can send and receive information to and from the communication unit 38 of the photographing device 30. The control unit 52 is provided with a communication control unit 52a, and the communication control unit 52a can exchange information with the control unit 32 of the imaging device 30 via the communication units 53 and 38.

  The control unit 52 can generate an operation signal for operating the imaging device 30 based on a user operation on the operation unit 51 and transmit the operation signal to the control unit 32 of the imaging device 30 via the communication units 53 and 38. It is like this.

  Further, the image capturing remote control device 50 is provided with a display unit 54, and the communication control unit 52a can receive a captured image from the image capturing device 30 and supply the captured image to the display unit 54. The display unit 54 can display an image captured by the image capturing device 30 on the display screen.

  The photographing remote control device 50 may be provided with a communication unit 55. The communication unit 55 is configured to be communicable with a communication unit 65 of the mobile remote control device 60 described later. The communication control unit 52a controls the communication unit 55 to enable the exchange of information between the control unit 52 and the control unit 62 of the mobile remote control device 60. Note that the communication unit 55 can be omitted if communication with the mobile remote control device 60 is not performed.

  As shown in FIG. 13B, the mobile remote control device 60 has an operation unit 61 and a control unit 62. The control unit 62 can be configured by a processor such as a CPU (not shown), and controls each unit of the mobile remote control device 60. The mobile remote control device 60 is provided with a storage unit 66, and the storage unit 66 can store various kinds of information and programs used in the control unit 62. The mobile remote control device 60 has a communication unit 63, and the communication unit 63 can send and receive information to and from the communication unit 48 of the mobile unit 40. The control unit 62 is provided with a communication control unit 62a, and the communication control unit 62a enables exchange of information with the control unit 42 of the mobile unit 40 via the communication units 63 and 48.

  The control unit 62 can generate an operation signal for operating the mobile unit 40 based on a user operation on the operation unit 61 and transmit the operation signal to the control unit 42 of the mobile unit 40 via the communication units 63 and 48. It is like this.

  The mobile remote control device 60 is provided with a display unit 64, and the display unit 64 can display various menu displays and the like for controlling the moving body 40 on the display screen.

  Further, the communication control unit 62a can receive the captured image from the moving body 40 and supply it to the display unit 64. The display unit 64 can display the captured image from the moving body 40 on the display screen.

  The mobile remote control device 60 may also be provided with a communication unit 65. The communication unit 65 is configured to be able to communicate with the communication unit 55 of the photographing remote control device 50. The communication control unit 62a controls the communication unit 65 to enable the exchange of information between the control unit 62 and the control unit 52 of the photographing remote control device 50. Note that the communication unit 65 can be omitted if communication with the photographing remote control device 50 is not performed.

  Next, the operation of the embodiment configured as described above will be described with reference to FIGS. 14 to 17B. FIG. 14 is a flowchart showing camera control, and FIG. 15 is a flowchart showing moving body control. 16A and 16B are explanatory diagrams showing a state and an operation screen at the time of operation, and FIGS. 17A and 17B are explanatory diagrams showing a state and an operation screen at the time of photographing.

  When trying to improve the quality of an image obtained by imaging, control of zoom, exposure, focus, etc. is likely to be complicated, and it becomes difficult to concentrate on the operation of the moving body. Therefore, in the present embodiment, an example is shown in which a moving body such as a drone and an imaging device are operated by different persons individually.

  As shown in FIG. 16A, for example, the operator 95 holds the housing 60a of the mobile remote control device 60 with the right hand 96R and operates the operation lever 61a as the operation unit 61 for operating the moving body 40 with the left hand 96L. Manipulate. As for the photographing device 30, as shown in FIG. 17A, it is assumed that the photographer 98 performs photographing control by operating the photographing remote control device 50 supported by the left hand 99L with the right hand 99R.

  FIG. 16B shows the operation screen 101 displayed on the display screen 64 a of the display unit 64 of the mobile remote control device 60. In the example of FIG. 16B, a map display 102 showing the position of the moving body 40 such as a drone is displayed on the operation screen 101. A map is displayed on the map display 102, and the point display 102a indicates the position of the moving body 40. A message display area 103 is provided in the operation screen 101, and a message corresponding to the operation of the message display buttons 104 and 105 is displayed in the message display area 103. The message display button 104 is for displaying a message as to whether or not the posture of the moving body 40 is in the state change permission posture, that is, whether or not to permit the state change such as the movement of the center of gravity of the imaging device 30. In the example of FIG. 16B, the display of “state change not permitted” indicates that the moving body 40 is not in the state change permitted posture. If the user is in the state change permission posture, “state change permission” is displayed in the message display area 103, for example. Further, the message display button 105 is for displaying a display indicating the wind speed in the message display area 103.

  An example of the case where the drone 70 shown in FIGS. 5A and 5B is adopted as the moving body 40 of FIG. 12B will be described. The imaging device 30 is attached to the drone 70 which is the moving body 40 via the attachment member 75. The imaging device 30 is attached to the drone 70 that is the moving body 40 via the attachment member 75, and exchanges information with the control unit 42 built in the drone 70 via the cable 76. There is. The drone 70 has four propulsion units 71 to 74 corresponding to the propulsion unit 41 of FIG. 12B, and the propulsion units 71 to 74 maintain a desired posture.

  The imaging device 30 has a lens barrel 21a that expands and contracts according to zooming. In the imaging device 30, for example, the optical axis is arranged vertically below the center of gravity of the drone 70, and the lens barrel 21a is in the contracted state shown in FIG. 5A when the drone 70 is in a flight attitude when the pitching angle and the bank angle are both 0 degrees. And the stretched state shown in FIG. 5B.

  In step S71 of FIG. 15, the control unit 42 of the drone 70 uses the altimeter, radar, built-in camera, GPS, etc. to determine the current state. The control unit 42 enters a standby state for access from the mobile remote control device 60 to the moving body 40 in step S72. In step S73, the control unit 42 determines whether or not the navigation program exists or the mobile control signal is received. When the navigation program does not exist and the movement control signal has not been received, the control unit 42 determines in step S74 that the flight can be stopped. When landing on the ground, the control unit 42 determines that flight can be stopped and stops driving. When it is not determined that the flight can be stopped, the control unit 42 determines whether it is better to return (return) in step S75. When it is determined that the return is better, the control unit 42 performs the feedback control (step S76), and otherwise executes the hovering control (step S77).

  When the navigation program is present or the mobile control signal is received, the control unit 42 determines whether or not there is a mobile obstacle in step S78. If there is a movement obstacle, the control unit 42 shifts the processing to step S75, and if there is no movement obstacle, the control portion 42 moves according to the navigation program or the movement control signal (step S79). The control unit 42 transmits the image from the built-in camera 49 to the mobile remote control device 60 in step S80 so that the mobile remote control device 60 can confirm the flight state, and the state based on the state determination result acquired in step S71. The information is transmitted to the communication unit 63 of the mobile remote control device 60 via the communication unit 48 (step S81).

  A camera capable of outputting an infrared image may be adopted as the built-in camera 49. In this case, an infrared image can be output in step S80. The infrared image has different wavelengths and sensitivity characteristics from the normal image, and it is possible to obtain a pattern according to the water content of the subject. For example, when the drone 70 is used in fields such as agriculture and forestry, an infrared camera is used as the built-in camera 49 to determine the growth of agricultural products or the like based on the water content of the agricultural products or the like. .. For example, the wavelength sensitivity characteristic of the built-in camera 49 and the wavelength sensitivity characteristic of the imaging unit 31 of the photographing device 30 may be set to different characteristics. For example, the built-in camera 49 may be an infrared camera and the photographing device 30 may be a zoom camera.

  The processing of steps S71 to S81 in the drone 70 is performed regardless of whether or not the imaging device 30 is installed. If the imaging device 30 is installed, the drone 70 determines whether or not there is camera communication in step S82.

  On the other hand, the photographing device 30 is operated by the photographing remote control device 50. As shown in FIG. 17A, the photographer 98, for example, holds the photographing remote control device 50 with his / her left hand 99L and photographs with the right hand 99R while viewing the captured image (through image) displayed on the display screen 54a of the display unit 54. I do. FIG. 17B shows a display screen 54a of the display unit 54 of the photographing remote control device 50 in this case. In the example of FIG. 17B, the bird image 111a is displayed in the through image display area 111 on the display screen 54a.

  The photographing device 30 performs the control shown in FIG. In step S51 of FIG. 14, the control unit 32 determines whether communication can be performed between the communication unit 37 and the communication unit 47 of the mobile unit 40. In step S52, the control unit 32 performs automatic exposure control (AE), drives the image pickup unit 31, and acquires a picked-up image. In step S53, the control unit 32 determines whether or not it is the drone mode in which it is attached to the moving body and the image is taken. In the case of a mode other than the drone mode, the operation according to the operation is performed in that mode. When in the drone mode, the control unit 32 communicates with the drone 70 in step S54.

  Note that during the exposure control in step S52, autofocus control may be performed within a range that does not affect the navigation stability of the drone 70.

  In step S83, the control unit 42 of the drone 70 receives the request from the imaging device 30 and takes action according to the request. For example, the control unit 42 may perform a determination process as to whether or not the photographing can be permitted when a request for permission of the photographing is generated from the photographing device 30. Further, it may be possible to transmit to the mobile remote control device 60 that the request for permission of shooting from the shooting device 30 has been generated, and display that effect on the display screen 64a. Further, in response to a request from the imaging device 30, a warning may be issued or flight stop processing may be performed, and information regarding these processings may be transmitted to the mobile remote control device 60 and displayed on the display screen 64a. You may display that effect.

  The control unit 42 of the drone 70 does not permit the image capturing device 30 to perform image capturing when the image capturing device 30 may interfere with stability during flight or the like. The shooting permission instruction may be generated in response to a user operation. For example, the photographing permission button 106 may be arranged on the display screen 64a of the mobile remote control device 60. When the operator 95 operates this button 106, the photographing device 30 may be permitted to photograph.

  In step S55, the control unit 32 of the photographing device 30 determines whether or not the drone 70 gives the photographing permission. If the photographing permission is not given, the process proceeds to step S56 and the photographing related Stop various operations. For example, the photographing device 30 may shift to the energy saving mode.

  The control unit 32 of the photographing device 30 transmits the captured image to the photographing remote control device 50 in step S57. Next, in step S58, the control unit 32 determines whether the drone 70 has given permission to change the state.

  In step S84, the permission determination unit 42h of the drone 70 determines whether or not the posture of the moving body 40 is the state change permission posture. The permission determination unit 42h determines whether or not the state change permission is possible depending on whether or not the moving body 40 is in the state change permission posture. If the state change permission is not possible, the control unit 42 proceeds from step S85 to step S86 to make a response indicating that the state change is not permitted. If the state change permission is possible, the state change permission signal is sent in step S87. To send.

  When the state change signal is received from the drone 70 and the state change is permitted in step S58, the control unit 32 of the imaging device 30 moves the process to step S59 and changes the state according to an operation or the like. ..

  For example, in this case, the state permission signal is given from the photographing device 30 or directly from the drone 70 to the photographing remote control device 50, and the state change is permitted on the display screen 54a of the photographing remote control device 50. A display 112 indicating whether or not there is displayed. In the example of FIG. 17B, the display 112 of “zoom operation OK” indicates that the state change is permitted. In addition, when the state change is not permitted, the control unit 32 displays a warning display on the display screen 54a in step S60. For example, in this case, a display 112 such as "zoom operation NG" is displayed on the display screen 54a.

  On the display screen 54a, operation buttons 113a and 113b for zooming control of the image pickup unit 31 of the photographing device 30 are provided. The photographer 98 zooms in or zooms out the imaging unit 31 by touching the operation button 113a or 113b as necessary.

  Also in the present embodiment, as in the first embodiment, when the zoom operation NG display 112 is displayed, zoom control is not performed even if the operation buttons 113a and 113b are operated. It may be like this. Further, for example, when the display 112 of the zoom operation NG is displayed, if the operation buttons 113a and 113b are operated, a request for enabling the state change may be generated. That is, when this operation is performed in the photographing remote control device 50, the control unit 32 of the photographing device 30 determines in step S61 that a state change request is generated, and in step S62, the drone 70 can change the state. Send a request to

  In this case, the control unit 42 of the drone 70 may perform control such that the flight is temporarily stopped and the hovering operation is performed to shift to the state change permission posture. As a result of this control, if the state change permission posture is reached, the control unit 42 generates a state change permission signal in step S87.

  In the above description, it is described that the request for changing the state is generated by the user operation, but the request may be generated by the judgment of the control unit 32 of the photographing device 30. For example, if it is determined in advance that the predetermined image has been captured by the image analysis of the image capturing unit 31, a request for enabling the state change may be generated. As a result, for an important subject to be zoomed, it is possible to forcibly suspend the flight and automatically perform zooming while the drone 70 is stabilized.

  The control unit 32 determines whether or not the signal for the photographing operation (release operation) is received in step S63. When the control unit 32 determines that the release operation has been performed, the control unit 32 captures an image in step S64 and records the captured image in the recording unit 36. The control unit 32 may also transmit the captured image to the drone 70. The control unit 32 determines that the drone 70 is stopped in step S65, and if it is not stopped, the process returns to step S51.

  As described above, also in the present embodiment, the state change accompanied by the change of the center of gravity of the imaging device is permitted only when the posture of the moving body is the state change permission posture. This makes it possible to maintain the stability of the moving body and the shooting stability.

  It should be noted that in the above embodiment, an example in which communication is mainly performed between the moving body and the mobile remote control device and communication is performed between the imaging device and the shooting remote control device has been described. May be realized by performing cooperative control by communication between a photographing remote control device that remotely controls a photographing device and a mobile remote control device that remotely controls a moving body. Since the communicable distance between the photographing device and the photographing remote control device is relatively short compared to the communicable distance between the moving body and the mobile remote control device, the photographing remote control device and the mobile remote control device are In some cases, it may be advantageous to perform communication control between them to perform cooperative control. In this case, reliable control is possible even when the distance between the photographing remote control device and the moving body is relatively large. In addition, the remote control device or the mobile remote control device may be wholly or partially configurable by a smartphone. A smartphone has a relatively high degree of freedom in display. Therefore, the present embodiment has an advantage that it is possible to provide a user-friendly GUI even if a user operation is required for shooting control while maintaining stability.

  In each of the above-described embodiments, a digital camera is used as a device for shooting, but the camera may be a digital single-lens reflex camera or a compact digital camera, and a moving image such as a video camera or a movie camera may be used. It may be a camera, or of course, a camera built in a personal digital assistant (PDA) such as a mobile phone or a smartphone. Also, the word "drone" can be applied to any device that moves in balance. There are not only flying mobiles such as airplanes and helicopters, but also boats, boats, submarine-like underwater moving devices, or ground mobiles whose balance is important as is obvious for bicycles, and also for robots etc. The idea of the present application can be applied, and it may be rephrased as an invention in such a field. Further, it goes without saying that the moving body and the photographing device are concepts including a control unit for controlling each.

  The present invention is not limited to the above embodiments as they are, and can be embodied by modifying the constituent elements within a range not departing from the gist of the invention in an implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in each of the above embodiments. For example, some of all the constituent elements shown in the embodiment may be deleted. Furthermore, the constituent elements of different embodiments may be combined appropriately.

  It should be noted that even if the description of the claims, the description, and the operation flow in the drawings is made by using “first,” “next,” and the like for convenience, it is essential that they are performed in this order. It does not mean. Further, it goes without saying that the steps constituting these operation flows can be appropriately omitted as long as they do not affect the essence of the invention.

  Of the techniques described here, the control mainly described in the flowchart is often settable by a program and may be stored in a recording medium or a recording unit. The recording medium and the recording unit may be recorded at the time of product shipment, may be distributed recording medium, or may be downloaded via the Internet.

    10 ... Mobile photographing device, 10a ... Moving body, 10b ... Imaging part, 11 ... Control part, 12a ... Shooting control part, 13f ... Permission judging part, 14 ... Propulsion part, 18 ... Built-in camera.

Claims (11)

A photography device that can be attached to a moving body,
An imaging unit that can shoot in different shooting conditions,
A control unit that controls the change of the photographing state with the change of the center of gravity, based on the determination result of whether the posture of the moving body has reached a predetermined state change permission posture,
An imaging device comprising: The imaging device according to claim 1, further comprising a communication unit that receives the information on the state change permission posture transmitted by the mobile body. The photographing apparatus according to claim 1, wherein the change of the photographing state occurs based on a movement of a movable portion. The imaging device according to claim 3, wherein the movable portion is an optical system. A moving body,
An imaging unit attached to the moving body and capable of shooting in different shooting states,
A control unit that controls the change of the photographing state with the change of the center of gravity, based on the determination result of whether the posture of the moving body has reached a predetermined state change permission posture,
A mobile image capturing apparatus comprising: In the state change permission posture, the line segment connecting the center of gravity of the moving body and the center of gravity of the image capturing unit is in the vertical direction, and the direction of change of the center of gravity of the image capturing unit due to the change in the image capturing state is in the vertical direction. The mobile photographing apparatus according to claim 5, wherein the mobile photographing apparatus is in a posture. The mobile photographing apparatus according to claim 5, further comprising a built-in camera attached to the moving body. The mobile image capturing apparatus according to claim 7, wherein the built-in camera is capable of capturing an image in a wider angle than the image capturing unit. A moving body for shooting to which a shooting device is attached,
A posture determination unit that determines the posture with respect to the vertical direction,
A control unit for controlling the change of the photographing state with the change of the center of gravity of the photographing device, based on whether or not the posture judged by the posture judging unit is a predetermined state change permission posture;
A moving body for photography, comprising: Photographing the moving body according to the determination result of the upper Symbol controller to claim 9, characterized by comprising a transmission unit that transmits to the imaging device. A first communication unit that receives a posture determination result from a moving body;
A second communication unit that transmits information for controlling a shooting state of a shooting device attached to the moving body;
Only when the determination result received by the first communication unit indicates that the moving body is in the predetermined state change permission posture, the change permission of the photographing state with the change of the center of gravity of the photographing device is determined, A control unit that controls the imaging device via the second communication unit;
An imaging control device for a mobile body, comprising: