JP2019003035A - Controller, imaging apparatus, imaging system, flight object, control method, and program - Google Patents

Abstract

To solve a problem such that an imaging apparatus sometimes cannot properly correct a swing accompanied by drive of a rotary vane.SOLUTION: A controller controls swing correction of an imaging apparatus mounted on a flight object having a rotary vane and an electric motor for rotating the rotary vane. The controller may acquire a first acquisition unit for acquiring movement information of the imaging apparatus from a first sensor for detecting movement of the imaging apparatus. The controller may include a first control unit for controlling swing correction of the imaging apparatus based on the movement information and rotation speed of the electric motor or a current value of a current input into the electric motor.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a control device, an imaging device, an imaging system, a flying object, a control method, and a program.

In some cases, the imaging apparatus has a shake correction function in order to prevent the image from blurring due to so-called camera shake. Patent Document 1 describes correcting blurring of a video by specifying a blurring position of an image using blur angle information from a Gyro value and returning the video data at that position to the original position. ing. Japanese Patent Application Laid-Open No. 2004-228561 describes that blur correction control that is different from normal is executed when the usage status when the mirror unit and the shutter unit are driven matches or is similar to a specific usage status.
Patent Document 1 Japanese Patent Application Laid-Open No. 2006-208483 Patent Document 2 Japanese Patent Application Laid-Open No. 2015-34904

  An imaging device having a shake correction function may be mounted on a flying object including a rotary wing such as an unmanned aerial vehicle (UAV). An imaging device mounted on such a flying object may vibrate as the rotor blades are driven. However, the imaging apparatus may not be able to properly perform shake correction associated with the driving of the rotor blades.

  A control device according to an aspect of the present invention controls shake correction of an imaging device mounted on a flying object including a rotating wing and an electric motor that rotates the rotating wing. The control device may include a first acquisition unit that acquires motion information of the imaging device from a first sensor that detects the motion of the imaging device. The control device may include a first control unit that controls shake correction of the imaging device based on the motion information and the rotation speed of the motor or the current value of the current input to the motor.

  The first control unit may specify the rotation speed or the current value based on control information for controlling the flight of the flying object.

  The first control unit may specify the rotation speed based on position information of the rotor of the electric motor.

  The flying body may include a plurality of rotor blades and a plurality of electric motors that respectively rotate the plurality of rotor blades. The first control unit may control shake correction of the imaging device based on the motion information and the current values of the rotation speeds of the plurality of motors or the currents input to the plurality of motors.

  The first control unit drives at least one of the lens and the image sensor included in the imaging device based on the motion information of the imaging device and the rotation speed of the electric motor or the current value input to the electric motor. The correction may be controlled.

  The first control unit may include a deriving unit that derives driving information for controlling driving of at least one of the lens and the image sensor based on the motion information. The first control unit may include a generation unit that generates a correction value based on a rotation speed of the electric motor or a current value input to the electric motor. The first control unit may include a correction unit that corrects the drive information based on the correction value. The first control unit may control shake correction of the imaging device by driving at least one of the lens and the image sensor based on the drive information corrected by the correction unit.

  The control device includes a storage unit that stores the rotation speed of the motor or the current value input to the motor and correction information for correcting driving information for controlling driving of at least one of the lens and the image sensor in association with each other. It's okay. The generation unit may generate a correction value based on correction information corresponding to a rotation speed of the electric motor or a current value input to the electric motor.

  The generation unit may generate a correction value further based on attitude information indicating the attitude of the flying object.

  The control device may include a second acquisition unit that acquires position information of the rotor of the electric motor from a second sensor that detects the position of the rotor of the electric motor. The generation unit may generate a correction value based further on the position information.

  The control device may include a third acquisition unit that acquires movement information of at least one of the lens and the image sensor from a third sensor that detects movement of at least one of the lens and the image sensor. The control device controls the correction of the image captured by the image sensor on the basis of the comparison between the drive information derived by the deriving unit and the motion information acquired by the third acquisition unit. You may provide the 2nd control part which controls correction | amendment.

  An imaging device according to one embodiment of the present invention may include the control device. The imaging device may include a lens. The imaging device may include an image sensor. The imaging apparatus may include a drive unit that drives at least one of the lens and the image sensor.

  The imaging system which concerns on 1 aspect of this invention may be provided with the said imaging device. The imaging system may include a support mechanism that supports the imaging device so that the posture of the imaging device can be changed.

  An aircraft according to an aspect of the present invention may include the imaging system. The flying body may include a rotating wing. The flying body may include an electric motor.

  A control method according to an aspect of the present invention controls shake correction of an imaging device mounted on an aircraft including a rotating wing and an electric motor that rotates the rotating wing. The control method may include a step of acquiring motion information of the imaging device from a first sensor that detects the motion of the imaging device. The control method may include a step of controlling shake correction of the imaging device based on the motion information and the rotation speed of the motor or the current value of the current input to the motor.

  A program according to an aspect of the present invention causes a computer to control shake correction of an imaging device mounted on an aircraft including a rotating wing and an electric motor that rotates the rotating wing. The program may cause the computer to execute a step of acquiring motion information of the imaging device from a first sensor that detects the motion of the imaging device. The program may cause the computer to execute a step of controlling shake correction of the imaging device based on the motion information and the rotation speed of the motor or the current value of the current input to the motor.

  According to one aspect of the present invention, it is possible to more appropriately perform shake correction associated with driving of a rotor blade.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure which shows an example of the external appearance of an unmanned aircraft and a remote control device. It is a figure which shows an example of the functional block of an unmanned aircraft. It is a figure which shows an example of the relationship between a vibration frequency and a tracking rate. It is a figure for demonstrating the error between a target value and a detected value. It is a figure which shows an example of correction information. It is a figure which shows an example of the functional block regarding an optical shake correction. It is a figure for demonstrating the shake of the horizontal direction of an image. It is a figure for demonstrating the shake of the horizontal direction of an image. It is a figure which shows an example of a hardware constitutions.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  The claims, the description, the drawings, and the abstract include matters subject to copyright protection. The copyright owner will not object to any number of copies of these documents as they appear in the JPO file or record. However, in other cases, all copyrights are reserved.

  Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where a block is either (1) a stage in a process in which an operation is performed or (2) an apparatus responsible for performing the operation. May represent a “part”. Certain stages and “units” may be implemented by programmable circuits and / or processors. Dedicated circuitry may include digital and / or analog hardware circuitry. Integrated circuits (ICs) and / or discrete circuits may be included. The programmable circuit may include a reconfigurable hardware circuit. Reconfigurable hardware circuits include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGA), programmable logic arrays (PLA), etc. The memory element or the like may be included.

  Computer-readable media may include any tangible device that can store instructions executed by a suitable device. As a result, a computer readable medium having instructions stored thereon comprises a product that includes instructions that can be executed to create a means for performing the operations specified in the flowcharts or block diagrams. Examples of computer readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically erasable programmable read only memory (EEPROM), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, memory stick, integrated A circuit card or the like may be included.

  The computer readable instructions may include either source code or object code written in any combination of one or more programming languages. The source code or object code includes a conventional procedural programming language. Conventional procedural programming languages include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or Smalltalk, JAVA, C ++, etc. It may be an object-oriented programming language and a “C” programming language or a similar programming language. Computer readable instructions may be directed to a general purpose computer, special purpose computer, or other programmable data processing device processor or programmable circuit locally or in a wide area network (WAN) such as a local area network (LAN), the Internet, etc. ). The processor or programmable circuit may execute computer readable instructions to create a means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

  FIG. 1 shows an example of the external appearance of an unmanned aerial vehicle (UAV) 10 and a remote control device 300. The UAV 10 includes a UAV main body 20, a gimbal 50, a plurality of imaging devices 60, and an imaging device 100. The gimbal 50 and the imaging device 100 are an example of an imaging system. The UAV 10 is an example of a flying object that flies by a rotary wing. The flying object is a concept including UAV, other aircraft moving in the air, and the like.

  The UAV main body 20 includes a plurality of rotor blades. The UAV main body 20 causes the UAV 10 to fly by controlling the rotation of a plurality of rotor blades. For example, the UAV main body 20 causes the UAV 10 to fly using four rotary wings. The number of rotor blades is not limited to four.

  The imaging device 100 is an imaging camera that images a subject included in a desired imaging range. The gimbal 50 supports the imaging device 100 so that the posture of the imaging device 100 can be changed. The gimbal 50 supports the imaging device 100 in a rotatable manner. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 supports the imaging device 100 so as to be rotatable about the pitch axis using an actuator. The gimbal 50 further supports the imaging apparatus 100 using an actuator so as to be rotatable about the roll axis and the yaw axis. The gimbal 50 may change the posture of the imaging device 100 by rotating the imaging device 100 about at least one of the yaw axis, the pitch axis, and the roll axis.

  The plurality of imaging devices 60 are sensing cameras that image the surroundings of the UAV 10 in order to control the flight of the UAV 10. Two imaging devices 60 may be provided in the front which is the nose of UAV10. Two other imaging devices 60 may be provided on the bottom surface of the UAV 10. The two imaging devices 60 on the front side may be paired and function as a so-called stereo camera. The two imaging devices 60 on the bottom side may also be paired and function as a stereo camera. Based on images picked up by a plurality of image pickup devices 60, three-dimensional spatial data around the UAV 10 may be generated. The number of imaging devices 60 included in the UAV 10 is not limited to four. The UAV 10 only needs to include at least one imaging device 60. The UAV 10 may include at least one imaging device 60 on each of the nose, the tail, the side surface, the bottom surface, and the ceiling surface of the UAV 10. The angle of view that can be set by the imaging device 60 may be wider than the angle of view that can be set by the imaging device 100. That is, the imaging range of the imaging device 60 may be wider than the imaging range of the imaging device 100. The imaging device 60 may have a single focus lens or a fisheye lens.

  The remote operation device 300 communicates with the UAV 10 to remotely operate the UAV 10. The remote operation device 300 may communicate with the UAV 10 wirelessly. The remote control device 300 transmits various drive commands related to the movement of the UAV 10 such as ascending, descending, accelerating, decelerating, moving forward, backward, and rotating to the UAV 10.

  FIG. 2 shows an example of functional blocks of the UAV 10. The UAV 10 includes a UAV control unit 30, a memory 32, a communication interface 34, an electric motor 36, a hall sensor 38, a rotor blade 40, a GPS receiver 41, an inertial measurement device 42, a magnetic compass 43, a barometric altimeter 44, a gimbal 50, and an imaging device 60. And an imaging device 100.

  The communication interface 34 communicates with other devices such as the remote operation device 300. The communication interface 34 may receive instruction information including various commands for the UAV control unit 30 from the remote operation device 300. In the memory 32, the UAV control unit 30 controls the electric motor 36, the GPS receiver 41, the inertial measurement device (IMU) 42, the magnetic compass 43, the barometric altimeter 44, the gimbal 50, the imaging device 60, the imaging device 100, and the like. Stores necessary programs etc. The memory 32 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 32 may be provided inside the UAV main body 20. It may be provided so as to be removable from the UAV main body 20.

  The UAV control unit 30 controls the flight and imaging of the UAV 10 according to a program stored in the memory 32. The UAV control unit 30 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The UAV control unit 30 controls the flight and imaging of the UAV 10 according to a command received from the remote control device 300 via the communication interface 34. The electric motor 36 is driven according to the control information from the UAV control unit 30 and rotates the rotor blade 40 to fly the UAV 10. The electric motor 36 is, for example, a brushless motor having a three-phase stator and a rotor.

  The UAV control unit 30 may specify the environment around the UAV 10 by analyzing a plurality of images captured by the plurality of sensing imaging devices 60. The UAV control unit 30 controls the flight while avoiding obstacles, for example, based on the environment around the UAV 10. The UAV control unit 30 may generate three-dimensional spatial data around the UAV 10 based on a plurality of images captured by the plurality of imaging devices 60, and control the flight based on the three-dimensional spatial data.

  The hall sensor 38 detects the position of the rotor included in the electric motor 36. The hall sensor 38 is an example of a second sensor. The GPS receiver 41 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 41 calculates the position of the GPS receiver 41, that is, the position of the UAV 10 based on the received signals. The IMU 42 detects the posture of the UAV 10. The IMU 42 detects, as the posture of the UAV 10, acceleration in the three axial directions of the front, rear, left, and right of the UAV 10, and angular velocity in the three axial directions of pitch, roll, and yaw. The magnetic compass 43 detects the heading of the UAV 10. The barometric altimeter 44 detects the altitude at which the UAV 10 flies. The barometric altimeter 44 detects the atmospheric pressure around the UAV 10, converts the detected atmospheric pressure into an altitude, and detects the altitude.

  The imaging device 100 includes an imaging unit 102 and a lens unit 200. The lens unit 200 is an example of a lens device. The imaging unit 102 includes an image sensor 120, an imaging control unit 110, a gyro sensor 116, and a memory 130. The image sensor 120 may be configured by a CCD or a CMOS. The image sensor 120 outputs image data of an optical image formed through the plurality of lenses 210 to the imaging control unit 110. The gyro sensor 116 detects the movement of the imaging device 100. The gyro sensor 116 detects the movement (shake) of the imaging apparatus 100 in the pitch direction and the yaw direction. The gyro sensor 116 may detect movement in the roll direction. The gyro sensor 116 may detect movement (shake) in the vertical direction (pitch direction) and the horizontal direction (yaw direction) of the imaging device 100 in a normal posture in which the optical axis of the imaging device 100 is oriented in the horizontal direction. . The gyro sensor 116 is an example of a first sensor. The imaging control unit 110 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The imaging control unit 110 may control the imaging device 100 in accordance with an operation command for the imaging device 100 from the UAV control unit 30. The imaging control unit 110 executes at least one of optical shake correction and electronic shake correction.

  The memory 130 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 130 stores a program and the like necessary for the imaging control unit 110 to control the image sensor 120 and the like. The memory 130 may be provided inside the housing of the imaging device 100. The memory 130 may be provided so as to be removable from the housing of the imaging apparatus 100.

  The lens unit 200 includes a plurality of lenses 210, a lens driving unit 212, a position sensor 214, and a lens control unit 220. The plurality of lenses 210 may function as a zoom lens, a varifocal lens, and a focus lens. At least one of the plurality of lenses 210 is an optical shake correction lens. At least some or all of the plurality of lenses 210 are arranged to be movable along the optical axis. The lens unit 200 may be an interchangeable lens that is detachably attached to the imaging unit 102. The lens driving unit 212 moves the lens 210 along the optical axis. At least one lens driving unit 212 among the plurality of lens driving units 212 may move the lens 210 in a direction perpendicular to the optical axis in order to realize optical shake correction. The at least one lens driving unit 212 may move the lens 210 in at least one of the pit direction and the yaw direction in order to realize optical shake correction. The lens driving unit 212 is an example of a driving unit. The position sensor 214 detects the position of the lens 210. The position sensor 214 is an example of a third sensor that detects the movement of the lens 210. The lens control unit 220 drives the at least one lens driving unit 212 according to the lens driving information from the imaging unit 102 and moves the at least one lens 210 along the optical axis direction or a direction perpendicular to the optical axis. . The drive information includes, for example, a zoom control command, a focus control command, and a shake correction control command. In this embodiment, an example in which the lens 210 is driven as optical shake correction will be described. However, the imaging apparatus 100 may drive the image sensor 120 as optical shake correction. Alternatively, the imaging apparatus 100 may drive the lens 210 and the image sensor 120 as optical shake correction.

  The imaging control unit 110 includes an optical shake correction control unit 140 and an electronic shake correction control unit 150. The optical shake correction control unit 140 controls optical shake correction. The electronic shake correction control unit 150 controls electronic shake correction. The optical shake correction control unit 140 is an example of a first control unit. The electronic shake correction control unit 150 is an example of a second control unit.

  The optical shake correction control unit 140 performs shake correction based on the movement information of the imaging device 100 from the gyro sensor 116. The optical shake correction control unit 140 derives the shake amounts of the imaging device 100 in the pitch direction and the yaw direction based on the motion information of the imaging device 100 from the gyro sensor 116. The optical shake correction control unit 140 generates drive information for driving the lens 210 in a direction to cancel the derived shake amount. The optical shake correction control unit 140 controls the lens drive unit 212 based on the drive information to drive the lens 210 in a direction in which the shake amount is canceled. The optical shake correction control unit 140 may feedback control the movement of the lens 210 based on the movement information of the lens 210 from the position sensor 214 and the driving information of the lens 210.

  The lens driving unit 212 drives the lens 210 based on the driving information. However, an error may occur between the target movement amount that the lens 210 should move according to the drive information and the actual movement amount. FIG. 3 shows an example of the relationship between the vibration frequency of the vibration generated in the imaging apparatus 100 and the tracking rate (detected value d / target value c) of the lens 210. The target value c indicates the target movement amount of the lens 210. The detection value d indicates the actual movement amount of the lens 210 detected by the position sensor 214. The closer the tracking rate is to 1, the more the lens 210 is moved to an ideal position. That is, the closer the tracking rate is to 1, the more accurately the shake correction is performed.

  The vibration frequency of the vibration of the imaging device 100 accompanying the rotation of the rotary blade 40 depends on the characteristics of the electric motor 36 and the rotary blade 40, but is in the vicinity of 50 Hz to 70 Hz, for example. As shown in FIG. 3, the tracking rate is close to 1 when the vibration frequency of the imaging apparatus 100 is around 10 Hz. On the other hand, when the vibration frequency of the imaging device 100 is in the vicinity of 50 Hz to 70 Hz, the follow-up rate deviates from 1 due to the influence of the characteristics of the lens driving unit 212 and the like. That is, when the vibration frequency of the imaging apparatus 100 is in the vicinity of 50 Hz to 70 Hz, an error occurs in the movement of the lens 210 based on the drive information, and the shake correction is not accurately performed. For example, the vibration frequency of normal shake correction that is likely to occur when the user takes an image with the imaging device 100 in his / her hand without being mounted on the UAV 10 is around 10 Hz. At such a vibration frequency, the follow-up rate is close to 1, which is not a problem. However, when the imaging apparatus 100 is mounted on the UAV 10, vibrations having a vibration frequency in the vicinity of 50 Hz to 70 Hz may be generated in the imaging apparatus 100 as the rotating blades 40 rotate. In this case, an error may occur between the target value c and the actual detection value d of the lens 210, and shake correction may not be performed accurately.

  Therefore, when the imaging apparatus 100 according to the present embodiment vibrates at a specific vibration frequency, the driving information is corrected so as to cancel the error of the movement of the lens 210 based on the driving information. Thereby, even when the vibration frequency of the imaging apparatus 100 is in the vicinity of 50 Hz to 70 Hz, the tracking rate is brought close to 1. The specific vibration frequency may be a vibration frequency whose follow-up rate is lower than a predetermined threshold.

  As shown in FIG. 4, between (1) the target value c indicating the amount of movement of the lens 210 and (2) the detected value d indicating the actual amount of movement of the lens 210 detected by the position sensor 214. (3) An error (cd) occurs. The optical shake correction control unit 140 corrects the driving information of the lens 210 with (4) the correction value Δ in order to cancel the error. As a result, (5) error (cd) is canceled.

  The imaging control unit 110 includes an acquisition unit 112. The acquisition unit 112 acquires motion information of the imaging device 100 from the gyro sensor 116. The acquisition unit 112 may acquire the angular velocity in the pitch direction and the angular velocity in the yaw direction of the imaging device 100 as motion information of the imaging device 100 from the gyro sensor 116. The acquisition unit 112 may acquire the position information of the rotor of the electric motor 36 from the hall sensor 38 via the UAV control unit 30. The acquisition unit 112 may acquire movement information of the lens 210 that functions as a shake correction lens from the position sensor 214. The acquisition unit 112 is an example of a first acquisition unit, a second acquisition unit, and a third acquisition unit.

  The optical shake correction control unit 140 includes a derivation unit 142, a generation unit 144, and a correction unit 146. The deriving unit 142 derives driving information for controlling the driving of the lens 210 based on the motion information of the imaging apparatus 100. The deriving unit 142 may derive the shake amount in the pit direction and the yaw direction of the imaging device 100 by integrating the angular velocity in the pitch direction and the angular velocity in the yaw direction of the imaging device 100, respectively. The deriving unit 142 may derive driving information for driving the lens 210 in a direction in which the shake amount is canceled.

  The generation unit 144 generates the correction value Δ based on the rotation speed of the electric motor 36 or the current value input to the electric motor 36. The generation unit 144 may acquire control information for controlling the flight of the UAV 10 from the UAV control unit 30 and specify the rotational speed of the electric motor 36 or the current value input to the electric motor 36 based on the control information. The generation unit 144 acquires control information indicating a drive command for driving the electric motor 36 from the UAV control unit 30, and specifies a rotation speed of the electric motor 36 or a current value input to the electric motor 36 based on the control information. It's okay. The generation unit 144 may acquire the position information of the rotor of the electric motor 36 from the hall sensor 38 via the UAV control unit 30 and specify the rotation speed of the electric motor 36 based on the position information of the rotor. The generation unit 144 may specify the current value of the current input to the electric motor 36 based on a signal from a current sensor that detects the current input to the electric motor 36.

  The memory 130 may store the rotational speed of the electric motor 36 or the current value input to the electric motor 36 and correction information for correcting the driving information of the lens 210 in association with each other. The memory 130 is an example of a storage unit. For example, as illustrated in FIG. 5, the memory 130 may store a correction pattern defined by a frequency and an amplitude and a rotation speed (rad / s) of the electric motor 36 in association with each other as correction information. For each rotation speed of the electric motor 36, the difference between the target value c and the detection value d is actually measured, and correction information is generated in advance from the difference and stored in the memory 130.

  The generation unit 144 may generate the correction value Δ based on the correction information corresponding to the rotation speed of the electric motor 36 or the current value input to the electric motor 36. The generation unit 144 may read a correction pattern corresponding to the rotation speed of the electric motor 36 from the memory 130 and generate the correction value Δ based on the correction pattern. The generation unit 144 acquires the position information of the rotor of the electric motor 36 from the Hall sensor 38, synchronizes the driving information of the lens 210 and the correction pattern based on the position information of the rotor, and corrects the phase and correction of the target value c. The correction value Δ may be generated so that the phase of the value Δ matches.

  The direction of vibration of the imaging device 100 accompanying the rotation of the electric motor 36 differs depending on the orientation of the imaging device 100. Therefore, the generation unit 144 may generate the correction value Δ in consideration of the attitude of the imaging device 100. The memory 130 may store a correction pattern for each posture of the imaging device 100. The memory 130 may store a correction pattern for each angle range indicating the attitude of the imaging device 100. The angle range may be defined, for example, as an angle of 0 degrees when the imaging apparatus 100 is oriented in the horizontal direction.

  The correcting unit 146 corrects the driving information derived by the deriving unit 142 based on the correction value Δ generated by the generating unit 144. The optical shake correction control unit 140 drives the lens 210 via the lens drive unit 212 based on the drive information corrected by the correction unit 146, and executes shake correction of the imaging apparatus 100. The correction unit 146 may sequentially correct the drive information by sequentially adding the correction value Δ generated by the generation unit 144 to the target value c indicated by the drive information derived by the derivation unit 142.

  The UAV 10 includes a plurality of rotor blades 40 and a plurality of electric motors 36 that respectively rotate the plurality of rotor blades 40. The influence on the vibration of the imaging device 100 differs for each of the plurality of electric motors 36. Therefore, the generation unit 144 may generate a correction value Δ for each of the plurality of electric motors 36. The correction unit 146 may correct the drive information based on a plurality of correction values Δ corresponding to the plurality of electric motors 36. The correction unit 146 may correct the drive information by merging the correction values Δ of the plurality of electric motors 36 and adding the merged correction value to the target value indicated in the drive information. The memory 130 may store a table as shown in FIG.

  FIG. 6 shows an example of functional blocks related to optical shake correction. The acquisition unit 112 acquires motion information of the imaging device 100 from the gyro sensor 116. The deriving unit 142 derives the shake amount of the imaging device 100 based on the motion information of the imaging device 100 provided from the acquisition unit 112. The deriving unit 142 derives driving information of the lens 210 so as to drive the lens 210 so as to cancel the shake amount. On the other hand, the generation unit 144 acquires the rotation speed of the electric motor 36 via the UAV control unit 30. The generation unit 144 acquires correction information corresponding to the rotation speed of the electric motor 36 from the memory 130. The generation unit 144 may acquire posture information of the imaging device 100 from the gyro sensor 116 and may acquire correction information corresponding to the posture of the imaging device 100 and the rotation speed of the electric motor 36 from the memory 130. Further, the generation unit 144 acquires position information of the rotor of the electric motor 36 from the hall sensor 38. And the production | generation part 144 produces | generates correction value (DELTA) according to the position of the rotor of the electric motor 36 based on correction information. The generation unit 144 generates a correction value Δ synchronized with the rotation period of the electric motor 36. The correction unit 146 corrects the drive information based on the drive information provided from the derivation unit 142 and the correction value Δ provided from the generation unit 144. The correcting unit 146 provides the corrected driving information to the lens driving unit 212. The lens driving unit 212 drives the lens 210 in a direction to cancel the shake amount of the imaging apparatus 100 based on the corrected driving information. The deriving unit 142 may acquire the movement information of the lens 210 from the position sensor 214 and may perform a drive information feedback process based on the movement information of the lens 210.

  Here, even if the drive information is corrected based on the correction value Δ, the lens 210 may not move to the target value c due to an external factor such as a difference in the surrounding environment of the imaging apparatus 100. That is, even if the drive information is corrected based on the correction value Δ, an error may occur between the target value c and the detection value d. Therefore, in order to further eliminate such an error, the electronic shake correction control unit 150 may control the electronic shake correction.

  For example, when the image sensor 120 is configured with a CMOS sensor, an image acquired from the image sensor 120 is captured for each horizontal line. For example, an image acquired from the image sensor 120 is captured for each of the horizontal lines L1 to L6 as shown in FIG. The number of horizontal lines of the image is not limited to the number shown in FIG. For example, when the shake direction of the imaging apparatus 100 is the horizontal direction, the image is blurred in units of horizontal lines as shown in FIG. Even if the drive information is corrected by the correction value Δ, horizontal shake may remain as shown in FIG. In such a case, the electronic shake correction control unit 150 may correct the remaining horizontal shake by image processing.

  The electronic shake correction control unit 150 captures an image by the image sensor 120 based on a comparison between the drive information derived by the deriving unit 142 and the motion information of the lens 210 that functions as the shake correction lens acquired by the acquisition unit 112. The shake correction of the imaging apparatus 100 may be controlled by controlling the correction of the image that has been performed. For each timing at which an image is captured in units of horizontal lines, the electronic shake correction control unit 150 includes a horizontal target value c indicated by driving information of the lens 210 corresponding to the timing, and a position sensor 214 corresponding to the timing. The difference from the acquired horizontal detection value d is acquired. The electronic shake correction control unit 150 stores the difference in the memory 130 for each horizontal line. Then, the electronic shake correction control unit 150 corrects the image acquired from the image sensor 120 for each horizontal line based on the difference in order to cancel the difference for each horizontal line. Accordingly, the electronic shake correction control unit 150 corrects, for example, an image blurred in the horizontal direction as shown in FIG. 8 into an image with little horizontal shake as shown in FIG.

  As described above, according to the imaging apparatus 100 according to the present embodiment, it is possible to prevent the shake correction from being performed accurately when the imaging apparatus 100 captures an image while the UAV 10 is flying. For example, in addition to the case where the imaging apparatus 100 is mounted on the UAV 10 and used, the shake correction is accurately performed even when the imaging apparatus 100 is used without being mounted on the UAV 10 such as being held by a user. Can be done.

  FIG. 9 illustrates an example computer 1200 in which aspects of the present invention may be embodied in whole or in part. A program installed in the computer 1200 can cause the computer 1200 to function as an operation associated with the apparatus according to the embodiment of the present invention or as one or more “units” of the apparatus. Alternatively, the program can cause the computer 1200 to execute the operation or the one or more “units”. The program can cause the computer 1200 to execute a process according to an embodiment of the present invention or a stage of the process. Such a program may be executed by CPU 1212 to cause computer 1200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

  A computer 1200 according to this embodiment includes a CPU 1212 and a RAM 1214, which are connected to each other by a host controller 1210. The computer 1200 also includes a communication interface 1222 and an input / output unit, which are connected to the host controller 1210 via the input / output controller 1220. Computer 1200 also includes ROM 1230. The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit.

  The communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores therein a boot program executed by the computer 1200 at the time of activation and / or a program depending on the hardware of the computer 1200. The program is provided via a computer-readable recording medium such as a CR-ROM, a USB memory, or an IC card or a network. The program is installed in the RAM 1214 or the ROM 1230 that is also an example of a computer-readable recording medium, and is executed by the CPU 1212. Information processing described in these programs is read by the computer 1200 to bring about cooperation between the programs and the various types of hardware resources. An apparatus or method may be configured by implementing information operations or processing in accordance with the use of computer 1200.

  For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214 and performs communication processing on the communication interface 1222 based on the processing described in the communication program. You may order. The communication interface 1222 reads transmission data stored in a RAM 1214 or a transmission buffer area provided in a recording medium such as a USB memory under the control of the CPU 1212 and transmits the read transmission data to a network, or The reception data received from the network is written into a reception buffer area provided on the recording medium.

  In addition, the CPU 1212 allows the RAM 1214 to read all or necessary portions of a file or database stored in an external recording medium such as a USB memory, and executes various types of processing on the data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to an external recording medium.

  Various types of information, such as various types of programs, data, tables, and databases, may be stored on a recording medium and subjected to information processing. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval that are described throughout the present disclosure for data read from the RAM 1214 and specified by the instruction sequence of the program. Various types of processing may be performed, including / replacement, etc., and the result is written back to RAM 1214. In addition, the CPU 1212 may search for information in files, databases, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. The entry that matches the condition is searched from the plurality of entries, the attribute value of the second attribute stored in the entry is read, and thereby the first attribute that satisfies the predetermined condition is associated. The attribute value of the obtained second attribute may be acquired.

  The programs or software modules described above may be stored on a computer-readable storage medium on or near the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the program is transferred to the computer 1200 via the network. provide.

  The execution order of each process such as operation, procedure, step, and stage in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if it is described using “first”, “next”, etc. for the sake of convenience, it means that it is essential to carry out in this order. It is not a thing.

10 UAV
20 UAV body 30 UAV control unit 32 Memory 34 Communication interface 36 Electric motor 38 Hall sensor 40 Rotor blade 41 GPS receiver 42 Inertial measurement device 43 Magnetic compass 44 Barometric altimeter 50 Gimbal 60 Imaging device 100 Imaging device 102 Imaging unit 110 Imaging control unit 112 Acquisition unit 116 Gyro sensor 120 Image sensor 130 Memory 140 Optical shake correction control unit 142 Derivation unit 144 Generation unit 146 Correction unit 150 Electronic shake correction control unit 200 Lens unit 210 Lens 212 Lens drive unit 214 Position sensor 220 Lens control unit 300 Remote operation device 1200 Computer 1210 Host controller 1212 CPU
1214 RAM
1220 Input / output controller 1222 Communication interface 1230 ROM

Claims (15)

A control device that controls shake correction of an imaging device mounted on a flying object including a rotary wing and an electric motor that rotates the rotary wing,
A first acquisition unit that acquires movement information of the imaging device from a first sensor that detects the movement of the imaging device;
A control device comprising: a first control unit that controls shake correction of the imaging device based on the motion information and a rotation speed of the motor or a current value of a current input to the motor.   The control device according to claim 1, wherein the first control unit specifies the rotation speed or the current value based on control information for controlling flight of the flying object.   The control device according to claim 1, wherein the first control unit specifies the rotation speed based on position information of a rotor of the electric motor. The flying object includes a plurality of the rotor blades and a plurality of the electric motors that rotate the plurality of rotor blades, respectively.
The first control unit controls shake correction of the imaging device based on the motion information and current values of rotation speeds of the plurality of electric motors or currents input to the plurality of electric motors; The control device according to claim 1.   The first control unit drives at least one of a lens and an image sensor included in the imaging device based on movement information of the imaging device and a rotation speed of the electric motor or a current value input to the electric motor. The control device according to claim 1, wherein shake control of the imaging device is controlled. The first controller is
A deriving unit for deriving driving information for controlling driving of at least one of the lens and the image sensor based on the motion information;
A generating unit that generates a correction value based on a rotation speed of the electric motor or a current value input to the electric motor;
A correction unit that corrects the drive information based on the correction value;
The first control unit controls shake correction of the imaging device by driving at least one of the lens and the image sensor based on the drive information corrected by the correction unit. Control device. A storage unit that stores the rotation speed of the motor or a current value input to the motor and correction information for correcting drive information for controlling driving of at least one of the lens and the image sensor in association with each other; Prepared,
The control device according to claim 6, wherein the generation unit generates the correction value based on correction information corresponding to a rotation speed of the electric motor or a current value input to the electric motor.   The control device according to claim 6, wherein the generation unit generates the correction value further based on posture information indicating a posture of the flying object. A second acquisition unit that acquires position information of the rotor of the electric motor from a second sensor that detects a position of the rotor of the electric motor;
The control device according to claim 6, wherein the generation unit generates the correction value further based on the position information. A third acquisition unit that acquires movement information of at least one of the lens and the image sensor from a third sensor that detects movement of at least one of the lens and the image sensor;
By controlling correction of an image captured by the image sensor based on a comparison between the driving information derived by the deriving unit and the motion information acquired by the third acquiring unit, the imaging apparatus The control device according to claim 6, further comprising: a second control unit that controls the shake correction of the image. A control device according to any one of claims 1 to 10,
A lens,
An imaging apparatus comprising: an image sensor that captures an image formed by the lens; and a drive unit that drives at least one of the lens and the image sensor. An imaging device according to claim 11;
An imaging system comprising: a support mechanism that supports the imaging device so that the posture of the imaging device can be changed. An imaging system according to claim 12,
The rotor blade;
An aircraft including the electric motor. A control method for controlling shake correction of an imaging device mounted on a flying object including a rotary wing and an electric motor that rotates the rotary wing,
Obtaining movement information of the imaging device from a first sensor that detects the movement of the imaging device;
And a step of controlling shake correction of the imaging device based on the motion information and a rotation speed of the motor or a current value of a current input to the motor. A program for causing a computer to control shake correction of an imaging device mounted on a flying object including a rotary wing and an electric motor that rotates the rotary wing,
Obtaining movement information of the imaging device from a first sensor that detects the movement of the imaging device;
A program for causing the computer to execute the step of controlling shake correction of the imaging device based on the motion information and a rotation speed of the motor or a current value of a current input to the motor.

Images