JP2019066556A - Control device, imaging apparatus, flying body, control method, and program - Google Patents

Abstract

To provide a lens control device for preventing the accuracy of zoom tracking, even when a subject distance is changed.SOLUTION: The control device controls a lens device including a zoom lens and a focus lens. The control device includes a first specification part which specifies the first change amount of a focus position for indicating the position of the focus lens, for maintaining a focusing state with respect to a subject whose distance from the lens device is not changed, when a zoom position for indicating the position of the zoom lens is changed and a second specification part which specifies the second change amount of the focus position, for maintaining the focusing state with respect to the subject whose distance from the lens device is changed.SELECTED DRAWING: Figure 10

Description

  The present invention relates to a control device, an imaging device, an aircraft, a control method, and a program.

Patent Document 1 discloses a lens device that performs zoom tracking control for moving a focus lens in order to correct a focus shift accompanying movement of a zoom lens.
Patent Document 1: Japanese Patent Application Laid-Open No. 2016-224096

  The above zoom tracking control assumes that the distance from the imaging device to the subject is constant. However, the distance may not always be constant, and the accuracy of zoom tracking may decrease.

  A control device according to an aspect of the present invention controls a lens device provided with a zoom lens and a focus lens. The control device is configured to maintain the in-focus state with respect to a subject whose distance from the lens apparatus does not change when the zoom position indicating the position of the zoom lens changes, the first change amount of the focus position indicating the position of the focus lens May be provided. The control device may include a second specifying unit that specifies a second change amount of the focus position for maintaining the in-focus state with respect to the subject whose distance from the lens device changes. The control device may include a control unit that controls the focus position based on the first change amount and the second change amount.

  In response to the command for moving the zoom lens from the first zoom position to the second zoom position, the first identification unit determines the first change amount when the zoom lens moves from the first zoom position to the second zoom position. You may specify. In response to the command, the second identification unit may identify the second change amount based on the plurality of images captured before the zoom lens moves to the second zoom position.

  The second identification unit is configured to calculate a second change amount based on the plurality of images including the first image captured when the zoom lens is at the first zoom position and the second image captured before the first image. You may specify

  The second identification unit identifies a first distance indicating the distance from the lens device to the subject based on the first image, and identifies a second distance indicating the distance from the lens device to the subject based on the second image, The second change amount may be specified based on the first distance and the second distance.

  The second identification unit may identify the second change amount based on the difference between the first distance and the second distance.

  The first identification unit is configured to set the first focusing unit based on the information indicating the relationship between the focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens device does not change and the zoom position The amount of change may be specified.

  The control unit may control the focus position based on the first change amount and the second change amount when the lens device does not satisfy a predetermined imaging condition. The control unit may control the focus position based on the first change amount, not based on the second change amount, when the lens device satisfies a predetermined imaging condition.

  The control unit controls the focus position based on the first change amount and the second change amount when the predetermined imaging condition that the distance from the lens device to the ground is equal to or more than the predetermined distance is not satisfied. Good. When the control unit satisfies the predetermined imaging condition that the distance from the lens device to the ground is equal to or more than the predetermined distance, the control unit does not calculate the focus position based on the first amount of change, not based on the second amount of change. You may control.

  When the distance from the lens device to the ground is equal to or greater than a predetermined distance and the imaging direction of the lens device does not satisfy a predetermined imaging condition that the component includes a component in a predetermined direction, the control unit The focus position may be controlled based on the first change amount and the second change amount. When the distance from the lens device to the ground is equal to or greater than a predetermined distance and the imaging direction of the lens device includes a component in a predetermined direction, the control unit satisfies the second imaging condition. The focus position may be controlled based on the first change amount, not based on the change amount. The predetermined direction may be vertical.

  The control unit controls the focus position based on the first change amount and the second change amount when the predetermined imaging condition that the distance from the lens device to the subject is equal to or more than the predetermined distance is not satisfied. Good. When the control unit satisfies the predetermined imaging condition that the distance from the lens device to the subject is equal to or more than the predetermined distance, the control unit is not based on the second change amount, but on the basis of the first change amount. You may control.

  The control unit may control the focus position based on the first change amount and the second change amount when the lens device does not satisfy a predetermined imaging condition that the lens device operates in a predetermined imaging mode. The control unit may control the focus position based on the first amount of change, not based on the second amount of change, when the imaging device satisfies a predetermined imaging condition that the lens device operates in a predetermined imaging mode. .

  The lens apparatus may be mounted to the aircraft. The control unit may control the focus position on the basis of the first change amount and the second change amount when the predetermined imaging condition that the flight body is in flight is not satisfied. The control unit may control the focus position based on the first amount of change, not based on the second amount of change, when a predetermined imaging condition that the aircraft is in flight is satisfied.

  An imaging device according to an aspect of the present invention includes the control device and a lens device.

  An aircraft according to an aspect of the present invention flies with the imaging device. The vehicle may include a support mechanism that rotatably supports the imaging device.

  A control method according to an aspect of the present invention controls a lens apparatus provided with a zoom lens and a focus lens. The control method is the first change amount of the focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens device does not change when the zoom position indicating the position of the zoom lens changes. May be provided. The control method may include identifying a second change amount of the focus position to maintain the in-focus state with respect to the subject whose distance from the lens device changes. The control method may include controlling the focus position based on the first change amount and the second change amount.

  A program according to an aspect of the present invention is a program for causing a computer to control a lens device provided with a zoom lens and a focus lens. The program changes the first change amount of the focus position indicating the position of the focus lens to maintain the in-focus state with respect to the subject whose distance from the lens apparatus does not change when the zoom position indicating the position of the zoom lens changes. The computer may execute the identifying step. The program may cause the computer to execute a step of identifying a second amount of change of the focus position in order to maintain the in-focus state with respect to the subject whose distance from the lens device changes. The program may cause the computer to control the focus position based on the first change amount and the second change amount.

  According to one aspect of the present invention, it is possible to suppress a decrease in zoom tracking accuracy when the distance from the lens device to the subject changes.

  Note that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a subcombination of these feature groups can also be an invention.

It is a figure which shows an example of the external appearance of a unmanned aerial vehicle and a remote control. It is a figure which shows an example of the functional block of a unmanned aerial vehicle. It is a figure which shows an example of a zoom tracking curve. FIG. 7 is a flowchart illustrating an example of a procedure of zoom control of the imaging device. It is a figure showing a flow chart which shows an example of a procedure of derivation of the 2nd amount of change of a focus position based on change of distance to a photographic subject. It is a figure for demonstrating the to-be-photographed object distance in the optical system of a lens part, a focal distance, etc. FIG. It is a figure for demonstrating the to-be-photographed object distance in the optical system of a lens part, a focal distance, etc. FIG. It is a figure showing a flow chart which shows an example of a procedure of derivation of the 1st amount of change of a focus position based on change of a zoom position. It is a figure for demonstrating the procedure of specification of the focus position based on a zoom tracking curve. It is a figure for demonstrating the mode of a change of the focus position by zoom tracking control. It is a figure for demonstrating an example of a hardware configuration.

  Hereinafter, the present invention will be described through the embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Moreover, not all combinations of features described in the embodiments are essential to the solution of the invention.

  Hereinafter, although this invention is demonstrated through embodiment of invention, the following embodiment does not limit the invention which concerns on a claim. Further, not all combinations of features described in the embodiments are essential to the solution of the invention. It will be apparent to those skilled in the art that various changes or modifications can be added to the following embodiments. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

  The claims, the description, the drawings, and the abstract contain matters that are subject to copyright protection. The copyright holder will not object to any copy of these documents as they appear in the Patent Office file or record. However, in all other cases, all copyrights are reserved.

  Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, wherein the blocks are responsible for (1) process steps or (2) operations being performed. May represent a "part" of The particular stages and "parts" may be implemented by programmable circuits and / or processors. Dedicated circuitry may include digital and / or analog hardware circuitry. Integrated circuits (ICs) and / or discrete circuits may be included. Programmable circuits may include reconfigurable hardware circuits. Reconfigurable hardware circuits include logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, flip flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. Memory elements, etc. may be included.

  Computer readable media may include any tangible device capable of storing instructions for execution by a suitable device. As a result, a computer readable medium having instructions stored thereon will comprise an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-Ray (RTM) Disc, Memory Stick, Integrated A circuit card or the like may be included.

  Computer readable instructions may include either source code or object code written in any combination of one or more programming languages. Source code or object code includes a conventional procedural programming language. Conventional procedural programming languages include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or like Smalltalk, JAVA, C ++, etc. It may be an object oriented programming language, and a "C" programming language or similar programming language. Computer readable instructions may be local or to a wide area network (WAN) such as a local area network (LAN), the Internet, etc., relative to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing device. May be provided via The processor or programmable circuitry may execute computer readable instructions to create a means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

  FIG. 1 shows an example of the appearance of an unmanned aerial vehicle (UAV) 10 and a remote control device 300. The UAV 10 includes a UAV main body 20, a gimbal 50, a plurality of imaging devices 60, and an imaging device 100. The gimbal 50 and the imaging device 100 are examples of an imaging system. The UAV 10 is an example of an aircraft moving in the air. The term “air vehicle” is a concept including UAVs, other aircraft moving in the air, airships, helicopters and the like.

  The UAV body 20 comprises a plurality of rotors. The plurality of rotors are an example of the propulsion unit. The UAV body 20 causes the UAV 10 to fly by controlling the rotation of a plurality of rotors. The UAV body 20 causes the UAV 10 to fly using, for example, four rotors. The number of rotors is not limited to four. The UAV 10 may also be a fixed wing aircraft that does not have a rotor.

  The imaging device 100 is a camera for imaging which captures an object included in a desired imaging range. The gimbal 50 rotatably supports the imaging device 100. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 rotatably supports the imaging device 100 on the pitch axis using an actuator. The gimbal 50 rotatably supports the imaging device 100 about each of the roll axis and the yaw axis using an actuator. The gimbal 50 may change the attitude of the imaging device 100 by rotating the imaging device 100 about at least one of the yaw axis, the pitch axis, and the roll axis.

  The plurality of imaging devices 60 are cameras for sensing that capture the periphery of the UAV 10 to control the flight of the UAV 10. Two imaging devices 60 may be provided at the front, which is the nose of the UAV 10. Two further imaging devices 60 may be provided on the bottom of the UAV 10. The two front imaging devices 60 may be paired to function as a so-called stereo camera. The two imaging devices 60 on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the UAV 10 may be generated based on the images captured by the plurality of imaging devices 60. The number of imaging devices 60 provided in the UAV 10 is not limited to four. The UAV 10 may include at least one imaging device 60. The UAV 10 may include at least one imaging device 60 on each of the nose, aft, sides, bottom, and ceiling of the UAV 10. The angle of view that can be set by the imaging device 60 may be wider than the angle of view that can be set by the imaging device 100. The imaging device 60 may have a single focus lens or a fisheye lens.

  The remote control device 300 communicates with the UAV 10 to remotely control the UAV 10. The remote control device 300 may communicate with the UAV 10 wirelessly. The remote control device 300 transmits instruction information indicating various commands related to the movement of the UAV 10 such as rising, falling, acceleration, deceleration, forward, reverse, and rotation to the UAV 10. The instruction information includes, for example, instruction information for raising the altitude of the UAV 10. The instruction information may indicate the altitude at which the UAV 10 should be located. The UAV 10 moves so as to be located at the altitude indicated by the instruction information received from the remote control device 300. The instruction information may include a lift instruction to raise the UAV 10. The UAV 10 goes up while accepting the up command. Even if the UAV 10 receives an ascent instruction, if the UAV 10's altitude reaches the upper limit altitude, the UAV 10 may limit the ascent.

  FIG. 2 shows an example of a functional block of the UAV 10. The UAV 10 includes a UAV control unit 30, a memory 32, a communication interface 36, a propulsion unit 40, a GPS receiver 41, an inertial measurement device 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60 and the imaging device 100.

  The communication interface 36 communicates with other devices such as the remote control device 300. The communication interface 36 may receive instruction information including various instructions for the UAV control unit 30 from the remote control device 300. The memory 32 includes a propulsion unit 40, a GPS receiver 41, an inertial measurement unit (IMU) 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60, and a memory 32. And the program etc. required to control the imaging device 100 are stored. The memory 32 may be a computer readable recording medium, and may include at least one of a SRAM, a DRAM, an EPROM, an EEPROM, and a flash memory such as a USB memory. The memory 32 may be provided inside the UAV body 20. It may be provided to be removable from the UAV main body 20.

  The UAV control unit 30 controls the flight and imaging of the UAV 10 in accordance with a program stored in the memory 32. The UAV control unit 30 may be configured by a microprocessor such as a CPU or an MPU or a microcontroller such as an MCU. The UAV control unit 30 controls the flight and imaging of the UAV 10 according to the command received from the remote control device 300 via the communication interface 36. The promotion unit 40 promotes the UAV 10. The propulsion unit 40 has a plurality of rotors and a plurality of drive motors for rotating the plurality of rotors. The propulsion unit 40 rotates the plurality of rotary blades via the plurality of drive motors in accordance with the instruction from the UAV control unit 30 to fly the UAV 10.

  The GPS receiver 41 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 41 calculates the position (latitude and longitude) of the GPS receiver 41, that is, the position (latitude and longitude) of the UAV 10, based on the plurality of received signals. The IMU 42 detects the attitude of the UAV 10. The IMU 42 detects, as the posture of the UAV 10, accelerations in the front, rear, left, right, and top three axial directions of the UAV 10, and angular velocities in three axial directions of pitch, roll, and yaw. The magnetic compass 43 detects the heading of the UAV 10. The barometric altimeter 44 detects the altitude at which the UAV 10 flies. The barometric pressure altimeter 44 detects the barometric pressure around the UAV 10, converts the detected barometric pressure into a height, and detects the height. The temperature sensor 45 detects the temperature around the UAV 10. The humidity sensor 46 detects the humidity around the UAV 10.

  The imaging device 100 includes an imaging unit 102 and a lens unit 200. The lens unit 200 is an example of a lens device. The imaging unit 102 includes an image sensor 120, an imaging control unit 110, and a memory 130. The image sensor 120 may be configured by a CCD or a CMOS. The image sensor 120 outputs image data of an optical image formed through the plurality of lenses 210 to the imaging control unit 110. The imaging control unit 110 may be configured by a microprocessor such as a CPU or an MPU, a microcontroller such as an MCU, or the like. The imaging control unit 110 may control the imaging device 100 according to an operation command of the imaging device 100 from the UAV control unit 30. The memory 130 may be a computer readable recording medium, and may include at least one of SRAM, DRAM, EPROM, EEPROM, and flash memory such as USB memory. The memory 130 stores programs and the like necessary for the imaging control unit 110 to control the image sensor 120 and the like. The memory 130 may be provided inside the housing of the imaging device 100. The memory 130 may be provided to be removable from the housing of the imaging device 100.

  The lens unit 200 includes a plurality of lenses 210, a plurality of lens drivers 212, and a lens controller 220. The plurality of lenses 210 may function as a zoom lens, a varifocal lens, and a focus lens. At least a part or all of the plurality of lenses 210 is disposed movably along the optical axis. The lens unit 200 may be an interchangeable lens provided detachably to the imaging unit 102. The lens driving unit 212 moves at least a part or all of the plurality of lenses 210 along the optical axis via a mechanical member such as a cam ring. The lens driver 212 may include an actuator. The actuator may include a stepping motor. The lens control unit 220 drives the lens driving unit 212 according to the lens control command from the imaging unit 102 to move one or more lenses 210 along the optical axis direction via the mechanical member. The lens control command is, for example, a zoom control command and a focus control command.

  The lens unit 200 further includes a memory 222 and a position sensor 214. The lens control unit 220 controls movement of the lens 210 in the optical axis direction via the lens driving unit 212 in accordance with the lens operation command from the imaging unit 102. The lens control unit 220 controls movement of the lens 210 in the optical axis direction via the lens driving unit 212 in accordance with the lens operation command from the imaging unit 102. Some or all of the lenses 210 move along the optical axis. The lens control unit 220 performs at least one of the zoom operation and the focusing operation by moving at least one of the lenses 210 along the optical axis. The position sensor 214 detects the position of the lens 210. Position sensor 214 may detect the current zoom position or focus position.

  The lens drive unit 212 may include a shake correction mechanism. The lens control unit 220 may execute shake correction by moving the lens 210 in the direction along the optical axis or in the direction perpendicular to the optical axis via the shake correction mechanism. The lens drive unit 212 may drive a shake correction mechanism by a stepping motor to execute shake correction. The shake correction mechanism may be driven by a stepping motor to move the image sensor 120 in the direction along the direction of the optical axis or in the direction perpendicular to the optical axis to perform the shake correction.

  The memory 222 stores control values of the plurality of lenses 210 moving through the lens driving unit 212. The memory 222 may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.

  The imaging device 100 configured in this way has a so-called zoom tracking function. When the zoom position indicating the position of the zoom lens changes, the zoom tracking function maintains the in-focus state with respect to a subject whose distance from the imaging device 100 does not change. It is a function to change the focus position indicating the position. Zoom tracking is performed, for example, by moving the focus lens to a target focus position corresponding to a target zoom position identified according to a zoom tracking curve according to the subject distance as shown in FIG. The zoom tracking curve is an example of information indicating the relationship between the focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens unit 200 does not change, and the zoom position of the zoom lens. The information is determined by the characteristics of the optical system of the lens unit 200, and may be stored, for example, in the memory 222.

  For example, in the case where the optical system of the lens unit 200 is changed from the state of the zoom position (1) and the focus position (1) to the state of the zoom position (2) when the subject is at the infinite end, the imaging device 100 In accordance with the zoom tracking curve 400, the focus position (2) corresponding to the zoom position (2) is identified. The imaging apparatus 100 derives the change amount of the zoom position based on the zoom position (1) and the zoom position (2), and derives the change amount of the focus position from the focus position (1) and the focus position (2). . Then, the imaging device 100 controls the zoom position and the focus position based on the respective change amounts.

  The above zoom tracking is based on the premise that the distance from the lens unit 200 to the subject is constant while moving the zoom lens to a desired zoom position. However, the distance may change while the zoom position is changing. For example, when the imaging device 100 is mounted on a flying object such as the UAV 10, the change in the position of the flying object or the subject during the flight of the flying object may reduce the accuracy of zoom tracking. Therefore, in the present embodiment, even when the distance from the lens unit 200 to the subject changes, the reduction in the zoom tracking accuracy is suppressed.

  The imaging control unit 110 includes a specifying unit 112, a zoom position control unit 114, and a focus position control unit 116. When the zoom position indicating the position of the zoom lens changes, the identifying unit 112 selects the first focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens unit 200 does not change. Identify the amount of change. The identifying unit 112 identifies a second change amount of the focus position for maintaining the in-focus state with respect to the subject whose distance from the lens unit 200 changes. The identifying unit 112 is an example of a first identifying unit and a second identifying unit. The zoom position does not necessarily have to directly indicate the physical position of at least one lens constituting the zoom lens in the optical axis direction. For example, when using a cam mechanism to move the zoom lens, the zoom position may be a value corresponding to the displacement position of the cam member. When the zoom lens is moved using a cylindrical cam, the zoom position may have a value corresponding to the rotational angle of the cylindrical cam. The focus position may not necessarily directly indicate the physical position of at least one lens constituting the zoom lens in the optical axis direction. For example, the zoom position may be a value corresponding to the number of steps from the reference rotational position of the stepping motor for driving the drive mechanism for moving the focus lens in the optical axis direction.

  In response to an instruction to move the zoom lens from the first zoom position to the second zoom position, the identifying unit 112 identifies a first change amount when the zoom lens moves from the first zoom position to the second zoom position. . The identifying unit 112 further identifies a second change amount based on the plurality of images captured before the zoom lens moves to the second zoom position. The identifying unit 112 generates a second change amount based on a plurality of images including a first image captured when the zoom lens is at the first zoom position and a second image captured before the first image. You may specify. The identifying unit 112 may identify a first distance indicating a distance from the lens unit 200 to the subject based on the first image. The identifying unit 112 may identify a second distance indicating the distance from the lens unit 200 to the subject based on the second image. The identifying unit 112 may identify the second change amount based on the first distance and the second distance. The identifying unit 112 may identify the second change amount based on the difference between the first distance and the second distance.

  The focus position control unit 116 controls the focus position based on the first change amount and the second change amount. The zoom position control unit 114 controls the position of the zoom lens according to an instruction to move the zoom lens from the first zoom position to the second zoom position.

  In zoom tracking, in addition to the first change amount specified based on the change in the position of the zoom lens, the focus position control unit 116 performs focusing based on the second change amount specified based on the change in the distance to the subject. Control the position. As a result, even when the distance to the subject changes, the imaging control unit 110 can control the zoom tracking with good control. At least the first image captured at the first zoom position and the second image captured before the first image before the zoom lens moves from the first zoom position to the second zoom position Based on the change amount of the distance to the subject is specified. The identifying unit 112 identifies a second change amount of the focus position before the zoom lens moves to the second zoom position based on the change amount. While the zoom lens moves from the first zoom position to the second zoom position, the focus position control unit 116 adds a first amount of change based on a change in the position of the zoom lens and a second change based on a change in the distance to the subject. The position of the focus lens can be controlled based on the amount of change. Therefore, even when the distance to the subject changes, it is possible to suppress the reduction in the accuracy of the zoom tracking.

  Here, for example, if the distance from the lens unit 200 to the subject is relatively long, even if the distance to the subject changes, the accuracy of the zoom tracking has virtually no effect. That is, depending on the imaging condition of the imaging device 100, zoom tracking based on the second change amount based on the change in the distance to the subject may not be necessary. Therefore, when the imaging device 100 does not satisfy a predetermined imaging condition, the focus position control unit 116 may control the focus position based on the first change amount and the second change amount. On the other hand, when the lens device satisfies the predetermined imaging condition, the focus position control unit 116 may control the focus position based on the first change amount, not based on the second change amount.

  When the focus position control unit 116 does not satisfy a predetermined imaging condition that the distance from the lens unit 200 to the ground is equal to or more than a predetermined distance, the focus position is determined based on the first change amount and the second change amount. You may control If the focus position control unit 116 satisfies a predetermined imaging condition that the distance from the lens unit 200 to the ground is equal to or greater than a predetermined distance, the focus position control unit 116 does not base on the second change amount, but on the basis of the first change amount. , May control the focus position. The distance from the lens unit 200 to the ground may be specified based on, for example, a signal from an infrared sensor provided in the UAV main body 20 or the imaging device 100. An infrared sensor may be provided, for example, on the UAV body 20 so that the infrared radiation is emitted vertically downward. The infrared sensor may detect the distance from the infrared sensor to the reflection surface as the distance from the lens unit 200 to the ground based on the reflected light of the emitted infrared light.

  The focus position control unit 116 sets predetermined imaging conditions in which the distance from the lens unit 200 to the ground is equal to or greater than a predetermined distance and the imaging direction of the lens unit 200 includes a component in a predetermined direction. If not satisfied, the focus position may be controlled based on the first change amount and the second change amount. The focus position control unit 116 sets predetermined imaging conditions in which the distance from the lens unit 200 to the ground is equal to or greater than a predetermined distance and the imaging direction of the lens unit 200 includes a component in a predetermined direction. If satisfied, the focus position may be controlled based on the first amount of change, not on the basis of the second amount of change. The predetermined direction may be a direction in which the imaging device 100 can capture the direction of the ground, and may be, for example, vertically downward. For example, the imaging device 100 is mounted on the UAV 10, and during the flight of the UAV 10, the ground surface is photographed by aerial photography. In such a case, since the distance to the subject is long, even if the distance to the subject changes, the accuracy of the zoom tracking has virtually no effect. Therefore, when the above conditions are satisfied, the focus position control unit 116 may control the focus position based on the first change amount, not based on the second change amount. On the other hand, even if the imaging direction of the lens unit 200 includes, for example, a component in the vertical direction, when the imaging apparatus 100 captures an object having a relatively short distance, a change in the distance to the object affects the zoom tracking accuracy. There is a possibility of giving. For example, when the imaging device 100 performs portrait shooting by flying the UAV 10, a change in the distance to the subject may affect the accuracy of zoom tracking. Therefore, when shooting is performed under such conditions, the focus position control unit 116 may control the focus position based on the first change amount and the second change amount.

  The focus position control unit 116 controls the focus position based on the first change amount and the second change amount when the imaging device 100 does not satisfy a predetermined imaging condition that the imaging device 100 operates in a predetermined imaging mode. Good. The focus position control unit 116 may control the focus position based on the first amount of change, not based on the second amount of change, when a predetermined imaging condition to operate in a predetermined imaging mode is satisfied. . The focus position control unit 116 is preset, for example, when the imaging apparatus 100 operates in a shooting mode other than a predetermined shooting mode assuming a scene in which the distance to the subject is relatively short, such as a portrait mode. It may be determined that the shooting conditions are not satisfied, and the focus position may be controlled based on the first change amount and the second change amount. The predetermined shooting mode may be any shooting mode for shooting a scene that has virtually no effect on the zoom tracking accuracy even if the distance to the subject changes.

  When the focus position control unit 116 does not satisfy a predetermined imaging condition that the distance from the lens unit 200 to the subject is equal to or more than a predetermined distance, the focus position is determined based on the first change amount and the second change amount. You may control When the focus position control unit 116 satisfies a predetermined imaging condition that the distance from the lens unit 200 to the subject is equal to or more than a predetermined distance, the focus position control unit 116 does not base on the second change amount but on the first change amount. , May control the focus position.

  The focus position control unit 116 may control the focus position based on the first change amount and the second change amount when the predetermined imaging condition that the UAV 10 is in flight is not satisfied. When the predetermined imaging condition that the UAV 10 is in flight is satisfied, the focus position control unit 116 may control the focus position based on the first amount of change, not based on the second amount of change.

  As described above, even if the distance to the subject changes, the zoom tracking based on the second change amount based on the change in the distance to the subject is not performed if the zoom tracking accuracy does not really affect. This can reduce the processing load in zoom tracking control.

  FIG. 4 is a flowchart illustrating an example of the procedure of zoom control of the imaging device 100. The imaging control unit 110 receives a zoom instruction to move the zoom lens based on a user operation (S100), and the imaging control unit 110 starts the zoom tracking operation in response to the zoom instruction. Further, the imaging control unit 110 determines whether or not a predetermined imaging condition for which the zoom tracking operation may be performed without considering the change in the distance to the subject is satisfied (S102). When a predetermined imaging condition is not satisfied, the imaging control unit 110 determines that the zoom tracking operation should be performed in consideration of a change in the distance to the subject. Then, the imaging control unit 110 derives the amount of change of the area of the main subject from the captured image (S104). The imaging control unit 110 first specifies the area of the main subject from the image. For example, the imaging control unit 110 may identify the area of the main subject by determining the subject existing at the center of the image as the main subject. The imaging control unit 110 may identify the area of the main subject by determining the subject with the largest movement within the predetermined period as the main subject. The imaging control unit 110 sets the difference between the area of the main subject of the image taken at the first time and the area of the main subject of the image taken at the second time before the first time to the area of the main subject. It may be derived as the amount of change.

  Next, the imaging control unit 110 determines whether the amount of change is equal to or greater than a predetermined threshold (S106). If the amount of change is smaller than a predetermined threshold, the imaging control unit 110 determines that the change in the distance to the subject has little influence on the accuracy of the zoom tracking, and returns to step S102. If the amount of change is equal to or greater than a predetermined threshold, the imaging control unit 110 determines that the change in the distance to the subject may affect the accuracy of the zoom tracking, and the second change in the focus position The quantity is derived (S108).

  When the zoom tracking operation is started, the imaging control unit 110 holds the current zoom position (S110). The imaging control unit 110 may obtain information indicating the current zoom position of the zoom lens detected by the position sensor 214 from the lens control unit 220, and hold the zoom position as the current zoom position. Furthermore, the imaging control unit 110 derives the target zoom position based on the zoom command (S112). The imaging control unit 110 may specify the zoom position indicated by the zoom command as the target zoom position. The imaging control unit 110 may specify the target zoom position from the movement amount of the zoom lens indicated by the zoom command and the current zoom position.

  Next, the imaging control unit 110 derives a first change amount of the focus position corresponding to the movement amount of the focus lens to be moved based on the change of the zoom position (S114). The focus position control unit 116 determines the change amount of the focus position (S116). When the second change amount is derived, the focus position control unit 116 may determine the change amount of the focus position by adding the second change amount to the first change amount. When the second change amount is not derived, the focus position control unit 116 may determine the first change amount as the change amount of the focus position. Alternatively, the focus position control unit 116 determines one of the first change amount and the second change amount as the change amount of the focus position, and thereafter, the focus position control unit 116 determines the other of the first change amount and the second change amount. May be determined as the amount of change in focus position. The focus position control unit 116 drives the focus lens based on the determined amount of change via the lens control unit 220 (S118). The focus position control unit 116 may change the focus position of the focus lens to a desired focus position at one time based on the first change amount and the second change amount. Alternatively, after the focus position control unit 116 once changes the focus position of the focus lens based on one of the first change amount and the second change amount, the focus position control unit 116 is further based on the other of the first change amount and the second change amount. The focus position of the focus lens may be changed. That is, the focus position control unit 116 may change the focus position of the focus lens to a desired focus position in stages based on the first change amount and the second change amount.

  On the other hand, the zoom position control unit 114 determines the change amount of the zoom position from the current zoom position and the target zoom position (S120). The zoom position control unit 114 drives the zoom lens based on the determined change amount of the zoom position via the lens control unit 220 (S122).

  The imaging control unit 110 determines whether the zoom position has reached the final zoom position specified according to the zoom instruction (S124). If the zoom position has not reached the final zoom position specified in accordance with the zoom command, the imaging control unit 110 repeats the processing from step S110. On the other hand, if the final zoom position has been reached, the imaging control unit 110 ends the process.

  FIG. 5 is a flowchart illustrating an example of a procedure of deriving a second amount of change in focus position based on a change in distance to a subject. 6 and 7 are diagrams for explaining the subject distance L1, the focal distance f, and the like in the optical system of the lens unit 200. FIG.

  The identifying unit 112 identifies the subject size y1 on the image plane for the image captured in the previous frame. The identifying unit 112 identifies the size of the subject 502 on the image plane imaged through the lens 210, as shown in FIG. 6 (S200). The identifying unit 112 derives the image magnification α1 of the previous frame (S202). The image magnification α1 is uniquely derived based on the zoom position and the focus position in the previous frame. The identifying unit 112 may derive an image magnification α1 (= IM1: design value image magnification) based on predetermined information indicating the relationship between the zoom position and the focus position and the image magnification.

  Further, the specifying unit 112 derives the subject distance L1 and the actual subject size Y (S204). The identifying unit 112 may derive the subject distance L1 according to the equation L1 = f / IM1. Furthermore, the identifying unit 112 may derive the subject size Y according to the equation Y = y1 × L1 / f.

  Subsequently, as illustrated in FIG. 7, the specifying unit 112 specifies the subject size y2 of the subject 504 on the image plane formed through the lens 210 for the image captured in the current frame (S206). The identifying unit 112 derives an image magnification α2 (= IM2: design value image magnification) in the current frame based on the zoom position and the focus position in the current frame (S208). Further, the specifying unit 112 derives the subject distance L2 and the actual subject size Y (S210). The identifying unit 112 may derive the subject distance L2 in accordance with the equation L2 = f / IM2. The identifying unit 112 may derive the subject size Y according to the equation Y = y2 × L2 / f.

The identifying unit 112 derives a change amount ΔL of the subject distance L (S212). The identifying unit 112 may derive the change amount ΔL in accordance with the equation ΔL = L1−L2. Further, the specifying unit 112 may derive the image plane change amount Δx. Specifying unit 112, according to the equation Δx = L1 (1- (IM1 / IM2)) × IM2 2, may derive an image plane variation [Delta] x. The identifying unit 112 may derive a second change amount of the focus position based on the image plane change amount Δx. The identifying unit 112 may derive the second change amount of the focus position by multiplying the image plane change amount Δx by a predetermined coefficient K. The coefficient K is a coefficient predetermined for focus position conversion with respect to image plane position change.

  The identifying unit 112 predicts the position of the subject in the image of the future frame imaged at the second zoom position based on the image imaged at the current frame and at least one image imaged at the previous frame The second change amount of the focus position may be derived from the prediction.

  FIG. 8 is a flowchart illustrating an example of a procedure of deriving a first amount of change in focus position based on a change in zoom position.

  The identifying unit 112 acquires the current zoom position (S300). The identifying unit 112 may obtain the current zoom position of the zoom lens via the lens control unit 220. The identifying unit 112 acquires the current focus position (S302). The specifying unit 112 may obtain the current focus position of the focus lens via the lens control unit 220. The identifying unit 112 derives the drive range at the current zoom position (S304). As shown in FIG. 9, the identifying unit 112 sets the focus position corresponding to the current zoom position on the zoom tracking curve 400 at infinity and the focus position corresponding to the current zoom position on the zoom tracking curve 402 at the near end. Drive range 404 may be identified. Further, the specifying unit 112 derives the ratio of the focus position to the drive range at the current zoom position (S306). The identifying unit 112 derives, for example, the ratio b / a in the drive range 404 shown in FIG.

  The identifying unit 112 derives the zoom position of the target based on the zoom command (S308). The identifying unit 112 derives a drive range at the target zoom position (S310). The identifying unit 112 derives, for example, a drive range 406 at the target zoom value shown in FIG. The identifying unit 112 derives the focus position of the target from the ratio at the zoom position of the target and the drive range. The identifying unit 112 derives a target focus position from, for example, the ratio b / a and the drive range 406 illustrated in FIG. 9. The identifying unit 112 derives a first change amount of the focus position from the current focus position and the target focus position (S314).

  The identifying unit 112 may derive a zoom tracking curve corresponding to the current distance to the subject from zoom tracking curves corresponding to at least two distances to the object defined in advance. Furthermore, the identifying unit 112 may derive, from the derived zoom tracking curve, the focus position of the target when the distance to the subject does not change.

  For example, as shown in FIG. 10, the zoom lens moves from the first zoom position to the second zoom position. In this case, the specifying unit 112 specifies the temporary focus position (1) according to the zoom tracking curve 401 specified from the first zoom position and the current focus position, and specifies the first change amount 412. However, the distance to the subject may actually change. Therefore, the specifying unit 112 specifies the second change amount 414 of the focus position based on the change amount of the distance to the subject. The identifying unit 112 identifies the amount of change in the distance to the subject based on the first image captured at the first zoom position and the second image captured before the first image, and selects the second focus position. The variation 414 may be identified. While the zoom lens moves from the first position to the second position, the focus position control unit 116 moves the change amount 416 by taking into account the first change amount 412 and the second change amount 414, and thus the focus lens Is moved to the first focus position. The focus position control unit 116 may move the focusing lens at one time while the zoom lens moves from the first position to the second position. In addition, while moving from the first zoom position to the second zoom position, the focus position control unit 116 moves the first change amount 412, the focus lens after moving the second change amount 414, and moves the focus lens. You may

  Further, the zoom lens moves from the second zoom position to the third zoom position. In this case, the specifying unit 112 specifies the provisional focus position (2) according to the zoom tracking curve 402, and specifies the first change amount 422. Further, the specifying unit 112 specifies the second change amount 424 of the focus position based on the change amount of the distance to the subject. While moving from the second zoom position to the third zoom position, the focus position control unit 116 moves the focus lens by the amount of change 426 in consideration of the first amount of change 422 and the second amount of change 424 to perform focusing. The lens may be moved to the second focus position. By moving the focus lens at one time in consideration of the first change amount 422 and the second change amount 424, the movement amount of the focus lens can be reduced. The focus position control unit 116 moves the focus lens by the first change amount 422 while moving the focus lens by the second change amount 424 while moving from the second zoom position to the third zoom position. By moving the focus lens, the focus lens may be moved to the second focus position.

  According to the imaging apparatus 100 according to the present embodiment, even when the distance from the lens unit 200 to the subject changes, the focus position is adjusted during zoom tracking based on the amount of change in the distance to the subject. It is possible to suppress a decrease in tracking accuracy.

  FIG. 11 shows an example of a computer 1200 in which aspects of the invention may be fully or partially embodied. A program installed on computer 1200 can cause computer 1200 to act as an operation or one or more "parts" of the device according to embodiments of the invention. Alternatively, the program may cause the computer 1200 to execute the operation or one or more “parts”. The program can cause the computer 1200 to execute the process according to the embodiment of the present invention or the steps of the process. Such programs may be executed by CPU 1212 to cause computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

  The computer 1200 according to the present embodiment includes a CPU 1212 and a RAM 1214, which are interconnected by a host controller 1210. Computer 1200 also includes a communication interface 1222, an input / output unit, which are connected to host controller 1210 via an input / output controller 1220. Computer 1200 also includes a ROM 1230. The CPU 1212 operates in accordance with programs stored in the ROM 1230 and the RAM 1214 to control each unit.

  The communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores therein a boot program or the like executed by the computer 1200 upon activation, and / or a program dependent on the hardware of the computer 1200. The program is provided via a computer readable recording medium or network such as a CR-ROM, a USB memory or an IC card. The program is installed in the RAM 1214 or ROM 1230, which is also an example of a computer-readable recording medium, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200 and brings about coordination between the programs and the various types of hardware resources. An apparatus or method may be configured by implementing the operation or processing of information in accordance with the use of computer 1200.

  For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes the communication program loaded in the RAM 1214, and performs communication processing to the communication interface 1222 based on the processing described in the communication program. You may order. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214 or a USB memory under the control of the CPU 1212 and transmits the read transmission data to the network, or The received data received from the network is written in a reception buffer area or the like provided on the recording medium.

  In addition, the CPU 1212 allows the RAM 1214 to read all or necessary portions of a file or database stored in an external recording medium such as a USB memory, and executes various types of processing on data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to the external storage medium.

  Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium and subjected to information processing. The CPU 1212 describes various types of operations, information processing, condition judgment, conditional branching, unconditional branching, information retrieval, which are described throughout the present disclosure for data read from the RAM 1214 and specified by a program instruction sequence. Various types of processing may be performed, including / replacement etc, and the results written back to RAM 1214. Further, the CPU 1212 may search for information in a file in a recording medium, a database or the like. For example, when a plurality of entries each having the attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. Search for an entry matching the condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby associate the first attribute satisfying the predetermined condition with the first attribute An attribute value of the second attribute may be acquired.

  The programs or software modules described above may be stored on computer readable storage medium on or near computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer readable storage medium, whereby the program can be transmitted to the computer 1200 via the network. provide.

  The execution order of each process such as operations, procedures, steps, and steps in the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly “before”, “preceding” It is to be noted that “it is not explicitly stated as“ etc. ”and can be realized in any order as long as the output of the previous process is not used in the later process. With regard to the operation flow in the claims, the specification, and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It is apparent to those skilled in the art that various changes or modifications can be added to the above embodiment. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

  The execution order of each process such as operations, procedures, steps, and steps in the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly “before”, “preceding” It is to be noted that “it is not explicitly stated as“ etc. ”and can be realized in any order as long as the output of the previous process is not used in the later process. With regard to the flow of operations in the claims, the specification and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

10 UAV
Reference Signs List 20 UAV main body 30 UAV control unit 32 memory 36 communication interface 40 propulsion unit 41 GPS receiver 42 inertia measurement device 43 magnetic compass 44 barometric altimeter 45 temperature sensor 46 humidity sensor 50 gimbal 60 imaging device 100 imaging device 102 imaging unit 110 imaging control unit 112 identification unit 114 zoom position control unit 116 focus position control unit 120 image sensor 130 memory 200 lens unit 210 lens 212 lens drive unit 214 position sensor 220 lens control unit 222 memory 300 remote control device 1200 computer 1210 host controller 1212 CPU
1214 RAM
1220 I / O controller 1222 communication interface 1230 ROM

Claims (17)

A control device for controlling a lens device provided with a zoom lens and a focus lens, comprising:
When the zoom position indicating the position of the zoom lens changes, a first change amount of the focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens device does not change A first identification unit to identify,
A second identification unit that identifies a second change amount of the focus position for maintaining the in-focus state with respect to the subject whose distance from the lens device changes;
A control unit configured to control the focus position based on the first change amount and the second change amount. In response to an instruction to move the zoom lens from the first zoom position to the second zoom position,
The first identification unit identifies the first change amount when the zoom lens moves from the first zoom position to the second zoom position.
The control device according to claim 1, wherein the second identification unit identifies the second change amount based on a plurality of images captured before the zoom lens moves to the second zoom position.   The second identification unit is configured based on the plurality of images including a first image captured when the zoom lens is at the first zoom position, and a second image captured before the first image. The control device according to claim 2, wherein the second change amount is specified.   The second identification unit identifies a first distance indicating a distance from the lens device to the subject based on the first image, and indicates a distance from the lens device to the subject based on the second image. The control device according to claim 3, wherein a second distance is specified, and the second change amount is specified based on the first distance and the second distance.   The control device according to claim 4, wherein the second identification unit identifies the second change amount based on a difference between the first distance and the second distance.   The first identification unit may use information indicating a relationship between a focus position indicating the position of the focus lens for maintaining an in-focus state with respect to a subject whose distance from the lens device does not change, and a zoom position of the zoom lens. The control device according to claim 1, wherein the first change amount is specified based on the second change amount. The control unit
When the lens device does not satisfy a predetermined imaging condition, the focus position is controlled based on the first change amount and the second change amount,
The control device according to claim 1, wherein the focus position is controlled based on the first amount of change not based on the second amount of change when the lens device satisfies a predetermined imaging condition. The control unit
When the predetermined imaging condition that the distance from the lens device to the ground is equal to or more than a predetermined distance is not satisfied, the focus position is controlled based on the first change amount and the second change amount. ,
When the predetermined imaging condition is satisfied that the distance from the lens device to the ground is equal to or more than a predetermined distance, the focus position is determined based on the first change amount, not based on the second change amount. The control device according to claim 7, which controls The control unit
When the distance from the lens device to the ground is equal to or greater than a predetermined distance, and the imaging direction of the lens device does not satisfy the predetermined imaging condition including a component of the predetermined direction, Controlling the focus position based on the first change amount and the second change amount,
When the distance from the lens device to the ground is equal to or greater than a predetermined distance, and the imaging direction of the lens device satisfies the predetermined imaging condition including a component of the predetermined direction, the second The control device according to claim 7, wherein the focus position is controlled based on the first change amount, not based on the change amount. The control unit
When the predetermined imaging condition that the distance from the lens device to the subject is equal to or more than a predetermined distance is not satisfied, the focus position is controlled based on the first change amount and the second change amount. And
When the predetermined imaging condition is satisfied that the distance from the lens device to the subject is equal to or more than a predetermined distance, the focusing is performed based on the first change amount, not based on the second change amount. The control device according to claim 7, which controls the position. The control unit
When the predetermined imaging condition that the lens device operates in a predetermined imaging mode is not satisfied, the focus position is controlled based on the first change amount and the second change amount,
When the predetermined imaging condition that the lens device operates in the predetermined imaging mode is satisfied, the focus position is controlled based on the first change amount, not based on the second change amount. The control device according to claim 7. The lens device is mounted on a flying object,
The control unit
If the predetermined imaging condition that the aircraft is in flight is not satisfied, the focus position is controlled based on the first change amount and the second change amount,
The focus position is controlled based on the first change amount, not based on the second change amount, when the predetermined imaging condition that the aircraft is in flight is satisfied. Control device. A controller according to any one of claims 1 to 12;
An imaging device comprising the lens device;   An aircraft flying with the imaging device according to claim 13.   15. The vehicle according to claim 14, further comprising a support mechanism rotatably supporting the imaging device. A control method for controlling a lens device provided with a zoom lens and a focus lens, comprising:
When the zoom position indicating the position of the zoom lens changes, a first change amount of the focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens device does not change Stage of identification,
Identifying a second amount of change of the focus position to maintain the in-focus state with respect to the subject whose distance from the lens device changes;
Controlling the focus position based on the first change amount and the second change amount. A program for causing a computer to control a lens device provided with a zoom lens and a focus lens,
When the zoom position indicating the position of the zoom lens changes, a first change amount of the focus position indicating the position of the focus lens for maintaining the in-focus state with respect to the subject whose distance from the lens device does not change Stage of identification,
Identifying a second amount of change of the focus position to maintain the in-focus state with respect to the subject whose distance from the lens device changes;
A program for causing the computer to execute the step of controlling the focus position based on the first change amount and the second change amount.

Images