JP2020181056A - Controller, mobile body, method for control, and program - Google Patents

Abstract

To deal with the situation in which imaging control becomes unstable by a change of the distance to a specific object in some cases.SOLUTION: A controller may include a circuit formed to derive a first contrast evaluation value for evaluating a contrast value in a first region by applying a first contrast evaluation filter to the first region as a region including an object in a first image, execute a focus control of the imaging device on the basis of the first contrast evaluation value, specify a second contrast evaluation filter on the basis of a first distance between the imaging device and the object when the imaging device took the first image and a second distance between the imaging device and the object when the imaging device took a second image, derive a second contrast evaluation value for evaluating a contrast value in a second region by applying a contrast evaluation filter to the second region as a region including an object in a second image, and execute a focus control of the imaging device on the basis of the second contrast evaluation value.SELECTED DRAWING: Figure 5

Description

The present invention relates to a control device, an image pickup device, a moving body, a control method, and a program.

Patent Document 1 discloses that the focusing position specified according to the distance to the subject is corrected according to the spatial frequency associated with the photographing mode.
[Prior art literature]
[Patent Document]
[Patent Document 1] Japanese Unexamined Patent Publication No. 2017-68195

Imaging controls such as autofocus control, autoexposure control, and auto white balance may become unstable as the distance to a specific subject changes.

The control device according to one aspect of the present invention applies the first contrast evaluation filter to the first region, which is a region including the subject, in the first image captured by the imaging device, thereby applying the first region. A circuit configured to derive a first contrast evaluation value for evaluating the contrast value of the above may be provided. The circuit may be configured to perform focusing control of the imaging device based on the first contrast evaluation value. The circuit is based on the first distance between the imager and the subject when the imager captures the first image and the second distance between the imager and the subject when the imager captures the second image. The second contrast evaluation filter may be specified. The circuit derives a second contrast evaluation value for evaluating the contrast value in the second region by applying the second contrast evaluation filter to the second region in the second image, which is a region including the subject. It may be configured as follows. The circuit may be configured to perform focusing control of the image pickup device based on the second contrast evaluation value.

The first contrast evaluation filter may have a band including the first spatial frequency. The second contrast evaluation filter may have a band including a second spatial frequency different from the first spatial frequency.

Specifying the second contrast evaluation filter may include specifying the second contrast evaluation filter based on the ratio of the first distance to the second distance and the information of the first contrast evaluation filter.

The circuit may be configured to control the focus lens of the image pickup device to focus at infinity when the distance between the image pickup device and the subject is longer than a predetermined distance.

The control device according to one aspect of the present invention derives a first evaluation value of the first region based on the image information of the first region, which is a region including the subject, in the first image captured by the imaging device. A circuit configured as described above may be provided. The circuit may be configured to perform the first imaging control of the imaging apparatus based on the first evaluation value. The circuit may be configured to derive a second evaluation value of the second region based on the image information of the second region, which is a region including the subject, in the second image in which the subject is captured by the imaging device. .. The circuit is the first distance between the image pickup device and the subject when the image pickup device captures the first image, the second distance between the image pickup device and the subject when the image pickup device captures the second image, and the second. The second image pickup control of the image pickup apparatus may be executed based on the first evaluation value and the second evaluation value.

Executing the second imaging control derives a predicted evaluation value, which is a predicted evaluation value of the second region, based on the first distance, the second distance, and the first evaluation value, and derives the predicted evaluation value and the second evaluation value. It may include executing the imaging control of the imaging apparatus based on the evaluation value.

Derivation of the predicted evaluation value may include deriving the predicted evaluation value based on the ratio of the first distance to the second distance and the first evaluation value.

Derivation of the first evaluation value may include deriving the first brightness evaluation value for evaluating the brightness of the first region as the first evaluation value. Executing the first imaging control may include executing the exposure control of the imaging device based on the first luminance evaluation value. Derivation of the second evaluation value may include deriving the second brightness evaluation value for evaluating the brightness of the second region as the second evaluation value. Executing the second imaging control may include executing the exposure control of the imaging device based on the first distance, the second distance, the first luminance evaluation value, and the second luminance evaluation value.

Executing the second imaging control derives a predicted luminance evaluation value, which is a predicted luminance evaluation value of the second region of interest, based on the first distance, the second distance, and the first luminance evaluation value, and derives the predicted luminance evaluation value. It may include performing exposure control of the imaging device based on the evaluation value and the second luminance evaluation value.

Derivation of the predicted luminance evaluation value may include deriving the predicted luminance evaluation value based on the ratio of the first distance to the second distance and the first luminance evaluation value.

Derivation of the first evaluation value may include deriving the first color temperature of the first region as the first evaluation value. Executing the first imaging control may include executing the white balance control of the imaging apparatus based on the first color temperature. Derivation of the second evaluation value may include deriving the second color temperature of the second region as the second evaluation value. Performing the second imaging control may include performing white balance control of the imaging device based on the first distance, the second distance, the first color temperature, and the second color temperature.

Executing the second imaging control derives the predicted color temperature, which is the predicted color temperature of the second region, based on the first distance, the second distance, and the first color temperature, and derives the predicted color temperature and the second color temperature. It may include performing exposure control of the imaging device based on the color temperature.

Derivation of the predicted color temperature may include deriving the predicted color temperature based on the ratio of the first distance to the second distance and the first color temperature.

The imaging device according to one aspect of the present invention may include the above control device and an image sensor.

The moving body according to one aspect of the present invention may be a moving body provided with the above-mentioned imaging device.

The moving body may move along the imaging direction of the imaging device until the distance to the subject is from the first distance to the second distance.

The moving body may be a flying body. The flying object may ascend or descend along the imaging direction of the imaging device from the first distance to the second distance to the subject.

In the control method according to one aspect of the present invention, the first contrast evaluation filter is applied to the first area of interest, which is the area of the subject selected by the image pickup device, in the first image captured by the image pickup device. Therefore, a step of deriving the first contrast evaluation value for evaluating the contrast value of the first interest region may be provided. The control method may include a step of executing focusing control of the image pickup apparatus based on the first contrast evaluation value. The control method is the first distance between the image pickup device and the subject when the image pickup device captures the first image and the second distance between the image pickup device and the subject when the image pickup device captures the second image. Based on this, a step of identifying the second contrast evaluation filter may be provided. The control method evaluates the contrast value of the second interest region by applying the second contrast evaluation filter to the second interest region, which is the region of the subject selected by the imaging device in the second image. A step of deriving the second contrast evaluation value may be provided. The control method may include a step of executing focusing control of the image pickup apparatus based on the second contrast evaluation value.

The control method according to one aspect of the present invention derives a first evaluation value of the first region based on the image information of the first region, which is a region including the subject, in the first image captured by the imaging device. It may have stages. A step of executing the first image pickup control of the image pickup apparatus based on the first evaluation value may be provided. The control method may include a step of deriving a second evaluation value of the second region based on the image information of the second region, which is a region including the subject, in the second image in which the subject is captured by the imaging device. .. The control method includes a first distance between the image pickup device and the subject when the image pickup device captures the first image, a second distance between the image pickup device and the subject when the image pickup device captures the second image, and the like. A step of executing the second image pickup control of the image pickup apparatus may be provided based on the first evaluation value and the second evaluation value.

The program according to one aspect of the present invention may be a program for operating a computer as the control device.

According to one aspect of the present invention, it is possible to prevent the imaging control such as autofocus control, automatic exposure control, and auto white balance from becoming unstable due to a change in the distance to a specific subject.

The outline of the above invention does not list all the necessary features of the present invention. Sub-combinations of these feature groups can also be inventions.

It is a figure which shows an example of the appearance of an unmanned aerial vehicle and a remote control device. It is a figure which shows an example of the functional block of an unmanned aerial vehicle. It is a figure for demonstrating the contrast evaluation filter. It is a figure for demonstrating the contrast evaluation filter. It is a figure which shows a mode that an unmanned aerial vehicle moves along the image pickup direction of an image pickup apparatus. It is a figure which shows the flowchart which shows an example of the procedure of AF control by an image pickup control part. It is a figure for demonstrating the luminance evaluation value. It is a figure for demonstrating the luminance evaluation value. It is a figure which shows the flowchart which shows an example of the procedure of AE control by an image pickup control part. It is a figure which shows the flowchart which shows an example of the procedure of AWB control by an image pickup control part. It is a figure which shows an example of a hardware configuration.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention. It will be apparent to those skilled in the art that various changes or improvements can be made to the following embodiments. It is clear from the description of the claims that such modified or improved forms may also be included in the technical scope of the present invention.

The claims, description, drawings, and abstracts include matters that are subject to copyright protection. The copyright holder will not object to any person's reproduction of these documents as they appear in the Patent Office files or records. However, in other cases, all copyrights are reserved.

Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, wherein the block is (1) a stage of the process in which the operation is performed or (2) a device having a role of performing the operation. May represent the "part" of. Specific stages and "parts" may be implemented by programmable circuits and / or processors. Dedicated circuits may include digital and / or analog hardware circuits. It may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include a reconfigurable hardware circuit. Reconfigurable hardware circuits include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGA), programmable logic arrays (PLA), etc. It may include a memory element such as.

The computer-readable medium may include any tangible device capable of storing instructions executed by the appropriate device. As a result, the computer-readable medium having the instructions stored therein will include the product, including instructions that can be executed to create means for performing the operation specified in the flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable media include floppy (registered trademark) disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), Electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray (RTM) disk, memory stick, integrated A circuit card or the like may be included.

Computer-readable instructions may include either source code or object code written in any combination of one or more programming languages. Source code or object code includes traditional procedural programming languages. Traditional procedural programming languages are assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or Smalltalk®, JAVA®, C ++. It may be an object-oriented programming language such as, and a "C" programming language or a similar programming language. Computer-readable instructions are used locally or on a local area network (LAN), wide area network (WAN) such as the Internet, to the processor or programmable circuit of a general purpose computer, special purpose computer, or other programmable data processing device. ) May be provided. The processor or programmable circuit may execute computer-readable instructions to create means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers and the like.

FIG. 1 shows an example of the appearance of the unmanned aerial vehicle (UAV) 10 and the remote control device 300. The UAV 10 includes a UAV main body 20, a gimbal 50, a plurality of imaging devices 60, and an imaging device 100. The gimbal 50 and the imaging device 100 are examples of an imaging system. The UAV 10 is a concept including a moving body, an air vehicle moving in the air, a vehicle moving on the ground, a ship moving on the water, and the like. An airship that moves in the air is a concept that includes UAVs, other aircraft that move in the air, airships, helicopters, and the like.

The UAV main body 20 includes a plurality of rotor blades. The plurality of rotor blades are an example of a propulsion unit. The UAV main body 20 flies the UAV 10 by controlling the rotation of a plurality of rotor blades. The UAV body 20 flies the UAV 10 using, for example, four rotor blades. The number of rotor blades is not limited to four. Further, the UAV 10 may be a fixed-wing aircraft having no rotor blades.

The imaging device 100 is an imaging camera that captures a subject included in a desired imaging range. The gimbal 50 rotatably supports the imaging device 100. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 rotatably supports the image pickup device 100 on a pitch axis using an actuator. The gimbal 50 further rotatably supports the image pickup device 100 around each of the roll axis and the yaw axis by using an actuator. The gimbal 50 may change the posture of the image pickup device 100 by rotating the image pickup device 100 around at least one of the yaw axis, the pitch axis, and the roll axis.

The plurality of imaging devices 60 are sensing cameras that image the surroundings of the UAV 10 in order to control the flight of the UAV 10. Two imaging devices 60 may be provided in front of the nose of the UAV 10. Yet two other imaging devices 60 may be provided on the bottom surface of the UAV 10. The two image pickup devices 60 on the front side may form a pair and function as a so-called stereo camera. The two image pickup devices 60 on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the UAV 10 may be generated based on the images captured by the plurality of imaging devices 60. The number of image pickup devices 60 included in the UAV 10 is not limited to four. The UAV 10 may include at least one imaging device 60. The UAV 10 may be provided with at least one imaging device 60 on each of the nose, nose, side surface, bottom surface, and ceiling surface of the UAV 10. The angle of view that can be set by the image pickup device 60 may be wider than the angle of view that can be set by the image pickup device 100. The image pickup apparatus 60 may have a single focus lens or a fisheye lens.

The remote control device 300 communicates with the UAV 10 to remotely control the UAV 10. The remote control device 300 may communicate wirelessly with the UAV 10. The remote control device 300 transmits instruction information indicating various commands related to the movement of the UAV 10, such as ascending, descending, accelerating, decelerating, advancing, reversing, and rotating, to the UAV 10. The instruction information includes, for example, instruction information for raising the altitude of the UAV 10. The instruction information may indicate the altitude at which the UAV 10 should be located. The UAV 10 moves so as to be located at an altitude indicated by the instruction information received from the remote control device 300. The instruction information may include an ascending instruction to ascend the UAV 10. The UAV10 rises while accepting the rise order. Even if the UAV10 accepts the ascending command, the ascending may be restricted if the altitude of the UAV10 has reached the upper limit altitude.

FIG. 2 shows an example of the functional block of the UAV 10. The UAV 10 includes a UAV control unit 30, a memory 37, a communication interface 36, a propulsion unit 40, a GPS receiver 41, an inertial measurement unit 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, and an imaging device. 60 and an image pickup device 100 are provided.

The communication interface 36 communicates with another device such as the remote control device 300. The communication interface 36 may receive instruction information including various commands from the remote control device 300 to the UAV control unit 30. In the memory 37, the UAV control unit 30 has a propulsion unit 40, a GPS receiver 41, an inertial measurement unit (IMU) 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an image pickup device 60, and the like. The program and the like necessary for controlling the image pickup device 100 are stored. The memory 37 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, USB memory, and solid state drive (SSD). The memory 37 may be provided inside the UAV main body 20. It may be provided so as to be removable from the UAV main body 20.

The UAV control unit 30 controls the flight and imaging of the UAV 10 according to the program stored in the memory 37. The UAV control unit 30 may be composed of a CPU, a microprocessor such as an MPU, a microcontroller such as an MCU, or the like. The UAV control unit 30 controls the flight and imaging of the UAV 10 according to a command received from the remote control device 300 via the communication interface 36. The propulsion unit 40 promotes the UAV 10. The propulsion unit 40 has a plurality of rotary blades and a plurality of drive motors for rotating the plurality of rotary blades. The propulsion unit 40 makes the UAV 10 fly by rotating a plurality of rotor blades via the plurality of drive motors in accordance with a command from the UAV control unit 30.

The GPS receiver 41 receives a plurality of signals indicating the time transmitted from the plurality of GPS satellites. The GPS receiver 41 calculates the position (latitude and longitude) of the GPS receiver 41, that is, the position (latitude and longitude) of the UAV 10 based on the plurality of received signals. The IMU 42 detects the posture of the UAV 10. The IMU 42 detects the acceleration in the three axial directions of the front-back, left-right, and up-down of the UAV 10 and the angular velocity in the three-axis directions of pitch, roll, and yaw as the posture of the UAV 10. The magnetic compass 43 detects the nose orientation of the UAV 10. The barometric altimeter 44 detects the altitude at which the UAV 10 flies. The barometric altimeter 44 detects the barometric pressure around the UAV 10, converts the detected barometric pressure into an altitude, and detects the altitude. The temperature sensor 45 detects the ambient temperature of the UAV 10. The humidity sensor 46 detects the humidity around the UAV 10.

The image pickup apparatus 100 includes an image pickup section 102 and a lens section 200. The lens unit 200 is an example of a lens device. The image pickup unit 102 includes an image sensor 120, an image pickup control unit 110, a memory 130, and a ranging sensor 140. The image sensor 120 may be configured by CCD or CMOS. The image sensor 120 captures an optical image formed through a plurality of lenses 210, and outputs the captured image to the image pickup control unit 110. The image pickup control unit 110 may be composed of a CPU, a microprocessor such as an MPU, a microcontroller such as an MCU, or the like. The image pickup control unit 110 may control the image pickup device 100 in response to an operation command of the image pickup device 100 from the UAV control unit 30. The image pickup control unit 110 is an example of a circuit. The memory 130 may be a computer-readable recording medium and may include at least one of flash memories such as SRAM, DRAM, EPROM, EEPROM, USB memory, and solid state drive (SSD). The memory 130 stores a program or the like necessary for the image pickup control unit 110 to control the image sensor 120 or the like. The memory 130 may be provided inside the housing of the image pickup apparatus 100. The memory 130 may be provided so as to be removable from the housing of the image pickup apparatus 100.

The distance measuring sensor 140 measures the distance to the subject. The distance measuring sensor 140 may be an infrared sensor, an ultrasonic sensor, a stereo camera, a TOF (Time Of Flight) sensor, or the like.

The lens unit 200 includes a plurality of lenses 210, a plurality of lens driving units 212, and a lens control unit 220. The plurality of lenses 210 may function as a zoom lens, a varifocal lens, and a focus lens. At least a part or all of the plurality of lenses 210 are arranged so as to be movable along the optical axis. The lens unit 200 may be an interchangeable lens that is detachably provided to the imaging unit 102. The lens driving unit 212 moves at least a part or all of the plurality of lenses 210 along the optical axis via a mechanical member such as a cam ring. The lens driving unit 212 may include an actuator. The actuator may include a stepping motor. The lens control unit 220 drives the lens drive unit 212 in accordance with a lens control command from the image pickup unit 102 to move one or more lenses 210 along the optical axis direction via a mechanical member. The lens control command is, for example, a zoom control command and a focus control command.

The lens unit 200 further includes a memory 222 and a position sensor 214. The lens control unit 220 controls the movement of the lens 210 in the optical axis direction via the lens drive unit 212 in response to a lens operation command from the image pickup unit 102. The lens control unit 220 controls the movement of the lens 210 in the optical axis direction via the lens drive unit 212 in response to a lens operation command from the image pickup unit 102. Part or all of the lens 210 moves along the optical axis. The lens control unit 220 executes at least one of the zoom operation and the focus operation by moving at least one of the lenses 210 along the optical axis. The position sensor 214 detects the position of the lens 210. The position sensor 214 may detect the current zoom position or focus position.

The lens driving unit 212 may include a runout correction mechanism. The lens control unit 220 may execute the shake correction by moving the lens 210 in the direction along the optical axis or in the direction perpendicular to the optical axis via the shake correction mechanism. The lens driving unit 212 may drive the runout correction mechanism by a stepping motor to perform runout correction. The runout correction mechanism may be driven by a stepping motor to move the image sensor 120 in a direction along the optical axis or in a direction perpendicular to the optical axis to perform runout correction.

The memory 222 stores the control values of the plurality of lenses 210 that move via the lens driving unit 212. The memory 222 may include at least one of flash memories such as SRAM, DRAM, EPROM, EEPROM, and USB memory.

In the image pickup apparatus 100 configured in this way, while tracking a specific subject, the image pickup device 100 may focus on the specific subject or execute exposure control and white balance control. In such a case, as the distance to the subject increases, the area of interest (ROI) becomes smaller, and imaging control such as autofocus (AF), autoexposure (AE), or auto white balance (AWB) can be performed appropriately. It may not be.

Therefore, according to the image pickup apparatus 100 according to the present embodiment, the image pickup control such as AF control, AE control, or AWB control is adjusted according to the distance to the subject.

The image pickup control unit 110 derives the contrast evaluation value of the image by applying the contrast evaluation filter to the image captured by the image pickup device 100. The image pickup control unit 110 executes contrast AF by specifying the position of the focus lens to focus on a specific subject based on the contrast evaluation value. The image pickup control unit 110 executes contrast AF by specifying the position of the focus lens at which the contrast evaluation value peaks.

The contrast evaluation filter derives the contrast evaluation value of the image by multiplying the contrast evaluation value for each spatial frequency by the evaluation coefficient for each spatial frequency and adding up the contrast evaluation values obtained by multiplying the evaluation coefficient. A filter is fine. The contrast evaluation filter may be a filter that derives the contrast evaluation value of an image by weighting the contrast evaluation value for each spatial frequency for each spatial frequency and adding up the weighted contrast evaluation values. For example, as shown in FIG. 3A, the imaging control unit 110 may apply the first contrast evaluation filter 400 having the highest evaluation coefficient of the spatial frequency f ₀ to the image to derive the contrast evaluation value. The spatial frequency f ₀ is predetermined by the pixel pitch of the image sensor 120, the optical performance of the lens 210, and the like. The image 500 shows an image captured by the image pickup apparatus 100, and the frame in the image 500 shows the region of interest 510.

The UAV 10 equipped with the image pickup apparatus 100 moves away from the subject 600 along the imaging direction 550 of the imaging apparatus 100, for example, as shown in FIG. In this case, if the angle of view of the image pickup apparatus 100 is fixed, the size of the subject 600 in the image 500 becomes smaller as shown in FIG. 3B, and the area of interest 510 in the image 500 becomes smaller accordingly.

When the subject 600 in the image 500 becomes smaller, the spatial frequency of the image in the region of interest 510 becomes extremely small or large. Therefore, when the image pickup control unit 110 applies the contrast evaluation filter having the highest evaluation coefficient of the spatial frequency f ₀ to the image and derives the contrast evaluation value, the contrast evaluation value changes. Therefore, the contrast AF based on the contrast evaluation value becomes unstable.

Therefore, the image pickup control unit 110 shifts the central spatial frequency of the contrast evaluation filter according to the distance to the subject. The image pickup control unit 110 may shift the central spatial frequency of the contrast evaluation filter to the high frequency side as the distance to the subject increases. The image pickup control unit 110 shifts the spatial frequency having the highest evaluation coefficient according to the distance to the subject. The imaging control unit 110 may shift the spatial frequency having the highest evaluation coefficient to the high frequency side as the distance to the subject increases.

The image pickup control unit 110 applies the first contrast evaluation filter 400 to the first area of interest, which is the area of the subject selected by the image pickup device 100, in the first image captured by the image pickup device 100. , The first contrast evaluation value for evaluating the contrast value of the first interest region is derived. The first region of interest is an example of the first region. The image pickup control unit 110 may set the first area of interest by allowing the user to specify a subject from the first image captured by the image pickup device 100. The image pickup control unit 110 may identify a predetermined subject from the first image captured by the image pickup apparatus 100 by image matching according to a predetermined condition. The image pickup control unit 110 may set a region including the specified subject as the first region of interest. The first contrast evaluation filter 400 may filter has a peak of the evaluation factor to the first spatial frequency f ₀ is a spatial frequency determined in advance. The first contrast evaluation filter 400 has a bandwidth that includes a first spatial frequency _{f 0.}

The image pickup control unit 110 executes focusing control of the image pickup apparatus based on the first contrast evaluation value. The image pickup control unit 110 may execute the contrast AF control based on the first contrast evaluation value. The image pickup control unit 110 specifies the position of the focus lens at which the first contrast evaluation value peaks while moving the focus lens, and moves the focus lens to the specified position to execute focusing control. Good.

When the distance to the subject is relatively short, even if the distance to the subject changes, it is unlikely that the spatial frequency of the image in the region of interest 510 becomes extremely small or large. Therefore, when the distance to the subject is relatively short, it is not necessary to shift the central spatial frequency of the contrast evaluation filter according to the distance to the subject. Therefore, when the distance to the subject is included in the first distance range 701 from the closest to the first reference distance X ₁ , the image pickup control unit 110 sets the contrast evaluation filter to the first spatial frequency f ₀ without shifting the contrast evaluation filter. The first contrast evaluation filter 400 having a peak of the evaluation coefficient may be applied to derive the contrast evaluation value of the image.

Further, even when the distance to the subject is very long, even if the distance to the subject changes, it is unlikely that the spatial frequency of the image in the region of interest 510 becomes extremely small or large. In this case, the image pickup control unit 110 may fix the focus lens at a position where it is focused at infinity. Therefore, when the distance to the subject is included in the third distance range 703 that exceeds the predetermined second reference distance X ₂ , the image pickup control unit 110 may fix the focus lens at a position to focus at infinity. ..

The imaging control unit 110, if the distance to the object is included in the second distance range from the first reference distance X ₁ to the second reference distance X _2, according to the distance to the subject, the center spatial frequency of the contrast evaluation Filters May be shifted.

The UAV 10 moves along the imaging direction of the imaging device 100. The UAV 10 may move along the imaging direction of the imaging device 100 from the first distance to the second distance to the subject. The UAV 10 may increase or decrease from the first distance to the second distance along the image pickup direction of the image pickup apparatus 100. When the image pickup direction of the image pickup apparatus 100 is vertically downward, the UAV 10 may rise or fall in the vertical direction until the distance to the subject changes from the first distance to the second distance.

The image pickup control unit 110 is an image pickup device when the first image pickup device 100 captures the first image, the first distance between the image pickup device 100 and the subject 600, and the image pickup device 100 after the movement captures the second image. A second contrast evaluation filter is specified based on the second distance between the subject and the subject. When the distance to the subject after movement is included in the second distance range, the image pickup control unit 110 sets the first distance between the image pickup device 100 and the subject 600 when the image pickup device 100 captures the first image, and The second contrast evaluation filter 401 is specified based on the second distance between the image pickup device and the subject when the image pickup device 100 after the movement captures the second image. The image pickup control unit 110 may specify the second contrast evaluation filter based on the ratio of the first distance to the second distance and the information of the first contrast evaluation filter. The information of the first contrast evaluation filter may be information indicating the characteristics of the first contrast evaluation filter. The information of the first contrast evaluation filter may be information indicating the spatial frequency in which the peak of the evaluation coefficient is present. The first distance may be a predetermined first reference distance X ₁ . The second distance may be a distance X _C from the imaging device 100 after movement to the object. The image pickup control unit 110 may derive the second spatial frequency f ₁ by multiplying the first spatial frequency f ₀ by X _C / X ₁ . The second spatial frequency f ₁ is different from the first spatial frequency f ₀ . The imaging control unit 110, a second contrast evaluation filter 401 having a peak rating factor to the second spatial frequency f _1, which is derived may be identified. The image pickup control unit 110 may select the second contrast evaluation filter 401 so that the spatial frequency having the peak of the evaluation coefficient increases as the distance to the subject increases. The second contrast evaluation filter 401 having a band including a second spatial frequency _{f 1.}

When the distance to the subject is the second distance, the image pickup control unit 110 refers to the second area of interest, which is the area of the subject selected by the image pickup device 100, in the second image captured by the image pickup device 100. Then, by applying the second contrast evaluation filter 401, a second contrast evaluation value for evaluating the contrast value of the second region of interest is derived. The second region of interest is an example of the second region. The image pickup control unit 110 executes focusing control of the image pickup apparatus based on the second contrast evaluation value. The image pickup control unit 110 may execute the contrast AF control based on the second contrast evaluation value. The image pickup control unit 110 identifies the position of the focus lens at which the second contrast evaluation value peaks while moving the focus lens, and moves the focus lens to the specified position to execute focusing control. Good.

When the distance between the image pickup device 100 and the subject is longer than the second reference distance X ₂ , the image pickup control unit 110 may control the focus lens of the image pickup device so as to focus at infinity.

FIG. 5 is a flowchart showing an example of the AF control procedure by the image pickup control unit 110. The flowchart shown in FIG. 5 shows an AF control procedure when the imaging device 100 images the subject 600 while the UAV 10 moves away from the subject 600 along the imaging direction of the imaging device 100.

The image pickup control unit 110 identifies a region of interest in the image captured by the image pickup apparatus 100 (S100). The image pickup control unit 110 executes contrast AF by using a contrast evaluation filter corresponding to a predetermined first spatial frequency f ₀ (S102). Contrast evaluation filter corresponding to the first spatial frequency f ₀ is a filter having a peak of the evaluation factor to the first spatial frequency f _0. The contrast evaluation filter may be a filter that multiplies the contrast evaluation value for each spatial frequency by the evaluation coefficient for each spatial frequency and outputs the total of the multiplied contrast evaluation values as the contrast evaluation value of the image.

The UAV 10 then begins moving (S104). The UAV 10 may move in a direction away from the subject along the imaging direction of the imaging device 100. The image pickup control unit 110 acquires the current distance X _c to the subject (S106). The image pickup control unit 110 may acquire the distance from the distance measuring sensor 140 to the subject. The image pickup control unit 110 determines whether or not the current distance X _c is farther than the first reference distance X ₁ (S108).

If the current distance X _c is closer than the first reference distance X ₁ , the image pickup control unit 110 executes the normal contrast AF. That is, the imaging control unit 110, by using the contrast evaluation filter corresponding to the first spatial frequency f ₀ which is determined in advance, to perform a contrast AF.

On the other hand, if the current distance X _c is farther than the first reference distance X ₁ , the imaging control unit 110 is the product of the ratio of the first reference distance X ₁ and the distance X _c and the first spatial frequency f ₀ . The contrast evaluation filter corresponding to the two spatial frequencies f ₁ is specified (S110). The imaging control unit 110, by using the contrast evaluation filter corresponding to the second spatial frequency _{f 1,} to perform a contrast AF (S112).

The UAV 10 further moves along the imaging direction of the imaging device 100. The image pickup control unit 110 further acquires the current distance X _c to the subject (S114). The image pickup control unit 110 determines whether or not the distance X _c is farther than the second reference distance X ₂ (S116). If the distance X _c is closer than the second reference distance X ₂ , the imaging control unit 110 repeats the processes after step S110.

If the distance X _c is farther than the second reference distance X ₂ , the image pickup control unit 110 fixes the focus lens at a position where the focus lens is focused at infinity (S118).

As described above, according to the present embodiment, when the UAV 10 moves away along the imaging direction of the imaging device 100 and the imaging device 100 images a subject, AF control can be executed while keeping the contrast evaluation value constant.

The image pickup control unit 110 may also adjust the image pickup control other than the AF control according to the distance to the subject. The image pickup control unit 110 is the first evaluation value of the first area of interest based on the image information of the first area of interest, which is the area of the subject selected by the image pickup device, in the first image captured by the image pickup device 100. May be derived. The image pickup control unit 110 may execute the first image pickup control of the image pickup apparatus based on the first evaluation value. The image pickup control unit 110 performs a second evaluation of the second area of interest based on the image information of the second area of interest, which is the area of the subject selected by the image pickup device, in the second image in which the subject is imaged by the image pickup device. You may derive the value. The image pickup control unit 110 is a first distance between the image pickup device and the subject when the image pickup device captures the first image, and a second distance between the image pickup device and the subject when the image pickup device captures the second image. The second imaging control of the imaging apparatus may be executed based on the distance, the first evaluation value, and the second evaluation value.

The imaging control unit 110 derives a predicted evaluation value, which is a predicted evaluation value of the second region of interest, based on the first distance, the second distance, and the first evaluation value, and uses the predicted evaluation value and the second evaluation value. Based on this, the second imaging control may be executed by executing the imaging control of the imaging device. The imaging control unit 110 may derive a predicted evaluation value based on the ratio of the first distance to the second distance and the first evaluation value.

As the distance to the subject changes, the predicted evaluation values such as the predicted brightness and the predicted color temperature of the region of interest change. For example, the longer the distance, the lower the brightness. As the distance increases, the image becomes more bluish and the color temperature increases due to the influence of Rayleigh scattering, for example. The predicted evaluation value can be expressed as a function of distance in advance by experiments or the like. The image pickup apparatus 100 may store a function for each surrounding environment of the image pickup apparatus 100 in a memory 130 or the like.

The imaging control unit 110 may derive a first luminance evaluation value for evaluating the brightness of the first interest region as the first evaluation value. The image pickup control unit 110 may execute the exposure control of the image pickup apparatus based on the first luminance evaluation value. The image pickup control unit 110 may determine the first exposure control value based on the first luminance evaluation value. The image pickup control unit 110 may determine the aperture value and the exposure time based on the first exposure control value.

The image pickup control unit 110 may derive a second luminance evaluation value for evaluating the luminance of the second region of interest as the second evaluation value. The image pickup control unit 110 may execute the exposure control of the image pickup apparatus based on the first distance, the second distance, the first luminance evaluation value, and the second luminance evaluation value.

The imaging control unit 110 derives a predicted luminance evaluation value, which is a predicted luminance evaluation value of the second region of interest, based on the first distance, the second distance, and the first luminance evaluation value, and derives the predicted luminance evaluation value and the first luminance evaluation value. 2 Exposure control of the imaging device may be executed based on the brightness evaluation value. The imaging control unit 110 may derive a predicted luminance evaluation value based on the ratio of the first distance to the second distance and the first luminance evaluation value. The brightness evaluation value may be the brightness value of the image. When the distance to the subject is a predetermined distance (for example, the first reference distance X ₁ ), the brightness evaluation value of the image may be set to 1.

The image pickup control unit 110 may derive the first color temperature of the first region of interest as the first evaluation value. The image pickup control unit 110 may execute the white balance control of the image pickup apparatus based on the first color temperature. The image pickup control unit 110 may derive the second color temperature of the second region of interest as the second evaluation value. The image pickup control unit 110 may execute the white balance control of the image pickup apparatus based on the first distance, the second distance, the first color temperature, and the second color temperature. The imaging control unit 110 derives the predicted color temperature, which is the predicted color temperature of the second region of interest, based on the first distance, the second distance, and the first color temperature, and sets the predicted color temperature and the second color temperature. Based on this, the exposure control of the imaging device may be executed. The image pickup control unit 110 may derive the predicted color temperature based on the ratio of the first distance to the second distance and the first color temperature.

As the distance to the subject increases, the brightness of the subject in the image captured by the image pickup apparatus 100 decreases. The brightness decreases in proportion to the square of the distance to the subject. Therefore, the imaging control unit 110 predicts the luminance evaluation value as the predicted luminance evaluation value in consideration of the fact that the luminance becomes lower in proportion to the square of the distance to the subject, and the exposure control is performed based on the predicted luminance evaluation value. To execute.

For example, as shown in FIG. 6A, when the current distance to the subject is X _C1 , the luminance evaluation value is set to 1. As shown in FIG. 6B, when the current distance to the subject is X _C2 , the imaging control unit 110 predicts the predicted luminance evaluation value as 1 / (X _C2 / X _C1 ) ² .

FIG. 7 is a flowchart showing an example of the procedure of AE control by the image pickup control unit 110. The flowchart shown in FIG. 7 shows an AE control procedure when the imaging device 100 images a subject while the UAV 10 moves away along the imaging direction of the imaging device 100.

The image pickup control unit 110 identifies a region of interest in the image captured by the image pickup device 100 (S200). The UAV 10 moves along the imaging direction of the imaging device 100. The image pickup control unit 110 acquires the current distance X _c to the subject (S202). The image pickup control unit 110 identifies the first luminance evaluation value of the region of interest and executes AE (S204). The image pickup control unit 110 may specify the brightness value in the region of interest 510 as the first brightness evaluation value. The image pickup control unit 110 may specify the average luminance value in the region of interest 510 as the first luminance evaluation value. The image pickup control unit 110 executes AE by determining the first exposure control value based on the first luminance evaluation value of the region of interest and determining the aperture value and the exposure time based on the first exposure control value. You can do it.

Next, the image pickup control unit 110 further acquires the current distance X _c to the subject (S206). The image pickup control unit 110 determines whether or not the current distance X _c is farther than the first reference distance X ₁ (S208). If the current distance X _c is closer than the first reference distance X ₁ , the exposure control may be executed as usual based on the luminance value in the region of interest 510.

On the other hand, if the current distance X _c is farther than the first reference distance X ₁ , the imaging control unit 110 determines the first luminance evaluation value specified last time and the ratio of the previous distance X _c1 to the current distance X _C2. Based on this, the second luminance evaluation value is derived, and AE is executed based on the second luminance evaluation value (S210). The image pickup control unit 110 multiplies the previous first luminance evaluation value by the reciprocal (1 / (X _C2 / X _C1 ) ² ) of the squared ratio of the previous distance X _C1 and the current distance X _C2. Therefore, the second luminance evaluation value may be derived. The image pickup control unit 110 derives a predicted second luminance evaluation value in consideration of a change in the distance to the subject, and once executes AE based on the second luminance evaluation value. After that, the image pickup control unit 110 executes AE based on the brightness evaluation value of the region of interest of the image captured by the image pickup device 100. By executing the AE based on the predicted second brightness evaluation value, the brightness of the image captured by the image pickup apparatus 100 can be stabilized.

FIG. 8 is a flowchart showing an example of the procedure of AWB control by the imaging control unit 110. The flowchart shown in FIG. 8 shows the procedure of AWB control when the image pickup apparatus 100 images a subject while the UAV 10 moves away along the imaging direction of the image pickup apparatus 100.

The image pickup control unit 110 identifies a region of interest in the image captured by the image pickup device 100 (S300). The UAV 10 moves along the imaging direction of the imaging device 100. The image pickup control unit 110 acquires the current distance X _c to the subject (S302). The image pickup control unit 110 identifies the first color temperature of the region of interest and executes AWB (S304). The image pickup control unit 110 identifies the white (achromatic color) region and the color temperature in the region from the RGB integrated value in the region of interest. From the specified color temperature, the white balance (WB) coefficient to be multiplied by the pixel value of each RGB channel is determined in order to correct the change in the appearance of the subject due to the tint of the light source. As a result, the imaging control unit 110 may execute the AWB.

Next, the image pickup control unit 110 further acquires the current distance X _c to the subject (S306). The image pickup control unit 110 determines whether or not the current distance X _c is farther than the first reference distance X ₁ (S308). If the current distance X _c is closer than the first reference distance X ₁ , the AWB may be performed as usual based on the color temperature within the region of interest 510.

On the other hand, if the current distance X _c is farther than the first reference distance X ₁ , the imaging control unit 110 is based on the first color temperature identified last time and the ratio of the previous distance X _c1 to the current distance X _C2. Then, the second color temperature is derived, and AWB is executed based on the second color temperature (S310). The image pickup control unit 110 may derive the second color temperature according to a predetermined function with respect to the ratio of the previous distance X _c1 and the current distance X _C2 . The image pickup control unit 110 multiplies the first color temperature by a coefficient derived according to a predetermined function for the ratio of the previous distance X _c1 and the current distance X _C2 to obtain the second color. The temperature may be derived.

The image pickup control unit 110 derives the predicted second color temperature in consideration of the change in the distance to the subject, and once executes the AWB based on the second color temperature. After that, the image pickup control unit 110 executes the AWB based on the color temperature of the region of interest of the image captured by the image pickup device 100. By executing the AWB based on the predicted second color temperature, the white balance of the image captured by the image pickup apparatus 100 can be stabilized.

As described above, according to the image pickup apparatus 100 according to the present embodiment, it is possible to prevent the imaging control such as AF control, AE control, and AWB control from becoming unstable according to the change in the distance to a specific subject.

FIG. 9 shows an example of a computer 1200 in which a plurality of aspects of the present invention may be embodied in whole or in part. The program installed on the computer 1200 can cause the computer 1200 to function as an operation associated with the device according to an embodiment of the present invention or as one or more "parts" of the device. Alternatively, the program may cause the computer 1200 to perform the operation or one or more "parts". The program can cause the computer 1200 to perform a process or a step of the process according to an embodiment of the present invention. Such a program may be run by the CPU 1212 to cause the computer 1200 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212 and a RAM 1214, which are connected to each other by a host controller 1210. The computer 1200 also includes a communication interface 1222, an input / output unit, which are connected to the host controller 1210 via an input / output controller 1220. The computer 1200 also includes a ROM 1230. The CPU 1212 operates according to the programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit.

Communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by CPU 1212 in computer 1200. The ROM 1230 stores in it a boot program or the like executed by the computer 1200 at the time of activation, and / or a program depending on the hardware of the computer 1200. The program is provided via a computer-readable recording medium such as a CR-ROM, USB stick or IC card or network. The program is installed in RAM 1214 or ROM 1230, which is also an example of a computer-readable recording medium, and is executed by CPU 1212. The information processing described in these programs is read by the computer 1200 and provides a link between the program and the various types of hardware resources described above. The device or method may be configured to implement the operation or processing of information according to the use of the computer 1200.

For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and performs communication processing on the communication interface 1222 based on the processing described in the communication program. You may order. Under the control of the CPU 1212, the communication interface 1222 reads the transmission data stored in the transmission buffer area provided in the RAM 1214 or a recording medium such as a USB memory, and transmits the read transmission data to the network, or The received data received from the network is written to the reception buffer area or the like provided on the recording medium.

Further, the CPU 1212 makes the RAM 1214 read all or necessary parts of a file or a database stored in an external recording medium such as a USB memory, and executes various types of processing on the data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to an external recording medium.

Various types of information such as various types of programs, data, tables, and databases may be stored in recording media and processed. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval described in various parts of the present disclosure with respect to the data read from the RAM 1214, and is specified by the instruction sequence of the program. Various types of processing may be performed, including / replacement, etc., and the results are written back to the RAM 1214. Further, the CPU 1212 may search for information in a file, a database, or the like in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. Search for an entry that matches the condition from the plurality of entries, read the attribute value of the second attribute stored in the entry, and associate it with the first attribute that satisfies the predetermined condition. The attribute value of the second attribute obtained may be acquired.

The program or software module described above may be stored on a computer 1200 or in a computer readable storage medium near the computer 1200. Also, a recording medium such as a hard disk or RAM provided in a dedicated communication network or a server system connected to the Internet can be used as a computer-readable storage medium, thereby allowing the program to be transferred to the computer 1200 over the network. provide.

The execution order of each process such as operation, procedure, step, and step in the device, system, program, and method shown in the claims, the specification, and the drawing is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that such modified or improved forms may also be included in the technical scope of the present invention.

10 UAV
20 UAV main unit 30 UAV control unit 36 Communication interface 37 Memory 40 Propulsion unit 41 GPS receiver 42 Inertial measurement unit 43 Magnetic compass 44 Atmospheric pressure sensor 45 Temperature sensor 46 Humidity sensor 50 Gimbal 60 Imaging device 100 Imaging device 102 Imaging unit 110 Imaging control unit 120 Image sensor 130 Memory 200 Lens unit 210 Lens 212 Lens drive unit 214 Position sensor 220 Lens control unit 222 Memory 300 Remote control device 1200 Computer 1210 Host controller 1212 CPU
1214 RAM
1220 Input / Output Controller 1222 Communication Interface 1230 ROM

Claims (20)

First contrast evaluation that evaluates the contrast value of the first region by applying the first contrast evaluation filter to the first region, which is a region including the subject, in the first image captured by the imaging device. Derived the value,
Focus control of the image pickup apparatus is executed based on the first contrast evaluation value, and the focus control is executed.
The first distance between the image pickup device and the subject when the image pickup device captures the first image, and the first distance between the image pickup device and the subject when the image pickup device captures the second image. Identify the second contrast evaluation filter based on the two distances
By applying the second contrast evaluation filter to the second region in the second image, which is a region including the subject, a second contrast evaluation value for evaluating the contrast value of the second region is derived. And
A control device including a circuit configured to execute focusing control of the image pickup device based on the second contrast evaluation value. The control according to claim 1, wherein the first contrast evaluation filter has a band including a first spatial frequency, and the second contrast evaluation filter has a band including a second spatial frequency different from the first spatial frequency. apparatus. Identifying the second contrast evaluation filter
The control device according to claim 1, wherein the second contrast evaluation filter is specified based on the ratio of the first distance to the second distance and the information of the first contrast evaluation filter. The circuit is further configured to control the focus lens of the imaging device to focus at infinity when the distance between the imaging device and the subject is longer than a predetermined distance. Item 1. The control device according to item 1. The first evaluation value of the first region is derived based on the image information of the first region, which is the region including the subject, in the first image captured by the imaging device.
Based on the first evaluation value, the first imaging control of the imaging device is executed.
The second evaluation value of the second region is derived based on the image information of the second region, which is the region including the subject, in the second image in which the subject is captured by the imaging device.
The first distance between the image pickup device and the subject when the image pickup device captures the first image, and between the image pickup device and the subject when the image pickup device captures the second image. A control device including a circuit configured to execute a second image pickup control of the image pickup device based on a second distance, the first evaluation value, and the second evaluation value. Executing the second imaging control
Based on the first distance, the second distance, and the first evaluation value, a predicted evaluation value which is an evaluation value of the prediction of the second region is derived, and based on the predicted evaluation value and the second evaluation value. The control device according to claim 5, wherein the image pickup control of the image pickup device is executed. Derivation of the predicted evaluation value is
The control device according to claim 6, further comprising deriving the predicted evaluation value based on the ratio of the first distance to the second distance and the first evaluation value. Derivation of the first evaluation value includes deriving a first luminance evaluation value for evaluating the brightness of the first region as the first evaluation value.
Executing the first imaging control includes executing the exposure control of the imaging device based on the first luminance evaluation value.
Derivation of the second evaluation value includes deriving a second brightness evaluation value for evaluating the brightness of the second region as the second evaluation value.
Executing the second imaging control means executing the exposure control of the imaging device based on the first distance, the second distance, the first luminance evaluation value, and the second luminance evaluation value. The control device according to claim 6, which includes. Executing the second imaging control
Based on the first distance, the second distance, and the first brightness evaluation value, the predicted brightness evaluation value which is the predicted brightness evaluation value of the second region is derived, and the predicted brightness evaluation value and the second brightness evaluation value are derived. The control device according to claim 8, wherein the exposure control of the image pickup device is executed based on the brightness evaluation value. Derivation of the predicted brightness evaluation value is
The control device according to claim 9, further comprising deriving the predicted luminance evaluation value based on the ratio of the first distance to the second distance and the first luminance evaluation value. Derivation of the first evaluation value includes deriving the first color temperature of the first region as the first evaluation value.
Executing the first imaging control includes executing the white balance control of the imaging device based on the first color temperature.
Derivation of the second evaluation value includes deriving the second color temperature of the second region as the second evaluation value.
Executing the second image pickup control includes executing the white balance control of the image pickup apparatus based on the first distance, the second distance, the first color temperature, and the second color temperature. , The control device according to claim 6. Executing the second imaging control
Based on the first distance, the second distance, and the first color temperature, the predicted color temperature which is the predicted color temperature of the second region is derived, and based on the predicted color temperature and the second color temperature. The control device according to claim 11, further comprising performing exposure control of the image pickup device. Derivation of the predicted color temperature is
The control device according to claim 12, further comprising deriving the predicted color temperature based on the ratio of the first distance to the second distance and the first color temperature. The control device according to any one of claims 1 to 13.
An imaging device including an image sensor. A moving body that moves with the imaging device according to claim 14. The moving body according to claim 15, wherein the moving body moves along the imaging direction of the imaging device from the first distance to the second distance to the subject. The moving body is a flying body and
The moving body according to claim 16, wherein the flying object rises or falls from the first distance to the second distance along the imaging direction of the imaging device. A first contrast evaluation value that evaluates the contrast value of the first region by applying the first contrast evaluation filter to the first region, which is the region of the subject, in the first image captured by the imaging device. And the stage of deriving
Based on the first contrast evaluation value, the step of executing the focusing control of the image pickup apparatus and
The first distance between the image pickup device and the subject when the image pickup device captures the first image, and the first distance between the image pickup device and the subject when the image pickup device captures the second image. At the stage of identifying the second contrast evaluation filter based on the two distances,
The contrast value of the second region is evaluated by applying the second contrast evaluation filter to the second region, which is the region of the subject selected by the imaging device in the second image. 2 The stage of deriving the contrast evaluation value and
A control method for an imaging device including a step of executing focusing control of the imaging device based on the second contrast evaluation value. A step of deriving a first evaluation value of the first region based on the image information of the first region, which is a region including the subject, in the first image captured by the imaging device.
A step of executing the first imaging control of the imaging device based on the first evaluation value, and
A step of deriving a second evaluation value of the second region based on the image information of the second region, which is a region including the subject, in the second image in which the subject is captured by the imaging device.
The first distance between the image pickup device and the subject when the image pickup device captures the first image, and between the image pickup device and the subject when the image pickup device captures the second image. A control method including a step of executing a second image pickup control of the image pickup apparatus based on a second distance, the first evaluation value, and the second evaluation value. A program for operating a computer as a control device according to any one of claims 1 to 13.

Images