CN111602385B

CN111602385B - Specifying device, moving body, specifying method, and computer-readable recording medium

Info

Publication number: CN111602385B
Application number: CN201980008707.8A
Authority: CN
Inventors: 本庄谦一; 安田知长
Original assignee: SZ DJI Technology Co Ltd
Current assignee: SZ DJI Technology Co Ltd
Priority date: 2018-11-28
Filing date: 2019-11-11
Publication date: 2021-12-10
Anticipated expiration: 2039-11-11
Also published as: WO2020108284A1; CN111602385A; JP2020086242A; JP6641574B1

Abstract

A determination apparatus may include: and a first determination unit that determines a focus setting value of an imaging device mounted on the mobile body, a zoom setting value of the imaging device, and a moving speed of the mobile body at each of time points from the first time point to the second time point. The determining means may comprise: and a second specifying unit that specifies a third position that is a position of the moving object at a third time point prior to the first time point, based on a first position that is a position of the moving object at the first time point and a first speed that is a moving speed of the moving object at the first time point, so that the moving object can move at the first position at the first moving speed at the first time point.

Description

Specifying device, moving body, specifying method, and computer-readable recording medium

[ technical field ] A method for producing a semiconductor device

The present disclosure relates to a control device, a mobile body, a control method, and a program.

[ background of the invention ]

Patent document 1 describes: to provide a sliding zoom effect, the zoom function is automatically adjusted using image analysis in correspondence with the motion of the camera.

Patent document 1: japanese patent laid-open publication No. 2016-517639

[ summary of the invention ]

[ technical problem to be solved by the invention ]

It is desirable to make it possible for the image pickup apparatus to more simply capture an image having an effect of a slide zoom or the like.

[ MEANS FOR SOLVING PROBLEMS ] to solve the problems

An aspect of the present disclosure relates to a determination apparatus, which may include: and a first determination unit that determines a set value of focus of an imaging device mounted on the mobile body, a set value of zoom of the imaging device, and a moving speed of the mobile body at each of time points from the first time point to the second time point. The determining means may comprise: and a second specifying unit that specifies a third position that is a position of the moving object at a third time point prior to the first time point, based on the first position that is the position of the moving object at the first time point and a first moving speed that is a moving speed of the moving object at the first time point, so that the moving object can move at the first position at the first moving speed at the first time point.

The first determination section may determine a set value of focus, a set value of zoom, and a movement speed at each time point from the first time point to the second time point to change the zoom magnification of the image pickup apparatus from the first zoom magnification to the second zoom magnification and to change the focus distance of the image pickup apparatus from the first focus distance to the second focus distance during the first time point to the second time point.

The second zoom magnification may be n times the first zoom magnification. The second focus distance may be n times the first focus distance. The distance between the second position, which is the position of the mobile body from the first position to the second time point, may correspond to a difference between the first focus distance and the second focus distance.

The time required for the moving body to move from the first position to the second position at the first moving speed may be a first time, and the second position may be a position of the moving body at the second time point. The first determination unit may determine the focus setting value and the zoom setting value at each of time points from the first time point to the second time point so as to change the focal distance of the image pickup device from the first focal distance to the second focal distance and change the zoom magnification of the image pickup device from the first zoom magnification to the second zoom magnification during the movement of the movable body from the first position to the second position at the first movement speed, that is, during the first time.

The time required for the moving body to move from the first position to the second position at the first moving speed may be a first time, and the second position may be a position of the moving body at the second time point. The first determination unit may determine the focus setting value and the zoom setting value at each of time points from the first time point to the second time point so as to change the focal distance of the image pickup device from the first focal distance to the second focal distance during a period in which the moving body moves from the first position to the second position at the moving speed, that is, during a first time, and change the zoom magnification of the image pickup device from the first zoom magnification to the second zoom magnification during a second time shorter than the first time.

The first determination unit may determine that a moving speed of the moving body when the moving body moves from a fourth position, which is a position of the moving body at a fourth time point prior to the third time point, to the third position is smaller than the moving speed of the moving body when the moving body moves from the third time point to the first time point.

The fourth position may be the same as the first position.

The time required for the moving body to move from the first position to the second position at the first moving speed may be a first time, and the second position may be a position of the moving body at the second time point. The first determination unit may determine the focus setting value and the zoom setting value at each of time points from the first time point to the second time point to change the focal distance of the image pickup device from the first focal distance to the second focal distance and to change the zoom magnification of the image pickup device from the first zoom magnification to the second zoom magnification during the movement of the moving body from the first position to the second position at the movement speed, that is, during the first time. The second zoom magnification may be n times the first zoom magnification. The distance from the first position to the second position may be shorter than a distance n times the first focus distance.

A moving body according to an aspect of the present disclosure may be a moving body that includes the above-described determination device and an imaging device and moves. The moving body may include: and a first control unit that starts movement of the movable body from the third position to the first position, controls the movable body so that the speed of the movable body is maintained at the first movement speed from the first position to a second position at a second time point after the movable body is controlled so that the speed of the movable body becomes the first movement speed at the first time point. The moving body may include a second control section that performs control of changing a zoom magnification of the image pickup apparatus from the first zoom magnification to the second zoom magnification and changing a focus distance of the image pickup apparatus from the first focus distance to the second focus distance from the first time point to the second time point.

The first control unit may control the moving body such that the moving speed of the moving body becomes the first moving speed at the first position after the moving body is moved from the fourth position to the third position from a fourth time point prior to the third time point, and the fourth position is the position of the moving body at the fourth time point.

The movable body may include a notification section that notifies the outside that the movable body moves from the fourth position to the third position before moving from the first position to the second position.

The fourth position may be the same as the first position.

The first control unit moves the movable body from the fourth position to the third position at a moving speed slower than a moving speed of the movable body when the movable body moves from the third time point to the first time point.

A determination method according to an aspect of the present disclosure may include: and determining a set value of focus of an imaging device mounted on the mobile body, a set value of zoom of the imaging device, and a moving speed of the mobile body at each time point from the first time point to the second time point. The determination method may comprise the following stages: the position of the mobile body at a third time point before the first time point is determined based on a first position that is the position of the mobile body at the first time point and a first moving speed that is the moving speed of the mobile body at the first time point so that the mobile body can move at the first position at the first moving speed at the first time point.

The program according to one aspect of the present disclosure may be a program for causing a computer to function as the above-described determination device.

According to an aspect of the present disclosure, it is possible to make an image pickup apparatus more simply capture an image having an effect of a slide zoom or the like.

Moreover, the above summary is not exhaustive of all of the necessary features of the disclosure. Furthermore, sub-combinations of these feature sets may also constitute the invention.

[ description of the drawings ]

Fig. 1 is a diagram showing one example of the external appearance of an unmanned aerial vehicle and a remote operation device.

Fig. 2 is a diagram showing one example of functional blocks of an unmanned aerial vehicle.

Fig. 3 is a diagram showing one example of the positional relationship between the unmanned aerial vehicle and the subject.

Fig. 4 is a diagram illustrating one example of the relationship of the position of the focus lens and the position of the zoom lens.

Fig. 5 is a diagram for explaining a focal length of the lens system, a distance from an object side focal point to an object, and a distance from an image side focal point to an image plane.

Fig. 6 is a diagram showing one example of setting information representing focus setting values in association with a focal length and a focal distance.

Fig. 7 is a diagram illustrating one example of the relationship between the position of the focus lens and the position of the zoom lens.

Fig. 8 is a diagram showing one example of a relationship of a zoom tracking curve and a movement tracking curve.

Fig. 9A is a diagram illustrating an example of an image captured on the telephoto side by the image capturing apparatus.

Fig. 9B is a diagram illustrating an example of an image captured by the image pickup device at the wide-angle side.

Fig. 10A is a diagram illustrating an example of an image captured on the telephoto side by the image capturing apparatus.

Fig. 10B is a diagram illustrating an example of an image captured by the image pickup device at the wide-angle side.

Fig. 11A is a diagram illustrating an example of an image captured on the telephoto side by the image capturing apparatus.

Fig. 11B is a diagram illustrating an example of an image captured by the image pickup device at the wide-angle side.

Fig. 12A is a diagram for explaining a mode in which the imaging apparatus performs imaging by combining optical zooming and electronic zooming.

Fig. 12B is a diagram for explaining a mode in which the imaging apparatus performs imaging by combining optical zooming and electronic zooming.

Fig. 12C is a diagram for explaining a mode in which the imaging apparatus performs imaging by combining optical zooming and electronic zooming.

Fig. 13 is a diagram showing one example of a relationship between the position of the focus lens and the position of the zoom lens in the case where optical zooming and electronic zooming are combined.

Fig. 14 is a diagram illustrating one example of a change in the position of the focus lens in the case where electronic zooming is performed after optical zooming.

Fig. 15 is a diagram showing one example of the positional relationship between the unmanned aerial vehicle and the subject.

Fig. 16 is a diagram for explaining the image recording timing of the imaging device according to the movement of the unmanned aerial vehicle.

Fig. 17 is a diagram for explaining the image recording timing of the imaging device according to the movement of the unmanned aerial vehicle.

Fig. 18 is a diagram for explaining the image recording timing of the imaging device according to the movement of the unmanned aerial vehicle.

Fig. 19A is a diagram illustrating an example of an image captured by the image pickup device at the wide-angle side.

Fig. 19B is a diagram illustrating an example of an image captured on the telephoto side by the image capturing apparatus.

Fig. 20 is a flowchart showing one example of an image capturing process of the image capturing apparatus.

Fig. 21 is a diagram showing one example of the hardware configuration.

[ detailed description ] embodiments

The present disclosure will be described below with reference to embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, the combination of features illustrated in all embodiments is not necessarily essential to the inventive solution. It will be apparent to those skilled in the art that various changes and modifications can be made in the following embodiments. It is apparent from the description of the claims that such modifications or improvements can be included in the technical scope of the present disclosure.

The claims, the specification, the drawings, and the abstract of the specification contain matters to be protected by copyright. The copyright owner would not make an objection to the facsimile reproduction by anyone of the files, as represented by the patent office documents or records. However, in other cases, the copyright of everything is reserved.

Various embodiments of the present disclosure may be described with reference to flow diagrams and block diagrams, where a block may represent (1) a stage in a process of performing an operation or (2) a "part" of a device having a role in performing an operation. Certain stages and "sections" may be implemented by programmable circuits and/or processors. The dedicated circuitry may comprise digital and/or analog hardware circuitry. May include Integrated Circuits (ICs) and/or discrete circuits. The programmable circuitry may comprise reconfigurable hardware circuitry. The reconfigurable hardware circuit may include logical AND, logical OR, logical XOR, logical NAND, logical NOR, AND other logical operations, as well as storage elements such as flip-flops, registers, Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), AND the like.

The computer readable medium may comprise any tangible device that can store instructions for execution by a suitable device. As a result, a computer-readable medium having stored thereon instructions that are executable to create a means for implementing the operations specified in the flowchart or block diagram. As examples of the computer readable medium, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, and the like may be included. More specific examples of the computer-readable medium may include a floppy disk (registered trademark) disk, a floppy disk, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Static Random Access Memory (SRAM), a compact disc read only memory (CD-ROM), a Digital Versatile Disc (DVD), a blu-Ray (RTM) disc, a memory stick, an integrated circuit card, and so forth.

Computer readable instructions may include any one of source code or object code described by any combination of one or more programming languages. The source code or object code comprises a conventional procedural programming language. Conventional procedural programming languages may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or Smalltalk, JAVA (registered trademark), C + +, or the like, and the "C" programming language, or similar programming languages. The computer readable instructions may be provided to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing apparatus, either locally or via a Wide Area Network (WAN), such as a Local Area Network (LAN), the internet, or the like. A processor or programmable circuit may execute the computer readable instructions to create means for implementing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

Fig. 1 shows one example of the appearance of an Unmanned Aerial Vehicle (UAV)10 and a remote operation device 300. The UAV10 includes a UAV main body 20, a universal joint 50, a plurality of imaging devices 60, and an imaging device 100. The gimbal 50 and the image pickup apparatus 100 are one example of an image pickup system. The UAV10, i.e., a mobile body, is a concept including a flight vehicle moving in the air, a vehicle moving on the ground, a ship moving on water, and the like. The flying body moving in the air refers to a concept including not only the UAV but also other aircrafts, airships, helicopters, and the like moving in the air.

The UAV main body 20 includes a plurality of rotors. Multiple rotors are one example of a propulsion section. The UAV body 20 flies the UAV10 by controlling the rotation of the plurality of rotors. The UAV body 20 uses, for example, four rotors to fly the UAV 10. The number of rotors is not limited to four. Alternatively, the UAV10 may be a fixed wing aircraft without a rotor.

The imaging apparatus 100 is an imaging camera that captures an object included in a desired imaging range. The gimbal 50 rotatably supports the image pickup apparatus 100. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 rotatably supports the image pickup apparatus 100 with a pitch axis using an actuator. The gimbal 50 further rotatably supports the image pickup apparatus 100 centered on the roll axis and the yaw axis, respectively, using the actuators. The gimbal 50 may change the attitude of the image pickup apparatus 100 by rotating the image pickup apparatus 100 around at least one of a yaw axis, a pitch axis, and a roll axis.

The plurality of imaging devices 60 are sensing cameras for imaging the surroundings of the UAV10 in order to control the flight of the UAV 10. Two cameras 60 may be provided at the nose, i.e., front, of the UAV 10. Also, two other cameras 60 may be provided on the bottom surface of the UAV 10. The two image pickup devices 60 on the front side may be paired to function as a so-called stereo camera. The two imaging devices 60 on the bottom surface side may also be paired to function as a stereo camera. Three-dimensional spatial data around the UAV10 may be generated based on images taken by the plurality of cameras 60. The number of the imaging devices 60 provided in the UAV10 is not limited to four. The UAV10 may include at least one imaging device 60. The UAV10 may also include at least 1 imaging device 60 on the nose, tail, sides, bottom, and top of the UAV 10. The angle of view that can be set in the image pickup device 60 can be larger than that which can be set in the image pickup device 100. The imaging device 60 may also have a single focus lens or a fisheye lens.

The remote operation device 300 communicates with the UAV10 to remotely operate the UAV 10. The remote operation device 300 may be in wireless communication with the UAV 10. The remote operation device 300 transmits instruction information indicating various instructions related to the movement of the UAV10, such as ascending, descending, accelerating, decelerating, advancing, retreating, and rotating, to the UAV 10. The indication includes, for example, an indication to raise the altitude of the UAV 10. The indication may show the altitude at which the UAV10 should be located. The UAV10 moves to be located at an altitude represented by the indication received from the remote operation device 300. The indication may include a lift instruction to lift the UAV 10. The UAV10 ascends while receiving the ascending instruction. When the altitude of UAV10 has reached the upper limit altitude, UAV10 may be restricted from ascending even if an ascending command is received.

Fig. 2 shows one example of the functional blocks of the UAV 10. The UAV10 includes a UAV control unit 30, a memory 37, a communication interface 36, a propulsion unit 40, a GPS receiver 41, an inertial measurement unit 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60, and an imaging device 100.

The communication interface 36 communicates with other devices such as the remote operation device 300. The communication interface 36 may receive instruction information including various instructions to the UAV control 30 from the remote operation device 300. The memory 37 stores programs and the like necessary for the UAV control unit 30 to control the propulsion unit 40, the GPS receiver 41, the Inertial Measurement Unit (IMU)42, the magnetic compass 43, the barometric altimeter 44, the temperature sensor 45, the humidity sensor 46, the universal joint 50, the imaging device 60, and the imaging device 100. The memory 37 may be a computer-readable recording medium, and may include at least one of SRAM, DRAM, EPROM, EEPROM, USB memory, and flash memory such as Solid State Disk (SSD). The memory 37 may be disposed inside the UAV body 20. Which may be removably disposed on the UAV body 20.

The UAV control unit 30 controls the flight and shooting of the UAV10 according to a program stored in the memory 37. The UAV control 30 is one example of a control device. The UAV control unit 30 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The UAV control unit 30 controls the flight and shooting of the UAV10 in accordance with instructions received from the remote operation device 300 via the communication interface 36. The propulsion portion 40 propels the UAV 10. The propulsion unit 40 includes a plurality of rotors and a plurality of drive motors for rotating the rotors. The propulsion unit 40 rotates the plurality of rotors via the plurality of drive motors in accordance with instructions from the UAV control unit 30 to fly the UAV 10.

The GPS receiver 41 receives a plurality of signals showing times transmitted from a plurality of GPS satellites. The GPS receiver 41 calculates the position (latitude and longitude) of the GPS receiver 41, that is, the position (latitude and longitude) of the UAV10, based on the plurality of received signals. The IMU42 detects the pose of the UAV 10. The IMU42 detects the acceleration of the UAV10 in the three-axis directions of the front-back, left-right, and up-down directions, and the angular velocity of the UAV10 in the three-axis directions of the pitch axis, roll axis, and yaw axis. The magnetic compass 43 detects the orientation of the nose of the UAV 10. The barometric altimeter 44 detects the altitude of the UAV 10. The barometric altimeter 44 detects the barometric pressure around the UAV10 and converts the detected barometric pressure into altitude to detect altitude. The temperature sensor 45 detects the temperature around the UAV 10. The humidity sensor 46 detects the humidity around the UAV 10.

The imaging device 100 includes an imaging unit 102 and a lens unit 200. The image pickup apparatus 100 may have an electronic zoom function in addition to the optical zoom. The image pickup apparatus 100 may have at least one of an optical zoom function and an electronic zoom function. The lens part 200 is one example of a lens apparatus. The imaging unit 102 includes an image sensor 120, an imaging control unit 110, and a memory 130. The image sensor 120 may be formed of a CCD or a CMOS. The image sensor 120 captures an optical image imaged via the lens section 200, and outputs the captured image to the image capture control section 110. The imaging control unit 110 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The imaging control unit 110 may control the imaging apparatus 100 in accordance with an operation command of the imaging apparatus 100 from the UAV control unit 30. The image pickup control part 110 may enlarge an image output from the image sensor 120 and cut out a part of the image, thereby implementing an electronic zoom.

The memory 130 may be a computer-readable recording medium and may include at least one of SRAM, DRAM, EPROM, EEPROM, USB memory, and flash memory such as Solid State Disk (SSD). The memory 130 stores programs and the like necessary for the imaging control unit 110 to control the image sensor 120 and the like. The memory 130 may be provided inside the housing of the image pickup apparatus 100. The memory 130 may be provided to be detachable from the housing of the image pickup apparatus 100.

The lens part 200 includes a focus lens 210, a zoom lens 211, a lens driving part 212, a lens driving part 213, and a lens control part 220. Focus lens 210 is one example of a focus lens system. The zoom lens 211 is one example of a zoom lens system. The focus lens 210 and the zoom lens 211 may include at least one lens. At least a part or all of the focus lens 210 and the zoom lens 211 are configured to be movable along the optical axis. The lens portion 200 may be an interchangeable lens provided to be attachable to and detachable from the image pickup portion 102. The lens driving section 212 moves at least a part or all of the focus lens 210 along the optical axis via a mechanism member such as a cam ring, a guide shaft, or the like. The lens driving section 213 moves at least a part or all of the zoom lens 211 along the optical axis via a mechanism member such as a cam ring, a guide shaft, or the like. The lens control section 220 drives at least one of the lens driving section 212 and the lens driving section 213 in accordance with a lens control instruction from the image pickup section 102, and moves at least one of the focus lens 210 and the zoom lens 211 in the optical axis direction by a mechanism member to perform at least one of a zooming action and a focusing action. The lens control command is, for example, a zoom control command and a focus control command.

Lens portion 200 also includes memory 222, position sensor 214, and position sensor 215. The memory 222 stores control values of the focus lens 210 and the zoom lens 211 moved via the lens driving section 212 and the lens driving section 213. The memory 222 may include at least one of SRAM, DRAM, EPROM, EEPROM, USB memory, and other flash memories. The position sensor 214 detects the lens position of the focus lens 210. The position sensor 214 may detect the current focus position. The position sensor 215 detects the lens position of the zoom lens 211. The position sensor 215 may detect a current zoom position of the zoom lens 211.

In the imaging apparatus 100 mounted on the UAV10 as described above, while the UAV10 is moving, a zoom function of the imaging apparatus 100 is used to provide a moving image with a slide zoom effect such as changing the size of the background on the image plane while maintaining the size of the subject of interest on the image plane.

The UAV control section 30 includes an acquisition section 31, a determination section 32, and a determination section 33. The acquisition section 31 acquires a time T required to change the zoom magnification of the image pickup apparatus 100 from the first zoom magnification to the second zoom magnification, the first zoom magnification, and the second zoom magnification. The acquisition section 31 may acquire the time, the first zoom magnification, and the second zoom magnification that are stored in advance in the memory 130, the memory 37, or the like. The acquisition section 31 may acquire the time T, the first zoom magnification, and the second zoom magnification designated by the user via the remote operation device 300.

The zoom magnification may be an optical zoom magnification, an electronic zoom magnification, or a magnification that combines an optical zoom magnification and an electronic zoom magnification. The optical zoom magnification refers to a magnification from the wide-angle end. The electronic zoom magnification refers to the magnification of the image output from the image sensor 120.

The determination unit 32 determines a focus setting value of the imaging apparatus 100, a zoom setting value of the imaging apparatus 100, and a movement speed of the UAV10 at each time point from the first time point to the second time point based on the time T, the first zoom magnification, and the second zoom magnification. The determination section 32 may further determine the focus setting value of the imaging apparatus 100, the zoom setting value of the imaging apparatus, and the moving speed of the UAV10 at each time point from the first time point to the second time point based on the information indicating the first focal distance of the imaging apparatus 100 at the first time point and the information indicating the second focal distance thereof at the second time point. Here, the information indicating the first focal distance includes at least one of a distance to an object brought into focus at the first time point in the image pickup apparatus 100 and a position of the focus lens 210 for bringing the object into focus at the first time point. The information indicating the second focusing distance includes at least one of a distance from the image pickup apparatus 100 to the object brought into a focused state at the second point in time and a position of the focus lens 210 at which the object is brought into a focused state at the second point in time. The in-focus state refers to, for example, a state in which an evaluation value of contrast of an object in an image is a predetermined value or more.

For example, the first zoom magnification is 2 times, and the second zoom magnification is 1 time. As shown in fig. 3, the zoom magnification of the image pickup apparatus 100 at the first time point is set to 2 times, and the distance from the image pickup apparatus 100 to the object 500 (first focal distance) is set to L1. Then, the UAV10 is moved in the imaging direction so that the size of the object 500 at 2 times on the image plane matches the size of the object 500 at 1 time on the image plane. In this case, since the zoom magnification of the image pickup apparatus 100 at the second time point is 1 time, the distance from the image pickup apparatus 100 to the object 500 at the second time point (the second focus distance) is L2 (L1/2). That is, the UAV10 may move in the imaging direction by the difference between the first focal distance and the second focal distance (L1-L2 — L1).

The image pickup apparatus 100 moves the zoom lens 211 from the first time point to the second time point to change the zoom magnification from 2 times to 1 time. In addition, the image pickup apparatus 100 changes the focus distance of the focus lens 210 from the first focus distance to the second focus distance from the first time point to the second time point. The first focus distance corresponds to a distance from the image pickup apparatus 100 to a first focus position that should be focused at a first time point. The second focus distance corresponds to a distance from the image pickup apparatus 100 to a second focus position at which focusing should be performed at the second time point. Note that the image pickup apparatus 100 may be moved away from the object 500 without being moved close to the object 500. In this case, for example, the first zoom magnification is 1 time and the second zoom magnification is 2 times.

The image capturing apparatus 100 can perform shooting in such a manner that the in-focus state of a single object that is still is maintained from the first time point to the second time point. In this case, the first focus position is the same as the second focus position. The image pickup apparatus 100 can perform image pickup in such a manner that it focuses on a first object at a first time point and focuses on a second object having a distance from the image pickup apparatus 100 different from the first object at a second time point. In this case, the first focus position is different from the second focus position.

The determination section 32 determines the movement speed of the UAV10 required to move the UAV10 within the time T period by the difference of the second focal distance and the first focal distance.

The determination section 32 may determine the focus setting value of the image pickup apparatus 100 and the zoom setting value of the image pickup apparatus 100 at each time point from the first time point to the second time point based on first information indicating a relationship between the position of the focus lens and the position of the focus lens in the first focus distance and second information indicating a relationship between the position of the focus lens and the position of the focus lens in the second focus distance.

The determination section 32 may determine the focus setting value of the image pickup apparatus 100 and the zoom setting value of the image pickup apparatus 100 at each time point from the first time point to the second time point based on a so-called zoom tracking curve. For example, as shown in fig. 4, the determination section 32 may determine a movement tracking curve 603 representing the focus setting value of the image pickup apparatus 100 and the zoom setting value of the image pickup apparatus 100 at each time point from the first time point to the second time point, based on a zoom tracking curve 602 corresponding to the infinity-side focus distance of the first focus distance and a zoom tracking curve 601 corresponding to the closest-side focus distance of the second focus distance. The imaging control unit 110 outputs a zoom operation command and a focus operation command to the lens control unit 220 to control the position of the zoom lens and the position of the focus lens from the first time point to the second time point in accordance with a movement tracking curve 603 shown in fig. 4.

The determination section 32 may acquire data of a zoom tracking curve for each focus distance stored in the memory 222 of the lens section 200 and determine a movement tracking curve indicating a focus setting value of the image pickup apparatus 100 and a zoom setting value of the image pickup apparatus 100 at each time point from the first time point to the second time point based on the acquired data.

In the case where the imaging device 100 captures a dynamic image that produces a so-called slide zoom effect, the UAV10 may fly in the imaging direction of the imaging device 100 from the first time point to the second time point. The image pickup control section 110 may control the zoom lens 211 and the focus lens 210 from the first time point to the second time point so that the image pickup apparatus 100 maintains the size of the object on the image plane at the first location in the in-focus state at the first time point and the state of focusing on the object at the first location. The determination section 32 may determine the focus setting value of the image pickup apparatus 100 at each time point from the first time point to the second time point.

Fig. 5 is a diagram representing the focus lens 210 and the zoom lens 211 with one lens system L. H denotes the principal point of the image of the lens system L. F1 denotes the object side focus of the lens system L. F2 denotes the image side focus of the lens system L. F denotes a focal length, which is a distance from the image principal point H to the object side focus F1 or the image side focus F2. a denotes a distance from the object side focus F1 to the object 500. b denotes a distance from the image side focal point F2 to the image plane 121. In this case, according to the newton imaging formula, a, b, and f satisfy the following relationship. a. b and f are real numbers, respectively.

b＝f²×(1/a)

Setting a first zoom magnification at a first time point to Z₁Setting a second zoom magnification at a next time point following the first zoom magnification as Z₂A first zoom magnification Z₁And a second zoom magnification Z₂The ratio is set as n ═ Z₁/Z₂. The zoom magnification is set from the first zoom magnification Z in the imaging apparatus 100₁Change to the second zoom magnification Z₂In the case of (3), the UAV control unit 30 controls the UAV10 so that the distance a becomes a distance n × a. The imaging control unit 110 controls the zoom lens 211 via the lens control unit 220 so that the focal length f becomes the focal length n × f.

Second zoom magnification Z₂The distance b' of time is expressed as follows according to the newton imaging formula.

b’＝(n×f)²×(1/(n×a))＝n×f²×(1/a)＝n×b

That is, when the imaging apparatus 100 changes the zoom magnification from the first zoom magnification Z1 to the second zoom magnification Z2, the imaging controller 110 may control the focus lens 210 so that the distance from the image-side focus of the lens system L to the image plane becomes n × b. In a case where the distance a from the object-side focus of the lens system L to the object becomes n × a, the UAV control section 30 may control the zoom lens 211 and the focus lens 210 via the lens control section 220 so as to set the focal length f of the lens system L to n × f and set the distance b from the image-side focus of the lens system L to the image plane to n × b. The UAV control unit 30 can control the zoom lens 211 to set the focal length of the lens system L to n × f, and control the focus lens 210 to set the distance from the image-side focal point of the lens system L to the image plane to n × b. Thus, the image pickup apparatus 100 can capture an object while maintaining the size of the object on the image plane at the first location in the in-focus state at the first time point and the state of focusing on the object at the first location.

The determination unit 32 may determine a focus setting value and a zoom setting value for setting the focal length f of the lens system L to n × f and setting the distance b from the image-side focus of the lens system L to the image plane to n × b based on the time T required to change the zoom magnification of the imaging apparatus 100 from the first zoom magnification corresponding to the focal length f to the second zoom magnification corresponding to the focal length n × f, the first zoom magnification, the second zoom magnification, information indicating the distance a, and information indicating the distance n × a.

The determination section 32 may further determine a focus setting value and a zoom setting value for setting the focal length f of the lens system L to n × f and setting the distance b from the image side focus of the lens system L to the image plane to n × b based on first information indicating the relationship between the position of the focus lens 211 and the position of the focus lens 210 in the distance a and second information indicating the relationship between the position of the focus lens and the position of the focus lens in the distance n × a.

The determination section 32 may further determine a focus setting value and a zoom setting value for setting the focal length f of the lens system L to n × f and setting the distance b from the image-side focus of the lens system L to the image plane to n × b based on first information indicating a relationship between the position of the zoom lens 211 and the position of the focus lens 210 in the distance a and second information indicating a relationship between the position of the zoom lens 211 and the position of the focus lens 210 in the distance n × a.

The distance a may correspond to a distance from the image pickup apparatus 100 to a first focus position that should be focused at a first time point. The distance n × a may correspond to a distance from the image pickup apparatus 100 to a second focus position at which focusing should be performed at the second point in time. In this case, the determination section 32 may determine a focus setting value and a zoom setting value for setting the focal length f of the lens system L to n × f and setting the distance b from the image-side focus of the lens system L to the image plane to n × b so that the size of the object on the image plane at the first focus position photographed by the image pickup apparatus 100 at the first time point and the size of the object on the image plane at the second focus position photographed by the image pickup apparatus 100 at the second time point satisfy a predetermined condition. The predetermined condition may be a condition that a size of an object on the image plane at a first focus position photographed by the image pickup apparatus 100 at a first point in time coincides with a size of an object on the image plane at a second focus position photographed by the image pickup apparatus 100 at a second point in time.

The lens system L is actually constituted by a plurality of lens groups functioning as the zoom lens 211 or the focus lens 210. When the position of the zoom lens 211 changes, the distance b from the image-side focal point of the lens system L to the image plane may also change. According to the change in the position of the zoom lens 211, the image pickup apparatus 100 can change the position of the focus lens 210 so that the focal distance is not shifted according to the change in the position of the zoom lens 211. That is, the image pickup apparatus 100 can perform so-called zoom tracking control.

The determination section 32 may determine the focus setting value of the focus lens 210 for setting the distance from the image side focus of the lens system L to the image plane to n × b, from setting information indicating the focus setting value of the focus lens 210 in association with information corresponding to the focal length of the lens system L and information corresponding to the distance a (focusing distance) from the object side focus of the lens system L to the object. The setting information may be information referred to when the imaging control unit 110 performs zoom tracking control.

Fig. 6 shows one example of the setting information. The focus distance d0 represents, for example, an infinite end. The focus distance d8 represents the most proximal end. The setting information may represent the number of pulses of a stepping motor for driving the focus lens 210 as the setting value S of the focus lens 210 in association with the focal length and the distance a. The distance (range) represents a moving amount r of the focus lens 210 in a case where the focus lens 210 is moved from the infinity side to the closest side at a specific focal length (zoom magnification). The amount of movement of focus lens 210 when focus lens 210 is moved from the infinity side to the closest side varies depending on the magnitude of the focal length (zoom magnification). The memory 130 may also store setting information as shown in fig. 6. The memory 130 may store a focus setting value of the focus lens 210 for each focal length corresponding to a specific focus distance as setting information. In this case, the determination unit 32 may derive the focus setting value of the focus lens 210 for a specific focal length (zoom magnification) among other focal distances, at a time, based on the setting information. The memory 130 may store, for example, a focus setting value for each focal length corresponding to the infinity end, that is, information corresponding to a zoom tracking curve for the infinity end as setting information. The determination unit 32 may derive the focus setting value of the focus lens 210 at a specific focal length (zoom magnification) at each time among other focal distances, based on the setting information corresponding to the zoom tracking curve at the infinity end.

Fig. 7 shows a set of so-called zoom tracking curves representing setting information in a two-dimensional manner. The lower limit boundary indicated by a symbol 610 corresponds to a zoom tracking curve indicating the relationship between the position (zoom setting value) of the zoom lens 211 and the position (focus setting value) of the focus lens 210 in the case where the focus distance is an infinite end, and the upper limit boundary indicated by a symbol 611 corresponds to a zoom tracking curve indicating the relationship between the position (zoom setting value) of the zoom lens 211 and the position (focus setting value) of the focus lens 210 in the case where the focus distance is the most proximal end. The width 620 between the zoom tracking curve 610 and the zoom tracking curve 611 corresponds to a movement amount r of the focus lens 210 when the focus lens 210 is moved from the infinity side to the closest side at each zoom magnification.

From the first time point to the second time point, the image pickup apparatus 100 changes the zoom magnification of the lens system from the first zoom magnification to the second zoom magnification. In this case, the first zoom magnification is set to Z₁Setting the second zoom magnification as Z₂The ratio of the first zoom magnification to the second zoom magnification is set to be n ═ Z₂/Z₁. A first zoom magnification Z₁The focus setting value of the focus lens 210 at that time is set to S₁Second zoom magnification Z₂The focus setting value of the focus lens 210 at that time is set to S₂. Further, the zoom magnification Z at the first zoom magnification determined based on the setting information shown in fig. 7 referred to in the zoom tracking control₁The amount of movement (distance) of focus lens 210 when focus lens 210 is moved from the infinity side to the closest side is r₁. Further, at a second zoom magnification Z to be determined based on this setting information₂The amount of movement of focus lens 210 when focus lens 210 is moved from the infinity side to the closest side is r₂。

In this case, the determination section 32 may be based on n, r₁、r₂And S₁To determine S₂. Here, the focus distance at the first time point is d₁Will be in focus by a distance d₁Is set to P₁Setting the focusing distance at the second time point as d₂Will be in focus by a distance d₂Is set to P₂Let the most proximal focus distance be d_nThe constant is set to div. Here, n and r₁、r₂、S₁、S₂、d₁、d₂、P₁、P₂、d_nAnd div is a real number.

In this case, the following formula can be defined:

P₁＝diV(S₁/r₁)…(1)

P₁＝diV(d_n/d₁)…(2)

P₂＝diV(S₂/r₂)…(3)

P₂＝diV(d_n/d₂)…(4)

when the formula (3) is modified, S is obtained₂＝(r₂×P₂)/div…(5)。

If formula (4) is substituted into formula (5),

then becomes S₂＝r₂×div(d_n/d₂)/div＝(d_n/d₂)×r₂…(6)。

Due to d₂＝n×d₁So that formula (6) becomes

S₂＝d_n/(n×d₁)×r₂…(7)。

And, d_n/d₁The formula (1) and the formula (2) are changed into

d_n/d₁＝S₁/r₁…(8)。

If formula (8) is substituted into formula (7),

then becomes S₂＝(1/n)×(r₂/r₁)×S₁…(9)。

Therefore, the temperature of the molten metal is controlled,the determination section 32 may be as S₂＝(1/n)×(r₂/r₁)×S₁To determine S₂. The UAV control 30 may instruct the imaging device 100 to follow S₂The focus lens 210 is controlled. The UAV control section 30 may control the focus lens 210 via the lens control section 220 so that n, r₁、r₂、S₁And S₂The relationship therebetween satisfies a predetermined condition. The UAV control 30 may open the lens control 220 to control the focusing lens 210 to satisfy S₂＝(1/n)×(r₂/r₁)×S₁。

Fig. 8 shows examples of a zoom tracking curve by a focal distance, a movement tracking curve 630 in a case where the focal distance is changed from 1.0m to 2.0m, and a movement tracking curve 631 in a case where the focal distance is changed from 2.0m to 4.0 m. During the movement of the image pickup apparatus 100, the lens control section 220 may control the focus lens 210 and the zoom lens 211 according to, for example, the movement tracking curve 630 or the movement tracking curve 631.

The determination section 32 may determine the focus setting value of the image pickup apparatus 100, the zoom setting value of the image pickup apparatus 100, and the moving speed of the UAV10 at each time point from the first time point to the second time point such that the size of the object on the image plane at the first in-focus position captured by the image pickup apparatus 100 at the first time point and the size of the object on the image plane at the second in-focus position captured by the image pickup apparatus 100 at the second time point satisfy a predetermined condition. The predetermined condition may be a condition that a size of an object on the image plane at a first focus position photographed by the image pickup apparatus 100 at a first point in time coincides with a size of an object on the image plane at a second focus position photographed by the image pickup apparatus 100 at a second point in time.

The image capturing apparatus 100 can perform shooting so as to approach an object from a first time point to a second time point. In the case where the first focus position is the same as the second focus position, the image pickup apparatus 100 can perform shooting while moving relative to the object such that the first focus distance is longer than the second focus distance. In this case, the image pickup apparatus 100 captures, for example, an image 700 as shown in fig. 9A at a first time point at a first focus distance and a first zoom magnification, and captures an image 701 as shown in fig. 9B at a second time point at a second focus distance and a second zoom magnification smaller than the first zoom magnification. Thus, the moving image captured from the first time point to the second time point includes an appearance that the size of the background on the image plane is changed while the size of the object 500 of interest on the image plane is maintained.

In the case where the first focus position is different from the second focus position, the determination section 32 may determine the focus setting value of the image pickup apparatus 100, the zoom setting value of the image pickup apparatus 100, and the moving speed of the UAV10 at each time point from the first time point to the second time point so that the size of the object on the image plane at the first focus position captured by the image pickup apparatus 100 at the first time point and the size of the object on the image plane at the second focus position captured by the image pickup apparatus 100 at the second time point satisfy a predetermined condition. Under such a condition, a moving image captured from the first time point to the second time includes an expression of a state in which a background is in focus on a first object of interest existing at the first focus position at the first time point while changing in size on the image plane to a state in which a background is in focus on a second object of interest existing at the second focus position at the second time point.

The first subject of interest may also be the same as the second subject of interest. That is, the object of interest existing at the first focus position at the first time point may also move to the second focus position at the second time point. For example, the image pickup apparatus 100 captures an image 710 including the object 500 in an in-focus state as illustrated in fig. 10A at a first time point at a first focus distance and a first zoom magnification. An image 711 including the subject 500 in an in-focus state as illustrated in fig. 10B is captured at a second time point at a second focus distance and a second zoom magnification smaller than the first zoom magnification. Thus, the moving image captured from the first time point to the second time point includes the appearance that the size of the object 500 moving in the period from the first time point to the second time point on the image plane is maintained while the size of the background on the image plane is changed.

In a case where the first focus position is different from the second focus position, the determination section 32 may determine the focus setting value of the image pickup apparatus 100, the zoom setting value of the image pickup apparatus 100, and the moving speed of the UAV10 at each time point from the first time point to the second time point such that the size of the object on the image plane at the first focus position captured by the image pickup apparatus 100 at the first time point and the size of the object on the image plane at the position corresponding to the first focus position captured by the image pickup apparatus 100 at the second time point satisfy a predetermined condition.

The predetermined condition in this case may be a condition that the size of the object on the image plane at the first focal position captured by the image capturing apparatus 100 at the first time point coincides with the size of the object on the image plane at the position corresponding to the first focal position captured by the image capturing apparatus 100 at the second time point. Under such a condition, a moving image captured from the first time point to the second time point includes an appearance that the size of the background on the image plane is changed while maintaining the size of the object of interest existing at the first in-focus position on the image plane. The moving image includes a representation that at a first point in time, a subject of interest at a first focus position enters an in-focus state, and at a second point in time, another subject of interest existing at a second focus position enters the in-focus state. The image capturing apparatus 100 captures an image 720 including the object 500 brought into focus and the object 501 in focus as shown in fig. 11A at a first focal distance and a first zoom magnification, for example, at a first time point. Further, an image 721 including the object 500 in focus and the object 501 not in focus as shown in fig. 11B is photographed at a second time point at a second focus distance and a second zoom magnification smaller than the first zoom magnification.

In the case where the first focus position is different from the second focus position, the determination section 32 may determine the focus setting value of the image pickup apparatus 100, the zoom setting value of the image pickup apparatus 100, and the moving speed of the UAV10 at each time point from the first time point to the second time point such that the size of the object on the image plane at the position corresponding to the second focus position captured by the image pickup apparatus 100 at the first time point and the size of the object on the image plane at the second focus position captured by the image pickup apparatus 100 at the second time point satisfy a predetermined condition.

The predetermined condition in this case may be a condition that the size on the image plane of the object at the position corresponding to the second focal position captured by the image capturing apparatus 100 at the first point in time coincides with the size on the image plane of the object at the second focal position captured by the image capturing apparatus 100 at the second point in time. Under such a condition, a moving image captured from the first time point to the second time point includes an appearance that the size of the background on the image plane is changed while maintaining the size of the object of interest present at the second focus position on the image plane. The moving image includes a representation that, at a first point in time, a subject of interest present at a position corresponding to a second in-focus position is not in an in-focus state, and a subject of interest present at the second in-focus position at the second point in time enters the in-focus state.

In the case of zooming to the telephoto side, it is more difficult to obtain an in-focus state than in the case of zooming to the wide-angle side. One of the reasons is that, in the case of zooming to the telephoto side, when the slide zoom is started, it is difficult to find an object to be focused. Therefore, preferably, the first focus distance at the first point in time is longer than the second focus distance at the second point in time. That is, it is preferable that the UAV10 move so as to approach the object of interest from the first time point to the second time point, and be captured by the imaging apparatus 100. Thus, from the first time point to the second time point, the in-focus state of the object of interest is easily maintained.

For example, the imaging apparatus 100 is actually moved relative to the object, and the focal distance from the first time point to the second time point is acquired by the acquisition unit 31. Subsequently, it is also possible to move the image pickup apparatus 100 again with respect to the object, causing the image pickup apparatus 100 to pick up a moving image that produces the slide-zoom effect. At this time, while the image pickup apparatus 100 is moving so as to approach the object, the zoom magnification may be changed from the telephoto side to the wide-angle side, and the focal distance may be acquired by the acquisition unit 31. Thereby, the image pickup apparatus 100 more easily acquires the focusing distance for focusing on the object from the first time point to the second time point. In addition, when the image pickup apparatus 100 picks up a moving image in which the slide zoom effect is obtained, while the image pickup apparatus 100 is moving away from the object, it is possible to control the focus lens and the zoom lens in accordance with the previously acquired focusing distance, and to perform shooting by changing the zoom magnification from the wide-angle side to the telephoto side.

The determination section 32 may determine respective control values of the optical zoom and the electronic zoom as zoom setting values of the image pickup apparatus 100 at each time point from the first time point to the second time point based on the time T, the first zoom magnification, and the second zoom magnification. The determination section 32 may determine each control value of the optical zoom and the electronic zoom as a zoom setting value of the image pickup apparatus 100 to switch from the optical zoom to the electronic zoom. The determination section 32 may determine each control value of the optical zoom and the electronic zoom as a zoom setting value of the image pickup apparatus 100 to switch from the electronic zoom to the optical zoom.

The determination section 32 may determine the focus setting value of the focus lens 210 and the zoom setting value of the zoom lens 211 at respective time points from the first time point to the second time point based on the time T, the first zoom magnification, and the second zoom magnification. The determination section 32 may determine the focus setting value of the focus lens 210 and the zoom setting value of the zoom lens 211 at respective time points from the first time point to the second time point, according to a predetermined relationship between the position of the focus lens 210 and the position of the zoom lens 211.

The determination section 32 may determine the focus setting value and the zoom setting value at each time point from the first time point to the second time point such that a size of an object on the image plane at a first focus position captured by the image pickup device at the first time point and a size of an object on the image plane at a second focus position captured by the image pickup device at the second time point satisfy a predetermined condition. The predetermined condition may be a condition that a size of an object on the image plane at a first focus position photographed by the image pickup apparatus 100 at a first point in time coincides with a size of an object on the image plane at a second focus position photographed by the image pickup apparatus 100 at a second point in time.

In a case where the imaging apparatus 100 changes the zoom magnification from the second zoom magnification to the third zoom magnification by electronic zooming from the second time point to the third time point, the determination section 32 may determine the focus setting value of the focus lens 210 at each time point from the second time point to the third time point. The determination section 32 may determine the focus setting value of the focus lens 210 at each time point from the second time point to the third time point based on the focus distance at the second time point and the velocity of the UAV 10.

The UAV control section 30 may move the focus lens 210 and the zoom lens 211 via the lens control section 220 in accordance with a predetermined relationship (e.g., a zoom tracking curve) between the position of the focus lens 210 and the position of the zoom lens 211 during movement of the image pickup apparatus 100 from the first time point to the second time point, thereby changing the zoom magnification of the image pickup apparatus 100 from the first zoom magnification to a second zoom magnification that is n times the first zoom magnification and changing the focus distance of the image pickup apparatus 100 from the first focus distance to a second focus distance that is n times the first focus distance.

Further, the UAV control section 30 may change the zoom magnification of the image pickup apparatus 100 from the second zoom magnification to a third zoom magnification that is m times the first zoom magnification by performing the electronic zoom during the movement of the image pickup apparatus 100 from the second time point to the third time point, and change the focus distance of the image pickup apparatus 100 from the second focus distance to the third focus distance that is m times the first focus distance by moving the focus lens 210. Here, the electronic zoom may be implemented by changing a size of a cutout in an image output from the image sensor 120. The image pickup apparatus 100 may move the focus lens 210 to change the focus distance according to the distance from the object without performing the optical zoom during the execution of the electronic zoom. Thus, the image pickup apparatus 100 can pick up a moving image that produces a slide zoom effect by electronic zooming.

For example, as shown in fig. 12A, the UAV10 flies in the imaging direction of the imaging apparatus 100 so that the distance from the object 500 changes from 1.0m to 2.0 m. During this time, the image pickup apparatus 100 performs optical zooming by controlling the focus lens 210 and the zoom lens 211 so that the zoom magnification is changed from 1 to 2 and the focusing distance is changed from 1.0 to 2.0 m. Further, the UAV10 flies in the imaging direction of the imaging apparatus 100 so that the distance to the object changes from 2.0m to 3.0 m. During this time, the image pickup apparatus 100 performs electronic zooming to change the zoom magnification from 2 times to 3 times, and changes the focus distance from 2.0m to 3.0m by controlling the focus lens 210.

The UAV control unit 30 may cause the imaging apparatus 100 to perform optical zooming after causing the imaging apparatus 100 to perform electronic zooming. In this case, the UAV control section 30 may change the zoom magnification of the image pickup apparatus from the first zoom magnification to a second zoom magnification that is n times the first zoom magnification by performing the electronic zoom during the movement of the image pickup apparatus 100 from the first time point to the second time point, and move the focus lens 210 via the lens control section 220, thereby changing the focus distance of the image pickup apparatus 100 from the first focus distance to a second focus distance that is n times the first focus distance.

Further, the UAV control section 30 may move the focus lens 210 and the zoom lens 211 according to a predetermined relationship between the position of the focus lens 210 and the position of the zoom lens 211 during movement of the image pickup apparatus 100 from the second time point to the third time point, thereby changing the zoom magnification of the image pickup apparatus 100 from the second zoom magnification to a third zoom magnification that is m times the first zoom magnification, and changing the focus distance of the image pickup apparatus 100 from the second focus distance to a third focus distance that is m times the first focus distance.

The determination unit 32 may determine the focus setting value of the image pickup apparatus 100 at each of the time points from the first time point to the second time point based on a time T required to change the zoom magnification of the image pickup apparatus 100 from the first zoom magnification to the second zoom magnification, the first zoom magnification, the second zoom magnification, information indicating the first focus distance, and information indicating the second focus distance. The determination unit 32 may determine the focus setting value and the zoom setting value at each time point from the second time point to the third time point based on a time required to change the zoom magnification of the image pickup apparatus 100 from the second zoom magnification to the third zoom magnification, the second zoom magnification, the third zoom magnification, information indicating the second focus distance, and information indicating the third focus distance. The determination section 32 may further determine the focus setting value and the zoom setting value for each time point from the second time point to the third time point based on first information indicating a relationship between the position of the focus lens 211 and the position of the focus lens 210 in the second focus distance and second information indicating a relationship between the position of the focus lens 211 and the position of the focus lens 210 in the third focus distance.

For example, as shown in fig. 12B, the UAV10 flies in the imaging direction of the imaging apparatus 100 so that the distance from the object 500 changes from 1.0m to 2.0 m. During this time, the image pickup apparatus 100 performs electronic zooming to change the zoom magnification from 1 to 2 times, and changes the focus distance from 1.0m to 2.0m by controlling the focus lens 210. Further, the UAV10 flies in the imaging direction of the imaging apparatus 100 so that the distance to the object changes from 2.0m to 3.0 m. During this time, the image pickup apparatus 100 performs optical zooming by controlling the focus lens 210 and the zoom lens 211 so that the zoom magnification is changed from 2 times to 3 times and the focus distance is changed from 2.0m to 3.0 m.

The UAV control 30 may cause the imaging apparatus 100 to perform optical zooming and electronic zooming simultaneously during at least a portion of the period. The UAV control section 30 may change the zoom magnification of the image pickup apparatus 100 from the first zoom magnification to a second zoom magnification that is n times the first zoom magnification and change the focus distance of the image pickup apparatus 100 from the first focus distance to a second focus distance that is n times the first focus distance by performing electronic zooming of the image pickup apparatus 100 during movement of the image pickup apparatus 100 from the first time point to the second time point and moving the focus lens 210 and the zoom lens 211 via the lens control section 220 according to a predetermined relationship (zoom tracking curve) between the position of the focus lens 210 and the position of the zoom lens 211.

The determination unit 32 may determine the focus setting value and the zoom setting value at each time point from the first time point to the second time point based on a time T required to change the zoom magnification of the image pickup apparatus 100 from the first zoom magnification to the second zoom magnification, the first zoom magnification, the second zoom magnification, information indicating the first focus distance, and information indicating the second focus distance. The determination section 32 may further determine the focus setting value and the zoom setting value for each time point from the first time point to the second time point based on first information indicating a relationship between the position of the focus lens 211 and the position of the focus lens 210 in the first focus distance and second information indicating a relationship between the position of the focus lens 211 and the position of the focus lens 210 in the second focus distance.

For example, as shown in fig. 12C, the UAV10 flies in the imaging direction of the imaging apparatus 100 so that the distance from the object 500 changes from 1.0m to 3.0 m. During this time, the image pickup apparatus 100 performs electronic zooming and optical zooming, changing the zoom magnification from 1 to 3 times, and changing the focus distance from 1.0m to 3.0 m.

Fig. 13 is a diagram illustrating one example of the relationship between the position of focus lens 210 and the position of zoom lens 211. Fig. 13 shows a zoom tracking curve 640 when the focus distance is 1.0m, a zoom tracking curve 641 when the focus distance is 2.0m, and a movement tracking curve 643 when the focus distance is 3.0 m.

As shown in fig. 12A, when the distance from the UAV10 to the object 500 is changed from 1.0m to 2.0 and the zoom magnification is changed from 1 × to 2 × the determination section 32 may derive a movement tracking curve 643 indicating a relationship between the position of the zoom lens 211 and the position of the focus lens 210 in the case where the zoom magnification is changed from 1 × to 2 times, based on the zoom tracking curve 640 when the focus distance is 1.0m and the zoom tracking curve 641 when the focus distance is 2.0m, for example. The determination section 32 may also determine the focus setting value of the focus lens 210 in the case of changing the zoom magnification from 2 times to 3 times by electronic zooming. Since the zoom lens 211 does not move, the determination section 32 may determine the focus setting value of the focus lens 210 such that the position of the focus lens 210 changes as a straight line indicated by a symbol 644.

Fig. 14 shows a case where the position of the focus lens 210 changes in a case where the image pickup apparatus 100 performs optical zooming and then performs electronic zooming. As shown in fig. 14, the UAV control section 30 may move the focus lens 210 via the lens control section 220 along a curve 650 determined based on the zoom tracking curve while the focus distance of the image pickup apparatus 100 is changed from 1.0m to 2.0 m. Further, the UAV control unit 30 may move the focus lens 210 via the lens control unit 220 along a curve 651 determined based on the moving speed of the imaging apparatus 100(UAV10) while the focal distance of the imaging apparatus 100 is changed from 2.0m to 3.0 m.

Here, there is a limit to the maximum speed at which the UAV10 can move. Thus, depending on the length of time T, or the distance of movement of UAV10 from the first point in time to the second point in time, UAV10 may not be able to move the distance of movement during time T.

There is a limit to the maximum speed at which the zoom lens 211 can be moved. Depending on the length of time T, zoom lens 211 may not be able to move from the first zoom magnification to the second zoom magnification during time T.

There is also a limit to the minimum speed at which the zoom lens 211 can be moved. Zoom lens 211 may not be movable from the first zoom magnification to the second zoom magnification within time T. That is, in order to move the zoom lens 211 within the time T, the speed of the zoom lens 211 may be slow.

When there is an obstacle on the route that the UAV10 moves from the first point in time to the second point in time, the UAV10 may not be able to move on the route.

In this way, the imaging apparatus 100 may not be able to capture a moving image that obtains the slide zoom effect according to the time T, the first zoom magnification, the second zoom magnification, the first focal distance, and the second focal distance.

Therefore, the determination section 33 may determine whether the image pickup apparatus 100 can capture a moving image in which the slide zoom effect is obtained, based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance.

The determination section 33 may determine whether or not the zoom magnification of the image pickup apparatus 100 can be changed from the first zoom magnification to the second zoom magnification within the time T based on at least one of the time T, the first zoom magnification, the second zoom magnification, the minimum speed and the maximum speed of the zoom lens 211. When the determination unit 33 determines that the zoom magnification of the imaging apparatus 100 can be changed from the first zoom magnification to the second zoom magnification within the time T, the determination unit 32 may determine the focus setting value of the imaging apparatus 100, the zoom setting value of the imaging apparatus 100, and the moving speed of the UAV10 at each time point from the first time point to the second time point.

The determination 33 may determine whether the UAV10 is able to move within the time T by the difference between the first focal distance and the second focal distance based on the time T, the difference between the first focal distance and the second focal distance, and the maximum speed of the UAV 10. In a case where the determination unit 33 determines that the UAV10 is movable within the time T by the difference between the first focal distance and the second focal distance, the determination unit 32 may determine the focus setting value of the imaging device 100, the zoom setting value of the imaging device 100, and the moving speed of the UAV10 at each time point from the first time point to the second time point.

The determination section 33 may determine whether there is an obstacle on the path that moves the UAV10 by the difference between the first focal distance and the second focal distance. In the case where the determination section 33 determines that there is no obstacle on the path, the focus setting value of the imaging apparatus 100, the zoom setting value of the imaging apparatus, and the moving speed of the UAV10 at each time point from the first time point to the second time point may be determined. The determination section 33 may determine whether there is an obstacle on the path that moves the UAV10 by the difference of the first focal distance and the second focal distance, based on the three-dimensional map stored in the memory 37 and the position information of the UAV 10. The determination section 33 may determine whether there is an obstacle on the path that moves the UAV10 by the difference between the first focal distance and the second focal distance, based on the image captured by the imaging apparatus 100 or the imaging apparatus 60 that is a stereo camera.

Here, in order to accurately impart an effect such as slide zoom to an image, it is desirable to coordinate and accurately perform zoom control and focus control of the imaging apparatus 100 and movement control of the UAV10 based on the set value of the focus of the imaging apparatus 100, the set value of the zoom of the imaging apparatus 100, and the movement speed of the UAV10 at each time point from the first time point to the second time point, which are determined by the determination unit 32.

Since the mass of UAV10 is different from the mass of zoom lens 211 and focus lens 210, the moment of inertia of UAV10 is different from the moment of inertia of zoom lens 211 and focus lens 210. Therefore, in the case where the movement of the UAV10 and the movements of the zoom lens 211 and the focus lens 210 start at the same time, it is necessary to consider and coordinate control of the difference in the moments of inertia between each other. However, it is not easy to accurately perform coordinated control of the UAV10, the zoom lens 211, and the focus lens 210.

Therefore, before the imaging device 100 starts to photograph, the UAV10 starts to move, and the zoom lens 211 and the focus lens 210 start to move at a stage when the UAV10 can move at a desired movement speed. That is, at the position where the imaging device 100 starts imaging, the UAV control unit 30 temporarily moves the UAV10 to a position where the UAV10 can run in order to allow the UAV10 to move at a desired movement speed. The UAV controller 30 controls the movement of the UAV10 such that the UAV10 starts moving from its position, and the moving speed of the UAV10 becomes a desired moving speed at the position at which the imaging device 100 starts imaging. Thus, movement of the zoom lens 211 and the focus lens 210 may be controlled without regard to the moment of inertia of the UAV 10.

For example, as shown in fig. 15, in a state where the UAV10 hovers at the position P1, the acquisition unit 31 acquires the focal distance L1 at the position P1 at which the imaging apparatus 100 starts imaging. Further, the acquisition unit 31 acquires a time T required to change the zoom magnification of the imaging apparatus 100 from the first zoom magnification to the second zoom magnification, the first zoom magnification, and the second zoom magnification.

The determination unit 32 determines the setting value of the focus of the imaging apparatus 100, the setting value of the zoom of the imaging apparatus 100, and the movement speed V1 of the UAV10 at each time point from the recording start time point T1 to the recording end time point T2 based on the time T, the first zoom magnification, and the second zoom magnification. The determination section 32 may determine the setting value of the focus, the setting value of the zoom, and the moving speed at each time point from the recording start time point T1 to the recording end time point T2 to change the zoom magnification of the image pickup apparatus 100 from the first zoom magnification to the second zoom magnification and to change the focus distance of the image pickup apparatus 100 from the first focus distance L1 to the second focus distance L2 during the recording start time point T1 to the recording end time point T2. For example, the second zoom magnification is n times the first zoom magnification, and the second focus distance L2 may be n times the first focus distance. The distance from position P1 to position P2 corresponds to the distance between the first focus distance L1 and the second focus distance L2.

Still further, the determination section 32 determines the position P0 of the UAV10 at the approach start time point T0 before the recording start time point T1 based on the position P1 of the recording start time point T1 of the UAV10 and the moving speed V1 of the UAV10 at the recording start time point T1, so that the UAV10 can move at the position P1 at the moving speed V1 at the recording start time point T1. The determination unit 32 is an example of a first determination unit and a second determination unit. The run-up distance required to achieve a desired moving speed of the UAV10 may be measured in advance by experiments, simulations, or the like, and the correspondence between the moving speed and the run-up distance may be stored in the memory 37 or the like. The determination section 32 may determine the run-up distance corresponding to the movement speed V1 by referring to the memory 37, and determine the position P0 of the UAV10 at the run-up start time point T0 based on the determined run-up distance.

The UAV controller 30 moves the UAV10 from the position P1 to the position P0 after the time point T00 before the time point T0 reaches the time point T0, and starts moving the UAV10 from the position P0 to the position P1 at the time point T0. The position of the UAV10 at the time point T00 may be different from the position P1 at the recording start time point T1. Then, at time T1, the UAV controller 30 controls the UAV10 to change the speed of the UAV10 to the moving speed V1, and then controls the UAV10 to maintain the speed of the UAV10 at the moving speed V1 from the position P1 to the position P2 of the UAV10 at time T2. The imaging control unit 110 executes control for changing the zoom magnification of the imaging apparatus 100 from the first zoom magnification to the second zoom magnification and changing the focal distance of the imaging apparatus 100 from the first focal distance to the second focal distance from time T1 to time T2.

As described above, in order to ensure the run-up distance, the UAV10 has a possibility of approaching the main object from the time point T00 to the time point T0. The user who is the main subject sometimes assumes that the shooting apparatus 100 shoots while the UAV10 is away. In this case, when UAV10 approaches the user, the user may misinterpret UAV10 as having a false action. Therefore, the imaging control unit 110 may further include a notification unit 34, as shown in fig. 2. The notification section 34 notifies the outside that the UAV10 will move from position P1 to position P0 before moving from position P1 to position P2. The notification unit 34 may notify the user of the approach of the UAV10 before the slide zoom is performed, from a speaker sound included in the remote operation device 300, via the communication interface 36. The notification portion 34 may notify the user via the communication interface 36 by displaying a message indicating that the UAV10 approaches before the slide zoom is performed on the display portion included in the remote operation apparatus 300.

Fig. 16 is a diagram for explaining the image recording timing of the imaging apparatus 100 according to the movement of the UAV 10. At time point T00, UAV10 hovers at position P1. Therefore, in order to ensure the run-up distance, the UAV10 moves to the position P0 determined by the determination section 32 at the time point T0. Such as the UAV10, toward the primary subject. At this time, the UAV10 may approach the user at a speed equal to or lower than the moving speed V2, which is slower than the moving speed V1, so as not to give a sense of discomfort to the user. That is, the UAV10 may keep the user free of fear by slowly approaching the user. The determination section 32 may determine the moving speed of the UAV10 at the time when the UAV10 moves from the position P1 at the time point T00 to the position P0 at the time point T0 as a moving speed that is slower than the moving speed of the UAV10 at the time when the UAV10 moves from the time point T0 to the time point T1. The UAV controller 30 may move the UAV10 from the position P1 to the position P0 at a moving speed slower than a moving speed of the UAV10 when the UAV10 moves from the time point T0 to the time point T1, thereby ensuring a run-up distance.

Then, the UAV10 sets the moving speed to the moving speed V1 with the run-up distance from the time point T0 to the time point T1. From time point T1 to time point T2, UAV10 flies from position P1 to position P2 while maintaining moving speed V1. At this time, from time T1 to time T2, the imaging apparatus 100 changes the zoom magnification from the first zoom magnification to the second zoom magnification. Further, from the time point T1 to the time point T2, the image pickup apparatus 100 records a moving image.

Here, the time required for the UAV10 to move from the position P1 to the position P2 of the UAV10 at the time point T2 at the moving speed V1 is time H1. The determination unit 32 may determine the focus setting value and the zoom setting value at each time point from time point T1 to time point T2 to change the focal distance of the imaging apparatus 100 from the first focal distance L1 to the second focal distance L2 and change the zoom magnification of the imaging apparatus 100 from the first zoom magnification to the second zoom magnification while the UAV10 moves from position P1 to position P2 at the movement speed V1, that is, at time H1. By specifying the respective parameters in this manner, the effect of changing the background can be given to the moving image while maintaining the size of the main subject in the image from time T1 to time T2.

On the other hand, from time T1 to time T2, there is a case where the size of the main subject in the image is also changed, and there is a need to provide an effect of changing the background at a different change rate to the moving image. For example, there is a case where the main subject in the image is increased and an effect of changing the background at a different change rate is desired to be given to the moving image.

In this case, as shown in fig. 17, the determination unit 32 may determine the focus setting value and the zoom setting value at each time point from the time point T1 to the time point T2 to change the focal distance of the imaging apparatus 100 from the first focal distance L1 to the second focal distance L2 during the movement of the UAV10 from the position P1 to the position P2 at the movement speed V1, that is, at the time H1, and change the zoom magnification of the imaging apparatus 100 from the first zoom magnification to the second zoom magnification at the time H2 shorter than the time H1. Thus, from time T1 to time T2, the size of the main subject in the image can be changed, and an effect of changing the background at a different change rate can be given to the moving image.

As shown in fig. 18, the above-described effect can also be obtained by setting the moving speed of the UAV10 to the moving speed V1' which is slower than the moving speed V1.

The determination unit 32 may determine the focus setting value and the zoom setting value at each time point from the time point T1 to the time point T2 to change the focal distance of the imaging apparatus 100 from the first focal distance L1 to the second focal distance L2 ' and change the zoom magnification of the imaging apparatus 100 from the first zoom magnification to the second zoom magnification while the UAV10 moves from the position P1 to the position P2 ' at the movement speed V1 ', that is, at a time H1. At this time, the moving speed V1' is slower than the moving speed V1. The second zoom magnification is n times the first zoom magnification, and the distance from the position P1 to the position P2' is shorter than the distance n times the first focus distance.

In this way, while the imaging apparatus 100 changes the zoom magnification from the first zoom magnification to the second zoom magnification that is n times the first zoom magnification, the moving speed is adjusted so that the UAV10 moves by a distance that is shorter than the distance that is n times the first focus distance. Thus, from time T1 to time T2, the size of the main subject in the image can be changed, and an effect of changing the background at a different rate of change can be given to the moving image.

As described above, by adjusting the speed at which the zoom magnification is changed or the moving speed of the UAV10, for example, at the slide-zoom start time point, as an image on the wide angle side, the image 730 including the object 500 in the in-focus state as shown in fig. 19A is captured at the first zoom magnification. At the slide-zoom end time point, as an image on the telephoto side, an image 711 containing the object 500 in focus as shown in fig. 19B is captured at a second zoom magnification that is larger than the first zoom magnification. This makes it possible to provide a moving image captured from the slide zoom start time point to the slide zoom end time point with an expression in which the rate of change in the size of the subject 500 on the image plane and the rate of change in the size of the background on the image plane are different.

From time T1 to time T2, the size of the main subject in the image may be reduced, and an effect of changing the background at a different rate of change may be given to the moving image. For example, while the imaging apparatus 100 changes the zoom magnification from the first zoom magnification to the second zoom magnification that is 1/n times the first zoom magnification, the moving speed may be adjusted so that the UAV10 moves by a distance that is n times the first focal distance.

Fig. 20 is a flowchart showing one example of an imaging process of the imaging apparatus 100 mounted on the UAV 10.

The UAV10 begins flying (S100). The UAV control unit 30 receives a mode setting instruction from the remote operation device 300, and sets the imaging mode of the imaging apparatus 100 to the slide zoom mode (S102). The UAV control unit 30 accepts selection of an object of interest via live view of the imaging apparatus 100 displayed on the display unit of the remote operation apparatus 300 (S104). The UAV control unit 30 may have a receiving unit that receives an object of interest from an image captured by the imaging apparatus 100. The receiving unit may receive a selection of a plurality of objects of interest from the image. The receiving section may accept selection of an object of interest at a slide-zoom start time point and an object of interest at a slide-zoom end time point. The receiving section may accept selection of an object of interest at each time point from the slide-zoom start time point to the slide-zoom end time point.

The UAV control unit 30 receives and sets a first zoom magnification at a first time point (recording start time point of the slide zoom), a second zoom magnification at a second time point (recording end time point of the slide zoom), and a time T which is an imaging time of the slide zoom via the remote operation device 300 (S106). The UAV control section 30 may set the first zoom magnification, the second zoom magnification, and the time T in accordance with setting information stored in advance in the memory 37 or the like. The UAV control section 30 may accept only whether to change from the telephoto side to the wide-angle side or from the wide-angle side to the telephoto side. The UAV control section 30 may set a predetermined zoom magnification on the telephoto side and a predetermined zoom magnification on the wide-angle side to the zoom magnifications at the first time point and the second time point based on whether to change from the telephoto side to the wide-angle side or from the wide-angle side to the telephoto side. The UAV control 30 may accept a time T from a predetermined plurality of candidate times. The UAV control section 30 can set the time T by accepting a desired time mode from among a long time mode, a medium time mode, and a short time mode, for example.

The acquisition unit 31 acquires information indicating a focus distance from the imaging device 100 to the object of interest (S108). The acquisition section 31 may acquire information indicating a first focus distance of the object of interest from a first time point. The acquisition section 31 may derive the second focal distance based on the first zoom magnification, the second zoom magnification, and the first focal distance. The acquisition section 31 may derive the second focus distance by multiplying the first focus distance by a ratio of the first zoom magnification and the second zoom magnification.

The determination unit 33 determines whether the image pickup apparatus 100 can capture a moving image in which the slide zoom effect is obtained, based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance (S110). The determination unit 33 determines whether or not the image pickup apparatus 100 can capture a moving image that achieves the slide zoom effect, based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance.

The determination section 33 may determine whether or not the zoom magnification of the image pickup apparatus 100 can be changed from the first zoom magnification to the second zoom magnification within the time T based on at least one of the time T, the first zoom magnification, the second zoom magnification, the minimum speed and the maximum speed of the zoom lens 211. The determination 33 may determine whether the UAV10 is able to move within the time T by the difference between the first focal distance and the second focal distance based on the time T, the difference between the first focal distance and the second focal distance, and the maximum speed of the UAV 10. The determination section 33 may determine whether there is an obstacle on the path that moves the UAV10 by the difference between the first focal distance and the second focal distance.

When the determination section 33 determines that the image pickup apparatus 100 cannot pick up a moving image for which the slide zoom effect is obtained, the user is notified of a setting change request via the remote operation apparatus 300. The determination section 33 may notify the user of the time T, the first focus distance, or the zoom magnification at which the slide zoom can be captured. When the determination unit 33 receives a setting change request from the user (S124), the UAV control unit 30 resets the zoom magnification and time in accordance with the setting change request (S106). Upon receiving a movement instruction from the user's UAV10, the UAV control unit 30 moves the UAV10 relative to the object to adjust the distance to the object.

When the setting change request is not made, the determination unit 33 notifies the user of an error indicating that the slide zoom cannot be captured via the remote operation device 300 (S126).

When the slide zoom is able to be captured, the determination unit 32 determines the set value of the focus of the imaging apparatus 100, the set value of the zoom of the imaging apparatus 100, and the first movement speed of the UAV10 at each of the first time point to the second time point (S112). The determination unit 32 may determine the set value of the focus of the imaging apparatus 100, the set value of the zoom of the imaging apparatus 100, and the first movement speed of the UAV10 at each time point from the first time point to the second time point based on the movement tracking curve at the first focal length at the first time point and the movement tracking curve at the second focal length at the second time point.

The determination unit 32 also determines the position P0 of the UAV at the starting point of the run-up based on the position P1 of the UAV10 at the starting point of the slide zoom and the first movement speed (S114). The determination section 32 may determine the run-up distance corresponding to the first movement speed based on a correspondence relationship between the movement speed and the run-up distance set in advance, and determine the position P0 of the UAV at the run-up start time point based on the run-up distance.

The UAV control section 30 moves the UAV10 to a position P0 at the starting point of run-up time (S116). The UAV controller 30 moves the UAV10 from position P1 to position P0 at a slower movement speed than the first movement speed. Next, the UAV controller 30 controls the UAV10 so that the UAV10 can move at the first moving speed at a position P1 of the UAV10 at the first time point which is the start time point of the slide zoom (S118).

The UAV control unit 30 controls the position of the zoom lens 211, the position of the focus lens 210, and the movement of the UAV10 based on the set value of the focus of the imaging apparatus 100, the set value of the zoom of the imaging apparatus 100, and the movement speed of the UAV10 at each time point from the first time point to the second time point (S120). Thus, the imaging apparatus 100 changes the zoom magnification and the focal length while changing the distance from the object from the first time point to the second time point. From the first time point to the second time point, the image capturing apparatus 100 captures an image while maintaining the size of the subject of interest on the image plane, for example, and stores data such as a captured moving image in the memory 130 or the like (S122). Thereby, the image pickup apparatus 100 can take a moving image maintaining the size and the in-focus state of the object of interest on the image plane while changing the background size or the blur amount. Further, since the run-up distance is secured, the slide zoom can be recorded from a state in which the moving speed of the UAV10 is stabilized to a desired moving speed. Thereby, the zoom lens 211 and the focus lens 210 can be controlled so as to impart a desired effect to an image without being affected by the moment of inertia of the UAV 10.

Note that, in the above example, an example has been described in which the UAV10 moves along the imaging direction of the imaging apparatus 100. However, the UAV10 may move so as to traverse the object, and control the posture of the imaging apparatus 100 by the gimbal 50 so that the imaging direction of the imaging apparatus 100 is directed toward the object side. The UAV10 may also control the orientation of the UAV10 so that the imaging direction of the imaging apparatus 100 is directed toward the object side while moving so as to pass through the object. The UAV10 may control the orientation of the UAV10 and the posture of the imaging apparatus 100 via the universal joint 50 so that the imaging direction of the imaging apparatus 100 faces the object side while moving so as to cross the object. The UAV10 can control at least one of the posture of the imaging apparatus 100 and the orientation of the UAV10 adjusted via the universal joint 50 such that the imaging direction of the imaging apparatus 100 is directed toward the object side while ascending or descending. As can be understood from fig. 4, the range of movable tracking is, for example, between the zoom tracking curve 601 and the zoom tracking curve 602. Thus, it may be provided that the UAV10 is able to move within a range of movable tracking. The movable range may be set as a three-dimensional space region. That is, by using the movement tracking mode, the movable region of the UAV10 may be controlled. The movable region of the UAV10 may be set to be a hollow sphere on a three-dimensional space centered on the subject or a hollow hemisphere on a three-dimensional space. The movable region of UAV10 may be set based on at least one of time T, the first zoom magnification, the second zoom magnification, the lowest speed of zoom lens 211, the maximum speed of zoom lens 211, and the maximum speed of UAV 10.

The imaging apparatus 100 may also adjust the aperture from the first time point to the second time point. The determination unit 32 may determine the aperture value of the image pickup apparatus 100 at each of the time points from the first time point to the second time point based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance. The determination section 32 may determine the control value of the aperture of the image pickup apparatus 100 at each time point from the first time point to the second time point so that the degree of blurring of the background from the first time point to the second time point does not change. The determination unit 32 may determine the diaphragm as a first control value at a first zoom magnification (telephoto side) at a first time point, and may determine the diaphragm as a second control value smaller than the first control value at a second magnification (wide-angle side) smaller than the first zoom magnification at a second time point.

The image capturing apparatus 100 may also adjust the F value from the first time point to the second time point. The determination unit 32 may determine the F value of the image pickup apparatus 100 at each of the time points from the first time point to the second time point based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance. The determination section 32 may determine the F value of the image pickup apparatus 100 at each time point from the first time point to the second time point so that the luminance (luminance value) in the image of the object of interest does not change from the first time point to the second time point. The determination unit 32 may determine the F value as a first control value at a first zoom magnification (telephoto side) at a first time point, and may determine the F value as a second control value larger than the first control value at a second magnification (wide-angle side) smaller than the first zoom magnification at a second time point.

The image capturing apparatus 100 can adjust the ISO sensitivity (gain) from the first time point to the second time point. The determination unit 32 may determine the ISO sensitivity of the image pickup apparatus 100 at each time point from the first time point to the second time point based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance. The determination section 32 may determine the ISO sensitivity and the shutter speed of the image pickup apparatus 100 at each time point from the first time point to the second time point based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance. The determination section 32 may determine the ISO sensitivity and shutter speed of the image pickup apparatus 100 at each time point from the first time point to the second time point based on the time T, the first zoom magnification, the second zoom magnification, the first focus distance, and the second focus distance to keep the exposure constant.

To reduce the flicker of the image, the image pickup apparatus 100 may disable the auto exposure function and the auto white balance function when operating in the slide zoom mode.

The UAV10 may move in such a manner that the selected object of interest is included in the central area of the image captured by the imaging device 100. Alternatively, the UAV10 may move in such a manner that an arbitrary point other than the attention object in the image captured by the imaging apparatus 100 at the first time point is included in the central area of the image. In performing the slide zoom, the electronic zoom may be performed after the optical zoom. In performing the slide zoom, the optical zoom may be performed after the electronic zoom. This can extend the movable distance of the UAV 10. Therefore, the sliding zooming effect can be better realized.

FIG. 21 illustrates one example of a computer 1200 that may fully or partially embody aspects of the disclosure. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "sections" of or operations associated with an apparatus according to an embodiment of the present disclosure. Alternatively, the program can cause the computer 1200 to execute the operation or the one or more "sections". The program enables the computer 1200 to execute the processes or the stages of the processes related to the embodiments of the present disclosure. Such programs may be executed by the CPU1212 to cause the computer 1200 to perform certain operations associated with some or all of the blocks in the flowchart and block diagrams described herein.

The computer 1200 of the present embodiment includes a CPU1212 and a RAM1214, which are connected to each other via a host controller 1210. The computer 1200 also includes a communication interface 1222, an input/output unit, which are connected to the host controller 1210 through the input/output controller 1220. Computer 1200 also includes a ROM 1230. The CPU1212 operates in accordance with programs stored in the ROM1230 and the RAM1214, thereby controlling the respective units.

The communication interface 1222 communicates with other electronic devices through a network. The hard disk drive may store programs and data used by CPU1212 in computer 1200. The ROM1230 stores therein a boot program or the like executed by the computer 1200 at runtime, and/or a program depending on hardware of the computer 1200. The program is provided through a computer-readable recording medium such as a CR-ROM, a USB memory, or an IC card, or a network. The program is installed in the RAM1214 or the ROM1230, which is also an example of a computer-readable recording medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by operations or processes according to information that may be implemented with the use of the computer 1200.

For example, in performing communication between the computer 1200 and an external device, the CPU1212 may execute a communication program loaded in the RAM1214 and instruct the communication interface 1222 to perform communication processing based on processing described by the communication program. The communication interface 1222 reads transmission data stored in a transmission buffer provided in a recording medium such as the RAM1214 or a USB memory and transmits the read transmission data to a network, or writes reception data received from the network in a reception buffer or the like provided in the recording medium, under the control of the CPU 1212.

Further, the CPU1212 may cause the RAM1214 to read all or a necessary portion of a file or a database stored in an external recording medium such as a USB memory, and perform various types of processing on data on the RAM 1214. Then, the CPU1212 may write back the processed data to the external recording medium.

Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium and processed by the information. With respect to data read from the RAM1214, the CPU1212 may execute various types of processing described throughout this disclosure, including various types of operations specified by an instruction sequence of a program, information processing, condition judgment, condition transition, unconditional transition, retrieval/replacement of information, and the like, and write the result back to the RAM 1214. Further, the CPU1212 can retrieve information in files, databases, etc., within the recording medium. For example, when a plurality of entries having attribute values of first attributes respectively associated with attribute values of second attributes are stored in a recording medium, the CPU1212 may retrieve an entry matching a condition specifying an attribute value of a first attribute from the plurality of entries and read an attribute value of a second attribute stored in the entry, thereby acquiring an attribute value of a second attribute associated with a first attribute satisfying a predetermined condition.

The programs or software modules described above may be stored on the computer 1200 or on a computer readable storage medium near the computer 1200. Further, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the internet may be used as the computer-readable storage medium, thereby providing the program to the computer 1200 through the network.

It should be noted that the execution order of the operations, the sequence, the steps, the stages, and the like in the devices, systems, programs, and methods shown in the claims, the description, and the drawings of the specification can be realized in any order as long as "before …", "in advance", and the like are not particularly explicitly indicated, and as long as the output of the preceding process is not used in the following process. The operational flow in the claims, the specification, and the drawings of the specification is described using "first", "next", and the like for convenience, but it is not necessarily meant to be performed in this order.

The present disclosure has been explained above using the embodiments, but the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes and modifications can be made in the above embodiments. It is apparent from the description of the claims that such modifications or improvements can be included in the technical scope of the present disclosure.

[ notation ] to show

10 UAV

20 UAV body

30 UAV control section

31 acquisition part

32 determination part

33 determination part

34 notification unit

36 communication interface

37 memory

40 advancing part

41 GPS receiver

42 inertia measuring device

43 magnetic compass

44 barometric altimeter

45 temperature sensor

46 humidity sensor

50 universal joint

60 image pickup device

100 image pickup device

102 image pickup part

110 image pickup control unit

120 image sensor

130 memory

200 lens part

210 focusing lens

211 zoom lens

212, 213 lens driving part

214, 215 position sensor

220 lens control part

222 memory

300 remote operation device

1200 computer

1210 host controller

1212 CPU

1214 RAM

1220 input/output controller

1222 communication interface

1230 ROM

Claims

1. A determination apparatus, comprising: a first determination unit that determines a set value of focus of an imaging device mounted on a mobile body, a set value of zoom of the imaging device, and a moving speed of the mobile body at each of time points from a first time point to a second time point, so that an image captured by the imaging device changes a background while maintaining a size of an object or changes the background at a change rate different from a size change rate of the object during the first time point to the second time point; and

and a second specifying unit that specifies a third position that is a position of the moving body at a third time point before the first time point, based on a first position that is a position of the moving body at the first time point and a first moving speed that is a speed of the moving body at the first time point, so that the moving body can move at the first position at the first moving speed at the first time point.

2. The determination device according to claim 1, wherein the first determination section determines the setting value of the focus, the setting value of the zoom, and the movement speed at each of time points from the first time point to the second time point to change the zoom magnification of the image pickup device from a first zoom magnification to a second zoom magnification and change the focus distance of the image pickup device from a first focus distance to a second focus distance during the first time point to the second time point.

3. The determination apparatus according to claim 2, wherein the second zoom magnification is n times the first zoom magnification,

the second focus distance is n times the first focus distance,

a distance between a second position that is a position of the mobile body at the second time point from the first position corresponds to a difference between the first focus distance and the second focus distance.

4. The determination device according to claim 2, wherein a time required for the mobile body to move from the first position to a second position at the first moving speed is a first time, the second position is a position of the mobile body at the second point in time,

the first determination unit determines the focus setting value and the zoom setting value at each of the time points from the first time point to the second time point so as to change the focal distance of the imaging device from the first focal distance to the second focal distance and change the zoom magnification of the imaging device from the first zoom magnification to the second zoom magnification during the movement of the mobile body from the first position to the second position at the first movement speed, that is, during the first time.

5. The determination device according to claim 2, wherein a time required for the mobile body to move from the first position to a second position at the first moving speed is a first time, the second position is a position of the mobile body at the second point in time,

the first determination unit determines the focus setting value and the zoom setting value at each of the time points from the first time point to the second time point so as to change the focus distance of the imaging device from the first focus distance to the second focus distance during the movement of the movable body from the first position to the second position at the movement speed, that is, during the first time, and change the zoom magnification of the imaging device from the first zoom magnification to the second zoom magnification at a second time shorter than the first time.

6. The determination device according to claim 1, wherein the first determination section determines the moving speed of the moving body when the moving body moves from a fourth position, which is a position of the moving body at a fourth time point that is prior to a third time point, to the third position to be smaller than the moving speed of the moving body when the moving body moves from the third time point to the first time point.

7. The determination device of claim 6, wherein the fourth location is the same as the first location.

8. The determination device according to claim 2, wherein a time required for the mobile body to move from the first position to a second position at the first moving speed is a first time, the second position is a position of the mobile body at the second point in time,

the first determination unit determines the focus setting value and the zoom setting value at each of the time points from the first time point to the second time point to change the focus distance of the image pickup apparatus from the first focus distance to the second focus distance and change the zoom magnification of the image pickup apparatus from the first zoom magnification to the second zoom magnification during the movement of the movable body from the first position to the second position at the movement speed, that is, during the first time,

the second zoom magnification is n times the first zoom magnification,

a distance from the first position to the second position is shorter than a distance n times the first focus distance.

9. A moving body that includes the determination device according to any one of claims 1 to 8 and the imaging device and that moves, comprising:

a first control unit that controls the moving body to move from the third position to the first position, and controls the moving body to maintain the speed of the moving body at the first moving speed from the first position to a second position at the second time point after the speed of the moving body is controlled to become the first moving speed at the first time point; and

a second control section that performs control of changing a zoom magnification of the image pickup apparatus from a first zoom magnification to a second zoom magnification and changing a focal distance of the image pickup apparatus from a first focal distance to a second focal distance from the first time point to the second time point.

10. The movable body according to claim 9 wherein the first control unit controls the movable body such that the moving speed of the movable body becomes the first moving speed at the first position after moving the movable body from a fourth position, which is a position of the movable body at a fourth time point, to the third time point from the fourth time point prior to the third time point, at the first time point.

11. The mobile body according to claim 10, further comprising: and a notification unit configured to notify an outside that the mobile object moves from the fourth position to the third position before moving from the first position to the second position.

12. The mobile body according to claim 11 wherein the fourth position is the same as the first position.

13. The movable body according to claim 10 wherein the first control portion moves the movable body from the fourth position to the third position at a speed that is less than a moving speed of the movable body when the movable body moves from a third time point to a first time point.

14. A method of determination comprising the stages of: determining a set value of focus of an image pickup device mounted on a moving body at each of time points from a first time point to a second time point, a set value of zoom of the image pickup device, and a moving speed of the moving body to cause an image picked up by the image pickup device to change a background while maintaining a size of an object or to change the background at a change rate different from a size change rate of the object during the first time point to the second time point; and

the position of the mobile body at a third time point prior to the first time point is determined based on a first position that is the position of the mobile body at the first time point and a first moving speed that is the speed of the mobile body at the first time point, so that the mobile body can move at the first position at the first moving speed at the first time point.

15. A computer-readable recording medium storing a program for causing a computer to function as the determination device according to any one of claims 1 to 8.