JP6515423B2 - CONTROL DEVICE, MOBILE OBJECT, CONTROL METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to a control device, a mobile unit, a control method, and a program.

Patent Document 1 discloses a camera that adjusts the position of a shooting lens and the amount of stop of an aperture so that a subject enters the depth of field.
Patent Document 1: International Publication No. 2016/056089

Problem to be solved

  If it is desired to take a picture without changing the position of the taking lens or the stop amount of the stop, the above method can not be adopted.

General disclosure

  The control device according to one aspect of the present invention is an imaging device in which an object to be imaged by the imaging device should exist based on at least one of the focal length of the imaging device, the aperture value, the permissible circle of confusion diameter, and the object distance. And a first determination unit that determines a distance range of The control device may include a first identification unit that identifies the distance from the imaging device to the object. The control device may include a control unit that controls the position of the imaging device so that the object is included in the distance range based on the distance range and the distance.

  The width of the distance range may be narrower than the width of the depth of field of the imaging device determined based on at least one of the focal length of the imaging device, the aperture value, the permissible circle of confusion diameter, and the subject distance. The subject distance may be based on the position of the lens of the imaging device.

  The controller may comprise an estimation unit for estimating the geographical position of the object at a first future point in time. The control device may include a second determination unit that determines a geographical area where the imaging device should exist at the first time point based on the estimated geographical position and the distance range. The controller may control the position of the imaging device such that the imaging device is within the geographical area at a first time.

  The control device may include a first point at which a geographical position is determined in advance from an image captured by the imaging device, and a selection unit that selects an object. The control device specifies a geographical position of the object based on the positional relationship between the first point and the object in the image captured by the imaging device, and the geographical position of the first point. You may have a department. The estimation unit may estimate the geographical position of the object at the first time based on the plurality of geographical positions of the object specified by the second specifying unit at the plurality of time points up to now.

  The control unit may cause the imaging device to image the object when the object is present within the distance range.

  The mobile unit according to an aspect of the present invention may move with the control device and the imaging device mounted thereon.

  The control unit may control the position of the moving body such that the object is included in the distance range.

  The control unit may cause the moving object to track the target so that the target is included in the distance range.

  The mobile may be an unmanned aerial vehicle. The controller may cause the unmanned aircraft to hover so that the object is included within the distance range.

  According to one aspect of the present invention, there is provided a control method according to an aspect of the present invention, in which an imaging device should have an object to be imaged based on at least one of a focal length of an imaging device, an aperture value, a permissible circle of confusion diameter Determining a distance range of The control method may comprise identifying a distance from the imaging device to the object. The control method may comprise controlling the position of the imaging device to include the object within the distance range based on the distance range and the distance.

  A program according to an aspect of the present invention is a program from an imaging device in which an object to be imaged by the imaging device should exist based on at least one of the focal length of the imaging device, the aperture value, the permissible circle of confusion diameter, and the object distance. The computer may perform the step of determining the distance range. The program may cause the computer to execute a step of specifying the distance from the imaging device to the object. The program may cause the computer to control the position of the imaging device so that the object is included in the distance range based on the distance range and the distance.

  Even when it is desired to capture an image without changing the imaging conditions of the imaging device, it is possible to prevent the object from falling out of the depth of field of the imaging device.

  The above summary of the invention does not enumerate all of the features of the present invention. A subcombination of these feature groups can also be an invention.

It is a figure showing an example of the appearance of a unmanned aerial vehicle. It is a schematic diagram for demonstrating the depth of field. It is a figure which shows an example of the functional block of a unmanned aerial vehicle. It is a figure for demonstrating an example of adjustment of the position of a distance range. It is a figure which shows an example of the time change of the movement distance of a target object and UAV. It is a figure which shows an example of the time change of the movement distance of a target object and UAV. It is a figure which shows an example of the image containing a control | reference point and a target object. 5 is a flowchart illustrating an example of a procedure of controlling the position of the imaging device. It is a flowchart which shows an example of the procedure which estimates the future position of a target object and controls the position of an imaging device. It is a figure showing an example of hardware constitutions.

  Hereinafter, the present invention will be described through the embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Moreover, not all combinations of features described in the embodiments are essential to the solution of the invention.

  The claims, the description, the drawings, and the abstract contain matters which are subject to copyright protection. The copyright holder will not object to any copy of these documents as they appear in the Patent Office file or record. However, in all other cases, all copyrights are reserved.

  Various embodiments of the present invention may be described with reference to the flowcharts and block diagrams. The blocks in the flowcharts and block diagrams may represent (1) steps of a process in which the operation is performed or (2) "parts" of the device responsible for performing the operation. Specific steps and "parts" are supplied with dedicated circuitry, programmable circuitry supplied with computer readable instructions stored on computer readable storage media, and / or computer readable instructions stored on computer readable storage media It may be implemented by a processor. Dedicated circuitry may include digital and / or analog hardware circuitry. Integrated circuits (ICs) and / or discrete circuits may be included. Programmable circuits include, for example, AND, OR, EXCLUSIVE OR, NOR, NOR, and other logical operations, such as Field Programmable Gate Array (FPGA), Programmable Logic Array (PLA), etc. Reconfigurable hardware circuitry may be included, including flip flops, registers, and memory elements.

  Computer readable storage media may include any tangible device capable of storing instructions for execution by a suitable device. As a result, a computer readable storage medium having instructions stored thereon will comprise an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of the computer readable storage medium include floppy disk, diskette, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory) Electrically erasable programmable read only memory (EEPROM), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (registered trademark) disc, memory stick , Integrated circuit cards, etc. may be included.

  Computer readable instructions may include either source code or object code written in any combination of one or more programming languages. Source code or object code includes a conventional procedural programming language. Conventional procedural programming languages include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or like Smalltalk, JAVA, C ++, etc. It may be an object oriented programming language, and a "C" programming language or similar programming language.

  Computer readable instructions may be local or to a wide area network (WAN) such as a local area network (LAN), the Internet, etc., relative to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing device. May be provided via The processor or programmable circuitry may execute computer readable instructions to create a means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

  FIG. 1 shows an example of the appearance of an unmanned aerial vehicle (UAV) 100. The UAV 100 includes a UAV body 102, a gimbal 200, an imaging device 300, and a plurality of imaging devices 230. The UAV 100 is an example of a mobile. The moving body is a concept including UAVs, other aircraft moving in the air, vehicles moving on the ground, ships moving on the water, and the like. The gimbal 200 and the imaging device 300 are an example of an imaging system.

  The UAV body 102 comprises a plurality of rotors. The UAV body 102 causes the UAV 100 to fly by controlling the rotation of a plurality of rotors. The UAV body 102 causes the UAV 100 to fly, for example, using four rotors. The number of rotors is not limited to four. Further, the UAV 100 may be a fixed wing aircraft that does not have a rotary wing.

  The imaging device 300 is a camera for capturing a moving image or a still image. The plurality of imaging devices 230 are sensing cameras that capture the periphery of the UAV 100 to control the flight of the UAV 100. Two imaging devices 230 may be provided at the front, which is the nose of the UAV 100. Two further imaging devices 230 may be provided on the bottom of the UAV 100. The two front imaging devices 230 may be paired to function as a so-called stereo camera. The two imaging devices 230 on the bottom side may also be paired and function as a stereo camera. Based on the images captured by the plurality of imaging devices 230, the distance from the UAV 100 to the object may be measured. Three-dimensional spatial data around the UAV 100 may be generated based on the images captured by the plurality of imaging devices 230. The number of imaging devices 230 included in the UAV 100 is not limited to four. The UAV 100 may include at least one imaging device 230. UAV 100 may include at least one imaging device 230 on each of the nose, aft, sides, bottom, and ceiling of UAV 100. The angle of view that can be set by the imaging device 230 may be wider than the angle of view that can be set by the imaging device 300. The imaging device 230 may have a single focus lens or a fisheye lens.

  In the imaging device 300 mounted on the UAV 100 configured as described above, the position of the UAV 100 is controlled so that the object imaged by the imaging device 300 does not deviate from the depth of field of the imaging device 300.

  FIG. 2 is a schematic view for explaining the depth of field. The focal length of the lens 510 is f, the aperture value of the diaphragm 514 is F, the subject distance indicating the distance from the lens 510 to the in-focus position 502 of the lens 510 is a, the distance from the lens 510 to the image plane of the image sensor 512 The image distance b and the permissible circle of confusion are defined as ε. The subject distance a changes with the position of the lens 510. The permissible circle of confusion diameter ε may change depending on the type of the imaging device 512. The permissible circle of confusion diameter ε may change depending on the size of the image sensor 512. A distance (shooting distance) from the image plane of the imaging element 512 to the target object 500 as a main subject is defined as L. The depth of field may be determined based on at least one of the focal length f of the imaging device, the aperture value F, the permissible circle of confusion diameter ε, and the subject distance a.

The depth of field 520 is the sum of the forward depth of field 522 and the backward depth of field 524. The front depth of field 522 is located on the front side from the focusing position 502. The rear depth of field 524 is located rearward of the focusing position 502. The forward depth of field 522 and the backward depth of field 524 are derived by the following equation.

  The UAV 100 positions the imaging device 300 such that the object 500 is included within the distance range 526 from the imaging device 300 where the object 500 should be present, so that the object 500 does not deviate from the depth of field 520. The position of UAV 100 may be controlled. The width of the distance range 526 may be narrower than the width of the depth of field 520. The distance range 526 may be a range from the distance Lf from the image plane of the imaging device 512 to the distance Lb from the image plane of the imaging device 512 (> distance Lf). Distance range 526 may include a predetermined percentage of forward depth of field 522 and a predetermined percentage of rear depth of field 524. The distance range 526 is, for example, a 95%, 90%, 85%, or 80% range on the side of the in-focus position 502 of the front depth of field 522, and 95 on the side of the in-focus position 502 of the back depth of field 524. A range of%, 90%, 85%, or 80% may be included.

  As described above, even when it is desired that shooting is performed without changing the imaging conditions such as the aperture value F and the focal length f of the imaging device 300 by the UAV 100 flying, the object 500 is a target of the imaging device 300. It is possible to prevent the camera from falling out of the depth of field 520.

  FIG. 3 shows an example of a functional block of the UAV 100. The UAV 100 includes a UAV control unit 110, a communication interface 150, a memory 160, a gimbal 200, a rotary wing mechanism 210, an imaging device 300, an imaging device 230, a GPS receiver 240, an inertial measurement unit (IMU) 250, a magnetic compass 260, and a barometric pressure. An altimeter 270 is provided.

  The communication interface 150 communicates with an external transmitter. The communication interface 150 receives various commands to the UAV control unit 110 from a remote transmitter. The memory 160 is a program necessary for the UAV control unit 110 to control the gimbal 200, the rotary wing mechanism 210, the imaging device 300, the imaging device 230, the GPS receiver 240, the IMU 250, the magnetic compass 260, and the barometric altimeter 270. Store. The memory 160 may be a computer readable recording medium, and may include at least one of a SRAM, a DRAM, an EPROM, an EEPROM, and a flash memory such as a USB memory. The memory 160 may be provided inside the UAV body 102. The memory 160 may be provided to be removable from the UAV body 102.

  The gimbal 200 supports the imaging direction of the imaging device 300 in an adjustable manner. The gimbal 200 rotatably supports the imaging device 300 about at least one axis. The gimbal 200 is an example of a support mechanism. The gimbal 200 may rotatably support the imaging device 300 about the yaw axis, the pitch axis, and the roll axis. The gimbal 200 may change the imaging direction of the imaging device 300 by rotating the imaging device 300 about at least one of the yaw axis, the pitch axis, and the roll axis. The rotary wing mechanism 210 has a plurality of rotary wings and a plurality of drive motors for rotating the plurality of rotary wings.

  The imaging device 230 images the periphery of the UAV 100 to generate image data. Image data of the imaging device 230 is stored in the memory 160. The GPS receiver 240 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 240 calculates the position of the GPS receiver 240, that is, the position of the UAV 100 based on the received plurality of signals. The inertial measurement unit (IMU) 250 detects the attitude of the UAV 100. The IMU 250 detects, as the posture of the UAV 100, accelerations in three axial directions in the front, rear, left, right, upper and lower directions of the UAV 100 and angular velocities in three axial directions of pitch, roll, and yaw. The magnetic compass 260 detects the heading of the UAV 100. The barometric altimeter 270 detects the altitude at which the UAV 100 flies.

  The UAV control unit 110 controls the flight of the UAV 100 according to a program stored in the memory 160. The UAV control unit 110 may be configured by a microprocessor such as a CPU or an MPU or a microcontroller such as an MCU. The UAV control unit 110 controls the flight of the UAV 100 in accordance with an instruction received from a remote transmitter via the communication interface 150.

  The UAV control unit 110 may specify an environment around the UAV 100 by analyzing a plurality of images captured by a plurality of imaging devices 230. The UAV control unit 110 controls the flight, for example, avoiding an obstacle based on the environment around the UAV 100. The UAV control unit 110 may generate three-dimensional space data around the UAV 100 based on a plurality of images captured by a plurality of imaging devices 230 and control flight based on the three-dimensional space data.

  The imaging device 300 includes an imaging unit 301 and a lens unit 401. The lens unit 401 may be a lens unit that can be removed from the imaging unit 301. The imaging unit 301 includes an imaging control unit 310, an imaging element 330, and a memory 340. The imaging control unit 310 may be configured by a microprocessor such as a CPU or an MPU or a microcontroller such as an MCU. The imaging control unit 310 may control the imaging device 300 in accordance with an operation command of the imaging device 300 from the UAV control unit 110.

  The memory 340 may be a computer readable recording medium, and may include at least one of a SRAM, a DRAM, an EPROM, an EEPROM, and a flash memory such as a USB memory. The memory 340 may be provided inside the housing of the imaging unit 301. The memory 340 may be provided to be removable from the housing of the imaging unit 301.

  The imaging device 330 may be configured by a CCD or a CMOS. The imaging element 330 is held inside the casing of the imaging device 300, and outputs image data of an optical image formed through the plurality of lenses 432 and the lens 442 to the imaging control unit 310. The imaging control unit 310 performs a series of image processing such as noise reduction, demosaicing, gamma correction, and edge coordination on the image data. The imaging control unit 310 stores the image data after the series of image processing in the memory 340. The imaging control unit 310 may output image data to the memory 160 via the UAV control unit 110 and store the image data.

  The lens unit 401 includes a lens control unit 410, a memory 420, a lens drive unit 430, a lens 432, a position sensor 434, a lens drive unit 440, a lens 442, a position sensor 444, an aperture drive unit 450, and an aperture 452. The lens 432 includes at least one lens. The lens 432 may be a zoom lens. The lens 442 includes at least one lens. The lens 442 may be a focus lens.

  The lens control unit 410 controls the movement of the lens 432 in the optical axis direction via the lens drive unit 430 in accordance with the lens operation command from the imaging unit 301. The lens control unit 410 controls the movement of the lens 442 in the optical axis direction via the lens drive unit 440 in accordance with the lens operation command from the imaging unit 301. Some or all of the lens 432 and the lens 442 move along the optical axis. The lens control unit 410 performs at least one of the zoom operation and the focus operation by moving at least one of the lens 432 and the lens 442 along the optical axis. The position sensor 434 detects the position of the lens 432. Position sensor 434 may detect the current zoom position. The position sensor 444 detects the position of the lens 442. Position sensor 444 may detect the current focus position.

  The lens driving unit 430 and the lens driving unit 440 may include an actuator. The lens 432 and the lens 442 may move along the optical axis direction via a lens drive mechanism upon receiving power from the respective actuators.

  The aperture 452 adjusts the amount of light incident on the imaging device 330. The diaphragm drive unit 450 may include an actuator. In response to a command from the lens control unit 410, the diaphragm drive unit 450 may drive an actuator to adjust the size of the diaphragm opening, that is, the diaphragm value.

  The memory 420 stores control values of the plurality of lenses 432 and the lenses 442 moving through the lens driving unit 430. The memory 420 may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.

  In the UAV 100 configured as above, the UAV control unit 110 controls the position of the UAV 100 so that the object 500 does not deviate from the depth of field of the imaging device 300.

  The UAV control unit 110 includes a distance range determination unit 111, a distance specification unit 112, a position control unit 113, a selection unit 114, a geographical position specification unit 115, an estimation unit 116, and a geographical area determination unit 117. A control unit other than the UAV control unit 110 may have all or part of the units included in the UAV control unit 110. For example, the imaging control unit 310 may include all or part of the units included in the UAV control unit 110. The transmitter for remotely controlling the UAV 100 may have all or part of the units included in the UAV control unit 110.

  The distance range determination unit 111 should have an object 500 to be imaged by the imaging device 300 based on at least one of the focal length f of the imaging device 300, the aperture value F, the permissible circle of confusion diameter ε, and the object distance a. A distance range 526 from the imaging device 300 is determined. The distance range determination unit 111 is an example of a first determination unit. The distance range determination unit 111 determines the front depth of field 522 according to the above equations (1) and (2) based on the focal length f of the imaging device 300, the aperture value F, the permissible circle of confusion ε, and the subject distance a. And the rear depth of field 524 may be derived. The distance range determination unit 111 may determine the distance range 526 based on the derived front depth of field 522 and the rear depth of field 524. The distance range determination unit 111 determines a predetermined ratio (for example, 90%) of the in-focus position 502 of the front depth of field 522 and a predetermined ratio of the in-focus position 502 of the back depth of field 524. Distance range 526 may be determined to include a percentage (eg, 90%).

  The distance specifying unit 112 specifies the distance L from the imaging device 300 to the object 500. The distance L may be the distance from the image plane of the imaging device 330 to the object 500. The distance specifying unit 112 is an example of a first specifying unit. The distance specifying unit 112 may specify the distance from the UAV 100 to the object 500 as the distance L based on a plurality of images captured by the plurality of imaging devices 230 using a triangular ranging method. The distance specifying unit 112 may specify the distance from the UAV 100 to the object 500 as the distance L using an ultrasonic sensor, an infrared sensor, a radar sensor, or the like.

  The position control unit 113 controls the position of the imaging device 300, that is, the position of the UAV 100 so that the object 500 is included in the distance range 526 based on the distance range 526 and the distance L. The position control unit 113 is an example of a control unit. The UAV control unit 110 may cause the imaging device 300 to image the object when the object is present within the distance range. The UAV control unit 110 may stop the imaging of the object by the imaging device 300 when the object does not exist within the distance range. While the UAV 100 is tracking an object, the UAV control unit 110 may control the position of the UAV 100 so that the object is present within the distance range determined by the distance range determination unit 111. While the UAV 100 hovers around the object, the UAV control unit 110 may control the position of the UAV 100 such that the object is present within the distance range determined by the distance range determination unit 111.

  FIG. 4 is a diagram for describing an example of adjustment of the position of the distance range. It is assumed that the object 500 moves to the position of the object 500 ′ and deviates from the distance range 600 of the depth of field 610. In this case, the position control unit 113 controls, for example, the position of the UAV 100 such that the object 500 'is at the center of the distance range. As the UAV 100 moves, the distance range moves from the distance range 600 to the distance range 600 ′.

  The position control unit 113 may adjust the position within the distance range 526 where the object 500 should be present, in accordance with the relative movement direction of the object 500 and the imaging device 300. For example, it is assumed that the object 500 is approaching the imaging device 300. In this case, the object 500 deviates from the end on the opposite side to the end on the focusing position 502 side of the front depth of field 522. That is, the object 500 is disengaged from the front end of the front depth of field 522. In this case, the position control unit 113 may control the position of the UAV 100 such that the object 500 is located within the rear depth of field 524. On the other hand, it is assumed that the object 500 is away from the imaging device 300. In this case, the object 500 deviates from the end on the opposite side to the end on the focusing position 502 side of the rear depth of field 524. That is, the object 500 is disengaged from the rear end of the rear depth of field 524. In this case, the position control unit 113 may control the position of the UAV 100 such that the object 500 is positioned within the front depth of field 522. When the relative moving direction between the object 500 and the imaging device 300 is constant, the object 500 captures an image by controlling the position of the UAV 100 so that the position of the object 500 is as described above. It is possible to further suppress the deviation from the depth of field 520 of the device 300. Alternatively, the frequency with which the position control unit 113 adjusts the position of the UAV 100 based on the distance range 526 can be reduced.

  The depth of field 520 is derived by Equations (1) and (2) as described above. Even if the focal distance f is constant, the depth of field 520 changes according to the size of the aperture value F, for example. For example, the greater the aperture value F, the shallower the depth of field 520. Since the permissible circle of confusion ε varies with the size of the imaging device 330, the depth of field 520 varies with the size of the imaging device 330 even if the focal length f is constant. The depth of field 520 changes depending on the size of the image sensor 330 even if the size of the permissible circle of confusion diameter ε is the same. As the size of the imaging device 330 is larger, the depth of field 520 is shallower.

  If the size (width) of the depth of field changes, the width of the distance range in which the object should exist also changes. As the distance range is narrower, the UAV 100 needs to keep the distance to the object constant more accurately. FIG. 5A and FIG. 5B show an example of the temporal change of the movement distance of the object 500 and the UAV 100. The width of the distance range 700 shown in FIG. 5B is narrower than the width of the distance range 700 shown in FIG. 5A. As shown in FIGS. 5A and 5B, the narrower the distance range 700, the shorter the time 702 from when the object 500 starts to shift from the center of the distance range 700 to when the end of the distance range 700 is reached. Thus, the shallower the depth of field, the faster the UAV 100 needs to operate. Therefore, the UAV 100 may operate so as to increase the acceleration as the depth of field is shallower. When the UAV 100 operates within a preset maximum acceleration, the maximum acceleration may be set to increase as the depth of field becomes shallower. The shallower the depth of field, the faster the distance between the UAV 100 and the object can be corrected. When it is desired to keep the distance between the UAV 100 and the object constant, for example, when the UAV 100 tracks or hovers while keeping the distance to the object constant, the maximum acceleration is increased as the depth of field is shallower. May be effective.

  The UAV control unit 110 may predict the movement of the object, determine the target position to be reached by the UAV 100 based on the predicted future position and distance range of the object, and move the UAV 100 to the target position. . In order to implement such control, the UAV control unit 110 may further include a selection unit 114, a geographical location unit 115, an estimation unit 116, and a geographical area determination unit 117.

  The selection unit 114 selects, from among the images captured by the imaging device 300, a reference point whose geographical position has been determined in advance and an object. The reference point is an example of the first point. The reference point at which the geographical position is determined in advance may be an immobile object. The reference point may be a landmark having a geographical position (latitude, longitude, altitude, etc.) registered in advance on a map database that can be referred to by the UAV control unit 110. The selection unit 114 may select a reference point and an object in response to an instruction from the user from the image captured by the imaging device 300.

  FIG. 6 shows an example of an image 800 including a reference point 802 and an object 804 captured by the imaging device 300. The user designates a mountain that can be a landmark from the image 800 as a reference point 802. The image 800 may be displayed, for example, on a display provided to a transmitter that remotely controls the UAV 100. Furthermore, the user designates an object 804 as a main subject in the image 800. The selecting unit 114 may compare the image 800 with the image corresponding to the landmark registered in advance in the map database, and search an area that can be the landmark from the image 800. The map information including the current position of the UAV 100, the position of the reference point 802, and the current position of the object 804 may be superimposed on the image 800 captured by the imaging device 300 and displayed on the display.

  The geographical position specifying unit 115 specifies the geographical position of the object based on the positional relationship between the reference point and the object in the image imaged by the imaging device 300 and the geographical position of the reference point. The geographical position specifying unit 115 is an example of a second specifying unit. The geographical position specifying unit 115 may generate a distance image indicating the distance to the imaging device 300 for each unit area in the image, using two images captured for each unit time by the imaging device 300. . The geographical position specifying unit 115 specifies the positional relationship between the reference point and the object using the distance image. The geographical position specifying unit 115 specifies the geographical position of the reference point, for example, with reference to the map database stored in the memory 160. The geographical location unit 115 identifies the geographical location of the object based on the identified positional relationship and the geographical location of the reference point.

  The estimation unit 116 estimates the geographical position of the object at a first future point in time. The estimation unit 116 estimates the geographical position of the object at the first time based on the plurality of geographical positions of the object specified by the geographical position specification unit 115 at the plurality of time points up to now. The estimation unit 116 may estimate the geographical position of the object for each unit time determined based on the frame rate.

  The geographical area determination unit 117 determines the geographical area where the imaging device 300 should exist at the first time point, based on the geographical position and the distance range of the object estimated by the estimation unit 116. That is, the geographical area determination unit 117 determines the geographical area where the UAV 100 should exist at the first time based on the geographical position and the distance range of the object estimated by the estimation unit 116. If the UAV 100 is within the geographical area at a first time, at least the object can be located within the depth of field of the imaging device 300 at a first time. The position control unit 113 controls the position of the imaging device 300 so that the imaging device 300 exists in the geographical area at the first time point. The position control unit 113 may determine an optimal target position in the geographical area by referring to the geographical area and the map database. The position control unit 113 may determine a position that can be reached at the shortest distance as a target position in the geographical area. The position control unit 113 refers to the map database to identify obstacles that impede flight before reaching the geographical area, and positions within the geographical area that can bypass the obstacle and reach the shortest distance. You may decide on the target position.

  FIG. 7 is a flowchart showing an example of the procedure for controlling the position of the imaging device 300. The selection unit 114 selects an object specified by the user from the images captured by the imaging device 300 (S010). The imaging condition of the imaging device 300 optimal for imaging the object is determined, and the imaging condition is maintained until the processing is completed. That is, the imaging device 300 maintains the focal length f, the aperture value F, and the subject distance a determined as the imaging condition. Then, the distance range determination unit 111 acquires information of the focal length f, the aperture value F, the permissible circle of confusion diameter ε, the subject distance a, and the image distance b from the imaging device 300. The distance specifying unit 112 specifies the depth of field of the imaging device 300 based on the focal length f, the aperture value F, the permissible circle of confusion diameter ε, and the subject distance a. The distance range determination unit 111 determines the distance range from the imaging device 300 where the object should exist, based on the depth of field, the subject distance a, and the image distance b (S102).

  The distance specifying unit 112 specifies the distance L from the imaging device 300 to the object (S104). The distance specifying unit 112 may specify, for example, the distance to the object derived from the plurality of images captured by the plurality of imaging devices 230 as the distance L. The position control unit 113 determines whether the specified distance L is within the distance range (S106). If the object does not exist within the distance range, the position control unit 113 controls the flight of the UAV 100 so as to adjust the position of the imaging device 300 so that the object is included within the distance range (S108). If the object is within the distance range, the UAV control unit 110 continues the flight control of the current UAV 100.

  The UAV control unit 110 continues the process from step S104 to step S108 until the process of controlling the position of the imaging device 300 is finished so that the object exists within the distance range (S110).

  According to the above-described procedure, the UAV 100 can cause the imaging device 300 to image the object without the object falling outside the depth of field of the imaging device 300.

  FIG. 8 is a flowchart showing an example of a procedure of estimating the future position of the object to control the position of the imaging device 300. The selection unit 114 selects a target designated by the user and a reference point to be a landmark from the image captured by the imaging device 300 (S200). The distance range determination unit 111 acquires information of the focal length f, the aperture value F, the permissible circle of confusion diameter ε, the subject distance a, and the image distance b from the imaging device 300. The distance specifying unit 112 specifies the depth of field of the imaging device 300 based on the focal length f, the aperture value F, the permissible circle of confusion diameter ε, and the subject distance a. The distance range determination unit 111 determines a distance range from the imaging device 300 where the object should exist, based on the depth of field, the subject distance a, and the image distance b (S202).

  The geographical position specifying unit 115 specifies the geographical position of the reference point with reference to the map database (S204). The geographical position specifying unit 115 creates a distance image indicating the distance to the imaging device 300 for each unit area in the image based on the plurality of images captured for each unit time by the imaging device 300 (S206). The geographical position specifying unit 115 specifies the geographical position of the object based on the geographical position of the reference point and the distance image (S208).

  The estimation unit 116 estimates the geographical position of the object at a first point in time in the future, based on the plurality of geographical positions of the object specified by the geographical position determination unit 115 so far (S210). The geographical area determination unit 117 determines the geographical area where the UAV 100 should exist at the first time based on the geographical position of the object at the first time and the distance range (S212). The position control unit 113 controls the flight of the UAV 100 based on the geographical area and the current geographical position of the UAV 100 (S214). The position control unit 113 controls the flight of the UAV 100 such that the UAV 100 is located within the geographical area at a first point of time.

  As described above, the UAV 100 estimates the position of the future object, and identifies the geographical area where the UAV 100 should exist in the future, based on the estimation result and the distance range in which the object should exist. The UAV 100 can determine an optimal flight path in consideration of obstacles and the like with reference to the identified geographical area and map database. Thus, the UAV 100 can fly in an optimal flight path so that the object does not deviate from the depth of field of the imaging device 300.

  FIG. 9 shows an example of a computer 1200 in which aspects of the present invention may be fully or partially embodied. A program installed on computer 1200 can cause computer 1200 to act as an operation or one or more "parts" of the device according to embodiments of the invention. Alternatively, the program may cause the computer 1200 to execute the operation or one or more “parts”. The program can cause the computer 1200 to execute the process according to the embodiment of the present invention or the steps of the process. Such programs may be executed by CPU 1212 to cause computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

  The computer 1200 according to the present embodiment includes a CPU 1212 and a RAM 1214, which are interconnected by a host controller 1210. Computer 1200 also includes a communication interface 1222, an input / output unit, which are connected to host controller 1210 via an input / output controller 1220. Computer 1200 also includes a ROM 1230. The CPU 1212 operates in accordance with programs stored in the ROM 1230 and the RAM 1214 to control each unit.

  The communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores therein a boot program or the like executed by the computer 1200 upon activation, and / or a program dependent on the hardware of the computer 1200. The program is provided via a computer readable recording medium or network such as a CR-ROM, a USB memory or an IC card. The program is installed in the RAM 1214 or ROM 1230, which is also an example of a computer-readable recording medium, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200 and brings about coordination between the programs and the various types of hardware resources. An apparatus or method may be configured by implementing the operation or processing of information in accordance with the use of computer 1200.

  For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes the communication program loaded in the RAM 1214, and performs communication processing to the communication interface 1222 based on the processing described in the communication program. You may order. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214 or a USB memory under the control of the CPU 1212 and transmits the read transmission data to the network, or The received data received from the network is written in a reception buffer area or the like provided on the recording medium.

  In addition, the CPU 1212 allows the RAM 1214 to read all or necessary portions of a file or database stored in an external recording medium such as a USB memory, and executes various types of processing on data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to the external storage medium.

  Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium and subjected to information processing. The CPU 1212 describes various types of operations, information processing, condition judgment, conditional branching, unconditional branching, information retrieval, which are described throughout the present disclosure for data read from the RAM 1214 and specified by a program instruction sequence. Various types of processing may be performed, including / replacement etc, and the results written back to RAM 1214. Further, the CPU 1212 may search for information in a file in a recording medium, a database or the like. For example, when a plurality of entries each having the attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. Search for an entry matching the condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby associate the first attribute satisfying the predetermined condition with the first attribute An attribute value of the second attribute may be acquired.

  The programs or software modules described above may be stored on computer readable storage medium on or near computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer readable storage medium, whereby the program can be transmitted to the computer 1200 via the network. provide.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It is apparent to those skilled in the art that various changes or modifications can be added to the above embodiment. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and steps in the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly "before", "before" It should be noted that it can be realized in any order, unless explicitly stated as etc., and unless the output of the previous process is used in the later process. With regard to the operation flow in the claims, the specification, and the drawings, even if it is described using “first,” “next,” etc. for the sake of convenience, it means that it is essential to carry out in this order. is not.

100 UAV
102 UAV main unit 110 UAV control unit 111 distance range determination unit 112 distance specification unit 113 position control unit 114 selection unit 115 geographical position specification unit 116 estimation unit 117 geographical area determination unit 150 communication interface 160 memory 200 gimbal 210 rotary wing mechanism 230 Imaging device 240 GPS receiver 260 Magnetic compass 270 Barometric altimeter 300 Imaging device 301 Imaging unit 310 Imaging unit 330 Imaging control unit 330 Imaging device 340 Memory 401 Lens unit 410 Lens control unit 420 Memory 430 Lens drive unit 432 Lens 434 Position sensor 440 Lens drive unit 442 Lens 444 Position sensor 450 Aperture drive unit 452 Aperture 500 Object 502 Focus position 510 Lens 512 Image sensor 514 Aperture 1200 Computer 1210 Host controller 1212 CPU
1214 RAM
1220 I / O controller 1222 communication interface 1230 ROM

Claims (13)

A first determination to determine a distance range from the image pickup apparatus in which an object to be picked up by the image pickup apparatus should exist based on at least one of focal length of the image pickup apparatus, aperture value, permissible circle of confusion diameter, and subject distance Department,
A first identification unit that identifies a distance from the imaging device to the object;
And a control unit configured to control the position of the imaging device such that the object is included in the distance range based on the distance range and the distance .
The control device, wherein a width of the distance range is narrower than a width of a depth of field of the imaging device determined based on at least one of a focal length of the imaging device, an aperture value, a permissible circle of confusion diameter, and a subject distance .   The control device according to claim 1, wherein the subject distance is based on a position of a lens of the imaging device. An estimation unit for estimating a geographical position of the object at a first future point in time;
And a second determination unit that determines a geographical area where the imaging device should be present at the first time based on the estimated geographical position and the distance range.
The control device according to claim 1, wherein the control unit controls the position of the imaging device such that the imaging device exists in the geographical area at the first time point. A first point at which a geographical position is determined in advance from an image captured by the imaging device, and a selection unit for selecting the object;
Second identifying a geographical position of the object based on a positional relationship between the first point and the object in an image captured by the imaging device, and a geographical position of the first point And a specific part,
The estimation unit estimates a geographical position of the object at the first time based on a plurality of geographical positions of the object specified by the second specifying unit at a plurality of time points up to now. The control device according to claim 3 . A first determination to determine a distance range from the image pickup apparatus in which an object to be picked up by the image pickup apparatus should exist based on at least one of the focal length of the image pickup apparatus, the aperture value, the permissible circle of confusion diameter, and the subject distance Department,
  A first identification unit that identifies a distance from the imaging device to the object;
  A control unit configured to control the position of the imaging device such that the object is included in the distance range based on the distance range and the distance;
  A first point at which a geographical position is determined in advance from an image captured by the imaging device, and a selection unit for selecting the object;
  Second identifying a geographical position of the object based on a positional relationship between the first point and the object in an image captured by the imaging device, and a geographical position of the first point A specific part,
  An estimation unit for estimating a geographical position of the object at a first future time based on a plurality of geographical positions of the object specified by the second specifying unit at a plurality of time points up to now;
  A second determination unit that determines a geographical area that the imaging device should be present at the first time based on the estimated geographical position and the distance range;
Equipped with
  The control unit controls the position of the imaging device such that the imaging device exists in the geographical area at the first time point. The control device according to any one of claims 1 to 5, wherein the control unit causes the imaging device to image the object when the object is present within the distance range.   The control device according to any one of claims 1 to 6, and a movable body mounted and moved with the imaging device.   The movable body according to claim 7, wherein the control unit controls the position of the movable body so that the object is included in the distance range. The mobile unit according to claim 8, wherein the control unit causes the mobile unit to track the target such that the target is included in the distance range. The moving body is an unmanned aerial vehicle,
The mobile unit according to claim 8, wherein the control unit causes the unmanned aircraft to hover so that the object is included in the distance range. Determining a distance range from the image pickup apparatus in which the object to be picked up by the image pickup apparatus should exist based on at least one of the focal length of the image pickup apparatus, the aperture value, the permissible circle of confusion diameter, and the subject distance;
Identifying a distance from the imaging device to the object;
Controlling the position of the imaging device so that the object is included in the distance range based on the distance range and the distance ;
The control method, wherein a width of the distance range is narrower than a depth of field of the imaging device determined based on at least one of a focal length of the imaging device, an aperture value, a permissible circle of confusion diameter, and a subject distance . Determining a distance range from the image pickup apparatus in which the object to be picked up by the image pickup apparatus should exist based on at least one of the focal length of the image pickup apparatus, the aperture value, the permissible circle of confusion diameter, and the subject distance;
  Identifying a distance from the imaging device to the object;
  Controlling the position of the imaging device to include the object within the distance range based on the distance range and the distance;
  Selecting a first point whose geographical position is previously determined from the image captured by the imaging device, and the object;
  Identifying the geographical position of the object based on the positional relationship between the first point and the object in the image captured by the imaging device, and the geographical position of the first point; ,
  Estimating a geographical position of the object at a first future time based on a plurality of geographical positions of the object specified at a plurality of time points up to now;
  Determining the geographical area that the imaging device should be present at the first time based on the estimated geographical position and the distance range;
Equipped with
  Controlling the position of the imaging device such that the imaging device is within the geographical area at the first point in time. A program for causing a computer to function as the control device according to any one of claims 1 to 6 .

Images