JP2019096965A - Determination device, control arrangement, imaging system, flying object, determination method, program - Google Patents

Abstract

To provide a determination device for determining a place where a flying object is not flying but stopped, and capable of making an imaging apparatus capture an image of desired composition, when it is desired to make the imaging apparatus capture an image of desired composition, in a state where the flying object such as an unmanned aircraft mounting the imaging apparatus is in a state not flying but resting.SOLUTION: A resting possible area 508 where an unmanned aircraft (UAV) 10 mounting an imaging apparatus can be rested without flying is identified from a resting candidate region 506, and then an imaging possible area 510 capable of capturing an image of composition satisfying predetermined conditions is determined from the resting possible area thus determined.SELECTED DRAWING: Figure 4A

Description

  The present invention relates to a determination device, a control device, an imaging system, an aircraft, a determination method, and a program.

Patent Document 1 describes that the unmanned aerial vehicle unit is vertically lowered and landed after imaging a structure.
Patent Document 1: JP-A-2017-138162

  It may be desired that the imaging device causes the imaging device to capture an image of a desired composition while the flying object such as an unmanned aerial vehicle equipped with the imaging device is stationary without flying such as in a landed state. In such a case, the imaging device may not be able to capture an image of a desired composition depending on the place where the flight body is stopped without flying.

  The determination apparatus according to an aspect of the present invention may include a specifying unit that specifies a restable area in which an aircraft equipped with an imaging device can stand without flying. The determination device may include a determination unit that determines an imageable area capable of capturing an image of a composition that satisfies a predetermined condition from among the stationary area.

  The determination unit can capture an image including a predetermined object and not including another object existing in a range up to a predetermined distance shorter than the distance from the imaging device to the predetermined object. The area may be determined as an imageable area.

  The determination apparatus may include a first reception unit that receives specification of a predetermined object.

  The first receiving unit may receive specification of a predetermined object from among the images captured by the imaging device.

  The determination device may include a second reception unit that receives specification of a planned stationary area to be stationary without causing the flight body to fly. The determination device may include an extraction unit that extracts a stationary candidate region that satisfies a predetermined condition indicating flat from the planned stationary region based on the image captured by the imaging device.

  The determination device may include an acquisition unit that acquires distance information indicating a distance from the imaging device to the stationary candidate area. The identifying unit may identify the stationary area from among the stationary candidate areas based on the distance information.

  The second receiving unit may receive the designation of the planned stationary region based on the image captured by the imaging device while the flying object is flying.

  The determination apparatus may include a third reception unit that receives specification of an image of a composition that satisfies a predetermined condition.

  The third reception unit may receive a specification of an image of a composition that satisfies a predetermined condition from among images captured by the imaging device while the flight body is flying.

  The determination unit is present on an extension of the imaging direction of the imaging device from the position of the aircraft when the imaging device captures an image of a composition that satisfies a predetermined condition while the aircraft is flying from the stationary area. An area to be captured may be determined as an image captureable area.

  The determination unit may determine the imageable area based on the range of the angle of view that can be set for the imaging device.

  The flying object may be equipped with a support mechanism that supports the imaging device such that the imaging direction of the imaging device can be adjusted. The determination unit may determine the imageable area based on the range of the imaging direction of the imaging device that can be adjusted by the support mechanism.

  The control device according to an aspect of the present invention may include the above-described determination device. A control unit controls the flying object to cause the flying object to stand still without flying to the image captureable area, and controls the imaging device to cause the imaging device to capture an image of a composition satisfying a predetermined condition May be provided.

  An imaging system according to an aspect of the present invention may include the control device. The imaging system may comprise an imaging device.

  An aircraft according to an aspect of the present invention flies with the imaging system.

  The flying object may include a support mechanism that supports the imaging device such that the attitude of the imaging device can be adjusted.

  The determination method according to an aspect of the present invention may include the step of identifying a stationary area where the aircraft carrying the imaging device can stand without flying. The determination method may include the step of determining, from among the stationary regions, an imageable region capable of capturing an image of a composition that satisfies a predetermined condition.

  A program according to an aspect of the present invention may cause a computer to execute a step of specifying a restable area in which an aircraft equipped with an imaging device can stand without flying. The program may cause the computer to execute, from among the stationary regions, determining an imageable region capable of capturing an image of a composition that satisfies a predetermined condition.

  According to one aspect of the present invention, it is possible to cause the imaging device to more reliably capture an image of a desired composition while the aircraft is stationary without flying.

  Note that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a subcombination of these feature groups can also be an invention.

It is a figure which shows an example of the external appearance of a unmanned aerial vehicle and a remote control. It is a figure which shows an example of the functional block of a unmanned aerial vehicle. It is a figure for demonstrating a stationary plan area | region, a stationary candidate area | region, a stationary possible area | region, and an imaging possible area. It is a figure for demonstrating the determination method of an imaging | photography possible area | region. It is the figure which looked at the condition shown to FIG. 4A from the top. It is a figure for demonstrating the determination method of an imaging | photography possible area | region. It is a flowchart which shows an example of imaging | photography procedure. It is a flowchart which shows an example of imaging | photography procedure. It is a figure for demonstrating an example of a hardware configuration.

  Hereinafter, although this invention is demonstrated through embodiment of invention, the following embodiment does not limit the invention which concerns on a claim. Further, not all combinations of features described in the embodiments are essential to the solution of the invention. It will be apparent to those skilled in the art that various changes or modifications can be added to the following embodiments. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

  The claims, the description, the drawings, and the abstract contain matters that are subject to copyright protection. The copyright holder will not object to any copy of these documents as they appear in the Patent Office file or record. However, in all other cases, all copyrights are reserved.

  Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, wherein the blocks are responsible for (1) process steps or (2) operations being performed. May represent a "part" of The particular stages and "parts" may be implemented by programmable circuits and / or processors. Dedicated circuitry may include digital and / or analog hardware circuitry. Integrated circuits (ICs) and / or discrete circuits may be included. Programmable circuits may include reconfigurable hardware circuits. Reconfigurable hardware circuits include logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, flip flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. Memory elements, etc. may be included.

  Computer readable media may include any tangible device capable of storing instructions for execution by a suitable device. As a result, a computer readable medium having instructions stored thereon will comprise an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-Ray (RTM) Disc, Memory Stick, Integrated A circuit card or the like may be included.

  Computer readable instructions may include either source code or object code written in any combination of one or more programming languages. Source code or object code includes a conventional procedural programming language. Conventional procedural programming languages include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or like Smalltalk, JAVA, C ++, etc. It may be an object oriented programming language, and a "C" programming language or similar programming language. Computer readable instructions may be local or to a wide area network (WAN) such as a local area network (LAN), the Internet, etc., relative to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing device. May be provided via The processor or programmable circuitry may execute computer readable instructions to create a means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

  FIG. 1 shows an example of the appearance of an unmanned aerial vehicle (UAV) 10 and a remote control device 300. The UAV 10 includes a UAV main body 20, a gimbal 50, a plurality of imaging devices 60, and an imaging device 100. The gimbal 50 and the imaging device 100 are examples of an imaging system. The UAV 10 is an example of an aircraft moving in the air. The term “air vehicle” is a concept including UAVs, other aircraft moving in the air, airships, helicopters and the like.

  The UAV body 20 comprises a plurality of rotors. The plurality of rotors are an example of the propulsion unit. The UAV body 20 causes the UAV 10 to fly by controlling the rotation of a plurality of rotors. The UAV body 20 causes the UAV 10 to fly using, for example, four rotors. The number of rotors is not limited to four. The UAV 10 may also be a fixed wing aircraft that does not have a rotor.

  The imaging device 100 is a camera for imaging which captures an object included in a desired imaging range. The gimbal 50 rotatably supports the imaging device 100. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 rotatably supports the imaging device 100 on the pitch axis using an actuator. The gimbal 50 rotatably supports the imaging device 100 about each of the roll axis and the yaw axis using an actuator. The gimbal 50 may change the attitude of the imaging device 100 by rotating the imaging device 100 about at least one of the yaw axis, the pitch axis, and the roll axis.

  The plurality of imaging devices 60 are cameras for sensing that capture the periphery of the UAV 10 to control the flight of the UAV 10. Two imaging devices 60 may be provided at the front, which is the nose of the UAV 10. Two further imaging devices 60 may be provided on the bottom of the UAV 10. The two front imaging devices 60 may be paired to function as a so-called stereo camera. The two imaging devices 60 on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the UAV 10 may be generated based on the images captured by the plurality of imaging devices 60. The number of imaging devices 60 provided in the UAV 10 is not limited to four. The UAV 10 may include at least one imaging device 60. The UAV 10 may include at least one imaging device 60 on each of the nose, aft, sides, bottom, and ceiling of the UAV 10. The angle of view that can be set by the imaging device 60 may be wider than the angle of view that can be set by the imaging device 100. The imaging device 60 may have a single focus lens or a fisheye lens.

  The remote control device 300 communicates with the UAV 10 to remotely control the UAV 10. The remote control device 300 may communicate with the UAV 10 wirelessly. The remote control device 300 transmits instruction information indicating various commands related to the movement of the UAV 10 such as rising, falling, acceleration, deceleration, forward, reverse, and rotation to the UAV 10. The instruction information includes, for example, instruction information for raising the altitude of the UAV 10. The instruction information may indicate the altitude at which the UAV 10 should be located. The UAV 10 moves so as to be located at the altitude indicated by the instruction information received from the remote control device 300. The instruction information may include a lift instruction to raise the UAV 10. The UAV 10 goes up while accepting the up command. Even if the UAV 10 receives an ascent instruction, if the UAV 10's altitude reaches the upper limit altitude, the UAV 10 may limit the ascent.

  FIG. 2 shows an example of a functional block of the UAV 10. The UAV 10 includes a UAV control unit 30, a memory 32, a communication interface 36, a propulsion unit 40, a GPS receiver 41, an inertial measurement device 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60 and the imaging device 100.

  The communication interface 36 communicates with other devices such as the remote control device 300. The communication interface 36 may receive instruction information including various instructions for the UAV control unit 30 from the remote control device 300. The memory 32 includes a propulsion unit 40, a GPS receiver 41, an inertial measurement unit (IMU) 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60, and a memory 32. And the program etc. required to control the imaging device 100 are stored. The memory 32 may be a computer readable recording medium, and may include at least one of a SRAM, a DRAM, an EPROM, an EEPROM, and a flash memory such as a USB memory. The memory 32 may be provided inside the UAV body 20. It may be provided to be removable from the UAV main body 20.

  The UAV control unit 30 controls the flight and imaging of the UAV 10 in accordance with a program stored in the memory 32. The UAV control unit 30 may be configured by a microprocessor such as a CPU or an MPU or a microcontroller such as an MCU. The UAV control unit 30 controls the flight and imaging of the UAV 10 according to the command received from the remote control device 300 via the communication interface 36. The promotion unit 40 promotes the UAV 10. The propulsion unit 40 has a plurality of rotors and a plurality of drive motors for rotating the plurality of rotors. The propulsion unit 40 rotates the plurality of rotary blades via the plurality of drive motors in accordance with the instruction from the UAV control unit 30 to fly the UAV 10.

  The GPS receiver 41 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 41 calculates the position (latitude and longitude) of the GPS receiver 41, that is, the position (latitude and longitude) of the UAV 10, based on the plurality of received signals. The IMU 42 detects the attitude of the UAV 10. The IMU 42 detects, as the posture of the UAV 10, accelerations in the front, rear, left, right, and top three axial directions of the UAV 10, and angular velocities in three axial directions of pitch, roll, and yaw. The magnetic compass 43 detects the heading of the UAV 10. The barometric altimeter 44 detects the altitude at which the UAV 10 flies. The barometric pressure altimeter 44 detects the barometric pressure around the UAV 10, converts the detected barometric pressure into a height, and detects the height. The temperature sensor 45 detects the temperature around the UAV 10. The humidity sensor 46 detects the humidity around the UAV 10.

  The imaging device 100 includes an imaging unit 102 and a lens unit 200. The lens unit 200 is an example of a lens device. The imaging unit 102 includes an image sensor 120, an imaging control unit 110, and a memory 130. The image sensor 120 may be configured by a CCD or a CMOS. The image sensor 120 captures an optical image formed through the plurality of lenses 210, and outputs the captured image data to the imaging control unit 110. The imaging control unit 110 may be configured by a microprocessor such as a CPU or an MPU, a microcontroller such as an MCU, or the like. The imaging control unit 110 may control the imaging device 100 according to an operation command of the imaging device 100 from the UAV control unit 30. The memory 130 may be a computer readable recording medium, and may include at least one of SRAM, DRAM, EPROM, EEPROM, and flash memory such as USB memory. The memory 130 stores programs and the like necessary for the imaging control unit 110 to control the image sensor 120 and the like. The memory 130 may be provided inside the housing of the imaging device 100. The memory 130 may be provided to be removable from the housing of the imaging device 100.

  The lens unit 200 includes a plurality of lenses 210, a plurality of lens drivers 212, and a lens controller 220. The plurality of lenses 210 may function as a zoom lens, a varifocal lens, and a focus lens. At least a part or all of the plurality of lenses 210 is disposed movably along the optical axis. The lens unit 200 may be an interchangeable lens provided detachably to the imaging unit 102. The lens driving unit 212 moves at least a part or all of the plurality of lenses 210 along the optical axis via a mechanical member such as a cam ring. The lens driver 212 may include an actuator. The actuator may include a stepping motor. The lens control unit 220 drives the lens driving unit 212 according to the lens control command from the imaging unit 102 to move one or more lenses 210 along the optical axis direction via the mechanical member. The lens control command is, for example, a zoom control command and a focus control command.

  The lens unit 200 further includes a memory 222 and a position sensor 214. The lens control unit 220 controls movement of the lens 210 in the optical axis direction via the lens driving unit 212 in accordance with the lens operation command from the imaging unit 102. The lens control unit 220 controls movement of the lens 210 in the optical axis direction via the lens driving unit 212 in accordance with the lens operation command from the imaging unit 102. Some or all of the lenses 210 move along the optical axis. The lens control unit 220 performs at least one of the zoom operation and the focusing operation by moving at least one of the lenses 210 along the optical axis. The position sensor 214 detects the position of the lens 210. Position sensor 214 may detect the current zoom position or focus position.

  The lens drive unit 212 may include a shake correction mechanism. The lens control unit 220 may execute shake correction by moving the lens 210 in the direction along the optical axis or in the direction perpendicular to the optical axis via the shake correction mechanism. The lens drive unit 212 may drive a shake correction mechanism by a stepping motor to execute shake correction. The shake correction mechanism may be driven by a stepping motor to move the image sensor 120 in the direction along the direction of the optical axis or in the direction perpendicular to the optical axis to perform the shake correction.

  The memory 222 stores control values of the plurality of lenses 210 moving through the lens driving unit 212. The memory 222 may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.

  The imaging device 100 configured in this way is often used for imaging the UAV 10 during flight. The time that can be taken by the imaging device 100 while the UAV 10 flies depends on the capacity of the battery mounted on the UAV 10 and the like. In order to increase the shooting time, it is desirable to make the flight of UAV 10 as short as possible. With the UAV 10 stationary without flying, for example, the imaging device 100 may be able to capture an image of a desired composition with the UAV 10 landing. If the imaging device 100 can perform imaging in such a state, consumption of the battery of the UAV 10 can be suppressed, and the imaging available time can be extended. However, depending on the location where the UAV 10 is to be stopped without flying, the imaging device 100 may not be able to capture an image of a desired composition due to the presence of an obstacle or the like.

  Therefore, according to the UAV 10 according to the present embodiment, an imageable area capable of capturing an image of a composition that satisfies a predetermined condition is searched. The UAV 10 stands still without flying to the searched imageable area, and the imaging apparatus 100 captures an image of a composition that satisfies a predetermined condition. The UAV 10 may, for example, land in the imageable area and stand still without flying. The UAV 10 does not fly by hooking a hook or the like provided on the UAV main body 20 or the like to a hook or a branch such as a branch of a tree provided on a building or pole. It may be stationary.

  The UAV control unit 30 includes a reception unit 61, an extraction unit 62, an acquisition unit 63, a specification unit 64, a determination unit 65, and an imaging instruction unit 70. The UAV control unit 30 is an example of a determination device that determines an image captureable area. At least a part of the functions of the UAV control unit 30 may be provided to a control unit other than the UAV control unit 30, for example, the imaging control unit 110 or the remote control device 300.

  The receiving unit 61 receives various instructions from the user via the remote control device 300 or the like. The reception unit 61 is an example of a first reception unit, a second reception unit, and a third reception unit. The receiving unit 61 receives specification of a predetermined object to be imaged. The accepting unit 61 may accept designation of a predetermined object from among the images captured by the imaging device 100. The accepting unit 61 may accept designation of a desired person, a structure, a landscape, etc. from the image captured by the imaging device 100. The reception unit 61 may receive an image including a desired object from among images stored in advance in the UAV 10, the remote control device 300, or a database on a network as a predetermined object. At this time, the UAV 10 may set an object based on the position information by the position information device provided in the remote control device 300. As the position information, for example, GPS (registered trademark) from the remote control device 300 or Bluetooth (registered trademark) can be used. For example, when shooting a person who operates the remote control device 300, the remote control device 300 can shoot an object by using position information.

  The receiving unit 61 receives the specification of the stationary scheduled area to be stationary without causing the UAV 10 to fly. The accepting unit 61 may accept the designation of the planned stationary region based on the image captured by the imaging device 100 while the UAV 10 is flying. The receiving unit 61 may receive specification of a plurality of planned stationary areas. The receiving unit 61 may receive a predetermined area in the image as a stationary scheduled area. The accepting unit 61 may accept the designation of a scheduled stationary area, which is an area that the user wants to stand still without causing the UAV 10 to fly, based on the image captured by the imaging device 100 while the UAV 10 is flying. The reception unit 61 may receive, from the image captured by the imaging device 100 while the UAV 10 is flying, a region where the user wants to land the UAV 10 designated by the user as a stationary scheduled region. The user may specify, for example, the stationary scheduled area 502 from an image 500 as shown in FIG. 3 captured by the imaging device 100, displayed on the display of the remote control device 300. The user may specify, for example, an area on which a UAV 10 is likely to land, such as on a cliff, on a roof, or on furniture, from among the images captured by the imaging device 100.

  The extraction unit 62 extracts a still candidate area satisfying a predetermined condition indicating being flat out of the planned stationary area based on the image captured by the imaging device 100. For example, the extraction unit 62 divides the image 500 into a plurality of blocks 504. The extraction unit 62 derives the feature amount for each of the blocks 504 in the planned stationary region. The feature amount may be luminance, pixel value, or the like. The extraction unit 62 selects one target block from the plurality of blocks 504, and sets a target block whose difference between the feature amounts of blocks around the target block and the feature amount of the target block is equal to or less than a predetermined threshold. Extract. The extraction unit 62 extracts a target block whose difference is equal to or less than the threshold as the stationary candidate area 506. The extraction unit 62 may extract a plurality of stationary candidate areas 506.

  The acquisition unit 63 acquires distance information from the imaging device 100 to the stationary candidate area 506. The acquisition unit 63 may acquire, as distance information, a distance image indicating the distance of each pixel constituting the image. The acquiring unit 63 causes the imaging device 100 to image two images including the still candidate area 506, and derives a distance by triangulation based on the parallax of each pixel of the two images, thereby a distance image indicating the distance of each pixel May be acquired as distance information.

  The identifying unit 64 identifies, on the basis of the distance information, a restable area in which the UAV 10 carrying the imaging device 100 can stand without flying. For example, the identifying unit 64 may identify a flat and horizontal area from among the stationary candidate areas 506 as the stationary area 508. The stationary candidate area 506 extracted by the extraction unit 62 includes, for example, an area where the UAV 10 can not land on a slope. Therefore, the specifying unit 64 may specify a flat, horizontal area on which the UAV 10 can land as the stationary area 508, from among the stationary candidate areas 506, based on the distance information. The identifying unit 64 may identify a flat area on which the UAV 10 can land and an area in which the inclination angle is equal to or less than a predetermined angle as the stationary area 508.

  The determination unit 65 determines an imageable area 510 in which the imaging apparatus 100 can capture an image of a composition satisfying a predetermined condition from among the stationary area 508. In a state where the UAV 10 has landed, the determination unit 65 determines, as the image captureable area 510, an area where the imaging device 100 can capture an image of a composition that satisfies a predetermined condition. The determination unit 65 may determine the imageable area 510 based on the range of the angle of view that can be set for the imaging device 100. The determination unit 65 may determine the imageable area 510 based on the range of the imaging direction of the imaging apparatus 100 adjustable by the gimbal 50. The determination unit 65 determines the relationship between the predetermined position of the object, the angle of view set for the imaging device 100, the imaging direction of the imaging device 100 set by the gimbal 50, and the respective positions within the stationary area 508. Based on the above, it may be determined whether or not the imaging device 100 can capture an image of a composition that satisfies a predetermined condition for each position in the stationary area 508. The determination unit 65 may determine each position where the imaging device 100 can capture an image of a composition that satisfies a predetermined condition in the stationary area 508 as the imaging available area 510. The determination unit 65 may determine a plurality of imageable areas 510. When the determination unit 65 determines a plurality of image captureable areas 510, the reception unit 61 may receive a designation of a desired area from among the plurality of image captureable areas 510 from the user. For example, when at least one of the yaw axis, the pitch axis, and the roll axis of the gimbal 50 is rotated, an object unrelated to the object such as the ground or a wall is photographed. In order to avoid such a state, the imaging direction of the adjustable imaging device 100 is determined in advance by the gimbals 50.

  The determination unit 65 includes an image that includes a predetermined object and does not include another object that exists in a range up to a predetermined distance shorter than the distance from the imaging device 100 to the predetermined object. An area where the image can be imaged may be determined as the imageable area 510. The determination unit 65 includes a predetermined object in the angle of view set in the imaging device 100, and exists in a range up to a predetermined distance shorter than the distance from the imaging device 100 to the predetermined object. An area in which the imaging device 100 can capture an image that does not include another object may be determined as the image captureable area 510. The determination unit 65 includes a predetermined object in an angle of view set in the imaging device 100 in the imaging direction set in the imaging device 100, and is shorter than a distance from the imaging device 100 to the predetermined object. A region where the imaging device 100 can capture an image not including another object present in a range up to a predetermined distance may be determined as the image captureable region 510.

  FIG. 4A is a diagram for describing a method of determining an image captureable area. FIG. 4B is a top view of the situation shown in FIG. 4A. The receiving unit 61 receives, for example, designation of a predetermined object 400. The extraction unit 62 satisfies a predetermined condition indicating that the user is flat from the planned stationary region, based on the feature quantities such as the luminance and the pixel value of the image captured by the imaging device 100. The stationary candidate area 506 is extracted. The identifying unit 64 identifies a flat and horizontal area where the UAV 10 can land as the stationary area 508, based on the distance information of the stationary candidate area 506. The determination unit 65 captures an image that includes a predetermined object 400 and does not include any other object within a range up to a predetermined distance shorter than the distance from the imaging device 100 to the predetermined object 400. An area that can be imaged by the device 100 is determined as an imaging available area 510. For example, when the UAV 10 lands at a location indicated by the UAV 10b, another object 450 exists within a range 402b up to a predetermined distance. Therefore, when the UAV 10 lands at a location indicated by the UAV 10 b, the imaging device 100 excludes the image of the composition satisfying the predetermined condition from the candidates for the imaging available area as being unable to capture an image. On the other hand, when the UAV 10 lands at a location indicated by the UAV 10a or UAV 10c, there is no other object including another object 450 within the range 402a or the range 402c up to a predetermined distance. Therefore, the determination unit 65 determines that the imaging apparatus 100 can capture an image of a composition that satisfies a predetermined condition, and determines an area including the locations indicated by the UAVs 10a and 10c as the imaging available area 510.

  The UAV 10 is landed on the imageable area 510 determined by the determination unit 65, and the imaging device 100 captures an image of an object 400 determined in advance. The UAV 10 may land on the imageable area 510 automatically. Alternatively, the user may operate the UAV 10 via the remote control device 300 to land the UAV 10 on the imageable area 510. As a result, it is possible to suppress the consumption of the battery accompanying the flight of the UAV 10 and obtain an image of a composition that satisfies a predetermined condition.

  The receiving unit 61 may receive specification of an image of a composition that satisfies a predetermined condition. The accepting unit 61 may accept designation of an image of a composition that satisfies a predetermined condition from among the images captured by the imaging device 100 during the flight of the UAV 10. The accepting unit 61 may accept specification of an image of a composition that satisfies a predetermined condition from among images stored in advance in the UAV 10, the remote control device 300, or a database on a network.

  The determination unit 65 is an extension of the imaging direction of the imaging device 100 from the position of the UAV 10 when the imaging device 100 captures an image of a composition that satisfies a predetermined condition while the UAV 10 is flying from the stationary area. The existing area may be determined as an imageable area.

  For example, as shown in FIG. 5, the UAV 10 is caused to fly to cause the imaging device 100 to image a predetermined object 400. At this time, the reception unit 61 receives specification of an image captured by the imaging device 100 as an image of a composition that satisfies a predetermined condition. The UAV control unit 30 stores, in the memory 32, information indicating the latitude, longitude, and altitude of the UAV 10 when the image is captured by the imaging device 100. Thereafter, while causing the UAV 10 to fly, the imaging device 100 causes the imaging device 100 to capture an image, and the extraction unit 62 extracts the stillness candidate region 506 from the captured image. Further, the specifying unit 64 specifies a flat and horizontal area out of the stationary candidate areas 506 as the stationary area 508. The determination unit 65 is an extension of the imaging direction of the imaging device 100 from the position of the UAV 10 when the imaging device 100 captures an image of a composition that satisfies a predetermined condition while the UAV 10 is flying from the stationary area 508. The area present on 410 may be determined as an imageable area 510. The determination unit 65 includes a predetermined object 400 in a region existing on the extension line 410 in the imaging direction of the imaging device 100, and is predetermined to be shorter than a distance from the imaging device 100 to the predetermined object 400. A region where the imaging device 100 can capture an image not including another object present in the range up to the distance may be determined as the image captureable region 510. For example, when the UAV 10 lands in the area indicated by the UAV 10b, another object 450 is present in the range 402b from the imaging device 100 to a predetermined distance. Exclude from the candidates. On the other hand, when the UAV 10 lands in the area indicated by the UAV 10a, no other object is present in the range 402a from the imaging device 100 to a predetermined distance, and the determination unit 65 sets the area as the imageable area 510. decide.

  FIG. 6 is a flowchart showing an example of the imaging procedure. The accepting unit 61 accepts the selection of the designated imaging mode which causes the imaging device 100 to image in a state where the UAV 10 stands still without flying (S100). The UAV 10 is made to fly to cause the imaging device 100 to capture an image. Furthermore, the receiving unit 61 receives the selection of the object to be imaged from the image captured by the imaging device 100 (S102). Furthermore, the receiving unit 61 receives the designation of the planned stationary region from the image captured by the imaging device 100 (S104). The extraction unit 62 extracts a stationary candidate region that satisfies a predetermined condition indicating being flat from the stationary scheduled regions based on the feature amount of the image (S106). The acquisition unit 63 acquires distance information of the stationary candidate area (S108).

  The identifying unit 64 identifies, from among the stationary candidate areas, a stationary area where the UAV 10 can stand without flying, based on the distance information (S110). The determination unit 65 determines whether the stationary area identified by the identification unit 64 exists (S112). If the stationary area does not exist, the process ends.

  When the stationary area exists, the determination unit 65 determines an imaging available area in which the imaging device 100 can capture an image of a composition satisfying the predetermined condition from among the stationary available areas (S114). The UAV 10 lands on the imageable area (S116). The UAV control unit 30 controls the imaging device 100 and the gimbal 50 to adjust the imaging direction and the angle of view of the imaging device 100 so as to capture an image of a composition that satisfies a predetermined condition (S118). After the adjustment, the imaging device 100 captures an image (S120).

  As described above, in a state where the UAV 10 has landed, when there is a region where the imaging device 100 can capture an image of a desired composition, the UAV 10 is landed in that region and causes the imaging device 100 to capture an image. As a result, it is possible to suppress the consumption of the battery accompanying the flight of the UAV 10, and it is possible to extend the photographing available time. Since the imaging device 100 can capture an image in a state where the UAV 10 stands still, it is possible to prevent blurring of the image captured by the imaging device 100 due to the vibration accompanying the flight of the UAV 10.

  FIG. 7 is a flowchart showing an example of the imaging procedure. The accepting unit 61 accepts the selection of the designated imaging mode for causing the imaging device 100 to image in a state where the UAV 10 stands still without flying (S200). The UAV 10 is made to fly, and the imaging device 100 captures an image of a composition that satisfies a predetermined condition (S202). The extraction unit 62 specifies the position of the UAV 10 when the imaging device 100 captures an image of a composition that satisfies a predetermined condition, and the imaging direction of the imaging device 100 (S204). The UAV 10 is further fly to cause the imaging device 100 to capture an image including an area where the UAV 10 is likely to land. The receiving unit 61 receives the designation of the scheduled stationary area from among the images captured by the imaging device 100 (S206).

  The extraction unit 62 extracts, from among the stationary candidates, stationary candidate regions that satisfy a predetermined condition indicating a flat surface existing on an extension of the imaging direction identified from the identified position (S208). The acquisition unit 63 determines whether there is a stationary candidate area (S210). If the stationary candidate area does not exist, the process ends.

  If there is a stationary candidate area, the acquisition unit 63 acquires distance information of the stationary candidate area (S212). The identifying unit 64 identifies, from among the stationary candidate areas, a stationary area where the UAV 10 can land, based on the distance information (S214). The determination unit 65 determines whether there is a stationary area (S216). If the stationary area does not exist, the process ends.

  If there is a restable area, the determination unit 65 determines, from among the restable areas, an immagable area capable of capturing an image of a composition that satisfies a predetermined condition (S218). The UAV 10 lands on the imageable area (S220). The UAV control unit 30 controls the imaging device 100 and the gimbal 50 to adjust the imaging direction and the angle of view of the imaging device 100 so as to capture an image of a composition that satisfies a predetermined condition (S222). After the adjustment, the imaging device 100 captures an image (S224).

  As described above, when the imaging device 100 can capture an image of a desired composition captured by the imaging device 100 while the UAV 10 is in flight, even when the UAV 10 is landing, the imaging device 100 can be captured while the UAV 10 is landing. To capture a desired image. As a result, it is possible to suppress the consumption of the battery accompanying the flight of the UAV 10, and it is possible to extend the photographing available time. Since the imaging device 100 can capture an image in a state where the UAV 10 stands still, it is possible to prevent blurring of the image captured by the imaging device 100 due to the vibration accompanying the flight of the UAV 10.

  FIG. 8 shows an example of a computer 1200 in which aspects of the present invention may be fully or partially embodied. A program installed on computer 1200 can cause computer 1200 to act as an operation or one or more "parts" of the device according to embodiments of the invention. Alternatively, the program may cause the computer 1200 to execute the operation or one or more “parts”. The program can cause the computer 1200 to execute the process according to the embodiment of the present invention or the steps of the process. Such programs may be executed by CPU 1212 to cause computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

  The computer 1200 according to the present embodiment includes a CPU 1212 and a RAM 1214, which are interconnected by a host controller 1210. Computer 1200 also includes a communication interface 1222, an input / output unit, which are connected to host controller 1210 via an input / output controller 1220. Computer 1200 also includes a ROM 1230. The CPU 1212 operates in accordance with programs stored in the ROM 1230 and the RAM 1214 to control each unit.

  The communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores therein a boot program or the like executed by the computer 1200 upon activation, and / or a program dependent on the hardware of the computer 1200. The program is provided via a computer readable recording medium or network such as a CR-ROM, a USB memory or an IC card. The program is installed in the RAM 1214 or ROM 1230, which is also an example of a computer-readable recording medium, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200 and brings about coordination between the programs and the various types of hardware resources. An apparatus or method may be configured by implementing the operation or processing of information in accordance with the use of computer 1200.

  For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes the communication program loaded in the RAM 1214, and performs communication processing to the communication interface 1222 based on the processing described in the communication program. You may order. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214 or a USB memory under the control of the CPU 1212 and transmits the read transmission data to the network, or The received data received from the network is written in a reception buffer area or the like provided on the recording medium.

  In addition, the CPU 1212 allows the RAM 1214 to read all or necessary portions of a file or database stored in an external recording medium such as a USB memory, and executes various types of processing on data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to the external storage medium.

  Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium and subjected to information processing. The CPU 1212 describes various types of operations, information processing, condition judgment, conditional branching, unconditional branching, information retrieval, which are described throughout the present disclosure for data read from the RAM 1214 and specified by a program instruction sequence. Various types of processing may be performed, including / replacement etc, and the results written back to RAM 1214. Further, the CPU 1212 may search for information in a file in a recording medium, a database or the like. For example, when a plurality of entries each having the attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. Search for an entry matching the condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby associate the first attribute satisfying the predetermined condition with the first attribute An attribute value of the second attribute may be acquired.

  The programs or software modules described above may be stored on computer readable storage medium on or near computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer readable storage medium, whereby the program can be transmitted to the computer 1200 via the network. provide.

  The execution order of each process such as operations, procedures, steps, and steps in the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly “before”, “preceding” It is to be noted that “it is not explicitly stated as“ etc. ”and can be realized in any order as long as the output of the previous process is not used in the later process. With regard to the operation flow in the claims, the specification, and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It is apparent to those skilled in the art that various changes or modifications can be added to the above embodiment. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

  The execution order of each process such as operations, procedures, steps, and steps in the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly “before”, “preceding” It is to be noted that “it is not explicitly stated as“ etc. ”and can be realized in any order as long as the output of the previous process is not used in the later process. With regard to the flow of operations in the claims, the specification and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

10 UAV
Reference Signs List 20 UAV main body 30 UAV control unit 32 memory 36 communication interface 40 promotion unit 41 GPS receiver 42 inertia measurement device 43 magnetic compass 44 barometric pressure altimeter 45 temperature sensor 46 humidity sensor 50 gimbal 60 imaging device 61 reception unit 62 extraction unit 63 acquisition unit 64 Identification unit 65 Determination unit 70 Imaging instruction unit 100 Imaging device 102 Imaging unit 110 Imaging control unit 120 Image sensor 130 Memory 200 Lens unit 210 Lens 212 Lens driver 214 Position sensor 220 Lens control unit 222 Memory 300 Remote control 400 Object 410 Extension Line 450 Other object 500 Image 502 Stationary area 506 Stationary candidate area 508 Stationary area 510 Imageable area 1200 Computer 1210 Host controller 1212 CPU
1214 RAM
1220 I / O controller 1222 communication interface 1230 ROM

Claims (17)

An identifying unit that identifies a stationary area where the aircraft carrying the imaging device can stand without flying;
A determination apparatus comprising: a determination unit configured to determine an image captureable area capable of capturing an image of a composition satisfying a predetermined condition from among the stationary areas.   The determination unit may capture an image including a predetermined object and not including another object existing in a range up to a predetermined distance shorter than a distance from the imaging device to the predetermined object. The determination apparatus according to claim 1, wherein an area that can be imaged by the apparatus is determined as the imageable area.   The determination apparatus according to claim 2, further comprising a first reception unit that receives specification of the predetermined object.   The determining apparatus according to claim 3, wherein the first receiving unit receives specification of the predetermined object from among the images captured by the imaging device. A second reception unit that receives specification of a stationary scheduled area to be stationary without causing the aircraft to fly;
An extraction unit for extracting a still candidate area satisfying a predetermined condition indicating flat from the predetermined stationary area based on the image captured by the imaging device;
An acquisition unit configured to acquire distance information indicating a distance from the imaging device to the stationary candidate area;
The determination apparatus according to claim 1, wherein the identification unit identifies the restable area from among the rest candidate areas based on the distance information.   The determining apparatus according to claim 5, wherein the second receiving unit receives the specification of the scheduled stationary area based on an image captured by the imaging device while the flying object is flying.   The determination apparatus according to claim 1, further comprising a third reception unit that receives specification of an image of a composition that satisfies the predetermined condition.   The determination apparatus according to claim 7, wherein the third reception unit receives specification of an image of a composition that satisfies the predetermined condition from among images captured by the imaging device while the flight body is flying. .   The determination unit is configured based on a position of the flying object when the imaging device captures an image of a composition that satisfies the predetermined condition while the flying object is flying from the restable area. The determination apparatus according to claim 8, wherein an area existing on an extension of an imaging direction is determined as the imaging available area.   The determination device according to claim 1, wherein the determination unit determines the image captureable area based on a range of angle of view that can be set for the imaging device. The flying object further includes a support mechanism that supports the imaging device such that the imaging direction of the imaging device can be adjusted.
The determination device according to claim 1, wherein the determination unit determines the imageable area based on a range of an imaging direction of the imaging device that can be adjusted by the support mechanism. A determining device according to any one of the preceding claims.
The flying object is controlled to be stopped without flying the flying object to the image captureable area, and the imaging device is controlled to cause the imaging device to capture an image of a composition satisfying the predetermined condition. A control device comprising a control unit. A controller according to claim 12;
An imaging system comprising the imaging device.   An aircraft flying with the imaging system according to claim 13.   The flying object according to claim 14, further comprising a support mechanism that supports the imaging device so that the attitude of the imaging device can be adjusted. Identifying a stationary area in which the aircraft carrying the imaging device can stand without flying;
Determining an image captureable area capable of capturing an image of a composition satisfying a predetermined condition from among the stationary areas. Identifying a stationary area in which the aircraft carrying the imaging device can stand without flying;
A program for causing a computer to execute a step of determining an image captureable area capable of capturing an image of a composition satisfying a predetermined condition from among the stationary areas.

Images