JP6543859B1 - IMAGE PROCESSING DEVICE, IMAGING DEVICE, MOBILE OBJECT, IMAGE PROCESSING METHOD, AND PROGRAM - Google Patents

Abstract

An image processing apparatus that produces an effect according to movement of a moving body on an image captured by an imaging apparatus mounted on the moving body. In an unmanned aerial vehicle UAV, an image processing device acquires an image captured by an imaging device mounted on a moving body, and indicates a moving direction of the moving body when the imaging device is capturing an image. A blur processing unit is provided that adds blur to an image based on a specific unit to be specified and a moving direction. The specifying unit specifies a moving direction of the moving body based on control information for controlling the moving body. [Selected figure] Figure 2

Description

  The present invention relates to an image processing apparatus, an imaging apparatus, a moving object, an image processing method, and a program.

It is disclosed that the character is subjected to a filtering process to reduce blurring according to the moving speed of the character.
Patent Document 1: JP-A-2010-145510

  It is desired to produce an effect corresponding to the movement of a moving object in an image captured by an imaging device mounted on the moving object.

  The image processing apparatus according to an aspect of the present invention may include an acquisition unit that acquires an image captured by an imaging device mounted on a mobile body. The image processing apparatus may include a specifying unit that specifies a moving direction of the moving body when the imaging apparatus captures an image. The image processing apparatus may include a tweak processing unit that blurs the image based on the movement direction.

  The identifying unit may identify the moving direction of the mobile based on control information for controlling the mobile.

  The identifying unit may identify the moving direction along the imaging surface of the imaging device. The forgery processing unit may perform forgery processing to add blur along the movement direction to the image.

  The forgery processing unit may perform a forgery processing for imparting a blur extending in the direction opposite to the movement direction to the image.

  The identifying unit may further identify the moving speed of the moving object when the imaging device captures an image. The forgery processing unit may give the image a blur of the intensity according to the moving speed.

  The forgery processing unit may apply a filter of a size according to the moving speed to blur the image.

  When the moving speed of the moving object is the first moving speed, the tweak processing unit may apply a first size filter to blur the image. When the moving speed of the moving object is the second moving speed higher than the first moving speed, the tweak processing unit may apply a second size filter larger than the first size to blur the image.

  When the moving speed of the moving object is smaller than a predetermined threshold, the forgery processing unit may apply a first size filter to blur the image. When the moving speed of the moving object is equal to or higher than the threshold, the forgery processing unit may apply a second size filter larger than the first size to blur the image.

  The identifying unit may identify a shutter speed of the imaging device when the imaging device captures an image. The forgery processing unit may apply a filter of a size corresponding to the shutter speed to blur the image.

  When the shutter speed of the imaging device is the first shutter speed, the tweak processing unit may apply a filter of the first size to blur the image. When the shutter speed of the imaging device is a second shutter speed that is larger than the first shutter speed, the tweak processing unit may apply a second size filter that is larger than the first size to blur the image.

  When the shutter speed of the imaging device is smaller than a predetermined third shutter speed smaller than the first shutter speed, the blurring processing unit may not blur the image.

  The identifying unit may identify a moving speed of the moving body when the imaging device captures an image, and a shutter speed of the imaging device. The forgery processing unit may give a blur of the intensity according to the moving speed and the shutter speed.

  The imaging device may be mounted on the movable body via a support mechanism that supports the imaging device such that the imaging direction of the imaging device can be adjusted. The identifying unit may identify an imaging direction of the imaging device with respect to the moving body when the imaging device captures an image. The identifying unit may identify the moving direction along the imaging surface based on the moving direction of the moving body and the imaging direction of the imaging device with respect to the moving body.

  The identifying unit may identify a moving speed of the moving body when the imaging device captures an image, and a rotation speed of the imaging device with respect to the moving body. The forgery processing unit may give the image a blur of the intensity according to the moving speed of the moving body and the rotational speed of the imaging device.

  An imaging device according to an aspect of the present invention may include an image sensor. The imaging device may include the above-described image processing device that performs image processing on an image output from an image sensor.

  The mobile unit according to one aspect of the present invention may be a mobile unit that moves with the imaging device.

  The moving body may include a support mechanism that supports the imaging device such that the imaging direction of the imaging device can be adjusted.

  The image processing method according to an aspect of the present invention may include the step of acquiring an image captured by an imaging device mounted on a mobile object. The image processing method may include the step of specifying the moving direction of the moving body when the imaging device captures an image. The image processing method may comprise blurring the image based on the direction of movement.

  The program according to an aspect of the present invention may be a program for causing a computer to function as the image processing apparatus.

  According to one aspect of the present invention, it is possible to produce an effect according to the movement of a moving object on an image captured by an imaging device mounted on the moving object.

  Note that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a subcombination of these feature groups can also be an invention.

It is a figure showing an example of the appearance of a UAV and a remote control. It is a figure which shows an example of the functional block of UAV. It is a figure which shows an example of the filter according to the moving direction and moving speed of UAV. It is a figure which shows an example of the filter according to the moving direction and moving speed of UAV. It is a figure which shows an example of the filter according to the moving direction and moving speed of UAV. It is a flowchart which shows an example of the procedure of a tweet process. It is a figure which shows an example of the relationship between the moving speed of UAV, and the size of a filter. It is a figure which shows an example of the relationship between the moving speed of UAV, and the size of a filter. It is a figure which shows an example of the relationship between shutter speed and the size of a filter. It is a figure which shows an example of the image before blur processing, and the image after blur processing. It is a figure for demonstrating an example of a hardware configuration.

  Hereinafter, although this invention is demonstrated through embodiment of invention, the following embodiment does not limit the invention which concerns on a claim. Further, not all combinations of features described in the embodiments are essential to the solution of the invention. It will be apparent to those skilled in the art that various changes or modifications can be added to the following embodiments. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

  The claims, the description, the drawings, and the abstract contain matters that are subject to copyright protection. The copyright holder will not object to any copy of these documents as they appear in the Patent Office file or record. However, in all other cases, all copyrights are reserved.

  Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, wherein the blocks are responsible for (1) process steps or (2) operations being performed. May represent a "part" of The particular stages and "parts" may be implemented by programmable circuits and / or processors. Dedicated circuitry may include digital and / or analog hardware circuitry. Integrated circuits (ICs) and / or discrete circuits may be included. Programmable circuits may include reconfigurable hardware circuits. Reconfigurable hardware circuits include logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, flip flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. Memory elements, etc. may be included.

  Computer readable media may include any tangible device capable of storing instructions for execution by a suitable device. As a result, a computer readable medium having instructions stored thereon will comprise an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-Ray (RTM) Disc, Memory Stick, Integrated A circuit card or the like may be included.

  Computer readable instructions may include either source code or object code written in any combination of one or more programming languages. Source code or object code includes a conventional procedural programming language. Conventional procedural programming languages include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or like Smalltalk, JAVA, C ++, etc. It may be an object oriented programming language, and a "C" programming language or similar programming language. Computer readable instructions may be local or to a wide area network (WAN) such as a local area network (LAN), the Internet, etc., relative to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing device. May be provided via The processor or programmable circuitry may execute computer readable instructions to create a means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

  FIG. 1 shows an example of the appearance of an unmanned aerial vehicle (UAV) 10 and a remote control device 300. The UAV 10 includes a UAV main body 20, a gimbal 50, a plurality of imaging devices 60, and an imaging device 100. The gimbal 50 and the imaging device 100 are examples of an imaging system. The UAV 10 is a concept including a moving object moving in the air, a vehicle moving on the ground, a ship moving on the water, and the like. A flying object moving in the air is a concept including UAV, other aircraft moving in the air, an airship, a helicopter, and the like.

  The UAV body 20 comprises a plurality of rotors. The plurality of rotors are an example of the propulsion unit. The UAV body 20 causes the UAV 10 to fly by controlling the rotation of a plurality of rotors. The UAV body 20 causes the UAV 10 to fly using, for example, four rotors. The number of rotors is not limited to four. The UAV 10 may also be a fixed wing aircraft that does not have a rotor.

  The imaging device 100 is a camera for imaging which captures an object included in a desired imaging range. The gimbal 50 rotatably supports the imaging device 100. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 rotatably supports the imaging device 100 on the pitch axis using an actuator. The gimbal 50 rotatably supports the imaging device 100 about each of the roll axis and the yaw axis using an actuator. The gimbal 50 may change the attitude of the imaging device 100 by rotating the imaging device 100 about at least one of the yaw axis, the pitch axis, and the roll axis.

  The plurality of imaging devices 60 are cameras for sensing that capture the periphery of the UAV 10 to control the flight of the UAV 10. Two imaging devices 60 may be provided at the front, which is the nose of the UAV 10. Two further imaging devices 60 may be provided on the bottom of the UAV 10. The two front imaging devices 60 may be paired to function as a so-called stereo camera. The two imaging devices 60 on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the UAV 10 may be generated based on the images captured by the plurality of imaging devices 60. The number of imaging devices 60 provided in the UAV 10 is not limited to four. The UAV 10 may include at least one imaging device 60. The UAV 10 may include at least one imaging device 60 on each of the nose, aft, sides, bottom, and ceiling of the UAV 10. The angle of view that can be set by the imaging device 60 may be wider than the angle of view that can be set by the imaging device 100. The imaging device 60 may have a single focus lens or a fisheye lens.

  The remote control device 300 communicates with the UAV 10 to remotely control the UAV 10. The remote control device 300 may communicate with the UAV 10 wirelessly. The remote control device 300 transmits instruction information indicating various commands related to the movement of the UAV 10 such as rising, falling, acceleration, deceleration, forward, reverse, and rotation to the UAV 10. The instruction information includes, for example, instruction information for raising the altitude of the UAV 10. The instruction information may indicate the altitude at which the UAV 10 should be located. The UAV 10 moves so as to be located at the altitude indicated by the instruction information received from the remote control device 300. The instruction information may include a lift instruction to raise the UAV 10. The UAV 10 goes up while accepting the up command. Even if the UAV 10 receives an ascent instruction, if the UAV 10's altitude reaches the upper limit altitude, the UAV 10 may limit the ascent.

  FIG. 2 shows an example of a functional block of the UAV 10. The UAV 10 includes a UAV control unit 30, a memory 37, a communication interface 36, a propulsion unit 40, a GPS receiver 41, an inertial measurement device 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60 and the imaging device 100.

  The communication interface 36 communicates with other devices such as the remote control device 300. The communication interface 36 may receive instruction information including various instructions for the UAV control unit 30 from the remote control device 300. The memory 37 includes a propulsion unit 40, a GPS receiver 41, an inertial measurement unit (IMU) 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an imaging device 60, and a memory 37. And the program etc. required to control the imaging device 100 are stored. The memory 37 may be a computer readable recording medium, and may include at least one of a SRAM, a DRAM, an EPROM, an EEPROM, and a flash memory such as a USB memory. The memory 37 may be provided inside the UAV main body 20. It may be provided to be removable from the UAV main body 20.

  The UAV control unit 30 controls the flight and imaging of the UAV 10 in accordance with the program stored in the memory 37. The UAV control unit 30 is an example of a second control unit. The UAV control unit 30 may be configured by a microprocessor such as a CPU or an MPU or a microcontroller such as an MCU. The UAV control unit 30 controls the flight and imaging of the UAV 10 according to the command received from the remote control device 300 via the communication interface 36. The promotion unit 40 promotes the UAV 10. The propulsion unit 40 has a plurality of rotors and a plurality of drive motors for rotating the plurality of rotors. The propulsion unit 40 rotates the plurality of rotary blades via the plurality of drive motors in accordance with the instruction from the UAV control unit 30 to fly the UAV 10.

  The GPS receiver 41 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 41 calculates the position (latitude and longitude) of the GPS receiver 41, that is, the position (latitude and longitude) of the UAV 10, based on the plurality of received signals. The IMU 42 detects the attitude of the UAV 10. The IMU 42 detects, as the posture of the UAV 10, accelerations in the front, rear, left, right, and top three axial directions of the UAV 10, and angular velocities in three axial directions of pitch, roll, and yaw. The magnetic compass 43 detects the heading of the UAV 10. The barometric altimeter 44 detects the altitude at which the UAV 10 flies. The barometric pressure altimeter 44 detects the barometric pressure around the UAV 10, converts the detected barometric pressure into a height, and detects the height. The temperature sensor 45 detects the temperature around the UAV 10. The humidity sensor 46 detects the humidity around the UAV 10.

  The imaging device 100 includes an imaging unit 102 and a lens unit 200. The lens unit 200 is an example of a lens device. The imaging unit 102 includes an image sensor 120, an imaging control unit 110, and a memory 130. The image sensor 120 may be configured by a CCD or a CMOS. The image sensor 120 captures an optical image formed through the plurality of lenses 210, and outputs the captured image to the imaging control unit 110. The imaging control unit 110 may be configured by a microprocessor such as a CPU or an MPU, a microcontroller such as an MCU, or the like. The imaging control unit 110 may control the imaging device 100 according to an operation command of the imaging device 100 from the UAV control unit 30. The imaging control unit 110 is an example of a first control unit. The memory 130 may be a computer readable recording medium, and may include at least one of SRAM, DRAM, EPROM, EEPROM, and flash memory such as USB memory. The memory 130 stores programs and the like necessary for the imaging control unit 110 to control the image sensor 120 and the like. The memory 130 may be provided inside the housing of the imaging device 100. The memory 130 may be provided to be removable from the housing of the imaging device 100.

  The lens unit 200 includes a plurality of lenses 210, a plurality of lens drivers 212, and a lens controller 220. The plurality of lenses 210 may function as a zoom lens, a varifocal lens, and a focus lens. At least a part or all of the plurality of lenses 210 is disposed movably along the optical axis. The lens unit 200 may be an interchangeable lens provided detachably to the imaging unit 102. The lens driving unit 212 moves at least a part or all of the plurality of lenses 210 along the optical axis via a mechanical member such as a cam ring. The lens driver 212 may include an actuator. The actuator may include a stepping motor. The lens control unit 220 drives the lens driving unit 212 according to the lens control command from the imaging unit 102 to move one or more lenses 210 along the optical axis direction via the mechanical member. The lens control command is, for example, a zoom control command and a focus control command.

  The lens unit 200 further includes a memory 222 and a position sensor 214. The lens control unit 220 controls movement of the lens 210 in the optical axis direction via the lens driving unit 212 in accordance with the lens operation command from the imaging unit 102. The lens control unit 220 controls movement of the lens 210 in the optical axis direction via the lens driving unit 212 in accordance with the lens operation command from the imaging unit 102. Some or all of the lenses 210 move along the optical axis. The lens control unit 220 performs at least one of the zoom operation and the focusing operation by moving at least one of the lenses 210 along the optical axis. The position sensor 214 detects the position of the lens 210. Position sensor 214 may detect the current zoom position or focus position.

  The lens drive unit 212 may include a shake correction mechanism. The lens control unit 220 may execute shake correction by moving the lens 210 in the direction along the optical axis or in the direction perpendicular to the optical axis via the shake correction mechanism. The lens drive unit 212 may drive a shake correction mechanism by a stepping motor to execute shake correction. The shake correction mechanism may be driven by a stepping motor to move the image sensor 120 in the direction along the direction of the optical axis or in the direction perpendicular to the optical axis to perform the shake correction.

  The memory 222 stores control values of the plurality of lenses 210 moving through the lens driving unit 212. The memory 222 may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.

  In the UAV 10 configured as described above, an effect according to the movement of the UAV 10 is produced on the image captured by the imaging device 100. For example, when the UAV 10 captures a moving image by the imaging device 100 outdoors, the shutter speed may be high, the motion of the subject may not be smooth between frames, and the motion of the subject may be unnatural. According to the imaging device 100 according to the present embodiment, such unnaturalness of the image is eliminated by applying blur to the image according to the movement of the UAV 10.

  The imaging control unit 110 includes an acquisition unit 112, a specification unit 114, and a decoy processing unit 116. The imaging control unit 110 is an example of an image processing apparatus. The acquisition unit 112, the identification unit 114, and the forgery processing unit 116 may be included in an apparatus other than the imaging apparatus 100, such as a personal computer. The acquisition unit 112 acquires an image captured by the imaging device 100 via the image sensor 120. The acquisition unit 112 may sequentially acquire each image constituting a moving image captured by the imaging device 100. The identifying unit 114 identifies the moving direction of the UAV 10 when the imaging device 100 captures an image. The identifying unit 114 may identify the moving direction along the imaging surface of the image sensor 120. The identifying unit 114 may identify a component parallel to the imaging surface of the image sensor 120 in the movement direction of the UAV 10. The identifying unit 114 may identify the moving direction of the UAV 10 based on control information for controlling the UAV 10 acquired via the UAV control unit 30. The identifying unit 114 identifies the moving direction of the UAV 10 based on the moving direction information of the UAV 10 from the GPS receiver 41 acquired via the UAV control unit 30, the attitude information of the UAV 10 from the inertial measurement device 42, and the like. You may

  The forgery processing unit 116 blurs the image based on the moving direction of the UAV 10. The forgery processing unit 116 may perform forgery processing for adding blur to the image along the moving direction of the UAV 10. The forgery processing unit 116 may perform forgery processing for applying a blur extending in the direction opposite to the moving direction of the UAV 10 to the image. The forgery processing unit 116 may perform the forgery processing by applying a filter that applies the blur extending in the direction opposite to the moving direction of the UAV 10 to the image. When the moving direction of the UAV 10 and the imaging direction of the imaging device 100 are the same, the blurring processing unit 116 applies the blurring process to the image by applying blur extending radially from the center of the image to perform the blurring process. You may The forgery processing unit 116 may apply, to the image, a motion blur filter that performs processing for turning the image in a specific direction. The forgery processing unit 116 may apply, to the image, a Gaussian filter that performs processing for perturbing the image in a specific direction.

  The identifying unit 114 may further identify the moving speed of the UAV 10 when the imaging device 100 captures an image. The identifying unit 114 may identify the moving speed of the UAV 10 based on the control information for controlling the UAV 10 acquired via the UAV control unit 30. The identifying unit 114 may identify the moving speed of the UAV 10 based on the control information of the propulsion unit 40. The identifying unit 114 may identify the moving speed of the UAV 10 based on the moving speed of the UAV 10 from the GPS receiver 41 acquired via the UAV control unit 30. The forgery processing unit 116 may give the image a blur of the intensity according to the moving speed of the UAV 10. The forgery processing unit 116 may apply a filter of a size according to the moving speed of the UAV 10 to blur the image. When the moving speed of the UAV 10 is the first moving speed, the tweak processing unit 116 may apply a filter of the first size to blur the image.

  When the moving speed of the UAV 10 is a second moving speed that is higher than the first moving speed, the tweak processing unit 116 may apply a filter of a second size larger than the first size to blur the image.

  When the moving speed of the UAV 10 is smaller than a predetermined threshold value, the forgery processing unit 116 may apply a first size filter to blur the image. When the moving speed of the UAV 10 is equal to or higher than the threshold, the forgery processing unit 116 may apply a filter of a second size larger than the first size to blur the image.

  Instead of changing the filter size according to the moving speed of the UAV 10, the forgery processing unit 116 may apply blurring to the image by applying a weighting filter according to the moving speed of the UAV 10. The blurring processing unit 116 may apply blur to the image by applying a filter that is weighted such that the blur intensity increases as the moving speed of the UAV 10 increases. The forgery processing unit 116 may apply blur to the image by applying a filter of a size and weighting according to the moving speed of the UAV 10.

  The imaging device 100 is mounted on the UAV 10 via the gimbal 50. By operating the gimbal 50, the imaging direction of the imaging device 100 with respect to the UAV 10 changes. Therefore, the identifying unit 114 may further identify the imaging direction of the imaging device 100 with respect to the UAV 10 when the imaging device 100 is imaging an image. Then, the specifying unit 114 may specify the moving direction along the imaging surface of the image sensor 120 of the UAV 10 based on the moving direction of the UAV 10 and the imaging direction of the imaging device 100 with respect to the UAV 10. The identifying unit 114 may identify the movement direction along the imaging surface of the image sensor 120 of the UAV 10 based on the control information of the gimbal 50 acquired from the UAV control unit 30.

  The identifying unit 114 may further identify the rotational speed of the imaging device 100 with respect to the UAV 10 when the imaging device 100 captures an image. The forgery processing unit 116 may give the image a blur of the intensity according to the moving speed of the UAV 10 and the rotational speed of the imaging device 100.

  The acquisition unit 112 associates the information indicating the moving direction of the UAV 10 when the image is captured and the moving speed of the UAV 10 as a time stamp with the memory 130 as an image sequentially captured through the image sensor 120. It may be stored sequentially. The identifying unit 114 may identify the moving direction of the UAV 10 when the image is captured and the moving speed of the UAV 10 by referring to the timestamp associated with the image.

  As shown in FIG. 3, when the imaging apparatus 100 captures a direction indicated by the arrow 402 (upward in the drawing) while moving the UAV 10 at the first movement speed in the direction indicated by the arrow 400 (rightward in the drawing). The forgery processing unit 116 may apply a 3 × 3 size filter 500 that applies a blur to the image that extends in the direction opposite to the moving direction of the UAV 10.

  As shown in FIG. 4, while the UAV 10 moves at a second movement speed higher than the first movement speed in the direction indicated by the arrow 400 (right direction in the drawing), the direction indicated by the arrow 402 (upward in the drawing) When imaging, the blurring processing unit 116 may apply a 5 × 5 filter 502 that applies blur to the image in the direction opposite to the moving direction of the UAV 10.

  As shown in FIG. 5, while the UAV 10 moves at the second moving speed in the direction indicated by the arrow 410 (the lower right direction on the sheet), the imaging device 100 captures an image in the direction indicated by the symbol 412 (from the front to the back of the sheet) If so, the forgery processing unit 116 may apply a 5 × 5 sized filter 504 that applies a blur to the image that extends in the direction opposite to the moving direction of the UAV 10.

  FIG. 6 is a flowchart showing an example of the procedure of the tanning process. The identifying unit 114 identifies the moving direction and the moving speed of the UAV 10 when the frame of the target of the tumbling process constituting the moving image is captured (S100). The forgery processing unit 116 determines the size of the filter and the weight in the filter based on the moving direction and the moving speed of the UAV 10 (S102). The forgery processing unit 116 selects the size of the filter based on the moving speed of the UAV 10 and is weighted so as to apply a blur extending in the direction opposite to the moving direction of the UAV 10 from the filters of the selected size. You may select a filter. The forgery processing unit 116 applies the determined filter to the target frame to perform forgery processing on the frame (S104). The forgery processing unit 116 outputs the target frame after applying the filter (S106).

  As described above, according to the present embodiment, since the forgery according to the moving direction and the moving speed of the UAV 10 is applied to the image, the effect according to the movement of the UAV 10 can be effectively generated in the image . For example, when the UAV 10 captures a moving image by the imaging device 100 outdoors, it is possible to obtain a moving image in which the subject does not move unnaturally even if the shutter speed is high.

  The forgery processing unit 116 may switch the size of the filter based on, for example, a predetermined threshold value Vth of the moving speed of the UAV 10. For example, as illustrated in FIG. 7, when the moving speed of the UAV 10 is smaller than the threshold value Vth, the concealment processing unit 116 may select a filter of a first size (for example, a 3 × 3 size). When the moving speed of the UAV 10 is equal to or higher than the threshold value Vth, the fork processing unit 116 may select a filter of a second size (for example, a 5 × 5 size).

  The forgery processing unit 116 may switch the size of the filter stepwise in accordance with the moving speed of the UAV 10. For example, as illustrated in FIG. 8, when the moving speed of the UAV 10 is smaller than the first threshold value Vth1, the concealment processing unit 116 may select a filter of a first size (for example, a 3 × 3 size). When the moving speed of the UAV 10 is equal to or greater than the first threshold Vth1 and smaller than the second threshold Vth2, the forgery processor 116 may select a filter of the second size (for example, 5 × 5 size). When the moving speed of the UAV 10 is equal to or greater than the second threshold Vth2 and smaller than the third threshold Vth3, the forgery processor 116 may select a filter of a third size (for example, a size of 7 × 7). When the moving speed of the UAV 10 is equal to or higher than the third threshold Vth3, the forgery processing unit 116 may select a filter of a fourth size (for example, a size of 9 × 9).

  The identifying unit 114 may further identify the shutter speed of the imaging device 100 when the imaging device 100 captures an image. The blurring processing unit 116 may apply a filter of a size corresponding to the shutter speed to blur the image. When the shutter speed of the imaging apparatus 100 is the first shutter speed, the blurring processing unit 116 may apply a first size filter to blur the image. When the shutter speed of the imaging device 100 is a second shutter speed that is larger than the first shutter speed, the blurring processing unit 116 may apply a filter of a second size larger than the first size to blur the image. When the shutter speed of the imaging device 100 is equal to or less than a predetermined third shutter speed smaller than the first shutter speed, the blurring processing unit 116 may not add blur to the image.

  When the shutter speed is smaller than the first threshold Vss1 as shown in FIG. 9, the blurring processor 116 does not have to blur the image. When the shutter speed is equal to or higher than the first threshold Vss1 and smaller than the second threshold Vss2, the forgery processor 116 may select a filter of the first size (for example, 3 × 3 size). When the second processing unit 116 performs processing on the second threshold Vss2 or more and smaller than the third threshold Vss3, the forgery processing unit 116 may select a filter of a second size (for example, 5 × 5 size). If the shutter speed is equal to or higher than the third threshold Vss3, the forgery processing unit 116 may select a filter of a third size (for example, a size of 7 × 7).

  The manipulation processing unit 116 may provide the image with a blur that extends in the direction opposite to the moving direction of the UAV 10, the intensity of which corresponds to the moving speed and the shutter speed of the UAV 10. The blurring processing unit 116 may apply a filter combining a filter of a size according to the moving speed of the UAV 10 and a filter of a size according to the shutter speed to the image to perform blur processing on the image. After applying a filter of a size according to the moving speed of the UAV 10 to the image, the forgery processing unit 116 applies a filter of a size according to the shutter speed to the image to make the image opposite to the moving direction of the UAV 10 You may perform the blurring process extended.

  FIG. 10 is a view showing an example of an image before the blurring process is performed by the blurring unit 116 and an image after the blurring process is performed. The image illustrated in FIG. 10 is an image captured by the imaging device 100 in a state in which the imaging device 100 is directed from the front to the back of the paper surface while the UAV 10 moves in the right direction on the paper surface. As described above, by performing the blurring process extending in the direction opposite to the moving direction of the UAV 10, it is possible to give the image a sense of realism accompanying the movement of the UAV 10.

  The specifying unit 114 may further specify the distance from the imaging device 100 to the subject (subject distance). The forgery processing unit 116 may apply a filter of a size according to the distance to the subject to the image to blur the image. When the subject distance is the first subject distance, the forgery processing unit 116 may apply a filter of the first size to the image. When the subject distance is a second subject distance shorter than the first subject distance, the forgery processing unit 116 may apply a filter of a second size larger than the first size to the image.

  The identifying unit 114 may divide the image into a plurality of areas, and identify the subject distance for each of the plurality of areas. The forgery processing unit 116 may apply a filter of a size according to the subject distance for each of the plurality of areas to each of the plurality of areas to perform the blur processing of the image.

  The identifying unit 114 may identify an object moving relative to the image. The forgery processing unit 116 may apply a filter to the area of the subject in the image to blur the subject in the image.

  FIG. 11 shows an example of a computer 1200 in which aspects of the invention may be fully or partially embodied. A program installed on computer 1200 can cause computer 1200 to act as an operation or one or more "parts" of the device according to embodiments of the invention. Alternatively, the program may cause the computer 1200 to execute the operation or one or more “parts”. The program can cause the computer 1200 to execute the process according to the embodiment of the present invention or the steps of the process. Such programs may be executed by CPU 1212 to cause computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

  The computer 1200 according to the present embodiment includes a CPU 1212 and a RAM 1214, which are interconnected by a host controller 1210. Computer 1200 also includes a communication interface 1222, an input / output unit, which are connected to host controller 1210 via an input / output controller 1220. Computer 1200 also includes a ROM 1230. The CPU 1212 operates in accordance with programs stored in the ROM 1230 and the RAM 1214 to control each unit.

  The communication interface 1222 communicates with other electronic devices via a network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores therein a boot program or the like executed by the computer 1200 upon activation, and / or a program dependent on the hardware of the computer 1200. The program is provided via a computer readable recording medium or network such as a CR-ROM, a USB memory or an IC card. The program is installed in the RAM 1214 or ROM 1230, which is also an example of a computer-readable recording medium, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200 and brings about coordination between the programs and the various types of hardware resources. An apparatus or method may be configured by implementing the operation or processing of information in accordance with the use of computer 1200.

  For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes the communication program loaded in the RAM 1214, and performs communication processing to the communication interface 1222 based on the processing described in the communication program. You may order. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214 or a USB memory under the control of the CPU 1212 and transmits the read transmission data to the network, or The received data received from the network is written in a reception buffer area or the like provided on the recording medium.

  In addition, the CPU 1212 allows the RAM 1214 to read all or necessary portions of a file or database stored in an external recording medium such as a USB memory, and executes various types of processing on data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to the external storage medium.

  Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium and subjected to information processing. The CPU 1212 describes various types of operations, information processing, condition judgment, conditional branching, unconditional branching, information retrieval, which are described throughout the present disclosure for data read from the RAM 1214 and specified by a program instruction sequence. Various types of processing may be performed, including / replacement etc, and the results written back to RAM 1214. Further, the CPU 1212 may search for information in a file in a recording medium, a database or the like. For example, when a plurality of entries each having the attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. Search for an entry matching the condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby associate the first attribute satisfying the predetermined condition with the first attribute An attribute value of the second attribute may be acquired.

  The programs or software modules described above may be stored on computer readable storage medium on or near computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer readable storage medium, whereby the program can be transmitted to the computer 1200 via the network. provide.

  The execution order of each process such as operations, procedures, steps, and steps in the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly “before”, “preceding” It is to be noted that “it is not explicitly stated as“ etc. ”and can be realized in any order as long as the output of the previous process is not used in the later process. With regard to the operation flow in the claims, the specification, and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It is apparent to those skilled in the art that various changes or modifications can be added to the above embodiment. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

10 UAV
Reference Signs List 20 UAV main body 30 UAV control unit 36 communication interface 37 memory 40 propulsion unit 41 receiver 42 inertia measurement device 43 magnetic compass 44 barometric pressure altimeter 45 temperature sensor 46 humidity sensor 50 gimbal 60 imaging device 100 imaging device 102 imaging unit 110 imaging control unit 112 Acquisition unit 114 Identification unit 116 Processing unit 120 Image sensor 130 Memory 200 Lens unit 210 Lens 212 Lens drive unit 214 Position sensor 220 Lens control unit 222 Memory 300 Remote control device 1200 Computer 1210 Host controller 1212 CPU
1214 RAM
1220 I / O controller 1222 communication interface 1230 ROM

Claims (19)

An acquisition unit that acquires an image captured by an imaging device mounted on a mobile object;
An identifying unit that identifies a moving direction of the moving object when the imaging device captures the image;
An image processing apparatus comprising: a tweak processing unit that adds blur to the image based on the movement direction.   The image processing apparatus according to claim 1, wherein the specifying unit specifies a moving direction of the moving body based on control information for controlling the moving body. The specifying unit specifies the moving direction along an imaging surface of the imaging device;
The image processing apparatus according to claim 1, wherein the shaking processing unit performs a shaking process for applying blurring along the movement direction to the image.   The image processing apparatus according to claim 3, wherein the tweak processing unit performs tweak processing for applying a blur extending in a direction opposite to the movement direction to the image. The specifying unit further specifies a moving speed of the moving body when the imaging device is capturing the image;
The image processing apparatus according to claim 1, wherein the tweak processing unit imparts a blur of an intensity corresponding to the movement speed to the image.   The image processing apparatus according to claim 5, wherein the tweak processing unit applies a filter of a size according to the movement speed to add blur to the image.   When the moving speed of the moving body is the first moving speed, the tumbling processing unit applies a filter of the first size to blur the image, and the moving speed of the moving body is the first moving speed. The image processing apparatus according to claim 6, wherein a second size filter larger than the first size is applied to blur the image when the second moving speed is higher.   When the moving speed of the moving body is smaller than a predetermined threshold, the forgery processing unit applies a first size filter to blur the image, and the moving speed of the moving body is equal to or more than the threshold. The image processing apparatus according to claim 6, wherein a second filter larger than the first filter is applied to blur the image. The specifying unit further specifies a shutter speed of the imaging device when the imaging device captures the image;
The image processing apparatus according to claim 6, wherein the tweak processing unit applies a filter having a size according to the shutter speed to blur the image.   When the shutter speed of the imaging device is the first shutter speed, the tumbling processing unit applies a filter of the first size to blur the image, and the shutter speed of the imaging device is the first shutter speed. 10. The image processing apparatus according to claim 9, wherein a second size filter larger than the first size is applied to blur the image when the second shutter speed is larger.   11. The image processing apparatus according to claim 10, wherein the blurring unit does not blur the image when the shutter speed of the imaging device is smaller than a predetermined third shutter speed smaller than the first shutter speed. . The specifying unit further specifies a moving speed of the moving body when the imaging device is imaging the image, and a shutter speed of the imaging device.
The image processing apparatus according to claim 1, wherein the forgery processing unit blurs the intensity according to the moving speed and the shutter speed. The imaging device is mounted on the movable body via a support mechanism that supports the imaging device so that the imaging direction of the imaging device can be adjusted.
The specifying unit further specifies an imaging direction of the imaging device with respect to the movable body when the imaging device is imaging the image, and a moving direction of the movable body and imaging of the imaging device with respect to the movable body The image processing apparatus according to claim 3, wherein the movement direction along the imaging surface is specified based on a direction. The specifying unit further specifies a moving speed of the moving body when the imaging device is capturing the image, and a rotational speed of the imaging device with respect to the moving body.
The image processing apparatus according to claim 13, wherein the tweak processing unit imparts, to the image, a blur of an intensity according to the moving speed of the moving body and the rotational speed of the imaging device. An image sensor,
An image processing apparatus according to any one of claims 1 to 13, which performs image processing on an image output from the image sensor.   A mobile which moves with the imaging device according to claim 15.   The movable body according to claim 16, further comprising a support mechanism that supports the imaging device such that the imaging direction of the imaging device can be adjusted. Acquiring an image captured by an imaging device mounted on a mobile object;
Identifying a moving direction of the moving body when the imaging device captures the image;
Applying blurring to the image based on the moving direction.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 14.