WO2018116417A1 - 制御装置、撮像装置、移動体、制御方法、及びプログラム - Google Patents
制御装置、撮像装置、移動体、制御方法、及びプログラム Download PDFInfo
- Publication number
- WO2018116417A1 WO2018116417A1 PCT/JP2016/088157 JP2016088157W WO2018116417A1 WO 2018116417 A1 WO2018116417 A1 WO 2018116417A1 JP 2016088157 W JP2016088157 W JP 2016088157W WO 2018116417 A1 WO2018116417 A1 WO 2018116417A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- actuator
- optical member
- imaging device
- condition
- holding force
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
- G02B7/04—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification
- G02B7/08—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification adapted to co-operate with a remote control mechanism
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B15/00—Special procedures for taking photographs; Apparatus therefor
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B5/00—Adjustment of optical system relative to image or object surface other than for focusing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
Definitions
- the present invention relates to a control device, an imaging device, a moving body, a control method, and a program.
- Patent Document 1 describes that when vibration is detected, electric power is supplied to the aperture motor to prevent the aperture opening from changing.
- Patent Document 1 Japanese Patent Application Laid-Open No. 2013-156358
- the control device may include a control unit that controls an actuator that holds an optical member of the imaging device.
- the actuator may hold the optical member with the first holding force when the first power is supplied.
- the actuator may hold the optical member with a second holding force larger than the first holding force when the second power larger than the first power is supplied.
- the control unit When the actuator holds the optical member with the first holding force, the control unit includes a condition relating to an altitude of the imaging apparatus, a condition relating to the acceleration of the imaging apparatus, a condition relating to the posture of the moving body that moves by mounting the imaging apparatus, The actuator is controlled to hold the optical member with the second holding force when a predetermined condition including at least one of a condition related to a moving mode of the moving object and a condition related to a command for controlling movement of the moving object is satisfied. You can do it.
- the predetermined condition may be a condition that the altitude of the imaging device is lower than a threshold value.
- the predetermined condition may be a condition that the acceleration of the imaging device is larger than a threshold value.
- the predetermined condition may be a condition that the acceleration direction of the imaging apparatus and the direction in which the optical member can move are within a predetermined angle range.
- the moving body may be an unmanned aerial vehicle.
- the predetermined condition may be a condition that the flight attitude of the unmanned aircraft is a predetermined flight attitude.
- the unmanned aerial vehicle may be capable of flying in a first flight mode and a second flight mode in which an acceleration greater than an acceleration that occurs in an unmanned aircraft flying in the first flight mode occurs.
- the predetermined condition may be a condition that the unmanned aircraft flies in the second flight mode.
- the predetermined condition may be a condition that the instruction is a predetermined instruction that causes the imaging apparatus to generate an acceleration greater than a threshold value.
- the predetermined instruction may include an instruction to land an unmanned aerial vehicle.
- the optical member may include at least one of a diaphragm, a shutter, a filter, and a lens of the imaging device.
- the actuator may be an electromagnetic actuator.
- the actuator may be a stepping motor.
- the optical member may be a diaphragm of the imaging device.
- the actuator may be a voice coil motor.
- the optical member may be a lens of an imaging device.
- An imaging device may include the control device.
- the imaging device may include an optical member.
- the imaging device may include an actuator.
- the moving body according to one embodiment of the present invention may move by mounting the imaging device.
- the control method according to one aspect of the present invention may be a control method for controlling an actuator that holds an optical member of an imaging apparatus.
- the actuator may hold the optical member with the first holding force when the first power is supplied.
- the actuator may hold the optical member with a second holding force larger than the first holding force when the second power larger than the first power is supplied.
- the control method includes a condition relating to the altitude of the imaging device, a condition relating to the acceleration of the imaging device, a condition relating to the posture of the moving body that moves by mounting the imaging device,
- the actuator is controlled to hold the optical member with the second holding force when a predetermined condition including at least one of a condition related to a moving mode of the moving object and a condition related to a command for controlling movement of the moving object is satisfied.
- a predetermined condition including at least one of a condition related to a moving mode of the moving object and a condition related to a command for controlling movement of the moving object is satisfied.
- the program according to an aspect of the present invention may be a program for causing a computer to function as a control unit that controls an actuator that holds an optical member of an imaging apparatus.
- the actuator may hold the optical member with the first holding force when the first power is supplied.
- the actuator may hold the optical member with a second holding force larger than the first holding force when the second power larger than the first power is supplied.
- the control unit When the actuator holds the optical member with the first holding force, the control unit includes a condition relating to an altitude of the imaging apparatus, a condition relating to the acceleration of the imaging apparatus, a condition relating to the posture of the moving body that moves by mounting the imaging apparatus, The actuator is controlled to hold the optical member with the second holding force when a predetermined condition including at least one of a condition related to a moving mode of the moving object and a condition related to a command for controlling movement of the moving object is satisfied. You can do it.
- the optical member held by the actuator can be prevented from being displaced due to the vibration of the imaging device.
- FIG. 1 Various embodiments of the present invention may be described with reference to flowcharts and block diagrams.
- the blocks in the flowcharts and block diagrams may represent (1) the stage of the process in which the operation is performed or (2) the “part” of the device responsible for performing the operation.
- Certain stages and “parts” are provided with dedicated circuitry, programmable circuitry supplied with computer readable instructions stored on a computer readable storage medium, and / or computer readable instructions stored on a computer readable storage medium. It may be implemented by a processor.
- Dedicated circuitry may include digital and / or analog hardware circuitry. Integrated circuits (ICs) and / or discrete circuits may be included.
- Programmable circuits may be logical products, logical sums, exclusive logical sums, negative logical products, negative logical sums, and other logical operations, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLA), for example. , Flip-flops, registers, and memory elements, including reconfigurable hardware circuitry.
- FPGAs field programmable gate arrays
- PLA programmable logic arrays
- a computer-readable storage medium may include any tangible device capable of storing instructions to be executed by a suitable device.
- a computer readable storage medium having instructions stored thereon comprises a product that includes instructions that can be executed to create a means for performing the operations specified in the flowcharts or block diagrams.
- Examples of computer readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory).
- EEPROM Electrically erasable programmable read only memory
- SRAM static random access memory
- CD-ROM compact disc read only memory
- DVD digital versatile disc
- Blu-ray registered trademark
- the computer readable instructions may include either source code or object code written in any combination of one or more programming languages.
- the source code or object code includes a conventional procedural programming language.
- Conventional procedural programming languages include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or Smalltalk, JAVA, C ++, etc. It may be an object-oriented programming language and a “C” programming language or a similar programming language.
- Computer readable instructions may be directed to a general purpose computer, special purpose computer, or other programmable data processing device processor or programmable circuit locally or in a wide area network (WAN) such as a local area network (LAN), the Internet, etc. ).
- the processor or programmable circuit may execute computer readable instructions to create a means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.
- FIG. 1 shows an example of the appearance of an unmanned aerial vehicle (UAV) 100.
- the UAV 100 includes a UAV main body 102, a gimbal 200, an imaging device 300, and a plurality of imaging devices 230.
- the UAV 100 is an example of a moving object.
- the moving body is a concept including, in addition to UAV, other aircraft that moves in the air, vehicles that move on the ground, ships that move on the water, and the like.
- the UAV main body 102 includes a plurality of rotor blades.
- the UAV main body 102 flies the UAV 100 by controlling the rotation of a plurality of rotor blades.
- the UAV main body 102 causes the UAV 100 to fly using four rotary wings.
- the number of rotor blades is not limited to four.
- the UAV 100 may be a fixed wing aircraft that does not have a rotating wing.
- the imaging device 300 is a camera for capturing a moving image or a still image.
- the plurality of imaging devices 230 are sensing cameras that image the surroundings of the UAV 100 in order to control the flight of the UAV 100.
- Two imaging devices 230 may be provided on the front surface that is the nose of the UAV 100.
- Two other imaging devices 230 may be provided on the bottom surface of the UAV 100.
- the two imaging devices 230 on the front side may be paired and function as a so-called stereo camera.
- the two imaging devices 230 on the bottom side may also be paired and function as a stereo camera.
- the distance from the UAV 100 to the object may be measured based on images captured by the plurality of imaging devices 230.
- Three-dimensional spatial data around the UAV 100 may be generated based on images captured by the plurality of imaging devices 230.
- the number of imaging devices 230 included in the UAV 100 is not limited to four.
- the UAV 100 only needs to include at least one imaging device 230.
- the UAV 100 may include at least one imaging device 230 on each of the nose, the tail, the side surface, the bottom surface, and the ceiling surface of the UAV 100.
- the angle of view that can be set by the imaging device 230 may be wider than the angle of view that can be set by the imaging device 300.
- the imaging device 230 may have a single focus lens or a fisheye lens.
- FIG. 2 shows an example of functional blocks of the UAV100.
- the UAV 100 includes a UAV control unit 110, a communication interface 150, a memory 160, a gimbal 200, a rotating blade mechanism 210, an imaging device 300, an imaging device 230, a GPS receiver 240, an inertial measurement device (IMU) 250, a magnetic compass 260, and an atmospheric pressure.
- An altimeter 270 is provided.
- the communication interface 150 communicates with an external transmitter.
- the communication interface 150 receives various commands for the UAV control unit 110 from a remote transmitter.
- the memory 160 stores programs necessary for the UAV control unit 110 to control the gimbal 200, the rotary blade mechanism 210, the imaging device 300, the imaging device 230, the GPS receiver 240, the IMU 250, the magnetic compass 260, and the barometric altimeter 270.
- the memory 160 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.
- the memory 160 may be provided inside the UAV main body 102.
- the memory 160 may be provided so as to be removable from the UAV main body 102.
- the gimbal 200 supports the imaging direction of the imaging device 300 so that it can be adjusted.
- the gimbal 200 supports the imaging device 300 rotatably around at least one axis.
- the gimbal 200 is an example of a support mechanism.
- the gimbal 200 may support the imaging device 300 rotatably about the yaw axis, the pitch axis, and the roll axis.
- the gimbal 200 may change the imaging direction of the imaging device 300 by rotating the imaging device 300 about at least one of the yaw axis, the pitch axis, and the roll axis.
- the rotary blade mechanism 210 includes a plurality of rotary blades and a plurality of drive motors that rotate the plurality of rotary blades.
- the imaging device 230 captures the surroundings of the UAV 100 and generates image data. Image data of the imaging device 230 is stored in the memory 160.
- the GPS receiver 240 receives a plurality of signals indicating times transmitted from a plurality of GPS satellites. The GPS receiver 240 calculates the position of the GPS receiver 240, that is, the position of the UAV 100, based on the received signals.
- the inertial measurement device (IMU) 250 detects the posture of the UAV 100. As the posture of the UAV 100, the IMU 250 detects the acceleration in the three axial directions of the UAV 100 in the front, rear, left, and right directions, and the angular velocity in the three axial directions of pitch, roll, and yaw.
- the magnetic compass 260 detects the heading of the UAV 100.
- the barometric altimeter 270 detects the altitude at which the UAV 100 flies.
- the UAV control unit 110 controls the flight of the UAV 100 in accordance with a program stored in the memory 160.
- the UAV control unit 110 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like.
- the UAV control unit 110 controls the flight of the UAV 100 according to a command received from a remote transmitter via the communication interface 150.
- the UAV control unit 110 may control the flight of the UAV 100 according to a flight mode selected from among a plurality of flight modes.
- the plurality of flight modes may include a normal mode and a sports mode.
- the normal mode is an example of a first flight mode.
- the sport mode is an example of a second flight mode in which acceleration greater than the acceleration generated in the UAV 100 flying in the first flight mode occurs.
- the speed limit of the UAV 100 flying in the sport mode may be larger than the speed limit of the UAV 100 flying in the normal mode.
- the UAV control unit 110 may specify the environment around the UAV 100 by analyzing a plurality of images captured by the plurality of imaging devices 230.
- the UAV control unit 110 controls the flight while avoiding obstacles based on the environment around the UAV 100, for example.
- the UAV control unit 110 may generate three-dimensional spatial data around the UAV 100 based on a plurality of images captured by the plurality of imaging devices 230, and control the flight based on the three-dimensional spatial data.
- the imaging apparatus 300 includes an imaging unit 301 and a lens unit 401.
- the lens unit 401 may be a lens unit that can be detached from the imaging unit 301.
- the imaging unit 301 includes an imaging control unit 310, an imaging element 330, a shutter driving unit 320, a shutter 322, a memory 340, and an acceleration sensor 324.
- the imaging control unit 310 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like.
- the imaging control unit 310 may control the imaging device 300 in accordance with an operation command for the imaging device 300 from the UAV control unit 110.
- the memory 340 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.
- the memory 340 may be provided inside the housing of the imaging unit 301.
- the memory 340 may be provided so as to be removable from the housing of the imaging unit 301.
- the shutter 322 is controlled by the shutter driving unit 320 to control the light receiving time of the image sensor 330.
- the shutter 322 and the shutter driving unit 320 may be provided in the lens unit 401.
- the shutter 322 may be constituted by at least one movable wing member so as to shield light incident on the image sensor 330.
- the at least one wing member may move along a plane perpendicular to the optical axis and shield light incident on the image sensor 330.
- the shutter 322 may be a focal plane shutter or a lens shutter.
- the shutter drive unit 320 includes an actuator.
- the actuator may be an electromagnetic actuator.
- the electromagnetic actuator may be an electromagnet or a solenoid.
- the shutter driving unit 320 drives the actuator to move the shutter 322 between the shielding position and the non-shielding position.
- the imaging device 330 may be configured by a CCD or a CMOS.
- the image pickup device 330 is held inside the housing of the image pickup apparatus 300 and outputs image data of an optical image formed through the plurality of lenses 432 and the lenses 442 to the image pickup control unit 310.
- the imaging control unit 310 performs a series of image processing such as noise reduction, demosaicing, gamma correction, and edge cooperation on the image data.
- the imaging control unit 310 stores image data after a series of image processing in the memory 340.
- the imaging control unit 310 may output and store the image data in the memory 160 via the UAV control unit 110.
- the acceleration sensor 324 detects the acceleration of the imaging device 300.
- the acceleration sensor 324 may be provided in the lens unit 401.
- the acceleration sensor 324 may be provided in the UAV main body 102.
- the lens unit 401 includes a lens control unit 410, a memory 420, a lens driving unit 430, a lens 432, a position sensor 434, a lens driving unit 440, a lens 442, a position sensor 444, an aperture driving unit 450, an aperture 452, a filter driving unit 460, And a filter 462.
- the lens 432 includes at least one lens.
- the lens 432 may be a zoom lens.
- the lens 442 includes at least one lens.
- the lens 442 may be a focus lens.
- the lens control unit 410 controls the movement of the lens 432 in the optical axis direction via the lens driving unit 430 in accordance with a lens operation command from the imaging unit 301.
- the lens control unit 410 controls the movement of the lens 442 in the optical axis direction via the lens driving unit 440 in accordance with a lens operation command from the imaging unit 301.
- Some or all of the lens 432 and the lens 442 move along the optical axis.
- the lens control unit 410 performs at least one of a zoom operation and a focus operation by moving at least one of the lens 432 and the lens 442 along the optical axis.
- the position sensor 434 detects the position of the lens 432.
- the position sensor 434 may detect the current zoom position.
- the position sensor 444 detects the position of the lens 442.
- the position sensor 444 may detect the current focus position.
- the lens driving unit 430 and the lens driving unit 440 may include an actuator.
- the actuator may be an electromagnetic actuator.
- the electromagnetic actuator may be a stepping motor, a solenoid, or a voice coil motor.
- the lens 432 and the lens 442 may move along the optical axis direction via a lens driving mechanism in response to power from each actuator.
- the diaphragm 452 adjusts the amount of light incident on the image sensor 330.
- the diaphragm 452 may include at least one wing member.
- the aperture driving unit 450 may include an actuator.
- the actuator may be an electromagnetic actuator.
- the electromagnetic actuator may be an electromagnet, a solenoid, or a stepping motor.
- the aperture driving unit 450 may receive an instruction from the lens control unit 410 and drive the actuator to adjust the overlapping degree of the plurality of wing members, thereby adjusting the size of the aperture opening.
- the plurality of wing members may move along a plane perpendicular to the optical axis in response to a force from the actuator.
- the filter 462 reduces the amount of light incident through the lens 432, or cuts light of a specific wavelength.
- the filter 462 may include at least one of an ND filter and an infrared cut filter.
- the filter driver 460 may include an actuator.
- the actuator may be an electromagnetic actuator.
- the electromagnetic actuator may be an electromagnet or a solenoid.
- the filter driving unit 460 receives a command from the lens control unit 410 and drives the actuator to remove or input the first position through which the incident light passes and the specific wavelength component of the input light.
- the filter 462 is moved between the second position where the light to be attenuated attenuates.
- the filter 462 may move along a plane perpendicular to the optical axis.
- the memory 420 stores control values of a plurality of lenses 432 and lenses 442 that move via the lens driving unit 430.
- the memory 420 may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory.
- the position of an optical member such as a diaphragm 452, a filter 462, a shutter 322, a lens 432, or a lens 442 driven by an actuator may be displaced. Therefore, the optical member may be held with the actuator energized so that the position of the optical member does not shift. However, if power is continuously supplied to the actuator while the optical member is not moving, power consumption increases. Less power is preferably supplied to the actuator.
- the vibration of the imaging device 300 is small, the optical member can be held with a sufficient holding force from the actuator even if the actuator is driven with relatively little electric power. However, when the vibration of the imaging device 300 is large, even if the actuator is driven with relatively little power, a sufficient holding force cannot be obtained from the actuator, and the position of the optical member may shift.
- the vibration generated in the imaging device 300 tends to be relatively large.
- the acceleration generated in the UAV 100 during the flight of the UAV 100 is about 2G.
- the acceleration generated when the UAV 100 lands is about 30G.
- the vibration generated in the UAV 100 varies depending on the flight state of the UAV 100.
- the imaging device 300 may continue to capture images while the UAV 100 is landing. After the UAV 100 has landed once, the image pickup apparatus 300 may be started by taking off again while maintaining the position of the optical member. Therefore, it is preferable that the position of the optical member does not shift due to an impact generated when the UAV 100 is landed.
- the power supplied to the actuator is increased. Therefore, the holding force obtained from the actuator is increased, and the positional deviation of the optical member is suppressed.
- the actuator may hold the optical member with the first holding force when the first power is supplied.
- the actuator may hold the optical member with a second holding force larger than the first holding force when the second power larger than the first power is supplied.
- the first power may be supplied to the actuator.
- the second power may be supplied to the actuator.
- a control unit such as the imaging control unit 310 or the lens control unit 410 that controls the actuator may control the power supplied to the actuator.
- a control unit that controls power supplied to the actuator may be provided in a transmitter that remotely controls the UAV 100, the UAV control unit 110, or the like.
- the control unit When the actuator holds the optical member with the first holding force, the control unit includes a condition relating to the altitude of the imaging apparatus 300, a condition relating to the acceleration of the imaging apparatus 300, a condition relating to the posture of the UAV 100, a condition relating to the movement mode of the UAV 100, The actuator may be controlled to hold the optical member with the second holding force when a predetermined condition including at least one of the conditions related to the command for controlling the movement of the UAV 100 is satisfied.
- the control unit may determine that a predetermined condition is satisfied when the altitude of the imaging apparatus 300 is lower than the threshold value.
- the control unit may control the actuator so as to hold the optical member with the second holding force.
- the control unit may determine that a predetermined condition is satisfied when the acceleration of the imaging apparatus 300 is greater than the threshold value. When the acceleration detected by the acceleration sensor 324 is larger than the threshold, the control unit may determine that a predetermined condition is satisfied.
- the control unit may determine that the predetermined condition is satisfied when the acceleration direction of the imaging apparatus 300 and the direction in which the optical member can move are within a predetermined angle range.
- the predetermined angle may be set on the basis of whether or not the optical member has actually caused an acceleration in various directions and the optical member has been displaced beyond the allowable range.
- the predetermined angle may be, for example, 90 degrees, 60 degrees, 30 degrees, 15 degrees, or 10 degrees.
- the diaphragm 452, the shutter 322, or the filter 462 moves along a plane perpendicular to the optical axis. Therefore, when the direction of acceleration generated in the imaging apparatus 300 has a component in a direction perpendicular to the optical axis, the diaphragm 452, the shutter 322, or the filter 462 is easy to move.
- the imaging apparatus 300 is maintained in a posture in which the optical axis is horizontal by the gimbal 200, when the UAV 100 flies up or down, the diaphragm 452, the shutter 322, or the filter 462 easily moves.
- the diaphragm 452, the shutter 322, or the filter 462 is likely to move when the UAV 100 flies horizontally.
- the lens 432 or the lens 442 moves along the optical axis. Therefore, when the direction of acceleration generated in the imaging apparatus 300 has a component in a direction parallel to the optical axis, the lens 432 or the lens 442 is likely to move.
- the imaging apparatus 300 is maintained in a posture in which the optical axis is horizontal by the gimbal 200, the lens 432 or the lens 442 easily moves when the UAV 100 flies horizontally.
- the imaging apparatus 300 is maintained in a downward posture by the gimbal 200, when the UAV 100 flies up or down, the lens 432 or the lens 442 easily moves.
- the direction of acceleration of the imaging apparatus 300 in which the optical member easily moves varies depending on the attitude of the imaging apparatus 300.
- the direction in which the optical member is easily moved may vary depending on the position where it is held.
- the filter 462 rotates around a fixed point to remove a specific wavelength component of the input light from the first position through which the incident light passes or to attenuate the input light. Move to 2 position.
- the direction in which the filter 462 is easy to move differs depending on whether the filter 462 is held at the first position or the second position.
- control unit refers to a table indicating the direction of acceleration of the imaging device 300 that should hold the optical member with the second holding force for each position where the optical member is held and the posture of the imaging device 300. It may be determined whether or not a predetermined condition is satisfied.
- the table may be stored in the memory 420 or the memory 340.
- the memory 420 or the memory 340 may store a table indicating the direction of acceleration of the imaging device 300 in which the filter 462 easily moves for each position where the filter 462 is held and the orientation of the imaging device 300.
- the lens control unit 410 or the imaging control unit 310 refers to the table, and the current acceleration direction of the imaging device 300 is determined with respect to the current position where the filter 462 is held and the current posture of the imaging device 300. If the conditions shown in the table are met, it may be determined that a predetermined condition is satisfied.
- the table may indicate the range of the acceleration direction of the imaging apparatus 300 in which the optical member easily moves for each position where the optical member is held and the range of the attitude of the imaging apparatus 300.
- the control unit may determine that the predetermined condition is satisfied when the flight attitude of the UAV 100 is a predetermined flight attitude.
- the UAV 100 When the UAV 100 is in a horizontal posture, the UAV 100 may land, so the control unit may determine that a predetermined condition is satisfied.
- the control unit When the UAV 100 is in the horizontal posture and the direction of acceleration generated in the imaging apparatus 300 is the vertical direction, the control unit may determine that a predetermined condition for the aperture 452, the shutter 322, or the filter 462 is satisfied. In this case, the control unit may supply the second electric power to the diaphragm 452, the shutter 322, or the actuator of the filter 462.
- the control unit may determine that a predetermined condition is satisfied when the UAV 100 flies in a sport mode in which a larger acceleration is likely to occur than in the normal mode.
- the control unit may determine that the predetermined condition is satisfied when the instruction related to the flight of the UAV 100 is a predetermined instruction that causes the imaging apparatus 300 to generate an acceleration greater than the threshold value.
- the predetermined instruction may include an instruction for landing the UAV 100.
- the predetermined command may include an acceleration command in which an acceleration generated in the UAV 100 is greater than a threshold value.
- FIG. 3 is a flowchart showing an example of the power control procedure of the actuator.
- the control unit acquires information serving as a determination criterion (S100). For example, when the lens control unit 410 controls an actuator that drives the diaphragm 452, the lens control unit 410 may acquire the acceleration of the imaging device 300 detected by the acceleration sensor 324 via the imaging control unit 310. The lens control unit 410 may acquire the posture information of the gimbal 200. The lens control unit 410 may acquire the UAV 100 altitude information, the UAV 100 flight mode, or a command related to the UAV 100 flight from the UAV control unit 110 via the imaging control unit 310.
- S100 determination criterion
- the control unit determines whether the acquired information satisfies a predetermined condition (S102). For example, the lens control unit 410 may determine whether or not the acceleration of the imaging device 300 is greater than or equal to a threshold value. The lens control unit 410 may determine whether the flight mode of the UAV 100 is the sports mode. The lens control unit 410 may determine whether the posture of the UAV 100 is a horizontal posture. The lens control unit 410 may determine whether the altitude of the UAV 100 is lower than a threshold value.
- a predetermined condition S102. For example, the lens control unit 410 may determine whether or not the acceleration of the imaging device 300 is greater than or equal to a threshold value. The lens control unit 410 may determine whether the flight mode of the UAV 100 is the sports mode. The lens control unit 410 may determine whether the posture of the UAV 100 is a horizontal posture. The lens control unit 410 may determine whether the altitude of the UAV 100 is lower than a threshold value.
- control unit may control the actuator so that the second electric power is supplied to the actuator in order to hold the optical member with the second holding force (S104).
- control unit may control the actuator so that the first electric power is supplied to the actuator in order to cause the actuator to hold the optical member with the first holding force (S106). ).
- the control unit may control the power supplied to the actuator by controlling the voltage applied to the actuator or the magnitude of the supplied current.
- the control unit may control the actuator so that a voltage of 1 V is applied to the actuator so that the actuator holds the optical member with the first holding force.
- the control unit may control the actuator so that a voltage of 5 V is applied to the actuator in order to cause the actuator to hold the optical member with the second holding force.
- the voltage applied to the actuator may vary depending on the contents of the conditions included in the predetermined conditions. For example, when the altitude is lower than 5 m, the control unit may control the actuator so as to apply a voltage of 5 V to the actuator.
- the control unit may control the actuator so that a voltage of 2 V is applied to the actuator.
- the control unit may control the actuator so that a voltage of 1.5 V is applied to the actuator while the UAV 100 is flying in a horizontal posture.
- the control unit may control the actuator so that a voltage of 1 V is applied to the actuator.
- the control unit may control the actuator so that a voltage of 1 V is applied to the actuator while the UAV 100 is flying in a posture other than the horizontal posture.
- FIG. 4 is a diagram showing an example of a temporal change in the power of the stepping motor that drives the diaphragm 452 and the motor rotation angle.
- the lens control unit 410 applies an AC voltage to the A phase and the B phase of the stepping motor, and sets the aperture of the aperture 452 to a desired aperture value.
- the lens control unit 410 controls the actuator so that the second power is supplied to the stepping motor in the period T2.
- the lens control unit 410 controls the actuator so that the first power smaller than the second power is supplied to the stepping motor when a predetermined period has elapsed. Control.
- the lens control unit 410 operates the actuator in the power saving mode in the period T3. After that, when the lens control unit 410 determines that a relatively high acceleration occurs in the imaging apparatus 300 or a predetermined condition that may possibly occur, the second power is supplied again to the stepping motor in the period T4. To control the actuator.
- the diaphragm 452 When the stepping motor is driven in the power saving mode, the diaphragm 452 can be held at a desired position by the holding force of the stepping motor even if a relatively small vibration occurs in the imaging apparatus 300.
- the holding power of the stepping motor that operates in the power saving mode is weak, and the diaphragm 452 may not be held at a desired position. Therefore, if the predetermined condition is satisfied while the stepping motor is operating in the power saving mode, the lens control unit 410 operates the actuator in a mode in which the power is higher than that in the power saving mode, and holds the actuator. Increase power. As a result, the diaphragm 452 can be held at a desired position even when a relatively large vibration occurs in the imaging apparatus 300.
- FIG. 5 is a diagram showing an example of a circuit configuration including a focus lens and a voice coil motor.
- the focus lens 500 is an example of the lens 442.
- the position sensor 520 is an example of the position sensor 444.
- the voice coil motor 510, the control gain setting unit 522, and the drive circuit 524 are examples of the lens drive unit 440.
- Voice coil motor 510 includes a coil 512 and a magnet 514.
- the difference between the target position commanded from the lens control unit 410 and the actual position of the focus lens 500 detected by the position sensor 520 is input to the control gain setting unit 522.
- the lens control unit 410 When the voice coil motor 510 is driven with the first power, for example, the lens control unit 410 outputs a command for setting the gain to 1 time to the control gain setting unit 522.
- the control gain setting unit 522 sets the gain to 1 and outputs a command corresponding to the difference to the drive circuit 524.
- the drive circuit 524 receives a command and outputs a drive current corresponding to the difference to the coil 512. Thereby, the focus lens 500 is controlled to the target position.
- the lens control unit 410 When the voice coil motor 510 is driven with the second power, for example, the lens control unit 410 outputs a command for setting the gain to double to the control gain setting unit 522.
- the control gain setting unit 522 sets the gain to double and outputs a command corresponding to the difference to the drive circuit 524.
- the magnitude of the gain By changing the magnitude of the gain, the magnitude of the drive current input to the coil 512 is changed.
- the force generated by the voice coil motor 510 that is, the holding force of the focus lens 500 is changed.
- FIG. 6 is a diagram illustrating an example of the relationship among the acceleration generated in the imaging apparatus 300, the lens position of the focus lens 500, and the gain setting.
- the lens control unit 410 outputs a command to the control gain setting unit 522 to set the gain to 2 times in response to detection of acceleration higher than the threshold value.
- the holding force of the focus lens 500 generated by the voice coil motor 510 is increased. Therefore, the amount by which the focus lens 500 deviates from the target position can be made smaller than when the gain is 1.
- the force for holding the optical member generated by the actuator may be controlled by controlling the gain of the drive current supplied to the actuator.
- FIG. 7 illustrates an example of a computer 1200 in which aspects of the present invention may be embodied in whole or in part.
- a program installed in the computer 1200 can cause the computer 1200 to function as an operation associated with the apparatus according to the embodiment of the present invention or as one or more “units” of the apparatus.
- the program can cause the computer 1200 to execute the operation or the one or more “units”.
- the program can cause the computer 1200 to execute a process according to an embodiment of the present invention or a stage of the process.
- Such a program may be executed by CPU 1212 to cause computer 1200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.
- the computer 1200 includes a CPU 1212 and a RAM 1214, which are connected to each other by a host controller 1210.
- the computer 1200 also includes a communication interface 1222 and an input / output unit, which are connected to the host controller 1210 via the input / output controller 1220.
- Computer 1200 also includes ROM 1230.
- the CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit.
- the communication interface 1222 communicates with other electronic devices via a network.
- a hard disk drive may store programs and data used by the CPU 1212 in the computer 1200.
- the ROM 1230 stores therein a boot program executed by the computer 1200 at the time of activation and / or a program depending on the hardware of the computer 1200.
- the program is provided via a computer-readable recording medium such as a CR-ROM, a USB memory, or an IC card or a network.
- the program is installed in the RAM 1214 or the ROM 1230 that is also an example of a computer-readable recording medium, and is executed by the CPU 1212.
- Information processing described in these programs is read by the computer 1200 to bring about cooperation between the programs and the various types of hardware resources.
- An apparatus or method may be configured by implementing information operations or processing in accordance with the use of computer 1200.
- the CPU 1212 executes a communication program loaded in the RAM 1214 and performs communication processing on the communication interface 1222 based on the processing described in the communication program. You may order.
- the communication interface 1222 reads transmission data stored in a RAM 1214 or a transmission buffer area provided in a recording medium such as a USB memory under the control of the CPU 1212 and transmits the read transmission data to a network, or The reception data received from the network is written into a reception buffer area provided on the recording medium.
- the CPU 1212 allows the RAM 1214 to read all or necessary portions of a file or database stored in an external recording medium such as a USB memory, and executes various types of processing on the data on the RAM 1214. Good. The CPU 1212 may then write back the processed data to an external recording medium.
- the CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval that are described throughout the present disclosure for data read from the RAM 1214 and specified by the instruction sequence of the program. Various types of processing may be performed, including / replacement, etc., and the result is written back to RAM 1214.
- the CPU 1212 may search for information in files, databases, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. The entry that matches the condition is searched from the plurality of entries, the attribute value of the second attribute stored in the entry is read, and thereby the first attribute that satisfies the predetermined condition is associated. The attribute value of the obtained second attribute may be acquired.
- the program or software module described above may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200.
- a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the program is transferred to the computer 1200 via the network.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Optics & Photonics (AREA)
- Studio Devices (AREA)
- Lens Barrels (AREA)
Abstract
Description
特許文献1 特開2013-156358号公報
110 UAV制御部
150 通信インタフェース
160 メモリ
200 ジンバル
210 回転翼機構
230 撮像装置
240 GPS受信機
260 磁気コンパス
270 気圧高度計
300 撮像装置
301 撮像部
310 撮像制御部
320 シャッタ駆動部
322 シャッタ
324 加速度センサ
330 撮像素子
340 メモリ
401 レンズ部
410 レンズ制御部
420 メモリ
430 レンズ駆動部
432 レンズ
434 位置センサ
440 レンズ駆動部
442 レンズ
444 位置センサ
450 絞り駆動部
452 絞り
460 フィルタ駆動部
462 フィルタ
500 フォーカスレンズ
510 ボイスコイルモータ
512 コイル
514 磁石
520 位置センサ
522 制御ゲイン設定部
524 駆動回路
1200 コンピュータ
1210 ホストコントローラ
1212 CPU
1214 RAM
1220 入力/出力コントローラ
1222 通信インタフェース
1230 ROM
Claims (16)
- 撮像装置の光学部材を保持するアクチュエータを制御する制御部
を備え、
前記アクチュエータは、第1電力が供給されている場合に第1保持力で前記光学部材を保持し、前記第1電力より大きい第2電力が供給されている場合に前記第1保持力より大きい第2保持力で前記光学部材を保持し、
前記制御部は、前記アクチュエータが前記第1保持力で前記光学部材を保持している場合に、前記撮像装置の高度に関する条件、前記撮像装置の加速度に関する条件、前記撮像装置を搭載して移動する移動体の姿勢に関する条件、前記移動体の移動モードに関する条件、及び前記移動体の移動を制御する命令に関する条件の少なくとも1つを含む予め定められた条件が満たされると、前記第2保持力で前記光学部材を保持するように前記アクチュエータを制御する、制御装置。 - 前記予め定められた条件は、前記撮像装置の高度が閾値より低いという条件である、請求項1に記載の制御装置。
- 前記予め定められた条件は、前記撮像装置の加速度が閾値より大きいという条件である、請求項1に記載の制御装置。
- 前記予め定められた条件は、前記撮像装置の加速度の方向と前記光学部材が移動可能な方向とが予め定められた角度の範囲内にあるという条件である、請求項1に記載の制御装置。
- 前記移動体は、無人航空機であり、
前記予め定められた条件は、前記無人航空機の飛行姿勢が予め定められた飛行姿勢であるという条件である、請求項1に記載の制御装置。 - 前記移動体は、無人航空機であり、
前記無人航空機は、第1飛行モード、及び前記第1飛行モードで飛行する前記無人航空機に生じる加速度より大きい加速度が生じる第2飛行モードで飛行可能であり、
前記予め定められた条件は、前記無人航空機が前記第2飛行モードで飛行するという条件である、請求項1に記載の制御装置。 - 前記移動体は、無人航空機であり、
前記予め定められた条件は、前記命令が、前記撮像装置に閾値より大きい加速度を生じさせる予め定められた命令であるという条件である、請求項1に記載の制御装置。 - 前記予め定められた命令は、前記無人航空機を着陸させる命令を含む、請求項7に記載の制御装置。
- 前記光学部材は、前記撮像装置の絞り、シャッタ、フィルタ、及びレンズの少なくとも1つを含む、請求項1に記載の制御装置。
- 前記アクチュエータは、電磁アクチュエータである、請求項1に記載の制御装置。
- 前記アクチュエータは、ステッピングモータであり、
前記光学部材は、前記撮像装置の絞りである、請求項1に記載の制御装置。 - 前記アクチュエータは、ボイスコイルモータであり、
前記光学部材は、前記撮像装置のレンズである、請求項1に記載の制御装置。 - 請求項1から12の何れか1つに記載の制御装置と、
前記光学部材と、
前記アクチュエータと
を備える、撮像装置。 - 請求項13に記載の撮像装置を搭載して移動する移動体。
- 撮像装置の光学部材を保持するアクチュエータを制御する制御方法であって、
前記アクチュエータは、第1電力が供給されている場合に第1保持力で前記光学部材を保持し、前記第1電力より大きい第2電力が供給されている場合に前記第1保持力より大きい第2保持力で前記光学部材を保持し、
前記制御方法は、
前記アクチュエータが前記第1保持力で前記光学部材を保持している場合に、前記撮像装置の高度に関する条件、前記撮像装置の加速度に関する条件、前記撮像装置を搭載して移動する移動体の姿勢に関する条件、前記移動体の移動モードに関する条件、及び前記移動体の移動を制御する命令に関する条件の少なくとも1つを含む予め定められた条件が満たされると、前記第2保持力で前記光学部材を保持するように前記アクチュエータを制御する段階を備える、制御方法。 - 撮像装置の光学部材を保持するアクチュエータを制御する制御部としてコンピュータを機能させるためのプログラムであって、
前記アクチュエータは、第1電力が供給されている場合に第1保持力で前記光学部材を保持し、前記第1電力より大きい第2電力が供給されている場合に前記第1保持力より大きい第2保持力で前記光学部材を保持し、
前記制御部は、前記アクチュエータが前記第1保持力で前記光学部材を保持している場合に、前記撮像装置の高度に関する条件、前記撮像装置の加速度に関する条件、前記撮像装置を搭載して移動する移動体の姿勢に関する条件、前記移動体の移動モードに関する条件、及び前記移動体の移動を制御する命令に関する条件の少なくとも1つを含む予め定められた条件が満たされると、前記第2保持力で前記光学部材を保持するように前記アクチュエータを制御する、プログラム。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017559714A JP6544542B2 (ja) | 2016-12-21 | 2016-12-21 | 制御装置、撮像装置、無人航空機、制御方法、及びプログラム |
PCT/JP2016/088157 WO2018116417A1 (ja) | 2016-12-21 | 2016-12-21 | 制御装置、撮像装置、移動体、制御方法、及びプログラム |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2016/088157 WO2018116417A1 (ja) | 2016-12-21 | 2016-12-21 | 制御装置、撮像装置、移動体、制御方法、及びプログラム |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018116417A1 true WO2018116417A1 (ja) | 2018-06-28 |
Family
ID=62626190
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2016/088157 WO2018116417A1 (ja) | 2016-12-21 | 2016-12-21 | 制御装置、撮像装置、移動体、制御方法、及びプログラム |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP6544542B2 (ja) |
WO (1) | WO2018116417A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020042183A (ja) * | 2018-09-11 | 2020-03-19 | 株式会社五鈴精工硝子 | 無人移動装置及びターレット切替え装置 |
JP2020118944A (ja) * | 2019-01-28 | 2020-08-06 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | 制御装置、撮像装置、移動体、制御方法、及びプログラム |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001197481A (ja) * | 2000-01-13 | 2001-07-19 | Mitsubishi Electric Corp | 撮影システム |
JP2013011732A (ja) * | 2011-06-29 | 2013-01-17 | Canon Inc | 光学機器およびその制御方法 |
JP2015163943A (ja) * | 2014-01-29 | 2015-09-10 | パナソニックIpマネジメント株式会社 | 撮像装置 |
JP2015195569A (ja) * | 2014-03-25 | 2015-11-05 | パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカPanasonic Intellectual Property Corporation of America | 移動体用撮影装置 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001083398A (ja) * | 1999-09-13 | 2001-03-30 | Hitachi Ltd | 撮像装置 |
JP2010044315A (ja) * | 2008-08-18 | 2010-02-25 | Hoya Corp | 駆動装置 |
-
2016
- 2016-12-21 JP JP2017559714A patent/JP6544542B2/ja not_active Expired - Fee Related
- 2016-12-21 WO PCT/JP2016/088157 patent/WO2018116417A1/ja active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001197481A (ja) * | 2000-01-13 | 2001-07-19 | Mitsubishi Electric Corp | 撮影システム |
JP2013011732A (ja) * | 2011-06-29 | 2013-01-17 | Canon Inc | 光学機器およびその制御方法 |
JP2015163943A (ja) * | 2014-01-29 | 2015-09-10 | パナソニックIpマネジメント株式会社 | 撮像装置 |
JP2015195569A (ja) * | 2014-03-25 | 2015-11-05 | パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカPanasonic Intellectual Property Corporation of America | 移動体用撮影装置 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020042183A (ja) * | 2018-09-11 | 2020-03-19 | 株式会社五鈴精工硝子 | 無人移動装置及びターレット切替え装置 |
JP7163114B2 (ja) | 2018-09-11 | 2022-10-31 | 株式会社五鈴精工硝子 | 無人移動装置及びターレット切替え装置 |
JP2020118944A (ja) * | 2019-01-28 | 2020-08-06 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | 制御装置、撮像装置、移動体、制御方法、及びプログラム |
Also Published As
Publication number | Publication date |
---|---|
JP6544542B2 (ja) | 2019-07-17 |
JPWO2018116417A1 (ja) | 2018-12-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6399498B1 (ja) | 制御装置、撮像システム、移動体、制御方法、及びプログラム | |
JP6496955B1 (ja) | 制御装置、システム、制御方法、及びプログラム | |
JP6493702B2 (ja) | 制御方法、撮像装置、撮像システム、移動体、制御方法、及びプログラム | |
JP2019066563A (ja) | 制御装置、レンズ装置、撮像装置、制御方法、及びプログラム | |
WO2018092283A1 (ja) | 制御装置、撮像システム、移動体、制御方法、およびプログラム | |
JP6630939B2 (ja) | 制御装置、撮像装置、移動体、制御方法、及びプログラム | |
WO2018116417A1 (ja) | 制御装置、撮像装置、移動体、制御方法、及びプログラム | |
JP6496953B2 (ja) | 制御装置、撮像装置、移動体、制御方法、及びプログラム | |
JP6565072B2 (ja) | 制御装置、レンズ装置、飛行体、制御方法、及びプログラム | |
JP6501091B1 (ja) | 制御装置、撮像装置、移動体、制御方法、及びプログラム | |
JP6331180B1 (ja) | 制御装置、撮像装置、撮像システム、飛行体、制御方法、及びプログラム | |
WO2019174343A1 (zh) | 活动体检测装置、控制装置、移动体、活动体检测方法及程序 | |
JP7048019B2 (ja) | 制御装置、飛行体、制御方法、及びプログラム | |
WO2018185940A1 (ja) | 撮像制御装置、撮像装置、撮像システム、移動体、撮像制御方法、及びプログラム | |
JP6543859B1 (ja) | 画像処理装置、撮像装置、移動体、画像処理方法、及びプログラム | |
JP6565071B2 (ja) | 制御装置、撮像装置、飛行体、制御方法、及びプログラム | |
WO2018163300A1 (ja) | 制御装置、撮像装置、撮像システム、移動体、制御方法、及びプログラム | |
US11363195B2 (en) | Control device, imaging device, imaging system, movable object, control method, and program | |
JP6459012B1 (ja) | 制御装置、撮像装置、飛行体、制御方法、及びプログラム | |
JP2019205047A (ja) | 制御装置、撮像装置、移動体、制御方法、及びプログラム | |
JP6818987B1 (ja) | 画像処理装置、撮像装置、移動体、画像処理方法、及びプログラム | |
JP6409239B1 (ja) | 撮像装置、撮像システム、飛行体、記録方法、及びプログラム | |
JP2020067558A (ja) | 制御装置、撮像装置、移動体、制御方法、及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2017559714 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16924340 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17/10/2019) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16924340 Country of ref document: EP Kind code of ref document: A1 |