JP7003357B2 - Control device, image pickup device, moving object, control method, and program - Google Patents

Description

The present invention relates to a control device, an image pickup device, a moving body, a control method, and a program.

Patent Document 1 discloses an image pickup apparatus having a function of specifying the type of an accessory attached to an image pickup lens unit. Patent Document 1 exemplifies a teleconverter lens, a wide converter lens, and the like as accessories.
[Prior Art Document]
[Patent Document]
[Patent Document 1] Japanese Unexamined Patent Publication No. 2010-875884

There is an upper limit to the speed at which the image sensor can output pixel information. However, the imaging rate can be increased by narrowing the range in which the image is read from the image sensor. However, if the range for reading the image from the image sensor is narrowed, the viewing angle becomes narrow.

The control device according to one aspect of the present invention includes a circuit configured to acquire information indicating an imaging rate. The circuit is configured to change the angle of view of the image pickup lens to a wide angle when the image pickup rate exceeds a predetermined value, and narrow the range of reading an image from an image sensor that receives light that has passed through the image pickup lens. The lens.

The image pickup lens may include a wide conversion lens that can be inserted into the subject optical path. The circuit may be configured to insert a wide conversion lens into the subject optical path to narrow the range of reading an image from the image sensor when the imaging rate exceeds a predetermined value.

The wide conversion lens may be movable between the insertion position inserted into the subject optical path and the retracted position retracted from the subject optical path. The circuit moves the wide conversion lens from the retracted position to the insertion position when the imaging rate transitions from a state that does not exceed a predetermined value to a state that exceeds a predetermined value, and the range for reading an image from the image sensor. May be narrowed.

The wide conversion lens may be movable between the insertion position inserted into the subject optical path and the retracted position retracted from the subject optical path. The circuit is a range in which the wide conversion lens is moved from the insertion position to the retracted position and the image is read from the image sensor when the imaging rate transitions from a state in which the imaging rate exceeds a predetermined value to a state in which the imaging rate does not exceed a predetermined value. May be wide.

The circuit may have a first read mode for reading an image from a first range of the image sensor and a second read mode for reading an image from a range 1 / n times the first range. The wide conversion lens may be designed so that the focal length of the image pickup lens is 1 / n times larger when it is inserted in the subject optical path than when it is not inserted in the subject optical path. The circuit may be configured to insert a wide conversion lens into the subject optical path and read the image from the image sensor in the second read mode when the imaging rate exceeds a predetermined value.

The circuit changes the angle of view of the image pickup lens to a wide angle when the image pickup rate instructed by the user of the image pickup device equipped with the image sensor exceeds a predetermined value, and narrows the range in which the image is read from the image sensor. May be configured in.

The image pickup apparatus according to one aspect of the present invention includes the above-mentioned image sensor and the above-mentioned control device.

The image pickup apparatus according to one aspect of the present invention includes the above-mentioned image sensor, the above-mentioned wide conversion lens, and the above-mentioned control device.

The moving body according to one aspect of the present invention may be a moving body provided with the above-mentioned imaging device.

The control method according to one aspect of the present invention includes a step of acquiring information indicating an imaging rate. The control method includes a step of changing the angle of view of the image pickup lens to a wide angle when the image pickup rate exceeds a predetermined value, and narrowing the range of reading an image from an image sensor that receives light that has passed through the image pickup lens. ..

The program according to one aspect of the present invention causes a computer to perform a procedure for acquiring information indicating an imaging rate. The program tells the computer a procedure to change the angle of view of the image pickup lens to a wide angle and narrow the range of reading the image from the image sensor that receives the light that has passed through the image pickup lens when the image pickup rate exceeds a predetermined value. Let it run.

According to one aspect of the present invention, an image can be read out from an image sensor at high speed while suppressing a decrease in the viewing angle.

The outline of the above invention does not list all the necessary features of the present invention. A subcombination of these feature groups can also be an invention.

An example of the appearance of the unmanned aerial vehicle (UAV) 10 and the remote control device 300 is shown. An example of the functional block of UAV10 is shown. The state where the wide conversion lens 218 is inserted in the subject optical path is shown. It shows a state in which the wide conversion lens 218 is retracted from the subject optical path. The case of taking an image at a frame rate of 30 fps is shown. The case of taking an image at a frame rate of 60 fps is shown. It is a flowchart which shows the procedure of the process which the image pickup control unit 110 executes. An example of the computer 1200 is shown.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention. It will be apparent to those skilled in the art that various changes or improvements can be made to the following embodiments. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

The claims, description, drawings, and abstracts include matters subject to copyright protection. The copyright holder will not object to any person's reproduction of these documents as long as they appear in the Patent Office files or records. However, in other cases, all copyrights are reserved.

Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, wherein the block is (1) a stage of the process in which the operation is performed or (2) a device having a role of performing the operation. May represent a "part" of. Specific steps and "parts" may be implemented by programmable circuits and / or processors. Dedicated circuits may include digital and / or analog hardware circuits. It may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include a reconfigurable hardware circuit. Reconfigurable hardware circuits include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip flops, registers, field programmable gate arrays (FPGA), programmable logic arrays (PLA), etc. It may include a memory element such as.

The computer readable medium may include any tangible device capable of storing instructions executed by the appropriate device. As a result, the computer-readable medium having the instructions stored therein will include the product, including instructions that can be executed to create means for performing the operation specified in the flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy (registered trademark) disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), Electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray (RTM) disk, memory stick, integrated A circuit card or the like may be included.

Computer-readable instructions may include either source code or object code written in any combination of one or more programming languages. Source code or object code includes traditional procedural programming languages. Traditional procedural programming languages are assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcodes, firmware instructions, state setting data, or Smalltalk®, JAVA®, C ++. It may be an object-oriented programming language such as, and a "C" programming language or a similar programming language. Computer-readable instructions are used locally or to a local area network (LAN), wide area network (WAN), etc., to the processor or programmable circuit of a general purpose computer, special purpose computer, or other programmable data processing device. ) May be provided. The processor or programmable circuit may execute computer-readable instructions to create means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

FIG. 1 shows an example of the appearance of an unmanned aerial vehicle (UAV) 10 and a remote control device 300. The UAV 10 includes a UAV main body 20, a gimbal 50, a plurality of image pickup devices 60, and an image pickup device 100. The gimbal 50 and the image pickup device 100 are examples of an image pickup system. The UAV 10 is a concept including a moving body, a flying body moving in the air, a vehicle moving on the ground, a ship moving on the water, and the like. An airship that moves in the air is a concept that includes UAVs, other aircraft that move in the air, airships, helicopters, and the like.

The UAV main body 20 includes a plurality of rotary wings. The plurality of rotor blades are an example of a propulsion unit. The UAV main body 20 flies the UAV 10 by controlling the rotation of a plurality of rotary blades. The UAV body 20 flies the UAV 10 using, for example, four rotor blades. The number of rotor blades is not limited to four. Further, the UAV 10 may be a fixed-wing aircraft having no rotor blades.

The image pickup apparatus 100 is a camera for taking an image of a subject included in a desired imaging range. The gimbal 50 rotatably supports the image pickup device 100. The gimbal 50 is an example of a support mechanism. For example, the gimbal 50 rotatably supports the image pickup device 100 on a pitch axis using an actuator. The gimbal 50 further rotatably supports the image pickup device 100 around each of the roll axis and the yaw axis by using an actuator. The gimbal 50 may change the posture of the image pickup device 100 by rotating the image pickup device 100 around at least one of the yaw axis, the pitch axis, and the roll axis.

The plurality of image pickup devices 60 are sensing cameras that image the surroundings of the UAV 10 in order to control the flight of the UAV 10. Two image pickup devices 60 may be provided in front of the nose of the UAV 10. Yet two other imaging devices 60 may be provided on the bottom surface of the UAV 10. The two image pickup devices 60 on the front side may be paired and function as a so-called stereo camera. The two image pickup devices 60 on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the UAV 10 may be generated based on the images captured by the plurality of image pickup devices 60. The number of image pickup devices 60 included in the UAV 10 is not limited to four. The UAV 10 may include at least one image pickup device 60. The UAV 10 may be provided with at least one image pickup device 60 on each of the nose, aft, side surface, bottom surface, and ceiling surface of the UAV 10. The angle of view that can be set by the image pickup device 60 may be wider than the angle of view that can be set by the image pickup device 100. The image pickup apparatus 60 may have a single focus lens or a fisheye lens.

The remote control device 300 communicates with the UAV 10 to remotely control the UAV 10. The remote control device 300 may wirelessly communicate with the UAV 10. The remote control device 300 transmits to the UAV 10 instruction information indicating various commands related to the movement of the UAV 10, such as ascending, descending, accelerating, decelerating, advancing, reversing, and rotating. The instruction information includes, for example, instruction information for raising the altitude of the UAV 10. The instruction information may indicate the altitude at which the UAV 10 should be located. The UAV 10 moves so as to be located at an altitude indicated by the instruction information received from the remote control device 300. The instruction information may include an ascending instruction to elevate the UAV 10. The UAV10 rises while accepting the rise order. Even if the UAV10 accepts an ascending order, the ascending may be restricted if the altitude of the UAV10 has reached the upper limit altitude.

FIG. 2 shows an example of a functional block of UAV10. The UAV 10 includes a UAV control unit 30, a memory 37, a communication interface 36, a propulsion unit 40, a GPS receiver 41, an inertial measurement unit (IMU) 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, and a gimbal 50. The image pickup device 60 and the image pickup device 100 are provided.

The communication interface 36 communicates with another device such as the remote control device 300. The communication interface 36 may receive instruction information including various commands from the remote control device 300 to the UAV control unit 30. In the memory 37, the UAV control unit 30 has a propulsion unit 40, a GPS receiver 41, an inertial measurement unit 42, a magnetic compass 43, a barometric altimeter 44, a temperature sensor 45, a humidity sensor 46, a gimbal 50, an image pickup device 60, and an image pickup device. Stores programs and the like necessary to control 100. The memory 37 may be a computer-readable recording medium and may include at least one of a flash memory such as a SRAM, DRAM, EEPROM, EEPROM, USB memory, and a solid state drive (SSD). The memory 37 may be provided inside the UAV main body 20. It may be provided so as to be removable from the UAV main body 20.

The UAV control unit 30 controls the flight and imaging of the UAV 10 according to the program stored in the memory 37. The UAV control unit 30 may be composed of a CPU, a microprocessor such as an MPU, a microcontroller such as an MCU, or the like. The UAV control unit 30 controls the flight and imaging of the UAV 10 according to a command received from the remote control device 300 via the communication interface 36. The propulsion unit 40 promotes the UAV 10. The propulsion unit 40 has a plurality of rotary blades and a plurality of drive motors for rotating the plurality of rotary blades. The propulsion unit 40 rotates a plurality of rotor blades via a plurality of drive motors in accordance with a command from the UAV control unit 30, and causes the UAV 10 to fly.

The GPS receiver 41 receives a plurality of signals indicating the time transmitted from the plurality of GPS satellites. The GPS receiver 41 calculates the position (latitude and longitude) of the GPS receiver 41, that is, the position (latitude and longitude) of the UAV 10 based on the plurality of received signals. The IMU 42 detects the posture of the UAV 10. The IMU 42 detects, as the posture of the UAV 10, the acceleration in the front-back, left-right, and up-down triaxial directions of the UAV 10, and the angular velocity in the three axial directions of pitch, roll, and yaw. The magnetic compass 43 detects the orientation of the nose of the UAV 10. The barometric altimeter 44 detects the altitude at which the UAV 10 flies. The barometric altimeter 44 detects the barometric pressure around the UAV 10, converts the detected barometric pressure into an altitude, and detects the altitude. The temperature sensor 45 detects the ambient temperature of the UAV 10. The humidity sensor 46 detects the humidity around the UAV 10.

The image pickup apparatus 100 includes an image pickup unit 102 and a lens unit 200. The lens unit 200 is an example of a lens device. The image pickup unit 102 includes an image sensor 120, an image pickup control unit 110, a memory 130, and a distance measuring sensor. The image sensor 120 may be configured by CCD or CMOS. The image sensor 120 captures an optical image formed by light received through the plurality of lenses 210, and outputs pixel information constituting the image to the image pickup control unit 110. The image pickup control unit 110 generates an image for recording by performing image processing based on the pixel information read from the image sensor 120, and records the image in the memory 130. The image pickup control unit 110 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The image pickup control unit 110 may control the image pickup device 100 in response to an operation command of the image pickup device 100 from the UAV control unit 30. The image pickup control unit 110 is an example of a circuit. The memory 130 may be a computer-readable recording medium and may include at least one of a flash memory such as a SRAM, DRAM, EEPROM, EEPROM, USB memory, and a solid state drive (SSD). The memory 130 stores a program or the like necessary for the image pickup control unit 110 to control the image sensor 120 or the like. The memory 130 may be provided inside the housing of the image pickup apparatus 100. The memory 130 may be provided so as to be removable from the housing of the image pickup apparatus 100.

The distance measuring sensor measures the distance to the subject. The distance measuring sensor may be an infrared sensor, an ultrasonic sensor, a stereo camera, a TOF (Time Of Flat) sensor, or the like.

The lens unit 200 includes a plurality of lenses 210, a plurality of lens drive units 212, a wide conversion lens 218, a lens drive unit 219, and a lens control unit 220. The plurality of lenses 210 may function as a zoom lens, a varifocal lens, and a focus lens. At least a part or all of the plurality of lenses 210 are arranged so as to be movable along the optical axis. The wide conversion lens 218 can be inserted into the subject optical path. The lens drive unit 219 inserts the wide conversion lens 218 into the subject optical path according to a control command from the lens control unit 220. Further, the lens driving unit 219 retracts the wide conversion lens 218 from the subject optical path according to the control command of the lens control unit 220.

The lens unit 200 may be an interchangeable lens that is detachably provided with respect to the image pickup unit 102. The lens driving unit 212 moves at least a part or all of the plurality of lenses 210 along the optical axis via a mechanical member such as a cam ring. The lens driving unit 212 may include an actuator. The actuator may include a stepping motor. The lens control unit 220 drives the lens drive unit 212 in accordance with a lens control command from the image pickup unit 102 to move one or a plurality of lenses 210 along the optical axis direction via a mechanical member. The lens control command is, for example, a zoom control command, a focus control command, a control command for instructing switching of the position of the wide conversion lens 218, and the like.

The lens unit 200 further includes a memory 222 and a position sensor 214. The lens control unit 220 controls the movement of the lens 210 in the optical axis direction via the lens drive unit 212 in response to a lens operation command from the image pickup unit 102. Part or all of the lens 210 moves along the optical axis. The lens control unit 220 executes at least one of the zoom operation and the focus operation by moving at least one of the lenses 210 along the optical axis. The position sensor 214 detects the position of the lens 210. The position sensor 214 may detect the current zoom position or focus position.

The lens driving unit 212 may include a shake correction mechanism. The lens control unit 220 may perform image stabilization by moving the lens 210 in a direction along the optical axis or in a direction perpendicular to the optical axis via the image stabilization mechanism. The lens drive unit 212 may drive the image stabilization mechanism by a stepping motor to perform image stabilization. The image stabilization mechanism may be driven by a stepping motor to move the image sensor 120 in a direction along the optical axis or in a direction perpendicular to the optical axis to perform image stabilization.

The memory 222 stores the control values of the plurality of lenses 210 that move via the lens driving unit 212. The memory 222 may include at least one of flash memories such as SRAM, DRAM, EPROM, EEPROM, and USB memory.

A control in which the image pickup control unit 110 reads out an image from the image sensor 120 will be described. The image pickup control unit 110 still acquires information indicating the frame rate. The frame rate is an example of the imaging rate. The image pickup rate is information indicating the number of times a moving image constituting a moving image is acquired per unit time.

When the frame rate exceeds a predetermined value, the image pickup control unit 110 inserts the wide conversion lens 218 into the subject optical path and narrows the range in which the image is read from the image sensor 120. By inserting the wide-angle conversion lens 218 in this way, the angle of view of the image pickup lens including the lens 210 and the wide-angle conversion lens 218 is changed to a wide angle.

The image pickup control unit 110 may acquire information indicating a frame rate specified by the user through the remote control device 300. When the frame rate instructed by the user of the image pickup apparatus provided with the image sensor 120 exceeds a predetermined value, the image pickup control unit 110 changes the angle of view of the lens 210 to a wide angle and transmits the light passing through the lens 210. The range for reading an image from the image sensor 120 that receives light is narrowed.

The wide conversion lens 218 is movable between the insertion position inserted into the subject optical path and the retracted position retracted from the subject optical path. The image pickup control unit 110 moves the wide conversion lens 218 from the retracted position to the insertion position when the frame rate transitions from a state in which the frame rate does not exceed a predetermined value to a state in which the frame rate exceeds a predetermined value, and the image sensor 120 Narrow the range of reading images from. Further, the image pickup control unit 110 moves the wide conversion lens 218 from the insertion position to the retracted position when the frame rate transitions from a state in which the frame rate exceeds a predetermined value to a state in which the frame rate does not exceed a predetermined value. The range of reading the image from the sensor 120 is widened.

The image pickup control unit 110 has a first read mode for reading an image from a first range of the image sensor 120, and a second read mode for reading an image from a range 1 / n times the first range. Note that n is a value larger than 1. The first range is, for example, a range in which effective pixels of the image sensor 120 are arranged. The image pickup control unit 110 reads out an image from the first range when the frame rate does not exceed a predetermined value.

When the wide conversion lens 218 is inserted in the subject optical path, the focal length of the entire lens system including the lens 210 and the wide conversion lens 218 is increased by 1 / n times as compared with the case where the wide conversion lens 218 is not inserted in the subject optical path. It is designed to be. In this case, when the frame rate exceeds a predetermined value, the image pickup control unit 110 inserts the wide conversion lens 218 into the subject optical path and reads an image from the image sensor 120 in the second read mode. As described above, the image pickup control unit 110 reads the pixel information of all the effective pixels from the image sensor 120 when the wide conversion lens 218 is not inserted in the subject optical path, and the wide conversion lens 218 is inserted in the subject optical path. In this case, the range in which the pixel information is partially read from the image sensor 120 is determined so that the viewing angle does not substantially change.

FIG. 3 shows a state in which the wide conversion lens 218 is inserted into the subject optical path. FIG. 4 shows a state in which the wide conversion lens 218 is retracted from the subject optical path. The coil 340 and the coil 350 shown in FIGS. 3 and 4 are examples of the components included in the lens driving unit 219.

The wide conversion lens 218 is fixed to the lens holding portion 310. A magnet 320 is fixed to the lens holding portion 310. The magnet 320 has an N pole 321 and an S pole 322. The wide conversion lens 218 moves between the insertion position and the retracted position by driving the coil 340 and the coil 350.

The lens holding portion 310 is rotatably provided around a rotation shaft 370 fixed to the lens portion 200. The lens control unit 220 moves the wide conversion lens 218 between the insertion position and the retracted position by passing a current through one of the coil 340 and the coil 350. The coil 340 forms an S pole on the N pole 321 side of the magnet 320 when a current is flowing through the coil 340. The coil 350 forms an N pole on the S pole 322 side of the magnet 320 when a current is flowing through the coil 350.

When a current is flowing through the coil 350, the S pole 322 of the magnet 320 is attracted to the coil 350. In this case, as shown in FIG. 3, the wide conversion lens 218 is located at the insertion position. When the wide conversion lens 218 is located at the insertion position, the center of the wide conversion lens 218 substantially coincides with the optical axis AX of the lens 210. When a current is flowing through the coil 340, the N pole 321 of the magnet 320 is attracted to the coil 340. In this case, as shown in FIG. 4, the wide conversion lens 218 is located in the retracted position.

The lens control unit 220 switches from the state in which the current is passed through the coil 340 to the state in which the current is passed through the coil 350 according to the control command acquired from the image pickup control unit 110, thereby causing the wide conversion lens 218. Is moved from the retracted position to the inserted position. Further, the lens control unit 220 moves the wide conversion lens 218 from the insertion position to the retracted position by switching from the state in which the current is passed through the coil 350 to the state in which the current is passed through the coil 340. ..

FIG. 5 shows a case where an image is taken at a frame rate of 30 fps. FIG. 6 shows a case where an image is taken at a frame rate of 60 fps.

The image sensor 120 has an ability to output pixel information of the number of effective pixels of the image sensor 120 30 times per second. Therefore, when imaging at a frame rate of 30 fps or less, the pixel information of the number of effective pixels possessed by the image sensor 120 can be output. For example, the image sensor 120 has an effective pixel count of 60000 × 4000 pixels in width and can output pixel information of 60,000 × 4000 pixels when imaging at a frame rate of 24 fps, 25 fps or 30 fps. Therefore, as shown in FIG. 5, when the frame rate is set to 30 fps, the image pickup control unit 110 arranges the wide conversion lens 218 in the retracted position and outputs the image information of 60,000 × 4000 pixels from the image sensor 120. read out. The operation mode when the image pickup control unit 110 reads the pixel information of the number of effective pixels from the image sensor 120 may be referred to as "all pixel read mode".

On the other hand, when the image sensor 120 takes an image at a frame rate exceeding 30 fps, the image sensor 120 can output only pixel information less than the number of effective pixels of the image sensor 120. For example, when imaging at a frame rate of 48 fps, 50 fps or 60 fps, the image sensor 120 can output only pixel information of less than 60,000 × 4000 pixels.

Therefore, as shown in FIG. 6, when the frame rate is set to 60 fps, the image pickup control unit 110 arranges the wide conversion lens 218 at the insertion position. As a result, the optical image formed on the image sensor 120 by the optical system is reduced. Then, the image pickup control unit 110 reads out the image information of a part of the pixels of the image sensor 120 by performing partial reading from the image sensor 120. The operation mode when the image pickup control unit 110 reads out a part of the pixel information from the image sensor 120 may be referred to as a "partial read mode".

Specifically, it is assumed that the focal length of the lens 210 is 24 mm, and when the wide conversion lens 218 is arranged at the insertion position, the focal lengths of the lens 210 and the wide conversion lens 218 are 16 mm. Further, it is assumed that the effective pixels of the image sensor 120 are arranged in an area of H in the horizontal direction and V in the vertical direction. In this case, when the wide conversion lens 218 is arranged at the insertion position, the image pickup control unit 110 has H × 2/3 in the horizontal direction and V × 2/3 in the vertical direction with the optical axis of the lens 210 as the center. Image information is read from the pixels arranged in the area 600. As a result, the number of pixels read from the image sensor 120 by the image pickup control unit 110 can be suppressed within a speed at which the image sensor 120 can output pixel information at 60 fps. Further, by moving the wide conversion lens 218 to the insertion position, it is possible to obtain an image having substantially the same viewing angle as when all pixels are read from the image sensor 120 even when partial reading is performed from the image sensor 120. ..

Further, when the wide conversion lens 218 is in the insertion position, the pixel information outside the predetermined range including the center of the image sensor 120 is not read out. Therefore, the lens system may be designed so that the image circle of the lens system including the lens 210 and the wide conversion lens 218 covers the vicinity of the center of the image sensor 120. Therefore, the lens system can be easily miniaturized.

FIG. 7 is a flowchart showing a procedure of processing executed by the image pickup control unit 110. This flowchart shows a procedure for changing the frame rate when the image pickup apparatus 100 shoots a moving image. Here, it is assumed that the frame rate is set to 30 fps and the wide conversion lens 218 is in the retracted position as the initial state of moving image shooting. Further, it is assumed that the readout mode of the image pickup control unit 110 is set to the all-pixel readout mode.

In S702, the image pickup control unit 110 determines whether or not the user has instructed to change the frame rate. If the frame rate change is not instructed, the process returns to S702. When the frame rate change is instructed, in S704, it is determined whether or not the changed frame rate exceeds a predetermined threshold value. The threshold value is the maximum frame rate at which the image sensor 120 can output the pixel information of the number of effective pixels of the image sensor 120. In the examples described in relation to FIGS. 5 and 6, the threshold value is 30 fps.

If it is determined in S704 that the changed frame rate does not exceed the threshold value, the process returns to S702. If the changed frame rate does not exceed the threshold value in the judgment of S704, if the changed frame rate exceeds the threshold value in the judgment of S704, the wide conversion lens 218 is inserted from the retracted position in S706. Switch to position. In S708, the pixel information read mode of the image pickup control unit 110 is switched to the partial read mode. For example, when the changed frame rate is 60 fps, as described in connection with FIG. 6, the image pickup control unit 110 switches to the operation mode in which the image sensor 120 partially reads out 4000 × 2666 pixels. Then, in S710, the image pickup control unit 110 switches to the instructed frame rate. Specifically, the image pickup control unit 110 determines the timing for reading out the pixel information from the image sensor 120 based on the instructed frame rate.

Subsequently, in S712, the image pickup control unit 110 determines whether or not to end the moving image shooting. For example, the image pickup control unit 110 determines whether or not the user has instructed to end the moving image shooting. If the movie shooting is not finished, the process returns to S702. When the moving image shooting is finished, the wide conversion lens 218 is moved to the retracted position in S714.

As described above, the image pickup control unit 110 automatically inserts the wide conversion lens 218 into the optical path according to the user's selection of the frame rate, and the operation mode of reading the pixel information from the image sensor 120. Is switched from the full pixel read mode to the partial read mode. As a result, even if the full pixel readout mode is switched to the partial readout mode, the viewing angle is not substantially switched. Therefore, the user can obtain a moving image captured at a desired frame rate and a desired viewing angle simply by selecting a frame rate without knowing the existence of the wide conversion lens 218.

As described above, the image pickup control unit 110 reduces the optical image formed on the image sensor 120 by inserting the wide conversion lens 218. When the magnification of the optical image can be reduced by moving the lens 210 in the optical axis direction, the image pickup control unit 110 moves the lens 210 in the optical axis direction instead of inserting the wide-angle conversion lens 218. By moving the lens 210, the angle of view of the lens 210 may be changed to a wide angle, and the range for reading an image from the image sensor 120 may be narrowed. In this case, the image pickup control unit 110 does not have to include the wide conversion lens 218 in the image pickup device 100 in this case.

In the above embodiment, the image pickup device 100 is an image pickup device mounted on the UAV 10. However, the image pickup device 100 does not have to be an image pickup device mounted on a moving body such as the UAV 10. For example, the image pickup device 100 may be an image pickup device supported by a handheld gimbal. Further, the image pickup device 100 may be an image pickup device that is not supported by the UAV 10 or the handheld gimbal. For example, the image pickup device 100 may be an image pickup device that can be handheld and supported by the user. The image pickup device 100 may be an image pickup device that is fixedly installed, such as a surveillance camera or the like.

FIG. 8 shows an example of a computer 1200 in which a plurality of aspects of the present invention may be embodied in whole or in part. The program installed on the computer 1200 can cause the computer 1200 to function as an operation associated with the device according to an embodiment of the present invention or as one or more "parts" of the device. Alternatively, the program may cause the computer 1200 to perform the operation or the one or more "parts". The program may cause the computer 1200 to perform a process according to an embodiment of the present invention or a step of the process. Such a program may be run by the CPU 1212 to cause the computer 1200 to perform certain operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212 and a RAM 1214, which are connected to each other by a host controller 1210. The computer 1200 also includes a communication interface 1222, an input / output unit, which are connected to the host controller 1210 via an input / output controller 1220. The computer 1200 also includes a ROM 1230. The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit.

The communication interface 1222 communicates with other electronic devices via the network. A hard disk drive may store programs and data used by the CPU 1212 in the computer 1200. The ROM 1230 stores in it a boot program or the like executed by the computer 1200 at the time of activation and / or a program depending on the hardware of the computer 1200. The program is provided via a computer-readable recording medium such as a CR-ROM, USB memory or IC card or network. The program is installed in RAM 1214, which is also an example of a computer-readable recording medium, or ROM 1230, and is executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and provides a link between the program and the various types of hardware resources described above. The device or method may be configured to implement the operation or processing of information according to the use of the computer 1200.

For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and performs communication processing with respect to the communication interface 1222 based on the processing described in the communication program. You may order. Under the control of the CPU 1212, the communication interface 1222 reads the transmission data stored in the transmission buffer area provided in the RAM 1214 or a recording medium such as a USB memory, and transmits the read transmission data to the network, or The received data received from the network is written to the reception buffer area or the like provided on the recording medium.

Further, the CPU 1212 makes the RAM 1214 read all or necessary parts of a file or a database stored in an external recording medium such as a USB memory, and executes various types of processing on the data on the RAM 1214. good. The CPU 1212 may then write back the processed data to an external recording medium.

Various types of information such as various types of programs, data, tables, and databases may be stored in recording media and processed. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval described in various parts of the present disclosure with respect to the data read from the RAM 1214. Various types of processing may be performed, including / replacement, etc., and the results are written back to the RAM 1214. Further, the CPU 1212 may search for information in a file, database, or the like in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 specifies the attribute value of the first attribute. Search for an entry that matches the condition from the plurality of entries, read the attribute value of the second attribute stored in the entry, and associate it with the first attribute that satisfies the predetermined condition. The attribute value of the second attribute obtained may be acquired.

The program or software module described above may be stored on a computer 1200 or in a computer readable storage medium near the computer 1200. Also, a recording medium such as a hard disk or RAM provided in a dedicated communication network or a server system connected to the Internet can be used as a computer readable storage medium, thereby allowing the program to be transferred to the computer 1200 over the network. offer.

The order of execution of each process such as operation, procedure, step, and step in the apparatus, system, program, and method shown in the claims, the specification, and the drawing is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

10 UAV
20 UAV main unit 30 UAV control unit 36 Communication interface 37 Memory 40 Propulsion unit 41 GPS receiver 42 Inertial measurement unit 43 Magnetic compass 44 Atmospheric pressure sensor 45 Temperature sensor 46 Humidity sensor 50 Gimbal 60 Imaging device 100 Imaging device 102 Imaging unit 110 Imaging control unit 120 Image sensor 130 Memory 200 Lens unit 210 Lens 212 Lens drive unit 214 Position sensor 218 Wide conversion lens 219 Lens drive unit 220 Lens control unit 222 Memory 300 Remote control device 310 Lens holder 320 Magnet 321 N pole 322 S pole 340 Coil 350 Coil 370 Rotating axis 600 Area 1200 Computer 1210 Host controller 1212 CPU
1214 RAM
1220 Input / Output Controller 1222 Communication Interface 1230 ROM

Claims (10)

Obtain information indicating the imaging rate and
When the image pickup rate exceeds a predetermined value, the angle of view of the image pickup lens is changed to a wide angle, and the range of reading an image from the image sensor that receives the light passing through the image pickup lens is narrowed. Equipped with a circuit
The image pickup lens includes a wide conversion lens that can be inserted into the subject optical path.
In the image pickup lens including the wide conversion lens, an image is read from the image sensor when the image circle of the image pickup lens has the wide conversion lens inserted in the subject optical path in the central portion of the image sensor. Designed to cover the above range
The circuit is
When the image pickup rate exceeds a predetermined value, the wide conversion lens is inserted into the subject optical path to narrow the range of reading an image from the image sensor.
Is configured as
Control device. The wide conversion lens is movable between an insertion position inserted into the subject optical path and a retracted position retracted from the subject optical path.
The circuit is
When the imaging rate transitions from a state in which the imaging rate does not exceed the predetermined value to a state in which the imaging rate exceeds the predetermined value, the wide conversion lens is moved from the retracted position to the insertion position, and the image is imaged from the image sensor. The control device according to claim 1 , wherein the control device is configured to narrow the reading range. The wide conversion lens can move between the insertion position inserted into the subject optical path and the retracted position retracted from the subject optical path.
The circuit is
When the imaging rate transitions from a state in which the imaging rate exceeds the predetermined value to a state in which the imaging rate does not exceed the predetermined value, the wide conversion lens is moved from the insertion position to the retracted position, and the image is imaged from the image sensor. The control device according to claim 2 , which is configured to widen the reading range. The circuit has a first read mode for reading an image from a first range of the image sensor and a second read mode for reading an image from a range 1 / n times the first range.
The wide conversion lens is designed to increase the focal length of the image pickup lens by 1 / n times when it is inserted into the subject optical path as compared with the case where it is not inserted into the subject optical path.
The circuit is
Claims 1 to 3 configured to insert the wide conversion lens into the subject optical path and read an image from the image sensor in the second read mode when the image pickup rate exceeds a predetermined value. The control device according to any one of the above. The circuit is
When the image pickup rate instructed by the user of the image pickup apparatus provided with the image sensor exceeds a predetermined value, the angle of view of the image pickup lens is changed to a wide angle, and the range for reading an image from the image sensor is narrowed. The control device according to any one of claims 1 to 4 . With the image sensor
An imaging device including the control device according to any one of claims 1 to 5 . With the image sensor
With the image pickup lens
An imaging device including the control device according to any one of claims 1 to 5 . A mobile body that moves with the image pickup apparatus according to claim 6 or 7. At the stage of acquiring information indicating the imaging rate,
When the image pickup rate exceeds a predetermined value, the angle of view of the image pickup lens is changed to a wide angle, and the range of reading an image from the image sensor that receives the light passing through the image pickup lens is narrowed .
The image pickup lens includes a wide conversion lens that can be inserted into the subject optical path.
In the image pickup lens including the wide conversion lens, an image is read from the image sensor when the image circle of the image pickup lens has the wide conversion lens inserted in the subject optical path in the central portion of the image sensor. Designed to cover the above range
When the image pickup rate exceeds a predetermined value, the step of changing the angle of view of the image pickup lens to a wide angle and narrowing the range of reading an image from the image sensor that receives the light passing through the image pickup lens is the image pickup. When the rate exceeds a predetermined value, the wide-angle conversion lens is inserted into the subject optical path to narrow the range of reading an image from the image sensor.
Control method. It ’s a program,
The procedure for acquiring information indicating the imaging rate and
When the image pickup rate exceeds a predetermined value, the computer is instructed to change the angle of view of the image pickup lens to a wide angle and narrow the range of reading an image from the image sensor that receives the light passing through the image pickup lens. Let it run
The image pickup lens includes a wide conversion lens that can be inserted into the subject optical path.
In the image pickup lens including the wide conversion lens, an image is read from the image sensor when the image circle of the image pickup lens has the wide conversion lens inserted in the subject optical path in the central portion of the image sensor. Designed to cover the above range
When the image pickup rate exceeds a predetermined value, the procedure for changing the angle of view of the image pickup lens to a wide angle and narrowing the range of reading an image from the image sensor that receives the light passing through the image pickup lens is the image pickup. It has a procedure of inserting the wide conversion lens into the subject optical path and narrowing the range of reading an image from the image sensor when the rate exceeds a predetermined value.
program.

Images