JP2023019114A - Imaging apparatus and imaging apparatus control method - Google Patents

Abstract

To provide an imaging apparatus which attains natural blurring.SOLUTION: The apparatus comprises: an image pick-up device which images a subject via lenses; a mechanism which shifts a portion of the lenses; a mechanism which shifts the image pick-up device; and means which performs control to output an image obtained by composing a plurality of images captured while driving the positions of the portion of the lenses and the image pick-up device in a predetermined relation.SELECTED DRAWING: Figure 3

Description

The present invention relates to an imaging apparatus and a control method for an imaging apparatus that expands the range of image quality control related to a captured image.

2. Description of the Related Art With the spread of digital cameras and portable communication devices, image capturing is widely performed. In addition to the personal enjoyment of images, means of widely disclosing images through electromagnetic media are often used, and opportunities to show images to others are increasing. Therefore, there is a growing demand for images with high image quality and unique image quality.

Blur is one of the important parameters that indicate image characteristics. In the method of adding blur to an image by image processing, which has been widely used in recent years, estimation processing is performed regarding the image area and amount to be filtered, but depending on the conditions, a processing error may occur, resulting in an optically unnatural image. could have become.

Japanese Patent Application Laid-Open No. 2002-200002 discloses a method of obtaining a larger amount of blur than that of an imaging lens by obtaining a distance distribution from parallax information and applying a blur filter according to the distance distribution to an image. there is

JP-A-9-181966

In the method of Patent Document 1, an accurate distance distribution can be obtained, such as a processing error at the boundary due to the low resolution of the distance distribution and an artifact due to the relationship between the angle at which the parallax information is acquired and the shape of the subject. image, resulting in an optically unnatural image.

In view of the above problems, an object of the present invention is to provide an imaging apparatus capable of adding natural blurring to a captured image.

In order to solve the above-described problems, an image pickup apparatus of the present invention includes an image pickup element for picking up an image of a subject through a lens, a mechanism for shifting a part of the lens, and a mechanism for shifting the image pickup apparatus. and means for performing control for outputting an image obtained by synthesizing a plurality of captured images while driving the imaging device in a predetermined relationship.

According to the present invention, it is possible to provide an imaging apparatus capable of adding natural blurring to a captured image.

FIG. 4 is an explanatory diagram of pupil positions and blurring; FIG. 10 is an explanatory diagram of movement of an image due to movement of the image stabilizing optical system; FIG. 10 is an explanatory diagram of enlarging the pupil by moving the anti-vibration optical system; FIG. 10 is an explanatory diagram when the subject distance is finite; FIG. 10 is an explanatory diagram of an example of the movement direction of the image stabilizing optical system and the movement direction of the imaging device; 4 is a flowchart for explaining operations in the first embodiment; FIG. 10 is an explanatory diagram of deformation of an image due to movement of the image stabilizing optical system; 9 is a flow chart for explaining the operation in the second embodiment; FIG. 10 is an explanatory diagram of possible shift amounts due to the shift of the image stabilizing optical system and the shift of the imaging element; FIG. 10 is an explanatory diagram of possible shift amounts due to the shift of the image stabilizing optical system and the shift of the imaging element; It is a flow chart explaining the operation in the 3rd example.

[First Embodiment]
Preferred embodiments of the present invention are described in detail below with reference to the drawings.

FIG. 1 is an explanatory diagram of pupil positions and blurring. FIG. 1A shows a state in which the image sensors are capturing images of subjects 101 and 102, in which the lens position is 103 and the image sensor is at position 105, and the lens position is 104 and the image sensor is at 106. The state in which the image is captured at the position is shown from above. FIG. 1B shows an image captured at the above position. An image 107 is obtained when the lens position is 103 and the imaging device is positioned at 105, and an image 108 is obtained when the lens position is 104 and the imaging device is positioned at 106. FIG.

The lens and the imaging element are arranged in a shifted manner so that the position of the subject 101 within the screen does not change. In the image 109 obtained by adding the images captured under the above two conditions, the position of the object 101 remains unchanged. The object 101 and the object 102 at different distances from the imaging device are added at different positions, and if a plurality of images taken while moving the lens position from 103 to 104 are added, the image will be blurred.

This state means that the lens has been moved from position 103 to position 104 to obtain an image formed by a virtual pupil 110 .

In FIG. 1, the formation of the virtual pupil was explained by moving one lens. The formation of a virtual pupil in the present invention is shown using FIG. A state in which an object 201 is imaged with a lens having a shiftable optical system is shown. As a structure for suppressing camera shake, a structure in which a part of the optical system is shifted is adopted in many lenses. FIG. 2 illustrates a system in which a part of the optical system closer to the imaging device than the front lens 202 can be shifted to a position 203 or 204 . A pupil position 205 is indicated when the shiftable optical system (hereinafter referred to as a shift optical system) is at the position 203, and a pupil position 206 is indicated when the shift optical system is at the position 204. FIG. In other words, it indicates that the position of the pupil also moves with the movement of the shift optical system.

The structure for suppressing camera shake suppresses camera shake by moving an image so as to offset the movement of the image on the imaging surface due to camera shake during imaging. If only the shift optical system is moved without camera shake, the image will move along with the movement of the optical system. In the present invention, this image movement is offset by movement of the imaging device.

Specifically, when the shift optical system moves from the neutral state (where the shift amount is zero) to the state of (A), the image moves upward. The imaging element is moved to the position 207 so as to move upward to offset the movement of the image, and the image 209 captured in that state has the same angle of view as the image captured in the neutral state. On the other hand, when the shift optical system moves from the neutral state to the state of (B), the image moves downward, contrary to (A). Also in this case, the captured image 210 has the same angle of view as in the neutral state.

A comparison of FIGS. 1 and 2 reveals that the shift optical system movement in FIG. 2 is similar to the lens movement in FIG. Since the positional relationship between the object 201 and the imaging device does not change, the positional relationship between the front lens and the object remains unchanged between (A) and (B). On the other hand, the movement of the shift optical system moves the pupil.

The enlargement of the blur amount according to the present invention will be described with reference to FIG. FIG. 2 shows that the position of the pupil can be changed by moving the shift optical system without moving the entire camera.

FIG. 3A shows a state in which continuous imaging is performed while the shift optical system is moved from position 301 to position 302 . The imaging elements are also moved to positions 306 and 307 in order to offset the movement of the image due to the movement of the shift optical system. By adding a plurality of images captured while moving the shift optical system, an image in which the pupil region is virtually expanded to the region 304 can be obtained. This state is optically equivalent to an image captured on an image sensor 309 on the optical axis by a pupil 308 of an optical system having a large pupil diameter, as shown in FIG. 3B.

Image addition may be performed by adding image data consisting of a two-dimensional array of numerical values, or by adding in the state of electrons on the photodiode of the image pickup device during the exposure time.

The amount of movement of the shift optical system and the amount of movement of the imaging device are set as follows. The amount of movement Sois of the image due to movement of the shift optical system is
Sois=f·tan θ Formula (1)
f is the focal length, and .theta. of the shifted image is the deflection angle.

The argument θ is a function of the shift amount Scoe,
θ=g(Scoe) Equation (2)
can be written as g differs depending on the lens.

On the other hand, the movement of the image on the image sensor due to the movement of the image sensor is the movement amount Siis of the image sensor itself, and the relationship for canceling the movement of the image due to the movement of the shift optical system by the movement of the image sensor is
Siis = Sois
= f tan θ
= f tan (g (Scoe)) Equation (3)
becomes.

If the movement amount of the shift optical system and the function g of the deflection angle are obtained in advance, the appropriate movement amount of the image sensor when the shift optical system is moved by a predetermined amount can be obtained from equation (3). Become.

The above is the amount of movement of the shift optical system and the amount of movement of the imaging device when the subject is at infinity. When the distance between the subject and the imaging device is short, the movement of the shift optical system changes the angle at which the subject is viewed.

The amount of movement of the shift optical system and the amount of movement of the imaging element when the subject is at a finite distance will be described with reference to FIG.

FIG. 4A shows a state in which the object 201 is at infinity. As the shift optical system 401 moves, the pupil of the optical system also moves to the position of 402 . 403 represents the size of a virtual pupil formed by moving the position of the pupil within the plane. Since the subject 201 is at infinity, the angle of the light beam entering the optical system does not change even if the shift optical system moves. Due to the movement of the shift optical system, it moves in the direction of the deflection angle θ according to the above equation (2).

FIG. 4B shows the case where the subject 201 is at a distance d from the principal point of the optical system. The angle at which the subject is viewed from the optical system changes as the shift optical system moves. The change u in the angle at which the subject is viewed due to the movement of the shift optical system is

is. When the incident angle of the shift optical system is changed by u, the change in the output angle is γu, where γ is the angular magnification of the shift optical system.

When the light enters the shift optical system perpendicularly, the exit angle is .theta., but when the object distance is d, it changes from .theta. by .gamma.u. In an optical system in which the movement direction of the shift optical system and the deflection angle are opposite directions, this change works in the direction of reducing the required amount of movement of the image pickup device. The required movement Siis2 is

becomes.

A plurality of images are captured by moving the shift optical system and the imaging element while maintaining the relationship of the above formula (3) or formula (5).

The amount of movement of the shift optical system and the imaging device in the x direction or the y direction when the optical axis is the z axis has been described above. The shift optical system and the imaging element move within the xy plane while maintaining the above-described relationship of the movement amount. The positional relationship of movement within the xy plane will be described with reference to FIG.

5(a), 5(b) and 5(c) show the embodiment of the present invention viewed from the z direction parallel to the optical axis when using a shift optical system in which the movement direction and the image movement direction are opposite. indicate.

Depending on the time, the shift optical system and the imaging device move to various positions within the xy plane as shown in (a), (b), and (c). Each relative relationship is arranged at an angle that is point symmetrical with respect to the optical axis of the optical system of the imaging device. The distance from each optical axis is Sois for the shift optical system, and Siis or Siis2 for the imaging element.

The locus of movement of the shift optical system in the xy plane may be circular at the maximum shiftable position, spiral while changing the radius, or polygonal. Furthermore, the trajectory may be arbitrarily set by the photographer instead of the trajectory set in advance.

In the present invention, the operation is performed so that the position of the subject does not change between a plurality of captured images. However, if an error occurs in operation, a desired output image can be obtained by synthesizing (adding) the images by shifting the positions during image addition. In this case, an area in which the number of synthesized images (the number of synthesized images) is smaller than that of a reference area (an area mainly including a subject, such as an area having the largest area) is generated in the image after synthesis. A region with a small number of composites may be removed by trimming, an output image may be generated with a small number of composites, or a low-pass filter may be applied to a region with a small (different) number of composites.

Among a plurality of captured images, an inappropriate image may be acquired, for example, when a person closes his or her eyes. By recognizing these images and excluding them from addition targets, the quality of the output image can be improved.

FIG. 5D is a block diagram showing a configuration example of an imaging device 500 according to this embodiment. First, the configuration of the camera housing section 501 will be described. A luminous flux from a subject passes through the imaging optical system 3 of the lens unit 502, forms an image on the imaging surface of the imaging device 6, and is received. The imaging optical system 3 includes the above-described shift optical system, and the drive unit 14 also includes a drive mechanism for shifting the shift optical system in a direction different from the optical axis (eg, perpendicular direction). In this embodiment, the drive unit 14 is driven and controlled by the system control unit 5, but the drive unit 14 itself may be provided with a drive control circuit.

Microlenses are arranged in a grid pattern on the surface of the imaging device 6 . A large number of microlenses constitute a microlens array 20 and function as pupil dividing means. Information related to the distance of the object (distance information) is obtained by acquiring signals from a plurality of photoelectric conversion units corresponding to each microlens and performing correlation calculations between the signals. For example, there is a distance image that represents the distribution of distance information of a subject in a captured image. The information of the distance image is two-dimensional information in which the value indicated by each pixel represents the distance information of the subject existing in the area of the captured image corresponding to the pixel. Examples of depth information representing the depth of an object in the depth direction include an image shift amount map and a defocus amount map. The image shift amount map is calculated from a plurality of viewpoint images from different viewpoints, and the defocus amount map is calculated by multiplying the image shift amount by a predetermined conversion coefficient. A distance map obtained by converting the defocus amount into object distance information represents the distance distribution from the imaging device to the object.

In this embodiment, an example of subject distance measurement based on the imaging plane phase difference detection method is shown. can be obtained.

In addition, since the output of the imaging device 6 provides signals representing the evaluation amount for focus adjustment and the exposure amount, AF control and AE (automatic exposure) control of the imaging optical system 3 are possible based on these signals. .

The image processing unit 7 has therein an A (analog)/D (digital) converter, a white balance circuit, a gamma correction circuit, an interpolation calculation circuit, etc., and is capable of generating image data for recording. Further, the image processing unit 7 adds a plurality of acquired images, generates a synthesized image, and outputs the synthesized image to the memory unit 8 .

The memory unit 8 includes a storage unit and a processing circuit necessary for storing image data and the like. The memory unit 8 performs compression processing and decompression processing of data such as images, moving images, audio, etc. by a predetermined method. The image data stored in the memory section 8 is read out and output to the display section 9 or the recording/playback section 10 . A display unit 9 includes an LCD (liquid crystal display device) or the like, and displays images according to control commands from the system control unit 5 . Further, the display unit 9 displays a predetermined message on the screen to notify the user.

The recording/reproducing unit 10 records image data and the like on a predetermined recording medium according to a control command from the system control unit 5, or reads and reproduces data from the recording medium. The predetermined recording medium is, for example, a semiconductor memory device or the like that can be used by being attached to the camera body.

A system control unit 5 is provided with a CPU (Central Processing Unit) and is a central part that controls each component of the imaging system. The system control unit 5 generates a timing signal or the like for imaging and outputs it to each unit, and also controls each component of the imaging system, the image processing system, and the recording/reproducing system in response to the operation instruction signal. The system control unit 5 performs processing for displaying or recording image data processed by the image processing unit 7, processing for transmitting the data to an external device using an output device, and the like.

The operation detection unit 11 detects a user's operation performed using an operation switch, a touch panel, or the like, and outputs an operation instruction signal to the system control unit 5 . For example, the operation detection unit 11 detects a user's imaging operation instruction and notifies the system control unit 5 of it. The system control unit 5 controls driving of the imaging device 6, operation of the image processing unit 7, compression processing by the memory unit 8, and the like, and controls display of images and information on the screen of the display unit 9. FIG.

The position/orientation detection unit 12 includes an angular velocity sensor, an acceleration sensor, and the like, detects the position and orientation of the imaging device 500 , and outputs a detection signal to the system control unit 5 . Based on the detection signal, the system control unit 5 acquires the position information, the amount of movement, and the orientation information of the imaging device 500 by a known method.

The light emitting unit 13 has a light source for illuminating the subject, and the amount of light emitted is controlled by a control signal from the system control unit 5 . The system control unit 5 irradiates the subject with illumination light from the light source of the light emitting unit 13 as necessary. The details of the light emission control of the light source will be described later in Examples.

Next, the configuration of the lens driving unit 502 will be described. The lens unit 502 includes a drive unit 14 and a position detection unit 15. FIG. The imaging optical system 3 of the lens unit 502 is made up of optical members such as a lens group and a diaphragm, and the optical axis 4 is indicated by a one-dot chain line in FIG.

The driving section 14 drives the lens section 503 in the r direction and the .theta. direction in FIG. The drive unit 14 shifts the lens unit 502 by means of a drive mechanism in the r direction and a drive mechanism in the θ direction. Each drive mechanism is not limited to a specific configuration, and various mechanisms can be employed. The position detection unit 15 detects the position (r, θ in FIG. 4 ) of the lens unit 502 and outputs a position detection signal to the drive unit 14 . The drive unit 14 performs feedback drive to reduce the deviation between the position detection signal and the drive target signal.

The operation of the imaging device 500 in this embodiment will be described with reference to FIG. Each step is executed by each unit according to the system control unit 5 in FIG. 5 or an instruction from the system control unit 5 .

First, in S<b>601 , the system control unit 5 accepts selection of blur enhancement mode by the photographer. In S602, the system control unit 5 receives the setting of the desired F-number by the photographer. In S603, the system control unit 5 receives composition setting by the photographer. When the system control unit 5 determines in S604 that the shutter has been half-pressed by the operation detection unit 11, the distance to the subject is measured in S605, and the driving unit 14 drives focusing in S606. In S607, driving of the shift optical system included in the imaging optical system 3 is started, and in S608, shift driving of the imaging device 6 is started. S606 to S608 may be performed at the same time, or their order may be changed. If the system control unit 5 determines in S609 that the shutter has been fully pressed, an image is acquired in S610, and image acquisition is repeated until the system control unit 5 determines that a predetermined time has elapsed. The predetermined time is related to the desired smoothness of the blurred image. Even when smoothness is required, it can be achieved in a short time. Further, the longer the predetermined time, the smoother the blurring of the output image. The predetermined time may be set in advance by the photographer, or may be determined automatically by the system control unit 5 from the imaging conditions. If it is determined that the camera has reached the predetermined time, the shift driving of the shift optical system and the image sensor is stopped in S612. In S613, a plurality of captured images are aligned and added, and output in S614.

As described above, in the present embodiment, a blurred image is obtained by capturing, adding, and outputting a plurality of images while moving the imaging device according to the movement of the shift optical system. According to this embodiment, there is provided an image pickup apparatus that can obtain a blur amount larger than the amount of blur that the image pickup apparatus can achieve in normal image pickup without moving the camera or reducing the number of pixels. can do.

[Second embodiment]
A second embodiment will be described with reference to FIG. In FIG. 7A, 709 shows a state in which an object 701 is imaged on the imaging element through an optical system 702 including the shift optical system when the shift optical system is at the origin position with a shift amount of 0 (703). In FIG. 7B, 710 indicates a state in which the subject is imaged on the image sensor 707 through the optical system including the shift optical system when the shift optical system is shifted to the position 704 . The imaging element is moved from position 707 to position 708 in order to offset the movement of the image due to the movement of the shift optics. 705 and 706 represent the positions of the pupils in each case. Although the image moves downward in FIG. 7 due to the shift of the shift optical system, the amount of movement of the image on the imaging device when the shift optical system moves a certain distance differs depending on the position within the screen. Generally, in the optical design of the shift optical system, the amount of movement of the image by the shift optical system is designed so that there is no difference within the screen, but the amount may not be negligible depending on the lens. Therefore, in Example 1, the movement amount Sois of the shift optical system is transformed as follows, where x and y are positions from the center of the screen.
= f tan θ
= f h (x, y) tan (g (Scoe)) Equation (6)
where Scoe is the amount of movement of the shift optical system, x and y are the distances from the center of the screen to the subject on the imaging device, and function g represents the deflection angle of the image at the center of the screen with respect to the movement of the shift optical system. function, and function h is a function that expresses the ratio of the amount of change in declination from the center of the screen at a location deviated from the center of the screen.

In order to offset the amount of movement of the image by the shift optical system and the amount of movement of the subject image by the movement of the imaging device, they are made equal.
Siis=f·h(x, y)·tan(g(Scoe)) Equation (7)
becomes.

The operation of the imaging device 500 according to this embodiment will be described with reference to FIG. Each step is executed by each unit according to the system control unit 5 in FIG. 5 or an instruction from the system control unit 5 .

First, in S<b>801 , the system control unit 5 accepts selection of blur enhancement mode by the photographer. In S802, the system control unit 5 receives the setting of the desired F-number by the photographer. In S<b>803 , the system control unit 5 receives composition setting by the photographer. When the system control unit 5 determines in S804 that the shutter has been half-pressed by the operation detection unit 11, in S805 the position of the subject within the imaging screen is specified. The subject position may be specified by the photographer, may be automatically recognized by the camera, or may be linked to the frame position for distance measurement. In S806, the system control unit 5 calculates the shift amount of the subject image on the imaging plane from the function of the subject position obtained in S805 and the amount of movement from the center of the screen within the screen prepared in advance. In S807, the distance to the object is measured, and in S808, the driving unit 14 performs focusing driving. The order of the set of S805 and S806 and the set of S807 and S808 may be reversed or may be performed simultaneously. In S809, driving of the shift optical system is started, and in S810, shift driving of the imaging element is started according to the shift amount obtained in S806. S809 to S810 may be performed at the same time, or their order may be changed. If the system control unit 5 determines that the shutter has been fully pressed, an image is acquired in S812, and image acquisition is repeated until it is determined that the imaging device has reached a predetermined time. The predetermined time may be set in advance by the photographer, or may be determined automatically by the camera based on the imaging conditions. If it is determined that the camera has reached the predetermined time, the shift driving of the shift optical system and the image sensor is stopped in S814. In S815, a plurality of captured images are aligned and added, and output in S816.

According to this embodiment, it is possible to obtain an amount of bokeh that is larger than the amount of bokeh that occurs in normal imaging without moving the camera, even with a lens in which the amount of image movement due to the movement of the shift optical system differs within the angle of view. It is possible to provide an imaging device with a

[Third embodiment]
A third embodiment will be described with reference to FIGS. In FIG. 9, the amount of shift by which the image can move on the imaging plane is the shift amount of the image due to the focal length in the case of optical image stabilization using a shift optical system and in the case of image sensor image stabilization by movement of the image sensor itself. It shows a comparison of possible quantities. Since the image pickup device itself is moved in the image pickup device image stabilization, the possible movement amount of the image pickup device is the possible image movement amount, which is constant regardless of the focal length of the lens as indicated by the solid line in FIG.

On the other hand, for optical vibration reduction, the possible shift amount can be set for each lens. The amount of movement of the image when the amount of angular blurring of the camera is .theta. is f.tan.theta., and if the same amount of blurring is to be handled, the longer the focal length, the larger the amount of movement must be. Therefore, it is designed so that the amount of possible movement of the image due to the optical vibration reduction increases in accordance with the focal length, as indicated by the dotted line in FIG.

The possible image movement amounts for optical image stabilization and image sensor image stabilization are reversed at the focal length 901 in FIG. The left side of 901 is the state (A) in which the possible shift amount is larger with image stabilization, and the right side is the state (B) in which the optical stabilization is larger.

In (A) and (B) of FIG. 10, their magnitude relationship is shown. In the case of (A), there is a margin of the width indicated by 1001 on the image pickup device vibration reduction side, and in the case of (B), there is a margin of the width indicated by 1002 on the optical vibration reduction side.

Therefore, in this embodiment, of the possible movement amounts of the images of the respective elements, the margins indicated by 1001 and 1002 are used for camera shake correction, and the remaining portions are used for blur enhancement.

The operation of the imaging device 500 according to this embodiment will be described with reference to FIG. Each step is executed by each unit according to the system control unit 5 in FIG. 5 or an instruction from the system control unit 5 .

First, in S<b>1101 , the system control unit 5 accepts selection of blur enhancement mode by the photographer. In imaging S1102, the system control unit 5 compares the amount of movement of the image by the shift optical system of the imaging lens and the amount of movement of the image by the imaging device, and calculates the feasible F value determined by the smaller amount of movement. In S1103, the difference between the amount of image movement by the shift optical system and the amount of image movement by the imaging element is calculated as the amount to be assigned to the camera shake correction function. In S1104, the photographer sets a desired F-number within the range of the F-number obtained in S1102. In S1105, the composition is determined by the photographer. If the system control unit 5 determines in S1106 that the shutter has been half-pressed, the distance to the object is measured in S1107, and the driving unit 14 drives the focusing in S1108. In S1109, the drive unit 14 starts driving the shift optical system, and in S1110, shift driving of the imaging element 6 is started. S1107 to S1110 may be performed at the same time, or their order may be changed. If the system control unit 5 determines in S1111 that the shutter has been fully pressed, an image is acquired in S1112, and image acquisition is repeated until the system control unit 5 determines that a predetermined time has elapsed. The predetermined time may be set in advance by the photographer, or may be determined automatically by the camera based on the imaging conditions. If it is determined that the camera has reached the predetermined time, the shift driving of the shift optical system and the image sensor is stopped in S1114. In S1115, a plurality of captured images are aligned and added, and output in S1116.

In this embodiment, it is possible to obtain an amount of blur that is larger than the amount of blur that can be achieved by an imaging device when performing normal imaging, without moving the camera and without reducing the number of pixels, while performing camera shake correction. An imaging device can be provided.

According to each of the above-described embodiments, it is possible to extend the blur amount of the lens without making it look unnatural by using existing lenses and camera mechanisms, which contributes to improving the degree of freedom in image quality control. .

Although preferred embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and combinations of configurations shown in the respective embodiments and various modifications and changes are possible.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which software program codes describing procedures for realizing the functions of the above-described embodiments are recorded is supplied to the system or device. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code stored in the storage medium.

In this case, the program code itself read from the storage medium implements the novel functions of the present invention, and the storage medium and program storing the program code constitute the present invention.

Storage media for supplying program codes include, for example, flexible disks, hard disks, optical disks, and magneto-optical disks. Also, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, non-volatile memory card, ROM, etc. can be used.

Moreover, the functions of the above-described embodiments are realized by making the program code read by the computer executable. Furthermore, based on the instructions of the program code, the OS (operating system) running on the computer performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments. is also included.

It also includes the following cases: First, a program code read from a storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

In addition, the present invention is not limited to equipment such as a digital camera whose main purpose is photography, but also includes a mobile phone, a personal computer (laptop type, desktop type, tablet type, etc.), a game machine, etc., which incorporates or is externally connected to an imaging device. applicable to any device that Accordingly, the term "imaging device" in this specification is intended to encompass any electronic device with an imaging function.

101, 102, 201, 701 Subject 103, 104, 202, 702 Lens 105, 106, 207, 208, 306, 307, 404, 502, 707, 708 Image sensor 203, 204, 301, 302, 401, 501, 703 , 704 shift optical system 709, 710 subject image on image sensor

Claims (10)

an imaging device that receives a light flux incident through an imaging optical system having a shift optical system that shifts in a direction different from the optical axis;
drive control means for performing control to drive the imaging optical system and the imaging device in a direction different from the optical axis;
and a control means for controlling imaging by the image pickup device while controlling the image pickup device and the drive control means, wherein the control means sets the moving direction of the shift optical system and the moving direction of the image pickup device in different directions. An image pickup apparatus characterized by controlling the image pickup of the image pickup element so as to perform exposure while moving the image pickup device. 2. The imaging apparatus according to claim 1, wherein said control means causes the moving direction of said shift optical system and the moving direction of said imaging element to be opposite directions. 3. The imaging apparatus according to claim 1, further comprising synthesizing means for synthesizing a plurality of images captured by said imaging device. 4. The imaging apparatus according to claim 3, further comprising output means for trimming and outputting an area having a different number of images synthesized with respect to a reference area from the image synthesized by said synthesizing means. 5. The imaging apparatus according to claim 4, further comprising an output unit that applies a low-pass filter to an area having a different number of images synthesized with respect to a reference area from the images synthesized by the synthesizing unit, and outputs the result. The control means controls the movable amount of the shift optical system in the direction different from the optical axis and the movable amount of the imaging element in the direction different from the optical axis, whichever is larger. 6. The imaging apparatus according to any one of claims 1 to 5, wherein the imaging device and the drive control means are controlled by assigning it to a movement amount for camera shake correction. The control means recognizes the position of an object in an image obtained by imaging with the image pickup device, and controls the amount of movement of the shift optical system and the amount of movement of the image pickup device by the drive control means according to the position of the object. 7. The imaging apparatus according to any one of claims 1 to 6, characterized in that: an imaging device that receives a light flux incident through an imaging optical system having a shift optical system that shifts in a direction different from the optical axis; and drive control means that controls driving of the imaging optical system and the imaging device. A control method for an imaging device comprising:
A control method for an imaging apparatus, comprising a control step of controlling imaging by the imaging device so as to perform exposure while moving the shifting optical system and the imaging device in different directions. 9. A computer-executable program describing the procedure of the control method for the imaging apparatus according to claim 8. A computer-readable storage medium storing a program for causing a computer to execute each step of the imaging apparatus control method according to claim 8 .

Images