JP5945389B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

The present invention relates to an image capturing apparatus , an image capturing method , and a program that have an image sensor that is partially used as a focus detection element and that captures a moving image by continuously capturing images.

  2. Description of the Related Art Conventionally, an image sensor that uses part of the image sensor as a focus detection photodiode is known (see Patent Document 1). In the image sensor disclosed in Patent Document 1, each pixel in a region that is also used for focus detection is divided into two pixels, and these two pixels are respectively connected to different exit pupils of the photographing lens via an on-chip microlens. A subject luminous flux that has passed through a region (hereinafter referred to as “pupil region”) is received. Then, the focus state is detected based on the phase difference between the two images represented by the two types of image signals that have passed through the different pupil regions.

Japanese Patent No. 3592147

  The image sensor disclosed in Patent Document 1 is excellent in that one image sensor can be used for both imaging and focus detection. However, since the image sensor disclosed in Patent Document 1 replaces a part of pixels constituting the image with a pixel for distance measurement (AF pixel), the image of this AF pixel portion is lost, and the image Will deteriorate. In order to prevent image degradation, the AF pixels may be reduced. However, if the AF pixels are reduced, the AF accuracy cannot be maintained.

  When such a defective pixel is present, the portion looks like noise when displaying a still image, a moving image, or a live view. In particular, when an enlarged image is displayed during live view display or when an enlarged recording is performed during moving image shooting, a much larger number of pixels than the conventional defective pixels become AF pixels. Will stand out. A method of correcting the defective pixel portion by an interpolation method or the like is conceivable. However, when correction such as interpolation is performed, the image quality at that position always deteriorates.

The present invention has been made in view of such circumstances, and a photographing apparatus and photographing capable of obtaining sufficient image quality even in the case of having an image sensor that also serves as a focus detection element. It is an object to provide a method and a program .

In order to achieve the above object, a photographic device according to a first aspect of the present invention is a photographic device capable of capturing a moving image, having an imaging element having focus detection pixels, converting a subject image into image data, and outputting the subject image. And a moving amount calculating means for calculating a relative moving amount of the image sensor, and a second frame that is captured after the first frame is captured after the first frame is imaged by the image sensor. By the change means for changing the relative position of the subject image and the image sensor based on the movement amount calculated by the movement amount calculation means and before the start of imaging of the frame, and smoothing processing between frames and interpolation means for performing interpolation processing in the image interpolation processing or one frame, a display means for performing a live view display based on the image data output from the image pickup device, the upper Enlarged image operation means for instructing enlarged display in performing live view display, and image cutout means for changing the cutout position of the image data output from the image sensor according to the movement amount When the enlargement display is instructed, the relative position between the subject image and the image sensor is changed by the changing unit based on the movement amount, and the cutout position of the image data is changed according to the movement amount. Based on the result of the interpolation process performed by the interpolation unit, the display unit performs live view display on the image data that has been changed by the cutout unit and the cutout position has been changed.

In the photographing apparatus according to a second aspect of the present invention, in the first aspect , the second frame is an image displayed on the display means next to the image of the first frame.
According to a third aspect of the present invention, there is provided a photographing apparatus according to the first aspect , further comprising recording means for recording a moving image based on the image data output from the image sensor, and the output from the image sensor. Image cut-out means for changing the cut-out position of the image data according to the movement amount, and when the recording of the moving image is instructed, the subject image and the image sensor The relative position is changed by the changing unit, the cutout position of the image data is changed by the image cutout unit according to the movement amount, and the image data in which the cutout position is changed is recorded by the recording unit.
In a photographing apparatus according to a fourth invention, in the first invention, the second frame is an image recorded next to the image of the first frame in the recording means.

According to a fifth aspect of the present invention, in the photographing apparatus according to the first aspect , the changing means moves the imaging element based on the movement amount or moves a lens for forming the subject image. Move based on.
According to a sixth aspect of the present invention, in the first aspect, the changing means changes the relative position between the subject image and the image sensor when performing the enlarged display or when the enlarged display is instructed. Run.

In the photographing apparatus according to a seventh aspect based on the first aspect , the movement amount calculation means randomly or regularly changes the relative position between the subject image and the imaging element.
According to an eighth aspect of the present invention, in the first aspect , the movement amount calculating means calculates the movement amount according to an image reduction ratio of image data output from the image sensor.
According to a ninth aspect of the present invention, in the first aspect , the movement amount calculating means changes the movement amount only when displaying or recording larger than a predetermined reduction ratio, and the predetermined reduction ratio. In the case of display or recording smaller than this, the movement amount is set to zero.

According to a tenth aspect of the present invention, there is provided a photographing method according to a tenth aspect of the present invention, wherein a moving amount is calculated according to a reduction ratio of image data in a photographing method for photographing a moving image by continuously photographing using an image sensor having an imager AF pixel. Before completing the imaging of the next frame after imaging of one frame is completed, the relative position of the subject image and the imaging element is changed according to the amount of movement, and interpolation processing by smoothing between frames, Alternatively, interpolation processing is performed within an image of one frame, live view display is performed based on the image data output from the image sensor, and when the enlargement display is instructed when performing the live view display, the imaging is performed. The cutout position of the image data output from the element is changed according to the movement amount, the cutout position of the image data is changed according to the movement amount, and this cutout is performed. To live view display on the display unit based on the image data obtained by changing the position to a result of the interpolation process by Interpolation means.

A program according to an eleventh aspect of the present invention is a program for executing a computer of a photographing apparatus that uses a photographing element having an imager AF pixel and continuously captures images to shoot a moving image in accordance with a reduction rate of image data. A step of calculating a movement amount, a step of changing a relative position of the subject image and the imaging element in accordance with the movement amount before imaging of the next frame is started after imaging of one frame is completed; Performing interpolation processing by smoothing between frames, or performing interpolation processing within an image of one frame, performing live view display based on the image data output from the image sensor, and performing the live view display. When the enlargement display is instructed, the image data output from the image sensor is cut out according to the movement amount. Change the location, in accordance with the amount of movement is changed the cutout position of the image data, and live view display on the display means based on the result of interpolation processing by Interpolation means image data obtained by changing the extraction position And causing the computer to execute steps.

According to the present invention, it is possible to obtain sufficient image quality because the relative position of the subject image and the image sensor is changed between frames even when an image sensor that partially serves as a focus detection element is provided. Imaging device , imaging method , and program can be provided.

1 is a block diagram illustrating an overall configuration mainly including an electric system of a digital camera according to a first embodiment of the present invention. It is a top view which shows the arrangement | sequence of the filter of the image pick-up element of the digital camera concerning 1st Embodiment of this invention. It is a figure which shows the structure of the image pick-up element of the digital camera concerning 1st Embodiment of this invention, (a) is a top view of an image pick-up element, (b) is BB sectional drawing of an image pick-up element. FIG. 2 is a plan view showing a configuration of an image sensor position moving unit in the digital camera according to the first embodiment of the present invention. 4 is a timing chart illustrating an example of synchronous communication between an interchangeable lens and a camera body in the digital camera according to the first embodiment of the present invention. It is a flowchart which shows the main flow of the digital camera concerning 1st Embodiment of this invention. It is a flowchart which shows the moving image and live view imaging | photography of the digital camera concerning 1st Embodiment of this invention. It is a figure explaining the calculation method of a movement amount in the digital camera concerning 1st Embodiment of this invention. It is a flowchart which shows the image process for moving images of the digital camera concerning 1st Embodiment of this invention. It is a flowchart which shows the process of NR between frames of the digital camera concerning 1st Embodiment of this invention. It is a block diagram which shows the whole structure mainly consisting of the electrical system of the digital camera concerning 2nd Embodiment of this invention.

  Hereinafter, preferred embodiments using a digital camera to which the present invention is applied will be described with reference to the drawings. FIG. 1 is a block diagram mainly showing an electrical configuration of the digital camera according to the first embodiment of the present invention. This digital camera includes a camera body 100 and an interchangeable lens 200 that can be attached to and detached from the camera body 100. In the present embodiment, the photographing lens is an interchangeable lens type, but the present invention is not limited to this, and a digital camera of a type in which the photographing lens is fixed to the camera body may of course be used.

  The interchangeable lens 200 includes a photographing lens 201, a diaphragm 203, a driver 205, a microcomputer 207, a photo interrupter (hereinafter referred to as PI) 208, a flash memory 209, and a manual focus ring (hereinafter referred to as MF ring) 210. An interface (hereinafter referred to as I / F) 300 is provided between the camera body 100 and the camera body 100 described later.

  The taking lens 201 is composed of a plurality of optical lenses for forming a subject image, and is a single focus lens or a zoom lens. A diaphragm 203 is arranged behind the optical axis of the photographing lens 201. The diaphragm 203 has a variable aperture, and restricts the amount of light of the subject light beam that has passed through the photographing lens 201. The photographing lens 201 can be moved in the optical axis direction by a driver 205, and the focus position of the photographing lens 201 is controlled based on a control signal from the microcomputer 207. In the case of a zoom lens, the focal length is also controlled. Is done. The driver 205 also controls the aperture of the diaphragm 203.

  The microcomputer 207 connected to the driver 205 is connected to the I / F 300, the PI 208, and the flash memory 209. The microcomputer 207 operates according to a program stored in the flash memory 209, communicates with a microcomputer 121 in the camera body 100 described later, and controls the interchangeable lens 200 based on a control signal from the microcomputer 121. Do.

  The flash memory 209 stores various information such as optical characteristics and adjustment values of the interchangeable lens 200 in addition to the above-described program. The I / F 300 is an interface for performing communication between the microcomputer 207 in the interchangeable lens 200 and the microcomputer 121 in the camera body 100. Synchronous communication between the microcomputer 207 and the microcomputer 121 performed via the I / F 300 will be described later with reference to FIG.

  An MF ring 210 is rotatably provided on the outer periphery of the interchangeable lens 200. The MF ring 210 is an operation member that is operated when the photographer manually focuses. The PI 208 generates a pulse signal according to the rotation of the MF ring 210 and outputs the pulse signal to the macro computer 207. The macro computer 207 detects the rotation direction, rotation speed, rotation amount, and the like of the MF ring 210 based on the pulse signal from the PI 208, and drives the driver 205 according to the detection result to perform manual focusing. .

  Further, the fact that the MF ring 210 is operated and the manual focus operation is being performed is transmitted to the microcomputer 121 in the camera body 100 via the I / F 300. In this embodiment, when the MF ring 210 is operated, a live view display is enlarged and displayed on a liquid crystal display (hereinafter referred to as an LCD) 135 to be described later. Therefore, the MF ring 210 performs the live view display. It functions as an enlarged image operation means for instructing enlargement display or instructing enlargement of an image when recording a moving image.

  A mechanical shutter 101 is disposed in the camera body 100 on the optical axis of the taking lens 201. The mechanical shutter 101 controls the passage time of the subject luminous flux and employs a known focal plane shutter or the like. An image sensor 103 is disposed behind the mechanical shutter 101 and at a position where a subject image is formed by the photographing lens 201.

  In the image sensor 103, photodiodes constituting each pixel are two-dimensionally arranged in a matrix, and each photodiode generates a photoelectric conversion current corresponding to the amount of received light, and this photoelectric conversion current is applied to each photodiode. Charges are accumulated by the connected capacitor. A Bayer color filter is arranged in front of each pixel. The Bayer array has lines in which R pixels and G pixels are alternately arranged in the horizontal direction and lines in which G pixels and B pixels are alternately arranged. Further, some of the R and B pixels are replaced with focus detection pixels (AF pixels). A detailed configuration of the image sensor 103 will be described later with reference to FIGS. 2 and 3. In this specification, a signal based on an image signal output from the image sensor 103 is referred to as image data including not only the signal A / D converted by the A / D converter 107 but also the image signal. There is a case.

  The image sensor 103 is shifted in the vertical direction and the horizontal direction within the plane orthogonal to the optical axis of the photographing lens 201 by the image sensor position moving unit 117. The image sensor position moving unit 117 includes a mechanical mechanism for moving the image sensor 103 by the driving force of the piezoelectric element, a circuit for performing drive control of the piezoelectric element, and the like. A detailed configuration of the image sensor moving unit 117 will be described later with reference to FIG.

  The image sensor position moving unit 117 functions as a changing unit that changes the relative position between the subject image formed by the photographing lens 201 and the image sensor 103. As will be described later, since the microcomputer 121 calculates the amount of movement of the image sensor 103, the image sensor position moving unit 117 starts imaging of the next frame after completion of imaging of one frame by the image sensor 103. Before, the relative position between the subject image and the image sensor is changed based on the calculated amount of movement (in this embodiment, the relative position between the subject image and the image sensor is changed by moving the image sensor 103. ing). Further, the changing unit changes the relative position between the subject image and the image sensor when performing the enlarged display or when the enlarged display is instructed.

  The image sensor 103 is connected to an analog processing unit 105. The analog processing unit 105 performs waveform shaping on the photoelectric conversion signal (analog image signal) read from the image sensor 103 while reducing reset noise and the like. Further, the gain is increased so as to obtain a more appropriate luminance. The analog processing unit 105 is connected to an A / D conversion unit 107. The A / D conversion unit 107 performs analog-to-digital conversion on the analog image signal, and sends the digital image signal (hereinafter referred to as image data) to the bus 119. Output.

  The bus 119 is a transfer path for transferring various data read or generated in the camera body 100 to the camera body 100. In addition to the A / D conversion unit 107, the bus 119 includes an image processing unit 109, an AE (Auto Exposure) processing unit 111, an AF (Auto Focus) processing unit 113, an image compression / decompression unit 115, a microcomputer 121, and an SDRAM. A (Synchronous DRAM) 127, a memory interface (hereinafter referred to as a memory I / F) 129, and a liquid crystal display driver (hereinafter referred to as an LCD driver) 133 are connected.

  The image processing unit 109 performs white balance correction (WB correction), synchronization processing, gamma / color reproduction processing, noise reduction processing, optical black subtraction processing (OB subtraction), edge enhancement processing, and the like, and is temporarily stored in the SDRAM 127. Image data is read out and various image processes are performed on the image data.

  As image processing by the image processing unit 109, when the MF ring 210 is operated during live view display or moving image recording, a part of the image from the image sensor 103 is cut out to generate an enlarged image. As described above, since the focus detection pixels of the image sensor 103 are not pixels constituting the image, a pixel defect occurs. Therefore, in this embodiment, when generating moving image data such as during live view display or moving image shooting, the image sensor position moving unit 117 is arranged so that the position of the focus detection pixel changes for each frame. Moves the image sensor 103 by a predetermined amount of movement, and the image processing unit 109 changes the cutout position of the image according to the predetermined amount of movement to generate an enlarged image. As described above, the image processing unit 109 functions as an image cutout unit that changes the cutout position of the image data output from the image sensor according to the movement amount.

  The image processing unit 109 also functions as an interpolation unit, and performs image data interpolation processing, for example, smoothing processing between a plurality of frames or interpolation processing within an image of one frame to reduce image noise. Note that inter-frame NR (Noise Reduction) as an example of smoothing processing between frames will be described later with reference to FIG.

  The AE processing unit 111 measures subject luminance based on the image data input via the bus 119, and outputs the subject luminance information to the microcomputer 117 via the bus 119. A dedicated photometric sensor may be provided for measuring the subject brightness, but in this embodiment, the subject brightness is calculated based on the image data.

  The AF processing unit 113 determines the defocus direction and the defocus amount of the photographing lens 201 by a so-called phase difference method based on the output of the AF photodiode 11b (see FIG. 3) provided in the focus detection pixel of the image sensor 103. Calculate and output to the microcomputer 121 via the bus 119. The microcomputer 121 outputs the calculated defocus direction and defocus amount to the microcomputer 207 in the interchangeable lens 200, and the driver 205 performs automatic focus adjustment of the photographing lens 201.

  The image compression / decompression unit 115 reads the image data from the SDRAM 127 when the image data is recorded on the recording medium 131, and compresses the read image data according to the JPEG compression method. Further, when reproducing the image data, the image data read from the recording medium 131 is decompressed. In this embodiment, the JPEG compression method is adopted as the image compression method. However, the compression method is not limited to this, and other compression methods such as TIFF and MPEG may be used.

  The microcomputer 121 functions as a control unit for the entire camera, and comprehensively controls various sequences of the camera. In addition to the I / F 300 described above, an operation unit 123 and a flash memory 125 are connected to the microcomputer 121.

  In addition, the microcomputer 121 functions as a movement amount calculation unit that calculates the predetermined amount described above. That is, as described above, when generating moving image data such as during live view display or moving image shooting, the subject on the image sensor 103 is changed so that the position of the focus detection pixel changes from frame to frame. Either one of the image and the image sensor 103 (in this embodiment, the image sensor 103) is moved by a predetermined movement amount, and the microcomputer 121 calculates the movement amount at this time. The amount of movement differs depending on the image reduction rate of the image data output from the image sensor, such as when the MF ring 210 is operated to generate an enlarged image. Specifically, the movement amount is changed only when displaying / recording larger than a predetermined reduction rate, and the movement amount is set to 0 when displaying / recording smaller than a predetermined reduction rate.

  The operation unit 123 includes operation members such as various input buttons and various input keys such as a power button, a release button, a moving image button, a playback button, a menu button, and the like, detects operation states of these operation members, and displays the detection result as a macro. The data is output to the computer 121. The microcomputer 121 executes various sequences according to the user's operation based on the detection result of the operation member from the operation unit 123. The power button is an operation member for instructing to turn on / off the power of the digital camera. When the power button is pressed, the power of the digital camera is turned on. When the power button is pressed again, the power of the digital camera is turned off.

  The release button includes a first release switch that is turned on when the button is half-pressed and a second release switch that is turned on when the button is further pressed halfway down and then fully pressed. When the first release switch is turned on, the microcomputer 121 executes a shooting preparation sequence such as AE processing and AF processing. When the second release switch is turned on, the mechanical shutter 101 and the like are controlled, image data based on the subject image is acquired from the image sensor 103 and the like, and a series of shooting sequences for recording the image data on the recording medium 131 are executed. Take a picture.

  The movie button is a button for starting and ending movie shooting. Since the moving image is not yet shot in the initial state, moving image shooting starts when the moving image button is pressed in this state, and moving image shooting ends when the moving image button is pressed during moving image shooting. Therefore, each time the moving image button is pressed, the start and end of moving image shooting are repeated alternately. The playback button is an operation button for executing a display mode in which a captured image recorded in the recording medium 131 is displayed on the LCD 135.

  The menu button displays a menu screen. In the present embodiment, the live view display is enlarged and displayed by operating the MF ring 210 or button operation during manual focus, and the enlargement ratio at that time is set on the menu screen. Of course, an operation button for setting the magnification may be provided.

  The flash memory 125 stores a program for executing various sequences of the microcomputer 121. The microcomputer 121 controls the digital camera based on this program. In addition, the flash memory 125 is a table (exposure condition determination table) showing combinations of exposure control values (ISO sensitivity, aperture value, shutter speed) that provide appropriate exposure with respect to the subject brightness, and white balance for white balance correction. Gain and color matrix coefficients are stored. The flash memory 125 stores various parameters and tables necessary for the operation of the digital camera, and the microcomputer 121 reads out these parameters and executes each process according to the program described above.

  The SDRAM 127 connected to the bus 119 is an electrically rewritable volatile memory for temporary storage of image data and the like. The SDRAM 127 temporarily stores the image data output from the A / D conversion unit 107 and the image data processed by the image processing unit 109, the image compression / decompression unit 115, and the like.

  A memory I / F 129 connected to the bus 119 is an interface for recording image data and the like on the recording medium 131 and reading the image data and the like from the recording medium 131. The recording medium 131 is, for example, a recording medium such as a memory card that is detachably attached to the camera body 100, but is not limited thereto, and may be a hard disk or the like built in the camera body 100. The memory I / F 129 and the recording medium 131 constitute recording means for recording a moving image based on image data output from the image sensor 103.

  The LCD driver 133 is connected to the LCD 135 and displays an image on the LCD 135 based on the image data read from the SDRAM 127 or the recording medium 131 and expanded by the image compression / decompression unit 115. The LCD 135 includes a liquid crystal display disposed on the back surface of the camera body 100 and performs image display. As the image display, there are a REC view display for displaying the recorded image data for a short time immediately after shooting, a playback display of a still image or a moving image file recorded on the recording medium 131, and a moving image display such as a live view display. included. When displaying compressed image data, the image data is displayed after being decompressed by the image compression / decompression unit 115 as described above. Further, the display unit is not limited to the liquid crystal display, and other display panels such as an organic EL may be adopted as a matter of course.

  Next, a detailed configuration of the image sensor according to the present embodiment will be described with reference to FIGS. 2 and 3. FIG. 2B is a plan view of the image sensor 103, and FIG. 2A is an enlarged view of a part of the image sensor 103. In FIG. 2 (a), a to are assigned as pixel numbers in the horizontal direction, and 1 to 16 are assigned as pixel numbers in the vertical direction. In this example, the pixel (A, 1) is an R pixel, the pixel (A, 2) is a B pixel, and the pixel (A, 1) and the pixel (A, 2) are G pixels. The combination of the four pixels is repeated in the horizontal direction and the vertical direction to form an image sensor. In addition, the part shown by the circle in each pixel shows a micro lens.

  In the present embodiment, the image sensors are arranged by replacing some R or B image generation pixels with focus detection pixels. That is, in the example shown in FIG. 2A, the pixel (d, 4), the pixel (e, 5), the pixel (x, 4), the pixel (x, 5), the pixel (d, 12), the pixel ( E, 13), pixels (x, 12), and pixels (x, 13) in total, including eight focus detection pixels. With these focus detection pixels, the phase difference in the vertical direction (direction of pixel coordinates 1 to 16) can be detected on the screen of the image sensor 103.

  In the present embodiment, the focus detection pixels are arranged so as to detect the phase difference along the vertical direction of the image sensor 103. However, the present invention is not limited to this, and the phase difference along the horizontal direction or the diagonal direction is not limited thereto. Focus detection pixels may be arranged so as to detect. In the present embodiment, the focus detection pixels are replaced with positions corresponding to R pixels and B pixels. However, the present invention is not limited to this, and the focus detection pixels may be replaced with positions corresponding to G pixels. .

  Next, a detailed structure of the focus detection pixel will be described with reference to FIG. FIG. 3A is a plan view of the focus detection pixel, and FIG. 3B is a cross-sectional view taken along the line BB. Photodiodes (not shown) are arranged in the captured image generation pixels arranged in a normal matrix, whereas the AF detection photo is arranged in the focus detection pixels as shown in FIG. A diode 11b is arranged. The AF detection photodiode 11 b is formed so as to be biased to one side with respect to the optical axis of the microlens 17. The color filter is provided with a luminance filter 15b in the focus detection pixel. The luminance filter 15b includes an ND filter, a transparent filter, a gray filter, and the like, and includes a filter that has no filter and is hollow.

  The upper side of the luminance filter 15 b is covered with the microlens 17. The focal length and the position of the microlens 17 are determined so that the position of the exit pupil 19 of the photographing lens 201 and the pixel (AF detection photodiode 11b) are substantially conjugate. A light shielding member 13 is provided on the back side of the luminance filter 15b. The light shielding member 13 causes only one side of the exit pupil 19 to reach the AF detection photodiode 11b, and shields the subject luminous flux other than that. In FIG. 3B, the AF detection photodiode 11b is formed to be biased to one side with respect to the optical axis of the microlens 17, but has the same size as the image generation pixel. It doesn't matter.

  Thus, the focus detection pixels in the present embodiment are configured. Therefore, the subject luminous flux 25 that has passed through a part of the exit pupil 19 of the photographing lens 201 passes through the microlens 17 and the luminance filter 15b and is projected onto the AF detection photodiode 11b. Light beams other than the subject light beam 25 are blocked by the light blocking member 13 and are not projected onto the AF detection photodiode 11b.

  In the present embodiment, since the AF detection photodiode 11b is arranged in the focus detection pixel of the image sensor 103, focus detection by phase difference detection is possible. That is, the focus detection pixel (AF photodiode 11b) is a first focus disposed at a position for receiving a light beam that has passed through one pupil region among subject light beams that have passed through different pupil regions of the photographing optical system. It includes a detection pixel and a second focus detection pixel arranged at a position for receiving the light beam that has passed through the other pupil region. For example, in FIG. 2A, the focus detection pixels (d, 4) are in a position to receive the light beam that has passed through one pupil region, while the focus detection pixels (e, 5) It is in a position to receive the light beam that has passed through the pupil region. Therefore, the phase difference can be obtained from one output of the focus detection pixels arranged in a predetermined direction and the other output.

  The AF detection photodiode 11b and the light shielding member 13 of the focus detection pixel are rectangular or square, but the shape and size can be changed as appropriate. When the positions of the AF detection photodiode 11b and the light shielding member 13 are changed, the position of the pupil conjugate to the light receiving surface of the AF detection photodiode 11b can be controlled. These shapes may be obtained by appropriately creating a mask at the time of exposure in the manufacturing process of the image sensor. In the present embodiment, the light shielding member 13 is provided. However, if the microlens 17 can accurately maintain the conjugate relationship between the light receiving surface of the AF detection photodiode 11b and the pupil region, the light shielding member is provided. Even if not, pupil division may be performed only with the shape of the photodiode 11b for AF detection.

  Next, the configuration of the image sensor position moving unit 117 will be described with reference to FIG. The aforementioned image sensor 103 is disposed on a holder 403, and this holder 403 is held by a fixed frame 401. That is, the holding plate 409c fixed to the fixed frame 401 with screws 413 and the holding plate 409c forming the leaf spring by inserting the protrusion 403a of the holder 403 through the ball 411 between the fixed frame 401. Is held by pressing. Similarly, in the holding plates 409a and 409b, the holder 403 is held by the fixed frame 401 via a ball.

  The holder 403 is pressed upward (Y direction in the figure) by the piezoelectric body A 405 a and the piezoelectric body B 405 b, and extensible springs A 407 a and B 407 b are interposed between the holder 403 and the fixed frame 401. ing. The piezoelectric body A405a expands and contracts in the Y direction. When the piezoelectric body A405a extends, the holder 403 is pushed upward against the elastic force of the spring A407a, and when the piezoelectric body A405a contracts, the holder 403 It is pushed downward by the elastic force of the spring A407a.

  A flexure A415a is connected to the piezoelectric body A405a, and power is supplied through the flexure A415a. As shown in FIG. 4, the piezoelectric body A 405 a and the piezoelectric body B 405 b are arranged on the left and right sides of the holder 403, and the holder 403 is held by holding plates 409 a to 409 c and balls 411, along the vertical direction (Y direction). Can be moved. The configuration and operation of the piezoelectric body B 405b are the same as those of the piezoelectric body A 405a, and thus detailed description thereof is omitted here. Although FIG. 4 shows only the moving mechanism only in the vertical direction, similarly, a moving mechanism (not shown) that moves in the horizontal direction (X direction) is provided.

  Next, an example of synchronous communication performed between the camera body 100 and the interchangeable lens 200 will be described with reference to FIG. In FIG. 5, the horizontal axis represents the flow of time, and the vertical axis represents the processing contents and timing. In the camera body process, in the process B1, a live view image is displayed and the AF defocus amount is calculated based on the image data acquired in the previous frame. In the process B2, AF calculation, various setting changes, and the like are performed based on the lens state data acquired by the lens state communication.

  The vertical synchronization signal is a signal output corresponding to each frame. In imaging / reading, a subject image is picked up by the image pickup device 103, and the picked-up image data is read out. Note that the image pickup / readout has a rhombus shape in FIG. 5. In this embodiment, a rolling shutter is used when a live view image is acquired, and image pickup and read out are sequentially performed for each pixel line. To do.

  In the communication BL in the lens communication, a lens state data request command is transmitted from the camera body 100 to the interchangeable lens 200, and this command requests transmission of data indicating the lens state of the interchangeable lens 200 to the camera body 100. In the communication LB, the interchangeable lens 200 transmits data indicating the lens state to the camera body 100 in response to the lens state data request command.

  The lens communication synchronization signal is generated in response to the vertical synchronization signal in the camera body 100, and this lens communication synchronization signal is output to the interchangeable lens 200 from the synchronization signal terminal of the I / F 300 on the camera body 100 side. The state of the lens position acquisition signal changes at a predetermined timing, for example, in the example shown in FIG.

  Also, the process L1 in the interchangeable lens 200 is to acquire the position information of the photographing lens 201 at the timing of changing the state of the lens position acquisition signal and to detect the operation state of the MF ring 210 at the reception timing of the lens communication synchronization signal. This is the process to be performed. The process L2 is a process of transmitting lens state data such as the position information of the photographing lens 201 and the operation state of the MF ring 210 in response to the lens state data request command received from the camera body 100.

  As shown in the timing chart of FIG. 5, in the synchronous communication in this embodiment, the process B1 is executed in the camera body 100 in synchronization with the vertical synchronization signal, and the lens communication synchronization signal is synchronized with the vertical synchronization signal. Send to interchangeable lens 200.

  When processing B1 is processed in the camera body 100, a lens state data request command is transmitted to the interchangeable lens 200 by communication BL. When the interchangeable lens 200 receives the lens state data request command, the interchangeable lens 200 detects the lens state and transmits the lens state data by communication LB. When receiving the lens state data, the camera body 100 executes the process B2.

  In the interchangeable lens 200, the process L1 for acquiring the lens position is executed in synchronization with the lens position acquisition signal. The lens position acquisition signal is generated at a predetermined timing, as described above, in the example of FIG. 5, at the time when ½ of the charge accumulation time at the screen center of the image sensor 103 has elapsed. The interchangeable lens 200 acquires position information of the photographing lens 201 by a lens position detection circuit (not shown) at the timing of the state change of the lens position acquisition signal. These synchronous communications are executed in synchronization with the lens communication synchronization signal as a whole.

  As described above, in the present embodiment, when the MF ring 210 is rotated, the microcomputer 207 in the interchangeable lens 200, based on the pulse signal from the PI 208, sets the MF at the end of the process L2. The fact that the ring 210 is being rotated is transmitted to the microcomputer 121 in the camera body 100.

  Next, the operation of the digital camera in this embodiment will be described with reference to FIGS. 6, 7, 9, and 10 are executed by the microcomputer 121 in accordance with a program stored in the flash memory 125.

  First, the main operation will be described using the flowchart shown in FIG. When the power button is operated and the power is turned on, the main flow shown in FIG. 6 starts operation. When the operation is started, first, initialization is performed (S1). Here, the recording flag indicating whether or not a moving image is being recorded is turned off. The recording flag indicates that a moving image is being recorded when turned on, and indicates that no moving image is recorded when turned off. Moreover, the synchronous communication demonstrated using FIG. 5 between the interchangeable lens 200 and the camera main body 100 is started.

  Once initialization has been carried out, it is next determined whether or not the playback button has been pressed (S3). Here, the determination is made based on the operation state of the playback button in the operation unit 123. If the result of this determination is that the playback button has been pressed, playback is next carried out (S19). Here, image data is read from the recording medium 131 and displayed on the LCD 135.

  If playback is executed in step S19 or if the result of determination in step S3 is that the playback button has not been pressed, it is next determined whether or not the movie button has been pressed (S5). In this step, the operation unit 123 detects the operation state of the moving image button and makes a determination based on the detection result. If the moving image button is pressed as a result of this determination, then the recording flag is reversed (S21). As described above, every time the movie button is pressed, the movie shooting start and end are alternately repeated. Therefore, in this step, if the recording flag is off, it is turned on. Turn off and invert the recording flag.

  If the recording flag is reversed in step S21, it is next determined whether or not moving image recording is in progress (S23). Here, the determination is made based on whether or not the recording flag is ON. That is, if the recording flag is on, it is determined that the moving image is being recorded. If the result of determination in step S23 is that movie recording is in progress, a movie file is generated (S25). Here, since moving image recording has started, a new moving image file is generated for recording.

  If a moving image file is generated in step S25, or if the result of determination in step S23 is that a moving image is not being recorded, or if the result of determination in step S5 is that the moving image button has not been pressed, As in step S23, it is determined whether moving image recording is in progress (S7).

  If the result of determination in step S7 is that video recording is not in progress, it is next determined whether or not the release button has been pressed halfway, in other words, whether or not the first release switch has been turned on from off (S9). ). This determination is made based on the detection result obtained by detecting the state of the first release switch linked to the release button by the operation unit 123. In this step, it is determined whether or not the first release switch has been changed from OFF to ON. If the ON state is maintained after the change, the determination result is No.

  If the result of determination in step S9 is that the first release has been pressed, AE is performed (S11). Here, the AE processing unit 111 measures subject brightness, determines an exposure control value such as an aperture value and a shutter speed, and determines a control value for performing live view display on the LCD 135 with appropriate exposure.

  When AE is performed in step S11, next, photographing is performed (S13). In this shooting, an image signal of the subject image is acquired by the image sensor 103. In this step, image data is not recorded on the recording medium 131.

  Once shooting is performed, AF is performed (S15). Here, based on image data from the AF detection photodiode 11b of the focus detection pixel of the image sensor 103, focus detection is performed by a so-called phase difference method. That is, the defocus direction and the defocus amount of the photographing lens 101 are obtained based on the image data shift of the AF detection photodiodes 11b arranged along the vertical direction. Then, based on the obtained defocus direction and defocus amount, the driver 205 controls the focus position of the photographing lens 201 via the microcomputer 207 in the interchangeable lens 200. Therefore, when the release button is pressed halfway when the moving image is not being recorded, the photographing lens 201 is focused once at that time.

  If the result of determination in step S9 is that the release button is not off → 1st transition, then whether or not the second release (2nd) has been pressed, in other words, the release button is fully pressed and the second release switch is turned on. It is determined whether or not (S27). In this step, the state of the second release switch linked to the release button is detected by the operation unit 123, and a determination is made based on the detection result.

  If the result of determination in step S27 is that the second release has been pressed, still image shooting is performed (S29). Here, the image sensor 103 performs exposure to acquire an image signal corresponding to the subject image.

  If still image shooting is performed in step S29, image processing is performed next (S31). Here, an image signal is read from the image sensor 103, the image processing unit 109 performs image processing on still image data based on the image signal, and the image compression / decompression unit 115 performs image compression processing. As described above, since a photodiode (not shown) of the imaging pixel is not arranged at a position where the focus detection pixel is located, an image signal for generating a captured image cannot be acquired. Therefore, in the case of still image shooting, interpolation processing may be performed using image data of pixels around the focus detection pixels.

  Once the still image processing is performed, the image data subjected to the image processing is recorded (S33). Here, the image data subjected to the image processing is recorded on the recording medium 131.

  If the result of determination in step S7 is that movie recording is in progress, AE is then performed as in step S11 (S35). Here, the AE processing unit 111 performs exposure control for moving image shooting and live view display.

  Subsequently, moving image / live view shooting is performed (S37). Here, an image signal is acquired by the image sensor 103 for moving image shooting and live view display. Further, in order to make the pixel defect due to the focus detection pixels inconspicuous, the image sensor position moving unit 117 moves the image sensor 103 by a predetermined movement amount for each frame. Note that the movement of the image sensor 103 may be executed only when an enlarged live view display is performed by an operation of the MF ring 210 or a button operation during manual focus. Detailed operation of this moving image / live view shooting will be described later with reference to FIGS.

  Once moving image / live view shooting has been performed, moving image processing is performed (S39). Here, image processing for moving image recording and image processing for live view display are performed. In step S37, when the image sensor 103 is moved by a predetermined amount, in order to display and record a stable image, the image data from the image sensor 103 is cut out according to the amount of movement moved in step S37. Change the position. Detailed operation of the moving image processing will be described later with reference to FIG.

  Once the moving image processing is performed, live view display is performed (S41). Here, display is performed on the LCD 135 using the moving image image data processed in step S39. If a moving image is being recorded, the recorded moving image can be confirmed. In addition, it is possible to observe a live view image that has been processed so that pixel defects due to focus detection pixels are not noticeable. When the MF ring 210 is operated, the live view display may be enlarged in the manual focus mode. In this case, a live view image enlarged at an enlargement rate set on the menu screen is displayed.

  Once live view display has been performed, it is next determined whether or not a moving image is being recorded, as in steps S23 and S7 (S43). If the result of this determination is that movie recording is in progress, movie recording is performed (S45). Here, the image data subjected to the moving image processing in step S39 is recorded on the recording medium 131 using the moving image file generated in step S25. In this case, by moving the image sensor 103 and cutting out the image according to the amount of movement, an image in which pixel defects are not noticeable can be recorded. Note that recording of a moving image when the MF ring 210 is operated may record a normal size image or may record an enlarged image.

  If still image recording is performed in step S33, or if the result of determination in step S43 is that moving image recording is not being performed, or if moving image recording is performed in S45 or AF is performed in step S15, the power is turned off. It is determined whether or not (S17). In this step, it is determined whether or not the power button of the operation unit 123 has been pressed again. If the result of this determination is that power is not off, processing returns to step S3. On the other hand, if the result of determination is that the power is off, the main flow is ended after performing the end operation of the main flow.

  Prior to describing the moving image / live view shooting subroutine in step S37, calculation of the movement amount of the image sensor 103 in the present embodiment will be described with reference to FIG. FIG. 8A shows the shooting of the previous frame, and FIG. 8B shows the shooting of the current frame. In the image sensor 103, focus detection pixels 103a are regularly arranged in the vertical and horizontal directions every several to several tens of pixels. Although not shown in FIG. 8, several to several tens of image generation pixels are arranged between adjacent focus detection pixels 103a.

  In the present embodiment, the position of the optical axis center 103c of the subject image does not move, but the image sensor 103 moves for each frame. Therefore, in the example of FIG. 8, the exposure position of the previous frame (FIG. 8A). ) With respect to 103d, the exposure position of the current frame moves upward by a and leftward by b. In this movement, the image sensor 103 is randomly moved so that the position of the focus detection pixel 103c in FIG. 8B does not overlap the position of the focus detection pixel 103c in FIG. In order to move randomly, a random number may be generated and determined. Of course, if the focus detection pixels do not overlap, regular changes may be made. Also, the colors at the Bayer (for example, the color at the center of the optical axis) may be shifted so that they are always the same or different.

  If the image sensor 103 is moved for each frame, the subject image moves for each frame. Therefore, in step S61 (see FIG. 9) described later, the image cutout position 103b is moved by the amount of movement of the image sensor 103. As a result, a stable image can be displayed and recorded.

  In addition, when a camera shake sensor is provided in the camera body 100 and so-called electronic camera shake correction is performed in which the cutout position is moved so as to cancel the camera shake detected by the camera shake sensor, the cutout position changes according to the camera shake. In this case, when the position of the end point detection pixel in the clipped image is the same as that of the previous frame, it is randomly shifted, and when it is different, the movement amount may be zero.

  Next, the moving image / live view shooting operation in step S37 will be described with reference to the flowchart shown in FIG. In the moving image / live view shooting flow, the exposure position movement amount is first calculated (S51). Here, the microcomputer 121 functioning as a movement amount calculation unit determines the position of the image sensor 103 so that the position of the focus detection pixel in the previous frame and the position of the focus detection pixel in the frame to be captured this time do not overlap. The amount of movement for moving in the horizontal and vertical directions is calculated. However, when the reduction display is performed more than a predetermined reduction ratio (for example, a reduction ratio such that the number of pixels after reduction is equal to or less than the number of pixels for focus detection, in the example shown in FIG. 2, 4/16 reduction display. ), The amount of movement is set to 0 because the focus detection pixels are not conspicuous. When the enlarged display or the enlarged image is recorded, the moving amount that moves the image sensor 103 is set. Of course, you may make it always move irrespective of a reduction rate or an enlargement rate. Further, the moving direction may be only horizontal or only vertical.

  Once the exposure position movement amount has been calculated in step S51, the sensor movement is then performed (S53). Here, in step S51, the image sensor position moving unit 117 moves the image sensor 103 based on the calculated movement amount.

  Once the sensor is moved, next exposure is performed (S55). Here, after the charge of the image sensor 103 is reset, exposure is performed with the ISO sensitivity and aperture value obtained from the result of the previous AE in step 35 and the calculated exposure time. When the exposure is completed, next, image data is read (S57). When the image data is read, the original flow is restored.

  Next, the operation of the moving image processing in step S39 (see FIG. 6) will be described using the flowchart shown in FIG. When the flow for moving image processing is entered, first, clipping is performed (S61). Here, the image processing unit 109 cuts out an image at a position corresponding to the moving direction and moving amount of the exposure position calculated in step S51 (see the cutting position 103b in FIG. 8). At this time, when electronic camera shake correction is performed, clipping is performed including a change of the clipping position by electronic camera shake correction.

  Once cut out, next, development processing is performed (S63). Here, the image processing unit 109 performs image processing such as OB subtraction, WB correction, synchronization, color matrix multiplication, gamma conversion, edge enhancement, and noise reduction (NR) on the Bayer array image data.

  Once development processing is performed, next, inter-frame NR is performed (S65). Here, using the image of the previous frame and the image of the current frame, noise reduction (NR) between frames using a known algorithm or the like so as to reduce the flicker of the image of the same coordinate between frames. I do. For example, smoothing is performed so that the difference between pixel values of corresponding coordinates is equal to or less than a predetermined value. At this time, when the image is not so much reduced or enlarged, the focus detection pixels are conspicuous. Therefore, the strength of inter-frame NR (the degree of smoothing, the smaller the flicker decreases) is reduced and the image is reduced. The strength should be weakened. Details of the inter-frame NR will be described later with reference to FIG. When inter-frame NR is performed, the original flow is restored.

  Next, the inter-frame NR will be described with reference to FIG. When the inter-frame NR is entered, first, the pixel number i = 0 is set (S71). The pixel numbers are sequentially given from, for example, the upper left corner of the image sensor 103. Subsequently, it is determined whether or not the pixel number i is smaller than the number of pixels (S73). In step S81, which will be described later, the value is incremented by one every time processing is completed. In this step, it is determined whether or not processing has been performed for all pixels.

  If the result of determination in step S73 is that the pixel number i is smaller than the number of pixels, it is next determined whether or not the i-th pixel is a focus detection pixel (AF pixel) and enlargement (S75). The pixel position of the focus detection pixel is determined by the focus detection element 103. Here, it is determined whether or not the i-th pixel position is the position of the focus detection pixel. In addition, since the enlarged display is performed on the LCD 135 during manual focus or the like, in this step, it is determined whether or not the enlarged display is performed by operating the MF ring 210.

  If the result of determination in step S75 is AF pixel and enlargement, strong inter-frame NR is executed (S77), whereas if AF pixel and enlargement is not, weak inter-frame NR is executed (S77). S79). By performing the inter-frame NR, the image can be smoothed. In particular, in the case of an enlarged display, in the case of an AF pixel (focus detection pixel), an image signal cannot be obtained from the position of the AF pixel. The pixel defect is made inconspicuous.

  If inter-frame NR is performed in step S77 or S79, then 1 is added to the pixel number i (S81), and the process returns to step S73. Thereafter, while adding 1 to the pixel number, a strong inter-frame NR is applied in the case of an AF pixel in enlarged display, and a weak inter-frame NR is applied in the case of none. (No is determined in step S73, and when the inter-frame NR is completed for all the pixels, the original flow is returned to.

  Thus, in the first embodiment of the present invention, the position of the image sensor 103 is moved so that the focus detection pixels do not overlap in the previous frame and the current frame. For this reason, even if the image of the focus detection pixel (AF pixel) portion is missing, the missing portion moves from frame to frame, so that the human eye does not appear to lack the image. In particular, in the case of live view display of an enlarged image or moving image recording of an enlarged image, image loss is conspicuous, but an image of sufficient quality can be obtained by moving the image sensor 103 as in this embodiment. be able to.

  In the first embodiment of the present invention, when cutting out from the read image data, the cutting is performed at a position corresponding to the moving amount and moving direction of the image sensor 103. When the image sensor 103 moves, the live view display and the recorded moving image also move, but a stable image can be obtained by the clipping process.

  In the first embodiment of the present invention, inter-frame NR is performed to reduce image noise. Although the image of the focus detection pixel portion is missing, by performing inter-frame NR, in the present embodiment, it is possible to reduce noise based on image loss and obtain a smooth image. The noise reduction means is not limited to the inter-frame NR. For example, the focus detection pixel portion may be interpolated based on the image data in one frame. You may make it replace with the last image data. Further, when the peripheral pixels of the focus detection pixels are also affected by the focus detection pixels due to the synchronization process or the like, these pixels may be processed in the same manner as the focus detection pixels.

  Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment of the present invention, the image sensor 103 is moved when the relative positional relationship between the subject image and the image sensor is changed by the changing means. In contrast, in the second embodiment, the changing unit moves the subject image by moving the optical system, and changes the relative positional relationship between the subject image and the image sensor. In other words, in the first embodiment, the image sensor 103 is moved by the image sensor position moving unit 117. On the other hand, in the second embodiment, the optical axis moving lens 202 and the optical axis moving lens control unit 206 are provided so that the subject image formed on the image sensor 103 is moved by the optical axis moving lens 202. Yes.

  FIG. 11 is a block diagram mainly showing an electrical configuration of a digital camera according to the second embodiment of the present invention. The same members as those in the block diagram shown in FIG. 1 according to the first embodiment are denoted by the same reference numerals, detailed description thereof will be omitted, and differences will be mainly described. The camera body 100 in the first embodiment is provided with the image sensor position moving unit 117, but in the second embodiment, the image sensor position moving unit 117 is omitted.

  An optical axis moving lens 202 is disposed on the optical axis of the taking lens 201 in the interchangeable lens 200. The optical axis moving lens 202 is configured to be movable in the vertical and horizontal directions within a plane perpendicular to the optical axis of the photographing lens 201. The optical axis moving lens 202 is moved by the optical axis moving lens control unit 206. When the optical axis moving lens 202 moves upward, the subject image formed on the image sensor 103 moves upward. On the other hand, when the optical axis moving lens 202 moves downward, the subject image formed on the image sensor 103 moves downward. Similarly, the subject image moves in the horizontal direction.

  The optical axis movement lens control unit 206 is connected to the microcomputer 207, and outputs a control command to the optical axis movement lens control unit 206 in accordance with an instruction from the microcomputer 121 in the camera body 100. The flash memory 209 stores a program of the microcomputer 207, various adjustment values, and the like. In this embodiment, optical axis movement related data relating to the amount of movement of the subject on the image sensor 103 is stored as various adjustment values according to the amount of movement of the optical axis movement lens 202.

  When the microcomputer 207 in the interchangeable lens 200 receives information on the required movement amount on the surface of the image sensor 103 from the microcomputer in the camera body 100, the microcomputer 207 is based on the optical axis movement related data stored in the flash memory 209. Thus, the movement amount of the optical axis movement lens is calculated, and the optical axis movement lens control unit 206 moves the optical axis movement lens 202.

  In the operation of this embodiment, the same processing as the flow shown in FIGS. 6, 7, 9, and 10 is performed. However, in calculating the exposure position movement amount in step S51 of FIG. 7, the movement amount of the optical axis movement lens 202 is calculated using the above-described optical axis movement related data. In step S53, the optical axis moving lens 202 is moved instead of moving the sensor.

  Thus, in the second embodiment of the present invention, the optical axis moving lens 202 is moved so that the focus detection pixels do not overlap in the previous frame and the current frame. For this reason, as in the case of the first embodiment, even if the image of the focus detection pixel (AF pixel) portion is missing, the missing portion moves, so that the image is missing in the human eye. I can't see. In the present embodiment, even when the moving mechanism of the image sensor 103 is not provided on the camera body 100 side, it is possible to prevent image degradation based on focus detection pixel defects on the interchangeable lens side.

  As described above, in each embodiment of the present invention, the relative movement amount between the subject image and the image sensor is calculated, and after the first frame is imaged by the image sensor, the first frame Thereafter, the relative position of the subject image and the image sensor is changed based on the movement amount calculated before the start of imaging of the second frame, which is a frame to be imaged thereafter. For this reason, even if the image of the focus detection pixel (AF pixel) portion is missing, an image of sufficient quality can be obtained.

  In each embodiment of the present invention, when the enlarged display is performed, the relative position between the subject image and the image sensor is changed based on the calculated movement amount, and the image data cut-out position is changed according to the movement amount. And the live view is displayed on the display means based on the image data whose cutout position is changed. In the live view display of the enlarged image, the image loss is conspicuous, but an image with sufficient quality can be obtained. Similarly, when recording a moving image, the relative position between the subject image and the image sensor is changed based on the calculated movement amount, and the cutout position of the image data is changed according to the movement amount. The image data whose position has been changed is recorded by the recording means. It is possible to record an image of sufficient quality when shooting a movie.

  In each embodiment of the present invention, in both live view display and moving image recording, the relative position of the subject image and the image sensor is prevented in order to prevent image quality degradation due to loss of the image of the focus detection pixel portion. However, the present invention is not limited to this, and only one of the live view display and the moving image recording may be used.

  In each embodiment of the present invention, when the subject image is enlarged, the relative position between the subject image and the image sensor is changed. However, the subject image and the image sensor are always changed regardless of enlargement (or reduction). The relative position may be changed, or the relative position may be changed only in either case.

  Further, in each embodiment of the present invention, the relative position of the subject image and the image sensor is changed every frame, but in consideration of the characteristics of the human eye, it may be every few frames. The timing of changing the relative position of the image sensor may be changed as appropriate. Further, when changing the relative position between the subject image and the image sensor, it may be random or regular for each frame. In any case, in consideration of the characteristics of the human eye, the image may be appropriately changed within a range in which the image defect due to the focus detection pixels is not noticeable.

  In each embodiment of the present invention, a digital camera has been described as an apparatus for photographing. However, the camera may be a digital single-lens reflex camera, a digital single-lens camera, or a compact digital camera, a video camera, a movie It may be a moving image camera such as a camera, or may be a camera built in a mobile phone, a personal digital assistant (PDA), a game machine, or the like. In any case, the present invention can be applied to any device capable of taking an image having a focus detection element including a focus detection pixel.

  Further, regarding the operation flow in the claims, the specification, and the drawings, even if it is described using words expressing the order such as “first” and “next” for the sake of convenience, it is essential to carry out in this order. It does not mean that.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

11b: photodiode for AF detection, 13 ... light shielding member, 15b ... luminance filter, 17 ... micro lens, 19 ... exit pupil, 25 ... subject luminous flux, 100 ... camera Main unit 101 ... Mechanical shutter 103 ... Image sensor 105 ... Analog processing unit 107 ... A / D conversion unit 109 ... Image processing unit 111 ... AE processing output unit DESCRIPTION OF SYMBOLS 113 ... AF process part, 115 ... Image compression / expansion part, 117 ... Image pick-up element position movement part, 119 ... Bus, 121 ... Microcomputer, 121 ... Clock generator, 123 ... Operation unit, 125: flash memory, 127: SDRAM, 129 ... memory I / F, 131 ... recording medium, 133 ... liquid crystal display driver (LCD driver) B), 135 ... liquid crystal display (LCD), 200 ... interchangeable lens, 201 ... photographic lens, 202 ... optical axis moving lens, 203 ... aperture, 205 ... driver, 206 ... Optical axis moving lens control unit, 207 ... Microcomputer, 208 ... Photo interrupter (PI), 209 ... Flash memory, 210 ... Manual focus ring (MF ring), 300 ... Interface (I / F), 401... Fixed frame, 401a to 401b... Projection, 403... Holder, 405a ... Piezoelectric body A, 405b ... Piezoelectric body B, 407a. A, 407b ... Spring B, 409a-409c ... Holding plate, 411 ... Ball, 413 ... Screw, 415a-415b ... Flexible

Claims (11)

In an imaging device that can capture video,
An image sensor that has focus detection pixels, converts a subject image into image data, and outputs the image data;
A movement amount calculating means for calculating a relative movement amount between the subject image and the image sensor;
Calculated by the movement amount calculation means after completion of imaging of the first frame by the imaging device and before the start of imaging of the second frame, which is a frame to be imaged after the first frame. Changing means for changing the relative position of the subject image and the image sensor based on the amount of movement;
Interpolation means for performing interpolation processing by smoothing between frames, or interpolation processing within an image of one frame;
Display means for performing live view display based on the image data output from the image sensor;
Enlarged image operation means for instructing enlarged display in performing the live view display,
Image cutout means for changing the cutout position of the image data output from the image sensor according to the amount of movement;
Have
When the enlargement display is instructed, the relative position between the subject image and the image sensor is changed by the changing unit based on the movement amount, and the image data cut-out position is changed according to the movement amount. An image-taking apparatus characterized in that the display means displays live view on the display means based on the result of the interpolation processing performed by the interpolation means. 2. The photographing apparatus according to claim 1, wherein the second frame is an image displayed on the display unit next to the image of the first frame. Furthermore, recording means for recording a moving image based on the image data output from the image sensor;
Image cutout means for changing the cutout position of the image data output from the image sensor according to the amount of movement;
Have
When the recording of the moving image is instructed, the relative position between the subject image and the image sensor is changed by the changing unit based on the movement amount, and the cutout position of the image data is changed according to the movement amount. 2. The photographing apparatus according to claim 1, wherein the image data is changed by the image cutout means, and the image data in which the cutout position is changed is recorded by the recording means. 4. The photographing apparatus according to claim 3 , wherein the second frame is an image recorded next to the image of the first frame in the recording unit. It said changing means, according to claim 1, characterized in that either moving the imaging element based on the amount of movement, or a lens for forming the subject image image moved based on the movement amount Shooting device. The imaging apparatus according to claim 1, wherein the changing unit changes the relative position between the subject image and the image sensor when performing enlarged display or when enlarged display is instructed. The photographing apparatus according to claim 1, wherein the movement amount calculation unit changes the relative position between the subject image and the imaging element at random or regularly. The imaging apparatus according to claim 1, wherein the movement amount calculation unit calculates the movement amount according to an image reduction rate of image data output from the imaging element. The movement amount calculation means changes the movement amount only when displaying or recording larger than a predetermined reduction rate, and sets the movement amount to 0 when displaying or recording smaller than the predetermined reduction rate. The imaging device according to claim 1 . In an image capturing method for capturing a moving image by continuously capturing an image using an image sensor having an imager AF pixel,
Calculate the amount of movement according to the reduction rate of the image data,
Before the imaging of the next frame is started after the imaging of one frame is completed, the relative position of the subject image and the imaging element is changed according to the movement amount,
Perform interpolation processing by smoothing between frames, or interpolation processing within one frame image,
Perform live view display based on the image data output from the image sensor,
When an enlargement display is instructed when performing the live view display, the image data output from the image sensor is changed in the cutout position according to the movement amount, and the image data is output according to the movement amount. It is changing the extraction position, the live view display on the display unit based on the image data obtained by changing the extraction position in the result of interpolation processing by interpolation means,
An imaging method characterized by the above. In a program for executing a computer of an imaging apparatus that uses an imaging device having an imager AF pixel and continuously captures an image to shoot a moving image,
Calculating the amount of movement according to the reduction ratio of the image data;
Changing the relative position of the subject image and the image sensor in accordance with the amount of movement before completing the imaging of the next frame after the imaging of one frame is completed;
Performing interpolation processing by smoothing between frames, or performing interpolation processing within an image of one frame;
Performing live view display based on the image data output from the image sensor;
When an enlargement display is instructed when performing the live view display, the image data output from the image sensor is changed in the cutout position according to the movement amount, and the image data is output according to the movement amount. is changing the extraction position, a step of live view display on the display means based on the result of interpolation processing by interpolation means image data obtained by changing the extraction position,
A program for causing the computer to execute.