JP2014122957A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of photographing a still picture without interrupting moving image photographing by performing AF during the moving image photographing.SOLUTION: The imaging device includes a first imaging element 100, a second imaging element 103, light splitting means 107 for making a luminous flux made incident on an optical system incident on the imaging elements, focus detection means 122, focal position changing means 112 for changing a focal position of the optical system, and image processing means 120, 121. The second imaging element has a pixel for focus detection including at least one photoelectric conversion part for receiving a luminous flux subjected to pupil division. The first and second imaging elements and the light splitting means are arranged such that an image is formed by a common optical system in the first and second imaging elements. Then, signals from pixels of the first and second imaging elements are independently processed by the image processing means, respectively.

Description

The present invention relates to an autofocus and still image shooting technique during moving image shooting of an image pickup apparatus, and more particularly to an image pickup apparatus having a plurality of image pickup elements.

2. Description of the Related Art Conventionally, there is a technology that includes an image sensor that captures an image and an AF sensor that performs phase difference AF (autofocus), and performs phase difference AF during moving image shooting. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for performing phase difference AF while a subject image is incident on an image sensor and an AF sensor with a half mirror and a moving image captured by the image sensor is displayed.

JP 2006-197406 A

However, with the technique disclosed in Patent Document 1 described above, it is necessary to stop the moving image when shooting a still image and restart the moving image after the still image shooting is completed. In addition, when trying to use an image sensor that outputs a still image and a moving image at the same time, the processing speed and circuit configuration increase. SUMMARY OF THE INVENTION An object of the present invention is to provide an image pickup apparatus that performs AF during moving image shooting and can take a still image without interrupting the moving image shooting.

In order to achieve the above object, the imaging apparatus of the present invention has the following characteristics.
A first imaging element in which pixels including a photoelectric conversion unit are two-dimensionally arranged, a second imaging element in which pixels including a photoelectric conversion unit are two-dimensionally arranged, and a light beam incident on a common optical system Splitting the light into the first image sensor and the second image sensor, focus detection means for performing focus detection based on signals from the pixels of the second image sensor, and the focus detection means Focus position changing means for changing the focus position of the common optical system based on the detection result of the first image processing, and image processing for processing signals from the pixels of the first image sensor and signals from the pixels of the second image sensor Means. Then, the second image sensor is a first photoelectric conversion unit for receiving a light beam divided through the first region of the exit pupil of the common optical system, and / or the common optical system. A focus detection pixel including a second photoelectric conversion unit for receiving a light beam divided through the second region shifted from the first region of the exit pupil of the first pupil; and The first image sensor, the second image sensor, and the light splitting unit are arranged so that an image is formed on the second image sensor by the common optical system. Signals from the pixels and signals from the pixels of the second image sensor are each independently processed by the image processing means.

According to the present invention, it is possible to provide an imaging apparatus that performs AF during moving image shooting and can capture a still image without interrupting moving image shooting.

It is a figure which shows the structure of the imaging device which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the imaging device which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the image pick-up element in embodiment of this invention. It is a figure which shows the structure of the image pick-up element in the 1st Embodiment of this invention. It is a figure which shows the concept of the focus detection in embodiment of this invention. 3 is a flowchart illustrating an operation of the imaging apparatus according to the first embodiment of the present invention. It is a figure which shows the concept of the focus detection in embodiment of this invention. It is a figure which shows the structure of the imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows the structure of the imaging device which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows operation | movement of the imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows the structure of the imaging device which concerns on the 3rd Embodiment of this invention. It is a figure which shows the structure of the image pick-up element in the 3rd Embodiment of this invention. It is a flowchart which shows operation | movement of the imaging device which concerns on the 3rd Embodiment of this invention. It is a figure which shows the output result of the image pick-up element in the 3rd Embodiment of this invention.

In the imaging apparatus of the present invention, the first imaging element including the pixels and the second imaging element including the focus detection pixels are arranged so that the image is formed by the common imaging optical system. Signals from the pixels of the second image sensor are each independently processed by the image processing means. Thereby, for example, a plurality of image sensors are arranged together with the light splitting means so as to have the same image plane magnification, and still images and moving images are independently photographed by the respective image sensors, so that an image sensor that outputs moving images is used. Focus detection is performed using the output, and AF operation can be performed simultaneously. Therefore, AF can be performed during moving image shooting, and a still image can be shot without interrupting moving image shooting.

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. With reference to FIG. 1, the configuration and operation of an imaging apparatus according to the first embodiment of the present invention will be described. In the imaging apparatus of FIG. 1, reference numeral 100 denotes a first imaging element that converts an optical image into an electrical signal. The image sensor 100 mainly shoots still images. Reference numeral 101 denotes an analog front end (hereinafter referred to as AFE) that performs digital conversion on an analog image signal output from the image sensor 100 in accordance with gain adjustment or a predetermined quantization bit. Reference numeral 102 denotes a timing generator (hereinafter referred to as TG) that controls the drive timing of the image sensor 100 and the AFE 101. Reference numeral 103 denotes a second image sensor that converts an optical image into an electrical signal. The image sensor 103 mainly shoots moving images. Reference numeral 104 denotes an AFE that performs digital conversion on an analog image signal output from the image sensor 103 in accordance with gain adjustment or a predetermined quantization bit. Reference numeral 105 denotes a TG that controls the drive timing of the image sensor 103 and the AFE 104. In this embodiment, the AFE 101 and TG 102 related to the first image sensor 100 and the AFE 104 and TG 105 related to the second image sensor 103 are used. However, the AFE and TG may be built in the image sensor. Is possible.

Reference numeral 118 denotes a RAM. The RAM functions as an image data storage unit that stores the image data digitally converted by the AFE 101 and the AFE 104 and the image data processed by the image processing unit 120 or the image processing unit 121 described below, and the CPU 124 described below operates. Combines the function of work memory. In the present embodiment, these functions are performed using the RAM 118, but other memories can be applied as long as the access speed is a level that does not cause a problem. Reference numeral 119 denotes a ROM that stores a program used when the CPU 124 operates. Here, the flash-ROM is shown in the present embodiment, but this is an example, and other memories may be applied as long as the access speed is at a level that does not cause a problem. Reference numeral 124 denotes a CPU that comprehensively controls the imaging apparatus. An image processing unit 120 performs processing such as correction / compression of a captured still image, which will be described later. Reference numeral 121 denotes an image processing unit that performs processing such as correction and compression of a captured moving image, which will be described later. In addition, a function of adding A image data and B image data, which will be described later, is provided. In this way, the signal from the pixel of the first image sensor and the signal from the pixel of the second image sensor are each independently processed by the image processing means, and an image is generated.

Reference numeral 122 denotes an AF calculation unit that performs focus detection from the pixel signal output from the second image sensor 103. Reference numeral 123 denotes a detachable flash memory for recording still image data and moving image data. In this embodiment, a flash memory is applied as a recording medium, but other data writable nonvolatile memory, hard disk, and the like may be used. Moreover, the form which incorporated these recording media may be sufficient. Reference numeral 116 denotes an operation unit for setting a shooting command, shooting conditions, and the like to the CPU 124. Reference numeral 117 denotes a display unit that displays captured still images, moving images, menus, and the like.

Reference numeral 111 denotes a first lens group disposed at the tip of the photographing optical system (common optical system), and is held so as to be able to advance and retract in the optical axis direction. Reference numeral 110 denotes a diaphragm, which adjusts the light amount at the time of photographing by adjusting the aperture diameter. Reference numeral 109 denotes a second lens group. The diaphragm 110 and the second lens group 109 integrally move forward and backward in the optical axis direction, and perform a zooming function (zoom function) in conjunction with the forward and backward movement of the first lens group 111. Reference numeral 108 denotes a third lens group that performs focus adjustment by advancing and retracting in the optical axis direction. Reference numeral 107 denotes a half mirror that divides a light beam from an incident subject into reflected light and transmitted light. Reflected light from the half mirror 107 enters the second image sensor 103, and transmitted light enters the first image sensor 100.

A focal plane shutter 106 adjusts the exposure time during still image shooting. In this embodiment, the exposure time of the image sensor 100 is adjusted by a focal plane shutter, but this is not a limitation. The imaging device 100 may have an electronic shutter function and adjust the exposure time with a control pulse. Reference numeral 112 denotes a focus drive circuit that is a focus position changing unit that changes the focus position of the optical system. The focus drive circuit 112 drives and controls the focus actuator 114 based on the focus detection result of the AF calculation unit 122, and controls the third lens group 108 in the optical axis direction. Drive forward and backward to adjust the focus. Reference numeral 113 denotes an aperture driving circuit that controls the aperture of the aperture 110 by drivingly controlling the aperture actuator 115.

FIG. 2 is a diagram illustrating the positions of the first image sensor 100, the second image sensor 103, and the half mirror 107. As described above, the half mirror 107 is arranged at a position / angle at which the reflected light of the half mirror 107 is incident on the image sensor 103 and the transmitted light is incident on the image sensor 100. That is, the first image sensor receives the transmitted light of the light dividing means, and the second image sensor receives the reflected light of the light dividing means. The distance a from the center of the half mirror 107 to the image sensor 100 and the distance b to the image sensor 103 are arranged equally. As a result, a primary image that is a subject image having the same magnification is incident on the first image sensor 100 and the second image sensor 103. With this configuration, even when the AF operation is performed using the image signal of the second image sensor 103, the image formed on the first image sensor 100 can be focused.

Next, the first image sensor 100 will be described. FIG. 3 shows the configuration of the image sensor 100. In FIG. 3, the imaging device includes a pixel array 100a, a vertical selection circuit 100d that selects a row in the pixel array 100a, and a horizontal selection circuit 100c that selects a column in the pixel array 100a. Further, a readout circuit 100b that reads out signals of pixels selected by the vertical selection circuit 100d and the horizontal selection circuit 100c among the pixels in the pixel array 100a is provided. The vertical selection circuit 100d selects a row of the pixel array 100a, and validates the readout pulse output from the TG 102 based on the horizontal synchronization signal output from the CPU 124 in the selected row. The readout circuit 100b includes an amplifier and a memory for each column, and the pixel signal of the selected row is stored in the memory via the amplifier. The pixel signals for one row stored in the memory are sequentially selected in the column direction by the horizontal selection circuit 100c and output to the outside through the amplifier 100e. This operation is repeated for the number of rows, and the signals of all the pixels are output to the outside.

FIG. 4A shows the configuration of the pixel array 100a. The pixel array 100a of the first image sensor 100 is configured by arranging a plurality of pixels in a two-dimensional array in order to provide a two-dimensional image. In FIG. 4A, 100f is a microlens, and 100g is a photodiode (PD) that performs photoelectric conversion. One microlens 100f is arranged at the top for each pixel and one PD. This pixel is arranged with the h1 pixel in the horizontal direction and the i1 pixel in the vertical direction.

The configuration and reading operation of the second image sensor 103 are the same as those of the first image sensor 100 shown in FIG. A pixel array of the image sensor 103 is shown in FIG. In FIG. 4B, 103f is a microlens, and 103g and 103h are PDs. One microlens 103f is arranged at the top for each pixel and two PDs. That is, the focus detection pixel includes a plurality of photoelectric conversion units for one microlens. When the area where the microlens 103f is shared is one pixel, this pixel is arranged side by side with h2 pixels in the horizontal direction and i2 pixels in the vertical direction. The signals accumulated in the PD 103g and PD 103h are separately output to the outside by the read operation described above. Since another image having a phase difference is incident on the PD 103g and PD 103h, the PD 103g is an A image photoelectric conversion unit and the PD 103h is a B image photoelectric conversion unit. In the present embodiment, two PDs are arranged for one microlens, but this is not restrictive. The present invention can be applied to any configuration in which a plurality of PDs are arranged vertically or horizontally with respect to one microlens.

Next, image data output by the A image photoelectric conversion unit and the B image photoelectric conversion unit of the second image sensor 103 will be described. 5A and 5B show the relationship between the focus state and the phase difference in the image sensor 103. FIG. Reference numeral 103a denotes a cross section of the pixel array. Reference numeral 128 denotes the above-described microlens, and reference numerals 129 and 130 denote an A image photoelectric conversion unit and a B image photoelectric conversion unit, respectively. Reference numeral 125 denotes a photographing lens that is considered as one lens by combining the first lens group 111, the second lens group 109, and the third lens group 108 shown in FIG. The light emitted from the subject 126 passes through each region of the photographing lens 125 with the optical axis 127 as the center, and forms an image on the image sensor. Here, the exit pupil and the center or center of gravity of the photographic lens are the same.

According to such a configuration, the pupil of the imaging optical system is divided symmetrically with respect to the center when the imaging optical system is viewed from the A-image photoelectric conversion unit and when viewed from the B-image photoelectric conversion unit. Is equivalent to In other words, the light beam from the imaging optical system is so-called pupil divided into two light beams. Then, each of the divided light beams (first light beam and second light beam) is a first photoelectric conversion unit and a second photoelectric conversion unit for A image photoelectric conversion for receiving the pupil-divided light beam, respectively. And the B image photoelectric conversion unit. The first light beam is a light beam that is pupil-divided through the first region of the exit pupil, and the second light beam is pupil-divided through the second region that is offset from the first region of the exit pupil. Light flux. Thus, the light beam from a specific point on the subject 126 passes through the split pupil corresponding to the A image photoelectric conversion unit A and enters the A image photoelectric conversion unit A and the B image photoelectric conversion unit B. Is split into a light beam ΦLb incident on the B image photoelectric conversion unit B through the split pupil corresponding to.

Since these two light beams are incident from the same point on the subject 126, when the imaging optical system is in focus, the imaging element passes through the same microlens as shown in FIG. Reach the top point. Therefore, the image signals respectively obtained by the A image photoelectric conversion unit 129 and the B image photoelectric conversion unit 130 coincide with each other. However, as shown in FIG. 5B, in the state where the focus is shifted by Y, the arrival positions of the light beams ΦLa and ΦLb are shifted from each other by the change in the incident angle of the light beams ΦLa and ΦLb to the microlens. Therefore, there is a phase difference between the image signals obtained from the A image photoelectric conversion unit 129 and the B image photoelectric conversion unit 130, respectively. Two subject images (A image and B image) having a phase difference are photoelectrically converted by the A image photoelectric conversion unit 129 and the B image photoelectric conversion unit 130, and are separately output to the outside, and an AF operation described later. Used for.

Since the first image sensor 100 captures a still image and the second image sensor 103 captures a moving image, the pixel number h1 * i1 pixel of the image sensor 100 is larger than the pixel number h2 * i2 pixel of the image sensor 103. It has a large configuration. That is, the number of pixels of the second image sensor is smaller than the number of pixels of the first image sensor. Since the image sensor 103 has a smaller number of pixels than the image sensor 100, the area of the PD is increased, so that the sensitivity is high. Therefore, the light beam split by the half mirror 107 is split at a ratio of M: N between the transmitted light and the reflected light, and the ratio N of the reflected light incident on the imaging element 103 having high sensitivity is smaller than M. To do. That is, the intensity of the divided light beam incident on the second image sensor is lower than the intensity of the divided light beam incident on the first image sensor.

Next, the operation of the imaging apparatus in the present embodiment will be described using the flowchart of FIG. First, it waits until the moving image photographing switch included in the operation unit 116 is pressed in step S101. When the moving image shooting switch is pressed, moving image shooting is started in step S102. When moving image shooting is started, the second image sensor 103, the AFE 104, and the TG 105 are powered on, and the CPU 124 performs moving image shooting setting. After the setting, the TG 105 outputs a read pulse to the image sensor 103 based on the synchronization signal output from the CPU 124, and the image sensor 103 starts a read operation at a predetermined frame rate. In the present embodiment, the charge accumulation / reading operation for moving images is performed using the electronic shutter function based on the slit rolling operation, but this is not restrictive. The A image photoelectric conversion unit data and the B image photoelectric conversion unit data output from the image sensor 103 are transferred to the RAM 118 by the CPU 124. Thereafter, the data is transferred to the image processing unit 121, and the data of the A image photoelectric conversion unit and the B image photoelectric conversion unit under the same microlens are added for each pixel. This creates a frame of the moving image. Thereafter, correction processing, compression, and the like are performed, and a moving image is displayed on the display unit 117 (live view). When moving image recording is selected using the menu displayed on the display unit 117 and the operation unit 116 before shooting, the moving image is sequentially recorded in the flash memory 123.

In step S103, it is determined whether the moving image shooting switch has been pressed again. If the movie shooting switch has not been pressed, movie shooting is continued, and the process of step S104 is performed. When the movie shooting switch is pressed, movie shooting is terminated.

In step S104, it is determined whether an AF switch included in the operation unit 116 has been pressed. If the AF switch is pressed here, AF calculation is performed in step S105. In step S105, the CPU 124 receives A image data corresponding to the A image using the A image photoelectric conversion unit data stored in the RAM 118 and B image data corresponding to the B image using the B image photoelectric conversion unit data. Transfer to the AF calculation unit 122.

FIG. 7A shows A image data and B image data in the case of FIG. The horizontal axis represents the pixel position, and the vertical axis represents the output. When the image is in focus, the A image data and the B image data match. FIG. 7B shows the A image data and the B image data in the case of FIG. 5B that is not in focus. At this time, the A image data and the B image data have a phase difference depending on the state described above, and the pixel position is shifted by the shift amount X. The AF calculation unit 122 serving as a focus detection unit calculates the shift amount X for each moving image frame, thereby calculating the focus shift amount, that is, the Y value in FIG. That is, the focus detection unit performs focus detection by the phase difference detection method using the output of the focus detection pixels of the second image sensor. The AF calculation unit 122 transfers the calculated Y value to the focus drive circuit 112.

In step S <b> 106, the focus drive circuit 112 calculates the amount of movement of the third lens group 108 based on the Y value acquired from the AF calculation unit 122, and outputs a drive command to the focus actuator 114. The third lens group 108 is moved to a focus position by the focus actuator 114 and is brought into focus by the image sensor 103. At this time, the first imaging device 100 and the second imaging device 103 have the same primary image formation with the same image plane magnification and the same depth of field. When the image is in focus, the image sensor 100 is also in focus.

In step S107, it is determined whether the still image shooting switch included in the operation unit 116 has been pressed. If the still image shooting switch has been pressed, a still image is shot in step S108. When still image shooting is started, the first image sensor 100, the AFE 101, and the TG 102 are powered on, and the CPU 124 performs still image shooting settings. After the setting, the CPU 124 operates the focal plane shutter 106 to perform an exposure operation on the image sensor 100. Thereafter, based on the synchronization signal output from the CPU 124, the TG 102 outputs a read pulse to the image sensor 100, and the image sensor 100 starts a read operation. Image data output from the image sensor 100 is converted into digital data by the AFE 101 and then stored in the RAM 118. The CPU 124 transfers the image data stored in the RAM 118 to the image processing unit 120, and the image processing unit 120 performs image data correction processing, compression, and the like. Thereafter, the image data is recorded in the flash memory 123. Then, it returns to step S103 and repeats the operation | movement of step S103 to step S108.

If the AF switch is not pressed in step S104, the process proceeds to step S107, and it is determined whether or not the still image shooting switch is pressed. The same applies when the AF operation is set to OFF from the menu display using the display unit 117 and the operation unit 116.

With the above operation, while performing moving image shooting (live view or moving image recording), the phase difference AF operation is performed to focus the image incident on the image sensor 103 or the image sensor 100, and still image shooting is performed at the same time. it can.

In the present embodiment, the second image sensor 103 is a pixel in which all pixels can measure the distance and performs the phase difference AF. However, the present invention is not limited to this. A configuration in which phase difference AF is performed using signals from discretely arranged distance-measurable pixels may be used. The configuration of pixels capable of ranging in this case may be a configuration in which one PD is provided under the microlens, and pupil division is performed with a configuration in which the left or right, or the top or bottom is shielded by the light shielding layer. In other words, the second imaging device receives the light beam divided through the first region of the exit pupil, or / and the second shifted from the first region of the exit pupil. It suffices to have a focus detection pixel including a second photoelectric conversion unit that receives a light beam that has been pupil-divided through the region. The image sensor may be configured to include an image pixel and a focus detection pixel.

In the present embodiment, the second image sensor 103 includes a pixel capable of distance measurement and performs phase difference AF. However, the present invention is not limited to this. The second image sensor 103 also has the same pixel configuration as the first image sensor 100 having one PD under one microlens, and adopts a contrast AF method in which the contrast of the read moving image is detected and an AF operation is performed. It is also possible to adopt a configuration. That is, the focus detection unit can detect the contrast from the output of the pixel of the second image sensor and perform focus detection by a contrast detection method.

In addition, in the present embodiment, the A image data and B image data of the image sensor 103 are added by the image processing unit to generate a moving image, but this is not a limitation. In the case where A image data and B image data are not required, a configuration may be adopted in which A image data and B image data are added and output for some or all pixels in the image sensor. In the present embodiment, the operation of capturing a still image during moving image capturing has been described. However, the present invention is not limited to this. It is also possible to take a still image while not taking a movie.

(Embodiment 2)
Hereinafter, the configuration and operation of the imaging apparatus according to the second embodiment of the present invention will be described with reference to FIG. In the image pickup apparatus of FIG. 8, reference numeral 200 denotes a first image pickup element that converts an optical image into an electric signal. The image sensor 200 mainly shoots still images. An AFE 201 performs digital conversion on an analog image signal output from the image sensor 200 in accordance with gain adjustment or a predetermined quantization bit. Reference numeral 202 denotes a TG that controls the drive timing of the image sensor 200 and the AFE 201.

Reference numeral 203 denotes a second image sensor that converts an optical image into an electrical signal. The image sensor 203 mainly shoots moving images. Reference numeral 204 denotes an AFE that performs digital conversion on an analog image signal output from the image sensor 203 in accordance with gain adjustment or a predetermined quantization bit. Reference numeral 205 denotes a TG that controls the drive timing of the image sensor 203 and the AFE 204. Also in this embodiment, AFE 201 and TG 202 related to the first image sensor 200 and AFE 204 and TG 205 related to the second image sensor 203 are used. However, the AFE and TG are built in the image sensor. Is also possible.

The components indicated by 206 to 224 correspond to the components 106 to 124 of the first embodiment, respectively. The difference is as follows. The reflected light of the half mirror 207 that splits the incident light flux from the subject into reflected light and transmitted light is incident on the first image sensor 200, and the transmitted light is incident on the second image sensor 203.

FIG. 9 is a diagram illustrating the positions of the image sensor 200, the image sensor 203, and the half mirror 207. The image that has passed through the half mirror tends to become unclear due to the optical aberration of the half mirror. As will be described later, since a still image is captured by the image sensor 200 having more pixels than the image sensor 203, a clearer image is required. Therefore, in this embodiment, as described above, the half mirror 207 is disposed at a position and an angle at which the reflected light of the half mirror 207 is incident on the image sensor 200 and the transmitted light is incident on the image sensor 203. The distance c from the center of the half mirror 207 to the image sensor 200 is equal to the distance d to the image sensor 203. As a result, primary imaging, which is a subject image with the same magnification, is incident on the image sensor 200 and the image sensor 203. With this configuration, even when an AF operation is performed using an image signal of the image sensor 203 described later, the image formed on the image sensor 200 can be focused.

The configurations and functions of the first image sensor 200 and the second image sensor 203 are the same as the configurations and functions described in the first embodiment, and a description thereof will be omitted. In the configuration of the imaging device according to the present embodiment, since the imaging device 200 captures a still image and the imaging device 203 captures a moving image, the number of pixels h1 * i1 of the imaging device 200 is the number of pixels h2 of the imaging device 203. * It is larger than i2 pixels. Since the image sensor 203 has a smaller number of pixels than the image sensor 200, the area of the PD is increased, so that the sensitivity is high. The light beam split by the half mirror 207 is split at a ratio of M: N between the transmitted light and the reflected light, and the ratio M of the transmitted light incident on the imaging element 203 having a higher sensitivity is smaller than N.

Next, the operation of the imaging apparatus in the present embodiment will be described using the flowchart of FIG. First, it waits until the moving image photographing switch included in the operation unit 116 is pressed in step S201. When the moving image shooting switch is pressed, moving image shooting is started in step S202. When moving image shooting is started, the image pickup device 203, the AFE 204, and the TG 205 are turned on, and the CPU 224 performs moving image shooting setting. After the setting, the TG 205 outputs a read pulse to the image sensor 203 based on the synchronization signal output from the CPU 224, and the image sensor 203 starts a read operation at a predetermined frame rate. In the present embodiment, the charge accumulation / reading operation of the moving image is performed using the electronic shutter function based on the slit rolling operation, but this is not restrictive.

The A image photoelectric conversion unit data and the B image photoelectric conversion unit data output from the image sensor 203 are transferred to the RAM 218 by the CPU 224. Thereafter, the data is transferred to the image processing unit 221, and the data of the A image photoelectric conversion unit and the B image photoelectric conversion unit under the same microlens are added for each pixel. This creates a frame of the moving image. Thereafter, correction processing, compression, and the like are performed, and a moving image is displayed on the display unit 217 (live view). If moving image recording is selected using the menu displayed on the display unit 217 and the operation unit 216 before shooting, the moving image is sequentially recorded in the flash memory 223.

In step S203, it is determined whether the moving image shooting switch has been pressed again. If the movie shooting switch has not been pressed, movie shooting is continued, and the process of step S204 is performed. When the movie shooting switch is pressed, movie shooting is terminated. In step S204, AF calculation is performed. The CPU 224 performs AF processing on the A image data corresponding to the A image using the A image photoelectric conversion unit data stored in the RAM 218 and the B image data corresponding to the B image using the B image photoelectric conversion unit data. 222. FIG. 7A shows A image data and B image data in the case of FIG. The horizontal axis represents the pixel position, and the vertical axis represents the output. When the image is in focus, the A image data and the B image data match. FIG. 7B shows the A image data and the B image data in the case of FIG. 5B that is not in focus. At this time, the A image data and the B image data have a phase difference depending on the state described above, and the pixel position is shifted by the shift amount X. The AF calculation unit 222 calculates the shift amount X for each moving image frame, thereby calculating the focus shift amount, that is, the Y value in FIG. The AF calculation unit 222 transfers the calculated Y value to the focus drive circuit 212. In step S <b> 205, the focus drive circuit 212 calculates the amount of movement of the third lens group 208 based on the Y value acquired from the AF calculation unit 222, and outputs a drive command to the focus actuator 214. The third lens group 208 is moved to the in-focus position by the focus actuator 214 and is brought into focus by the image sensor 203. At this time, since the primary imaging with the same image plane magnification is incident on the imaging device 200 and the imaging device 203 and the depth of field is the same, imaging is performed in a state where the imaging device 203 is in focus. The element 200 is also in focus.

In step S206, it is determined whether a still image shooting switch included in the operation unit 216 has been pressed. If the still image shooting switch has been pressed, the process proceeds to step S207. In step S207, it is determined whether or not the focus drive, that is, the movement of the third lens group 208 is stopped. If not, wait until it stops. When stopped, a still image is taken. When still image shooting is started in step S208, the image sensor 200, AFE 201, and TG 202 are powered on, and the CPU 224 performs still image shooting settings. After the setting, the CPU 224 operates the focal plane shutter 206 to perform an exposure operation on the image sensor 200. Thereafter, based on the synchronization signal output from the CPU 224, the TG 202 outputs a read pulse to the image sensor 200, and the image sensor 200 starts a read operation. Image data output from the image sensor 200 is converted into digital data by the AFE 201 and then stored in the RAM 218. The CPU 224 transfers the image data stored in the RAM 218 to the image processing unit 220, and the image processing unit 220 performs image data correction processing, compression, and the like. Thereafter, the image data is recorded in the flash memory 223. Thereafter, the process returns to step S203, and the operations from step S203 to step S208 are repeated.

An outline of the above operation is shown in FIG. Between time t1 and t2, A image data and B image data are output as an operation of one frame of the moving image. The AF calculation described above is performed from the A image data and B image data acquired at times t1 and t2, and focus driving (movement of the third lens group 208) is performed at time t2. Focus drive ends at time t3. During moving image shooting, the operation from time t1 to t3 is repeated, and the AF operation is always performed. The still image shooting switch is pressed at time t4, but still image shooting is not started at time t4 because the focus is driven. After the focus drive ends at time t5, a synchronization signal for still image shooting is output at time t6 and still image shooting is started. At time t7, the focal plane shutter 206 travels in the order of the front curtain and the rear curtain, and the image sensor 200 is exposed. Thereafter, image data is output from the image sensor 200 at t8, and is recorded in the flash memory 223 as a still image after the above-described processing. In this way, by taking a still image after the focus drive is completed, it is possible to avoid taking an image that is not in focus during focus drive, and a clear still image can be obtained.

With the above operation, while performing moving image shooting (live view or moving image recording), the phase difference AF operation is performed to focus the image incident on the image sensor 203 or the image sensor 200, and still image shooting can be performed simultaneously. it can. Note that the present embodiment can be modified as described in the first embodiment.

(Embodiment 3)
Hereinafter, the configuration and operation of the imaging apparatus according to the third embodiment of the present invention will be described with reference to FIG. The configuration of the imaging apparatus according to the present embodiment illustrated in FIG. That is, the constituent elements indicated by 300 to 324 correspond to the constituent elements 100 to 124 of the first embodiment, respectively. The positional relationship among the image sensor 300, the image sensor 303, and the half mirror is the same as that in the first embodiment.

The first image sensor 300 of this embodiment will be described. The configuration of the image sensor 300 is different from the configuration of the image sensor 100 of the first embodiment for the pixel array unit. A pixel array portion of the image sensor 300 is shown in FIG. In FIG. 13A, 300a is a microlens, and 300c and 300b are PDs. One microlens is arranged at the top for each pixel and two PDs. When the area where the microlens 300a is shared is one pixel, this pixel is arranged with j1 pixels in the horizontal direction and k1 pixels in the vertical direction. The signals accumulated in the PD 300b and PD 300c are separately output to the outside by a read operation. Since another image having a phase difference is incident on the PD 300b and the PD 300c, the PD 300b is an A image photoelectric conversion unit and the PD 300c is a B image photoelectric conversion unit. In the present embodiment, two PDs are arranged for one microlens, but the present invention is not limited to this. The present invention can be applied to any configuration in which a plurality of PDs are arranged vertically or horizontally with respect to one microlens. As described above, in the present embodiment, the first image sensor also has focus detection pixels.

In the present embodiment, the A image data corresponding to the A image using the A image photoelectric conversion unit data output from the image sensor 300 and the B image data corresponding to the B image using the B image photoelectric conversion unit data. Is used as follows. When the A image data and the B image data are not in focus as described in the first embodiment, image data having a phase difference corresponding to the focus shift is obtained. FIGS. 15A and 15B are images when the same subject is photographed, FIG. 15A is A image data, and FIG. 15B is B image data. FIGS. 15A and 15B show data when the person is in focus, and it can be seen that the background is shifted (has a phase difference). That is, the phase difference between A image data and B image data depends on the distance to the subject. Therefore, the phase difference between the A image data and the B image data is replaced with a so-called parallax, and when the A image data and the B image data are displayed so as to be incident independently on the left and right eyes, the image is recognized as having a stereoscopic effect. be able to.

The imaging apparatus of the present embodiment has a 3D still image shooting mode in which A image data and B image data are made independent and recorded in a format that can be displayed as a three-dimensional image.

Next, the second image sensor 303 will be described. The configuration of the image sensor 303 is the same as that of the second image sensor 103 of the first embodiment. A pixel array of the image sensor 303 is shown in FIG. In FIG. 13B, 303a is a microlens, and 303b and 303c are PDs. One microlens is arranged at the top for each pixel and two PDs. When the area where the microlens 303a is shared is one pixel, this pixel is arranged with j2 pixels in the horizontal direction and k2 pixels in the vertical direction. The signals accumulated in the PD 303b and PD 303c are separately output to the outside by a read operation. Since another image having a phase difference is incident on the PD 303b and the PD 303c, here, the PD 303g is an A image photoelectric conversion unit, and the PD 303h is a B image photoelectric conversion unit. In this embodiment, two PDs are arranged for one microlens, but this is not a limitation.

Next, the operation of the imaging apparatus according to the present embodiment will be described using the flowchart of FIG. The operations in steps S301 to S307 in FIG. 14 are the same as the operations in steps S101 to S107 described in the first embodiment, and thus description thereof is omitted. In step S308, it is determined whether or not the 3D still image shooting mode is ON. If the 3D still image shooting mode is turned on using the menu displayed on the display unit 317 and the operation unit 316 before shooting, a 3D still image is shot in step S309. When 3D still image shooting is started, the image pickup device 300, the AFE 301, and the TG 302 are turned on, and the CPU 324 performs still image shooting setting. After the setting, the CPU 324 operates the focal plane shutter 306 to perform an exposure operation on the image sensor 300. Thereafter, based on the synchronization signal output from the CPU 324, the TG 302 outputs a read pulse to the image sensor 300, and the image sensor 300 starts a read operation.

By the reading operation, A image data and B image data are output from the image sensor 300. These are converted into digital data by the AFE 301 and then stored in the RAM 318. The CPU 324 transfers the A image data and the B image data stored in the RAM 318 to the image processing unit 320, and the image processing unit 320 performs correction processing, compression, and the like, respectively. Thereafter, the A image data and the B image data are recorded in the flash memory 323 in a predetermined file format.

If the 3D still image shooting mode is OFF in step S308, normal still image shooting is performed in step S310. When normal still image shooting is started, the image pickup device 300, the AFE 301, and the TG 302 are powered on, and the CPU 324 performs still image shooting setting. After the setting, the CPU 324 operates the focal plane shutter 306 to perform an exposure operation on the image sensor 300. Thereafter, based on the synchronization signal output from the CPU 324, the TG 302 outputs a read pulse to the image sensor 300, and the image sensor 300 starts a read operation. By the reading operation, A image data and B image data are output from the image sensor 300. These are converted into digital data by the AFE 301 and then stored in the RAM 318. The CPU 324 transfers the A image data and the B image data stored in the RAM 318 to the image processing unit 320, and the data of the A image photoelectric conversion unit and the B image photoelectric conversion unit under the same microlens for each pixel. to add. As a result, a normal still image is generated. Thereafter, correction processing, compression, and the like are performed, and the normal still image is recorded in the flash memory 323. Thereafter, the process returns to step S303, and the operations from step S303 to step S310 are repeated.

With the above operation, while performing moving image shooting (live view or moving image recording), the phase difference AF operation is performed to focus the image incident on the image sensor 303 or the image sensor 300, and at the same time, three-dimensional display is possible. You can take pictures.

In the present embodiment, the moving image is picked up by the image pickup device 303 and the three-dimensional still image is picked up by the image pickup device 300. However, the present invention is not limited to this. The moving image by the image sensor 300 may be configured to shoot a moving image that can be displayed three-dimensionally. Further, the AF operation may be performed from the pixel signals of both the image sensor 300 and the image sensor 303. For example, the image sensor 300 has an A image photoelectric converter and a B image photoelectric converter arranged side by side, and the image sensor 303 has an A image photoelectric converter and a B image photoelectric converter arranged vertically. Thus, a method of detecting the phase difference between the left and right directions and the up and down directions may be used. Further, in the present embodiment, the changes described in the first embodiment can be made.

As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

100... First image sensor 103, second image sensor 107, half mirror (light splitting means) 112, focus drive circuit (focus position changing means) 120, 121 image processing unit ( Image processing means), 122 .. AF calculation section (focus detection means)

Claims (10)

A first imaging element in which pixels including a photoelectric conversion unit are two-dimensionally arranged;
A second imaging element in which pixels including a photoelectric conversion unit are two-dimensionally arranged;
Light splitting means for causing the light beam incident on the common optical system to enter the first image sensor and the second image sensor, respectively;
Focus detection means for performing focus detection based on a signal from a pixel of the second image sensor;
A focus position changing means for changing a focus position of the common optical system based on a detection result of the focus detecting means;
Image processing means for processing signals from the pixels of the first image sensor and signals from the pixels of the second image sensor;
With
The second image sensor includes a first photoelectric conversion unit for receiving a light beam divided through the first region of the exit pupil of the common optical system, and / or the exit of the common optical system. Having a focus detection pixel including a second photoelectric conversion unit for receiving a light beam divided into pupils through a second region shifted from the first region of the pupil;
The first image sensor, the second image sensor, and the light dividing means are arranged so that an image is formed on the first image sensor and the second image sensor by the common optical system.
An image pickup apparatus, wherein a signal from a pixel of the first image sensor and a signal from a pixel of the second image sensor are each independently processed by the image processing means. The imaging apparatus according to claim 1, wherein the number of pixels of the second imaging element is smaller than the number of pixels of the first imaging element. The imaging apparatus according to claim 1, wherein the focus detection pixel includes a plurality of photoelectric conversion units for one microlens. The focus detection means includes
4. The imaging apparatus according to claim 1, wherein focus detection is performed by a phase difference detection method using an output of the focus detection pixel of the second imaging element. 5. The focus detection means includes
4. The image pickup apparatus according to claim 1, wherein contrast is detected from an output of a pixel of the second image pickup element, and focus detection is performed by a contrast detection method. 5. The first imaging element receives light transmitted through the light splitting unit,
6. The imaging apparatus according to claim 1, wherein reflected light of the light splitting unit is incident on the second imaging element. 7. The first imaging element receives the reflected light of the light splitting means,
The imaging apparatus according to claim 1, wherein the second imaging element receives light transmitted through the light splitting unit. 8. The intensity of the divided light beam incident on the second image sensor is lower than the intensity of the divided light beam incident on the first image sensor. 9. The imaging apparatus according to item 1. The first image sensor mainly has a function of photographing a still image,
The image pickup apparatus according to claim 1, wherein the second image pickup element mainly has a function of taking a moving image. The imaging apparatus according to claim 1, wherein the first imaging element has a focus detection pixel.

Images