JP7027133B2 - Focus detection device and focus detection method - Google Patents

Description

The present invention relates to a focus detection device and a focus detection method.

Conventionally, a focus detection device that performs phase difference type focus detection using a pixel signal generated by an image sensor is known. Among this type of focus detection device, a focus detection device that enables focus detection even during live view display is also known. For example, the image pickup apparatus proposed in Patent Document 1 generates an image by changing the exposure time of a row required for performing a phase difference type focus detection on an image pickup element to the exposure time of another row. The pixel signal required for this purpose and the pixel signal required for performing the focus detection of the phase difference method are generated.

Japanese Patent Application Laid-Open No. 2015-161906

The image pickup apparatus of Patent Document 1 changes the exposure time for each row of the image pickup element, and is basically configured on the premise of detecting the phase difference in the horizontal direction. Therefore, even if the method of the image pickup apparatus of Patent Document 1 is used as it is, it tends to take time to detect the phase difference in the vertical direction. Therefore, if an attempt is made to detect the phase difference in both the horizontal direction and the vertical direction in order to secure the focus detection performance, there is a possibility that sufficient time for the live view display cannot be secured during continuous shooting or the like.

The present invention has been made in view of the above circumstances, and provides a focus detection device and a focus detection method capable of securing time for live view display while ensuring focus detection performance of a phase difference method. The purpose is to do.

In order to achieve the above object, the focus detection device of the first aspect of the present invention photoelectrically converts each light beam passing through different emission pupil regions of an image pickup optical system for one microlens to obtain a pixel. It has a pixel portion in which a plurality of light receiving portions divided in a first pupil division direction and a second pupil division direction different from the first pupil division direction so as to generate a signal are arranged, and the first pupil division direction is described. An image pickup unit that outputs a focus detection pixel signal corresponding to the pupil division direction or the second pupil division direction, a display pixel signal for display not corresponding to pupil division, or a still image pixel signal for recording, and the above . An image to be processed for displaying an image based on a display pixel signal or a pixel signal to which the focus detection pixel signal is added, and a process for recording a still image on a recording medium based on the still image pixel signal. A processing circuit, a focus detection circuit that performs processing for focus detection based on the focus detection pixel signal, and still image shooting, which is an operation of recording a still image based on the still image pixel signal on the recording medium. While executing the image pickup unit, the image pickup unit is subjected to the first operation including the generation and reading of the focus detection pixel signal or the display pixel signal , and the second operation alternately during the still image shooting. The control circuit includes a control circuit for controlling and a light measuring unit for measuring the subject brightness, and the control circuit comprises the pixel signals generated by the plurality of pixel portions in the first operation regardless of the subject brightness. After being added in the second pupil division direction in the pixel portion, the addition is performed in the second pupil division direction without being added in the first pupil division direction among the plurality of pixel portions. The image pickup unit is controlled so that the focus detection pixel signal is generated by being thinned out or thinned out, and in the second operation when the subject brightness is high , the display pixel signal is generated and the subject brightness is increased. In the second operation when the value is low, the pixel signals generated by the plurality of pixel portions are added in the first pupil division direction in the pixel portion, and then between the plurality of pixel portions. The image pickup unit is controlled so that the focus detection pixel signal added in the first pupil division direction is generated , and the image processing circuit converts the display pixel signal into the display pixel signal when the subject brightness is high. It is the focus detection pixel signal generated by performing a process for displaying an image based on the above and adding the image in the first pupil division direction among the plurality of pixel portions when the subject brightness is low. The focus detection pixel signal paired in the second pupil division direction is added to the plurality of pixel portions. Performs processing for displaying an image based on the calculated pixel signal .
In order to achieve the above object, the focus detection device of the second aspect of the present invention photoelectrically converts each light beam passing through different emission pupil regions of an image pickup optical system into a pixel for one microlens. It has a pixel portion in which a plurality of light receiving portions divided in a first pupil division direction and a second pupil division direction different from the first pupil division direction so as to generate a signal are arranged, and the first pupil division direction is described. An image pickup unit that outputs a focus detection pixel signal corresponding to the pupil division direction or the second pupil division direction, a display pixel signal for display not corresponding to pupil division, or a still image pixel signal for recording, and the display. An image processing circuit that performs processing for displaying an image based on a pixel signal or a pixel signal to which the focus detection pixel signal is added, and processing for recording a still image on a recording medium based on the still image pixel signal. A focus detection circuit that performs processing for focus detection based on the focus detection pixel signal, and still image shooting, which is an operation of recording a still image based on the still image pixel signal on the recording medium, are executed, and the still image is taken. A control circuit that controls the image pickup unit so that the first operation and the second operation including the generation and reading of the focus detection pixel signal or the display pixel signal are alternately performed during image capture. The control circuit is generated by the plurality of pixel portions in the first operation regardless of the continuous shooting speed corresponding to the interval of still image shooting when the still image shooting is continuously executed. After the pixel signal is added in the second pupil division direction in the pixel portion, the second pupil is not added in the first pupil division direction among the plurality of pixel portions. The image pickup unit is controlled so that the focus detection pixel signal is generated by being added or thinned in the division direction, and the display pixel is used in the second operation when the continuous shooting speed exceeds a predetermined value. In the second operation in which the image pickup unit is controlled so that a signal is generated and the continuous shooting speed is equal to or less than the predetermined value, the pixel signal generated by the plurality of pixel units is the pixel signal. The image pickup unit so as to generate the focus detection pixel signal added in the first pupil division direction among the plurality of pixel units after being added in the first pupil division direction in the pixel unit. The image processing circuit performs processing for displaying an image based on the display pixel signal when the continuous shooting speed exceeds a predetermined value, and when the continuous shooting speed is equal to or less than the predetermined value. In addition, the focus detection pixel signal added in the first pupil division direction among the plurality of pixel portions. Therefore, processing is performed to display an image based on the pixel signal obtained by adding the focus detection pixel signals paired in the second pupil division direction in the plurality of pixel portions.

In order to achieve the above object, the focus detection method according to the third aspect of the present invention is a pixel by photoelectrically converting each light beam passing through different emission pupil regions of an image pickup optical system for one microlens. It has a pixel portion in which a plurality of light receiving portions divided in a first pupil division direction and a second pupil division direction different from the first pupil division direction so as to generate a signal are arranged, and the first pupil division direction is described. Focus using an image pickup unit that outputs a focus detection pixel signal corresponding to the pupil division direction or the second pupil division direction, a display pixel signal for display not corresponding to pupil division, or a still image pixel signal for recording. It is a detection method that measures the subject brightness and performs processing for displaying an image based on the display pixel signal or the pixel signal to which the focus detection pixel signal is added based on the subject brightness. That, processing for recording a still image on a recording medium based on the still image pixel signal, processing for focus detection based on the focus detection pixel signal, and processing for performing focus detection. While performing still image shooting, which is an operation of recording a still image based on the still image pixel signal on the recording medium, each of them generates and reads out the focus detection pixel signal or the display pixel signal during the still image shooting. The image pickup unit is controlled so as to alternately perform the first operation and the second operation including the above, and controlling the image pickup unit is the first operation regardless of the subject brightness . In the operation, after the pixel signal generated by the plurality of pixel portions is added in the second pupil division direction in the pixel portion, the first pupil division is performed among the plurality of pixel portions. The image pickup unit is controlled so that the focus detection pixel signal is generated by being added or thinned out in the second pupil division direction without being added in the direction, and the first image is obtained when the subject brightness is high . In the second operation, when the display pixel signal is generated in the second operation and the subject brightness is low, the pixel signal generated by the plurality of the pixel portions is the first in the pixel portion. This includes controlling the image pickup unit so that the focus detection pixel signal added for the first pupil division direction is generated among the plurality of pixel units after the addition in the pupil division direction. The process for displaying the image is the process for displaying the image based on the display pixel signal when the subject brightness is high, and the plurality of processes for displaying the image when the subject brightness is low. The focus detection pixel signal generated by being added in the first pupil division direction between the pixel portions. This includes processing for displaying an image based on a pixel signal obtained by adding the focus detection pixel signals paired in the second pupil division direction at the plurality of pixel portions .

According to the present invention, it is possible to provide a focus detection device and a focus detection method that can secure time for live view display while ensuring the focus detection performance of the phase difference method.

FIG. 1 is a block diagram showing an example of a configuration of an image pickup device including a focus detection device according to an embodiment of the present invention. FIG. 2 is a diagram showing the configuration of an example image sensor 208. FIG. 3 is a diagram showing a detailed configuration of the pixel portion. FIG. 4A is a diagram showing the addition of pixel signals when the pixels are treated as left and right aperture pixels. FIG. 4B is a diagram showing the addition of pixel signals when the pixels are treated as upper and lower aperture pixels. FIG. 5A is a flowchart showing the operation of the image pickup apparatus according to the embodiment of the present invention. FIG. 5B is a flowchart showing the operation of the image pickup apparatus according to the embodiment of the present invention. FIG. 6 is a flowchart showing imaging and readout processing for AF and LV display. FIG. 7 is a timing chart for explaining high-speed operation. FIG. 8 is a diagram showing an example of setting the pixel addition of the first operation in the high-speed operation. FIG. 9 is a diagram showing pixel data stored in the DRAM by the first operation in the high-speed operation. FIG. 10 is a diagram showing an example of setting the pixel addition of the second operation in the high-speed operation. FIG. 11 is a diagram showing pixel data stored in the DRAM by the second operation in the high-speed operation. FIG. 12 is a timing chart for explaining the low-luminance operation. FIG. 13 is a diagram showing an example of setting the pixel addition of the first operation in the low luminance operation. FIG. 14 is a diagram showing pixel data stored in the DRAM by the first operation in the low-luminance operation. FIG. 15 is a diagram showing an example of setting the pixel addition of the second operation in the low luminance operation. FIG. 16 is a diagram showing pixel data stored in the DRAM by the second operation in the low luminance operation.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of a configuration of an image pickup device including a focus detection device according to an embodiment of the present invention. In FIG. 1, the solid line with an arrow indicates the flow of data, and the broken line with an arrow indicates the flow of a control signal.

As shown in FIG. 1, the image pickup apparatus 1 includes an interchangeable lens 100 and a camera body 200. The interchangeable lens 100 is configured to be attached to and detached from the camera body 200. The interchangeable lens 100 and the camera body 200 are connected so that they can communicate with each other when the interchangeable lens 100 is attached to the camera body 200. The image pickup device 1 does not necessarily have to be an interchangeable lens type image pickup device. For example, the image pickup device 1 may be a lens-integrated image pickup device.

The interchangeable lens 100 includes an image pickup optical system 102, a drive unit 104, a lens CPU 106, and a lens side storage unit 108. Here, each block of the interchangeable lens 100 is composed of, for example, hardware. However, it does not necessarily have to be configured by hardware, and a part may be configured by software. Further, each block of the interchangeable lens 100 does not have to be composed of a single hardware or software, and may be composed of a plurality of hardware or software.

The image pickup optical system 102 is an optical system for forming a luminous flux from a subject on the image pickup element 208 of the camera body 200. The image pickup optical system 102 has a focus lens 1021 and a diaphragm 1022. The focus lens 1021 is configured so that the focal position of the imaging optical system 102 can be adjusted by moving in the optical axis direction.

The aperture 1022 is arranged on the optical axis of the focus lens 1021. The aperture of the aperture 1022 is variable. The aperture 1022 adjusts the amount of light flux from the subject that passes through the focus lens 1021 and is incident on the image pickup device 208. The drive unit 104 drives the focus lens 1021 and the aperture 1022 based on the control signal output from the lens CPU 106. Here, the image pickup optical system 102 may be configured as a zoom lens. In this case, the drive unit 104 also performs zoom drive.

The lens CPU 106 is configured to be able to communicate with the CPU 212 of the camera body 200 via the interface (I / F) 110. The lens CPU 106 controls the drive unit 104 according to the control of the CPU 212 of the camera body 200. Further, the lens CPU 106 transmits information such as the aperture value (F value) of the aperture 1022 and the lens information stored in the lens side storage unit 108 to the CPU 212 via the I / F 110. The lens CPU 106 does not necessarily have to be configured as a CPU. That is, the same function as the lens CPU 106 may be realized by an ASIC, FPGA, or the like. Further, the same function as the lens CPU 106 may be realized by software.

The lens-side storage unit 108 stores lens information regarding the interchangeable lens 100. The lens information includes, for example, information on the focal length of the imaging optical system 102 and information on aberrations.

The camera body 200 includes a mechanical shutter 202, a drive unit 204, an operation unit 206, an image sensor 208, an image stabilization circuit 210, a CPU 212, an image processing circuit 214, an image compression expansion unit 216, and a focus detection circuit. It has a 218, an exposure control circuit 220, a display unit 222, a bus 224, a DRAM 226, a main body side storage unit 228, and a recording medium 230. Here, each block of the camera body 200 is composed of, for example, hardware. However, it does not necessarily have to be configured by hardware, and a part may be configured by software. Further, each block of the camera body 200 may not be composed of a single hardware or software, or may be composed of a plurality of hardware or software.

The mechanical shutter 202 is configured to be openable and closable, and adjusts the incident time of the light flux from the subject on the image pickup element 208 (exposure time of the image pickup element 208). As the mechanical shutter 202, for example, a focal plane shutter is adopted. The drive unit 204 drives the mechanical shutter 202 based on the control signal from the CPU 212.

The operation unit 206 includes various operation buttons such as a power button, a release button, a moving image button, a play button, and a menu button, and various operation members such as a touch panel. The operation unit 206 detects the operation state of various operation members and outputs a signal indicating the detection result to the CPU 212.

The image pickup element 208 is arranged on the optical axis of the image pickup optical system 102, behind the mechanical shutter 202, and at a position where a light flux from the subject is imaged by the image pickup optical system 102. The image pickup device 208 takes an image of the subject and generates a pixel signal related to the subject. The image sensor 208 will be described in detail later.

The image stabilization circuit 210 moves the image sensor 208 in a direction parallel to the light receiving surface thereof so that the image stabilization generated in the camera body 200 is suppressed. By moving the image sensor 208 according to the camera shake, the blurring of the subject image generated in the image data due to the camera shake is suppressed. The image stabilization circuit may be provided in the interchangeable lens 100. The image stabilization circuit in this case is configured to move the image stabilization optical system included in the image pickup optical system 102.

The CPU 212 controls the entire camera body 200 according to a program stored in the body side storage unit 228. The CPU 212 controls, for example, image pickup by the image pickup device 208. Further, the CPU 212 outputs a control signal for driving the focus lens 1021 to the lens CPU 106 according to the focus state of the focus lens 1021 detected by the focus detection circuit 218. Further, the CPU 212 outputs the exposure set value calculated by the exposure control circuit 220 to the lens CPU 106 and the image pickup device 208. Here, the CPU 212 does not necessarily have to be configured as a CPU. That is, the same function as the CPU 212 may be realized by an ASIC, FPGA, or the like. Further, the same function as the CPU 212 may be realized by software.

The image processing circuit 214 performs various image processing on the pixel data. For example, when shooting a still image, the image processing circuit 214 performs image processing for recording a still image to generate still image data. Similarly, the image processing circuit 214 performs image processing for moving image recording to generate moving image data at the time of moving image shooting. Further, the image processing circuit 214 performs image processing for display at the time of live view display to generate display image data.

The image compression expansion unit 216 compresses the image data (still image data or moving image data) generated by the image processing circuit 214 at the time of recording the image data. Further, when the image data is reproduced, the image data recorded in the compressed state on the recording medium 230 is decompressed.

The focus detection circuit 218 detects the focus of the focus lens 1021 by a known phase difference method using the focus detection pixel data output from the image pickup element 208.

The exposure control circuit 220 as a photometric unit calculates an exposure set value based on the pixel data of the image sensor 208. The exposure control circuit 220 measures the subject brightness from the pixel data of the image pickup element 208, and calculates the exposure setting value necessary for adjusting the brightness of the subject at the time of shooting to an appropriate value from the measured subject brightness. The exposure set value includes the aperture amount (aperture value) of the aperture 1022 and the exposure time (shutter speed) of the image sensor 208.

The display unit 222 is a display unit such as a liquid crystal display or an organic EL display, and is arranged on the back surface of the camera body 200, for example. The display unit 222 displays an image under the control of the CPU 212. The display unit 222 is used for displaying a live view, displaying a recorded image, and the like.

The bus 224 is connected to an image sensor 208, a CPU 212, an image processing circuit 214, an image compression expansion unit 216, a focus detection circuit 218, an exposure control circuit 220, a display unit 222, a DRAM 226, a main body side storage unit 228, and a recording medium 230. It operates as a transfer path for transferring various data generated in these blocks.

The DRAM 226 is an electrically rewritable memory, and temporarily stores various data such as pixel data, still image data, moving image data, display image data, and processing data in the CPU 212 output from the image sensor 208. In addition, SDRAM may be used for temporary storage.

The main body side storage unit 228 stores various data such as a program used by the CPU 212 and adjustment values of the camera main body 200. The recording medium 230 is configured to be built in or loaded in the camera body 200, and records image data for recording as an image file in a predetermined format. The DRAM 226, the main body side storage unit 228, and the recording medium 230 may each be configured with one memory or the like, or may be configured by combining a plurality of memories or the like.

Next, the image pickup device 208 will be described. FIG. 2 is a diagram showing the configuration of an example image sensor 208. As shown in FIG. 2, the image pickup element 208 includes an input circuit 301, a control circuit 302, a pixel unit 303, a vertical scanning circuit 304, an analog processing circuit 305, an analog-to-digital conversion (ADC) processing circuit 306, and the like. It has a memory circuit 307, a horizontal scanning circuit 308, and an output circuit 309. Here, the imaging unit in the present embodiment is configured to have a pixel unit 303. The imaging unit may further include a vertical scanning circuit 304, an analog processing circuit 305, an ADC processing circuit 306, and a memory circuit 307. Further, the analog processing circuit 305 and the ADC processing circuit 306 may be provided outside the image pickup device 208.

The input circuit 301 receives a control signal related to the operation of the image pickup device 208 from the CPU 212 and inputs the control signal to the control circuit 302. The control signal related to this operation includes a synchronization signal (vertical synchronization signal and horizontal synchronization signal), a reference clock, and a signal for setting the operation of the image pickup device 208.

The control circuit 302 is composed of, for example, a CPU, an ASIC, or a logic circuit (digital circuit), and controls the operation of each part of the image pickup element 208 based on the control signal input from the input circuit 301.

A plurality of pixel units 303 are arranged two-dimensionally, and incident light is photoelectrically converted to generate a pixel signal. FIG. 3 is a diagram showing a detailed configuration of the pixel unit 303. Here, the left side of FIG. 3 shows a view of the pixel portion 303 as viewed from the front, and the right side of FIG. 3 shows a view of the pixel portion 303 as viewed from the side surface. As shown in FIG. 3, the pixel unit 303 has a microlens 303a, a color filter 303b, and a pixel 303c.

The microlens 303a concentrates the light flux that has passed through the image pickup optical system 102 on the light receiving portion of the corresponding pixel 303c. As will be described later, one pixel 303c is composed of four light receiving portions whose pupils are divided into two horizontal and two vertical. The microlens 303a concentrates the light flux that has passed through the different exit pupil regions of the image pickup optical system 102 on different light receiving units.

The color filter 303b is, for example, a color filter having a Bayer array of primary colors. In the primary color system bayer arrangement, rows in which red (R) filters and green (Gr) filters are alternately arranged and rows in which blue (B) filters and green (Gb) filters are alternately arranged are alternately arranged in the column direction. It is an arranged array. As shown in FIG. 3, one color of the color filter 303b corresponds to one pixel 303c. Therefore, light of the same color is incident on the four light receiving portions constituting the pixel 303c. The color filter 303b does not necessarily have to be a color filter having a Bayer array of primary colors.

The pixel 303c is composed of four light receiving portions lt, rt, lb, and rb which are divided and arranged in the horizontal direction which is the first pupil division direction and the vertical direction which is the second pupil division direction. ing. In the figure, the notation of "lt", "rt", "lb", and "rb" is added to "R", "Gr", "Gb", and "B" indicating the color of the color filter. .. The light receiving units lt, rt, lb, and rb receive the light flux emitted from the same portion of the subject. Each of the light receiving units lt, rt, lb, and rb is composed of, for example, a photodiode, and outputs a pixel signal based on the accumulated charge according to the received light flux. Here, the light receiving unit lt receives the light flux that has passed through the exit pupil region at the lower right of the image pickup optical system 102 among the emitted light flux. The light receiving unit rt receives the light flux that has passed through the exit pupil region at the lower left of the image pickup optical system 102 among the emitted light flux. The light receiving unit lb receives the light beam that has passed through the exit pupil region on the upper right of the imaging optical system 102 among the emitted light flux. The light receiving unit rb receives the light flux that has passed through the exit pupil region on the upper left of the image pickup optical system 102 among the emitted light flux.

Due to the structure in which the pixel 303c is divided into four light receiving portions lt, rt, lb, and rb, the pixel 303c can be treated as either a left-right aperture pixel or a top-bottom aperture pixel for phase difference detection. The left and right aperture pixels are a pair of a pixel that receives a light flux passing through the exit pupil region on the left side of the imaging optical system and a pixel that receives a light flux passing through the exit pupil region on the right side of the imaging optical system. The upper and lower aperture pixels are a pair of a pixel that receives a light flux that has passed through the upper exit pupil region of the imaging optical system and a pixel that receives a light flux that has passed through the lower exit pupil region of the imaging optical system. Whether the pixel 303c is treated as the left and right aperture pixels or the upper and lower aperture pixels is switched by the control signal from the control circuit 302. Further, it is possible to treat all four light receiving units lt, rt, lb, and rb as a total aperture pixel, and it can be set by a control signal from the control circuit 302. In this case, it is used as a pixel for still image shooting that does not require phase difference information.

When the pixel 303c is treated as a left and right opening pixel, the control circuit 302 is a pixel signal and a light receiving part of a light receiving part lt (Rlt, Grlt, Gblt, Blt) which is a light receiving part of the left half as shown on the left side of FIG. 4A. The pixel signals of lb (Rlb, Grlb, Gblb, Blb) are added (mixed) and output, and the pixel signals of the light receiving unit rt (Rrt, Grrt, Gbrt, Brt), which is the light receiving unit on the right half, and the light receiving unit. The pixel signal of rb (Rrb, Grrb, Gbrb, Brb) is added and output. As a result, as shown on the right side of FIG. 4A, the left aperture pixel signal l (Rl, Grl, Gbl, Bl) having the information of the light beam passing through the exit pupil region on the right side of the image pickup optical system and the left side of the image pickup optical system. Right aperture pixel signals r (Rr, Grr, Gbr, Br) having information on the light beam passing through the exit pupil region of the above are output, respectively.

When the pixel 303c is treated as an upper and lower opening pixel, the control circuit 302 is a pixel signal and a light receiving part of a light receiving part lt (Rlt, Grlt, Gblt, Blt) which is a light receiving part of the upper half as shown on the left side of FIG. 4B. The pixel signals of rt (Rrt, Grrt, Gbrt, Brt) are added (mixed) and output, and the pixel signals of the light receiving section lb (Rlb, Grlb, Gblb, Blb), which is the light receiving section of the lower half, and the light receiving section. The pixel signal of rb (Rrb, Grrb, Gbrb, Brb) is added and output. As a result, as shown on the right side of FIG. 4B, the upper aperture pixel signal t (Rt, Grt, Gbt, Bt) having the information of the luminous flux passing through the exit pupil region on the lower side of the image pickup optical system and the image pickup optical system. Lower aperture pixel signals b (Rb, Grb, Gbb, Bb) having information on the luminous flux passing through the upper exit pupil region are output, respectively. Further, when the pixel 303c is treated as a full-open pixel, the control circuit 302 has a pixel signal of the light receiving portion lt (Rlt, Grlt, Gblt, Blt) which is a light receiving portion of the left half and a light receiving portion lb (Rlb, Grlb, Gblb). , Blb), the pixel signal of the light receiving part rt (Rrt, Grrt, Gbrt, Brt) which is the light receiving part of the right half, and the pixel signal of the light receiving part rb (Rrb, Grrb, Gbrb, Brb) are added. And output.

Further, the pixel unit 303 of the present embodiment may be configured so that pixel signals can be added and output even between different pixel units 303. For example, the pixel unit 303 adds and outputs pixel signals of a plurality of left-opening pixels existing in the horizontal or vertical direction or between the right openings, or a plurality of upper openings existing in the horizontal or vertical direction or each other. It may be configured to add and output the pixel signals of the lower aperture pixels. The addition number of the pixel signals is set by, for example, the control circuit 302. By reading after adding the pixel signals, the reading time can be shortened. When the read time is sufficient, the pixel signals may not be added between the pixel units 303.

The vertical scanning circuit 304 resets the electric charge accumulated in the pixel 303c of the pixel unit 303 line by line each time it receives the accumulation start signal as a control signal from the control circuit 302, and then starts accumulating the electric charge in the pixel unit 303. Let me. Further, the vertical scanning circuit 304 ends the accumulation of electric charges in the pixel 303c of the pixel unit 303 one line at a time each time it receives the accumulation end signal as a control signal from the control circuit 302, and the accumulated electric charge is analog as a pixel signal. Transfer to the processing circuit 305.

The analog processing circuit 305 performs analog processing on the pixel signals sequentially transferred from the pixel unit 303. The analog processing circuit 305 includes, for example, a preamplifier that amplifies the pixel signal in an analog manner, a correlated double sampling (CDS) processing circuit that removes reset noise from the pixel signal, and the like.

The ADC processing circuit 306 converts the pixel signal output from the analog processing circuit 305 into pixel data, which is a digital signal, according to the control signal from the control circuit 302. The ADC processing circuit 306 is configured as, for example, a column-type ADC processing circuit.

The memory circuit 307 temporarily stores the pixel data output from the ADC processing circuit 306 according to the control signal from the control circuit 302. The memory circuit 307 is composed of a volatile memory or the like. The memory circuit 307 may be configured to perform digital addition of pixel data. In this case, the memory circuit 307 is configured to store the added value of the pixel data output from the ADC processing circuit 306.

The horizontal scanning circuit 308 receives the control signal from the control circuit 302 and transfers the pixel data stored in the memory circuit 307 to the output circuit 309 in column order.

The output circuit 309 arranges the pixel data transferred by the horizontal scanning circuit 308 to generate a pixel data string. Further, the output circuit 309 converts the pixel data string into a predetermined output signal format such as a serial signal and a differential signal and outputs the signal.

Hereinafter, the operation of the image pickup apparatus 1 according to the present embodiment will be described. 5A and 5B are flowcharts showing the operation of the image pickup apparatus according to the present embodiment. Here, FIGS. 5A and 5B show the operation when the AF mode of the image pickup apparatus 1 is the continuous AF mode. The continuous AF mode is an AF mode suitable for a moving subject, and is an AF mode that keeps focusing so as to follow the subject.

The operations of FIGS. 5A and 5B are started when the power-on operation of the image pickup apparatus 1 by the user is detected. When the power-on operation is detected, in step S101, the CPU 212 determines whether or not the 1st release switch of the release button is in the on state. The 1st release switch is, for example, a switch that is turned on in response to a half-press operation of the release button by the user. When it is determined in step S101 that the 1st release switch is in the ON state, the process proceeds to step S104. When it is determined in step S101 that the 1st release switch is not in the ON state, the process proceeds to step S102.

In step S102, the CPU 212 captures display pixel data for live view (LV) display. At this time, the CPU 212 outputs a control signal to the drive unit 204 so that the mechanical shutter 202 is fully opened, and controls the lens CPU 106 to drive the aperture 1022 by a predetermined amount (for example, an open aperture). Output a signal. After that, the CPU 212 outputs a control signal to the image pickup element 208 to start imaging for LV display by the image pickup element 208. Then, each time the imaging for LV display is completed, the control circuit 302 starts reading the pixel signal from the pixel unit 303. When reading out the pixel signal, the control circuit 302 may add the pixel signals of the same aperture (same color) output from the pixel 303c of the pixel unit 303. The display pixel data output from the image sensor 208 is stored in the DRAM 226.

In step S103, the CPU 212 displays the LV. At this time, the CPU 212 causes the image processing circuit 214 to generate display image data. In response to this, the image processing circuit 214 performs necessary processing on the display pixel data to generate display image data for display. The display image data is obtained by adding and averaging the pixel data of the light receiving units lt, rt, lb, and rb belonging to the same pixel unit 303. The CPU 212 causes the display unit 222 to display an LV image based on the display image data generated by the image processing circuit 214. After that, the process proceeds to step S126.

In step S104, the CPU 212 performs imaging and reading for autofocus (AF) and LV display. The imaging and readout processing for AF and LV display in step S104 will be described in detail later. Here, it is assumed that the focus detection pixel data for AF is stored in the DRAM 226 by imaging and readout for AF, and the display pixel data for display is stored in the DRAM 226 by imaging and readout for LV. continue.

In step S105, the CPU 212 displays the LV.

In step S106, the CPU 212 causes the focus detection circuit 218 to execute the focus detection operation. The focus detection circuit 218 performs a correlation calculation using the focus detection pixel data paired with the focus detection pixel data stored in the DRAM 226. The paired focus detection pixel data in the case of horizontal phase difference detection is the left aperture pixel data l and the right aperture pixel data r, and the paired focus detection pixel data in the case of vertical phase difference detection is above. The opening pixel data t and the lower opening pixel data b. After the correlation calculation, the focus detection circuit 218 determines the reliability of the focus detection. The reliability determination is determined based on, for example, the contrast obtained from the pixel data and the correlation value calculated as a result of the correlation calculation.

In step S107, the focus detection circuit 218 calculates the out-of-focus amount. That is, the focus detection circuit 218 is focused on the focus lens 1021 from the two image spacing values (the amount of image shift corresponding to the extreme value of the correlation value) in the focus detection region determined to be highly reliable as a result of the reliability determination in step S106. Calculate the amount of focus shift with respect to the in-focus position of. After that, the process proceeds to step S108.

In step S108, the focus detection circuit 218 performs an area selection process for selecting a focus detection region corresponding to the focus lens position used to drive the focus lens 1021. After the area selection process, the process proceeds to step S109. The area selection process is performed, for example, by selecting a focus detection area that indicates the amount of focus shift corresponding to the focus lens position corresponding to the closest (that is, the closest) focus lens position.

In step S109, the CPU 212 determines whether or not the focus lens 1021 is in the focused state. The determination in step S109 is performed, for example, by determining whether or not the amount of focus shift in the focus detection region selected in the area selection process is within a predetermined allowable range. When the amount of focus shift is within the allowable range, it is determined that the subject is in focus. When it is determined in step S109 that the focus lens 1021 is not in the in-focus state, the process proceeds to step S110. When it is determined in step S109 that the focus lens 1021 is in the in-focus state, the process proceeds to step S111.

In step S110, the CPU 212 outputs a control signal to the lens CPU 106 so that the focus lens 1021 is driven according to the focus lens position calculated for the focus detection region selected in step S108. The lens CPU 106 receives this control signal and drives the focus lens 1021 via the drive unit 104. After that, the process proceeds to step S126.

In step S111, the CPU 212 performs imaging and reading for autofocus (AF) and LV display similar to those in step S104. Here, the description will be continued assuming that the focus detection pixel data is stored in the DRAM 226 by imaging and readout for AF, and the display pixel data is stored in the DRAM 226 by imaging and readout for LV.

In step S112, the CPU 212 displays the LV.

In step S113, the CPU 212 causes the focus detection circuit 218 to execute the focus detection operation. The focus detection circuit 218 performs a correlation calculation using the focus detection pixel data paired with the focus detection pixel data stored in the DRAM 226. After the correlation calculation, the focus detection circuit 218 determines the reliability of the focus detection. In step S114, the focus detection circuit 218 performs a focus shift amount calculation. In step S115, the focus detection circuit 218 performs an area selection process.

In step S116, the focus detection circuit 218 stores the information related to the focus detection as history information in, for example, DRAM 226. The information related to focus detection includes, for example, information on the amount of focus shift calculated in step S114 and information on the focus detection region selected in step S115.

In step S117, the CPU 212 determines whether or not the 2nd release switch is turned on. The 2nd release switch is a switch that is turned on in response to, for example, a user pressing the release button fully. When it is determined in step S117 that the 2nd release switch is not in the ON state, the process proceeds to step S118. When it is determined in step S117 that the 2nd release switch is in the ON state, the process proceeds to step S120.

In step S118, the CPU 212 determines whether or not the focus lens 1021 is in the focused state. When it is determined in step S118 that the focus lens 1021 is not in the in-focus state, the process proceeds to step S119. When it is determined in step S118 that the focus lens 1021 is in the in-focus state, the process proceeds to step S125.

In step S119, the CPU 212 outputs a control signal to the lens CPU 106 so that the focus lens 1021 is driven according to the focus lens position calculated for the focus detection region selected in step S115. The lens CPU 106 receives this control signal and drives the focus lens 1021 via the drive unit 104. After that, the process proceeds to step S125.

In step S120, the CPU 212 causes the focus detection circuit 218 to execute the motion prediction operation. In response to this, the focus detection circuit 218 performs a moving object prediction operation. The moving object prediction calculation is a process of predicting the position to be driven of the focus lens 1021 this time from the history of the result (focus lens position) of the past focus shift amount calculation stored in step S116.

In step S121, the CPU 212 starts the operation of the mechanical shutter 202 in order to perform imaging (main exposure) for acquiring a still image. The operation of the mechanical shutter 202 includes an opening / closing operation of the mechanical shutter 202 before and after the main exposure, and a full opening operation of the mechanical shutter 202 for starting image pickup for live view and AF after the main exposure. First, the CPU 212 switches the control signal of the drive unit 204 so that the mechanical shutter 202 is fully closed. Then, after performing the main exposure in step S123, the CPU 212 controls the drive unit 204 so that the mechanical shutter 202 is fully opened.

In step S122, the CPU 212 instructs the lens CPU 106 to simultaneously drive the focus lens 1021 and the aperture 1022, and starts the operation. Here, the drive position of the focus lens 1021 is the position predicted in the moving object prediction calculation in step S120. Further, the aperture amount of the aperture 1022 is an aperture amount corresponding to the aperture value calculated based on the subject brightness measured by the immediately preceding photometric calculation.

In step S123, the CPU 212 starts the main exposure. This exposure is an image pickup for acquiring image data for recording. In the main exposure, the CPU 212 starts the imaging of the image pickup device 208. After the end of the exposure period, the control circuit 302 reads a pixel signal as a still image pixel signal from each light receiving unit of the image pickup element 208. After reading the still image pixel signal, the CPU 212 causes the image processing circuit 214 to perform a process for generating an image pixel signal for recording. In response to this, the image processing circuit 214 performs processing necessary for generating image data for recording to generate still image data for recording. After the image processing is completed, the CPU 212 compresses the still image data for recording by the image compression expansion unit 216. After the compression is completed, the CPU 212 records the compressed still image data for recording as an image file on the recording medium 230.

In step S124, the CPU 212 instructs the lens CPU 106 to open the aperture 1022.

In step S125, the CPU 212 determines whether or not the 1st release switch is in the ON state. When it is determined in step S125 that the 1st release switch is in the ON state, the process returns to step S111. After that, if the 2nd release switch is on, still image shooting will be performed continuously. When it is determined in step S125 that the 1st release switch is not in the ON state, the process proceeds to step S126.

In step S126, the CPU 212 determines whether or not to turn off the power of the camera body 200. For example, it is determined that the power is turned off when the power is instructed to be turned off by the operation of the user's operation unit 206 or when the user's operation unit 206 is not operated for a predetermined time. When it is determined in step S126 that the power of the camera body 200 is not turned off, the process returns to step S101. When it is determined in step S126 that the power of the camera body 200 is turned off, the process ends.

Next, the processing of step S104 and step S111 will be described. FIG. 6 is a flowchart showing imaging and reading processing for AF and LV display in steps S104 and S111. In the present embodiment, different imaging and readout are performed depending on whether the pixel signal needs to be read out at high speed and the focus detection performance is more important than reading out the pixel signal at high speed.

In step S201, the control circuit 302 determines whether or not the high-speed read condition is satisfied. The determination as to whether or not the high-speed read condition is satisfied is determined based on, for example, the information sent from the CPU 212. This information includes the continuous shooting speed setting, aperture value, subject brightness, and history of focusing results. For example, when all of the following conditions (1)-(3) are satisfied or the condition of (4) is satisfied, it is determined that the high-speed read condition is satisfied. When it is determined in step S201 that the high-speed read condition is satisfied, the process proceeds to step S202. When it is determined in step S201 that the high-speed read condition is not satisfied, the process proceeds to step S203.
(1) Continuous shooting speed (frame rate) is equal to or higher than a predetermined value (2) Aperture value (F number) is equal to or lower than a predetermined value (3) Subject brightness is equal to or higher than a predetermined value (4) The amount of focus shift cannot be detected continuously a predetermined number of times. The continuous shooting speed, which is the condition of (1), corresponds to the interval of continuous shooting. When the frame rate of the main exposure is high, that is, the interval between continuous shots is short, it is desirable to shorten the time required for AF by performing high-speed reading while shortening the exposure time for AF. The aperture value, which is the condition of (2), is represented by, for example, an F number. When the aperture value is not more than a predetermined value, it becomes easy to acquire a bright image, so that the focus detection performance can be ensured even if the exposure time for AF is shortened. When the subject brightness, which is the condition of (3), is equal to or higher than a predetermined value, it becomes easy to acquire a bright image, so that the focus detection performance can be ensured even if the exposure time for AF is shortened. Regarding the subject brightness, it is necessary to determine at least whether or not the subject brightness in the focus detection area selected by the area selection process is equal to or higher than a predetermined value. It may be determined whether or not the subject brightness in the all-focus detection region is equal to or higher than a predetermined value, or the subject brightness in the peripheral focus detection region including the focus detection region selected in the area selection process is equal to or higher than the predetermined value. It may be determined whether or not. It was determined that the determination as to whether or not the amount of focus shift, which is the condition of (4), could be detected is unreliable, for example, when the contrast of the subject in each focus detection region is low or when the minimum value of the correlation value is large. Sometimes it is determined that the out-of-focus amount could not be detected.

In step S202 when it is determined in step S201 that the high-speed read condition is satisfied, the control circuit 302 operates at high speed. After the high-speed operation ends, the process ends.

The high-speed operation will be described below. FIG. 7 is a timing chart for explaining high-speed operation. Here, “VD” in FIG. 7 indicates the timing of the synchronization signal (vertical synchronization signal) input to the control circuit 302. VD (1) is a vertical synchronization signal indicating the timing of the first operation including imaging and readout for AF, and VD (2) is a second operation including imaging and readout for LV display. It is a vertical synchronization signal indicating timing. In the high-speed operation, the first operation and the second operation are alternately performed in response to the input of the vertical synchronization signal. Then, the first operation and the second operation are performed once in one frame. In this case, the processing of steps S104 and S111 is performed for each frame. The length of one frame is determined according to the update interval of the LV display screen, the continuous shooting speed indicating the continuous shooting interval at the time of continuous shooting, and the like. Hereinafter, each of the first operation and the second operation in the high-speed operation will be described in detail.

First, the first operation in high-speed operation will be described. The first operation shown by AF (rl) in FIG. 7 is an operation for generating and reading a focus detection pixel signal with respect to the first pupil division direction. The first pupil division direction is, for example, the horizontal direction of the pixel unit 303. Prior to the first operation, the control circuit 302 outputs the left aperture pixel signal l (Rl, Grl, Gbl, Bl) and the right aperture pixel signal r (Rr, Grr, Gbr, Br) from the pixel unit 303. The setting of the pixel unit 303 is switched so as to be. Further, the control circuit 302 sets the pixel addition between the pixel units 303 in order to shorten the pixel signal reading time and improve the S / N.

Here, the setting of pixel addition will be described. In the following description, the pixel addition setting in the pixel unit 303 is represented by adding "H" indicating the horizontal direction or "V" indicating the vertical direction to n / m. Here, n is the number of pixel signals used for addition. Further, m is the number of pixel signals to be added. For example, in the case of V1 / 9 pixel addition, one of nine pixel signals of the same aperture arranged in the vertical direction is added, that is, only one of the nine pixel signals is read, and the remaining eight. Represents thinning out. Further, in the case of V5 / 9 pixel addition, it is shown that 5 out of 9 pixel signals of the same aperture arranged in the vertical direction are added and read, and the remaining 4 are thinned out. Further, in the case of H1 / 1 pixel addition, it is shown that the pixel signal is read out without addition and thinning in the horizontal direction.

FIG. 8 shows an example of setting the pixel addition of the first operation in the high-speed operation. In the first operation shown in FIG. 8, the addition is not performed in the horizontal direction which is the first pupil division direction, and the addition is performed only in the vertical direction which is the second pupil division direction. .. In the example of FIG. 8, the pixel signals of the same aperture (left openings or right openings) are set to add 1/1 pixel (that is, without addition) in the horizontal direction, and the same aperture (left) in the vertical direction. It is set to add 5/9 pixels to the pixel signals of (opens to each other or right openings to each other). The addition number of the pixel signals is appropriately set according to, for example, the frame rate.

The number of rows of the pixel signal is compressed by the setting shown in FIG. By reducing the number of rows of the pixel signal, the read time of the pixel signal is shortened. On the other hand, since the number of columns of the pixel signal does not change, the detection accuracy of the phase difference in the horizontal direction can be ensured. Further, as will be described later, in the second operation in the high-speed operation, the focus detection pixel signal regarding the second pupil division direction is not generated and read in order to secure the display quality at the time of display. Therefore, in the high-speed operation, the focus of the subject is detected only by the focus detection pixel signal obtained in the first operation. Here, in the first operation in the high-speed operation, the addition number in the vertical direction is suppressed to 5 pixels or less. By suppressing the addition number in the vertical direction in this way, information on the phase difference in the vertical direction remains in the pixel signal. The focus detection pixel signal generated based on such a pixel signal will have information on the phase difference in the substantially oblique direction. In the example of FIG. 8, five pixel signals are set to be added in the vertical direction, but the number of additions in the vertical direction may be 5 pixels or less.

After the setting as shown in FIG. 8, the CPU 212 outputs a control signal to the image pickup device 208 so as to perform imaging at the exposure time required for generating the focus detection pixel signal. This exposure time is set based on the subject brightness and the like.

Upon receiving the input of the control signal from the CPU 212, the control circuit 302 starts imaging (charge accumulation) for each row of the pixel unit 303. Then, the control circuit 302 controls the vertical scanning circuit 304 to sequentially output pixel signals from the pixel unit 303 in the row where the imaging is completed.

Here, the detection of the phase difference in the horizontal direction is performed by using, for example, a pair of Gr's left-opening pixel signal Grl and right-opening pixel signal Grr and a pair of Gb's left-opening pixel signal Gbl and right-opening pixel signal Gbr. That is, the pair of the left aperture pixel signal Rl and the right aperture pixel signal Rr of R and the pair of the left aperture pixel signal Bl and the right aperture pixel signal Br of B do not necessarily have to be read out. Therefore, at the time of reading, as shown in FIG. 9, only the pair of the left aperture pixel signal Grl and the right aperture pixel signal Grr and the pair of the left aperture pixel signal Gbl and the right aperture pixel signal Gbr are packed and read out. May be good. Of course, the pair of the left aperture pixel signal Rl and the right aperture pixel signal Rr and the pair of the left aperture pixel signal Gbl and the right aperture pixel signal Gbr may also be read together.

The pixel signal read from the pixel unit 303 is analog-processed in the analog processing circuit 305. The analog-processed pixel signal is converted into pixel data, which is a digital signal, in the ADC processing circuit 306. The pixel data is stored in the memory circuit 307. As described above, the addition shown in FIG. 8 may be performed in the memory circuit 307.

The horizontal scanning circuit 308 receives the control signal from the control circuit 302 and transfers the pixel data stored in the memory circuit 307 to the output circuit 309 in column order. The output circuit 309 arranges the pixel data transferred by the horizontal scanning circuit 308 to generate a pixel data string, converts the generated pixel data string into a predetermined output signal format such as a serial signal and a differential signal, and outputs the data. do. This pixel data string is stored in the DRAM 226 as shown in FIG. 9, for example. In this way, the first operation is completed.

The pixel data string stored in the DRAM 226 by the first operation is used for the correlation calculation for calculating the out-of-focus amount. The focus detection circuit 218 performs a correlation calculation using a pair of left aperture pixel data Grl and right aperture pixel data Grr and a pair of left aperture pixel data Gbl and right aperture pixel data Gbr stored in the DRAM 226.

Next, the second operation in the high-speed operation will be described. The second operation shown by LV in FIG. 7 is an operation for generating and reading (LV) a display pixel signal. In the high-speed operation, the focus detection pixel signal for the second pupil division direction, which will be described later, is not generated and read out. Prior to the second operation, the control circuit 302 outputs the upper aperture pixel signal t (Rt, Grt, Gbt, Bt) and the lower aperture pixel signal b (Rb, Grb, Gbb, Bb) from the pixel unit 303. The setting of the pixel unit 303 is switched so as to be. Further, the control circuit 302 sets the pixel addition between the pixel units 303 in order to shorten the pixel signal reading time.

FIG. 10 shows an example of setting the pixel addition of the second operation in the high-speed operation. In the second operation in the high speed operation, the pixel signal is used only for generating the display pixel signal. The display image data is obtained by adding and averaging the pixel data of the light receiving units lt, rt, lb, and rb belonging to the same pixel unit 303. Therefore, the control circuit 302 is set to add the upper and lower aperture pixel signals t and b (pixel signals connected by “+” in the figure) and output them from the pixel unit 303. Further, the control circuit 302 sets the pixel addition between the pixel units 303 in order to shorten the pixel signal reading time. As shown in FIG. 10, in the second operation in the high-speed operation, the addition is set to be performed in both the horizontal direction which is the first pupil division direction and the vertical direction which is the second pupil division direction. .. In the example of FIG. 10, it is set to add H3 / 3 pixels in the horizontal direction and to add V3 / 3 pixels in the vertical direction.

By the setting of FIG. 10, the number of rows and the number of columns of the pixel signal are each compressed to about one-third. By reducing the number of rows and columns of the pixel signal, the read time of the pixel signal is shortened. The number of pixel signals to be added is not limited to this. However, in the second operation, the image quality at the time of display deteriorates as the number of additions of the pixel signals increases. Therefore, it is desirable that the number of pixel signal additions is appropriately determined by the trade-off between the image quality at the time of display and the read time (frame rate). In FIG. 10, the control circuit 302 is set to add the upper and lower opening pixel signals t and b (pixel signals connected by “+” in the figure) and output them from the pixel unit 303. The left and right opening pixel signals l and r may be added and set to be output from the pixel unit 303. Further, the control circuit 302 is set to output the full aperture pixel signal from the pixel unit 303, and even if the pixel signal obtained by adding the pixel signals connected by “+” in FIG. 10 is output from the pixel unit 303. good.

After the setting as shown in FIG. 10, the CPU 212 outputs a control signal to the image pickup device 208 so as to perform imaging at the exposure time required for generating the display pixel signal. This exposure time is set based on the subject brightness and the like.

Upon receiving the input of the control signal from the CPU 212, the control circuit 302 starts imaging (charge accumulation) for each row of the pixel unit 303. Then, the control circuit 302 controls the vertical scanning circuit 304 to sequentially output pixel signals from the pixel unit 303 in the row where the imaging is completed.

The pixel signal read from the pixel unit 303 is analog-processed in the analog processing circuit 305. The analog-processed pixel signal is converted into pixel data, which is a digital signal, in the ADC processing circuit 306. The pixel data is stored in the memory circuit 307. In the second operation, pixel signals of R, Gr, Gb, and B are required for the display image data. Therefore, in the memory circuit 307, the pixel signal R which is an addition signal of the upper opening pixel signal Rt and the lower opening pixel signal Rb, the pixel signal Gr which is the addition signal of the upper opening pixel signal Grt and the lower opening pixel signal Grb, and the upper The pixel signal Gb, which is an addition signal of the opening pixel signal Gbt and the lower opening pixel signal Gbb, and the pixel signal B, which is an addition signal of the upper opening pixel signal Bt and the lower opening pixel signal Bb, are stored without being packed.

The horizontal scanning circuit 308 receives the control signal from the control circuit 302 and transfers the pixel data stored in the memory circuit 307 to the output circuit 309 in column order. The output circuit 309 arranges the pixel data transferred by the horizontal scanning circuit 308 to generate a pixel data string, converts the generated pixel data string into a predetermined output signal format such as a serial signal and a differential signal, and outputs the data. do. This pixel data string is stored in the DRAM 226 as shown in FIG. 11, for example. In this way, the second operation is completed.

The pixel data string stored in the DRAM 226 by the second operation is used for the LV display. The image processing circuit 214 generates display image data from the pixel data R, Gr, Gb, and B stored in the DRAM 226.

In the high-speed operation as described above, the read time can be shortened. Further, in the second operation in the high-speed operation, the focus detection pixel signal is not generated and read out. Therefore, in the second operation, the time for the generation and reading of the focus detection pixel signal can be used for the generation and reading for the display pixel signal. Therefore, it is possible to maintain high display quality while enabling high-speed operation. On the other hand, since the addition number in the vertical direction is suppressed in the first operation, it is possible to accurately detect the focus of the subject only by detecting the phase difference in the horizontal direction.

Here, the explanation returns to FIG. In step S203 when it is determined in step S201 that the high-speed read condition is not satisfied, the control circuit 302 performs a low-luminance operation. After the low-luminance operation ends, the process ends.

Hereinafter, the low-luminance operation will be described. FIG. 12 is a timing chart for explaining the low-luminance operation. Here, “VD” in FIG. 12 indicates the timing of the synchronization signal (vertical synchronization signal) input to the control circuit 302 as in FIG. 7. VD (1) is a vertical synchronization signal indicating the timing of the first operation including imaging and readout for AF, and VD (2) is a second operation including imaging and readout for LV display. It is a vertical synchronization signal indicating timing. Even in the low-luminance operation, the first operation and the second operation are alternately performed in response to the input of the vertical synchronization signal. Then, the first operation and the second operation are performed once in one frame. In this case, the processing of steps S104 and S111 is performed for each frame. Hereinafter, each of the first operation and the second operation in the low-luminance operation will be described in detail.

First, the first operation in the low-luminance operation will be described. The first operation shown by AF (rl) in FIG. 12 is an operation for generating and reading a focus detection pixel signal with respect to the first pupil division direction similar to the first operation in the high-speed operation. Prior to the first operation, the control circuit 302 outputs the left aperture pixel signal l (Rl, Grl, Gbl, Bl) and the right aperture pixel signal r (Rr, Grr, Gbr, Br) from the pixel unit 303. The setting of the pixel unit 303 is switched so as to be. Further, the control circuit 302 sets the pixel addition between the pixel units 303 in order to shorten the pixel signal reading time.

FIG. 13 shows an example of setting the pixel addition of the first operation in the low-luminance operation. As shown in FIG. 13, in the first operation, the addition is not performed in the horizontal direction which is the first pupil division direction, and the addition is performed only in the vertical direction which is the second pupil division direction. Will be done. In the example of FIG. 13, the pixel signals of the same openings (left openings or right openings) are set to be added by H1 / 1 pixels in the horizontal direction, and the same openings (left openings or right openings) are added in the vertical direction. ) Is set to add V9 / 9 pixels. The number of additions of the pixel signals in the vertical direction is appropriately set according to, for example, the frame rate.

By the setting of FIG. 13, the number of rows of the pixel signal is compressed to 1/9. By reducing the number of rows of the pixel signal, the read time of the pixel signal is shortened. On the other hand, since the number of columns of the pixel signal does not change, the detection accuracy of the phase difference in the horizontal direction is ensured. Further, as will be described later, in the second operation in the low-luminance operation, the focus detection pixel signal with respect to the second pupil division direction is generated and read out. Therefore, in the first operation in the low-luminance operation, it is sufficient that the phase difference in the horizontal direction can be detected with high accuracy. Therefore, in the first operation in the low-luminance operation, the S / N is improved by increasing the addition number in the vertical direction.

After the setting as shown in FIG. 13, the CPU 212 outputs a control signal to the image pickup device 208 so as to perform imaging at the exposure time required for generating the focus detection pixel signal. This exposure time is set based on the subject brightness and the like.

Upon receiving the input of the control signal from the CPU 212, the control circuit 302 starts imaging (charge accumulation) for each row of the pixel unit 303. Then, the control circuit 302 controls the vertical scanning circuit 304 to sequentially output pixel signals from the pixel unit 303 in the row where the imaging is completed.

Here, the detection of the phase difference in the horizontal direction is performed by using, for example, a pair of Gr's left-opening pixel signal Grl and right-opening pixel signal Grr and a pair of Gb's left-opening pixel signal Gbl and right-opening pixel signal Gbr. That is, the pair of the left aperture pixel signal Rl of R and the right aperture pixel signal Rr and the pair of the left aperture pixel signal Bl of B and the right aperture pixel signal Br do not necessarily have to be read out. On the other hand, as shown in FIG. 14, a pair of the left aperture pixel signal Rl and the right aperture pixel signal Rr and a pair of the left aperture pixel signal Gbl and the right aperture pixel signal Gbr may also be read out.

The pixel signal read from the pixel unit 303 is analog-processed in the analog processing circuit 305. The analog-processed pixel signal is converted into pixel data, which is a digital signal, in the ADC processing circuit 306. The pixel data is stored in the memory circuit 307. As described above, the addition shown in FIG. 13 may be performed in the memory circuit 307. Further, in the first operation of the low-luminance operation, the pixel data of the same aperture vertically adjacent to each other read from the image pickup element 208 may be further added in, for example, the memory circuit 307. As a result, the S / N can be further improved.

The horizontal scanning circuit 308 receives the control signal from the control circuit 302 and transfers the pixel data stored in the memory circuit 307 to the output circuit 309 in column order. The output circuit 309 arranges the pixel data transferred by the horizontal scanning circuit 308 to generate a pixel data string, converts the generated pixel data string into a predetermined output signal format such as a serial signal and a differential signal, and outputs the data. do. This pixel data string is stored in the DRAM 226 as shown in FIG. 14, for example. In this way, the first operation is completed.

The pixel data string stored in the DRAM 226 by the first operation is used for the correlation calculation for calculating the out-of-focus amount. The focus detection circuit 218 performs a correlation calculation using a pair of left aperture pixel data Grl and right aperture pixel data Grr and a pair of left aperture pixel data Gbl and right aperture pixel data Gbr stored in the DRAM 226.

Next, the second operation in the low-luminance operation will be described. The second operation shown by LV in FIG. 7 is an operation mainly for generating and reading (LV) a display pixel signal. However, in the present embodiment, the generation and reading (AF (tb)) of the focus detection pixel signal regarding the second pupil division direction are also performed in the second operation. Prior to the second operation, the control circuit 302 outputs the upper aperture pixel signal t (Rt, Grt, Gbt, Bt) and the lower aperture pixel signal b (Rb, Grb, Gbb, Bb) from the pixel unit 303. The setting of the pixel unit 303 is switched so as to be. Further, the control circuit 302 sets the pixel addition between the pixel units 303 in order to shorten the pixel signal reading time.

FIG. 15 shows an example of setting the pixel addition of the second operation in the low-luminance operation. In the second operation, the addition is set so that the addition is performed in both the horizontal direction which is the first pupil division direction and the vertical direction which is the second pupil division direction. In the example of FIG. 15, H3 / 3 pixels are added in the horizontal direction, and V3 / 3 pixels are added in the vertical direction. In the second operation in the low-luminance operation, it is desirable that 3 ≦ n ≦ 5, 3 ≦ m ≦ 5, and n ≦ m in the horizontal direction, and 1 ≦ n ≦ 3 in the vertical direction. It is desirable that 1 ≦ m ≦ 3 and n ≦ m. Similar to the first operation of the high-speed operation described above, the addition number of pixels is suppressed to 5 pixels or less, so that the detection accuracy of the phase difference in the vertical direction can be ensured.

By the setting of FIG. 15, the number of rows and the number of columns of the pixel signal are each compressed to about one-third. By reducing the number of rows and columns of the pixel signal, the read time of the pixel signal is shortened. The number of pixel signals to be added is not limited to this. However, in the second operation, the image quality at the time of display deteriorates as the number of additions of the pixel signals increases. Further, as the number of additions of the pixel signals in the vertical direction increases, the accuracy of detecting the phase difference in the vertical direction also decreases. Therefore, it is desirable that the number of pixel signal additions is appropriately determined by a trade-off between the readout time, the image quality at the time of display, and the detection accuracy of the phase difference in the vertical direction.

After the setting as shown in FIG. 15, the CPU 212 outputs a control signal to the image pickup device 208 so as to perform imaging at the exposure time required for generating the focus detection pixel signal. This exposure time is set based on the subject brightness and the like.

Upon receiving the input of the control signal from the CPU 212, the control circuit 302 starts imaging (charge accumulation) for each row of the pixel unit 303. Then, the control circuit 302 controls the vertical scanning circuit 304 to sequentially output pixel signals from the pixel unit 303 in the row where the imaging is completed.

The pixel signal read from the pixel unit 303 is analog-processed in the analog processing circuit 305. The analog-processed pixel signal is converted into pixel data, which is a digital signal, in the ADC processing circuit 306. The pixel data is stored in the memory circuit 307. In the second operation, pixel signals of R, Gr, Gb, and B are required for the display image data. Therefore, the upper aperture pixel signals Rt, Grt, Gbt, and Bt and the lower aperture pixel signals Rb, Grb, Gbb, and Bb are stored in the memory circuit 307 without being packed.

The horizontal scanning circuit 308 receives the control signal from the control circuit 302 and transfers the pixel data stored in the memory circuit 307 to the output circuit 309 in column order. The output circuit 309 arranges the pixel data transferred by the horizontal scanning circuit 308 to generate a pixel data string, converts the generated pixel data string into a predetermined output signal format such as a serial signal and a differential signal, and outputs the data. do. This pixel data string is stored in the DRAM 226 as shown in FIG. 16, for example. In this way, the second operation is completed.

The pixel data string stored in the DRAM 226 by the second operation is used for the LV display. As described above, the display image data is obtained by adding and averaging the pixel data of the light receiving units lt, rt, lb, and rb belonging to the same pixel unit 303. Therefore, the image processing circuit 214 generates display image data by adding and averaging the upper opening pixel data t (Rt, Grt, Gbt, Bt) and the lower opening pixel data b (Rb, Grb, Gbb, Bb).

Further, the pixel data string stored in the DRAM 226 by the second operation is used for the correlation calculation for calculating the focus shift amount. The focus detection circuit 218 performs the correlation calculation using the pair of the upper aperture pixel data Grt and the lower aperture pixel data Grb stored in the DRAM 226. Further, the focus detection circuit 218 performs the correlation calculation using the pair of the upper aperture pixel data Gbt and the lower aperture pixel data Gbb stored in the DRAM 226. That is, the upper and lower aperture pixel data Grt, Grb, Gbt, and Gbb are read out twice from the DRAM 226.

In the low-luminance operation as described above, the phase difference can be detected in both the horizontal direction and the vertical direction. Therefore, in the first operation, by increasing the addition number in the vertical direction, it is possible to secure the detection accuracy of the phase difference in the horizontal direction even under low luminance. Further, in the second operation, it is possible to secure the detection accuracy of the phase difference in the vertical direction while ensuring the reading time of the pixel signal for display.

As described above, in the present embodiment, in the first operation when high-speed operation is required, the pixel signal is not added in the first pupil division direction, and the pixels are obtained only in the second pupil division direction. By adding the signals, the focus detection performance in the first pupil division direction can be ensured. Further, by suppressing the addition number of the pixel signals in the second pupil division direction to a predetermined number of pixels or less, the diagonal direction including the focus detection substantially in the second pupil division direction in the first operation. Focus detection can be performed. In addition, in the second operation, the time for the live view display can be secured by not generating and reading the pixel signal for the focus detection.

Further, in the first operation when the focus detection performance at low luminance is required, the pixel signal is not added in the first pupil division direction, but the pixel signal is added only in the second pupil division direction. By setting it appropriately, it is possible to secure the focus detection performance in the first pupil division direction. Further, in the second operation, the addition number of the pixel signals in the second pupil division direction is suppressed to a predetermined number of pixels or less to secure the time for the live view display, and the second pupil division direction. Focus detection performance can be ensured.

As described above, in the present embodiment, it is possible to secure the time for the live view display while ensuring the focus detection performance of the phase difference method during continuous shooting.

[Modification example]
In the above-described embodiment, the first pupil division direction is the horizontal direction and the second pupil division direction is the vertical direction. On the contrary, the first pupil division direction may be the vertical direction and the second pupil division direction may be the horizontal direction. In this case, the pixel unit 303 is set so that the upper and lower aperture pixel signals are output in the first operation, and the pixel unit 303 is set so that the left and right aperture pixel signals are output in the second operation. Is done.

Further, in the above-described embodiment, the pixel addition setting and the like in the pixel unit 303 are performed by the control circuit 302 provided inside the image pickup device 208. The control circuit 302 may be provided outside the image pickup device 208. In this case, for example, the CPU 212 may be configured to perform the same processing as the control circuit 302.

Further, in the first operation described above, the readout time is shortened by adding the pixel signals of the same aperture arranged in the vertical direction, which is the second pupil division direction, and then outputting the pixels from the pixel unit 303. On the other hand, the read time can be shortened by thinning out the pixel signals and reading them.

Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications and applications are possible within the scope of the gist of the present invention.

Further, in the explanation of each operation flowchart described above, the operation is described using "first", "next", etc. for convenience, but it does not mean that it is essential to perform the operation in this order. do not have.

Further, each process according to the above-described embodiment can be stored as a program that can be executed by the CPU 212 or the control circuit 302, which is a computer. In addition, it can be stored and distributed in a storage medium of an external storage device such as a magnetic disk, an optical disk, or a semiconductor memory. Then, the CPU 212 or the control circuit 302 reads the program stored in the storage medium of the external storage device, and the operation is controlled by the read program, so that the above-mentioned processing can be executed.

The present invention is not limited to the above embodiment, and can be variously modified at the implementation stage without departing from the gist thereof. In addition, each embodiment may be carried out in combination as appropriate, in which case the combined effect can be obtained. Further, the above-described embodiment includes various inventions, and various inventions can be extracted by a combination selected from a plurality of disclosed constituent requirements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, if the problem can be solved and the effect is obtained, the configuration in which the constituent elements are deleted can be extracted as an invention.

1 image sensor, 100 interchangeable lens, 102 image pickup optical system, 104 drive unit, 106 lens CPU, 108 lens side storage unit, 110 interface (I / F), 200 camera body, 202 mechanical shutter, 204 drive unit, 206 operation Unit, 208 image sensor, 210 camera shake correction circuit, 212 CPU, 214 image processing circuit, 216 image compression expansion unit, 218 focus detection circuit, 220 exposure control circuit, 222 display unit, 224 bus, 226 DRAM, 228 main unit side storage unit , 230 recording medium, 301 input circuit, 302 control circuit, 303 pixel section, 303a microlens, 303b color filter, 303c pixel, 304 vertical scanning circuit, 305 analog processing circuit, 306 analog-to-digital conversion (ADC) processing circuit, 307 memory. Circuit, 308 horizontal scanning circuit, 309 output circuit, 1021 focus lens, 1022 aperture.

Claims (4)

It is different from the first pupil division direction and the first pupil division direction so as to generate a pixel signal by photoelectric conversion of each light beam passing through different exit pupil regions of the imaging optical system for one microlens. It has a pixel unit in which a plurality of light receiving units divided in the second pupil division direction are arranged, and is used for a focus detection pixel signal corresponding to the first pupil division direction or the second pupil division direction, and pupil division. An image pickup unit that outputs a display pixel signal for display that does not correspond or a still image pixel signal for recording .
A process for displaying an image based on the display pixel signal or a pixel signal to which the focus detection pixel signal is added, and a process for recording a still image on a recording medium based on the still image pixel signal. Image processing circuit and
A focus detection circuit that performs processing for focus detection based on the focus detection pixel signal, and
While performing still image shooting, which is an operation of recording a still image based on the still image pixel signal on the recording medium, each of them generates and reads out the focus detection pixel signal or the display pixel signal during the still image shooting. A control circuit that controls the image pickup unit so as to alternately perform the first operation and the second operation including
A photometric unit that measures the brightness of the subject,
Equipped with
In the control circuit, after the pixel signals generated by the plurality of pixel portions in the first operation are added in the pixel portion in the second pupil division direction regardless of the subject brightness . The focus detection pixel signal is generated so that the focus detection pixel signal is generated by being added or thinned in the second pupil division direction without being added in the first pupil division direction among the plurality of pixel portions. Control the image pickup unit,
In the second operation when the subject brightness is high , the display pixel signal is generated, and in the second operation when the subject brightness is low, the pixel signals generated by the plurality of pixel portions are generated. After being added in the first pupil division direction in the pixel portion, the focus detection pixel signal added in the first pupil division direction is generated among the plurality of pixel portions. By controlling the image pickup unit,
The image processing circuit performs processing for displaying an image based on the display pixel signal when the subject brightness is high, and the first among the plurality of pixel portions when the subject brightness is low. The focus detection pixel signal generated by being added in the pupil division direction of the above, and based on the pixel signal obtained by adding the focus detection pixel signals paired in the second pupil division direction in the plurality of pixel portions. A focus detector that performs processing to display an image . The subject brightness is the brightness of the focus detection area, which is the area for the focus detection set in the image pickup unit.
The focus detection device according to claim 1 , wherein the control circuit determines whether or not the subject brightness is low based on the aperture value of the image pickup optical system and the brightness of the focus detection region. It is different from the first pupil division direction and the first pupil division direction so as to generate a pixel signal by photoelectric conversion of each light beam passing through different exit pupil regions of the imaging optical system for one microlens. It has a pixel unit in which a plurality of light receiving units divided in the second pupil division direction are arranged, and is used for a focus detection pixel signal corresponding to the first pupil division direction or the second pupil division direction, and pupil division. An image pickup unit that outputs a display pixel signal for display that does not correspond or a still image pixel signal for recording.
An image processing circuit that performs processing for displaying an image based on the display pixel signal or a pixel signal to which the focus detection pixel signal is added, and processing for recording a still image on a recording medium based on the still image pixel signal. When,
A focus detection circuit that performs processing for focus detection based on the focus detection pixel signal, and
While performing still image shooting, which is an operation of recording a still image based on the still image pixel signal on the recording medium, each of them generates and reads out the focus detection pixel signal or the display pixel signal during the still image shooting. A control circuit that controls the image pickup unit so as to alternately perform the first operation and the second operation including
Equipped with
The control circuit has the pixels generated by the plurality of pixel portions in the first operation regardless of the continuous shooting speed corresponding to the still image shooting interval when the still image shooting is continuously executed. After the signal is added in the second pupil division direction in the pixel portion, the signal is not added in the first pupil division direction among the plurality of pixel portions, but in the second pupil division direction. The image pickup unit is controlled so that the focus detection pixel signal is generated by being added or thinned out.
In the second operation when the continuous shooting speed exceeds a predetermined value, the image pickup unit is controlled so that the display pixel signal is generated.
In the second operation when the continuous shooting speed is equal to or less than the predetermined value, the pixel signals generated by the plurality of pixel portions are added to the first pupil division direction in the pixel portions. Later, the image pickup unit is controlled so that the focus detection pixel signal added with respect to the first pupil division direction is generated among the plurality of pixel units .
The image processing circuit performs processing for displaying an image based on the display pixel signal when the continuous shooting speed exceeds a predetermined value, and when the continuous shooting speed is equal to or less than the predetermined value, the plurality of images are displayed. The focus detection pixel signal added in the first pupil division direction between the pixel portions, and the focus detection pixel signal paired in the second pupil division direction in the plurality of pixel portions. A focus detection device that performs processing for displaying an image based on the pixel signal obtained by adding the above. It is different from the first pupil division direction and the first pupil division direction so as to generate a pixel signal by photoelectric conversion of each light beam passing through different exit pupil regions of the imaging optical system for one microlens. It has a pixel unit in which a plurality of light receiving units divided in the second pupil division direction are arranged, and is used for a focus detection pixel signal corresponding to the first pupil division direction or the second pupil division direction, and pupil division. It is a focus detection method using an image pickup unit that outputs a display pixel signal for display that does not correspond or a still image pixel signal for recording .
Measuring the subject brightness and
Processing for displaying an image based on the pixel signal obtained by adding the display pixel signal or the focus detection pixel signal based on the subject luminance is performed.
Processing for recording a still image on a recording medium based on the still image pixel signal, and
Processing for focus detection based on the focus detection pixel signal and
While performing still image shooting, which is an operation of recording a still image based on the still image pixel signal on the recording medium, each of them generates and reads out the focus detection pixel signal or the display pixel signal during the still image shooting. Controlling the image pickup unit so as to alternately perform the first operation including and the second operation, and
Equipped with
Controlling the image pickup unit means that in the first operation, the pixel signals generated by the plurality of pixel units are added to the second pupil division direction in the pixel unit regardless of the subject brightness. After that, the focus detection pixel signal is generated by being added or thinned in the second pupil division direction without being added in the first pupil division direction among the plurality of pixel portions. By controlling the image pickup unit so as to
In the second operation when the subject brightness is high , the display pixel signal is generated, and in the second operation when the subject brightness is low, the pixel signals generated by the plurality of pixel portions are generated. After being added in the first pupil division direction in the pixel portion, the focus detection pixel signal added in the first pupil division direction is generated among the plurality of pixel portions. Including controlling the image pickup unit
The process for displaying the image is a process for displaying an image based on the display pixel signal when the subject brightness is high, and the plurality of pixels when the subject brightness is low. The focus detection pixel signal generated by being added in the first pupil division direction between the portions, and the focus detection pixel signal paired in the second pupil division direction in the plurality of pixel portions. A focus detection method including processing for displaying an image based on a pixel signal obtained by adding a pixel signal .

Images