JP2015011216A - Imaging device, and control method and control program of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To optimize a detection time lag and detection accuracy in performing AF control according to a photographic scene.SOLUTION: An imaging device includes an image sensor 102 that can output image signals in a plurality of drive modes having different frame rates during one frame period, and selects the optimal image for detecting an imaging evaluation value according to the feature of a determined photographic scene.

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a control program, and more particularly to an imaging apparatus capable of detecting an imaging evaluation value related to photometric distance measurement according to image data.

  Conventionally, when obtaining position information of a subject used for focus control in an image pickup apparatus, the position information is obtained based on an image signal output from an image pickup device. In addition, an optical signal indicating a subject is directly input to a dedicated detection device, and position information is obtained based on a phase difference in an image indicated by the optical signal. In the case where position information is obtained based on image data, the imaging device can be downsized in that a dedicated detection device is not required.

  FIG. 10 is a diagram for explaining the timing of autofocus imaging operation (AF evaluation imaging) during live view in a conventional imaging apparatus. As shown in the figure, in the conventional imaging apparatus, the imaging timing is defined by a vertical synchronizing signal (VD). When the AF control signal is turned on, an image for an AF evaluation image is captured according to VD after the live view imaging period, and when the AF control signal is turned off, the live view imaging period starts again. As described above, the live view imaging period for obtaining the live view image and the AF operation period for obtaining the AF evaluation image are serially arranged along the time axis, and the live view image and the AF evaluation image are simultaneously displayed. It is not configured to image.

  For this reason, as shown in the figure, since the AF evaluation image is captured during the AF operation period located between the live view image capturing periods (frames), the live view image and the AF evaluation image are between. There is a time lag.

  In addition, although live view display is performed even when an AF evaluation image is captured, live view display is performed based on the AF evaluation image. As shown in FIG. 10, when the AF evaluation image is captured, the frame rate is set higher than that of the live view imaging period. Therefore, the thinning-out rate in the reading of the image sensor increases, and the deterioration of the image quality is inevitable. .

  In order to avoid this point, for example, there is a configuration in which focus signal detection pixels are provided separately from the image signal pixels in the pixel portion of the image sensor. In this configuration, the focus detection / automatic exposure (reading the imaging signal for use in the focus detection signal and the photometric information for automatic exposure from the imaging element together with the live view readout mode for reading the imaging signal for live view display ( AE) read mode. These read modes are repeated cyclically for each frame (see Patent Document 1).

JP 2009-89105 A

  However, in Patent Document 1, since the focus signal detection pixel is provided in the pixel portion, the area of the image signal pixel is inevitably reduced, and the focus signal detection pixel is an image signal (image signal). Since it is not used when obtaining the image quality, the image quality is lowered accordingly. In addition, since the focus signal detection pixel has a lower sensitivity than the imaging signal pixel, the detection accuracy of the focus signal is lowered depending on the shooting scene.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging device capable of optimizing focus detection accuracy and focus detection speed by scene discrimination, a control method therefor, and a control program.

  In order to achieve the above-described object of the present invention, according to the present invention, an imaging apparatus having a photographing optical system for forming an optical image of a subject and capturing an optical image to generate an image signal is a photoelectric conversion of the optical image. A first element portion having a pixel portion in which pixels for generating an image signal are arranged in a matrix, and a second pixel portion provided with a reading means for reading the image signal from the pixel portion, However, the imaging device has a first reading unit that reads an image signal from the first pixel group of the pixel unit, a second reading unit that reads an image signal from the second pixel group of the pixel unit, and the imaging device outputs An imaging condition determination unit that acquires and determines information about an imaging condition of an optical image from an image signal, and an image signal that is read by the first reading unit and the second reading unit according to the determination result And a imaging evaluation value detecting means for detecting the imaging evaluation value related to the imaging of.

  According to the imaging apparatus of the present invention, an imaging device capable of optimizing focus detection accuracy and focus detection speed by scene discrimination, a control method thereof, and a control program are provided.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows the structure of the image pick-up element which the imaging device which concerns on 1st Example of this invention has. It is a figure for demonstrating the read-out structure of the pixel part of the image pick-up element which the imaging device which concerns on 1st Example of this invention uses. It is a figure which shows the imaging timing of the imaging device which concerns on 1st Example of this invention. It is a figure which shows the flowchart of operation | movement in AF mode of the imaging device which concerns on 1st Example of this invention. It is a figure which shows the structure of the imaging device which concerns on 2nd Example of this invention. It is a figure which shows the flowchart of operation | movement in AF mode of the imaging device which concerns on 2nd Example of this invention. It is a figure which shows the determination table of the scene determination in the imaging device which concerns on 2nd Example of this invention. It is a figure which shows the structure of the imaging device which concerns on the 3rd Example of this invention. It is a figure which shows the timing of the autofocus imaging operation in the live view mode in the conventional imaging device.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the first embodiment of the present invention. The illustrated imaging apparatus is applied to, for example, an electronic still camera with a moving image function or a video camera.

  In the figure, an imaging apparatus 100 includes a lens 101, an image sensor 102, an image signal processing unit 103, a compression / decompression unit 104, a lens drive control unit 105, an imaging signal evaluation value detection unit 106, a scene determination unit 107, and a system control unit 108. Is provided. In addition, the imaging apparatus 100 further includes a light emitting unit 109, an operation unit 110, a storage unit 111, and a display unit 112.

  The lens 101 is a lens group that constitutes a photographing optical system. The lens 101 includes a focus lens. The focus lens is a focus adjustment lens. The focus lens is configured to be able to change its position along the optical axis direction. Based on the values detected by the lens drive control unit 105 and the imaging evaluation value detection unit 106, the focus lens is driven and controlled, and functions as a focus adjustment unit that executes a focus adjustment process. The light that has passed through the lens 101 is formed as an optical image of a subject on an image forming surface of an image sensor 102 formed of a CMOS image sensor or the like, and is photoelectrically converted into a pixel signal by a pixel 201 described later.

  The image sensor 102 includes a pixel 201 and an A / D converter, and is, for example, a so-called XY readout type CMOS image sensor. The image sensor 102 performs an imaging operation such as exposure, signal readout, and reset under the control of the system control unit 108, and outputs an imaging signal (also referred to as an image signal).

  The imaging evaluation value detection unit 106 detects an imaging evaluation value from the image signal output from the image sensor 102. At that time, the imaging evaluation value is detected at the timing output from the system control 107. Detailed operation will be described later.

  Here, the imaging evaluation value is a parameter necessary for controlling the imaging apparatus, correcting the captured image, and the like. For example, it is an evaluation value necessary for the basic operation of the imaging apparatus, such as an AF evaluation value, a WB (white balance) evaluation value, an AE (Automatic Exposure) evaluation value. The AF evaluation value is an evaluation value for focusing on the subject at the time of imaging, and is mainly necessary when controlling the focus lens. The WB evaluation value is an evaluation value necessary for correcting a color at the time of imaging, and is a parameter necessary for development. The AE evaluation value is an evaluation value necessary for appropriately adjusting the exposure at the time of shooting, and is mainly required when setting the aperture, shutter speed, and sensitivity.

  The system control unit 108 determines the control amount of the lens 101 based on the AF evaluation value that is one of the parameters obtained as the imaging evaluation value, and outputs this control amount to the lens drive control unit 105. The lens drive control unit 105 adjusts the subject focus by driving the lens 101 in the optical axis direction based on the control amount of the AF evaluation value obtained from the system control 108.

  The image signal processing unit 103 performs image processing on the image signal that is the output of the image sensor 102 under the control of the system control unit 108 to generate image data. Specifically, image data is generated by performing signal processing such as white balance adjustment processing, color correction processing, and AE processing based on the imaging evaluation value detected by the imaging evaluation value detection unit.

  The compression / decompression unit 104 operates under the control of the system control unit 108 and performs compression encoding processing on the image data output from the image signal processing unit 103 in a predetermined still image data format. For example, the predetermined still image data format is a JPEG (Joint Photographic Coding Experts Group) system or the like. The compression / decompression unit 104 performs decompression / decoding processing on the encoded image data transmitted from the system control unit 108. Note that the compression / decompression unit 104 may perform compression encoding / decompression decoding processing on moving image data using an MPEG (Moving Picture Experts Group) method or the like.

  The scene determination unit 107 determines a shooting scene from shooting conditions obtained from the system control unit, and sends information for changing shooting parameters, image processing parameters, and the like at the time of shooting to the system control unit 108 according to the determined shooting scene. . Here, the imaging evaluation value detection unit 106 determines whether to detect the imaging evaluation value from an imaging evaluation value detection image signal or a display image signal, which will be described later, based on the scene determination information.

  The system control unit 108 is a microcontroller including, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU of the system control unit 108 performs overall control of the entire imaging apparatus 100 by executing a program stored in the ROM.

  When the light emitting unit 109 determines that the exposure value of the subject is low by the AE processing by the image signal processing unit 103, the light emitting unit 109 illuminates the subject with light. As the light emitting unit 109, for example, a strobe device using a xenon tube or an LED light emitting device is used. The operation unit 110 includes, for example, various operation keys such as a shutter release button, a lever, and a dial, and provides an operation signal according to a user input operation to the system control unit 108.

  The recording unit 111 is, for example, a recording medium such as a portable semiconductor memory, an optical disk, an HDD (Hard Disk Drive), or a magnetic tape, and stores the image data compressed and encoded by the compression / decompression unit 104 as an image file. . Further, the recording unit 111 reads out the image file designated by the system control unit 108 and outputs it to the system control unit 108.

  The image display unit 112 includes, for example, a display device such as an LCD (Liquid Crystal Display) and an interface circuit for the LCD, and displays an image represented by image data sent from the system control unit 108 on the display device.

  FIG. 2 is a diagram for explaining the configuration of the image sensor 102 shown in FIG. FIG. 2A is a perspective view of the image sensor, and FIG. 2B is a block diagram showing the configuration thereof.

  In FIG. 2A, the image sensor 102 has a first chip (first element portion) 20 and a second chip 21 (second element portion), and the first chip is formed on the second chip 21. 20 are stacked. The first chip 20 includes a plurality of pixels 201 arranged in a matrix, and the first chip 20 is stacked with the pixel array facing the light incident side (that is, the optical chip is positioned on the light receiving side). doing). In the second chip 21, pixel drive circuits such as column scanning circuits 213-a and 213-b and a row scanning circuit 212 described later are formed.

  In this manner, if the pixel 201 is formed on the first chip 20 and the pixel driving circuit is formed on the second chip 21, the manufacturing process of the peripheral circuit and the pixel portion of the image sensor 102 can be divided. As a result, it is possible to increase the speed, miniaturization, and functionality of the peripheral circuit by thinning the wiring and increasing the density.

  As shown in FIG. 2B, in the first chip 20, the pixels 201 are arranged in a matrix, and each pixel 201 has a transfer signal line 203, a reset signal line 204, and a horizontal direction (row direction). It is connected to the row selection signal line 205. Further, it is connected to the column signal lines 202-a and 202-b in the vertical direction (column direction). Note that each of the column signal lines 202-a and 202-b connects pixels to different readout destinations in units of rows.

  Each of the pixels 201 includes a photodiode PD, which is a photoelectric conversion element, a transfer transistor M1, a reset transistor M2, an amplification transistor M3, selection transistors M4, and a floating diffusion FD, as illustrated. In the illustrated example, each of the transistors is an n-channel MOSFET (MOS Field-Effect Transistor).

  A transfer signal line 203, a reset signal line 204, and a row selection signal line 205 are connected to the gates of the transfer transistor M1, the reset transistor M2, and the selection transistor M4, respectively. These signal lines 203 to 205 extend in the horizontal direction and simultaneously drive pixels in the same row. As a result, the operation of the line-sequential operation type rolling shutter or the all-row simultaneous operation type global shutter can be controlled. Further, the column signal line 202-a or 202-b is connected to the source of the selection transistor M4 in units of rows.

  The photodiode PD accumulates electric charges generated by photoelectric conversion. The P side of the photodiode PD is grounded, and the N side is connected to the source of the transfer transistor M1. When the transfer transistor M1 is turned on, the charge of the photodiode PD is transferred to the FD, and since there is a parasitic capacitance in the FD, the charge transferred to the FD is accumulated.

  A power supply voltage Vdd is applied to the drain of the amplification transistor M3, and its gate is connected to the FD. The amplification transistor M3 amplifies the charge (that is, voltage) of the FD and converts it into a voltage signal. The selection transistor M4 is for selecting a pixel from which a signal is read out by the row selection signal line 205 in units of rows, and the drain thereof is connected to the source of the amplification transistor M3. The source of the selection transistor M4 is connected to the column signal line 202.

  When the selection transistor M4 is turned on by the row signal selection line 205, a voltage signal corresponding to the voltage of the FD is output to the column signal line 202. A power supply voltage Vdd is applied to the drain of the reset transistor M2, and its source is connected to the FD. When the reset transistor M2 is turned on by the reset signal line 204, the voltage of the FD is reset to the power supply voltage Vdd.

  The second chip 21 is provided with a column ADC block 211, and the column ADC 211 is connected to the column signal line 202-a or 202-b. Further, the second chip 21 is provided with a row scanning circuit 212, column scanning circuits 213-a and 213-b, a timing control circuit 214, and horizontal signal lines (output means) 215-a and 215-b. .

  The timing control circuit 214 controls the operation timing of the row scanning circuit 212, the column scanning circuits 213-a and 213-b, and the column ADC block 211 under the control of the system control unit 108. The row scanning circuit 212 scans each row, and the column scanning circuits 213a and 213b scan each column.

  The horizontal signal lines 215-a and 215-b transfer the output signal (image signal) of the column ADC block 211 according to the timing controlled by the column scanning circuits 213-a and 213-b, respectively. Thereby, an image signal for live view (second image signal, that is, an image display signal) is output to the column signal line 202-a, and an image signal for detecting an imaging evaluation value (first image signal) is output to the column. The signal is output to the signal line 202-b.

  In FIG. 3, row numbers 1 and 2 are rows (first pixel group) for capturing an image for evaluation evaluation value detection, and row numbers 3 to 8 are rows for capturing a live view image (first). 2 pixel group). In the illustrated example, readout scanning is sequentially performed in units of rows, and readout scanning is repeatedly performed in units of 8 rows.

  In imaging evaluation value detection imaging, 3 pixels out of 4 pixels of the same vertical color are skipped and read out in order to emphasize the frame rate. On the other hand, in live view imaging, in order to emphasize image quality, 1 pixel is thinned out from 4 pixels of the same vertical color and 3 pixels are added. In other words, in imaging for imaging evaluation value detection, the first pixel group is read at the first frame rate. In live view imaging, the second pixel group is read out at a second frame rate that is slower than the first frame rate.

  As described above, by dividing the imaging evaluation value detection imaging and the live view imaging for each selected row, it is possible to acquire image signals of different frame rates with different data sizes in different charge accumulation times.

  The voltage signals (analog signals) output to the column signal lines 202-a and 202-b are converted from analog signals to digital signals (image signals) in the column ADC block 211 shown in FIG. The image signal that is the output of the column ADC block 211 is read out from the column ADC block 211 to the horizontal signal line 215-a or 215-b by the column scanning circuit 213-a or 213-b and output (first reading). Means, second reading means).

  Next, as an example of the imaging evaluation value, an operation for detecting an optimal imaging evaluation value (AF evaluation value) for the AF operation will be described with reference to FIG.

  FIG. 4A is a timing chart when an AF evaluation value (autofocus evaluation value) is detected from an imaging evaluation value detection image as a result of scene determination described later. As shown in the figure, the imaging timing is defined by the vertical synchronization signal, and in the AF evaluation mode, the system control unit 108 raises the AF control signal at the falling edge of the vertical synchronization signal at time T0 (H level). Subsequently, when the vertical synchronization signal rises, the system control unit 108 simultaneously starts and performs imaging operations for image display signals and imaging evaluation value detection in synchronization with the vertical synchronization signal.

  The imaging evaluation image signal read out from the pixel unit 20 via the horizontal signal line 215-b in the period T0 to TF1 is input to the imaging evaluation value detection unit 106, and AF evaluation values are calculated in TF1 to TF2. The The AF evaluation value is calculated at a timing controlled by the system control unit 108 based on contrast information and phase difference information obtained from the imaging evaluation value detection image signal output from the image sensor 102. Thereafter, in the periods TF <b> 2 to TF <b> 3, the imaging evaluation value detection unit 106 outputs the AF evaluation value to the system control unit 108.

  In the illustrated example, one frame of the live view image is captured during the period of one vertical synchronization signal, and three frames of the AF evaluation value detection image (AF scanning) are captured. When the system control unit 108 sets the vertical synchronization signal to the L level, the AF evaluation within one frame period of the live view image ends.

  The system control unit 108 compares the AF evaluation value with a predetermined AF expected value described later. When the AF evaluation value satisfies a predetermined evaluation condition with the expected AF value, the system control unit 108 causes the AF control signal to fall (time T1). When the AF control signal falls, only the AF evaluation imaging is stopped and the live view imaging is continuously performed.

  FIG. 4B is a diagram illustrating a timing chart when an AF evaluation value is detected from a live view image in accordance with a result of scene determination described later.

  As in FIG. 4A, when the imaging timing is defined by the vertical synchronization signal and the AF evaluation mode is set, the system control unit 108 outputs the AF control signal at the falling edge of the vertical synchronization signal at time T0. Start up (H level). Subsequently, when the vertical synchronization signal rises, the system control unit 108 performs the imaging operation only for the image display signal in synchronization with the vertical synchronization signal.

  In the period T0 to TF4, the live view image signal read from the pixel 201 via 215-a is input to the imaging evaluation value detection unit 106 and the image signal processing unit 103, and in the periods TF4 to TF5, the AF evaluation value Is calculated. The imaging evaluation value detection unit 106 outputs the AF evaluation value at a timing controlled by the system control unit 108 based on contrast information and phase difference information obtained from the image signal of the live view image output from the image sensor 102. calculate. Thereafter, in the periods TF5 to TF6, the imaging evaluation value detection unit 106 outputs the AF evaluation value to the system control unit 108.

  In the illustrated example, one frame of the live view image is captured during the period of one vertical synchronization signal, and the AF evaluation value detection image (AF scanning) is not captured. When the system control unit 108 sets the vertical synchronization signal to the L level, the AF evaluation within one frame period of the live view image ends.

  The system control unit 108 compares the AF evaluation value detected from the live view image with a predetermined AF expected value described later, and if the AF evaluation value satisfies a predetermined evaluation condition with the AF expected value, an AF control signal is output. Fall down (time T7). When the AF control signal falls, the AF evaluation detection operation from the display image is stopped, and live view imaging is continuously performed.

  As shown in FIG. 4A, in the present embodiment, the imaging evaluation value detection image can set a frame rate faster than the operation of the live view image. It becomes possible to control more quickly than in the past.

  However, the imaging apparatus has a dark tracking limit by exposure control determined by a program diagram. Therefore, the tracking limit of the dark part by exposure control differs between the imaging evaluation value detection image with a fast frame rate and the live view image with a slow frame rate. In the present embodiment, the difference in the tracking limit of the dark part is generated by three steps.

  The scene discriminating unit 107 discriminates the characteristics of the shooting scene from the sensitivity setting value set as the shooting (exposure) condition by the system control unit 108 at the time of shooting the live view image and the exposure amount calculated from the shutter speed. The imaging evaluation value detection unit 106 detects an imaging evaluation value from either the imaging evaluation value detection image or the live view image according to the determination result of the scene determination unit 107.

  Next, the operation of the imaging apparatus according to the first embodiment in the AF evaluation mode will be described in detail with reference to the flowchart of FIG.

  The illustrated flowchart is performed under the control of the system control unit 108. For the sake of explanation, the imaging evaluation value detected in the imaging evaluation value detection image is AF_Kα, and the imaging evaluation value detected in the live view image is AF_Kβ.

  After the power is turned on by the user and various initial settings are made (step S502), the mode is shifted to an operation mode such as live view mode or moving image recording, and shooting is started (step S520).

  The system control unit 108 determines whether or not the AF evaluation mode is set (step S502). That is, the system control unit 108 determines whether or not the autofocus mode is set. If it is not the AF evaluation mode (No in step S503), the system control unit 108 starts only live view imaging (step S520), and proceeds to step S521 described later.

  On the other hand, in the AF evaluation mode (Yes in step S503), the system control unit 108 turns on the AF control signal (H level) (step S504). Subsequently, the system control unit 108 initializes by assigning 0 to a variable n for counting the number of AF evaluation imaging (step S505). In step S506, the system control unit 108 detects the exposure amount S of the imaging condition when performing live view shooting.

Next, the scene determination unit 107 compares the exposure amount E detected in step S506 with the expected exposure amount Ev (imaging condition determination). The system control unit 108 determines whether or not the exposure amount of live view shooting satisfies the following equation (1), that is, a predetermined sensitivity condition with respect to Ev that is an expected value of exposure setting (step S507).
E <Ev (1)
If the exposure amount Ev of the live view satisfies the expression (1) (Yes), the process proceeds to S508, and imaging for AF evaluation value detection is started.

  Next, as described in FIG. 4A, the system control unit 108 starts AF evaluation imaging. (Step S508) After starting the AF evaluation imaging, the system control unit 108 increments the variable n by 1 (step S509). Thereafter, under the control of the system control unit 108, the imaging evaluation value detection unit 106 detects the AF evaluation value AF_Kα from the AF evaluation image signal obtained by the AF evaluation imaging (step S510).

Subsequently, the system control unit 108 determines whether or not the AF evaluation value AF_Kα satisfies the following expression (2), that is, a predetermined evaluation condition with respect to K_minα and K_maxα, which are AF expected values (step S511).
K_minα <AF_Kα <K_maxα (2)
Here, the AF evaluation value expected values K_minα and K_max α are set to the minimum value and the maximum value of the expected AF evaluation value, and are previously set in the system control unit 108 when designing the imaging apparatus or adjusting the imaging apparatus. To be recorded.

  If the AF evaluation value AF_Kα does not satisfy Expression (2) (No in step S511), the system control unit 108 obtains a feedback control amount based on the AF evaluation value AF_Kα. Then, the system control unit 108 drives and controls the lens drive control unit 105 according to the feedback control amount to drive the focus lens provided in the lens 101 (step S512).

  Subsequently, the system control unit 108 determines whether or not the variable (AF evaluation value imaging count) n is a predetermined number (here, 3) (step S513). If the number of AF evaluation value imaging is less than 3 (No in step S511), the system control unit 108 returns to the processing in step S510 and performs AF evaluation imaging. On the other hand, if the AF evaluation value imaging count is 3 (Yes in step S513), the system control unit 108 returns to the processing of step S505 and sets the AF evaluation value imaging count n to zero.

  When the AF evaluation value AF_Kα satisfies Expression (2) (Yes in step S511), the system control unit 108 turns off the AF control signal (L level) (step S514), and stops the AF evaluation imaging in the image sensor 102. (Step S515). Thereafter, the system control unit 108 advances the operation to step S522.

If the exposure amount E of the live view image does not satisfy the expression (1) in step S507 (No), the process proceeds to S516, and the AF evaluation value AF_Kβ is detected from the live view captured image. Subsequently, the system control unit 108 determines whether or not the AF evaluation value AF_Kβ satisfies the following equation (3) with respect to K_minβ and K_maxβ, which are AF expected values, that is, a predetermined surface price condition is satisfied (step S517). ).
K_minβ <AF_K <K_maxβ (3)
Here, the expected AF values K_minβ and K_maxβ are set as minimum and maximum values of expected AF evaluation values, and are recorded in advance in the system control unit 108 at the time of designing the imaging device or adjusting the imaging device. .

  When the AF evaluation value AF_Kβ does not satisfy Expression (3) (No in step S517), the system control unit 108 obtains a feedback control amount according to the AF evaluation value AF_Kβ. Then, the system control unit 108 drives and controls the lens drive control unit 105 according to the feedback control amount to drive the focus lens provided in the lens 101 (step S518), and the process proceeds to step S522.

  When the AF evaluation value AF_Kβ satisfies Expression (3) (Yes in step S517), the system control unit 108 turns off the AF control signal (L level) (step S519), and proceeds to step S522. In step S522, if there is an instruction to end the live view operation (Yes), the live view operation ends, and if a live view video is continuously performed (No), the process returns to step S503, and the live view operation is performed. , AF operation is performed.

  As described above, the present embodiment is premised on an imaging apparatus including an imaging element capable of performing live view imaging and imaging evaluation value detection imaging simultaneously during one frame period. Then, an image for detecting the AF evaluation value is selected from the AF evaluation value image and the live view image according to the result of the scene discrimination based on the exposure condition at the time of live view imaging. As a result, the time lag at the time of AF evaluation can be shortened when shooting a bright subject, and the accuracy of the AF evaluation value can be improved for dark portions. In this embodiment, the expected AF value to be compared with the AF evaluation value is set to a different expected value for each of the AF evaluation value image and the live view image. However, the same expected value may be used.

  In this embodiment, the AF evaluation value is used as an example of the imaging evaluation value. However, other imaging evaluation values such as a WB evaluation value (white balance data) and an AE evaluation value (exposure control data) are used. It is possible to detect an optimum evaluation value with a similar configuration. Also in the WB evaluation value and the AE evaluation value, whether the evaluation value is detected from either the imaging evaluation value detection image or the live image is switched according to the result of the scene discrimination in step S507. This makes it possible to optimize the time lag and accuracy of WB and AE evaluation value detection.

<Second Embodiment>
In the first embodiment of the present invention, a configuration has been described in which a shooting scene is determined based only on exposure amount information, and detection is performed from either an AF evaluation value AF evaluation value image or a live view image according to the result. In this embodiment, the imaging evaluation value is detected with higher accuracy by discriminating the shooting scene based on not only the exposure amount information but also face information, luminance information, color information, and the like. Hereinafter, an image pickup apparatus according to a second embodiment of the present invention will be described with reference to FIGS. In the figure, the same parts as those of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  FIG. 6 is a block diagram of the imaging apparatus according to the present embodiment. The difference from the first embodiment is that a face information detection unit 601, a luminance information detection unit 602, and a color information detection unit 603 are provided, and the other configurations are the same.

  The face information detection unit 601 performs face detection processing on the image signal output from the image sensor 102, detects the face of a person or animal (such as a pet) reflected in the frame image, and detects the detected face area. The face information is output to the scene determination unit 107.

  The luminance information detection unit 602 performs luminance detection processing on the image data image output from the image signal processing unit 103, divides the image frame image into a plurality of regions, and obtains an average luminance in each region. The luminance information detecting unit 602 obtains luminance information such as a luminance difference between the central portion of the frame image and its peripheral portion, a central luminance value, and the like using these average luminances. The luminance information detected by the luminance information detection unit 602 is output to the scene determination unit 107.

  The color information detection unit 603 performs color detection processing on the image data image output from the image signal processing 103, and detects color information such as the average saturation and the area of the high saturation area. The color information detected by the color information detection unit 603 is output to the scene determination unit 107.

  The scene discriminating unit 107 detects the background of the photographic scene from the image data image processed by the signal processing unit 103 based on the information input from the face information detecting unit 601, the luminance information detecting unit 602, and the color information detecting unit 603. And the subject of the shooting scene. Each information sent from the face information detection unit 601, the luminance information detection unit 602, and the color information detection unit 603 is temporarily stored by the scene determination unit 107 and updated as needed.

Next, a shooting scene discrimination operation in the imaging apparatus according to the present embodiment will be described.
The scene discriminating unit 107 uses the luminance information detected by the luminance detecting unit 602 and the color information detected by the color detecting unit 603 for the live view image signal output from the image sensor 102 to capture a shooting scene. Determine the background. Furthermore, using the face information detected by the face detection unit 601, a subject in the shooting scene is determined.

First, the case of determining the background of the shooting scene will be described.
The scene determination unit 107 analyzes the luminance information detected by the luminance detection unit 602 and the color information detected by the color detection unit 603, and determines whether the area of the blue sky color region on the image is equal to or greater than a threshold value. . If the area of the blue sky area is equal to or greater than the threshold, the scene determination unit 107 determines that the background of the shooting scene is blue. In addition, the scene determination unit 107 analyzes the luminance information from the luminance detection unit 602 and the color information from the color detection unit 603. As a result, when the scene determination unit 107 determines that the luminance on the image satisfies a predetermined histogram distribution or dispersion condition, the scene determination unit 107 determines that the background of the shooting scene is a night scene. For example, in the case of a dark part such as a night view, the luminance on the image is a state in which the low luminance part occupies most and the high luminance part occurs only once. Further, the scene determination unit 107 analyzes the luminance information from the luminance detection unit 602 and the color information from the color detection unit 603, and determines whether both the average saturation area and the high saturation area on the image are equal to or greater than a threshold value. If it is equal to or greater than the threshold, it is determined that the shooting scene is a vivid scene.

Next, an operation for determining a subject in a shooting scene will be described.
The scene determination unit 107 analyzes the face information from the face detection unit 601, and determines that the subject in the shooting scene is a person when a face is detected from the image signal.
As described above, the scene determination unit 107 determines both the background of the scene and the subject, combines these determination results, and outputs one determination result to the system control unit 107.

  Next, the operation of the imaging apparatus according to the present embodiment in the AF evaluation mode will be described in detail with reference to the flowchart of FIG. In order to explain the detailed operation of the present embodiment, the face information detection value detected from the live image is Xl, the luminance information detection value is Yl, and the color information detection value is Zl, which is different from the first embodiment. Only the operation will be described.

  The scene discriminating unit 107 compares the detected value of each detected information with each expected value, face information X, luminance information Y, and color information Z, and selects an AF evaluation value AF_Kα or AF_Kβ as shown in FIG. To do. According to this result, the branch in step S702 is determined.

  In step S701, the face information Xl, the luminance information Yl, and the color information Zl are detected by the face detection unit 601, the luminance detection unit 602, and the color detection unit 603 with respect to the acquired live view image. In S702, the imaging evaluation value to be reflected in the next shooting as shown in the table of FIG. 8 from each information value detected from the imaging evaluation value detection image and the live view image in S701 by the scene determination unit 107. To decide.

For example, if the relationship between each information value is as shown below,
Face information value: Xl> X
Luminance information value: Yl> Y
Color information value: Zl> Z
In step S702, imaging for AF evaluation value detection is started (that is, branching is Yes), the process proceeds to step S508, and the system control unit 108 increments the variable n by 1 (step S509). The subsequent steps are the same as in the first embodiment.

As another example, if the relationship between information values is as shown below,
Face information value: Xl ≦ Xl
Luminance information value: Yl ≦ Y
Color information value: Zl ≦ Z
In step S702, the AF evaluation value detection imaging is not started (that is, the branch is NO), and the process proceeds to S516 to detect the AF evaluation value AF_Kβ from the live view captured image. The subsequent steps are the same as in the first embodiment. The other cases of scene discrimination follow the relationship shown in FIG.

  As described above, the present embodiment is premised on an imaging apparatus including an imaging device capable of performing live view imaging and imaging evaluation value detection imaging simultaneously during one frame period. Then, an image for detecting the AF evaluation value is selected from the AF evaluation value image or the live view image in accordance with the result of determining the shooting scene based on the luminance / color / face information. Thereby, the time lag at the time of performing AF evaluation can be shortened at the time of photographing a bright subject, and the accuracy of the AF evaluation value can be improved for a dark part.

  In this embodiment, when scene determination is performed, face information, luminance information, and color information are detected from the imaging evaluation value detection image and the live view image, but the present invention is not limited thereto. By detecting the sensitivity described in the first embodiment and using the result as one factor for scene discrimination, it is possible to further optimize the AF evaluation value detection time lag and detection speed.

  In this embodiment, the imaging evaluation value is determined from the captured image when at least two of face information, luminance information, and color information, which are information necessary for scene discrimination, are large information values. It is not limited. Each information value may be weighted and compared by the imaging evaluation value.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. In this embodiment, the imaging evaluation value detection unit 106 is formed not inside the image sensor 102 but inside, for example, the pixel driving circuit of the second chip 21 shown in FIG.

  FIG. 9 is a configuration diagram of the image sensor 102 incorporating the imaging evaluation value detection unit. In the figure, the same parts as those in FIG.

  In the figure, reference numeral 901 denotes an imaging evaluation value detection unit that detects an imaging evaluation value, for example, an AF evaluation value (autofocus data), and outputs only the detected evaluation value to the system control unit 106. The imaging evaluation value detection unit 901 operates in accordance with timing controlled by the system control unit 108 based on contrast information and phase difference information obtained from an image signal obtained by the imaging element 102.

  The switches 902 and 903 are controlled by the system control unit 108, and when detecting the imaging evaluation value, based on the discrimination result of the scene discrimination unit 107, either the image signal for detecting the imaging evaluation value or the image signal for live view is selected. Whether to selectively input to the imaging evaluation value detection unit 901 is switched. The switch 902 is a switch for selecting whether to output the pixel signal corresponding to the imaging evaluation value illustrated in FIG. 3 to the signal processing unit 103 or the imaging evaluation value detection unit 901. The switch 903 is a switch that is turned on when the pixel signal corresponding to the selected row for live view illustrated in FIG. 3 is used for the imaging evaluation value.

  The operation of the image sensor when the image sensor of FIG. 9 is used in the imaging apparatus according to the first embodiment will be described with reference to the flowchart of FIG.

  In step S503, the AF operation is started. As described in the first embodiment, when the live view exposure amount Ev satisfies the expression (1) (YES) in step S507, the process proceeds to step S508, and imaging for AF evaluation value detection starts. At this time, the switch 902 is connected to the imaging evaluation value detection unit 901 side, and the switch 903 is turned off.

  After starting the AF evaluation imaging, the system control unit 108 increments the variable n by 1 (step S509). The imaging evaluation value detection unit 901 detects the AF evaluation value AF_Kα from the AF evaluation value image.

  If the exposure value Ev of the live view does not satisfy the expression (1) (No) in step S507, the process proceeds to S516. In S516, the switch 903 is turned on, and the imaging evaluation value detection unit 901 detects the AF evaluation value AF_Kβ from the image signal for live view. At this time, the switch 902 is connected to the signal processing unit 104 side. However, since the operation for the AF evaluation value is not performed, the pixel signal is not output to the signal processing unit. Since the imaging evaluation value detection unit 901 transfers only the AF evaluation value to the system control unit 108, the amount of data transferred to the outside can be reduced compared to the first embodiment, and the power consumption can be reduced. It becomes. Since the subsequent operation is the same as that of the first embodiment, the description thereof is omitted. The same operation is performed when the image sensor of FIG. 9 is used in the image pickup apparatus according to the second embodiment.

  In the third embodiment described above, by configuring the imaging evaluation value in the image sensor, the amount of data transferred when the imaging evaluation value is detected in the first and second actual examples can be reduced. It can be reduced.

  The embodiments of the present invention have been described based on the first to third embodiments. However, the present invention is not limited to these embodiments, and the scope of the present invention is not deviated. Various forms are also included in the present invention.

  For example, as a function control method of the above-described embodiment, the control unit may cause the imaging device to execute each function by controlling each unit of the device. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable storage medium, for example. Each of the above control method and control program has at least a control step and a display control step.

  The present invention can also be realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (18)

In an imaging device that has a photographing optical system that forms an optical image of a subject and that captures the optical image to generate an image signal,
A second element provided with a first element part having a pixel part in which pixels for photoelectrically converting the optical image to generate an image signal are arranged in a matrix, and a reading means for reading out the image signal from the pixel part A first readout unit that reads out an image signal from the first pixel group of the pixel unit, and a second readout unit that reads out the image signal from the second pixel group of the pixel unit. An image sensor having
Imaging condition determining means for acquiring and determining information on the imaging condition of the optical image from the image signal output by the imaging element;
An imaging evaluation value detection unit that detects an imaging evaluation value related to imaging of the optical image from any one of the image signals read by the first reading unit and the second reading unit according to the determination result;
An imaging apparatus comprising:   The information on the imaging condition is an exposure amount in imaging the optical image, and the imaging condition determination unit determines the exposure amount by comparing with a predetermined exposure amount. Imaging device.   The imaging condition determining means includes luminance information detecting means for detecting luminance information relating to an image signal output from the imaging element, color information detecting means for detecting color information relating to an image signal output from the imaging element, and the imaging A face information detecting unit that detects face information of a photographed person based on an image signal output from the element, and relates to the imaging condition according to a combination of the luminance information, the color information, and the face information; The imaging apparatus according to claim 1, wherein determination of information is determined.   Control means for controlling the reading means is further provided, and the control means controls at least one of the first reading means and the second reading means at a predetermined frame rate according to a result of determination of the information regarding the imaging condition. The imaging device according to claim 1, wherein the imaging device is driven.   The imaging apparatus according to claim 4, wherein a frame rate of a pixel signal read from the second pixel group is slower than a frame rate of the pixel signal read from the first pixel group.   The imaging apparatus according to claim 1, wherein the imaging evaluation value detection unit detects at least one of autofocus data, exposure control data, and white balance data.   The imaging apparatus according to claim 6, wherein the control unit further controls driving of the imaging optical system according to an imaging evaluation value detected by the imaging evaluation value detection unit.   The imaging apparatus according to claim 6 or 7, wherein the control unit further controls exposure of the photographing optical system according to an imaging evaluation value detected by the imaging evaluation value detection unit.   The imaging apparatus according to claim 6, wherein the control unit controls white balance of the image signal in accordance with the imaging evaluation value detected by the imaging evaluation value detection unit.   The imaging evaluation value detection unit is disposed in the second element unit, and the second element unit further detects an image signal read from the first reading unit and the second reading unit. The imaging apparatus according to claim 1, further comprising a switch for selectively inputting to the means and a converting means for converting the image signal into a digital signal.   The imaging apparatus according to claim 10, wherein the control unit switches the switch according to a determination result of information related to the imaging condition.   The imaging device according to claim 1, wherein the first pixel group and the second pixel group include pixels in different rows of the pixel unit.   The first element portion and the second element portion are stacked, and the first element portion is positioned on the side that receives the optical image. The imaging device according to one item. An imaging optical system that forms an optical image of a subject, and an imaging device that captures the optical image and generates an image signal, and the imaging device matrixes pixels that photoelectrically convert the optical image to generate an image signal A first element portion having pixel portions arranged in a shape and a second element portion provided with a reading means for reading an image signal from the pixel portion, wherein the reading means is a first portion of the pixel portion. In a control method of an imaging apparatus including an imaging element having a first readout unit that reads out an image signal from the pixel group and a second readout unit that reads out an image signal from the second pixel group of the pixel unit,
An imaging condition determination step for acquiring and determining information related to an imaging condition of the optical image from an image signal output from the imaging element, and reading by the first reading unit and the second reading unit according to the determination result An imaging evaluation value detection step for detecting an imaging evaluation value related to imaging of the optical image from any of the image signals;
A control method comprising:   A program for causing a computer to execute the control method according to claim 14.   A computer-readable storage medium storing the program according to claim 15.   A program that causes a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 13.   A storage medium storing a program that causes a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 13.

Images