JP5836673B2 - IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, AND COMPUTER PROGRAM - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging apparatus control method, and a computer program, and is particularly suitable for use in detecting deterioration in image quality of a captured image.

In recent years, imaging devices such as digital cameras and video cameras using imaging elements such as CCDs and CMOS sensors have become widespread.
As such an image pickup apparatus, an apparatus capable of switching between a still image shooting mode for recording a still image and a shooting mode (so-called moving image shooting mode) for repeatedly displaying or recording an image pickup operation and using each of the shooting modes is increasing. ing.

In recent years, in order to improve the operability of an imaging apparatus, an imaging apparatus having a live view function has also become common. The live view function is a function that enables the user to check the subject by repeatedly displaying the obtained image using the moving image shooting mode.
When the operation time for moving image shooting becomes longer or the operation time for live view becomes longer, devices inside the camera such as the image sensor generate heat, and the internal temperature of the camera rises. There is a problem in that the amount of noise such as dark current noise increases in the image sensor due to a rise in the internal temperature of the camera, and this noise causes a decrease in image quality.

  In response to this problem, Patent Document 1 discloses an imaging apparatus that detects the temperature of an image sensor using a temperature sensor, predicts degradation of image quality based on the detected temperature information, and limits or warns a live view operation. Proposed.

JP 2009-033508 A

  However, as in the technique described in Patent Document 1, when a temperature sensor is used to predict deterioration in image quality, it is necessary to place the temperature sensor close to the image sensor. In addition, it is necessary to predict deterioration in image quality in consideration of detection variations of the temperature sensor. For this reason, it is necessary to limit or warn the operation of the live view at a timing earlier than the actual deterioration of the image quality.

Further, in a large-sized image sensor that is mounted on a digital single-lens reflex camera, the temperature distribution of the image sensor is often not uniform due to a circuit layout or the like inside the image sensor. Therefore, in order to accurately detect image quality degradation based on the detection result of the temperature of the image sensor, it is necessary to accurately grasp the temperature distribution of the entire image sensor.
Therefore, in order to accurately detect the temperature distribution of the image sensor using the temperature sensor as in the technique described in Patent Document 1, it is necessary to arrange a plurality of temperature sensors close to the image sensor. For this reason, there existed the subject of the increase in a number of parts, and ensuring of the space which arrange | positions a temperature sensor. Even when a temperature distribution near the image sensor is measured using a plurality of temperature sensors, the temperature distribution is not the temperature distribution of the image sensor itself. Therefore, even if the image quality degradation is predicted using the temperature distribution, there is a problem that it is difficult to accurately predict the image quality degradation.

  Further, for such a problem, it is conceivable to detect deterioration in image quality based on a captured image. Since it is difficult to predict deterioration in image quality with high accuracy in an image in which a subject is captured, it is necessary to acquire a dark output. However, it is difficult to close the mechanical shutter in order to produce a dark output in a continuous operation state such as a moving image. Therefore, it is possible to obtain only a dark output of only a part of the area such as the optical black portion that is optically shielded in the image pickup device, and it is difficult to predict the deterioration of the image quality of the entire image pickup device.

The present invention has been made in view of the above problems, and an object of the present invention is to accurately detect deterioration in image quality of a captured image.
Specifically, for example, it is an object to accurately detect deterioration in image quality of a still image captured immediately after shooting a moving image or a live view image.
Another object is to accurately detect deterioration of the image quality of the moving image in the entire image sensor at the time of shooting the moving image, for example.

A first example of an imaging apparatus according to the present invention is a pixel having a photoelectric conversion element, and reads out a plurality of pixels arranged in two dimensions and a charge photoelectrically converted by the photoelectric conversion element, and converts the read charge into An output means for outputting a moving image or a live view image based on the image, and a charge different from the charge photoelectrically converted by the photoelectric conversion element is read from at least a part of the plurality of pixels, and image quality deterioration detection based on the read charge Acquisition means for acquiring data, and detection means for detecting deterioration in image quality using the image quality deterioration detection data acquired by the acquisition means, and the output means includes the plurality of pixels, reading out charges from the different pixels from the pixel charges are read out by the acquisition unit, the acquisition unit 1 off a moving image or live view image is output by the output means In over beam period, a period in which performs readout of charge for obtaining the moving image or the live view image, the both periods performed non period, different electric charge and which is photoelectrically converted in the photoelectric conversion element the charge of the plurality of pixels, and read out from the pixels are shielded from light, said detecting means detecting a deterioration of the image quality of a still image to be output after the output of the moving image or live view image Features.
A second example of the imaging apparatus of the present invention is a pixel having a photoelectric conversion element, and reads out a plurality of pixels arranged two-dimensionally and charges photoelectrically converted by the photoelectric conversion element, and converts the read charges into An output means for outputting a moving image or a live view image based on the image, and a charge different from the charge photoelectrically converted by the photoelectric conversion element is read from at least a part of the plurality of pixels, and image quality deterioration detection based on the read charge Acquisition means for acquiring data, and detection means for detecting deterioration in image quality using the image quality deterioration detection data acquired by the acquisition means, wherein the acquisition means is output by the output means Of the one frame period of the moving image or the live view image, at least a period during which no charge is read for obtaining the moving image or the live view image. The charge different from the charge photoelectrically converted by the photoelectric conversion element during a period in which the imaging device is driven under a condition different from that when the charge is read by the output means is obtained from at least a part of the plurality of pixels. It is characterized by reading.
A third example of the imaging apparatus of the present invention is a pixel having a photoelectric conversion element, and reads out and reads out a plurality of pixels arranged in two dimensions and a charge including a charge photoelectrically converted by the photoelectric conversion element. Output means for outputting a moving image based on the charged electric charge, and data for detecting image quality deterioration based on the read electric charge, which is read from at least a part of the plurality of pixels, and is different from the electric charge photoelectrically converted by the photoelectric conversion element. Acquisition means for acquiring image quality, and detection means for detecting image quality deterioration using the image quality deterioration detection data acquired by the acquisition means, wherein the acquisition means is a moving image output by the output means. The charge different from the charge photoelectrically converted by the photoelectric conversion element in at least a period during which the charge for obtaining the moving image is not read out in one frame period of the image, Designating and reading a part of the number of pixels is performed with different designated pixels in each of a plurality of frame periods, and the detection means detects deterioration in image quality of the moving image, and the output means If there is an instruction from the user during the output of the moving image, the moving image is enlarged based on charges read from some of the plurality of pixels corresponding to the instruction. And the obtaining means designates some of the plurality of pixels corresponding to the instruction.

  According to the present invention, it is possible to accurately detect deterioration in image quality of a captured image.

It is a figure which shows the whole structure of an imaging device. It is a figure which shows schematic structure of an image pick-up element. It is a figure which shows the circuit of the unit pixel of an image pick-up element. It is a figure which shows the whole layout of an image pick-up element. It is a timing chart which shows operation of an image sensor at the time of live view operation. It is a timing chart which shows operation of one line of an image sensor at the time of live view operation. 6 is a timing chart illustrating a first example of the operation of the image sensor when acquiring image quality degradation detection data. 12 is a timing chart illustrating a second example of the operation of the image sensor when acquiring image quality degradation detection data. It is a timing chart which shows operation of an image sensor in each output mode. 12 is a timing chart illustrating a third example of the operation of the image sensor when acquiring image quality degradation detection data. It is a figure which shows the pixel area | region which is an acquisition unit of the image quality degradation detection data. It is a figure which shows the pixel area which acquires the electronic zoom area | region at the time of video recording, and the image quality degradation detection data.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
First, a first embodiment of the present invention will be described.
FIG. 1 is a diagram illustrating an example of the overall configuration of the imaging apparatus.
In FIG. 1, the photographing lens 110 is provided with a motor (not shown). The imaging apparatus is provided with a mechanism for driving the motor and adjusting the focus of the photographing lens 110 in accordance with a processing result of AF 142 described later.

The lens control unit 111 transmits information from the photographing lens 110 to the system control circuit 150 and controls the operation of the photographing lens 110.
The lens control unit 111 includes control signal generating means. Motor driving such as an operation for adjusting the focus of the photographing lens 110 is performed by a pulse signal obtained by the control signal generating means.
The shutter 112 controls the exposure amount of the image sensor 114. The shutter 112 may be a known mechanical shutter or a liquid crystal shutter that can be shielded electrically.

The image sensor 114 converts the optical image into an electric signal. In the present embodiment, for example, a CMOS image sensor is used as the image sensor 114. The internal configuration of the image sensor 114 will be described later with reference to FIG.
The LPF 115 is a low-pass filter for cutting an extra wavelength of light transmitted through the photographing lens 110 (an unnecessary wavelength that affects color reproduction). The LPF 115 is disposed between the photographing lens 110 and the image sensor 114.

The AFE 116 is an analog front end circuit including an A / D converter that converts an analog signal output from the image sensor 114 into a digital signal, a clamp circuit (offset adjustment circuit), and a D / A converter. is there.
The DFE 117 is a digital front end circuit that inputs a digital signal output from each pixel via the AFE 116 and performs digital processing such as correction and rearrangement on the digital signal. The DFE 117 acquires correction data stored in a memory 152 or a nonvolatile memory 156, which will be described later, as necessary, and uses the correction data for digital signal correction processing.

The TG 118 is a timing generation circuit that supplies a clock signal and a control signal to the image sensor 114, the AFE 116, and the DFE 117. The TG 118 is controlled by the system control circuit 150. Specifically, the TG _{118 includes} TG (1) 118a and TG (2) 118b that can generate reference signals TG _sig1 and TG _sig2 of a plurality of systems. When the live view image is output, a drive signal is supplied to the display area of the captured image from the reference signal TG _sig1 generated by the TG (1) 118a. When acquiring the image quality degradation detection data during the output of the live view image, the drive signal is supplied from the reference signal TG _sig2 generated by the TG (2) 118b. That is, in the imaging apparatus of the present embodiment, it is possible to switch between the first output mode for obtaining a live view image and the second output mode for obtaining image quality degradation detection data by the operation of the TG 118.

The image processing circuit 120 performs predetermined pixel interpolation processing and color conversion processing on the data from the DFE 117 or the data from the memory control circuit 122. The image processing circuit 120 performs predetermined arithmetic processing using the captured image data as necessary.
The memory control circuit 122 controls the AFE 116, DFE 117, TG 118, the image processing circuit 120, the image display memory 124, and the memory 130.
Data from the DFE 117 is written into the image display memory 124 or the memory 130 via the image processing circuit 120 and the memory control circuit 122 or via the memory control circuit 122.

The image processing circuit 120 corrects the charge signal output from the image sensor 114 based on correction data stored in a memory 152 described later, and performs development processing such as color conversion on each output signal. To go and image.
In addition, the image processing circuit 120 according to the present embodiment includes a data operation unit 120a, and can perform focus detection and brightness detection from an image. Furthermore, the image processing circuit 120 can send a control signal to the lens control unit 111 via the system control circuit 150 based on the data detected by the data calculation unit 120a. With this control signal, an operation of adjusting the focus of the photographing lens 110 can be performed.

The image display unit 128 includes, for example, a TFT liquid crystal display unit. Images are continuously displayed on the screen of the image display unit 128 when a live view image is output and a moving image is captured. The user can confirm the movement of the subject by viewing this image.
The memory 130 stores captured still images and moving images. The memory 130 has a storage capacity sufficient to store a predetermined number of still images and a moving image for a predetermined time.
The shutter control unit 140 controls the operation of the shutter 112.

The AF 142 is a distance measurement control unit that is a distance measurement unit for performing AF (autofocus) processing.
The AE 146 is a photometric control unit that is a photometric means for performing AE (automatic exposure) processing. The flash unit 148 is used for photographing in the dark and generates a flash. The AE 146 also has a flash photographing function in cooperation with the flash unit 148.
The system control circuit 150 controls the entire image processing apparatus. The system control circuit 150 includes a CPU and the like.

The memory 152 stores variables for operating the system control circuit 150, programs, and the like.
The nonvolatile memory 156 stores a program and the like which will be described later. The nonvolatile memory 156 is realized by, for example, an electrically erasable / recordable flash memory. The non-volatile memory 156 stores various parameters, setting values such as ISO sensitivity, setting modes, various correction data, and the like.
The operation unit 160 includes various user interfaces for inputting various operation instructions to the system control circuit 150. For example, the operation unit 160 includes a main switch (startup switch), a shutter switch, a mode setting dial for switching the shooting mode, and the like. By pressing a release switch included in the operation unit 160, the two switches (SW1, SW2) are turned on stepwise.

  Operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash dimming) processing are performed by the first-stage operation of the release switch (SW1 ON). The Then, in the second stage operation (SW2 ON), a signal from the image sensor 114 is written as image data in the memory 130 via the AFE 116, the DFE 117, and the memory control circuit 122 under the control of the shutter control unit 140 and the like. Then, an exposure process is performed. Then, a development process based on the calculation in the image processing circuit 120 and the memory control circuit 122 and a recording process in which the image data in the memory 130 is compressed and written to the recording medium 190 are started as a series of processes.

  The operation unit 160 further includes a live view operation switch for continuously displaying the subject on the image display unit 128, a mode setting dial for switching various shooting modes, and a single shooting for switching between single shooting / continuous shooting. / Continuous shooting switch. In addition, the operation unit 160 supplies power to a continuous ranging operation setting switch for repeatedly performing AF processing and lens focusing processing, an ISO sensitivity setting switch for setting photographing sensitivity, and various systems. Including a power switch and the like.

The power supply control unit 182 controls the power supply unit 186 and includes a battery detection circuit, a DC-DC converter, and the like. The power supply unit 186 is a power supply unit including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter.
The recording medium 190 is a detachable recording medium such as a memory card or a hard disk.

Next, an example of the configuration of the image sensor 114 (CMOS image sensor) will be described with reference to FIGS.
FIG. 2 is a diagram illustrating an example of a schematic configuration of the image sensor 114. The image sensor 114 has a normal HOB area (horizontal optical black area) 51 in which pixels are shielded, a VOB area (vertical optical black area) 52 in which pixels are shielded in the same manner as the HOB area 51, and a normal light shielding. And an effective portion 53 which is a non-pixel area. In the present embodiment, the output of the VOB area 52 is used for detection of image quality degradation.

FIG. 3 is a diagram illustrating an example of a circuit of the unit pixel 60 (for one pixel) of the image sensor 114.
The photodiode 61 is an example of a photoelectric conversion element. The photodiode 61 receives a light image formed by the photographing lens 110, generates a charge (photoelectric conversion), and accumulates it. In the following description, the photodiode 61 is referred to as PD 61 as necessary. The transfer switch 62 is composed of a MOS transistor. In the following description, the transfer switch 62 is referred to as TX62 as necessary. The floating diffusion 64 is a capacitor. In the following description, the floating diffusion 64 is referred to as FD 64 as necessary. The electric charge accumulated in the PD 61 is transferred to the FD 64 by the operation of the TX 62 and converted into a voltage, and is output from the amplifier 65 as a pixel signal by the source follower. The row selection switch 66 operates when a pixel signal is output to the vertical output line 67. The reset switch 63 resets the potential of the FD 64 and PD 61. The reset switch 63, the amplifier 65, and the row selection switch 66 are composed of MOS transistors.

FIG. 4 is a diagram illustrating an example of the overall layout of the image sensor 114.
In the example shown in FIG. 4, the unit pixels 60 shown in FIG. 3 are arranged in a two-dimensional matrix (pixels 60 (1, 1) to 60 (n, m)).
The accumulation control of each unit pixel 60 is performed by a signal from the vertical scanning circuit 77a or 77b which is a vertical selection unit. In the present embodiment, the vertical scanning circuits 77a and 77b are constituted by shift registers.

The reference signals for the vertical scanning circuits 77a and 77b are reference signals TG _sig1 and TG _sig2 sent from the TG ₁₁₈ .
It is the control line of each horizontal line that sets which signal of the vertical scanning circuits 77a and 77b is used as the control signal for each horizontal line of the pixel. The control lines include a charge transfer control line TX that controls the TX 62, a reset control line RES, and a row selection control line SEL. Switch groups SWt_x_1 and SWt_x_2 are connected to the charge transfer control line TX. A switch group SWr_x_1 and SWr_x_2 are connected to the reset control line RES. A switch group SWs_x_1 or SWs_x_2 is connected to the row selection control line SEL. Here, x is 1 to n (n is an integer of 2 or more and the number of rows of the two-dimensional matrix).

When the switch group SWt_x_1, SWr_x_1, and SWs_x_1 are ON (closed), the switch group SWt_x_2, SWr_x_2, and SWs_x_2 are OFF, and the control signal (reference signal TG _sig1 ) of the vertical scanning circuit 77a operates. Further, when the switch group SWt_x_2, SWr_x_2, and SWs_x_2 are ON, the switch group SWt_x_1, SWr_x_1, and SWs_x_1 are OFF, and the control signal (reference signal TG _sig2 ) of the vertical scanning circuit 77b is operated.

  The switch groups SWt_x_1, SWr_x_1, SWs_x_1 and the switch groups SWt_x_2, SWr_x_2, SWs_x_2 are turned on / off by signals from the system control circuit 150. Only one of the switch groups SWt_x_1, SWr_x_1, SWs_x_1 and the switch groups SWt_x_2, SWr_x_2, SWs_x_2 is turned on.

  Signals from the vertical scanning circuit 77a include a control signal φTXa for the transfer switch 62, a control signal φRESa for the reset switch 63, and a control signal φSELa for the row selection switch 66. The control signal φTXa of the transfer switch 62 is output to each pixel 60 via the switch group SWt_x_1. The control signal φRESa of the reset switch 63 is output to each pixel 60 via the switch group SWr_x_1. The control signal φSELa of the row selection switch 66 is output to each pixel 60 via the switch group SWs_x_1.

  On the other hand, the signal from the vertical scanning circuit 77b includes a control signal φTXb for the transfer switch 62, a control signal φRESb for the reset switch 63, and a control signal φSELb for the row selection switch 66. The control signal φTXb of the transfer switch 62 is output to each pixel via the switch group SWt_x_2. The control signal φRESb of the reset switch 63 is output to each pixel via the switch group SWr_x_2. The control signal φSELb of the row selection switch 66 is output to each pixel 60 via the switch group SWs_x_2.

The vertical output line 67 (y) is connected to the pixels arranged in the vertical direction. Here, y is 1 to m (m is an integer of 2 or more and the number of columns of the two-dimensional matrix).
The vertical output line 67 (y) is connected to the SN circuit 75a or 75b for each line. Control such as output selection of the S-N circuits 75a and 75b (75 (1) to 75 (m)) is performed by the horizontal scanning circuits 76a and 76b. Read signals ΦPTN and ΦPTS (not shown) are supplied to the S-N circuits 75a and 75b. The S-N circuits 75a and 75b read out the signal component and noise component of each pixel 60 based on the readout signals ΦPTN and ΦPTS, perform differential operations, and output them to the output amplifiers 74a and 74b. The output amplifiers 74a and 74b can change the driving capability using a current switching circuit (not shown).

Horizontal scanning circuit 76a where a horizontal selection means, the 76 b, TG (1) the reference signal generated by 118a TG _sig1, or TG (2) signal with the reference signal TG _sig2 generated in 118b is sent . The horizontal scanning circuits 76a and 76b can individually send drive signals corresponding to the respective outputs.
In the present embodiment, the horizontal scanning circuits 76a and 76b are composed of shift registers in the same manner as the vertical scanning circuits 77a and 77b.

  In addition, when outputting image quality degradation detection data, the vertical scanning circuits 77a and 77b and the horizontal scanning circuits 76a and 76b are used to detect pixels from only a desired partial pixel area used for detection of degradation in image quality, not all pixel areas. The data (pixel value) is read out. That is, the drive signal is controlled so that the vertical scanning circuits 77a and 77b and the horizontal scanning circuits 76a and 76b are connected only to a desired pixel region.

In the present embodiment, the pixel area used at the time of detecting image quality degradation is the VOB area 52, and for example, pixel data in the first and second rows is used as image quality degradation detection data. That is, when the pixels in the first and second rows, which are the VOB area 52, are selected, the drive signal sent to the SN circuits 75 (1), 75 (m), etc. is output using the reference signal TG _sig2 as a reference signal. The The output from the S-N circuit 75 is output to a subsequent processing circuit (AFE 116 or the like) via output amplifiers 74a1, 74a2, 74b1, and 74b2.
The output amplifier 74 includes four output amplifiers 74a1, 74a2, 74b1, and 74b2, and signals are distributed and output according to the output path.

Next, an example of the rolling accumulation operation of the image sensor 114 during the live view operation will be described with reference to FIG. FIG. 5 is a timing chart for explaining an example of the outline of the operation of the image sensor 114 during the live view operation.
During the live view operation, “reset-accumulation-transfer-read” is performed to obtain a first output (image output for display) for each row. As a result, by using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76, one frame of live view display data that is display data during the live view operation is read. After that, as the time until the start of data acquisition for the next frame, an “adjustment period for operating at a frame rate (for example, 30 fps)” determined in advance to perform stable continuous operation as the system of the imaging apparatus. A certain frame adjustment period is provided. By doing so, a live view operation is performed at a predetermined frame rate.

FIG. 6 is a timing chart for explaining an example of detailed driving of one row of the image sensor 114 during the live view operation. An example of detailed operation timing for driving one row during the live view operation shown in FIG. 5 will be described with reference to FIG.
First, the resetting of the pixel portion starts at time T0. At time T0, the vertical scanning circuit 77a activates the reset signal ΦRESa (n) for controlling the reset switch 63 and the transfer signal ΦTXa (n) for controlling the transfer switch 62. Thereby, PD61 and FD64 will be in the reset state.

Subsequently, at time T1, the vertical scanning circuit 77a turns off the transfer signal ΦTXa (n) and the reset signal ΦRESa (n). At this stage, the charge accumulation operation to the PD 61 starts.
Next, the vertical scanning circuit 77a activates the reset signal ΦRESa (n) at time T2 immediately before reading out the charge, and turns off the reset signal ΦRESa (n) at time T3. That is, the dark current (charge) accumulated in the FD 64 during the accumulation operation is reset during the period of time T2 to T3.
Subsequently, at time T <b> 4, the vertical scanning circuit 77 a activates the row selection signal ΦSELa (n) that controls the row selection switch 66 to transfer the charges accumulated in the FD 64 to the vertical output line 67.

Next, in the period of time T5 to T6, the horizontal scanning circuit 76 activates the readout signal ΦPTN (n), outputs the potential of the FD 64 to the SN circuit 75, and reads out the noise component. That is, reading is performed in a reset state without an optical signal.
Next, in the period of time T7 to T8, the vertical scanning circuit 77a activates the transfer signal ΦTXa (n), and transfers the charge accumulated in the PD 61 to the FD 64. That is, the period of time T1 to T8 is an exposure period (charge accumulation period) as the imaging device.

Subsequently, in the period of time T8 to T9, the horizontal scanning circuit 76 activates the read signal ΦPTS (n), outputs the potential of the FD 64 to the SN circuit 75, and reads the signal component. That is, the optical signal stored in the PD 61 and transferred to the FD 64 is read.
Next, at time T <b> 10, the vertical scanning circuit 77 a turns off the row selection signal ΦSELa (n), and at the same time, the horizontal scanning circuit 76 transmits a control signal to the SN circuit 75. The S-N circuit 75 sequentially starts a read operation of outputting a signal obtained by subtracting the output of noise in the reset state of the FD 64 from the accumulated charge of light to the output amplifier 74.

The above is an example of the accumulation-reading operation for one horizontal line in the moving image shooting mode. Similarly, the reset signal ΦRESa (n + 1), transfer signal ΦTXa (n + 1), row selection signal ΦSELa (n + 1), read signal ΦPTN (n + 1), and read signal ΦPTS (n + 1) are used for the next target row. The operation described above is performed.
Here, the drive signal at the time of obtaining the image output for live view display is output using the reference signal TG _sig1 as a reference signal.

  The above is the basic operation of the live view operation. In the present embodiment, the still image captured immediately after the live view operation is obtained by using the frame adjustment period during the live view operation and obtaining the output of the image quality degradation detection data so as to be within the frame adjustment period. Predict deterioration of image quality.

Next, an example of an operation for acquiring image quality deterioration detection data will be described with reference to FIG. FIG. 7 is a timing chart for explaining an example of an outline of the operation of the image sensor 114 when acquiring image quality degradation detection data.
First, as described with reference to FIG. 5, “reset-accumulation-transfer-read” is performed to obtain an image output for live view display using the reference signal TG _sig1 as a reference signal. That is, reading of one frame of image data for live view display is performed using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76.

Next, an operation for obtaining image quality degradation detection data is performed within the frame rate adjustment period. Here, when acquiring the frame of the image quality degradation detection data, the reference signal TG _sig2 is used as a reference signal. As a data area for image quality deterioration detection data, a part of the VOB area 52, for example, the pixel area in the first and second rows is used. That is, the first row is selected based on the reference signal TG _Sig2 output from the TG 118. Then, reset-accumulation-transfer-read is performed. That is, the image quality deterioration detection data in the first row is read using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76.
For the second row, reset-accumulation-transfer-read is performed by the same operation as the first row. As described above, two rows of data in the VOB area 52 are acquired as the image quality deterioration detection data.

Next, using the reference signal TG _sig1 as a reference signal, “reset-accumulation-transfer-read” is performed to obtain an image output of the next frame for live view display. That is, using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76, one frame of data for live view operation is read. The above operations are repeated at a predetermined frame rate, and output of live view display image data and output of image quality deterioration detection data are performed.

Then, the degree of image quality deterioration is determined using the acquired image quality deterioration detection data. For example, the image processing circuit 120 calculates the standard deviation of the value of the image quality deterioration detection data. Then, the image processing circuit 120 determines whether or not the calculated standard deviation value is greater than or equal to a predetermined value. If the result of this determination is that the standard deviation value is greater than or equal to a predetermined value, it is determined that the image quality has deteriorated. When it is determined that the image quality is deteriorated, the image processing circuit 120 displays a warning display on the screen of the image display unit 128 on which the live view image is displayed to inform the user that the image quality is deteriorated. Do. Further, instead of performing such warning display, the image processing circuit 120 may end the live view operation.
The above is an example of the operation of acquiring image quality deterioration detection data.

  Here, as the image quality degradation detection data, it is necessary to estimate with high accuracy the degradation of the image quality of a still image taken immediately after. The deterioration of the image quality of the still image due to the temperature rise inside the imaging device is caused by an increase in the amount of noise such as noise due to dark current. At this time, as the charge accumulation time increases, the degradation of the image quality of the still image becomes more conspicuous due to noise caused by dark current. Therefore, it is possible to improve the detection accuracy of image quality degradation by acquiring the image quality degradation detection data with the charge accumulation time extended as much as possible.

As described above, in the present embodiment, in one frame, in addition to the output operation for obtaining an image for normal live view display, the image quality is determined during the frame adjustment period in which the pixel signal of the image is not read out. Acquire deterioration detection data. Then, using this image quality deterioration detection data, the deterioration of the image quality of the still image picked up immediately after is detected. At this time, image quality deterioration detection data is acquired from the VOB area 52. For this reason, it is possible to extend the charge accumulation time within a range that falls within the frame adjustment period independently of the image exposure condition. Therefore, it is possible to accurately detect deterioration in image quality due to a rise in the internal temperature of the imaging device of a still image captured immediately after the live view operation.
Here, in order to improve detection accuracy of image quality degradation, other operating conditions other than the charge accumulation time may be different from the conditions during the output operation for obtaining an image for normal live view display. For example, the image quality degradation detection data may be acquired with a high ISO sensitivity at which the image quality degradation is significant. Further, since the deterioration of image quality is affected by the amount of heat generated inside the image pickup apparatus during the charge accumulation period, the magnitude of the drive current of the output amplifier 74 is set to an output for obtaining an image for normal live view display. The image quality deterioration detection data may be acquired by changing the value to a value larger than that at the time of operation.

  As described above, in this embodiment, it is possible to change the conditions at the time of acquiring image quality degradation detection data independently of the conditions at the time of acquiring data for live view display. Therefore, by setting the driving conditions for the imaging device suitable for detecting image quality degradation, and driving the imaging device under the driving conditions to generate image quality degradation detection data, the detection of image quality degradation can be further enhanced. It becomes possible to carry out with accuracy.

The present embodiment is not limited to switching the output mode from the live view image display mode to the still image shooting mode. The present invention can be similarly applied when the output mode is switched from the moving image shooting mode for recording a moving image to a recording medium to the still image shooting mode.
Further, temperature information of the image sensor 114 or the like is acquired using an output from a temperature sensor (not shown), and the above-described deterioration in image quality is detected when the acquired temperature exceeds a predetermined temperature. good.
In this embodiment, the image quality deterioration detection data is acquired for each frame. However, the image quality deterioration detection data may be acquired once for a plurality of frames.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, the image quality degradation detection data is acquired during the frame adjustment period. On the other hand, in the present embodiment, a part of the image sensor 114 is not used for live view display, but is used for detection of image quality deterioration, and a live display data acquisition period and frame are used. Data for detecting image quality deterioration is acquired in both the adjustment period and the period. By doing so, it is possible to further extend the charge accumulation period for obtaining the image quality degradation detection data regardless of the frame rate of the live view image. As described above, the present embodiment and the first embodiment are mainly different in part of the operation of the imaging apparatus when acquiring image quality degradation detection data. Therefore, in the description of the present embodiment, the same parts as those of the first embodiment are denoted by the reference numerals attached to FIGS.

In the present embodiment, for example, the pixel areas in the first and second rows of the VOB area 52 are used only for detection of image quality deterioration, and are not used for live view display.
An example of the operation for acquiring the image quality degradation detection data will be described with reference to FIG. FIG. 8 is a timing chart showing an example of an outline of the operation of the image sensor 114 when acquiring image quality degradation detection data.
First, as described with reference to FIG. 5, reset-accumulation-transfer-readout for obtaining an output of an image for live view display is performed using the reference signal TG _sig1 as a reference signal. That is, reading of one frame of image data for live view display is performed using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76.

Here, in the present embodiment, the pixel area used as image data for live view display is a pixel area excluding the first and second rows of the VOB area 52. That is, by using the reference signal TG _sig1 as a reference signal, the horizontal scanning circuits 76a and 76b and the vertical scanning circuits 77a and 77b are driven to select the pixel region except the first and second rows of the VOB region 52.

Next, an example of an operation for obtaining image quality deterioration detection data will be described. When acquiring image quality degradation detection data, the reference signal TG _sig2 is used as a reference signal. As a data area used for detection of image quality deterioration, a part of the VOB area 52 (in the example of the present embodiment, the first and second rows) is used. That is, the first row is selected based on the reference signal TG _Sig2 output from the TG 118. Then, reset-accumulation-transfer-read is performed for the first row. That is, the image quality deterioration detection data in the first row is read using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76.
For the second row, reset-accumulation-transfer-read is performed by the same operation as the first row. As described above, two rows of data in the VOB area 52 are acquired as the image quality deterioration detection data.

Here, the pixel areas for the first and second rows of the VOB area 52 are set as pixel areas that are not used when data for live view display is acquired. Therefore, as shown in FIG. 8, for the pixel regions for the first and second rows of the VOB region 52, the charge region is independent of the pixel region used for acquiring image data for live view display. It is possible to set the accumulation time. That is, even at the timing when the pixel area of the image data for live view display is read, the charge accumulation is continued in the pixel areas for the first and second rows of the VOB area 52. Then, after the desired accumulation time has elapsed, using the reference signal TG _sig2 , acquisition of image quality degradation detection data can be continuously performed within the frame adjustment period.

Then, the degree of image quality deterioration is determined from the acquired image quality deterioration detection data. For example, the image processing circuit 120 calculates the standard deviation of the image quality deterioration detection data. Then, the image processing circuit 120 determines whether or not the calculated standard deviation value is greater than or equal to a predetermined value. As a result of this determination, if the standard deviation value is greater than or equal to a predetermined value, the image quality is degraded. When it is determined that the image quality is deteriorated, the image processing circuit 120 notifies the user that the image quality is deteriorated on the screen of the image display unit 128 on which the live view image (moving image) is displayed. A warning is displayed. Further, instead of performing such warning display, the image processing circuit 120 may end the live view operation.
The above is an example of the operation of acquiring image quality deterioration detection data.

  Here, in the present embodiment, the pixel area for detecting image quality degradation is set as an independent area different from the pixel area for displaying the live view image. Therefore, an accumulation time longer than one frame period of the live view image can be set as the charge accumulation time for obtaining the image quality degradation detection data. That is, charge can be accumulated in a long time such as 1 second or 2 seconds. Therefore, it is possible to detect the deterioration of the image quality of a still image taken immediately after the image with higher accuracy.

The present embodiment is not limited to switching the output mode from the live view image display mode to the still image shooting mode. The present invention can be similarly applied when the output mode is switched from the moving image shooting mode for recording a moving image to a recording medium to the still image shooting mode.
Further, temperature information of the image sensor 114 or the like may be acquired using a temperature sensor output (not shown), and the above-described deterioration in image quality may be detected when the acquired temperature is equal to or higher than a predetermined temperature.
In this embodiment, the image quality deterioration detection data is acquired for each frame. However, the image quality deterioration detection data may be acquired once for a plurality of frames.

(Third embodiment)
Next, a third embodiment of the present invention will be described. In the first and second embodiments, a case where a deterioration in image quality of a still image is detected when a still image is captured when a live view image is displayed or a moving image is captured is taken as an example. I gave it as an explanation. On the other hand, in the present embodiment, degradation of the image quality of the moving image is detected when the moving image is captured. As described above, the present embodiment and the first and second embodiments are mainly different in processing due to different targets for detecting deterioration of image quality. Therefore, in the description of the present embodiment, the same parts as those in the first and second embodiments are denoted by the reference numerals given in FIGS.

First, an example of the overall configuration of the imaging apparatus is the same as that shown in FIG. However, in the first and second embodiments, the TG 118 switches between the first output mode for obtaining a live view image and the second output mode for obtaining image quality degradation detection data. On the other hand, in the present embodiment, the first output mode (moving image shooting mode) in which a moving image is captured and recorded on a recording medium by the TG 118, and image quality degradation is detected (image quality degradation detection data is stored). The second output mode (acquired) (image quality deterioration detection mode) is switched. In the moving image shooting mode, a drive signal is supplied from the reference signal TG _sig1 generated by the TG (1) 118a. On the other hand, in the image quality deterioration detection mode, a drive signal is supplied from the reference signal TG _sig2 generated by TG (2) 118b.

When a moving image is output in the moving image shooting mode, read signals ΦPTNa and ΦPTSa (not shown) generated by the reference signal TG _sig1 are output to the SN circuits 75a and 75b. On the other hand, when outputting image quality deterioration detection data in the image quality deterioration detection mode, read signals ΦPTNb and ΦPTSb (not shown) generated by the reference signal TG _sig2 are output to the SN circuits 75a and 75b.

In the present embodiment, the pixel area used for moving image output in the moving image shooting mode is an area of the entire screen. That is, when a moving image is output in the moving image shooting mode, the drive signal sent to the SN circuits 75 (1), 75 (m), etc. is for light output described later using the reference signal TG _sig1 as a reference signal. SN driving is performed. On the other hand, the drive signal sent to the SN circuits 75 (1), 75 (m), etc. when acquiring the image quality deterioration detection data in the image quality deterioration detection mode uses the reference signal TG _sig2 as a reference signal as a whole screen. Of these, only the necessary region is subjected to NN driving described later.

Next, FIG. 9 is a timing chart for explaining an example of the operation of the image sensor 114 in each output mode (moving image shooting mode and image quality deterioration detection mode). Specifically, FIG. 9A is a timing chart for explaining an example of the operation of the image sensor 114 in the moving image shooting mode, and FIG. 9B is an example of the operation of the image sensor 114 in the image quality deterioration detection mode. It is a timing chart to explain.
First, FIG. 9A will be described.
First, resetting of the pixel portion is started at time T0. At time T0, the vertical scanning circuit 77a activates the reset signal ΦRESa (n) and the transfer signal ΦTXa (n). Thereby, PD61 and FD64 will be in the reset state.

Subsequently, at time T1, the vertical scanning circuit 77a turns off the transfer signal ΦTXa (n) and the reset signal ΦRESa (n). At this stage, the charge accumulation operation to the PD 61 starts.
Next, the vertical scanning circuit 77a activates the reset signal ΦRESa (n) at time T2 immediately before reading out the charge, and turns off the reset signal ΦRESa (n) at time T3. That is, the dark current (charge) accumulated in the FD 64 during the accumulation operation is reset during the period of time T2 to T3.
Subsequently, at time T <b> 4, the vertical scanning circuit 77 a activates the row selection signal ΦSELa (n) to transfer the charges accumulated in the FD 64 to the vertical output line 67.

Next, in the period of time T5 to T6, the horizontal scanning circuit 76 activates the readout signal ΦPTN (n), outputs the potential of the FD 64 to the SN circuit 75, and reads out the noise component. That is, reading is performed in a reset state without an optical signal.
Next, in the period of time T7 to T8, the vertical scanning circuit 77a activates the transfer signal ΦTXa (n), and transfers the charge accumulated in the PD 61 to the FD 64. That is, the period of time T1 to T8 is an exposure period (charge accumulation period) as the imaging device.

Subsequently, in the period of time T8 to T9, the horizontal scanning circuit 76 activates the read signal ΦPTS (n), outputs the potential of the FD 64 to the SN circuit 75, and reads the signal component. That is, the optical signal stored in the PD 61 and transferred to the FD 64 is read.
Next, at time T <b> 10, the vertical scanning circuit 77 a turns off the row selection signal ΦSELa (n), and at the same time, the horizontal scanning circuit 76 transmits a control signal to the SN circuit 75. The S-N circuit 75 sequentially starts a read operation of outputting a signal obtained by subtracting the output of noise in the reset state of the FD 64 from the accumulated charge of light to the output amplifier 74.
The above is an example of the accumulation-read operation for one row in the horizontal direction. Similarly, for the next target row, using the reset signal ΦRESa (n + 1), the transfer signal ΦTXa (n + 1), the row selection signal ΦSELa (n + 1), the read signal ΦPTNa (n + 1), and the read signal ΦPTSa (n + 1). The operation described above is performed.

Subsequently, FIG. 9B will be described.
Basically, the operation is the same as that shown in FIG. 9A except for the operation of the transfer signal ΦTXb (n) (ΦRESa (n) = ΦRESb (n), ΦSELa (n) = ΦSELb (n)). The difference from FIG. 9A is that the vertical scanning circuit 77b does not activate the transfer signal ΦTXb (n) in the period of time T7 to T8. Therefore, the charge accumulated in the PD 61 is not transferred to the FD 64 during this period.

Therefore, when the horizontal scanning circuit 76 activates the readout signal ΦPTSb (n) during the period of time T8 to T9, the signal read out to the SN circuit 75 is an optical signal (charge accumulated in the PD 61). Instead, the charge of the FD 64 is at the reset potential.
That is, the operation in FIG. 9B is an operation in the dark output mode in which only the noise component of the circuit system is output without the light output of the PD 61 being made.
The above is an example of the reading operation for one row in the horizontal direction in the image quality degradation detection mode. Similarly, the reset signal ΦRESb (n + 1), the transfer signal ΦTXb (n + 1), the row selection signal ΦSELb (n + 1), the read signal ΦPTNb (n + 1), and the read signal ΦPTSb (n + 1) are used for the next target row. The operation described above is performed. In the present embodiment, degradation of image quality is detected using dark output data including circuit-related noise caused by a temperature rise inside the imaging apparatus.

Next, an example of an operation of acquiring image quality deterioration detection data during moving image shooting will be described with reference to FIG. FIG. 10 is a timing chart for explaining an example of an outline of the operation of the image sensor 114 when acquiring image quality degradation detection data.
At the time of moving image shooting, “reset-accumulation-transfer-read” is performed to obtain a first output (display image output) for each row. Thus, one frame of image data for moving image shooting is read out using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76. After that, as a time until the start of data acquisition of the next frame, a frame that is an adjustment period for operating at a predetermined frame rate (for example, 30 fps) in order to perform stable continuous operation as the system of the imaging apparatus Set an adjustment period. In this way, a moving image shooting operation is performed at a predetermined frame rate.
In this embodiment, using this frame adjustment period, image quality deterioration detection data (dark output data) is obtained so that it falls within this frame period, and image quality deterioration is reduced using this image quality deterioration detection data. Predict.
First, using the reference signal TG _sig1 as a reference signal, reset-accumulation-transfer-readout for obtaining an image output for moving image shooting is performed. That is, one frame of image data for moving image shooting is read using the vertical scanning circuit 77a, the SN circuit 75, and the horizontal scanning circuit 76. Here, using the reference signal TG _sig1 , the output of image data (light output image) for moving image shooting is obtained in the SN drive mode.

Next, an operation for obtaining image quality degradation detection data is performed within the frame rate adjustment period. Here, using the reference signal TG _sig2 as a reference signal, an output (dark output) of image quality deterioration detection data is obtained in the NN drive mode.
In order to accurately detect image quality deterioration of the entire image sensor 114, the data area of the image quality deterioration detection data needs to be the entire pixel area of the image sensor 114. However, in order to satisfy a predetermined frame rate, the output of all the pixel regions of the image sensor 114 cannot be acquired within the frame rate adjustment period. Therefore, in the present embodiment, data of pixel regions that fall within the frame rate adjustment period among all the pixel regions of the image sensor 114 is acquired. That is, within one frame period, a part of the pixel area of the image sensor 114 is designated, and image degradation detection data is acquired from the designated pixel area. Then, the pixel area is changed for each frame, and the image quality deterioration detection data is acquired from all the pixel areas of the image sensor 114 over a period of a plurality of frames (“R1 area data NN reading” in FIG. 10). , “R2 area data NN read”).

FIG. 11 is a diagram illustrating an example of a pixel area that is a unit for acquiring image quality degradation detection data. In the example shown in FIG. 11, the entire pixel region of the image sensor 114 is divided into six in the horizontal direction, five in the vertical direction, and divided into 30 pixel regions.
In the present embodiment, image quality deterioration detection data for the pixel region R1 is acquired in the first frame adjustment period. In the frame adjustment period of the next frame, data of the pixel region R2 is acquired. In this way, by sequentially switching the pixel area from which the image quality deterioration detection data is acquired, the dark output data of all the pixel areas of the image sensor 114 can be acquired as the image quality deterioration detection data with 30 frames as one cycle. . Therefore, even when the temperature distribution of the image sensor 114 is non-uniform and the degree of image quality degradation varies depending on the location of the image sensor 114, it is possible to accurately detect image quality degradation.

Then, the degree of image quality deterioration is determined using the acquired image quality deterioration detection data for each pixel region Ri (i = 1 to 30). For example, the image processing circuit 120 calculates the standard deviation of the image quality deterioration detection data. Then, the image processing circuit 120 determines whether or not the calculated standard deviation value is greater than or equal to a predetermined value. If the result of this determination is that the standard deviation value is greater than or equal to a predetermined value, it is determined that the image quality has deteriorated. When it is determined that the image quality is deteriorated, the image processing circuit 120 displays a warning on the screen of the image display unit 128 on which the moving image is displayed to notify the user that the image quality is deteriorated. . Further, instead of performing such a warning display, the image processing circuit 120 may end the moving image shooting operation.
The above is an example of the operation of acquiring image quality deterioration detection data.

  As described above, in this embodiment, in one frame, apart from the output operation for obtaining a moving image, the image quality degradation detection data is acquired during the frame adjustment period in which the pixel signal of the image is not read. To do. At this time, acquiring the image quality degradation detection data that can be acquired in one frame adjustment period from the plurality of pixel areas is sequentially performed in the plurality of frame periods, and the image quality degradation detection data is obtained from all the pixel areas of the image sensor 114. get. Therefore, it is possible to accurately detect the deterioration of the image quality of the entire image sensor 114 without deteriorating the operational feeling of the image capturing apparatus when capturing a moving image.

Here, in the present embodiment, the example in which the pixel areas of the image sensor 114 are sequentially switched to acquire the image quality deterioration detection data of all the pixel areas of the image sensor 114 has been described. However, for example, only the image quality deterioration detection data of the pixel region suitable for extracting the characteristics of the temperature distribution of the image sensor 114 may be acquired. For example, areas corresponding to the four corners and the center of the image sensor 114 (pixel areas R1, R5, R13, R18, R26, and R30 in the example shown in FIG. 11) may be sequentially acquired.
Further, temperature information of the image sensor 114 or the like may be acquired using a temperature sensor output (not shown), and the above-described deterioration in image quality may be detected when the acquired temperature is equal to or higher than a predetermined temperature.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. In the present embodiment, when a moving image is enlarged and photographed with an electronic zoom, the deterioration of the image quality of the moving image is detected. As described above, in the third embodiment, deterioration of the image quality of a moving image that has not been enlarged is detected, whereas in this embodiment, deterioration of the image quality of a moving image enlarged by electronic zoom is detected. Mainly different. Therefore, in the description of the present embodiment, the same parts as those in the first to third embodiments are denoted by the reference numerals given in FIGS.

This embodiment is different from the third embodiment in that the pixel area for acquiring the image quality degradation detection data is changed in conjunction with the electronic zoom area at the time of moving image shooting by the electronic zoom.
FIG. 12 is a diagram illustrating an example of an electronic zoom region at the time of moving image shooting (FIG. 12A) and a pixel region (FIG. 12B) from which image quality degradation detection data is acquired.
As shown in FIG. 12A, pixel regions (pixel regions R7 to R10, R12 to R15, R17 to R20, R22 to R25) indicated by hatching in FIG. The image data at the time of moving image shooting is read using only the electronic zoom area. At this time, the pixel area of the image quality deterioration detection data acquired within the frame rate adjustment period is changed in conjunction with the electronic zoom area. That is, as shown in FIG. 12B, the image quality is deteriorated only in portions corresponding to the pixel regions (pixel regions R7 to R10, R12 to R15, R17 to R20, and R22 to R25) filled in gray in FIG. Get detection data. That is, in the first frame, image quality deterioration detection data for the pixel region R7 is acquired, and in the next frame, image quality deterioration detection data for the pixel region R8 is acquired. Then, the image quality deterioration detection data is sequentially acquired from all of the portions corresponding to the pixel areas painted in gray in FIG. As a result, the image quality deterioration of the pixel area corresponding to the image data used at the time of moving image shooting can be reduced in a smaller frame period as compared with the case where the image quality deterioration detection data of all the pixel areas of the image sensor 114 is acquired. It becomes possible to detect. Therefore, it is possible to accurately detect deterioration in image quality in a shorter time.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

(Other examples)
The present invention is also realized by executing the following processing. That is, first, software (computer program) for realizing the functions of the above embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the computer program.

114 imaging device, 120 image processing circuit, 128 image display unit

Claims (27)

A pixel having a photoelectric conversion element, a plurality of pixels arranged two-dimensionally,
An output means for reading out the electric charge photoelectrically converted by the photoelectric conversion element and outputting a moving image or a live view image based on the read electric charge;
An acquisition means for reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels, and acquiring data for image quality degradation detection based on the read charge;
Detecting means for detecting deterioration of image quality using the image quality deterioration detection data acquired by the acquisition means;
The output unit reads the charge from a pixel different from the pixel from which the charge is read by the acquisition unit among the plurality of pixels.
Said acquisition means, in one frame period of a moving image or live view image output by said output means, a period in which performs readout of charge for obtaining the moving image or the live view image, the period is not performed in both periods, a different charge from the photoelectrically converted charges in the photoelectric conversion elements, among the plurality of pixels, and read out from the pixels are shielded from light,
The image pickup apparatus , wherein the detection unit detects deterioration in image quality of a still image output after the moving image or the live view image is output . The acquisition unit is configured to output charges different from the charges photoelectrically converted by the photoelectric conversion element when the imaging device is driven under a condition different from that when the output unit reads the charges. The image pickup apparatus according to claim 1, wherein reading is performed from at least a part of the image pickup apparatus. A pixel having a photoelectric conversion element, a plurality of pixels arranged two-dimensionally,
An output means for reading out the electric charge photoelectrically converted by the photoelectric conversion element and outputting a moving image or a live view image based on the read electric charge;
An acquisition means for reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels, and acquiring data for image quality degradation detection based on the read charge;
Detecting means for detecting deterioration of image quality using the image quality deterioration detection data acquired by the acquisition means;
The acquisition unit is a period during which at least one of the frame periods of the moving image or live view image output by the output unit is not reading out charges for obtaining the moving image or live view image, A charge different from the charge photoelectrically converted by the photoelectric conversion element is read from at least a part of the plurality of pixels during a period in which the imaging device is driven under a condition different from that when the output means reads the charge. An imaging apparatus characterized by that. The acquisition means is configured to acquire charges different from charges photoelectrically converted by the photoelectric conversion element during a period in which charge is not read for obtaining the moving image or live view image within one frame period. Read out from the light-shielded pixels,
The imaging apparatus according to claim 3 , wherein the detection unit detects deterioration in image quality of a still image output after the moving image or the live view image is output. The condition different from that when the charge is read by the output means is that the driving current of the imaging device is larger than when the charge is read by the output means, or the charge is read by the output means. The imaging apparatus according to any one of claims 2 to 4, wherein the ISO sensitivity is higher than that of the time. A pixel having a photoelectric conversion element, a plurality of pixels arranged two-dimensionally,
Output means for reading out charges including charges photoelectrically converted by the photoelectric conversion element, and outputting a moving image based on the read charges;
An acquisition means for reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels, and acquiring data for image quality degradation detection based on the read charge;
Detecting means for detecting deterioration of image quality using the image quality deterioration detection data acquired by the acquisition means;
The acquisition means is a charge photoelectrically converted by the photoelectric conversion element during at least a period during which a charge for obtaining the moving image is not read out of one frame period of the moving image output by the output means. The charge different from the above is performed by designating a part of the plurality of pixels to be read out with different designated pixels in each of the plurality of frame periods,
The detection means detects deterioration in image quality of the moving image,
If there is an instruction from a user while outputting the moving image, the output means outputs the moving image based on charges read from some of the plurality of pixels corresponding to the instruction. Magnify and output
The acquisition device specifies a part of pixels corresponding to the instruction among the plurality of pixels. The imaging apparatus according to claim 6 , wherein the acquisition unit specifies different pixels in each frame, and specifies all of the plurality of pixels over a period of a plurality of frames . The imaging apparatus according to claim 6 , wherein the acquisition unit specifies a part of pixels that are set in advance as pixels for detecting deterioration in image quality among the plurality of pixels. 2. The output unit according to claim 1, wherein when the deterioration of image quality is detected by the detection unit, the output unit outputs information indicating the fact to the screen on which the moving image or the live view image is displayed. The imaging device according to any one of to 5 . 9. The output device according to claim 6, wherein when the detection unit detects a deterioration in image quality, the output unit outputs information indicating the fact to a screen on which the moving image is displayed. The imaging apparatus of Claim 1. It said output means by said detecting means, the deterioration of image quality is detected, according to claim 1, characterized in that to terminate the output of the moving image or live view image, 2, 3, 4, 5 or 9, The imaging device described in 1. 11. The imaging apparatus according to claim 6, wherein the output unit ends the output of the moving image when image quality deterioration is detected by the detection unit. A pixel having a photoelectric conversion element, a method for controlling an imaging apparatus having a plurality of pixels arranged two-dimensionally,
An output step of reading out the charge photoelectrically converted by the photoelectric conversion element, and outputting a moving image or a live view image based on the read charge;
An acquisition step of reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels and acquiring data for image quality degradation detection based on the read charge;
Using the image quality degradation detection data acquired by the acquisition step, and detecting the degradation of the image quality,
The output step reads out charge from a pixel different from the pixel from which the charge is read out in the obtaining step among the plurality of pixels.
The acquisition step, in one frame period of a moving image or live view image output by said output step, a period in which performs readout of charge for obtaining the moving image or the live view image, the period is not performed in both periods, a different charge from the photoelectrically converted charges in the photoelectric conversion elements, among the plurality of pixels, and read out from the pixels are shielded from light,
The method for controlling an imaging apparatus, wherein the detecting step detects deterioration in image quality of a still image output after the output of the moving image or live view image . In the acquisition step, when the imaging device is driven under a condition different from that when the charge is read out in the output step, a charge different from the charge photoelectrically converted by the photoelectric conversion element is transferred to the plurality of pixels. method for controlling an image sensing apparatus according to claim 1 3, characterized in that read from at least a portion of. A pixel having a photoelectric conversion element, a method for controlling an imaging apparatus having a plurality of pixels arranged two-dimensionally,
An output step of reading out the charge photoelectrically converted by the photoelectric conversion element, and outputting a moving image or a live view image based on the read charge;
An acquisition step of reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels and acquiring data for image quality degradation detection based on the read charge;
Using the image quality degradation detection data acquired by the acquisition step, and detecting the degradation of the image quality,
The acquisition step is a period in which at least one charge period for obtaining the moving image or live view image is not read out of one frame period of the moving image or live view image output by the output step, A charge different from the charge photoelectrically converted by the photoelectric conversion element is read from at least a part of the plurality of pixels during a period in which the imaging device is driven under a condition different from that when the charge is read by the output process. And a method of controlling the imaging apparatus. In the acquisition step, a charge different from the charge photoelectrically converted by the photoelectric conversion element in a period in which the charge for obtaining the moving image or the live view image is not read within one frame period, Read out from the light-shielded pixels,
The method according to claim 15, wherein the detecting step detects deterioration in image quality of a still image output after the moving image or the live view image is output. The condition different from when the charge is read by the output process is that the driving current of the imaging device is larger than when the charge is read by the output process, or the charge is read by the output process. The method for controlling an image pickup apparatus according to any one of claims 14 to 16, wherein the ISO sensitivity is higher than that of the time. A pixel having a photoelectric conversion element, a method for controlling an imaging apparatus having a plurality of pixels arranged two-dimensionally,
An output step of reading out charges including charges photoelectrically converted by the photoelectric conversion element and outputting a moving image based on the read charges;
An acquisition step of reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels and acquiring data for image quality degradation detection based on the read charge;
Using the image quality degradation detection data acquired by the acquisition step, and detecting the degradation of the image quality,
The acquisition step includes a charge photoelectrically converted by the photoelectric conversion element during at least a period during which the charge for obtaining the moving image is not read out of one frame period of the moving image output by the output step. The charge different from the above is performed by designating a part of the plurality of pixels to be read out with different designated pixels in each of the plurality of frame periods,
The detection step detects deterioration of image quality of the moving image,
In the output step, when there is an instruction from a user during the output of the moving image, the moving image is based on charges read from some of the plurality of pixels corresponding to the instruction. Magnify and output
The acquisition step specifies a part of pixels corresponding to the instruction from among the plurality of pixels. The method of controlling an imaging apparatus according to claim 18 , wherein in the acquisition step , different pixels are specified in each frame, and all of the plurality of pixels are specified over a period of a plurality of frames . The method for controlling an imaging apparatus according to claim 18 , wherein in the obtaining step, a part of pixels set in advance as pixels for detecting deterioration in image quality is specified from the plurality of pixels. 2. The output step of outputting information indicating the fact to the screen on which the moving image or the live view image is displayed when image quality deterioration is detected by the detection step. The control method of the imaging device according to any one of 3 to 17 . 21. The information output device according to any one of claims 18 to 20, wherein the output step outputs information indicating that to the screen on which the moving image is displayed when image quality deterioration is detected by the detection step. A control method for an imaging apparatus according to claim 1. The output step ends the output of the moving image or the live view image when deterioration in image quality is detected by the detection step. 13, 14, 15, 16, 17, or The control method of the imaging device of 21 . 23. The method according to claim 18, wherein the output step ends the output of the moving image when image quality deterioration is detected by the detection step. . A computer program for causing a computer to control an imaging device having a plurality of pixels arranged in two dimensions, the pixel having a photoelectric conversion element,
An output step of reading out the charge photoelectrically converted by the photoelectric conversion element, and outputting a moving image or a live view image based on the read charge;
An acquisition step of reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels and acquiring data for image quality degradation detection based on the read charge;
Using the image quality deterioration detection data acquired by the acquisition step, the detection step of detecting image quality deterioration is executed by a computer,
The output step reads out charge from a pixel different from the pixel from which the charge is read out in the obtaining step among the plurality of pixels.
The acquisition step, in one frame period of a moving image or live view image output by said output step, a period in which performs readout of charge for obtaining the moving image or the live view image, the period is not performed in both periods, a different charge from the photoelectrically converted charges in the photoelectric conversion elements, among the plurality of pixels, and read out from the pixels are shielded from light,
The computer program according to claim 1, wherein the detecting step detects deterioration in image quality of a still image output after the output of the moving image or the live view image . A computer program for causing a computer to control an imaging device having a plurality of pixels arranged in two dimensions, the pixel having a photoelectric conversion element,
An output step of reading out the charge photoelectrically converted by the photoelectric conversion element, and outputting a moving image or a live view image based on the read charge;
An acquisition step of reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels and acquiring data for image quality degradation detection based on the read charge;
Using the image quality deterioration detection data acquired by the acquisition step, the detection step of detecting image quality deterioration is executed by a computer,
The acquisition step is a period in which at least one charge period for obtaining the moving image or live view image is not read out of one frame period of the moving image or live view image output by the output step, A charge different from the charge photoelectrically converted by the photoelectric conversion element is read from at least a part of the plurality of pixels during a period in which the imaging device is driven under a condition different from that when the charge is read by the output process. A computer program characterized by the above. A computer program for causing a computer to control an imaging device having a plurality of pixels arranged in two dimensions, the pixel having a photoelectric conversion element,
An output step of reading out charges including charges photoelectrically converted by the photoelectric conversion element and outputting a moving image based on the read charges;
An acquisition step of reading out charge different from the charge photoelectrically converted by the photoelectric conversion element from at least a part of the plurality of pixels and acquiring data for image quality degradation detection based on the read charge;
Using the image quality deterioration detection data acquired by the acquisition step, the detection step of detecting image quality deterioration is executed by a computer,
The acquisition step includes a charge photoelectrically converted by the photoelectric conversion element during at least a period during which the charge for obtaining the moving image is not read out of one frame period of the moving image output by the output step. The charge different from the above is performed by designating a part of the plurality of pixels to be read out with different designated pixels in each of the plurality of frame periods,
The detection step detects deterioration of image quality of the moving image,
In the output step, when there is an instruction from a user during the output of the moving image, the moving image is based on charges read from some of the plurality of pixels corresponding to the instruction. Magnify and output
The acquisition step specifies a part of pixels corresponding to the instruction among the plurality of pixels.

Images