JP2010093630A - Imaging apparatus and method of controlling imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of obtaining a still image having continuity in an exposure period without influencing photography of a moving image when photographing the still image during the photography of the moving image. <P>SOLUTION: The image apparatus 11 which has an imaging element 1 for photoelectrically converting subject light and accumulating electric charges and can photograph a still image while photographing a moving image includes a driving means 3 of driving the imaging element 1 with individually divided in an exposure period for moving image and an exposure period for auxiliary still image when picking up the subject light, a moving image processing means 5 of generating the moving image on the basis of electric charges for moving image photoelectrically converted through exposure to the imaging element 1 in the exposure period for moving image, and a still image processing means 6 of generating the still image on the basis of auxiliary still image electric charges photoelectrically converted through exposure to the imaging element 1 in the exposure period for auxiliary still image, and of the moving image electric charges. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of capturing a still image while capturing a moving image, and a method for controlling the imaging apparatus.

  2. Description of the Related Art Conventionally, imaging devices that can capture both moving images and still images are known. Such an imaging apparatus has a still image shooting mode and a moving image shooting mode, and these operation modes are switched by a user operation.

In recent years, there has also been proposed an imaging apparatus capable of shooting a still image while shooting a moving image. When shooting a still image while shooting a moving image, a plurality of frames of the moving image are extracted and combined. Thus, still images are generated (Patent Documents 1 and 2).
JP 2005-311665 A JP 2007-274210 A

  By the way, even when shooting a still image during shooting of a moving image, it is desirable to obtain images having continuity in the exposure period from the image sensor irradiated with subject light in all the exposure periods specified by the user.

  On the other hand, in moving image shooting, it is desirable to generate a moving image with a frame generated with an appropriate exposure time according to the subject, regardless of still image shooting during moving image shooting.

  However, conventionally, when shooting a still image while shooting a moving image, a plurality of frames of the moving image are extracted and subjected to a combining process. Therefore, the exposure process is a frame combining process obtained for moving image shooting. End up. For this reason, it is not possible to obtain a still image having a continuity during the exposure period, and this problem is particularly noticeable during long-second exposure during moving image shooting.

  For example, when trying to shoot a still image of a fast-moving subject, conventionally, a still image is generated by synthesizing a frame that is generated for a moving image and includes a non-exposed period. Is not continuous in the exposure period.

  Specifically, as shown in FIG. 14, when shooting with a long second exposure in a state where the ball is flying from the lower left to the upper right, in still image shooting during moving image shooting, as shown in FIG. Frames 1 to 3 are generated from a signal generated by moving image charges exposed during the exposure period. However, since the periods T100 and T101 different from the moving image exposure period are not exposed, the still image obtained from the synthesis process of the frames 1 to 3 has no continuity.

  In addition, in still image shooting during moving image shooting, if a moving image is shot with the exposure period adjusted to be long with priority given to the continuity of the exposure period of the still image, an appropriate moving image cannot be obtained. Arise.

  Therefore, the present invention provides an imaging apparatus and an imaging apparatus control method capable of obtaining a still image having continuity in an exposure period without affecting moving image shooting when performing still image shooting during moving image shooting. The purpose is to provide.

  In order to achieve the above object, an image pickup apparatus of the present invention is an image pickup apparatus that includes an image pickup element that photoelectrically converts subject light and accumulates charges, and is capable of shooting a still image while shooting a moving image. When capturing light, a driving unit that drives the image sensor in an exposure period for moving images and an exposure period for auxiliary still images, and photoelectric conversion that is exposed to the image sensor during the exposure period for moving images A moving image processing means for generating a moving image based on the moving image charge, and an auxiliary still image charge photoelectrically converted by exposure to the imaging element during the auxiliary still image exposure period and the moving image And still image processing means for generating a still image based on the charge.

  An image pickup apparatus control method according to the present invention is an image pickup apparatus control method that includes an image pickup device that photoelectrically converts subject light and accumulates charges, and that can take a still image during moving image shooting. When capturing, a driving step of driving the image sensor separately into a moving image exposure period and an auxiliary still image exposure period, and the image sensor is exposed to photoelectric conversion during the moving image exposure period. A moving image processing step for generating a moving image based on the moving image charge; the moving image charge; and a still image charge photoelectrically converted by exposure to the imaging device during an auxiliary still image exposure period. And a still image processing step of generating a still image based on the still image processing step.

  According to the present invention, when still image shooting is performed during moving image shooting, it is possible to obtain a still image having continuity in the exposure period without affecting moving image shooting.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram for explaining an imaging apparatus according to the first embodiment of the present invention.

  As shown in FIG. 1, the imaging device 11 of this embodiment includes a lens 2, an imaging device 1, a TG (timing generator) 3, an image processing unit 4, a memory 7, and a CPU 8 that controls each block.

  The image sensor 1 converts subject light that has passed through the lens 2 into an electrical signal by photoelectric conversion, and the TG 3 controls the drive timing of the electronic shutter of the pixels of the image sensor 1.

  The image processing unit 4 includes a moving image processing unit 5 and a still image processing unit 6. The moving image processing unit 5 generates frames for moving images and still images during moving image shooting from the moving image voltages output from the image sensor 1 via the moving image voltage output line 9, and the memory 7 and Output to the still image processing unit 6.

  The still image processing unit 6 generates an auxiliary still image frame from the auxiliary still image voltage output from the image sensor 1 via the auxiliary still image voltage output line 10. In the present embodiment, the moving image voltage output line 9 and the auxiliary still image voltage output line 10 are provided separately. The still image processing unit 6 adds and synthesizes the auxiliary still image frame and the frame input from the moving image processing unit 5 to generate a still image having continuity in the exposure period, and outputs the still image to the memory 7.

  Next, with reference to FIG. 2 and FIG. 3, an operation example for obtaining a still image having continuity in the exposure period during moving image shooting in the imaging apparatus of the present embodiment will be described.

  FIG. 2 is a flowchart for explaining an operation example of the imaging device 11 in still image shooting during moving image shooting. Each process in FIG. 2 is executed by the image sensor 14 and the CPU 8, and the CPU 8 performs a process according to a program loaded in the RAM and stored in the ROM or the like.

  First, in step S100, the CPU 8 proceeds to step S101 when the image sensor 1 receives subject light via the lens 2 in response to an instruction to start moving image shooting by the user.

  In step S101, the CPU 8 controls the exposure timing for moving images with the electronic shutter using TG3 to generate moving image charges, and proceeds to step S102.

  In step S102, the moving image charge generated in step S101 is converted into a voltage by the processing of the image sensor 14 and output to the moving image processing unit 5 via the moving image voltage output line 9, and the process proceeds to step S103.

  In step S103, the CPU 8 controls the moving image processing unit 5 to generate a moving image, outputs the generated moving image to the memory 7 and the still image processing unit 6, and proceeds to step S104.

  In step S104, the CPU 8 determines whether still image shooting has been started. If still image shooting has started, the process proceeds to step S105, and if still image shooting has not started, the process returns to step S102.

  In step S105, the CPU 8 controls the exposure timing of the moving image charge information and the auxiliary still image charge information with the electronic shutter using TG3 to generate the moving image charge and the auxiliary still image charge, and then in step S106. Proceed to

  In step S <b> 106, the CPU 8 controls the image sensor 1 to convert the voltage for the auxiliary still image voltage and outputs it to the still image processing unit 6 via the auxiliary still image voltage output line 10. Further, the CPU 8 controls the imaging device 1 to convert the voltage of the moving image voltage to the moving image processing unit 5 via the moving image voltage output line 9, and the process proceeds to step S107.

  In step S107, the CPU 8 adds and synthesizes the auxiliary still image frame generated by the still image processing unit 6 and the moving image frame output from the moving image processing unit 5 to the still image processing unit 6 to generate a still image. And output to the memory 7. Further, the CPU 8 outputs the moving image frame generated by the moving image processing unit 5 to the memory 7 as a moving image, and returns to step S104.

  FIG. 3 is a timing chart regarding generation of a moving image frame and an auxiliary still image frame in the image sensor 1 in still image shooting during moving image shooting.

  In FIG. 3, a period T1 is an exposure period of one frame, and moving image shooting is started at timing T40.

  At timing T41, moving image exposure is performed during a period T50, and the moving image charge photoelectrically converted by the photodiode 1000 (see FIG. 5) constituting the pixel 301 of the image sensor 1 is M floating diffusion 403-. 1 is accumulated.

  At timing T42, as indicated by the hatched lines, a moving image line output 412 for one frame is performed, and the moving image processing unit 5 generates a moving image frame (when no still image is captured) 406.

  At timing T43, when still image shooting during moving image shooting is started, auxiliary still image exposure is performed during the period T51, and the auxiliary still image charge photoelectrically converted by the photodiode 1000 is converted into the S floating diffusion 403. -2.

  At timing T44, as indicated by the horizontal line, auxiliary still image line output 413 is performed, and the still image processing unit 6 generates an auxiliary still image frame 408.

  At timing T45, when still image shooting during moving image shooting ends, generation of auxiliary still image frames ends.

  The moving image frame (for moving image shooting and still image composition) 407 obtained during the time period T52 is output to the still image processing unit 6 as a still image frame, and is added and synthesized with the auxiliary still image frame 408 to be continuous. A static still image is generated.

  The moving image frame 406 and the moving image frame 407 are generated as moving images by the moving image processing unit 5.

  As described above, in still image shooting during moving image shooting, it is possible to obtain a still image that is continuous during the exposure period and a moving image that is not affected by still image shooting.

  FIG. 4 is a diagram illustrating an outline of a configuration example of the image sensor 1.

  As shown in FIG. 4, the image sensor 1 has a plurality of pixels 301 arranged on a two-dimensional matrix. The vertical scanning line circuit 300 includes a voltage column readout line (row selector (see FIG. 5) control line) 302 for moving images and a column readout line (row selector 303 (see FIG. 5)) for auxiliary still images. Control line) 303. The voltage output line 310 outputs a moving image voltage, and the voltage output line 311 outputs an auxiliary still image voltage.

  The CDS 312 removes noise from the moving image voltage, and the CDS 305 removes noise from the auxiliary still image voltage. The horizontal scanning line circuit 306 includes a moving image voltage output line 9 that outputs a moving image voltage to the moving image processing unit 5 and an auxiliary still image voltage output line 10 that outputs an auxiliary still image voltage to the still image processing unit 6. And select control.

  Next, the operation of the image sensor 1 will be described. When the column readout line 302 is selected by the control of the vertical scanning line circuit 300, the moving image charge in the selected column is converted into a voltage in the pixel 301. And output from the voltage output line 310. The moving image voltage output from the voltage output line 310 is noise-removed through the CDS 312 and output to the moving image processing unit 5 through the moving image voltage output line 9 under the control of the horizontal scanning line circuit 306.

  When the column readout line 303 is selected under the control of the vertical scanning line circuit 300, the auxiliary still image charge in the selected column is converted in voltage in the pixel 301 and then output from the voltage output line 311. The auxiliary still image voltage output from the voltage output line 311 is noise-removed via the CDS 305 and output to the still image processing unit 6 via the auxiliary still image voltage output line 10 under the control of the horizontal scanning line circuit 306. The

  FIG. 5 is a diagram illustrating a circuit configuration example of the pixel 301 of the image sensor 1 illustrated in FIG. 4.

  In FIG. 5, a photodiode 1000 converts optical information into electric charge and accumulates it. The M (moving image) reset gate 401-1 resets the moving image charge accumulated in the photodiode 1000. Exposure is performed when the M lead-out gate 402-1 is turned on, and the electric charge photoelectrically converted by the photodiode 1000 is read out to the M floating diffusion 403-1.

  The M source follower amplifier 404-1 converts the charge information stored in the M floating diffusion 403-1 into voltage information. The row selector 302 selects a row of moving image pixels that outputs the moving image voltage to the moving image processing circuit 5 via the moving image voltage output line 310.

  The S (auxiliary still image) reset gate 401-2 resets the auxiliary still image charge accumulated in the photodiode 1000. Exposure is performed by turning on the S lead-out gate 402-2, and the electric charge photoelectrically converted by the photodiode 1000 is read out to the S floating diffusion 403-2.

  The S source follower amplifier 404-2 converts the charge information stored in the S floating diffusion 403-2 into voltage information. The row selector S303 selects a column of auxiliary still image pixels that outputs the auxiliary still image voltage to the still image processing circuit 6 via the auxiliary still image image voltage output line 311.

  FIG. 6 is a timing chart showing the timing for generating moving image frames and auxiliary still image frames in still image shooting during moving image shooting by outputting moving image charges and auxiliary still image charges stored in the pixels 301 one frame at a time. It is.

  In FIG. 6, a period T1 is an exposure period (1 VD) of one frame. In this timing chart, moving image shooting has already started before the timing T2, and the moving image processing unit 5 has generated a moving image frame 406.

  First, output of a moving image frame when a still image is shot during moving image shooting will be described.

  At timing T2, the M reset gate 401-1 and the row selector 302 are turned on, and the charge accumulated in the M floating diffusion 403-1 is reset.

  At timing T3, the M reset gate 401-1 and the row selector 302 are turned off, the M lead-out gate 402-1 is turned on during the period T13 that is the moving image exposure period, and the M floating diffusion 403-1 is hatched. Charge is accumulated as shown.

  At timing T4, the M reset gate 401-1 and the row selector 302 are turned on, and the electric charge accumulated in the M floating diffusion 403-1 is converted into voltage information by the M source follower amplifier 404-1.

  During the period T15, the voltages of all the pixels constituting one frame of the moving image frame are output to the moving image processing unit 5 for each horizontal line when the row selector 302 is turned on.

  Then, the moving image processing unit 5 generates a moving image frame 407, and the moving image shooting frame is output to the still image processing unit 6 and the memory 7.

  Next, output of an auxiliary still image frame when still image shooting is performed during moving image shooting will be described.

  When the still image shooting period 410 during moving image shooting starts at timing T7, the S reset gate 401-2 and the row selector S303 are turned on at timing T8. As a result, the charges accumulated in the S floating diffusion 403-2 are reset.

  At timing T9, the S reset gate 401-2 and the row selector S303 are turned off, and the S lead-out gate 402-2 is turned on during the period T14, which is the auxiliary still image exposure period. As a result, charges are accumulated in the S floating diffusion 403-2 as indicated by the horizontal line.

  At timing T10, the S reset gate 401-2 and the row selector 303 are turned on, and the charge information stored in the S floating diffusion 403-2 is converted into voltage information by the S source follower amplifier 404-2.

In the period T16, the voltages of all the pixels constituting one frame of the auxiliary still image frame are output to the still image processing unit 6 for each horizontal line when the row selector 303 is turned on.
An auxiliary still image frame 408 is generated by image processing of the voltage output to the still image processing unit 6.

  The generation of the auxiliary still image frame is performed until the end timing T12 of the still image shooting during moving image shooting, and all the charges for the auxiliary still image are output to the still image processing unit 6, and the auxiliary still image frame is generated and ended. To do.

  After the above procedure, a still image during moving image shooting is generated by adding and combining the moving image frame 407 and the auxiliary still image frame 408 output from the moving image processing unit 5 during still image shooting in the still image processing unit 6. Therefore, it is possible to generate a still image having continuity during the exposure period.

  Note that in FIG. 6, since the transistor is turned on for a long period, it seems that there is no continuity in the exposure period of the still image frame in still image shooting during moving image shooting. However, in practice, the exposure period T1 of one frame is 1/30 second and 1/60 second, whereas the ON time of the transistor is sufficiently short, so that the exposure period of the still image frame is approximately continuous. You may think that there is sex.

  In addition, the moving image shooting can be output without being affected by the still image shooting since the moving image processing unit 5 performs image processing separately from the still image frame.

  In FIG. 6, in a period for reading out one frame of a moving image and an auxiliary still image, an actual CMOS sensor reads out each pixel constituting one horizontal line. For this reason, the ON / OFF timing of the S reset gate 401-2 and the row selector S303 should be described for each horizontal line.

  However, in FIG. 6, since one frame is described at the same timing, it is omitted including the timing difference between the ON and OFF timings of the S reset gate 401-2 and the row selector S303 when reading every horizontal line. At the same time, it is turned on and off.

(Second Embodiment)
Next, with reference to FIG. 7, an imaging apparatus according to a second embodiment of the present invention will be described. Note that the description of the same parts as those in the first embodiment is omitted.

  In the present embodiment, a case where the exposure period for moving images on the image sensor 1 is shortened by the amount of light of the subject is taken as an example.

  In FIG. 7, the period required to output the voltage for one frame from the image sensor 1 to the image processing unit 4 is 1 VD / 2.

  The period T1 is an exposure period of one frame in the image sensor 1, and the exposure for the moving image for the period T34 is performed at the timing T30.

  At the timing T31, during the period T36, the moving image voltage constituting one frame of the moving image is output to the moving image processing unit 5 through the moving image voltage output line 9 as indicated by hatching for each horizontal line. The At timing T31, exposure for auxiliary still images is performed for the period T35.

  During the period from the timing T37 to the period T38, the still image processing unit 6 is configured such that the auxiliary still image voltage constituting one frame of the auxiliary still image is indicated by a horizontal line for each horizontal line via the auxiliary still image voltage output line 10. Is output.

  As described above, in the present embodiment, since each of the output units for the moving image voltage and the auxiliary still image voltage can be output in parallel from the image sensor 1, one frame is obtained at 1 VD. Each moving image frame and auxiliary still image frame can be obtained. Other configurations and operational effects are the same as those of the first embodiment.

(Third embodiment)
Next, an imaging apparatus according to a third embodiment of the present invention will be described with reference to FIGS. In addition, about the part which overlaps with the said 1st Embodiment, the same code | symbol is attached | subjected to each figure and the description is abbreviate | omitted.

  FIG. 8 is a block diagram for explaining an image pickup apparatus according to a third embodiment of the present invention, FIG. 9 is a schematic diagram showing a configuration example of the image sensor, and FIG. 10 is a circuit configuration diagram of pixels of the image sensor.

  As shown in FIG. 8, the imaging device 13 of this embodiment includes an imaging element 14. As shown in FIGS. 8 and 9, the imaging device 14 outputs a single output of a moving image voltage to the moving image processing unit 5 and an auxiliary still image voltage to the moving image processing unit 5. This is done in a time division manner via the line 12.

  As shown in FIG. 10, the pixel 309 of the image sensor 14 includes an M floating diffusion (first holding means) 403-1 that holds a moving image charge and an auxiliary still image charge, and an S floating diffusion ( Second holding means) 403-2.

  In addition, in the pixel 309, the moving image voltage output line and the auxiliary still image voltage output line are connected by a wired OR, and as shown in FIGS. 9 and 10, the moving image voltage output line and the auxiliary still image output line are connected. The output line for the image voltage is shared to form one voltage output line 313.

  Next, with reference to FIG. 11, an operation timing at which the image sensor 14 outputs the moving image voltage and the auxiliary still image voltage in a time division manner will be described.

  In this embodiment, a case where the exposure period for moving images on the image sensor 14 is shortened by the amount of light of the subject is taken as an example. Note that the period required for outputting the voltage for one frame from the image sensor 14 to the image processing unit 4 is 1 VD / 2.

  In FIG. 11, a period T1 is an exposure period (1VD) of one frame in the image sensor 14, and exposure for moving images is performed for a time period T24 at timing T20.

  From timing T21 to period T26, the moving image voltage constituting one frame of the moving image is output to the moving image processing unit 5 via the output line 12 for each horizontal line as shown by the hatched lines.

  Since the output period of the moving image voltage to the moving image processing unit 5, that is, 1VD / 2 is shorter than the exposure period T25 for the auxiliary still image, the moving image processing unit of all voltages constituting one frame of the moving image. The output for every horizontal line to 6 is completed.

  The auxiliary still image voltage starts to be output to the still image processing unit 6 via the output line 12 at the timing T20 as indicated by a horizontal line for each horizontal line, but at the timing T21, the exposure for the auxiliary still image starts. . That is, the period T24 is shorter than 1VD / 2. For this reason, all the voltages constituting one frame of the auxiliary still image cannot be output to the still image processing unit 6.

  Therefore, under the control of the vertical scanning line circuit 300, the output of the auxiliary still image voltage is interrupted for the period T26, and the above-described moving image voltage is output for each horizontal line.

  Then, after the output of the moving image voltage to the moving image processing unit 5 is completed at timing T22, the output of the remaining auxiliary still image voltage is started as indicated by a horizontal line for each horizontal line, and at timing T23, 1 is output. The output of the auxiliary still image voltage for the frame is completed.

  As described above, in the present embodiment, the output of the moving image voltage and the output of the auxiliary still image voltage are performed in a time-sharing manner. Obtainable. Other configurations and operational effects are the same as those of the first embodiment.

(Fourth embodiment)
Next, with reference to FIG. 12, an imaging apparatus according to a fourth embodiment of the present invention will be described. Note that the description of the same parts as those in the first embodiment is omitted.

  In FIG. 12, when still image shooting is completed during the exposure period T13 for moving images, in order to obtain an appropriate still image, the charge that has been exposed and photoelectrically converted in the period from timing T15 to T17 is converted. An auxiliary still image frame is generated based on the obtained voltage. Then, a still image is generated using the generated auxiliary still image frame. Here, timing T15 is the start timing of the exposure period T13 for moving images. Timing T16 is the end timing of the still image shooting period 410. T17 is the timing for reading the final auxiliary still image by nondestructive reading.

  Regarding the reading of the electric charges exposed to the M floating diffusion 403-2, the non-destructive reading 500 is performed by turning on only the row selector 302 and not turning on the M reset gate in the period of T300. As a result, an appropriate auxiliary still image frame can be obtained without erasing the charge originally intended for moving images.

  A still image during moving image shooting is generated by adding and combining the moving image frame 407 and the auxiliary still image frame 408. Other configurations and operational effects are the same as those of the first embodiment.

(Fifth embodiment)
The outline of the fifth embodiment is that a moving image charge and a charge accumulating portion (holding portion) for accumulating still image charges are common when performing still image shooting during moving image shooting. And the output line of the still image signal drive an image sensor having a common pixel. The readout of the moving image signal is performed in the non-destructive readout, and the readout of the still image is performed in the destructive readout, so that it is possible to obtain a still image having continuity during moving image capturing.

  With reference to FIGS. 8 and 16 to 19, an imaging apparatus according to a fifth embodiment of the present invention will be described. In addition, about the part which overlaps with the said 1st, 3rd embodiment, the same code | symbol is attached | subjected to each figure and the description is abbreviate | omitted.

  Note that the description of the same parts as those in the first and third embodiments is omitted.

  FIG. 8 is a block diagram for explaining an image pickup apparatus according to the fifth embodiment of the present invention.

  FIG. 16 is a flowchart for explaining an operation example of the imaging apparatus 3 in still image shooting during moving image shooting.

  Each process in FIG. 16 is executed by the image sensor 14 and the CPU 8 that executes the program loaded in the RAM and stored in the ROM or the like.

  First, in step S200, the CPU 8 proceeds to step S201 when the image sensor 14 receives subject light via the lens 2 in response to an instruction to start moving image shooting by the user.

  In step S201, the CPU 8 uses TG3 to control the exposure timing for moving images with the electronic shutter to generate moving image charges, and proceeds to step S202.

  In step S202, the moving image charge generated in step S201 is converted into a voltage by the processing of the image sensor 14 and output to the moving image processing unit 5 via the voltage output line 10 for moving image and still image. The process proceeds to S203.

  In step S203, the CPU 8 controls the moving image processing unit 5 to generate a moving image from the moving image signal output from the imaging device 14, and outputs the generated moving image to the memory 7, and the process proceeds to step S204. move on.

  In step S204, the CPU 8 determines whether or not still image shooting has started. If still image shooting has started, the process proceeds to step S205, and if still image shooting has not started, the process returns to step S202.

  In step S205, the CPU 8 controls the exposure timing of the moving image charge information and the still image charge information with the electronic shutter using TG3 to generate the moving image charge and the still image charge, and then proceeds to step S206. .

  In step S <b> 206, the CPU 8 controls the image sensor 14 to convert the still image signal into a voltage and outputs it to the still image processing unit 6 via the still image voltage output line 10.

  Further, the CPU 8 controls the image pickup device 1 to convert the voltage of the moving image voltage to the moving image processing unit 5 via the voltage output line 10 for moving image and still image, and proceeds to step S207. .

  In step S207, the CPU 8 generates a plurality of still image frames from the still image signal output from the image sensor 14 generated by the still image processing unit 6, and adds and synthesizes the still images with continuity. Generate and output to the memory 7. In addition, the CPU 8 controls the moving image processing unit 5 to generate a moving image from the moving image signal output from the imaging element 14, outputs the generated moving image to the memory 7, and returns to step S204.

  FIG. 17 is a diagram illustrating an outline of a configuration example of the image sensor 16.

  First, the configuration of FIG. 17 will be described.

  In the image sensor 16, a plurality of pixels 314 are arranged on a two-dimensional matrix.

  The row selector 304 is a common column read line for moving images and still images (control line of the row selector 304 (see FIG. 18)).

  The vertical scanning line circuit 300 controls the control line 304 when performing column readout of a moving image voltage or a still image voltage.

  The voltage output line 313 outputs a moving image voltage and a still image voltage from each pixel.

  The CDS 315 removes noise from the voltage output from the voltage output line 313.

  The vertical scanning line circuit 306 is an output voltage selection line for a moving image voltage and a still image horizontal line.

  Next, the operation of the image sensor 16 will be described.

  When the column readout line 304 is selected by the control of the vertical scanning line circuit 300, the electric charge exposed in the pixel 314 of the selected column is converted into a voltage for moving image or a voltage for still image, and then voltage output. Output from line 313.

  The moving image or still image voltage output from the voltage output line 313 is denoised via the CDS 315 and output to the image processing unit 4 via the voltage output line 12 under the control of the horizontal scanning line circuit 306. .

  FIG. 18 is a diagram illustrating a circuit configuration example of the pixel 314 of the image sensor 16 illustrated in FIG. 17.

In FIG. 18, a photodiode 1000 converts optical information into electric charge and accumulates it.
The MS reset gate 401-3 resets the electric charge accumulated in the photodiode 1000.

  Exposure is performed by turning on the MS lead-out gate 402-3, and the electric charge photoelectrically converted by the photodiode 1000 is read out to the M floating diffusion 403-3.

  The MS source follower amplifier 404-3 converts the charge information exposed to the MS floating diffusion 403-3 into voltage information.

  The row selector 304 selects a row of voltage output lines for outputting the moving image voltage to the moving image processing circuit 5 and the still image processing circuit 6.

  When reading the exposed charge during the exposure period (1VD) of one frame, the row selector 304 is turned on and the MS reset gate 401-3 is not turned on, so that the MS floating diffusion 403-3 is exposed. Read the charge by non-destructive readout.

  When reading the signal exposed at the end of one frame, after reading the signal when the row selector 304 is turned on, the MS reset gate 401-3 is turned on to destruct and read the charge exposed to the MS floating diffusion 403-3. Read by.

  FIG. 19 is a timing chart showing the timing for generating moving image frames and still image frames in still image shooting during moving image shooting by outputting moving image charges and still image charges stored in the pixels 314 one frame at a time. .

  In FIG. 19, a period T1 is an exposure period (1 VD) of one frame.

  In FIG. 19, reference numeral 407 denotes a moving image frame output from the image sensor 16.

  Reference numeral 409 denotes a still image frame output from the image sensor 16.

  In this timing chart, both moving image shooting and still image shooting have already been performed.

  First, output of a moving image frame when a still image is shot during moving image shooting will be described.

  At timing T2, the MS reset gate 401-3 and the row selector 304 are turned on, and the charges accumulated in the photodiode 1000 and the MS floating diffusion 403-3 are reset.

  At timing T3, the MS reset gate 401-3 and the row selector 304 are turned off, and the MS lead-out gate 402-3 is turned on during the period T13 which is the exposure period for moving images. As a result, charges indicated by horizontal lines are accumulated in the MS floating diffusion 403-3.

  At timing T200, the row selector 304 is turned on, and the charge information stored in the MS floating diffusion 403-3 is converted into voltage information by the MS source follower amplifier 404-3.

  At this time, the charge accumulated in the MS floating diffusion 403-3 is read by nondestructive reading.

  During the period T15, the voltages of all the pixels constituting one frame of the moving image frame are line output 414 to the moving image processing unit 5 for each horizontal line when the row selector 304 is turned on. An image frame 407 is generated.

  Next, output of a still image frame will be described.

  At timing T202, the row selector 304 is turned off, and the MS lead-out gate 402-3 is turned on during the period T14 that is an exposure period for still images. As a result, charges indicated by vertical lines are accumulated in the floating diffusion 403-3 in the MS floating diffusion 403-3.

  At timing T201, the row selector 304 and the reset gate 401-3 are turned on. In addition, the charge exposed to the portion indicated by the horizontal and vertical lines of the MS floating diffusion 403-3 is converted into voltage information by the MS source follower amplifier 404-3 at timing T202.

  During the period T16, the voltages of all the pixels constituting one frame of the still image frame are line-output 414 to the still image processing unit 6 every horizontal line when the row selector 304 is turned on, and the still image processing unit 6 A still image frame 409 is generated.

  After the above procedure, a continuous still image obtained by still image shooting during moving image shooting is generated by adding and combining the still image frame 409 with the still image processing unit 6.

(Sixth embodiment)
The outline of the sixth embodiment is that when performing still image shooting during moving image shooting, the charge storage unit for storing the charge for moving image and the charge for still image is common, the signal for moving image, An image sensor having a common pixel is driven by an output line of a still image signal. With regard to reading of the moving image signal, it is possible to obtain a still image having continuity during moving image shooting by performing destructive reading in both reading of the still image.

  Next, an image pickup apparatus according to the sixth embodiment of the present invention will be described with reference to timing charts shown in FIG. 8 and FIG. In addition, about the part which overlaps with the said 1st, 3 and 5 embodiment, the same code | symbol is attached | subjected to each figure and the description is abbreviate | omitted.

  In addition, the description which overlaps with the said 1st, 3 and 5 embodiment is abbreviate | omitted.

  In the sixth embodiment, the image sensor 314 of FIG. 18 is used for both reading out an electric charge exposed in the middle of an exposure period (1VD) of one frame and reading out an exposed signal at the end of one frame. Then, a signal when the row selector 304 is turned on is read out. Thereafter, by turning on the MS reset gate 401-3, the charge exposed to the MS floating diffusion 403-3 is read by destructive readout.

  FIG. 20 is a timing chart showing the timing of generating moving image frames and auxiliary still image frames in still image shooting during moving image shooting by outputting moving image charges and auxiliary still image charges stored in the pixels 314 one frame at a time. It is.

  In FIG. 20, a period T1 is an exposure period (1 VD) of one frame. In this timing chart, both moving image shooting and still image shooting have already been performed.

  First, output of a moving image frame when a still image is shot during moving image shooting will be described.

  At timing T2, the MS reset gate 401-3 and the row selector 304 are turned on, and the charge accumulated in the MS floating diffusion 403-3 is reset.

  At timing T3, the MS reset gate 401-3 and the row selector 304 are turned off, and the MS lead-out gate 402-3 is turned on during the period T13 which is the exposure period for moving images. As a result, charges are accumulated in the MS floating diffusion 403-1 as indicated by the horizontal line.

  At timing T200, the MS reset gate 401-3 and the row selector 304 are turned on, and the charge exposed to the MS floating diffusion 403-3 indicated by the horizontal line is converted into voltage information by the MS source follower amplifier 404-3. . At this time, the charge accumulated in the MS floating diffusion 403-3 is read by the destructive read 501.

  During the period T15, the voltages of all the pixels constituting one frame of the moving image frame are line output 414 to the moving image processing unit 5 for each horizontal line when the row selector 302 is turned on. An image frame 407 is generated.

  Next, the output of the auxiliary still image frame will be described.

  At timing T202, the row selector 304 is turned off, and the MS lead-out gate 402-3 is turned on during the period T14 that is an exposure period for still images. As a result, charges indicated by vertical lines are accumulated in the floating diffusion 403-3 in the MS floating diffusion 403-3.

  At timing T201, the row selector 304 is turned on, and the charge exposed to the portion indicated by the vertical line portion of the MS floating diffusion 403-3 is converted into voltage information by the MS source follower amplifier 404-3.

  During the period T16, the voltages of all the pixels constituting one frame of the auxiliary still image frame are line-output 414 to the still image processing unit 6 for each horizontal line when the row selector 304 is turned on. Thus, a still image frame 408 is generated.

  After the above procedure, a still image during moving image shooting is generated by adding and combining the moving image frame 407 and the auxiliary still image frame 408 output from the moving image processing unit 5 during still image shooting in the still image processing unit 6. Therefore, it is possible to generate a still image having continuity during the exposure period.

(Seventh embodiment)
Next, with reference to FIG. 13, an imaging apparatus according to a seventh embodiment of the present invention will be described. Note that the description of the same parts as those in the first embodiment is omitted.

  In the present embodiment, the light emission timing of a strobe (not shown) during still image shooting during moving image shooting will be described.

  First, the timing of strobe light emission 411 by front curtain synchronization during still image shooting during moving image shooting will be described.

  In FIG. 13, a period T1 is an exposure period of one frame. At the time of shooting a still image during moving image shooting by a user operation, a still image shooting period 410 starts at timing T20. At the leading curtain synchronization timing T21, the flash emission 411 is originally performed during the period T22.

  When the timing of the strobe light emission 411 overlaps with the period T27 of the moving image exposure period 405-1, TG3 at the timing T23 (exposure still image period T29 at the beginning of 405-1 in the still image shooting during moving image shooting). To control the timing of the strobe light emission 411.

  Next, the timing of the strobe light emission 411 by the trailing curtain sync during still image shooting during moving image shooting will be described.

  When shooting a still image during moving image shooting by a user operation, the still image shooting period 410 ends at timing T24. In the period T22 at the timing T25 of the second curtain synchronization, the strobe light emission 411 is originally performed.

  When the timing of the strobe light emission 411 overlaps the period T28 of the moving image exposure period 405-1, TG3 at the timing T26 (the last auxiliary still image exposure period T30 of 405-2 in still image shooting during moving image shooting). To control the timing of the strobe light emission 411.

  By controlling the strobe light emission 411 at the timing described above, it is possible to generate a moving image in which light is not reflected. Other configurations and operational effects are the same as those of the first embodiment.

  In addition, this invention is not limited to what was illustrated by said each embodiment, In the range which does not deviate from the summary of this invention, it can change suitably.

It is a block diagram for demonstrating the imaging device which is the 1st Embodiment of this invention. FIG. 10 is a flowchart for explaining an operation example of the imaging apparatus in still image shooting during moving image shooting. FIG. 6 is a timing chart regarding generation of a moving image frame and an auxiliary still image frame on an image sensor in still image shooting during moving image shooting. It is a figure which shows the outline of the structural example of an image pick-up element. It is a figure which shows the circuit structural example of the pixel of the image pick-up element shown in FIG. FIG. 10 is a timing chart showing timings of outputting moving image charges and auxiliary still image charges accumulated in pixels one frame at a time, and generating moving image frames and auxiliary still image frames in still image shooting during moving image shooting. It is a timing chart for demonstrating the imaging device which is the 2nd Embodiment of this invention. It is a block diagram for demonstrating the imaging device which is the 3rd Embodiment of this invention. It is the schematic which shows the structural example of an image pick-up element. It is a circuit block diagram of the pixel of the image pick-up element shown in FIG. It is a timing chart for demonstrating the operation timing which outputs the voltage for moving images, and the voltage for auxiliary | assistant still images in a time division in an image pick-up element. It is a timing chart for demonstrating the imaging device which is the 4th Embodiment of this invention. It is a timing chart for demonstrating the imaging device which is the 7th Embodiment of this invention. It is a figure for demonstrating a prior art example, and is a figure which shows the case where the state which the ball is flying from the lower left to the upper right is image | photographed by long-second exposure. It is a figure for demonstrating the still image in imaging | photography at the time of long-second exposure during the video recording in a prior art example. It is a flowchart for demonstrating the operation example of the imaging device in the still image photography in the video recording which is the 5th Embodiment of this invention. It is a figure which shows the outline of the structural example of an image pick-up element. It is a circuit block diagram of the pixel of the image pick-up element shown in FIG. FIG. 6 is a timing chart showing timings of outputting moving image charges and still image charges accumulated in pixels one frame at a time, and generating moving image frames and auxiliary still image frames in still image shooting during moving image shooting. The moving image charge and the auxiliary still image charge accumulated in the pixel according to the sixth embodiment of the present invention are output frame by frame to generate a moving image frame and an auxiliary still image frame in still image shooting during moving image shooting. It is a timing chart figure which shows a timing.

Explanation of symbols

1, 14 Image sensor 2 Lens 3 TG (timing generator)
4 Image processing unit 5 Moving image processing unit 6 Still image processing unit 7 Memory 8 CPU
9 Voltage output line for moving image 10 Voltage output line for auxiliary still image 11, 13 Imaging device 12 Voltage output line for moving image and still image 300 Vertical scanning line circuits 301, 309, 314 Pixel 302 Row selector M
303 Row selector S
304 Row selectors 305, 312, 315 CDS (correlated double sampling) circuit 306 Horizontal scanning line circuit 307 Video voltage output line 308 Auxiliary still image voltage output line 310 Voltage output line 311 Voltage output line 313 Voltage output line 401-1 M reset gate 401-2 S reset gate 401-3 MS reset gate 402-1 M lead-out gate 402-2 S lead-out gate 402-3 MS lead-out gate 403-1 M floating diffusion 403-2 S floating diffusion 403- 3 MS floating diffusion 404-1 M source follower amplifier 404-2 S source follower amplifier 1000 Photodiode

Claims (9)

An imaging device that includes an imaging device that photoelectrically converts subject light and accumulates charges, and is capable of shooting a still image while shooting a movie,
Driving means for driving the image sensor separately for a moving image exposure period and an auxiliary still image exposure period when capturing subject light;
Moving image processing means for generating a moving image based on the moving image charge photoelectrically converted by exposure to the imaging element during the moving image exposure period;
Still image processing means for generating a still image based on the auxiliary still image charge photoelectrically converted by exposure to the image sensor during the auxiliary still image exposure period and the moving image charge. An imaging device that is characterized. The image pickup apparatus according to claim 1, wherein the image pickup device includes a reading unit that reads the charge for the moving image and the charge for the auxiliary still image for one frame during an exposure period for one frame. . The pixel of the imaging device includes a first holding unit that holds the charge for the moving image and a second holding unit that holds the charge for the auxiliary still image.
The imaging apparatus according to claim 1 or 2, wherein The pixel of the imaging device has a holding unit that holds the moving image charge and the auxiliary still image charge in a common holding unit.
The imaging apparatus according to claim 1 or 2, wherein The image sensor reads the charge exposed during the exposure period for the moving image by nondestructive readout as the charge for the moving image, and the charge exposed during the exposure period for the moving image and the auxiliary still image Means for reading out the charge exposed during the exposure period as the charge for the still image;
The imaging apparatus according to claim 4. The image sensor is configured to detect the moving image for a period from the start of the last exposure period for the moving image to the end of the still image capturing when the still image capturing is completed in the middle of the exposure period for the moving image. Non-destructive reading means for reading out electric charges by non-destructive reading to the second holding means,
The imaging apparatus according to claim 3 or 5, wherein The imaging device is provided with an output line for a signal generated from the charge for the moving image and an output line for a signal generated from the charge for the auxiliary still image,
The imaging apparatus according to any one of claims 1 to 6, wherein 2. The image pickup device is provided with a common output line for outputting a signal generated from the charge for the moving image and a signal generated from the charge for the auxiliary still image. The imaging apparatus as described in any one of -6. An imaging device control method comprising an imaging device that photoelectrically converts subject light and accumulates charges, and capable of capturing a still image while capturing a moving image,
A driving step of driving the image sensor separately for an exposure period for moving images and an exposure period for auxiliary still images when capturing subject light;
A moving image processing step for generating a moving image based on the moving image charge photoelectrically converted by being exposed to the imaging element during the moving image exposure period;
A still image processing step of generating a still image based on the moving image charge and the still image charge photoelectrically converted by exposure to the imaging element during an auxiliary still image exposure period. Control method for imaging apparatus.

Images