JP7324039B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an imaging device and its control method.

2. Description of the Related Art In recent years, digital cameras capable of capturing still images while capturing moving images have been commercialized. With such a digital camera, when still image shooting is performed during moving image shooting, a moving image file and a still image file are separately recorded.

A common recording size for moving images is a size called full HD (FHD), and if still images are recorded in an FHD size suitable for moving images, the still images can be recorded by cutting them out from the moving images. However, since the FHD size is a size obtained by thinning out the image signal from the image sensor, the image quality of the still image is deteriorated from that of the original still image. Therefore, when still images are to be captured during moving image capturing, moving image capturing is stopped at the timing of still image capturing, processing to switch to high image quality drive for still images is performed, and still images are captured. After the still image shooting is finished, the moving image shooting is resumed.

However, in this process, since the process of stopping moving image shooting is included in the middle of moving image shooting, there arises a problem that the connection of recorded moving image files is partially interrupted. In order to solve this problem, a technique has been proposed for simultaneously outputting an image signal of a moving image size and an image signal of a still image size from an imaging device. Further, Japanese Patent Application Laid-Open No. 2002-200000 discloses that an image signal is compressed in an imaging device.

Japanese Patent Application Laid-Open No. 2006-25270

When the technique described in Patent Document 1 is used, a decoding circuit for decoding (decompressing) the compressed image signal is required in the image signal processing circuit that processes the image signal output from the image sensor. Furthermore, in the case of a configuration capable of simultaneously outputting an image signal of a moving image size and an image signal of a still image size from the imaging device as described above, the imaging signal processing circuit also requires two decoding circuits. becomes large.

In addition, if there is only one decoding circuit in the imaging signal processing circuit, it is not possible to receive the output of the moving image and the still image from the image sensor at the same time, and if the still image is shot during recording of the moving image, the moving image will be interrupted. still have issues with

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging apparatus capable of capturing still images without interrupting moving image recording regardless of the configuration of a decoding circuit included in a signal processing circuit. to provide.

An imaging apparatus according to the present invention includes a plurality of pixels for photoelectrically converting light from a subject, a first readout operation for reading out first image signals from the plurality of pixels, and a number of pixels larger than the first image signal. a readout means capable of performing a second readout operation for reading out a second image signal with a large number of pixels; and a first compressed image signal is generated by compressing the first image signal read out by the readout means. and a compression means for compressing the second image signal read by the reading means to generate a second compressed image signal; and the first image signal output from the image pickup element. decoding means for decoding the compressed image signal and the second compressed image signal; and sending the second compressed image signal to the decoding means during a period in which the decoding means does not decode the first compressed image signal. and a control means for controlling transfer, wherein the control means starts recording a new moving image based on the first image signal after recording of the moving image based on the first image signal ends. is instructed, recording of the new moving image is prohibited while the decoding means is decoding the second compressed image signal.

According to the present invention, it is possible to provide an imaging apparatus capable of capturing a still image without interrupting moving image recording regardless of the configuration of the decoding circuit included in the signal processing circuit.

FIG. 2 is a diagram showing the configuration of an imaging element in the imaging apparatus according to the first embodiment; FIG. 4 is a diagram showing a configuration example of pixels and column ADC blocks in the first embodiment; FIG. FIG. 4 is a diagram showing a pixel array of an image pickup device according to the first embodiment; 1 is a block diagram showing the configuration of an imaging device according to a first embodiment; FIG. The figure which shows the connection state of an image pick-up element and an image pick-up signal processing circuit in 4th Embodiment. FIG. 2 is a diagram showing a connection state between an image sensor and an image signal processing circuit according to the first embodiment; 4A and 4B are diagrams showing the control timing of the image sensor in the first embodiment; FIG. 4 is a timing chart of the imaging signal processing circuit according to the first embodiment; FIG. 4 is a timing chart of the imaging signal processing circuit according to the first embodiment; FIG. FIG. 4 is a flow chart diagram showing the operation of the imaging device according to the first embodiment; 4 is a timing chart of the imaging signal processing circuit according to the first embodiment; FIG. 4 is a flowchart showing the operation of the imaging device according to the first embodiment; FIG. 10 is a diagram showing control timings of the image sensor in the second embodiment; 9 is a flowchart showing the operation of the imaging device according to the second embodiment; FIG. 11 is a diagram showing control timings of an image sensor in the third embodiment; 9 is a flow chart showing the operation of the imaging device according to the third embodiment; The figure which shows the combination of the compression circuit and decoding circuit in 4th Embodiment. 9 is a flow chart showing the operation of the imaging device according to the third embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
(First embodiment)
FIG. 1(a) is a block diagram showing the configuration of an imaging device 100 according to the first embodiment of the present invention, and FIG. 1(b) is a schematic diagram showing the appearance of the imaging device 100. As shown in FIG. As shown in FIG. 1B, the imaging element 100 is formed from a first semiconductor chip 10 (imaging layer) and a second semiconductor chip 11 (circuit layer). of semiconductor chips 10 are stacked.

In the first semiconductor chip 10, a pixel portion composed of a plurality of pixels 101 arranged in a matrix is arranged on the light incident side, that is, on the optical image receiving side. Pixels 101 arranged in a matrix on the first semiconductor chip 10 are connected to transfer signal lines 103, reset signal lines 104, and row selection signal lines 105 in units of rows in the horizontal direction (row direction). . On the other hand, it is connected to the column output line 102a or 102b in the vertical direction (column direction). The pixels 101 connected to the column output line 102a are defined as a first pixel group, and the pixels 101 connected to the column output line 102b are defined as a second pixel group.

Column ADC blocks 111a and 111b provided in each column, row scanning circuit 112, column scanning circuits 113a and 113b, timing control circuit 114 and other pixel driving circuits are formed on the second semiconductor chip 11. . Further, a changeover switch 116, a frame memory 120, an in-element arithmetic unit 123, a resize conversion unit 119, a compression circuit 117, and a parallel/serial conversion unit (hereinafter referred to as P/S conversion unit) 118 are also formed.

In this way, by forming the pixels 101 on the first semiconductor chip 10 and forming the pixel driving circuit, the memory circuit, the arithmetic circuit, etc. on the second semiconductor chip 11, the imaging layer and the circuit layer of the imaging device 100 can separate the manufacturing process. As a result, it is possible to increase the speed, reduce the size, and increase the functionality by thinning and increasing the density of the wiring in the circuit layer.

The pixels 101 of the image sensor 100 are controlled to accumulate and read out charges by control signals from the row scanning circuit 112 via the transfer signal line 103 , the reset signal line 104 and the row selection signal line 105 . Signals are then read out from the pixels 101 in the row selected by the row scanning circuit 112 . In this embodiment, two output channels are provided as output channels. A first output channel consists of a column ADC block 111a, a column scanning circuit 113b and a signal line 115a, and a second output channel consists of a column ADC block 111b, a column scanning circuit 113a and a signal line 115a. Accordingly, signals can be read out in parallel from the pixels 101 of two rows.

A signal read from the pixels 101 of the first pixel group is sent to the column ADC block 111a of the first output channel via the column output line 102a of each column and AD-converted. Signals read from the pixels 101 of the second pixel group are sent to the column ADC block 111b of the second output channel via the column output line 102b of each column and AD-converted. After that, the columns to be read out are sequentially selected by the column scanning circuit 113a or 113b, and the AD-converted image signals are output to the switch 116 via the signal lines 115a or 115b. It is also possible to read out signals from the pixels 101 every other row using either one of the first and second output channels.

The timing control circuit 114 sends timing signals to the row scanning circuit 112, the column ADC blocks 111a and 111b, and the column scanning circuits 113a and 113b under the control of the overall control arithmetic unit 309. FIG.

The changeover switch 116 is a switch for selectively outputting the image signals output from the signal lines 115a and 115b to the frame memory 120 sequentially. The frame memory 120 temporarily stores the output image signal as image data.

The in-element calculation unit 123 performs calculations such as resizing processing and compression processing of image data according to the drive mode. The resizing unit 119 resizes the image data stored in the frame memory 120 to a required angle of view based on the result calculated by the in-element arithmetic unit 123 , and outputs the resized image data to the compression circuit 117 .

The compression circuit 117 compresses the image signal using a compression method such as a wavelet transform method (generates a compressed image signal). It is also possible to switch between compression and non-compression, and to convert the compression rate. Note that the compression method is not limited to the wavelet transform method. When compression is performed, the bit precision is lowered, so the image quality may be degraded, but since unnecessary components such as noise are also removed, the noise may be reduced.

When resizing or compression processing is unnecessary, transfer is performed directly from the switch 116 to the P/S conversion unit 118 . The P/S conversion unit 118 performs parallel/serial conversion on the image data and sends it to the imaging signal processing circuit 307 outside the imaging device 100 .

The imaging element 100 and the imaging signal processing circuit 307 are connected by a plurality of lanes, and signals of different pixels and signals of the same pixels are distributed to the main stream 121 and the substream 122 according to the drive mode. , or output from the main stream 121 only. In this way, the image signal can be output from the image sensor 100 at high speed by multi-stream driving in which two systems of image signals are output from the image sensor 100 in parallel.

In addition, as a readout driving method of the image sensor 100, there is a drive for reading out pixel signals of all pixels without reducing the number of pixels, and a readout in which the number of pixels is reduced by thinning out the number of pixels in the vertical direction to ⅓ or ⅕. It is possible to select drive, readout drive in which the number of pixels is reduced by adding a plurality of pixel signals in the horizontal direction, vertical thinning horizontal addition drive, and the like. It is also possible to select multi-stream driving in which driving for reading out all pixels and driving for ⅓ vertical thinning and horizontal addition are simultaneously output.

In the present embodiment, for example, all-pixel readout driving is executed in still image shooting readout, and ⅕ vertical thinning horizontal addition driving is executed in readout of moving image signals during standby before still image shooting. Further, for example, when still image shooting is performed during moving image recording, the image pickup device 100 is driven by all-pixel readout driving to output a still image signal, and the image pickup device 100 is driven by ⅓ vertical thinning horizontal addition driving. multi-stream driving for outputting moving image signals. That is, in either case, the total number of pixels of the still image signal is greater than the total number of pixels of the moving image signal.

FIG. 2 is a diagram showing details of the configuration of one pixel 101 and one column ADC block 111 of the image sensor 100 in this embodiment. The outline of the operation of the imaging element 100 according to the embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. Since the column ADC block 111a and the column ADC block 111b have the same configuration, they are described as the column ADC block 111 in FIG.

In FIG. 2, a photodiode (PD) 201 included in a pixel 101 photoelectrically converts received light into photocharges (here, electrons) having a charge amount corresponding to the amount of light. The transfer transistor 202 is connected between the cathode of the PD 201 and the gate of the amplification transistor 204, and is turned on when a transfer pulse φTRS is applied to the gate through the transfer signal line 103. FIG. A node electrically connected to the gate of the amplification transistor 204 constitutes a floating diffusion (FD) section 206 . Photoelectric charges photoelectrically converted by the PD 201 are transferred to the FD section 206 by turning on the transfer transistor 202 by the transfer pulse φTRS.

The reset transistor 203 has a drain connected to the pixel power supply Vdd, a source connected to the FD unit 206 , and a gate turned on when a reset pulse φRST is applied through the reset signal line 104 . Prior to the transfer of photocharges from the PD 201 to the FD section 206, the charge of the FD section 206 is reset to the pixel power supply Vdd by turning on the reset transistor 203. FIG.

The amplification transistor 204 has a gate connected to the FD unit 206 and a drain connected to the pixel power supply Vdd. The selection transistor 205 has a drain connected to the source of the amplification transistor 204, a source connected to the column output line 102, and a selection pulse φSEL applied to the gate through the row selection signal line 105 to turn on. .
While the selection transistor 205 is in the ON state, first, the potential of the FD section 206 after being reset by the reset transistor 203 is output to the column output line 102 as a reset level. Furthermore, by turning on the transfer transistor 202, the potential of the FD section 206 after transferring the photocharge is output to the column output line 102 as a signal level. In this embodiment, N-channel MOS transistors are used as the transistors 202-205.

The configuration of the pixel 101 is not limited to the configuration described above. For example, it is possible to adopt a circuit configuration in which the selection transistor 205 is connected between the pixel power supply Vdd and the drain of the amplification transistor 204 . Further, the present invention is not limited to the four-transistor configuration described above, and may be, for example, a three-transistor configuration in which the amplifying transistor 204 and the selection transistor 205 are used together.

A signal output from the pixel 101 via the column output line 102 is transmitted to the column ADC block 111 . Column ADC block 111 has comparator 211 , up/down counter (U/D CNT) 212 , memory 213 and DA converter (DAC) 214 .

The comparator 211 has a pair of input terminals, one of which is connected to the column output line 102 and the other of which is connected to the DAC 214 . The DAC 214 outputs a ramp signal whose signal level changes in a ramp-like manner over time based on the reference signal input from the timing control circuit 114 . The comparator 211 then compares the level of the ramp signal input from the DAC 214 and the level of the signal input from the column output line 102 . The timing control circuit 114 outputs a reference signal to the DAC 214 based on the command from the overall control calculation section 309 .

For example, the comparator 211 outputs a high-level comparison signal when the level of the image signal is lower than the level of the ramp signal, and outputs a low-level comparison signal when the level of the image signal is higher than the level of the ramp signal. Output. The up/down counter 212 is connected to the comparator 211, and counts the period during which the comparison signal is high level or low level, for example.

By this counting process, the output signal of each pixel 101 is converted into a digital value. An AND circuit may be provided between the comparator 211 and the up/down counter 212 , a pulse signal may be input to the AND circuit, and the number of pulse signals may be counted by the up/down counter 212 .

The memory 213 is connected to the up/down counter 212 and stores count values counted by the up/down counter 212 . Note that the column ADC block 111 counts the count value corresponding to the reset level of the pixel 101, counts the count value corresponding to the signal level after a predetermined imaging time, and stores the difference value between them in the memory 213. may After that, the count value stored in the memory 213 is transmitted as an image signal to the switch 116 via the signal line 115a or the signal line 115b in synchronization with the signal from the column scanning circuit 113. FIG.

It goes without saying that the column ADC block 111 is not limited to the configuration described above, and a known column ADC may be used.

FIG. 3 is a diagram schematically showing the pixel arrangement of the imaging device 100 in this embodiment. A Bayer array is applied to the color filters, and red (R) and green (Gr) color filters are alternately provided in order from the left to pixels in odd rows. In addition, green (Gb) and blue (B) color filters are alternately provided in order from the left in the even-numbered rows of pixels. Also, an on-chip microlens 201 is formed on the color filter 202 .

FIG. 4 is a block diagram showing the configuration of an imaging device 300 using the imaging element described in FIGS. 1 to 3 according to this embodiment.

In FIG. 4, an imaging apparatus 300 includes a lens 301, a lens driving unit 302, a mechanical shutter 303 (hereinafter referred to as a mechanical shutter), an aperture 304, a shutter/aperture driving unit 305, an imaging device 100, and an imaging signal processing circuit including a decoding circuit 316. 307.

The imaging apparatus 300 also includes a first memory unit 308, an overall control calculation unit 309, a recording medium control interface unit (hereinafter referred to as a recording medium control I/F unit) 310, a display unit 311, a recording medium 312, an external interface unit ( An external I/F unit 313 is provided. The imaging device 300 further includes a second memory section 314 and an operation section 315 .

Reflected light from the subject that has passed through the lens 301 is adjusted to an appropriate light amount by the diaphragm 304 and formed as a subject image on the imaging surface of the image sensor 100 . A subject image formed on the imaging surface of the imaging element 100 is photoelectrically converted by the pixels 101, further subjected to gain adjustment and A/D conversion for converting analog signals to digital signals, and R, Gr, Gb, It is sent to the imaging signal processing circuit 307 as a B signal.

The image signal compressed by the imaging device 100 is decoded by the decoding circuit 316 in the imaging signal processing circuit 307 . Also, in the imaging signal processing circuit 307, predetermined arithmetic processing is performed using the picked-up image signal, and the overall control arithmetic unit 309 performs exposure control and automatic focus adjustment control based on the obtained arithmetic result. As a result, TTL (through-the-lens) type AE (automatic exposure control) processing and EF (automatic flash light emission control) processing are performed.

Further, in the imaging signal processing circuit 307, predetermined arithmetic processing is performed using the imaged image signal, and TTL AWB (Auto White Balance) processing is also performed based on the obtained arithmetic result. In addition, various imaging signal processing such as low-pass filter processing and shading processing for reducing noise, various corrections, image signal compression, and the like are performed.

A lens driving unit 302 drives and controls the zooming, focusing, and the like of the lens 301 . A mechanical shutter 303 and an aperture 304 are driven and controlled by a shutter/aperture driver 305 . A general control calculation unit 309 controls the entire imaging apparatus 300 and performs various calculations.

A first memory unit 308 temporarily stores the image signal. The recording medium control interface unit 310 records or reads image signals onto or from a recording medium. The display unit 311 displays image signals. A recording medium 312 is a detachable recording medium such as a semiconductor memory, and records or reads image signals.

The external interface unit 313 is an interface for communicating with an external computer or the like. A second memory unit 314 stores the calculation result of the overall control calculation unit 309 . The operation unit 315 allows the user to set driving conditions for the imaging device 300 , and this information is sent to the general control calculation unit 309 to control the imaging device 300 as a whole. The operation unit 315 includes operation members such as a menu button for the user to set the imaging device and a playback button for checking the captured image.

FIG. 6 is a diagram showing the state of connection between the imaging element 100 and the imaging signal processing circuit 307 in this embodiment. As shown in FIG. 6, the imaging signal processing circuit 307 of this embodiment has only one decoding circuit.

The imaging device 100 simultaneously outputs a moving image signal and a still image signal. At this time, the moving image signal is compressed by the first compression circuit 117A and the still image signal is compressed by the second compression circuit 117B and transferred to the imaging signal processing circuit 307. FIG.

In the imaging signal processing circuit 307 , the moving image signal is decoded by the decoding circuit 316 . Also, the still image signal is temporarily stored in the first memory unit 308 via the imaging signal processing circuit 307 .

FIG. 7 is a control timing chart of the imaging device 100 in this embodiment. The imaging device 100 transfers the moving image signal to the imaging signal processing circuit 307 at 60 fps (frames/second). On the other hand, with respect to the still image signal, when the overall control calculation unit 309 receives an instruction based on the operation of the operation unit 315 by the user and determines to start still image shooting, the overall control calculation unit 309 receives the still image of the image sensor 100 . Allow signal transfer.

Further, when the overall control calculation unit 309 receives an instruction based on the user's operation of the operation unit 315 and determines that the still image shooting is finished, the overall control calculation unit 309 prohibits the transfer of the still image signal of the image sensor 100. . As a result, the imaging device 100 does not always output a moving image signal and a still image signal at the same time, but can be controlled to output the still image signal at the same time only when necessary.

FIG. 8 is a diagram showing the timing of decoding the still image signal in the imaging signal processing circuit 307 in this embodiment.

In the imaging signal processing circuit 307, the still image signal temporarily stored in the first memory unit 308 is decoded by the decoding circuit 316 during the blanking period during which the moving image signal is not transferred. After the decoding process is completed, the imaging signal processing circuit 307 creates a still image, records it on the recording medium 312, and releases the first memory unit 308. FIG.

Next, shooting modes will be described. The user can set the shooting mode using the operation unit 315 . As shooting modes, it has an auto mode and a video recording mode.

Auto mode is a still image shooting mode in which various parameters of the camera are automatically determined by a program incorporated in the imaging signal processing circuit 307 based on measured evaluation values.

The moving image recording mode is a mode in which various parameters of the camera are set to values suitable for moving image recording by a program incorporated in the imaging signal processing circuit 307 based on the measured evaluation values.

Although moving image recording is possible in auto mode, the transfer of image signals from the image sensor 100 needs to be interrupted once because the reading method of the image sensor 100 differs before and after moving image recording is started.

FIG. 9 is a diagram showing timings of moving image recording and still image recording in the auto mode in this embodiment. FIG. 9 also shows how the first and second compression circuits 117A and 117B of the imaging device 100 and the decoding circuit 316 of the imaging signal processing circuit are used.

Before moving image recording is started, the image pickup device 100 operates with 1/5 thinning drive, and the first and second compression circuits 117A and 117B and the decoding circuit 316 are not used. After that, the overall control calculation unit 309 receives an instruction based on the user's operation of the operation unit 315, and when it is determined to start moving image recording, the imaging device 100 is switched to multi-stream driving. In this case, the first compression circuit 117A and the decoding circuit 316 are used for compressing and decoding moving image signals.

The operation during moving image recording is the same as the timing control described with reference to FIG. 8, and the second compression circuit 117B is used only when still image shooting is performed. At this time, the still image signal is stored in the first memory unit 308 . Then, when the overall control calculation unit 309 receives an instruction based on the user's operation of the operation unit 315 and determines that the moving image recording is finished, the state is the same as before the start of moving image recording.

At this time, if the still image signal still remains in the first memory unit 308 , the still image signal stored in the first memory unit 308 is decoded by the decoding circuit 316 . After the decoding process is completed, a still image is created by the imaging signal processing circuit 307 and recorded on the recording medium 312, and the first memory section 308 is released. During the period until the first memory unit 308 is released, even if the general control calculation unit 309 receives an instruction based on the operation of the operation unit 315 by the user, the moving image recording prohibition period is set so that the moving image recording is not started. do.

FIG. 10 is a flow chart showing the control of FIG. In the present embodiment, the image sensor 100 on standby for moving image recording in the auto mode operates in 1/5 thinning drive. In S901, the overall control calculation unit 309 determines whether or not the user has given an instruction to start moving image recording.

If it is determined in S901 that the start of moving image recording has been instructed, the process advances to S902, and if it is determined that the start of moving image recording has not been instructed, S901 is repeated. In S902, the image sensor 100 is switched to multi-stream driving. In S903, encoding of the moving image is started.

At S904, it is determined whether there is a signal in the first memory unit 308 or not. If the signal exists in the first memory unit 308, the process proceeds to S905, and if the signal does not exist, the process proceeds to S909.

In S905, the signal decoding process of the first memory unit 308 is started. In S906, it is determined whether or not the decoding circuit 316 is used for the moving image signal, and if it is the time period for using the decoding circuit 316 for the moving image signal, the process proceeds to S907; The signal decoding process of 308 is continued. In S907, the signal decoding process of the first memory unit 308 is finished, and in S910, the moving image signal is decoded.

On the other hand, in S909, it is determined whether or not the decoding circuit 316 is to be used for the moving image signal. back to

After decoding the moving image signal in S910, the general control calculation unit 309 determines in S911 whether or not an instruction to start still image shooting has been given. If it is determined in S911 that the start of still image shooting has not been instructed, the process advances to S912, and if it is determined that the instruction has been given, the process advances to S913.

In S912, it is determined whether or not to end moving image recording. If moving image recording is to be ended, the process proceeds to S916; otherwise, the process returns to S904.

On the other hand, the still image signal is transferred to the imaging signal processing circuit 307 in S913, and the transferred still image signal is saved in the first memory unit 308 in S914. In S915, the transfer of the still image signal is prohibited, and the process proceeds to S912.

In S916, encoding of the moving image ends, and in S917, the imaging device 100 is switched to ⅕ thinning drive. At S<b>918 , it is determined whether there is a signal in the first memory unit 308 .

If it is determined in S918 that the signal does not exist in the first memory unit 308, the process proceeds to S922, and if it is determined that the signal exists, the process proceeds to S919.

In S919, moving image recording is prohibited, and in S920, the signal in the first memory unit 308 is decoded. In S921, prohibition of moving image recording is released, and the process proceeds to S922.

In S922, it is determined whether or not an operation member such as a menu button or a playback button has been pressed on the operation unit 315. FIG. If so, the process proceeds to S923; otherwise, the process returns to S901. In S923, the ⅕ thinning drive is terminated, and the sequence is terminated.

FIG. 11 is a diagram showing timings of moving image recording and still image recording in the moving image mode in this embodiment. FIG. 11 also shows how the first and second compression circuits 117A and 117B of the image sensor 100 and the decoding circuit 316 of the image signal processing circuit 307 are used.

Unlike the auto mode shown in FIG. 9, the image pickup device 100 is in multistream driving from before the start of moving image recording. In this case, the first compression circuit 117A and the decoding circuit 316 are used for compressing and decoding moving image signals.

After that, even if the overall control calculation unit 309 receives an instruction based on the user's operation of the operation unit 315 and it is determined to start moving image recording, the imaging device 100 does not switch the drive and starts moving image recording.

The operation during moving image recording is the same as the timing control described with reference to FIG. 8, and the second compression circuit 117B is used only when still image shooting is performed. At this time, the still image signal is stored in the first memory unit 308 . Then, when the general control calculation unit 309 receives an instruction based on the operation of the operation unit 315 by the user, it is determined that the moving image recording is finished, and still image signals still remain in the first memory unit 308, the first A decoding circuit 316 decodes the still image signal stored in the memory unit 308 .

At this time, the drive is temporarily switched to ⅕ thinning drive so that the still image signal can be decoded by the decoding circuit 316 without returning to the same state as before the start of moving image recording. After the decoding process is completed, the imaging device 100 is returned to the same state as before the start of moving image recording, and a still image is created by the imaging signal processing circuit 307 and recorded on the recording medium 312 . Then, the first memory unit 308 is released. The period until the first memory unit 308 is released is set as a movie recording prohibition period so that even if the general control calculation unit 309 receives an instruction based on the user's operation of the operation unit 315, the movie recording is not started. .

FIG. 12 is a flow chart showing the control of FIG. In this embodiment, the image pickup device 100 waiting for moving image recording in the moving image mode operates by multi-stream driving, and in S1101, the overall control calculation unit 309 determines whether or not an instruction to start moving image recording has been issued. If it is determined in S1101 that the moving image recording start has been instructed, the process advances to S1102, and if it is determined that the instruction has not been given, S1101 is constantly monitored. In S1102, encoding of the moving image is started.

At S<b>1103 , it is determined whether or not a signal exists in the first memory unit 308 . If the signal exists in the first memory unit 308, proceed to S1104; otherwise, proceed to S1108. In S1104, decoding of the signal of the first memory unit 308 is started.

In S1105, it is determined whether or not the decoding circuit 316 is to be used for the moving image signal. If it is determined that the decoding circuit 316 is to be used for the moving image signal, the process advances to step S1106; In S1106, the decoding of the signal in the first memory unit 308 ends, and in S1107, the moving image signal is decoded.

On the other hand, in S1108, it is determined whether or not the decoding circuit 316 is used for the moving image signal. If it is determined that the decoding circuit 316 is used for the moving image signal, the process proceeds to S1107, and if it is determined not to be used, the process returns to S1103.

After decoding the moving image signal in S1107, in S1109, the overall control calculation unit 309 determines whether or not an instruction to start still image shooting has been given. If it is determined in S1109 that the start of still image shooting has not been instructed, the process advances to S1110, and if it is determined that the instruction has been given, the process advances to S1111.

The still image signal is transferred to the imaging signal processing circuit 307 in S1111, and the still image signal is stored in the first memory unit 308 in S1112. In S1113, still image transfer is prohibited, and the process proceeds to S1110.

In S1110, it is determined whether or not to end moving image recording.If moving image recording is to be ended, the process proceeds to S1114.If moving image recording is not to be ended, the process returns to S1103.

In S1114, encoding of the moving image ends, and in S1115, it is determined whether or not there is a signal in the first memory unit 308. If it is determined in S1115 that the signal does not exist in the first memory unit 308, the process proceeds to S1121, and if it is determined that the signal exists, the process proceeds to S116.

In S1116, the image sensor 100 is switched to ⅕ thinning drive. In S1117, moving image recording is prohibited, and in S1118, the signal of the first memory unit 308 is decoded. In S1119, prohibition of moving image recording is released, and in S1120, the image sensor 100 is switched to multi-stream driving.

In S1121, it is determined whether or not an operation member such as menu or playback has been pressed on the operation unit 315 . If pressed, end multistream drive and end the sequence. If not pressed, the process returns to S1101.

As described above, according to the first embodiment, even if the imaging signal processing circuit 307 includes only one decoding circuit, the compressed image signal of the moving image size and the compressed image signal of the still image size can be combined. processing, and still image shooting can be performed without interrupting moving image recording.

(Second embodiment)
In the first embodiment, when the compressed moving image signal and still image signal are simultaneously transferred from the image sensor 100, they are supplied to the image signal processing circuit 307 and the first memory unit 308, respectively. On the other hand, in the second embodiment, the control performed by the imaging device 100 when the imaging signal processing circuit 307 has one decoding circuit will be described. Note that the configuration of the imaging device in the second embodiment is the same as that in the first embodiment, so the description thereof will be omitted.

FIG. 13 is a control timing chart of the imaging device 100 in the second embodiment. The imaging device 100 operates at 60 fps (frames/second), and transfers the moving image signal to the imaging signal processing circuit 307 once every two frames.

On the other hand, with respect to the still image signal, when the overall control calculation unit 309 receives an instruction based on the operation of the operation unit 315 by the user and determines to start still image shooting, the overall control calculation unit 309 receives the still image of the image sensor 100 . Allow signal transfer. At this time, the still image signal is transferred in a frame in which the moving image signal is not transferred.

Further, when the overall control calculation unit 309 receives an instruction based on the user's operation of the operation unit 315 and determines that the still image shooting is finished, the overall control calculation unit 309 prohibits the transfer of the still image signal of the image sensor 100. .

FIG. 14 is a flowchart showing control operations in the second embodiment.

In S1301, the imaging element 100 is set to transfer the moving image signal once every 2VD (vertical synchronization period) (transfer every other frame) before multistream driving is started.

In S1302, multistream driving is started, and in S1303, it is determined whether or not the start of still image recording has been instructed.

If the start of still image recording has been instructed in S1303, the process advances to S1304; otherwise, the process advances to S1306.

In S1304, it is determined whether or not the moving image signal is being transferred. If not, the process advances to S1305.
In S1305, the still image is transferred, and the process proceeds to S1306.

In S1306, it is determined whether the multi-stream driving is finished. Whether or not the multi-stream drive is terminated is determined by whether or not an operation member such as menu or playback has been pressed on the operation unit 315 .

If it is determined in S1306 to end the multi-stream driving, the process proceeds to S1307, and if it is determined not to end the multi-stream driving, the process returns to S1303.

In S1307, multistream driving is stopped and the sequence ends.

As described above, according to the second embodiment, even if the imaging signal processing circuit 307 includes only one decoding circuit, it processes a moving image size compressed image signal and a still image size compressed image signal. becomes possible. As a result, still image shooting can be performed without interrupting moving image recording.

(Third Embodiment)
In the third embodiment, as in the second embodiment, when the imaging signal processing circuit 307 has one decoding circuit, the control performed by the imaging element 100 will be described. Note that the configuration of the imaging apparatus in the third embodiment is the same as that in the first embodiment, so the description thereof will be omitted.

FIG. 15 is a control timing chart of the imaging device 100 in the third embodiment. The imaging device 100 operates at 60 fps (frames/second), and transfers to the imaging signal processing circuit 307 every frame.

On the other hand, with respect to the still image signal, when the overall control calculation unit 309 receives an instruction based on the operation of the operation unit 315 by the user and determines to start still image shooting, the overall control calculation unit 309 receives the still image of the image sensor 100 . While permitting signal transfer, prohibiting moving image signal transfer. Then, when the transfer of the still image signal is completed, the transfer of the moving image signal is started and the transfer of the still image signal is prohibited. For moving image recording of a frame for which transfer of moving image signals is prohibited, the moving image signal of the previous frame is used.

FIG. 16 is a flowchart showing control operations in the third embodiment.

In S1501, it is set so that the image sensor 100 does not transfer the moving image signal and the still image signal at the same time before multistream driving is started.

In S1502, multistream driving is started, and in S1503, it is determined whether or not the start of still image recording has been instructed. If the start of still image recording has been instructed in S1503, the process advances to S1504; otherwise, the process advances to S1506.

In S1504, it is determined whether or not a moving image signal is being transferred. If the moving image signal is not being transferred, the process advances to S1505, and if the moving image signal is being transferred, the process waits for the completion of the moving image signal transfer.

In S1505, the transfer of the moving image signal in the next VD (vertical synchronization signal) is stopped, only the still image is transferred, and the process proceeds to S1506.

In S1506, it is determined whether the multi-stream driving is finished. Whether or not the multi-stream drive is terminated is determined by whether or not an operation member such as menu or playback has been pressed on the operation unit 315 .

If it is determined in S1506 that the multi-stream driving is to end, the process advances to S1507; if it is determined not to end, the process returns to S1503.

In S1507, multistream driving is stopped and the sequence ends.

As described above, according to the third embodiment, even when the imaging signal processing circuit 307 includes only one decoding circuit, the compressed image signal of the moving image size and the compressed image signal of the still image size are processed. becomes possible. As a result, still image shooting can be performed without interrupting moving image recording.

(Fourth embodiment)
In the second and third embodiments, the method of dealing with the image sensor 100 side when the imaging signal processing circuit 307 has one decoding circuit has been described. In the third embodiment, when the imaging device 100 has two compression circuits and the imaging signal processing circuit 307 has two decoding circuits, control for selectively using them will be described. Note that the configuration of the image capturing apparatus of the third embodiment is the same as that of the first embodiment, and therefore the description thereof will be omitted.

FIG. 5 is a diagram showing the state of connection between the imaging element 100 in this embodiment and the conventional imaging signal processing circuit 307. As shown in FIG. The imaging device 100 includes a first compression circuit 117A and a second compression circuit. The imaging signal processing circuit 307 also includes a first decoding circuit 316A and a second decoding circuit 316B.

FIG. 17 is a diagram showing combinations of the number of compression circuits used by the imaging device 100 and the number of decoding circuits used by the imaging signal processing circuit 307. As shown in FIG. Depending on the combination, it is possible to select the corresponding method of the first embodiment, the second embodiment, or the third embodiment.

FIG. 17A is a schematic diagram of a case where the imaging device 100 uses two compression circuits 117A and 117B and the imaging signal processing circuit 307 uses two decoding circuits 316A and 316B. In this case, the imaging device 100 can simultaneously output a moving image signal and a still image signal, and the imaging signal processing circuit 307 can also perform decoding processing at the same time.

FIG. 17B is a schematic diagram of a case where the imaging device 100 uses two compression circuits 117A and 117B and the imaging signal processing circuit 307 uses one decoding circuit 316. FIG. In this case, the method of temporarily storing the still image signal in the first memory unit 308 of the first embodiment, the method of the second embodiment, or the method of the third embodiment is adopted.

FIG. 17C is a schematic diagram of a case where the imaging device 100 uses one compression circuit 117 and the imaging signal processing circuit 307 uses two decoding circuits 316A and 316B. In this case, the method of the second embodiment or the method of the third embodiment is adopted.

FIG. 17D is a schematic diagram of a case where the imaging device 100 uses one compression circuit and the imaging signal processing circuit 307 uses one decoding circuit 316 . In this case, the method of the second embodiment or the method of the third embodiment is adopted.

In order to reduce the power consumption of the imaging apparatus, it is advantageous not to operate the compression circuit of the imaging device 100 and the decoding circuit of the imaging signal processing circuit 307 as much as possible. Therefore, the user can use the operation unit 315 to select a combination of (a), (b), (c), and (d) in FIG. 17 according to the imaging conditions. .

Specifically, for example, when the exposure time determined by AE (automatic exposure control) in the imaging signal processing circuit 307 is shorter than 1/60 second, two compression circuits and two compression circuits shown in FIG. Select the method using the decoding circuit. If the exposure time is longer than 1/60 seconds, the transfer of image signals for each frame is not performed, so one of the methods shown in FIGS. 17(b), (c), and (d) is selected. Note that the power reduction effect is large when the method of FIG.

FIG. 18 is a flow chart showing the operation of changing the usage of the circuit according to the exposure time. In S1701, the imaging signal processing circuit 307 sets the exposure time while operating in multi-stream driving, and the overall control calculation unit 309 determines the length of the exposure time.

If the exposure time is shorter than 1/60 second in S1701, the process proceeds to S1702, and if it is 1/60 second or longer, the process proceeds to S1703.

In S1702, the method of FIG. 17A with two compression circuits and two decoding circuits is used.
In S1703, the method of FIG. 17(d) with one compression circuit and one decoding circuit is used.
In S1704, it is determined whether the multi-stream driving is finished. The termination of multistream driving is determined by whether or not an operation member such as menu or playback has been pressed on the operation unit 315 . If it is determined in S1704 to end the multi-stream driving, the process proceeds to S1705, and if it is determined not to end the multi-stream driving, the process returns to S1701.

In S1705, multistream driving is stopped and the sequence ends.

As described above, according to the fourth embodiment, it is possible to process a compressed image signal of a moving image size and a compressed image signal of a still image size while reducing power consumption, thereby interrupting moving image recording. Still image shooting can be performed without

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

100: image sensor, 301: lens, 302: lens driving unit, 303: mechanical shutter, 304: diaphragm, 305: shutter/aperture driving unit, 307: imaging signal processing circuit, 308: first memory unit, 309: overall control Operation unit 310: recording medium control I/F unit 311: display unit 312: recording medium 313: external I/F unit 314: second memory unit 315: operation unit 316: decoding circuit

Claims (15)

a plurality of pixels photoelectrically converting light from a subject; a first readout operation for reading out a first image signal from the plurality of pixels; and a second image signal having a larger number of pixels than the first image signal. a readout unit capable of performing a second readout operation; compressing a first image signal read out by the readout unit to generate a first compressed image signal; and readout by the readout unit. a compression means for compressing the outputted second image signal to generate a second compressed image signal;
decoding means for decoding the first compressed image signal and the second compressed image signal output from the imaging device;
control means for controlling transfer of the second compressed image signal to the decoding means while the decoding means is not decoding the first compressed image signal;
with
When an instruction to start recording a new moving image based on the first image signal is given after the recording of the moving image based on the first image signal is finished, the control means causes the decoding means to change the second 3. An image pickup apparatus, wherein recording of the new moving image is prohibited while the compressed image signal of the above is being decoded. Further comprising storage means for storing the second compressed image signal, the control means stores the second compressed image signal stored in the storage means while the decoding means is not decoding the first compressed image signal. 2. An imaging apparatus according to claim 1, wherein said storage means is controlled so as to transfer a compressed image signal to said decoding means. The compression means simultaneously outputs the first compressed image signal and the second compressed image signal, the control means transfers the first compressed image signal to the decoding means, and the second compressed image signal is transmitted to the decoding means. 3. The imaging apparatus according to claim 2, wherein the image signal is stored in said storage means. 4. The apparatus according to claim 2, wherein said control means reads out said second compressed image signal from said storage means and causes said decoding means to decode said second compressed image signal during a vertical synchronization period of said first image signal. Imaging device. 5. The control means prohibits reading of the first image signal when the decoding means decodes the second compressed image signal. The imaging device according to . The control means reads out the second image signal from the imaging device when a user instructs to capture the second image signal while the first image signal is being captured. 6. The imaging apparatus according to any one of claims 1 to 5, characterized by: The control means controls the reading means so as to transfer the second compressed image signal to the decoding means while the decoding means is not decoding the first compressed image signal. The imaging device according to claim 1, wherein: The reading means reads the first image signal every other frame, and the compressing means performs second compression based on the second image signal read in a frame in which the first image signal is not read. 8. The imaging apparatus according to claim 7, wherein an image signal is transferred to said decoding means. 8. The imaging apparatus according to claim 7, wherein said control means prohibits reading of said first image signal while said reading means is reading said second image signal. The compression means has two compression circuits, the decoding means has two decoding circuits, and the control means determines the number of compression circuits used in the compression means and the number of decoding circuits used in the decoding means. 10. The imaging apparatus according to any one of claims 1 to 9, characterized in that the number and number are switched according to imaging conditions. 3. The control means switches between the number of compression circuits used by the compression means and the number of decoding circuits used by the decoding means according to the length of exposure time of the imaging device. 11. The imaging device according to 10. 12. The imaging apparatus according to claim 1, wherein said first image signal is a moving image signal, and said second image signal is a still image signal. A method for controlling an imaging device having an imaging device having a plurality of pixels that photoelectrically convert light from a subject, comprising:
a first reading step of reading a first image signal from the plurality of pixels;
a second reading step of reading a second image signal having a larger number of pixels than the first image signal from the plurality of pixels;
compressing the first image signal read out by the first reading step to generate a first compressed image signal, and compressing the second image signal read out by the second reading step; a compression step of generating a second compressed image signal with
a decoding step of decoding the first compressed image signal and the second compressed image signal;
a control step of controlling to transfer the second compressed image signal to the decoding step while the first compressed image signal is not decoded in the decoding step;
has
In the control step, when an instruction to start recording a new moving image based on the first image signal is given after recording of the moving image based on the first image signal is completed, the decoding step is performed by the second image signal. and prohibiting the recording of the new moving image while the compressed image signal of the above is being decoded . a plurality of pixels photoelectrically converting light from a subject; a first readout operation for reading out a first image signal from the plurality of pixels; and a second image signal having a larger number of pixels than the first image signal. a readout unit capable of performing a second readout operation; compressing a first image signal read out by the readout unit to generate a first compressed image signal; and readout by the readout unit. a compression means for compressing the outputted second image signal to generate a second compressed image signal;
decoding means for decoding the first compressed image signal and the second compressed image signal output from the imaging device;
control means for controlling transfer of the second compressed image signal to the decoding means while the decoding means is not decoding the first compressed image signal;
with
The control means controls the reading means so as to transfer the second compressed image signal to the decoding means while the decoding means is not decoding the first compressed image signal. The means reads out the first image signal every other frame, and the compression means produces a second compressed image signal based on the second image signal read out in the frame in which the first image signal is not read out. to the decoding means. A method for controlling an imaging device having an imaging device having a plurality of pixels that photoelectrically convert light from a subject, comprising:
a first reading step of reading a first image signal from the plurality of pixels;
a second reading step of reading a second image signal having a larger number of pixels than the first image signal from the plurality of pixels;
compressing the first image signal read out by the first reading step to generate a first compressed image signal, and compressing the second image signal read out by the second reading step; a compression step of generating a second compressed image signal with
a decoding step of decoding the first compressed image signal and the second compressed image signal;
a control step of controlling to transfer the second compressed image signal to the decoding step while the first compressed image signal is not decoded in the decoding step;
has
In the controlling step, the reading step is controlled so as to transfer the second compressed image signal to the decoding step while the decoding step is not decoding the first compressed image signal. The step reads out the first image signal every other frame, and the compressing step produces a second compressed image signal based on the second image signal read out in a frame in which the first image signal is not read out. to the decoding step.

Images