JP2011087338A - Image pickup device and image pickup method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup device capable of obtaining high-sensitivity and high-resolution without the use of a special sensor by combining summation of distances between screens and summation of distances between pixels and by reducing time lag between the screens in the summation of the distances between the screens; and to provide an image pickup method. <P>SOLUTION: The image pickup device has N pieces of image pickup elements 11 disposed in a matrix, and a driving means 52 for driving the image pickup elements 11 and reading image pickup signals. The image pickup device further has a synthesizing means for synthesizing first image data and second image data to thereby form third image data. The first image data has M (M<N) pieces of pixel data that are obtained by summing image pickup signals from the plurality of image pickup elements that are in a predetermined positional relation during reading of the image pickup elements when the driving means 52 reads the image pickup signals from a portion of the image pickup elements 11. The second image data has O (N≥O>M) pieces of pixel data that are obtained by the driving means that reads the image pickup signals from all the pickup elements 11. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging control method capable of imaging with high sensitivity and high resolution.

  An imaging device is used in a digital camera. By the way, with the advancement of technology, in addition to the increase in the number of pixels of a sensor in a digital camera, the downsizing of the imaging device has progressed, and each pixel of the sensor has become very fine. Although the resolution was improved by the high pixels, the sensitivity of the sensor was lowered as a trade-off. As a result, camera shake is likely to occur in dark places. A method of electrically amplifying the sensor output has been used for a long time, but in this case, noise is also amplified together, so that a large amplification cannot be performed.

  Therefore, many inventions have been made as follows.

  An invention that improves sensitivity by adding a plurality of pixels (see Patent Document 1), and a special (honeycomb) CCD (Charge Coupled Device) in which pixels with high sensitivity and pixels with low sensitivity are mixed. An invention that performs pixel addition using sensitivity data to achieve a high dynamic range, high sensitivity, and high resolution (see Patent Document 2), and there is a difference when comparing CCD fields divided and transferred for still images. Invention in which high sensitivity is achieved by adding fields to low-luminance subjects when there are few (see Patent Document 3), which obtains high sensitivity by adding a plurality of pixels, and devise readout timing at the time of readout Thus, an invention that suppresses a decrease in resolution that is reduced by pixel addition (see Patent Document 4), a multi-pixel addition is performed during high-sensitivity shooting, and no addition is performed during high-resolution shooting. The invention (see Patent Document 5), the invention for changing the addition condition of horizontal pixel addition according to the brightness for high-sensitivity photography (see Patent Document 6), and adding screens taken multiple times The invention which makes high sensitivity by doing (refer patent document 7).

  The sensitivity is “1” when the image sensor is exposed to T for a predetermined time, and when the outputs of the two image sensors are added, the sensitivity is equal to “2”. Further, when the same image sensor is exposed for 2T, the sensitivity becomes an exposure amount (accumulated data amount or luminance) equivalent to “2”.

  Each of the inventions described in Patent Documents 1, 4, 5 and 6 has a method of reading by adding pixels at the time of reading as shown in FIG. In this method, by adding and reading the outputs of two or more pixels, the sensitivity can be increased by the number of added pixels, but this is equivalent to a reduction in the number of pixels, and all pixels were used. There is a problem that the resolution is lower than shooting.

  In addition, the invention described in Patent Document 2 is intended to increase the dynamic range and increase the sensitivity by mixing high sensitivity pixels and low sensitivity pixels in the sensor. In addition to being necessary, the subsequent processing becomes complicated, resulting in problems such as an increase in cost due to an increase in circuit scale and an increase in the size of the imaging apparatus. Further, when the sensors have the same size and the same number of pixels, the number of high-sensitivity pixels is smaller than the total number of pixels, so that there is a problem that the resolution is lowered in a dark place.

  Further, the invention described in Patent Document 3 performs addition between lines after reading, but this also has a problem that the resolution is lowered.

  The invention described in Patent Document 7 is an invention of a method for increasing sensitivity by adding still images taken continuously. In the method of this publication, a time difference between captured screens becomes a problem. In a high pixel imaging device such as 5 million pixels, it takes 200 msec or more to read out all pixels. For this reason, the difference in exposure timing between images is about 250 msec in consideration of the exposure time (in other words, in the present invention, images having different times of 250 msec or more are added), and hand-held shooting becomes difficult. In addition, the shooting of a moving object also causes blurring.

  In addition, there is an image pickup apparatus provided with a mechanical mechanism to prevent camera shake, but this requires a device for detecting a shake and a device for moving a lens or a sensor, resulting in a problem of an increase in cost. In addition, this method has a problem that it cannot cope with so-called object blurring in which the subject moves.

  There is also a method of increasing the amount of light of the flash, but in this case, it is necessary to increase the capacity of the capacitor for accumulating electric charges, which raises the problem of cost increase and enlargement. In addition, when there is a subject in both the near and near, if the amount of light emission is adjusted so that the near side is appropriate, there is a problem that the subject on the far side becomes dark.

  As a countermeasure against the above problems, the present invention combines high-sensitivity and high-resolution imaging without using a special sensor by combining inter-screen addition and inter-pixel addition and reducing the inter-screen time lag of inter-screen addition. An object of the present invention is to provide an apparatus and an imaging method.

  In order to achieve the above object, an imaging apparatus according to the present invention includes: N imaging elements arranged in a matrix; and a driving unit that drives the imaging elements and reads an imaging signal. When the driving means reads the imaging signals from all or a part of the imaging elements, M obtained by adding the imaging signals from the plurality of imaging elements having a predetermined positional relationship at the time of reading the imaging elements First image data having (M <N) pixel data, and O (N ≧ O> M) number of images obtained by reading out the imaging signals from all or a part of the imaging elements by the driving unit. The image processing apparatus includes combining means for combining the second image data having pixel data and combining the third image data.

  As a result, the first image data, which is a high-sensitivity low-resolution image, and the second image data, which is a low-sensitivity high-resolution image, are added, and the conventional high-sensitivity adjacent pixel addition is lost. The high resolution can be realized with high sensitivity by supplementing the resolution with a high resolution image.

  In order to achieve the above object, the image pickup apparatus of the present invention includes an exposure unit that exposes the image pickup device, the drive unit reads the first image data, and the exposure unit The period during which the image sensor is exposed to obtain the second image data can be configured to partially overlap.

  As described above, since the period for reading the first image data that is a high-sensitivity low-resolution image and the period for exposure for the second image data that is a low-sensitivity high-resolution image partially overlap, The timing lag between the first image data and the second image data can be reduced.

  In order to achieve the above object, an image pickup apparatus according to the present invention includes an exposure unit that exposes the image pickup device, a process such as image data combining of the first image data and the second image data. A transfer means for transferring to the signal processing unit to perform, the exposure means exposes the image sensor to obtain the first image data, and the transfer means transfers the first image data. Then, the exposure means performs exposure on the image sensor to obtain the second image data, and the transfer means transfers the second image data.

  Thus, the period of exposure and reading for the first image data that is a high-sensitivity low-resolution image is longer than the period of exposure and reading for the second image data that is a low-sensitivity high-resolution image. Since it is short, the timing lag between the first image data and the second image data can be reduced even if the second image data is obtained after obtaining the first image data. In other words, by performing exposure and transfer of high-sensitivity data with high sensitivity and a short transfer time first, the time lag of the exposure start timing that occurred in the conventional two-screen addition can be shortened, causing camera shake. Can be difficult.

  In order to achieve the above object, the imaging apparatus of the present invention has a high-frequency pass filter, and the synthesizing means is processed by the first image data and the high-frequency pass filter. The two image data can be combined.

  As described above, the second image data that is the low-sensitivity high-resolution image is subjected to the high-frequency pass filter processing and then combined with the first image data that is the high-sensitivity low-resolution image. By extracting and adding the edge component of the second image data that is a low-sensitivity high-resolution image, the possibility of overexposure (high-intensity clip) due to addition can be reduced.

  In order to achieve the above object, the imaging apparatus according to the present invention includes an image size conversion unit, and the synthesis unit includes the first image data enlarged by the image size conversion unit, The image data can be combined with the image data.

  As a result, it is possible to synthesize image data by taking a good place between the first image data that is a high-sensitivity low-resolution image and the second image data that is a low-sensitivity high-resolution image. In other words, by adjusting to a large size, it is possible to obtain high-quality images by using high-resolution data information without cutting it.

  In order to achieve the above object, the imaging apparatus of the present invention has subtracting means for subtracting a predetermined value from the first image data and the second image data, and the synthesizing means The first image data or the second image data obtained by subtracting a predetermined value by the subtracting unit can be used for combining.

  Thereby, the noise of a low-intensity part can be removed by subtracting a fixed value. This process is more effective for low-sensitivity high-resolution images.

  In order to achieve the above object, the imaging apparatus of the present invention includes a white balance adjustment unit, and the white balance adjustment unit is configured to adjust the white balance based on the first image data. be able to.

  As described above, by acquiring the AWB evaluation value (RGB integration) at the time of still image shooting from the first image data that is a high-sensitivity low-resolution image with a large signal level, stable AWB processing with less influence of noise is performed. be able to.

  In order to achieve the above object, the imaging apparatus of the present invention includes a subtracting unit that subtracts a predetermined value from the first image data, and the white balance adjusting unit includes the subtracting unit, The white balance can be adjusted based on the first image data before subtracting a predetermined value from the first image data.

  Thereby, it is possible to prevent the white balance from being shifted due to subtraction. If the AWB gain multiplication is performed after the subtraction, the subtracted effect is different. For example, assuming that R = B = 10 and G = 20, the R gain is 2 times, the B gain is 2 times, and the value to be subtracted is 3. If subtraction is performed first, R: (10-3) × 2 = 14, G: (20-3) = 17, B: (10-3) × 2 = 14, and the white balance is lost. If AWB gain multiplication is performed first, R: (10 × 2) −3 = 17, G: (20−3) = 17, and B: (10 × 2) −3 = 17, and the white balance is not lost.

  In order to achieve the above object, the imaging apparatus of the present invention can be configured to have a readout pixel number switching means for switching the number of imaging elements read by the driving means.

  Thereby, for example, the number of readout pixels of the high sensitivity data is changed depending on the number of recording pixels. When recording a small number of pixels, it is possible to further shorten the deviation of the two exposure start timings by reducing the number of pixels to be transferred.

  In order to achieve the above object, an imaging method according to the present invention includes a driving step of driving N imaging elements arranged in a matrix to read out an imaging signal, and the driving step is wholly or partly. A first driving step of reading out the imaging signals from the imaging elements and adding the imaging signals from the plurality of imaging elements in a predetermined positional relationship to obtain M (N> M) pixel data; A second driving step of reading out the imaging signals from all or a part of the imaging elements to obtain O (N ≧ O> M) pixel data, and obtained in the first driving step. A synthesis step of synthesizing the third image data by synthesizing the first image data and the second image data obtained in the second driving step;

  In order to achieve the above object, the imaging method of the present invention has an exposure step of exposing the imaging device to obtain second image data, and the first image data is obtained by the first driving step. And a period during which the image sensor is exposed to obtain the second image data by the exposure step can be configured to partially overlap.

  In order to achieve the above object, the imaging method of the present invention includes a first transfer step of transferring the first image data to a signal processing unit that performs processing such as composition of image data, and the second. A second transfer step for transferring the image data to the signal processing unit, a first exposure step for performing exposure for exposing the image sensor to obtain first image data, and exposing the image sensor. And a second exposure step for performing exposure for obtaining second image data, and after the first exposure step and the first transfer step, the second exposure step and the second transfer step. It can comprise so that it may have.

  In order to achieve the above object, the imaging method of the present invention includes a high-frequency pass filter step for performing high-frequency pass filter processing for passing high-frequency components to image data, and the synthesis step May be configured to synthesize the first image data and the second image data processed by the high-pass frequency pass filter step.

  In order to achieve the above object, the imaging method of the present invention has an image size conversion step of converting an image size of the first image data, and the synthesis step is enlarged by the image size conversion step. The first image data and the second image data can be combined.

  In order to achieve the above object, the imaging method of the present invention includes a first subtraction step for subtracting a predetermined value from the first image data, and a predetermined value from the second image data. A second subtracting step for subtracting, and the combining step includes the first image data or the second image subtracted from a predetermined value by the first subtracting step or the second subtracting step. The data can be configured to be combined.

  In order to achieve the above object, the imaging method of the present invention includes a white balance adjustment step, and the white balance adjustment step is configured to adjust the white balance based on the first image data. be able to.

  In order to achieve the above object, the imaging method of the present invention has a subtraction step of subtracting a predetermined value from the first image data, and the white balance adjustment step is performed by the subtraction step. The white balance can be adjusted based on the first image data before a predetermined value is subtracted from one image data.

  In order to achieve the above object, the imaging method of the present invention can be configured to include a pixel number switching step for switching the number of imaging elements read out by the driving step.

  The present invention provides a high-sensitivity and high-resolution imaging apparatus and imaging method without using a special sensor by combining inter-screen addition and inter-pixel addition and reducing the inter-screen time lag of inter-screen addition. it can.

It is a figure for demonstrating the block diagram of an imaging device. It is an external view of a camera. It is a thinning transfer readout diagram with pixel addition. It is a figure for demonstrating the divided | segmented image. It is a figure for demonstrating an AWB multiplication circuit. 6 is a timing chart of normal mode still image recording. It is a figure for demonstrating the all pixel field transfer read-out figure. It is a figure for demonstrating the timing chart of the conventional screen addition. 3 is a timing chart (part 1) for still image recording in a high sensitivity mode. 7 is a timing chart (part 2) for still image recording in the high sensitivity mode. 12 is a timing chart (part 3) for still image recording in the high sensitivity mode. It is a functional block block diagram. It is a figure for demonstrating the imaging method.

Embodiments of the present invention will be described below with reference to the drawings, taking a digital camera as an example.
(System configuration)
FIG. 1 is a block diagram of a digital camera according to this embodiment, and FIG. 2 is an external view of the camera.

  First, each part of the block diagram of the digital camera shown in FIG. 1 will be described.

  The lens (zoom and focus lens) in the lens unit is driven by the motor driver 15. The motor driver 15 is controlled by a CPU (Central Processing Unit) 135 included in the signal processing IC.

  The imaging unit includes a CCD 11, a TG (timing signal generator) 124 that drives the CCD 11, a CDS 121 that samples an image signal from output data from the CCD, an AGC (analog gain controller) 122, and an electrical signal output from the CCD 11 (analog image data). ) Is converted to a digital signal. Here, the CDS 121, the AGC 122, the A / D converter 123, and the TG 124 are referred to as an AFE (analog front end) 12.

  The TG 124 supplies timing signals to the CCD 11, CDS 121, AGC 122, and A / D converter 123 based on the screen horizontal synchronization signal (HD) and the screen vertical synchronization signal (VD) under the control of the CPU 135. Yes. The TG 124 generates timing for reading the image sensor to drive means that drives the image sensors arranged in a matrix and reads the image signal, and supplies the timing to the drive means. The driving unit transfers the charge accumulated in the image sensor based on the timing generated by the TG 124. That is, based on the timing generated by the TG 124, the charge accumulated in the image sensor is subjected to vertical transfer and horizontal transfer. Further, when vertical transfer and horizontal transfer are performed, thinning of the image sensor, addition of charges, and the like are performed based on the timing generated by the TG 124.

  As a result, it can be said that the TG 124 has a function of switching the number of imaging elements.

  Further, the TG 124 can perform electronic shutter output, and has a function of controlling the exposure time of the image sensor by changing the number of outputs of the electronic shutter.

  Further, the digital signal from the imaging unit is applied to the signal processing IC 13 for signal processing. The signal processing IC 13 in FIG. 1 includes a memory controller 131, a CCD I / F unit 132, an image processing unit 133, a resizing / filtering unit 134, a CPU 135 that performs system control, a display I / F unit 136, a JPEG codec unit 137, and an inter-pixel calculation. Part 138, a card controller part 139, and a communication I / F part 140.

  The CCD I / F unit 132 outputs a screen horizontal synchronization signal (HD) and a screen vertical synchronization signal (VD), and takes in a digital RGB signal input from the A / D converter 123 according to the synchronization signal.

  Here, the screen horizontal synchronization signal (HD) and the screen vertical synchronization signal (VD) are output from the signal processing IC 13, but may be synchronized as a format output from the TG 124.

  During the monitoring operation, the RGB data is sent to the image processing unit 133. The image processing unit 133 converts the RGB data into YUV data, and the image data converted into a size suitable for display by the resizing / filtering unit 134 is SDRAM. The data is output to the YUV data area 173 of (Synchronous Dynamic Random Access Memory) 17.

  The SDRAM 17 has a RAW-RGB data area 172 and a JPEG data area 174 in addition to the YUV data area 173. The YUV data area 173 stores YUV format image data, the RAW-RGB data area 172 stores raw RGB format image data from the imaging unit, and the JPEG data area 174 stores JPEG format image data. Data is stored.

  At the time of still image shooting following the monitoring operation, all the pixels of the CCD 11 are transferred in a plurality of times, so that each field data is stored in the RAW-RGB data area 172 of the SDRAM 17 via the memory controller 131.

  The image processing unit 133 converts the RGB data sent from the CCD I / F 132 or the RGB data temporarily stored in the SDRAM 17 into YUV data based on the image processing parameters set by the CPU 135 that performs system control. Output.

  The resizing / filtering unit 134 receives YUV data and RGB data as input, and performs size conversion to a size necessary for recording, size conversion to a thumbnail image, size conversion to a size suitable for display, and the like. The resizing / filtering unit 134 also has a spatial filter processing function. In order to reduce image quality deterioration such as a jagged feeling or mosaic that occurs when enlarging or reducing, a plurality of linear interpolations, bicubic interpolations, and the like are provided. An interpolation method can be selected, and the setting is changed by the CPU 135 according to the required image quality and speed, and filter processing is performed. For example, since a high processing speed is required at the time of monitoring, horizontal only linear interpolation and filter OFF (simple thinning) are selected, and at the time of still image shooting, bicubic interpolation is selected because resolution is required.

  Further, the resizing / filtering unit 134 can operate only the filter function without performing resizing by setting the resizing magnification to 1. As a filter setting, a high-pass filter that extracts only a component having a high spatial frequency (an edge component of an image) or a low-pass filter that extracts and smoothes only a component having a low spatial frequency can be set.

  The inter-pixel calculation unit 138 performs a calculation between corresponding pixels of the two image data. The image data to be handled is YUV format data and RGB format data, and the number of bits of each YUV or each RGB data can be selected from 8, 12, 16 bits. When the input image is A and B and the output image is C, the calculation is performed by the following equation.

Clip [α × A (x, y) + β × B (x, y)] = C (x, y) (1)
Clip [α × A (x, y) × β × B (x, y)] = C (x, y) (2)
Clip [α × A (x, y) / β × B (x, y)] = C (x, y) (3)
In addition, the inter-pixel calculation unit 138 can perform various calculations including the calculation of the following formula (4).

Clip [α × A (x, y) + β] = C (x, y) (4)
However, −128 ≦ α ≦ 127, −128 ≦ β ≦ 127
x and y are horizontal (x) and vertical (y) positions in each image. The origin is set at the center or upper left of the screen.

Clip [] performs clipping with the number of output bits selected in the output image C.
The display I / F unit 136 sends the display data written in the SDRAM 17 to a display device (LCD / TV or the like) 19 to display a photographed image. The display device 19 can be displayed on an LCD built in the camera, or can be output as a TV video signal and displayed on a television.

  The JPEG codec unit 137 compresses the YUV format image data written in the YUV data area 173 of the SDRAM 17 during recording and outputs JPEG-encoded data. During playback, the JPEG codec unit 137 reads the JPEG read from the memory card 20. The encoded data is expanded into YUV format image data and output.

  The card controller unit 139 reads the data in the memory card 20 to the SDRAM and writes the data on the SDRAM 17 to the memory card 20 according to instructions from the CPU 135.

  The CPU 135, which is the overall control unit, loads the program and control data stored in the ROM 18 that can be rewritten at the time of startup into the SDRAM 17, and controls the whole based on the program code. The CPU 135 controls the imaging operation and the image processing parameters in the image processing apparatus in accordance with an instruction from the button of the operation unit 14, an external operation instruction such as a remote controller (not shown), or a communication operation instruction by communication from an external terminal such as a personal computer. Perform settings, memory control, and display control.

  The communication I / F unit 140 transmits and receives an image file by communication with a PC (Personal Computer), a printer, or the like via a USB (Universal Serial Bus) line 21. Further, the upgrade of the control program of the CPU 135 is realized by receiving the program from the PC via the USB line 21. Note that the communication I / F unit 140 may be connected to a line other than the USB line.

The operation unit 14 corresponds to the various buttons and switches shown in FIG. 4 and is used by the photographer to instruct the operation of the imaging apparatus. The operation unit 14 is a release key for instructing photographing, an optical zoom, and a zoom for setting an electronic zoom magnification. Input means for externally performing buttons, exposure mode selection, recording size selection, and other various settings.
(Normal mode)
The still image shooting mode includes a conventional normal mode shooting and a high-sensitivity shooting mode.

  First, the shooting operation in the normal mode will be described using a digital camera with 5 million pixels as an example.

  When the photographing / playback switching dial 25 in FIG. 2 is set to photographing and the camera is activated by the power button 26, the lens is moved to a photographing position by the motor driver 15 in FIG. 1, and the LCD (display device 19 in FIG. 1) Displays a monitoring image which is an electronic viewfinder function.

  When the zoom telephoto button 27 is pressed from the activated state, the zoom lens moves to the telephoto side (hereinafter referred to as the telephoto side). When the zoom wide-angle button 28 is pressed, the zoom lens moves to the wide-angle side (hereinafter, wide side).

  The CCD drive format at this time is shown in FIG. 3A (pixel addition 1/4 line readout). In FIG. 3A, white pixels are read and black pixels are not read. In addition, two pixels in the vertical direction are added to the read pixel, and a pixel “added in the vertical direction” is further added to another pixel added in the vertical direction. Therefore, although the line is thinned to ¼, the data size is four times as large as the data size of one pixel (thus, four times as bright).

  In the case of a 5-million-pixel CCD with 2560 × 1920 recording pixels, as shown in FIG. 3A, by performing pixel addition 1/4 line readout, the number of horizontal pixels is halved and the number of vertical pixels is 1. Since / 8, data of 1280 × 240 pixels is output.

  The TG 124 in FIG. 1 outputs control signals for pixel readout and addition transfer.

  The TG 124 outputs a control signal to the CCD 11 with the screen horizontal synchronization signal (HD) and the screen vertical synchronization signal (VD) from the signal processing IC 13 as the starting point of operation. In synchronization with the timing signal from the TG 124, the AFE 12 also samples the CCD output image data by the CDS 121, amplifies it by the AGC 122, performs A / D conversion by the A / D converter 123, and outputs it as 12-bit RGB data. The CCD I / F 132 also captures RGB format image data into the signal processing IC 13 in synchronization with the image transfer clock output from the TG 124. The signal processing IC 13 first performs black level adjustment and white balance adjustment with the CCD I / F 132, and then performs gamma conversion to output image data to the image processing unit 133 as 8-bit RGB format image data. The image processing unit 133 performs YUV conversion processing or edge enhancement processing for converting RGB format image data into YUV format image data, and outputs the result to the resizing / filtering unit 134. The resizing / filtering unit 134 performs resampling after linear interpolation is performed on 1280 × 240 pixel YUV format image data that is sequentially sent from the image processing unit 133 in synchronization with CCD driving (scanning). As a result, the image is resized to 640 × 480 pixels for display and is output to the SDRAM 17 via the memory controller 131. This operation is repeated at a cycle of 60 frames / second, which is a CCD readout cycle.

As described above, the display I / F unit 136 adds a control signal necessary for TV display or LCD display to the generated 640 × 480 pixel YUV format image data, and then displays the TV or LCD as a display device. Output to.
(Auto white balance)
Auto white balance (hereinafter, AWB) control will be described.

  The CCD I / F unit 132 in FIG. 1 calculates an integrated value for each RGB (RGB integrated value), which is an AWB evaluation value described later, for each block obtained by dividing the image as shown in FIG. Using the integrated data, AWB control values (R gain, B gain) are determined, and white balance is adjusted.

  FIG. 5 shows an AWB multiplication circuit for performing the AWB control. This circuit can be realized by the CCD I / F unit 132, the image processing unit 133, and the resizing / filtering unit 134. In this AWB multiplication circuit, when control is required, a gain is appropriately set, which is not necessary. In this case, the gain is set to 1.

  Next, an example of gain calculation in AWB will be described. The integrated values for each RGB in each block integrated by the CCD I / F unit 132 are stored in a register inside the CCD I / F unit 132, and the CPU 135 can read out the integrated values. ΣR, ΣG, and ΣB are calculated by further adding this integrated value for each color. Since white becomes white is a state where R = G = B, R gain and B gain are calculated as follows.

Rgain = (ΣG / ΣR)
Bgain = (ΣG / ΣB) (5)
Ggain = 1
Rgain, Bgain, and Ggain calculated by this equation (5) become Rgain45, Bgain46, and Ggain47 in FIG. The Rgain 45, Bgain 46, and Ggain 47 are supplied to the white balance gain amplifiers 41, 42, and 43, and the input RGB is controlled. As a result, white balanced outputs R′G ′ and B ′ are obtained from the AWB multiplication circuit.

  Although the basic functions of white balance have been described with reference to FIGS. 4 and 5, the present invention may have other white balance functions.

  In monitoring, each integrated value is read in synchronization with the screen vertical synchronization signal VD, R gain and B gain are calculated, and the gain is set in the AWB multiplication circuit of the CCD I / F unit 132.

This value is multiplied with the data read in the next frame. In the still, RGB is integrated from the first and second frames transferred interlaced. An AWB gain is calculated from the integration result, and AWB control is performed by an AWB multiplication circuit in the input unit of the image processing unit.
(Normal mode timing chart)
FIG. 6 is a timing chart showing the operation of still image recording in the normal mode. The relationship between the vertical synchronization new signal A, the mechanical shutter B, and the exposure control C is shown. The period up to t1 is a monitoring (also referred to as electronic view fighter or live view) period, t1 to t2 is a still picture exposure period (T1), and t2 to t3 is a still picture data transfer period (T2). In FIG. 6, the actual exposure time T3 is shorter than the still image exposure period (T1) by the electronic shutter.

  When the release button 30 is pressed (photographing request), the motor drive 15 is actuated by the instruction of the CPU 135 in FIG. 1, the focus lens is moved, and for example, a focusing operation such as CCD-AF based on the principle of mountain climbing AF is performed. Is called. Subsequently, in accordance with an instruction from the CPU 135, the TG 124 performs shooting preparation processing such as exposure time setting and still image exposure time setting. In the still image exposure period T1, the recording exposure processing is started from the screen vertical synchronization signal (VD). Done. When the exposure time T3 has elapsed, the mechanical shutter is closed (t2). Thereafter, the RGB data for all the pixels of the CCD are decomposed into four fields as shown in FIG. 7, for example, in the still image data transfer period (T2) and read out from the CCD (one field in one vertical synchronization period). 4 fields are transferred in 4 vertical synchronization periods.) Once, they are taken into the RAW-RGB data area 172 of the SDRAM 17.

  In the monitoring, as shown in FIG. 3A, the vertical is thinned out to 1/8 (and added with other corresponding pixels in the horizontal direction) and read out. On the other hand, the still image data transfer is performed by dividing the vertical 1/4 into four times. Since the number of transferred pixels has increased 16 times, the transfer time is about 0.26 seconds (1/60 seconds × 16 pixels). Also, during the still image data transfer period, the CCD I / F unit 132 performs RGB integration processing for auto white balance, as shown in the figure.

The RGB (RAW) data captured in the RAW-RGB data area 172 of the SDRAM 17 is sent to the image processing unit 133 of the signal processing IC 13 and converted into YUV format image data. The YUV format image data is converted to a size that matches the number of recording pixels by the resizing / filtering unit 134, and then subjected to JPEG compression processing, header information is added, and the image data is stored in the memory card 20 as a JPEG file.
(High sensitivity mode timing chart)
Next, the exposure timing of high-sensitivity mode shooting according to the present invention will be described by comparing the timing chart (FIG. 8) when performing conventional screen addition with the timing chart (FIG. 9) of high-sensitivity mode shooting according to the present invention. .

  FIG. 8 is a timing chart of the screen addition operation in the case of performing conventional screen addition, and shows the relationship between the vertical synchronization new signal A, the mechanical shutter B, and the exposure control C.

  In the case of FIG. 8, still image exposure and still image transfer are performed twice, and the RGB data for two screens captured in the memory (SDRAM) are added. In a system with a small number of pixels, since the transfer time of a still image is short, the time lag between two exposures has not been a big problem. However, in the case of 5 million pixels, as described above, it takes about 0.26 seconds only for the transfer time of the still image. Therefore, if preparation before exposure is added, an exposure time lag of about 0.3 seconds occurs. As a result, blurring occurs as in the case of long exposure.

  FIG. 9 is a timing chart of the present invention. Before starting exposure for a high-definition still image, first, exposure for high sensitivity is performed.

  When a photographing request is input by the photographer with the release button 30 in FIG. 2, AF is performed. Next, the exposure time for high sensitivity is set in the period (1) in FIG. 9 (this setting is performed by the CPU 135 for the TG 124, and the TG 124 performs exposure time control according to this setting). In FIG. 9, the periods (2) and (3) are set as exposure times for high sensitivity. Since the periods of (1), (2) and (3) are vertical synchronization periods, the exposure time for high sensitivity is for two frame transfer periods of 60 frames / second, and is set to about 33 msec. . Exposure (charge accumulation) starts from the end point (t1) of the VD signal. Subsequently, in the period (3), the CPU 135 sets the high-sensitivity transfer mode and the exposure time for still image recording for the TG 124.

  Data transfer for high sensitivity is performed during the period (4). The readout data at this time is the pixel addition 1/2 line readout mode in FIG. Since the pixels are read out in the horizontal and vertical directions, the sensitivity is almost quadrupled. The number of readout pixels is half the number of horizontal pixels (half because two pixels are added), the number of vertical pixels is 1/4 (half of the pixels are thinned, and two pixels are added). Therefore, 1280 × 480 pixels are output. Since the vertical line is doubled compared to the case of normal monitoring shown in FIG. 3A, the readout time is also doubled to about 33 msec. This data is stored in the SDRAM 17 as RGB data via the CCD I / F unit 132.

  At the time of high sensitivity readout during the period (4), RGB integration for AWB is performed by the CCD I / F unit 132 from the RGB data for high sensitivity. By setting the RGB integration start time in the period (3), RGB integration can be performed from the data transferred in the period (4), which is the next frame. The creation of RGB integration for AWB from this RGB data for high sensitivity is because the data for high sensitivity is greater than the value of still image data for normal sensitivity, so the influence of noise is reduced and stable AWB control is achieved. There is a merit that it can be done. Furthermore, since the R gain and the B gain for AWB control can be multiplied by the CCD I / F unit 132 at the time of still image data transfer, the processing for applying the AWB gain before the addition in the subsequent stage becomes unnecessary. Time is shortened.

  Further, in parallel with the high-sensitivity data reading in the period (4), the exposure is performed with the exposure time for a still image that is a high-definition image set in the period (3). In FIG. 9, exposure is performed for one vertical synchronization period (1 VD period) of the period (4) (even in the same 1 VD period diagram, the periods (1), (2), and (3) On the other hand, the period of (4) is doubled). During this period, the CPU 135 sets the still image transfer mode (FIG. 7) for the TG 124.

  At the end of the period (4), the mechanical shutter is closed, and the four-division interlace transfer as shown in FIG. 7 is performed from the period (5). The transferred data is sequentially written into the SDRAM 17 via the CCD I / F unit 132. When the transfer is completed, the mechanical shutter is opened, a transfer mode for monitoring is set for the TG 124, and the next shooting can be performed.

  As described above, in the conventional multi-screen addition (FIG. 8), the time lag between the two exposures (time lag) is large. There was a problem. However, the time lag can be shortened by using the mode in which reading is performed at a high speed as in the present invention for the first exposure data.

  In FIG. 9, the exposure time for high sensitivity and the exposure time for still images are both set to about 33 msec, but the present invention is not limited to this.

  As shown in FIG. 10, the still image exposure time can be set to 1 VD or more (the period (5) is added to the period (4). The period (5) is the data transfer period. In the same manner, it is possible to further increase the exposure time for high sensitivity. This may be switched according to the required image quality. For example, when high resolution is required, the still image exposure can be lengthened in order to increase the ratio of still image data.

FIG. 11 is a timing chart in a system in which high-sensitivity data transfer and still image exposure cannot be performed in parallel. That is, the exposure means exposes the image sensor to obtain high sensitivity data, and further, the transfer means exposes the image sensor to obtain high-definition image data after transferring the high sensitivity data. In addition, the transfer means transfers high-definition image data. Even in this case, it can be seen that the time lag is reduced as compared with the conventional system of FIG. Further, since the period (1) is a time for setting the exposure time of the high sensitivity data, a time corresponding to the normal 1VD period is not necessary. Therefore, in a system that can change the TG setting, change the operation clock, and change the VD cycle, shortening the 1 VD period can further shorten the deviation of the two exposure timings.
(Adding image data)
Here, three cases of the addition processing of two captured image data will be described. Addition using RGB data, addition performed by generating edge information from still image RGB data, and addition using YUV data after YUV conversion will be sequentially described.
(Addition with RGB data)
RGB data for two screens of 1280 × 480 pixels as high-sensitivity RGB data and 2560 × 1920 pixels as still image data is stored in the SDRAM.

  When recording a still image with 2560 × 1920 pixels, the high sensitivity data (1280 × 480 pixels) is first enlarged to 2560 × 1920 pixels. The high-sensitivity RGB data stored in the SDRAM 17 is read and sent to the resize filter unit 134. In the resize filter unit 134, the input unit multiplies the AWB gain by the AWB multiplication circuit (FIG. 5). As this gain, a value calculated from the RGB integrated value acquired at the time of transferring the high sensitivity data is used. The RGB data multiplied by the AWB gain is enlarged while applying a low pass filter, and the result is written back to the SDRAM 17. The low-pass filter is used here, as shown in FIG. 3B, because a part of the data is read out as shown in FIG. 3B, and there is a possibility that a false color is generated or a straight line is jagged during YUV conversion. is there. For this reason, data used to increase sensitivity is subjected to a low-pass filter.

  For still image data, the AWB gain is calculated from the RGB integration of the high sensitivity data, and AWB gain multiplication is performed in the CCD I / F unit 132 during transfer of the still image. If the AWB gain is not multiplied, the resizing / filter unit 134 can perform only the AWB multiplication process. In this case, however, the processing time is increased, so that the CCD I / F unit 132 performs multiplication in advance. It is desirable to keep it.

  As a result of the above-described processing, the high-sensitivity RGB data and the still image RGB data are both in a state of being multiplied by 2560 × 1920 pixels, 12 bits, and an AWB gain.

  Next, the two images are sent to the inter-pixel operation unit 138. In equation (1), still image data is added as image A, high sensitivity data is added as image B, and the addition result and image C are added. Here, the addition result is clipped with 12 bits. The image C is written back to the SDRAM 17 again. Here, α and β are set to 1, but the sensitivity may be further increased by increasing β of the image B in which noise is reduced by applying a low-pass filter to be larger than 1.

  Subsequent processing is performed in the same process as normal imaging. The RGB data of the image C written back to the SDRAM 17 is sent to the image processing unit 133 and subjected to edge enhancement processing, color adjustment, and the like, converted into YUV format image data, and written back to the SDRAM 17. The image data in the YUV format is JPEG compressed, added with header information, etc., and stored in a memory card.

  Here, as an example, a method for storing still images with only the sensitivity increased without increasing the resolution will be described.

  When recording a still image with 640 × 480 pixels, first, the normal still image (2560 × 1920 pixels) shown in FIG. 7 is the same size as the high-sensitivity data shown in FIG. 3 (B). Reduce to (1280 × 480 pixels). The still image RGB data stored in the SDRAM 17 is read and sent to the resize / filter unit 134. Since the still image RGB data has already been multiplied by the AWB gain in the CCD I / F unit 132, the resize filter unit 134 does not multiply the gain. The input RGB data is reduced by resampling while applying an interpolation filter such as a linear interpolation filter for each color, and written back to the SDRAM 17.

  Next, the high sensitivity data is read into the resizing / filtering unit 134. An AWB multiplication circuit (FIG. 5) multiplies the AWB gain at the input unit, applies a low-pass filter, and writes back to the SDRAM 17 without performing a resizing process.

  The high-sensitivity RGB data and the still-image RGB data are all sent to the inter-pixel operation unit 138 in a state in which 1280 × 480 pixels, 12 bits, and AWB gain are multiplied.

  In equation (1), still image data is added as image A, high sensitivity data is added as image B, and the addition result and image C are added. Here, the addition result is clipped with 12 bits. The image C is written back to the SDRAM 17 again.

  Subsequent processing is performed in the same process as normal imaging. The RGB data of the image C written back to the SDRAM 17 is sent to the image processing unit, subjected to edge enhancement processing, color adjustment, and the like, converted to YUV data, sent to the resizing / filtering unit 134 as it is, and YUV 640. X480 pixels are written back to the SDRAM 17. YUV is JPEG-compressed, and is added to the header information and stored in the memory card.

  In this method, if the exposure time for both high-sensitivity data and still image data is 33 msec (= 1/30 sec), the total exposure time is about 66 msec, but the high-sensitivity data has four times the sensitivity. Therefore, the sensitivity is 2.5 times as high as that obtained when the conventional photographing system as shown in FIG. 6 is exposed for 66 msec. High-sensitivity and high-resolution imaging with less camera shake is realized.

  In order to further increase the sensitivity, it is more effective to lengthen the exposure time of the high sensitivity data. If the exposure time for high sensitivity is 66 msec and the exposure time for still images is 33 msec, the total exposure time is about 100 msec. This corresponds to an exposure amount of about 300 msec in the conventional photographing system. That is, the exposure time is 1/3, and the sensitivity is tripled.

  Further, when a still image is recorded with 640 × 480 pixels, the high-sensitivity data may be used in the pixel addition 1/4 line readout mode of FIG. The number of readout pixels in this readout mode is 1280 × 240 pixels. In this case, the high sensitivity data is resized to 1280 × 480 pixels, and the normal still image data is reduced to 1280 × 480 pixels, and then the addition process is performed.

Since this pixel addition 1/4 line readout mode is 1/60 sec cycle, the period (4) in FIG. 9 can be shortened, and the deviation of the two exposure start timings can be further shortened.
(Addition by generating edge information from RGB data for still images)
This addition method is a method in which still image RGB data is used as edge information for increasing the resolution. First, the high sensitivity data (1280 × 480 pixels) is expanded to 2560 × 1920 pixels. The high-sensitivity RGB data stored in the SDRAM 17 is read and sent to the resize filter unit 134. In the resize filter unit 134, the input unit multiplies the AWB gain by the AWB multiplication circuit (FIG. 5). As this gain, a value calculated from the RGB integrated value acquired at the time of transferring the high sensitivity data is used. The RGB data multiplied by the AWB gain is enlarged while applying a low pass filter, and the result is written back to the SDRAM 17. The reason why the low-pass filter is used here is to suppress the possibility of occurrence of a false color or a problem of jagged lines during YUV conversion as described above.

  Next, the still image RGB data is sent to the resizing / filtering unit 134. Since the still image RGB data has already been subjected to AWB gain multiplication, the gain is set to 1 here. This still image RGB data is subjected to a high-pass filter to extract only the edge component, and is written back to the SDRAM 17 without resizing.

  The two images are sent to the inter-pixel calculation unit. In equation (1), still image data is added as image A, high sensitivity data is added as image B, and the addition result and image C are added. Here, the addition result is clipped with 12 bits. The image C is written back to the SDRAM 17 again.

  Subsequent processing is performed in the same process as normal imaging. The RGB data of the image C written back to the SDRAM 17 is sent to the image processing unit, subjected to edge enhancement processing, color adjustment, and the like, converted to YUV data, and written back to the SDRAM. YUV is JPEG-compressed, and is added to the header information and stored in the memory card 20.

  In this method, sensitivity is increased by high sensitivity data, and resolution is increased by still image data.

  If the exposure time of both the high sensitivity data and the still image data is about 33 msec, the total exposure time is about 66 msec, but the sensitivity of the high sensitivity data is quadrupled. Such a conventional photographing system corresponds to an exposure amount of about 166 msec. The resolution is obtained from still image data, and the exposure time is 1 / 2.5 (= 66/166), which is twice as sensitive. High-sensitivity and high-resolution imaging that is less susceptible to camera shake can be realized.

  In order to further increase the sensitivity, it is more effective to lengthen the exposure time of the high sensitivity data. If the exposure time for high sensitivity is 66 msec and the exposure time for still images is 33 msec, the total exposure time is about 100 msec. This corresponds to an exposure amount of about 266 msec in the conventional photographing system. The exposure time is 1 / 2.6, and the sensitivity is 2.6 times.

In the above processing, in order to obtain a high-pass filter output from still image data, if there is noise in the still image data, there is a possibility that the noise will be extracted. Therefore, noise reduction can be performed before the processing by the resizing / filtering unit 134. The noise reduction process can be performed by the inter-pixel calculation unit. Random noise usually has a low level. For this reason, using Expression (4), a negative value is added to β to remove noise in the low luminance part. The resolution information is extracted by the filtering process from the image from which noise is removed, and a higher quality image is obtained.
(Addition by YUV data after YUV conversion)
RGB data for two screens of 1280 × 480 pixels as high-sensitivity RGB data and 2560 × 1920 pixels as still image data is stored in the SDRAM.

  First, the high sensitivity data (1280 × 480 pixels) is sent to the image processing unit 133, and after the AWB multiplication processing is performed by the AWB gain calculated based on the RGB integrated data of the CCD I / F unit 132, color adjustment is performed. Etc. to convert it into YUV data. The YUV data is sent to the resizing / filtering unit 134 as it is, the enlargement processing is performed to 2560 × 1920 pixels while applying a low-pass filter, and the result is written back to the SDRAM 17. The low-pass filter is used here because, as shown in FIG. 3B, a part of the data is read out and there is a possibility that the line becomes jagged. Like that.

  Next, the still image RGB data is sent to the image processing unit 133. For still image data, the AWB gain is calculated from the RGB integration of the high sensitivity data, and the AWB gain multiplication is performed in the CCD I / F unit 132 at the time of still image transfer. Adjustment, edge generation processing, etc. are performed and converted into YUV data. The YUV data is sent to the resizing / filtering unit 134, is subjected to a high-pass filter, takes out only the resolution component (edge component), and writes it back to the SDRAM 17 without performing enlargement / reduction processing.

  Up to this point, the high sensitivity RGB data and the still image RGB data are both YUV data of 2560 × 1920 pixels.

  Next, only Y data of two images is sent to the inter-pixel operation unit 138. The human eye processes only Y data because the resolution of color information is weak and it is luminance (Y) data that affects the resolution.

  In Expression (1), the still image Y data is added as image A, the high sensitivity Y data is added as image B, and the addition result Y data and image C are added. Here, the addition result is clipped at 8 bits. The image C is written back to the SDRAM 17 again.

  Subsequent processing is performed in the same process as normal imaging. Using the UV data created from the Y data of the image C written back to the SDRAM 17 and the data for high sensitivity, the resizing / filtering unit performs resizing to the size required for recording, and the JPEG codec unit 137 performs JEPG compression and header information. After the addition, it is stored in the memory card 20 as a JPEG image file.

  In this method, as in the second method, sensitivity is increased by high sensitivity data, and resolution is increased by still image data.

  If the exposure time of both the high sensitivity data and the still image data is about 33 msec, the total exposure time is about 66 msec, but the sensitivity of the high sensitivity data is quadrupled. Such a conventional photographing system corresponds to an exposure amount of about 166 msec. The resolution is obtained from still image data, and the exposure time is 1 / 2.5 (= 66/166), which is twice as sensitive. High-sensitivity and high-resolution imaging that is less susceptible to camera shake can be realized.

In order to further increase the sensitivity, it is more effective to lengthen the exposure time of the high sensitivity data. If the exposure time for high sensitivity is 66 msec and the exposure time for still images is 33 msec, the total exposure time is about 100 msec. This corresponds to an exposure amount of about 266 msec in the conventional photographing system. The exposure time is 1 / 2.6, and the sensitivity is 4 times.
(Functional configuration block)
The imaging apparatus of the present invention has, for example, the blocks shown in FIG. That is, the exposure unit 51, the drive unit / pixel number switching unit 52, the transfer unit 53, the signal processing unit 54, and the SDRAM 17. Further, the signal processing unit 54 includes a synthesizing unit 541, a high frequency pass filter 542, a size converting unit 543, a subtracting unit 544, and a white balance adjusting unit 545.

  The exposure means 51 exposes the CCD 11 that is an image sensor. The driving unit / pixel number switching unit 52 drives the CCD 11 to read out an imaging signal and switches the number of pixels read out from the CCD 11. The transfer unit 53 transfers the read image pickup signal to the signal processing unit 54.

  The signal processing unit 54 processes the transferred image data. For example, the synthesizing unit 541 synthesizes a plurality of image data. The high frequency pass filter 542 performs high frequency pass filter processing on the image data. The size converter 543 converts the size of the image data. The subtracting means 544 subtracts a predetermined value from the image data. The white balance adjustment unit 545 adjusts the white balance for the image data.

The SDRAM 17 stores image data obtained as a result of processing by the signal processing unit 54 or stores image data processed by the signal processing unit 54.
(Imaging method)
The imaging method of the present invention has the processing flow shown in FIG.

  There are two processing flow routes up to the signal processing step S15. One is a process for obtaining high-sensitivity image data, and the other is a process for obtaining high-definition image data.

  Processing for obtaining high-sensitivity image data includes a high-sensitivity image pixel switching step S11, a high-sensitivity image exposure step S12, a high-sensitivity image drive step S13, and a high-sensitivity image image transfer step S14.

  In the high-sensitivity image pixel switching step S11, in order to obtain high-sensitivity image data, the pixels to be thinned out and the pixels to be added are set by the driving unit with respect to the image sensor (for example, FIG. 3A). . This setting is set in the TG 124 by the CPU 135.

  The high-sensitivity image exposure step S12 is a step of exposing a CCD, which is an image pickup element arranged in a matrix, and the high-sensitivity image drive step S13 is a step in which the drive means outputs an image pickup signal to the exposed CCD. The high-sensitivity image image transfer step S14 is a step of transferring the read imaging signal to the signal processing unit.

  Similarly, the processing for obtaining high-definition image data is a high-definition image pixel switching step S21, a high-definition image exposure step S22, a high-definition image driving step S23, and a high-definition image image transfer step S24.

  In the high-definition image pixel switching step S21, pixels to be thinned out and pixels to be added are set by the driving means for the image sensor to obtain high-definition image data (for example, FIG. 7). This setting is set in the TG 124 by the CPU 135.

  The high-definition image exposure step S22 is a step of exposing a CCD, which is an image pickup device arranged in a matrix, and the high-definition image drive step S23 is a step in which the drive means outputs an image pickup signal to the exposed CCD. The high-definition image transfer step S24 is a step of transferring the read image pickup signal to the signal processing unit.

  In the high-sensitivity image pixel switching step S11, image signals are read from all or some of the image sensors, and the image signals from a plurality of the image sensors having a predetermined positional relationship are added, and M (N > M) Switch so as to obtain pixel data (high-sensitivity image data). On the other hand, in the high-definition image pixel switching step S21, the imaging signals are read from all or a part of the imaging elements so as to obtain O (N ≧ O> M) pixel data (high-definition image data). Switch.

  Note that the transfer process for the high-definition image 24 is performed after the transfer process in the high-sensitivity image transfer step S14.

  Further, the driving process in the high-sensitivity image driving step S13 and the exposure process in the high-definition image exposure step S22 may partially overlap.

  Further, after the high-sensitivity image image transfer step S14, the exposure process in the high-definition image exposure step S22 may be performed.

  High-sensitivity image pixel switching step S11, high-sensitivity image exposure step S12, high-sensitivity image drive step S13, and high-sensitivity image image transfer step S14, and high-definition image pixel switching step The high-definition image data obtained in S21, high-definition image exposure step S22, high-definition image drive step S23, and high-definition image image transfer step S24, respectively, is signal processing step S15. Signal processing such as image size conversion, subtraction, and white balance adjustment is performed.

  Thereafter, the high-sensitivity image data and the high-definition image data are synthesized in the synthesis step S16.

  The white balance adjustment is preferably performed using high-sensitivity image data.

  When subtraction and white balance adjustment are performed, white balance adjustment is performed after white balance adjustment.

  The high-sensitivity image data may be enlarged to have the same size as the high-definition image data and may be combined.

  As described above, the present invention adds a high-sensitivity low-resolution image with a short readout time and a high-sensitivity image with a relatively low readout time, and devise the readout timing to achieve high sensitivity and high resolution. Therefore, it is possible to realize an imaging apparatus that is less likely to be shaken without adding a camera shake mechanism. Also, stable white balance can be achieved by acquiring white balance data from highly sensitive data.

11 CCD
12 AFE
13 Signal processing IC
14 Operation Unit 15 Motor Driver 16 Strobe Module 17 SDRAM
18 ROM
19 Display device 20 Memory card 21 USB
44 AWB gain amplifier 51 Exposure unit 52 Drive unit / pixel number switching unit 53 Transfer unit 54 Signal processing unit 121 CDS
122 AGC
123 A / D
124 TG
131 Memory Controller 132 CCD I / F Unit 133 Image Processing Unit 134 Resizing / Filtering Unit 135 CPU
136 Display I / F unit 137 JPEG codec unit 138 Inter-pixel calculation unit 139 Card controller unit 140 Communication I / F unit 541 Combining unit 542 High frequency pass filter 543 Size converting unit 544 Subtracting unit 545 White balance adjusting unit

JP 2005-45435 A JP 2004-336468 A JP 2004-235901 A JP 2002-84548 A JP 11-298800 A Japanese Patent No. 3123415 JP-A-5-344417

Claims (4)

In an imaging apparatus having N light receiving elements arranged in a matrix and driving means for driving the light receiving elements and reading an imaging signal,
When the driving means reads the imaging signals from all or a part of the light receiving elements, the imaging means obtained from a plurality of the light receiving elements in a predetermined positional relationship is obtained by adding the signals when reading the light receiving elements. First image data having M (M <N) pixel data;
Second image data having O (N ≧ O> M) pixel data obtained by reading out the imaging signal from all or a part of the light receiving elements;
An image pickup apparatus comprising combining means for combining the third image data by combining the first and second image data. Exposure means for exposing the light receiving element;
A period in which the driving unit reads out the first image data and a period in which the exposure unit exposes the light receiving element to obtain second image data partially overlap each other. The imaging device according to claim 1. A driving step of driving N light receiving elements arranged in a matrix to read out an imaging signal;
In the driving step, the imaging signals are read from all or a part of the light receiving elements, and the imaging signals from a plurality of the light receiving elements in a predetermined positional relationship are added, so that M (N> M) A first driving step for obtaining pixel data, and a second driving step for reading out the imaging signals from all or part of the light receiving elements to obtain O (N ≧ O> M) pixel data,
And a synthesis step of synthesizing the third image data by synthesizing the first image data obtained in the first driving step and the second image data obtained in the second driving step. An imaging method characterized by the above. An exposure step of exposing the light receiving element to obtain second image data;
The period during which the first image data is read out by the first driving step and the period during which the light receiving element is exposed to obtain the second image data by the exposure step partially overlap. The imaging method according to claim 3.

Images