JP2006352716A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and method capable of attaching a camera-shake preventing function with high reliability to a digital camera or the like at a lower cost. <P>SOLUTION: When a pixel summation factor is set to "twice" at photographing, a CCD is driven in a 2-pixel summation mode, an image signal with excellent quality is read, wherein the luminance is amplified by 2-pixel summation in a longitudinal direction (SB5), and when the pixel summation factor is set to "four times" in photographing, the CCD is driven in a 4-pixel summation mode, an image signal with excellent quality is read, wherein the luminance is amplified by 4-pixel summation in the longitudinal direction and a lateral direction, (SB9). The imaging apparatus can read the image signal wherein a plurality of pixels each are summated by the same color among pixels of all the pixels, that is, the imaging apparatus can read the image signal with excellent quality wherein the luminance is amplified and no leakage of luminance information is caused. The shutter speed can be maintained toward a higher speed, and the deterioration in the image due to an object-shake can be reduced without the need for using an angular velocity sensor for shake detection of a camera body and a drive mechanism of an optical system. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method suitable for use in, for example, a digital camera.

Conventionally, in a digital camera or the like, as a camera shake prevention structure that automatically prevents (corrects) camera shake caused by shaking of the camera body during still image shooting, for example, a part of the optical system of the imaging lens can be configured to be movable. By providing a variable optical system (such as a variable apex angle prism) in the optical system, a method of performing imaging while moving the imaging position with respect to the imaging device (changing the optical axis) during the exposure period of the imaging device, A method of moving the image sensor itself is known. In addition, in order to control the imaging position and the movement of the image sensor, the camera body is provided with two angular velocity sensors horizontally and vertically to detect the shaking in each direction, and based on the detection result. It is general (see, for example, Patent Document 1 below).
JP 2004-348147 A

  However, in the conventional technology, the camera body shake direction and the angular velocity sensor that detects the shake amount are indispensable, and the drive amount of the optical axis correction unit accurately matches the detected shake amount and timing of the camera body. In addition, the electric circuit of the control system for controlling the drive timing becomes complicated, and a precise drive mechanism is also required, so that the entire camera device becomes expensive.

  In addition, there is a mechanically movable part to move the imaging position on the image sensor or to move the image sensor itself, which is disadvantageous for downsizing of the device and is subject to external impact. However, there is a problem that the reliability is naturally limited when the service period is extended or the service period is extended.

  Furthermore, the conventional technology has a problem that it is not possible to prevent subject blur only by preventing camera shake.

  The present invention has been made in view of such conventional problems, and an imaging apparatus and an imaging method capable of adding a low-cost and highly reliable anti-shake function to a digital camera or the like, and its realization. The purpose is to provide a program used for.

  In order to solve the above-mentioned problem, in the invention of claim 1, in the image pickup apparatus including an image pickup device for picking up an image of a subject, the image pickup device is driven, while the image pickup device is driven in a plurality of fields. The driving means having a pixel addition mode for adding and outputting pixel signals of a plurality of pixels of the same color adjacent to each other apart from each other is provided.

  In such a configuration, by driving the image pickup device in the pixel addition mode by the driving unit, the image signal obtained by adding the pixel signals of all the pixels for each of the same color from the image pickup device, that is, the luminance is amplified. In addition, it is possible to read out a high-quality image signal that does not leak luminance information.

  Further, in the invention of claim 2, the pixel addition mode is a pixel signal of a plurality of pixels of the same color adjacent to each other in a field, in which the imaging device is driven in a plurality of fields. In the driving mode, pixel signals of a plurality of pixels of the same color existing in a plurality of horizontal lines spaced apart from each other are added and output while maintaining the arrangement order of the colors in the horizontal direction.

  In such a configuration, it is possible to read out a high-quality image signal with no luminance information leakage while the luminance is amplified by vertical pixel addition.

  Further, in the invention of claim 3, the pixel addition mode adds and outputs pixel signals of pixels of the same color adjacent in the vertical direction on the odd horizontal lines adjacent to each other in the odd field, In the even field, it is assumed that the drive mode is such that pixel signals of pixels of the same color adjacent in the vertical direction on the even horizontal lines adjacent to each other are added and output.

  In such a configuration, the pixel addition in the vertical direction of the odd horizontal line is performed in the odd field, and the pixel addition in the vertical direction of the even horizontal line is performed in the even field, so that the luminance is amplified and the luminance information is not leaked. A good image signal can be read out.

  According to a fourth aspect of the present invention, in the pixel addition mode, the image pickup device is driven in a plurality of fields, and pixel signals of a plurality of pixels of the same color adjacent to each other are separated for each field. The pixel signals of a plurality of pixels of the same color existing in a plurality of horizontal lines that are separated from each other are added while maintaining the arrangement order of the colors in the horizontal direction, and adjacent to each other that are separated from each other in the same horizontal line The drive mode is such that pixel signals of a plurality of pixels of the same color are added and output.

  In such a configuration, the luminance is amplified by the vertical and horizontal pixel addition, and a high-quality image signal with no leakage of luminance information can be read out. That is, an image signal whose luminance is amplified at a higher magnification can be read out.

  According to the invention of claim 5, in the pixel addition mode, in the odd field, pixel signals of pixels of the same color adjacent in the vertical direction on the odd horizontal lines adjacent to each other are added, and the same odd number is added. Pixel signals of pixels of the same color adjacent to each other in the horizontal direction on the horizontal line are added and output, and pixel signals of pixels of the same color adjacent in the vertical direction on the even horizontal lines adjacent to each other in the even field are output. And a drive mode for adding and outputting pixel signals of adjacent pixels of the same color that are spaced apart in the horizontal direction on the same even horizontal line.

  In such a configuration, in the odd field, the vertical pixel addition of the odd horizontal line and the horizontal pixel addition on the odd horizontal line are performed. In the even field, the vertical pixel addition of the even horizontal line and the horizontal pixel addition on the even horizontal line are performed. By performing pixel addition, it is possible to amplify the luminance and read out a high-quality image signal that does not leak luminance information.

  In addition, in the invention of claim 6, the pixel addition mode is a plurality of types in which the number of pixels to be added is different when the pixel signals of the plurality of pixels of the same color are added to the image sensor and output. The pixel addition mode includes a pixel addition mode switching unit that switches a type of the pixel addition mode when driving the imaging device in the pixel addition mode.

  In such a configuration, image signals having different luminance amplification magnifications can be read from the image sensor.

  In the invention of claim 7, the drive means has a normal drive mode in which the image pickup element outputs the pixel signals of all the pixels without adding the pixels, and the image pickup element of the image pickup element is driven by the drive means. Drive mode switching means for switching the drive mode between the pixel addition mode and the normal drive mode is provided.

  Further, in the invention of claim 8, an acquisition means for acquiring the brightness of the subject is provided, and the pixel addition mode switching means is configured to change the pixel addition mode based on the brightness of the subject acquired by the acquisition means. The type was changed.

  In such a configuration, it is possible to read image signals having different luminance amplification magnifications according to the brightness of the subject.

  Further, in the invention of claim 9, there is provided acquisition means for acquiring the brightness of the subject, and the drive mode switching means is based on the brightness of the subject acquired by the acquisition means. The drive mode of the image sensor is switched between the pixel addition mode and the normal drive mode.

  In such a configuration, it is possible to automatically select and read both the image signal with the amplified luminance and the image signal with the non-amplified luminance from the image sensor according to the brightness of the subject.

  According to a tenth aspect of the present invention, there is provided an imaging method of an imaging apparatus including an imaging device for imaging a subject, wherein the imaging device is driven in a plurality of fields when the imaging device is driven. Thus, the imaging method is set to the pixel addition mode in which pixel signals of a plurality of pixels of the same color that are adjacent to each other are separated and output for each field.

  In the invention of claim 11, a driving mode for driving the image sensor is driven in a plurality of fields by a computer included in an image pickup apparatus including an image sensor that images a subject, For each field, a program for setting a pixel addition mode in which pixel signals of a plurality of pixels of the same color that are separated from each other and adjacent are output is set.

  As described above, in the present invention, by driving the image pickup device in the pixel addition mode by the driving unit, the image signal obtained by adding the pixel signals of all the pixels for each of the same color from the image pickup device, that is, It is possible to read out a high-quality image signal that is amplified in luminance and has no leakage of luminance information. Therefore, if it is adopted in a digital camera or the like, the shutter speed can be maintained at a higher speed side, so that an angular velocity sensor for detecting shaking of the camera body, an optical system drive mechanism, etc. can be used. It is possible to reduce image deterioration due to subject blur. As a result, it is possible to add a low-cost and highly reliable anti-shake function to a digital camera or the like.

  In addition, image signals having different luminance amplification magnifications can be read from the image sensor.

  Further, it is possible to read out both the image signal with the amplified luminance and the image signal with the non-amplified luminance from the image sensor.

  Further, it is possible to read out image signals having different luminance amplification magnifications according to the brightness of the subject.

  In addition, it is possible to automatically select and read both the image signal with the amplified luminance and the image signal with the non-amplified luminance from the image sensor according to the brightness of the subject.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an outline of the electrical configuration of a digital camera 1 according to the present invention. The digital camera 1 has a still image shooting mode and a moving image shooting mode as shooting modes.

  As shown, the digital camera 1 has a lens block 2. The digital camera 1 has zoom, AF (autofocus), and AE (automatic exposure) functions. The lens block 2 includes a lens group including a zoom lens and a focus lens (not shown), an aperture, A mechanical shutter is included.

  The actuator block 3 includes a zoom motor and a focus motor for driving the zoom lens and the focus lens, an aperture actuator for controlling the opening degree by driving the aperture, and a shutter for opening and closing the mechanical shutter. Actuator. In this embodiment, the aperture that can be controlled is two stages. The driver circuit 4 is composed of the motors of the actuator block 3 and drivers for driving the actuators. The driver circuit 4 generates various drive signals according to instructions of the CPU 5 that controls the entire digital camera 1, and sends them to the actuator block 3. By supplying, each part of the lens block 2 is driven. The mechanical shutter can be replaced with an electronic shutter.

  The digital camera 1 also includes a CCD 6 on which subject light that has passed through the lens block 2 enters, a correlated double sampling circuit (CDS) 7, a programmable gain amplifier (PGA) 8, and an A / D converter (A / D). 9 is provided.

  The CCD 6 is a solid-state imaging device in which a Bayer array primary color filter is provided on a photosensitive surface on which an optical image of a subject is formed. The CCD 6 generates a timing signal generated by a timing generator (TG) 10 according to a command from the CPU 5. Based on this, the image signal is driven by the vertical / horizontal driver 11 and outputs an analog imaging signal corresponding to the optical image of the subject to the correlated double sampling circuit 7. In the present embodiment, the vertical / horizontal driver 11 constitutes the driving means of the present invention.

  The correlated double sampling circuit 7 reduces noise included in the input imaging signal by correlated double sampling, and outputs the reduced noise to the programmable gain amplifier 8. The programmable gain amplifier 8 adjusts the gain of the signal after noise reduction to a gain corresponding to the ISO sensitivity. The A / D converter 9 samples the image signal after gain adjustment, converts it to a digital signal having a predetermined number of bits, and outputs the digital signal to the image processing unit 12.

  The image processing unit 12 generates R, G, and B color component data (hereinafter, RGB data) for each pixel based on the input digital imaging signal (Bayer data) using the SDRAM 13 as a working memory in accordance with a command from the CPU 5. RGB interpolation processing, YUV conversion processing for generating YUV data consisting of luminance signals (Y) and color difference signals (U, V) from RGB data for each pixel, and digital for improving image quality such as auto white balance and edge enhancement Perform signal processing. The YUV data converted by the image processing unit 12 is sequentially stored in the SDRAM 13.

  The YUV data stored in the SDRAM 13 is converted into a video signal every time one frame is accumulated in the SDRAM 13 and sent to a liquid crystal monitor (LCD) 15 having a backlight (BL) 14 to obtain a through image. Displayed on the screen. Further, when shooting is performed with a shutter key (full pressing operation in the present embodiment) in the still image shooting mode, the image data temporarily stored in the SDRAM 13 is compressed by the CPU 5, and finally in a predetermined format. It is recorded in the external memory 16 as a still image file. During moving image shooting in the moving image shooting mode, a plurality of image data stored in the SDRAM 13 at a predetermined frame rate are sequentially compressed by the CPU 5 and finally recorded as a moving image file in the external memory 16.

  The external memory 16 is composed of, for example, various memory cards, and the still image file and the moving image file recorded in the external memory 16 are read and decompressed at any time by the CPU 5 according to the user's selection operation in the playback mode, and YUV After being expanded in the SDRAM 13 as data, it is displayed on the liquid crystal monitor 15.

  The digital camera 1 also includes a flash memory 17 that is a non-volatile memory capable of rewriting stored data. The flash memory 17 stores various programs for causing the CPU 5 to control the respective units, that is, programs for performing AE control, AF control, AWB control, and the like, and various data used for the control. In particular, the flash memory 17 includes a program for causing the CPU 5 to function as a pixel addition mode switching unit and an acquisition unit of the present invention, and control data constituting a program diagram (to be described later) used for AE control during still image shooting. Stored.

  The digital camera 1 includes a shutter button (not shown), a power key, a mode changeover switch, a key input unit 18 including a zoom up and zoom down button, a rechargeable battery 19 such as a nickel metal hydride battery, and the like. It has a power supply control circuit 20 for supplying power to each unit, and a microcomputer 21 for controlling them. The microcomputer 21 constantly scans the key input unit 18 for operation of various switches and the like, and when any operation key is operated by the user, an operation signal corresponding to the operation content is sent to the CPU 5. The shutter button has a so-called half shutter function that can be operated in two stages, a half-press operation and a full-press operation.

  Further, the CPU 5 includes a light emitting tube such as a xenon tube and a driving circuit for the light emitting tube, and a strobe circuit 22 for emitting auxiliary light as necessary during still image shooting. Further, the digital camera 1 has a recording function for recording ambient sounds in a moving image shooting mode or the like, and the CPU 5 is connected to an audio processing block 25 to which a microphone (MIC) 23 and a speaker (SP) 24 are connected. ing. The audio processing block 25 converts the audio signal input from the microphone 23 into digital data and inputs the digital data to the CPU 5. On the other hand, for example, the audio data recorded in the external memory 16 together with the moving image file is converted into an audio signal and drives the speaker 24. To do.

  Next, the above-described program diagram 100 stored in the flash memory 17 will be described. FIG. 2 is a diagram showing the program diagram 100. The program diagram 100 is setting information indicating a plurality of shooting conditions necessary for obtaining an appropriate exposure when the subject is at an arbitrary brightness (Lv value), as in the known one. The program diagram 100 of the form includes setting information related to pixel addition magnification in addition to general shutter speed, aperture (F value), and ISO sensitivity setting information as shooting conditions. The pixel addition will be described later.

  The program diagram 100 according to the present embodiment is a normal mode program diagram A indicated by a broken line in the figure, which is used during normal still image shooting, and the blur reduction prepared in the digital camera 1 in advance. This is composed of a program diagram B for the blur reduction mode shown by a solid line in the figure, which is used when taking a still image with the mode set to ON.

  The program diagram A (broken line) for the normal mode is a program diagram showing a combination of photographing conditions for limiting the ISO sensitivity (amplification factor of the amplifier 8) to “100” or less, that is, sensitivity with little noise generated in the photographed image. Yes, as shown in the figure, when the Lv value is “9” or less, the shutter speed becomes “1/64” or less according to the Lv value at that time. On the other hand, the program diagram B (solid line) for the blur reduction mode is a combination of shooting conditions in which the shutter speed is set to “1/64” or more as much as possible, that is, the speed at which camera shake is expected to occur during shooting. In accordance with the Lv value at that time, the ISO sensitivity increases in order until it reaches a maximum value “400” that can be set, and the pixel addition magnification reaches a maximum value “4 times” that can be set It goes up in order.

  In the present embodiment, the ISO sensitivities “50” to “400” shown in the program diagram 100 are obtained by setting the gain of the programmable gain amplifier 8 to 8 db, 14 db, 20 db, and 26 db. This is a realized value. Here, when the gain of the programmable gain amplifier 8 is set to 26 db (ISO sensitivity 400) and the pixel addition magnification described above is set to “2 times” and “4 times”, the programmable gain amplifier 8 is set. Can be obtained when the ISO gain is set to ISO sensitivities “800” and “1600”.

  The pixel addition magnification is a value corresponding to the number of pixels to be added when an image signal is read out from the CCD 6 during photographing. That is, in the present embodiment, as a read mode of the image signal from the CCD 6, in addition to the normal read mode, a two-pixel addition mode for reading out signal charges by adding two pixels in the vertical (vertical) direction, and a signal charge 4 pixel addition mode for reading out 4 pixels by adding 4 pixels in the vertical (vertical) direction and vertical (vertical) direction is prepared. The above-mentioned pixel addition magnification of “2 times” is a mode in which the CCD 6 drive mode is set to 2 pixels. Similarly, “4 times” of the pixel addition magnification indicates setting information for the readout mode indicating that the drive mode of the CCD 6 should be set to the 4-pixel addition mode.

  Here, a method of reading an image signal in the above-described 2-pixel addition mode and 4-pixel addition mode will be described with reference to the drawings.

  FIG. 3 is a schematic diagram of the CCD 6 showing the principle of the reading method in the two-pixel addition mode. The CCD 6 includes a large number of photosensitive CCDs (R, G, B in the figure) constituting pixels, a hold accumulation unit HOLD composed of CCDs that do not have a photosensitive function arranged in the horizontal direction, and a horizontal transfer unit 61. A vertical transfer unit (not shown) that vertically transfers the signal charges of the CCDs and an output circuit 62 are provided. Although not shown, the normal readout mode is a mode in which signal charges of all pixels (5 million pixels in the present embodiment) are read out from the CCD 6 by general frame readout without pixel addition.

  On the other hand, the 2-pixel addition mode is a mode in which the signal charge for one frame is read out in 4/8 lines and 2 fields as shown in FIG. That is, in the first field, only the signal charges of the odd-numbered lines (15, 13, 11,...) In the vertical direction are vertically transferred. At that time, the signal charges for one line are transferred using the hold accumulation unit HOLD. By temporarily holding the transfer time for one line, the signal charges of odd lines adjacent to each other are added, that is, the signal charges of two pixels of the same color are vertically added and transferred to the horizontal transfer unit 61. Thereafter, the signals of the odd lines obtained by vertically adding the signal charges of two lines composed of the same color pixel array (R, G, R, G,...) Are read out, and thereafter the signal charges of all the odd lines are read. Read sequentially. Next, in the second field, vertical addition similar to that in the first field is performed on the even-numbered lines (16, 14, 12,...) In the vertical direction, and the same color pixel array (G, B, G, different from the odd lines) is obtained. , B,...) Are read out from the even lines obtained by vertically adding the signal charges of two lines, and thereafter, the signal charges of all even lines are read out in order.

  FIG. 4 is a conceptual diagram showing Bayer data consisting of the imaging signals read out as described above. As shown in the figure, in the two-pixel addition mode, the number of pixels in the vertical direction is one half of the whole (2.5 million pixels). However, while the pixel information for the total number of pixels is reflected, the Bayer data in which the pixel value (luminance information) is amplified twice as a whole is obtained. FIG. 4 shows the positional relationship between the pixels (R, G, B) obtained by the vertical addition described above and the pixels that cannot be obtained, and the obtained pixels (R, G, B) at the time of shooting. ) Is temporarily stored in the SDRAM 13. In addition, the R, G, and B pixel positions after addition of the two pixels shown in the figure are the barycentric positions of the two pixels to be added.

  FIG. 5 is a schematic diagram of the CCD 6 showing the principle of the readout method in the 4-pixel addition mode described above. Similar to the two-pixel addition mode described above, the four-pixel addition mode is a mode in which signal charges for one frame are read in 4/8 lines and two fields as shown in the figure.

  That is, in the first field, only the signal charges of (15, 13, 11,...) Of the odd-numbered lines in the vertical direction are vertically transferred, and at that time, the signal charges for one line using the hold accumulation unit HOLD. Are temporarily held for the transfer time for one line, thereby adding the signal charges of odd lines adjacent to each other, that is, vertically adding the signal charges of two pixels of the same color and transferring them to the horizontal transfer unit 61. At the timing when the signal charge vertically added in the horizontal transfer unit 61 is transferred for two pixels, the signal charge next vertically added is added (horizontal addition) while being shifted by two pixels in the horizontal direction. Thereafter, signal charges for two lines composed of the same color pixel array (R, G, R, G,...) Are vertically added, and signals of two pixels of the same color existing on the same line every other pixel. The signals of odd lines to which charges are horizontally added are read out, and then the signal charges of all odd lines are read in order.

  Similarly, in the second field, vertical and horizontal additions similar to those in the first field are performed on the even-numbered lines (16, 14, 12,...) In the vertical direction, and the same color pixel array (G , B, G, B,...) Are added vertically, and signal charges for two pixels of the same color existing every other pixel on the same line are added horizontally. The signal is read, and thereafter, the signal charges of all even lines are read in order.

  FIG. 6 is a conceptual diagram showing Bayer data composed of the imaging signals read out as described above. As shown in the figure, in the four-pixel addition mode, the number of pixels in the vertical direction and the horizontal direction is each half (total pixels). Although the number of pixels is 1.25 million pixels, the pixel information corresponding to the total number of pixels is reflected, and Bayer data in which the pixel value (luminance information) as a whole is amplified four times is obtained. FIG. 6 shows the positional relationship between the pixels (R, G, B) obtained by the vertical addition and the horizontal addition described above and the pixels that cannot be obtained. Bayer data consisting only of G, B) is temporarily stored in the SDRAM 13. Also, the R, G, and B pixel positions after addition of the four pixels shown in the figure are the center of gravity of the original four pixels to be added.

  Next, the operation according to the present invention of the digital camera 1 having the above configuration will be described with reference to the drawings. FIG. 7 is a flowchart showing the contents of the process according to the present invention executed by the CPU 5 when the still image shooting mode is set by the user operating the mode changeover switch in the digital camera 1. In this flowchart, descriptions of AF control, AWB control, and the like are omitted.

  When the still image shooting mode is set, the CPU 5 causes the CCD 6 to perform an imaging operation of the subject at a timing according to a predetermined slew rate, and generates a through image based on the image data acquired by the imaging operation, and Display of the generated through image on the liquid crystal monitor 15 is started (step SA1). If the shutter button is not half-pressed (NO in step SA2), the through image is updated at regular intervals.

  During this time, although not shown, the CPU 5 sequentially accepts various key operations while displaying a through image, and the user performs predetermined key operations to set the above-described blur reduction mode on / off as necessary. Can be done. Further, during this period, AE processing using a through display program diagram suitable for displaying a through image is performed based on sequentially acquired image data.

  On the other hand, if the user presses the shutter button halfway while displaying a through image (YES in step SA2), the CPU 5 determines the brightness of the subject based on the luminance information of the image data acquired in the immediately preceding imaging operation. (Lv value) is determined (step SA3). Subsequently, it is confirmed whether or not the blur reduction mode provided in the digital camera 1 is set by the user, and if it is not set (NO in step SA4), the normal program diagram 100 described above is used. Based on the program diagram A for mode (broken line), the photographing condition corresponding to the brightness (Lv value) of the subject acquired in step SA3 is determined (step SA6). If the blur reduction mode is set (YES in step SA4), the brightness (Lv value) of the subject acquired in step SA3 is supported based on the program diagram B (solid line) for the blur reduction mode. The photographing condition to be determined is determined (step SA5).

  Thereafter, if the shutter button is not fully pressed (NO in step SA7), the process returns to the above-described step SA2, and if the half-press operation of the shutter button is continued (YES in step SA2), the above-described steps SA3 to SA6. If the shutter button half-press operation is not continued (NO in step SA2), the process waits for the shutter button half-press again. If the shutter button is fully pressed without releasing the half-pressed state of the shutter button (YES in step SA7), still image shooting processing (details will be described later) under the shooting conditions determined immediately before is performed. By performing this, image data for recording is generated (step SA8), and the generated image data is compressed and recorded in the external memory 16 (step SA9). Thereafter, the process returns to step SA1 to repeat the above-described operation.

  Details of the above-described still image shooting process will be described below. FIG. 8 is a flowchart showing the processing contents of the CPU 5 regarding the still image shooting processing (step SA8). In such processing, the CPU 5 controls the exposure of the CCD 6 with a predetermined shutter speed and aperture (F value) under the shooting conditions determined at that time (step SB1), and then the shooting conditions determined at that time. The following processing is executed according to the pixel addition magnification at.

  First, when the pixel addition magnification is 1 in step SB2, that is, when the blur reduction mode is not set or when the blur reduction mode is set and the brightness (Lv value) of the subject is, for example, “ The process when it is 8 ”or more will be described. In such a case, regardless of whether or not the blur reduction mode is set, first, normal readout is performed with the gain corresponding to the ISO sensitivity under the shooting conditions determined at that time set in the programmable gain amplifier 8. Depending on the mode, an image signal composed of signal charges for all the pixels is read from the CCD 6 (step SB3). Next, the image processing unit 12 performs RGB interpolation processing on the imaging signal (Bayer data) after A / D conversion, that is, color component data that each pixel of interest does not have, for the peripheral pixels having the color component data. RGB data for each pixel is generated by interpolation processing to obtain an average value of pixel values, and is composed of luminance component (Y) and color difference components (Cb, Cr) data for each pixel by YUV conversion processing based on the RGB data. YUV data is generated (step SB4).

  Next, the processing when the pixel addition magnification is 4 in the step SB2, that is, when the blur reduction mode is set and the brightness (Lv value) of the subject is “6” or less will be described. To do. In such a case, first, with the programmable gain amplifier 8 gain set to 26 db (ISO sensitivity 400), the CCD 6 is driven in the 4-pixel addition mode, and the number of pixels in the vertical and horizontal directions is 2 minutes each. An image signal consisting of signal charges corresponding to 1.25 million pixels, which is 1, is read (step SB5).

  Furthermore, the phase adjustment process in the horizontal direction and the vertical direction is performed on the imaging signal (Bayer data) read out and A / D converted here, and new Bayer data is acquired (step SB6).

  Such processing will be described with reference to the drawings. That is, each pixel constituting the Bayer data acquired by adding the four pixels is located at the center of gravity of the original four pixels added in the pixel space on the exposure surface of the CCD 6, so that each pixel as shown in FIG. The intervals are not the same. Therefore, when RGB interpolation is performed on the Bayer data, that is, as shown in FIG. 9, for example, if the target pixel 200 is an R pixel, the G component data of the target pixel is based on the four G pixels located around the target pixel 200. In addition, when the B component data of the target pixel is generated based on the four B pixels located in the vicinity, the correct G component data and B component data cannot be obtained. As a result, jaggies are generated in the finally obtained image.

  In order to avoid the phase adjustment processing, the Bayer data is reconstructed by converting the pixel value of each pixel constituting the Bayer data into a pixel value of a pixel having an equal interval between adjacent pixels in the pixel space. It is processing. FIG. 10 is a schematic diagram showing the phase adjustment process in the horizontal direction. If a new pixel of the same color is arranged at a position where the ratio of the distance from one to the distance from the other is 3: 5 between two pixels of the same color adjacent to each other on the same horizontal line, a new pixel The intervals between them are all the same. In addition, the pixel value of the new pixel can be obtained by calculating a weighted average of the pixel values of the original pixels of the same color located on both sides according to the distance to the new pixel. .

  Therefore, for example, a new R pixel 200 is arranged between the R pixels 201 and 202 shown in the drawing at a position where the distance from the R pixel 201 is 3/8 and the distance from the R pixel 202 is 5/8. The pixel value of the new R pixel 200 is a value obtained by weighted averaging the pixel values of the R pixels 201 and 202 according to the distance to the new R pixel 200 (the value of the R pixel 201 × the value of the 5/8 + R pixel 202). X 3/8), and Bayer data is reconstructed with the data of the new R pixel 200 and the like. FIG. 11 is a schematic diagram showing the phase adjustment processing in the vertical direction. In the vertical direction as well, new pixel data is acquired by the same method as in the horizontal direction, and Bayer data is reconstructed thereby. .

  Then, after the phase adjustment, RGB interpolation processing is performed on the reconstructed Bayer data, and YUV conversion processing is performed on the RGB data generated thereby, and each of the luminance component (Y) and color difference components (Cb, Cr) for each pixel is performed. YUV data consisting of data is generated (step SB7). Further, the generated YUV data is subjected to a double enlargement process in the vertical and horizontal directions, and the same amount as when the pixel addition magnification is “1” is obtained by interpolating pixels lost by the addition of four pixels. Image data for recording having the number of pixels (5 million pixels) is generated (step SB8). The interpolation at the time of the enlargement process is, for example, by a method similar to that in the known digital zoom (such as linear interpolation from adjacent peripheral pixels).

  Next, when the pixel addition magnification is 2 in Step SB2, that is, the blur reduction mode is set and the brightness (Lv value) of the subject is between “6” and “7”, for example. The processing at that time will be described. In such a case, first, the CCD 6 is driven in the two-pixel addition mode with the programmable gain amplifier 8 gain set to 26 db (ISO sensitivity 400), and the number of pixels in the vertical direction is ½. An image signal composed of signal charges for 10,000 pixels is read (step SB9).

  Further, vertical phase adjustment processing is performed on the image signal (Bayer data) that has been read out and A / D converted, and new Bayer data is acquired (step SB10). This process is the same as the vertical phase adjustment process in the case where the pixel addition magnification described above is four times.

  Then, after the phase adjustment, RGB interpolation processing is performed on the reconstructed Bayer data, and YUV conversion processing is performed on the RGB data generated thereby, and each of the luminance component (Y) and color difference components (Cb, Cr) for each pixel is performed. YUV data composed of data is generated (step SB11). Further, the generated YUV data is subjected to a double enlargement process in the vertical direction, and the same pixel as when the pixel addition magnification is “1 ×” is obtained by interpolating the pixels lost by the two-pixel addition. A number (5 million pixels) of image data for recording is generated (step SB12). The interpolation at the time of the enlargement process is, for example, by a method similar to that in the known digital zoom (such as linear interpolation from adjacent peripheral pixels).

  Then, the CPU 5 completes the still image shooting process at the time when any of the above-described steps SB4, SB8, and SB12 is completed, and then returns to the process of FIG. 7 to generate the YUV generated by any of the above processes. The data is compressed as image data for recording and recorded in the external memory 16.

  Here, in the above-described still image shooting processing, pixel addition (two-pixel addition or four-pixel addition) is performed on an image captured by the CCD 6 before performing RGB interpolation, resulting in RGB data after RGB interpolation. The phase adjustment for preventing the phase shift of the color component is performed on the Bayer data, and the Bayer data is reconstructed and then the RGB interpolation is performed. However, the phase adjustment is performed in the RGB interpolation process. You may make it perform.

  That is, FIG. 12 is a flowchart corresponding to FIG. 8, showing a modification of the still image shooting process.

  Also in this modified example, the CPU 5 controls the exposure of the CCD 6 by a predetermined shutter speed and aperture (F value) under the photographing conditions determined at that time (step SC1), and then determines at that time. The following processing is executed according to the pixel addition magnification under the shooting conditions.

  First, when the pixel addition magnification is 1 (“1” in Step SC2), the same processing as Steps SB3 and SB4 in the flowchart of FIG. 8 is performed (Steps SC3 to SC3), and the image processing unit 12 generates YUV data (steps SC3 to SC5).

  Next, processing when the pixel addition magnification is 4 in the step SC2 will be described. In such a case, first, with the programmable gain amplifier 8 gain set to 26 db (ISO sensitivity 400), the CCD 6 is driven in the 4-pixel addition mode, and the number of pixels in the vertical and horizontal directions is 2 minutes each. An image signal consisting of signal charges corresponding to 1.25 million pixels, which is 1, is read (step SC6).

  Subsequently, the image processing unit 12 is caused to perform RGB interpolation processing including phase adjustment in the horizontal direction and vertical direction with respect to the imaging signal (Bayer data) after A / D conversion (step SC7). In such processing, unlike the general interpolation processing in step SC4, the color component data that the pixel does not have for each pixel of interest is set to a simple average value of the pixel values of the surrounding pixels that have the color component data. Instead, the average value (weighted average) is obtained by weighting each pixel value of the surrounding pixels according to the distance from the target pixel.

  More specifically, if the target pixel is an R pixel having an R component as shown in FIG. 9, G component data is generated based on four G pixels located in the periphery of the R pixel, and the pixel is located in the periphery. B component data is generated based on the four B pixels. At this time, each pixel is read by the four-pixel addition described above, and the positional relationship of each pixel when the exposure surface of the CCD 6 is assumed (actually, the position of the center of gravity of the added four pixels) ), The distance between the target pixel (R pixel) and each of the surrounding four pixels (G pixel or B pixel) is not the same. Accordingly, when the mere average value of the peripheral pixel values is used as the pixel value of the target pixel, a phase shift occurs in the color component and jaggy occurs in the finally obtained image. The pixel values of the surrounding four pixels to be reflected in the target pixel are once weighted by multiplying the weighting coefficient corresponding to the difference in distance, and the average value thereof is set as the pixel value of the target pixel.

  That is, with reference to FIG. 10, for a certain target pixel 200, a horizontal distance between two peripheral pixels (here, R pixels) 201 and 202 existing on the same horizontal line among the peripheral pixels. Is “8”, the horizontal distance from the left peripheral pixel 201 to the target pixel 200 is “3”, and the horizontal distance from the right peripheral pixel 201 to the target pixel 200 is “5”. Is set to 5/8 times as a weighting coefficient, and 3/8 times is set as a weighting coefficient to the right peripheral pixel 202. Then, the pixel values of both the peripheral pixels 201 and 202 are multiplied by the above weighting coefficient, and the average value thereof is set as the pixel value of the target pixel 200. Here, the weighting method of the peripheral pixels is limited to the horizontal direction, but actually, the weighting is performed by the same method not only in the horizontal direction but also in the vertical direction.

  Thereafter, YUV data is generated by YUV conversion processing based on the RGB data for each pixel generated by the RGB interpolation processing including the phase adjustment described above (step SC8), and further, 2 in the vertical and horizontal directions with respect to the generated YUV data. The image data for recording having the same number of pixels (5 million pixels) as when the pixel addition magnification is “1 ×” is obtained by performing the double enlargement process and interpolating the pixels lost by the four-pixel addition. Generate (step SC9).

  Next, processing when the pixel addition magnification is 2 in the step SB2 will be described. In such a case, first, the CCD 6 is driven in the two-pixel addition mode with the programmable gain amplifier 8 gain set to 26 db (ISO sensitivity 400), and the number of pixels in the vertical direction is ½. An image signal composed of signal charges for 10,000 pixels is read (step SC10).

  Then, the image processing unit 12 is caused to perform RGB interpolation processing including phase adjustment in the vertical direction with respect to the imaging signal (Bayer data) after A / D conversion (step SC11). In such processing, unlike step SC7 described above, when obtaining the color component data that the pixel does not have for each pixel of interest from the pixel values of the peripheral pixels having the color component data, the pixel values of the peripheral pixels are temporarily set. After performing weighting by multiplying the weighting coefficient according to only the difference in the vertical distance with respect to the target pixel, the average value thereof is set as the pixel value of the target pixel. Thereby, RGB data is generated for each pixel with no color component phase shift.

  Thereafter, YUV data is generated by YUV conversion processing based on the RGB data for each pixel generated by the RGB interpolation processing including the phase adjustment described above (step SC12), and further doubled in the vertical direction with respect to the generated YUV data. By performing the enlargement process, the pixels lost by the two-pixel addition are interpolated to generate image data for recording with the same number of pixels (5 million pixels) as when the pixel addition magnification is “1”. (Step SC13). This modification can also prevent the image from being jaggy.

  As described above, in the digital camera 1 of the present embodiment, when the blur reduction mode is set at the time of shooting, it is expected that the brightness of the subject and the control speed of the shutter speed in the normal mode will cause camera shake. Even if the brightness is lower than the speed, the brightness of the image photographed by the above-described two-pixel addition or four-pixel addition is amplified to ensure proper exposure. Therefore, it is possible to reduce image deterioration due to camera shake during shooting by maintaining the shutter speed at a higher speed without using an angular velocity sensor for detecting camera shake or an optical drive mechanism. . As a result, a low-cost and highly reliable camera shake prevention function can be added by the digital camera.

  Even when the subject is dark as well as when the subject is bright, it is possible to deal with image shake (subject shake) due to movement of the subject as well as camera shake caused by the shake of the camera body.

  In addition, maintaining the shutter speed on the higher speed side is realized by amplifying the brightness of the photographed image by the above-described two-pixel addition or four-pixel addition. Unlike the case where the maintenance is realized by increasing the gain of the programmable gain amplifier 8, there is no deterioration of the image due to an increase in noise.

  When shooting a bright subject, the image data of all the pixels is read from the CCD 6 without performing 2-pixel addition or 4-pixel addition. In this case, the number of acquired pixels is unnecessarily reduced, that is, The quality of the captured image does not deteriorate.

  In this embodiment, the pixel addition mode (in this embodiment, the image signal reading mode from the CCD 6) is switched based on the brightness (Lv value) of the subject at the time of shooting. The mode may be switched based on the shutter speed once determined by AE control.

  In addition, pixel addition (two-pixel addition or four-pixel addition) in an image captured by the CCD 6 has been described as being performed inside the CCD 6 when the imaging signal is read from the CCD 6. However, similar pixel addition is output from the CCD 6, for example. For the subsequent image data, arbitrary hardware may be used, or software processing by a computer such as the CPU 5 may be used.

  The pixel addition is not necessarily performed for pixels in the Bayer data state, but may be performed for RGB data after RGB interpolation or Y, Cb, Cr data after YUV conversion.

  Further, the phase adjustment for preventing the phase shift of the color component generated in the RGB data after the RGB interpolation due to the pixel addition may be performed as a pre-process of the RGB interpolation process as in the present embodiment. As in the modification shown in FIG. 12, it may be performed in the RGB interpolation process. Note that when phase adjustment is performed in advance on Bayer data before RGB interpolation is performed as in the present embodiment, the RGB interpolation processing can be simplified and existing interpolation processing circuits and interpolation processing software can be used. Can be used as is. Further, in the case of performing the RGB interpolation processing as in the modification, a phase adjustment circuit and phase adjustment software only for reconstructing Bayer data are not required.

  In addition, when 4-pixel addition is performed, the phase adjustment in both the vertical and horizontal directions is performed. As a matter of course, a higher quality image can be obtained by performing in both the vertical and horizontal directions as in the present embodiment.

  The pixel addition mode (in this embodiment, the image signal readout mode from the CCD 6) is automatically switched based on the brightness (Lv value) of the subject at the time of shooting. May be switched by manual operation as required.

  In addition, the pixel addition mode (in this embodiment, the image signal readout mode from the CCD 6) is switched based on the brightness (Lv value) of the subject at the time of shooting, but the pixel addition mode is changed by AE control. You may make it switch based on the shutter speed once determined.

  Further, in the present embodiment, the image size (pixel size) when recording a photographed image is fixed to the maximum size that can be captured by the CCD 6, but in a digital camera, recording is possible. In general, a plurality of image sizes are prepared, and the user can select and set a desired image size as appropriate. Therefore, in the present embodiment, in the above-described still image shooting process (FIGS. 8 and 12), the YUV data is simply enlarged at an enlargement rate corresponding to only the pixel addition magnification (2 ×, 4 ×) at that time. However, in a configuration in which the image size can be selected, for example, it is desirable to change the processing in step SB8 and step SB12 in FIG. 8 to the following enlargement or reduction processing.

First, the print image size set (selected) at that time is confirmed, and then the print image size and the YUV data size at that time (this may be determined from the pixel addition magnification). Compare Here, if the recorded image size is larger, the enlargement ratio necessary for enlarging the YUV data to that size is calculated. Conversely, if the recorded image size is smaller, the YUV data is reduced to that size. Calculate the required zoom factor. Thereafter, the YUV data is enlarged or reduced at the calculated magnification.
However, when calculating the enlargement magnification or reduction magnification described above, if the pixel addition magnification at that time is 2 times and the aspect ratio of the YUV data is different from the normal aspect ratio, the magnifications in the vertical direction and the horizontal direction are set. Individually calculated, the YUV data is enlarged or reduced at different magnifications in the vertical and horizontal directions. In addition, when the recorded image size and the YUV data size are compared initially, if the two sizes are the same, the still image shooting process is completed without performing the above-described magnification calculation and enlargement / reduction process. To do.

  In this way, the photographed image can be recorded with the recording image size set at that time regardless of the pixel addition magnification at that time, that is, the brightness of the subject. Therefore, for example, it is possible to prevent the size of the recorded image from changing every time the image is taken due to the difference in brightness of the subject, and the usability of the recorded image after shooting and the quality of the digital camera can be reduced. Can be maintained. Further, the recorded image size can be obtained by one enlargement or reduction process, and unnecessary enlargement or reduction process can be eliminated.

  In the present embodiment, the case where the present invention is applied at the time of still image shooting / recording has been described. However, the present invention can also be applied at the time of moving image shooting / recording or during live view display. That is, according to the program diagram as shown in FIG. 2, the AE control may be performed at the time of moving image recording / recording or when displaying a through image.

1 is a block diagram of a digital camera according to the present invention. It is a figure which shows the program diagram stored in flash memory. It is a schematic diagram of CCD which shows the principle of the reading method by 2 pixel addition mode. It is the conceptual diagram which showed the Bayer data which consists of an imaging signal read by 2 pixel addition mode. It is a schematic diagram of CCD which shows the principle of the reading method by 4 pixel addition mode. It is the conceptual diagram which showed the Bayer data which consists of an imaging signal read by 4 pixel addition mode. It is a flowchart which shows the processing content of still image shooting mode. It is a flowchart which shows the content of the still image shooting process in still image shooting mode. It is a figure which shows the attention pixel in the RGB interpolation process when a pixel addition magnification is 4 times. It is explanatory drawing which shows the relationship between the attention pixel in the RGB interpolation process in case a pixel addition magnification is 4 times, and the surrounding pixel of a horizontal direction. It is explanatory drawing which shows the relationship between the said attention pixel and the surrounding pixel of the vertical direction. FIG. 9 is a flowchart corresponding to FIG. 8 and showing a modified example of still image shooting processing.

Explanation of symbols

1 Digital Camera 2 Lens Block 3 Actuator Block 4 Driver Circuit 5 CPU
6 CCD
7 Correlated double sampling circuit (CDS)
8 Programmable gain amplifier (PGA)
9 A / D converter (A / D)
10 Timing generator (T / G)
11 Vertical / Horizontal Driver 12 Image Processing Unit 13 SDRAM
14 Backlight (BL)
15 LCD monitor (LCD)
16 External Memory 17 Flash Memory 18 Key Input Unit 19 Battery 20 Power Supply Control Circuit 21 Microcomputer 22 Strobe 23 Microphone 24 Speaker 25 Audio Processing Block

Claims (11)

In an imaging apparatus including an imaging device for imaging a subject,
Drive means having a pixel addition mode for driving the image sensor while driving the image sensor in a plurality of fields and adding and outputting pixel signals of a plurality of pixels of the same color that are spaced apart from each other and adjacent to each other An imaging apparatus comprising:   In the pixel addition mode, the image sensor is driven in a plurality of fields, and pixel signals of a plurality of pixels of the same color that are adjacent to each other are separated for each field, and are present in a plurality of horizontal lines that are separated from each other. 2. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is a drive mode in which pixel signals of a plurality of color pixels are added and output while maintaining a horizontal color arrangement order.   In the pixel addition mode, pixel signals of the same color pixels adjacent in the vertical direction on the odd horizontal lines adjacent to each other in the odd field are added and output, and on the even horizontal lines adjacent to each other in the even field. 3. The imaging apparatus according to claim 2, wherein the imaging apparatus is a drive mode in which pixel signals of pixels of the same color adjacent in the vertical direction are added and output.   In the pixel addition mode, the image sensor is driven in a plurality of fields, and pixel signals of a plurality of pixels of the same color that are adjacent to each other are separated for each field, and are present in a plurality of horizontal lines that are separated from each other. Add the pixel signals of multiple pixels of the same color while maintaining the horizontal color arrangement order, and add the pixel signals of multiple adjacent pixels of the same color existing on the same horizontal line. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is in a drive mode for outputting the image.   The pixel addition mode adds pixel signals of pixels of the same color adjacent in the vertical direction on the odd horizontal lines adjacent to each other in the odd field, and is adjacent to each other in the horizontal direction on the same odd horizontal line. The pixel signals of the same color pixels are added and output. In the even field, the pixel signals of the same color pixels adjacent in the vertical direction on the even horizontal lines adjacent to each other are added, and on the same even horizontal line. 5. The image pickup apparatus according to claim 4, wherein the image pickup apparatus is a drive mode in which pixel signals of pixels of the same color adjacent to each other in the horizontal direction are added and output. The pixel addition mode is composed of a plurality of types of pixel addition modes in which the number of pixels to be added is different when the pixel signals of the plurality of pixels of the same color are added to the image sensor and output,
6. The imaging apparatus according to claim 1, further comprising a pixel addition mode switching unit that switches a type of the pixel addition mode when the image pickup element is driven in the pixel addition mode. The drive means has a normal drive mode for causing the image sensor to output pixel signals of all pixels without pixel addition,
The image pickup apparatus according to claim 1, further comprising drive mode switching means for switching a drive mode of the image pickup element by the drive means between the pixel addition mode and the normal drive mode. An acquisition means for acquiring the brightness of the subject;
8. The imaging apparatus according to claim 6, wherein the pixel addition mode switching unit switches the type of the pixel addition mode based on the brightness of the subject acquired by the acquisition unit. An acquisition means for acquiring the brightness of the subject;
The drive mode switching means switches the drive mode of the image sensor by the drive means between the pixel addition mode and the normal drive mode based on the brightness of the subject acquired by the acquisition means. The imaging device according to claim 7. An imaging method of an imaging apparatus including an imaging element that images a subject,
A driving mode for driving the image sensor is a pixel addition mode in which the image sensor is driven in a plurality of fields, and pixel signals of a plurality of pixels of the same color adjacent to each other are separated and output for each field. The imaging method characterized by setting to. In a computer included in an image pickup apparatus including an image pickup device that picks up an image of a subject,
A driving mode for driving the image sensor is a pixel addition mode in which the image sensor is driven in a plurality of fields, and pixel signals of a plurality of pixels of the same color adjacent to each other are separated and output for each field. A program to let you set.

Images