JP2007027845A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of eliminating black-level offsets between fields to obtain a beautiful image. <P>SOLUTION: An imaging sensor 303 comprises an imaging area 31; a front OB 323 having a photoelectric conversion element structure; a rear OB 324 having no photoelectric conversion elements; and a dummy pixel 33 formed at a horizontal transmission 303H. Black-level clamping is executed, based on the output signal of the rear OB 324. A black-level signal is taken out of the front and rear OB's 323, 324 and the dummy pixel 33 each. For example, when temperature rises in the imaging sensor 303, the difference between the output signal of the rear OB 324 and that of the front OB 323 is added to an effective pixel output signal and the black level is corrected. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus including an image pickup sensor such as a CCD (charge coupled device) image sensor, and more particularly to an image pickup apparatus that can accurately adjust the black level of image data acquired by the image pickup sensor.

  In recent years, in an imaging apparatus such as a digital camera, an increase in the number of pixels exceeding several million pixels has progressed, and an increase in the density of imaging sensors mounted on such an imaging apparatus has been invited. Due to such high density, the sensitivity per pixel decreases, and the amplification factor for the output of the image sensor tends to increase correspondingly. In addition, the multi-field readout method has become the mainstream in high-pixel imaging sensors, but the transfer capacity of the vertical transfer unit becomes smaller due to the higher density, so the number of readout fields tends to increase to ensure a dynamic range. It is in.

  By the way, the imaging sensor may generate a dark current due to its own temperature rise or the like (in response to the temperature rise). Therefore, for example, when a multi-field readout method of about 5 fields is adopted, the dark current occupying the output of the imaging sensor due to heat generation of the imaging sensor during the readout between the first readout field and the fifth readout field. The ratio of will be different.

  Conventionally, in order to eliminate the influence of the dark current, an optical black part (optical black) formed by applying an optical light shielding structure to a part of a photoelectric conversion element part included in an image sensor is provided and imaged. A method is known in which a sensor output is obtained from a pixel portion of an area and an output (light-shielding output) is obtained from the optical black portion, and a black level is fixed (clamped) to a light-shielding output reference. According to this method, even if the dark current level between fields is different in multi-field readout, if the black level is clamped for each field pixel signal in the optical black portion of each field, the dark level is increased. The black level offset based on the current level difference can be suppressed to a range that does not substantially cause a problem.

  On the other hand, if a part of the photoelectric conversion element part is an optical black part, the photoelectric conversion element structure has recently been changed due to a problem that improper clamping is performed when a pixel defect exists in the optical black part. Imaging sensors equipped with optical black parts that are not provided are becoming mainstream. In such an imaging sensor, the dark current level often differs between the imaging area and the optical black portion due to factors such as temperature change of the imaging sensor and exposure time. In addition to this, in the case of multi-field readout, the dark current level difference between the above-described fields is superimposed, so that the black level offset between the fields may become apparent. For example, when a long exposure is performed with a high sensitivity setting in multi-field readout, if a clamping operation is performed based only on output information of an optical black portion that does not have a photoelectric conversion element structure, horizontal streak noise is generated. There's a problem.

Japanese Patent Application Laid-Open No. 2004-26883 discloses an imaging sensor configured to divide an imaging area of an imaging sensor into two parts, read image signals from the respective areas, and combine them to form a single image. There is disclosed a technique for eliminating a black level offset between imaging regions by providing a dummy pixel in the horizontal transfer section and performing a clamping operation based on the dummy pixel signal. However, when the multi-field readout is performed in the imaging apparatus of Patent Document 1, the clamp operation is performed based on the same dummy pixel signal in any field. The problem cannot be solved.
JP 2002-300477 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an imaging apparatus capable of accurately clamping a black level. In particular, in an imaging apparatus that employs an imaging sensor equipped with an optical black portion that does not have a photoelectric conversion element structure and that reads an output from the imaging sensor using a multi-field readout method, the black level offset between fields is reduced. An object of the present invention is to provide an imaging apparatus that can eliminate (correctly correct) and obtain a beautiful image.

  An imaging apparatus according to a first aspect of the present invention is based on an imaging sensor having a photoelectric conversion element section that generates a signal charge according to the amount of received light, a predetermined optical black section, and output information from the optical black section. An imaging apparatus comprising image processing means capable of performing predetermined image processing, wherein the optical black portion includes a first optical black portion having at least a first output characteristic, and the first output. A second optical black portion having a second output characteristic different from the characteristics, and the image processing means is capable of acquiring output information from both the first optical black portion and the second optical black portion. It is characterized by being.

  According to this configuration, the image processing unit can acquire output information from both the first optical black portion and the second optical black portion having different output characteristics. Therefore, for example, at least two types of information related to the black level of the image data can be acquired, and these can be appropriately reflected in predetermined image processing in the image processing means.

  An imaging apparatus according to a second aspect of the present invention includes a photoelectric conversion element section that generates a signal charge according to the amount of received light, a predetermined optical black section, and a simulated black level pixel section that can generate output information equivalent to a black level. And image processing means capable of acquiring output information from both the optical black portion and the simulated black level pixel portion and performing predetermined image processing based on the output information. It is characterized by.

  According to this configuration, the image processing means can acquire output information from both the optical black portion and the simulated black level pixel portion capable of generating output information in accordance with the black level. Therefore, for example, at least two types of information related to the black level of the image data can be acquired, and these can be appropriately reflected in predetermined image processing in the image processing means.

  According to a third aspect of the present invention, there is provided an imaging sensor including a photoelectric conversion element unit that generates a signal charge according to the amount of received light, a predetermined optical black unit, and a dummy pixel unit formed in a horizontal transfer unit. And an image processing means capable of performing predetermined image processing based on output information from the optical black portion and the dummy pixel portion, wherein the optical black portion is a previous optical black portion And the rear optical black portion, wherein the image processing means can acquire output information from the dummy pixel portion and / or the front optical black portion and the rear optical black portion.

  According to this configuration, the image processing unit can acquire output information from the dummy pixel portion and / or the front optical black portion and the rear optical black portion. Therefore, for example, at least two types of information related to the black level of the image data (dummy pixel portion and rear optical black portion, or front optical black portion and rear optical black portion) or three types can be acquired. As appropriate, it can be reflected in predetermined image processing in the image processing means.

  In the above configuration, it is desirable that either the front optical black portion or the rear optical black portion is an optical black portion having a photoelectric conversion element structure. The optical black part with a photoelectric conversion element structure is an optical black part formed by applying an optical shading structure to a part of the photoelectric conversion element part that constitutes the imaging area. Is equivalent to the imaging area. Therefore, if the output information from the optical black part having the photoelectric conversion element structure is referred to (for example, if the output difference with respect to the output information from the optical black part not having the photoelectric conversion element structure is acquired), the imaging sensor Appropriate image processing (black level adjustment, etc.) can be performed on image data acquired during a temperature rise or long exposure.

  In the image processing means, the first output information and / or second output information obtained from the dummy pixel portion and / or the front optical black portion, and the third output information obtained from the rear optical black portion, It is desirable that the information can be used as information for black level adjustment of image data. In an imaging sensor that is widely used at present, the rear optical black portion is an optical black portion that does not have a photoelectric conversion element structure. The front optical black portion is an optical black portion having a photoelectric conversion element structure. Furthermore, the dummy pixel unit is a pixel unit provided in a horizontal transfer unit in which light does not enter even under adverse conditions such as backlight. In such a configuration of the image sensor, the first output information obtained from the dummy pixel portion or the previous optical black portion is based on the third output information obtained from the post optical black portion in which an error due to a pixel defect does not occur. By using together the second output information obtained from the above, or by using both the first and second output information together, for example, black level adjustment (black level correction) between fields in multi-field readout can be performed accurately. become.

  In this case, the image sensor has temperature detection means capable of acquiring temperature information of the imaging sensor, and the image processing means has the first output information to the third output information based on the temperature information given from the temperature detection means. Either or a plurality of them can be selected and used as information for adjusting the black level of the image data. According to this configuration, in the image processing unit, according to the temperature state of the imaging sensor, using appropriate information from the first output information to the third output information, or using a difference between these information, The black level of the image data is adjusted.

  Alternatively, exposure time control means for controlling an exposure time of the image sensor is provided, and the image processing means is configured to output the first output information to the third output based on the exposure time information set by the exposure time control means. Any one or a plurality of pieces of information can be selected and used as information for adjusting the black level of the image data. According to this configuration, in the image processing unit, according to the exposure time information set by the exposure time control unit, using appropriate information from the first output information to the third output information, or the information The black level of the image data is adjusted using the difference.

  The image processing unit further includes sensitivity setting means for setting sensitivity of the imaging sensor, and the image processing means is any one of the first output information to the third output information based on sensitivity information set by the sensitivity setting means. Alternatively, a plurality can be selected and used as information for adjusting the black level of the image data. According to this configuration, in the image processing means, according to the sensitivity information set by the sensitivity setting means, using appropriate information from the first output information to the third output information, or the difference between these information Is used to adjust the black level of the image data.

  In the configurations of claims 3 and 4, the clamping operation for the output information from the photoelectric conversion element portion is configured to be performed based on the output information from the optical black portion having no photoelectric conversion element structure. (Claim 9). According to this configuration, since the clamping operation is performed based on the output information from the optical black portion that does not have the photoelectric conversion element structure, the black level of the image data can be clamped without being affected by pixel defects.

  The reference output value of the optical black portion whose output information is used for the clamping operation and the optical black portion or the dummy pixel portion which is not used for the clamping operation in the configuration of any one of claims 3, 4 and 9. Preferably, the image processing means uses the difference as offset correction information. According to this configuration, it is possible to adjust the black level of the image data based on the difference information stored in the storage means at the time of factory shipment or the like.

  In the configuration according to any one of claims 1 to 10, it is preferable that output information of the imaging sensor is read by a multiple field readout method (claim 11). According to this configuration, offset correction between fields in multi-field reading can be performed.

  Further, in the configuration of claim 5, the output information of the imaging sensor is read by a multiple field readout method, and any one or a plurality of the first output information to the third output information is between fields. It is desirable to use it as information for black level adjustment (claim 12). According to this configuration, it is possible to correct the black level offset between fields in multi-field reading using the first output information to the third output information or the difference information thereof.

  According to the first aspect of the present invention, output information can be acquired from both the first optical black portion and the second optical black portion having different output characteristics and reflected in image processing. Appropriate image processing can be performed accordingly.

  According to the second aspect of the present invention, the output information can be acquired from both the optical black portion and the simulated black level pixel portion capable of generating output information according to the black level and reflected in the image processing. Appropriate image processing according to the imaging state can be performed.

  According to the third aspect of the present invention, output information can be acquired from the dummy pixel portion and / or the front optical black portion and the rear optical black portion and reflected in the image processing. Appropriate image processing can be performed.

  According to the fourth aspect of the present invention, appropriate image processing (black level adjustment, etc.) is performed on image data acquired when the temperature of the image sensor rises, when exposed for a long time (high sensitivity shooting), or the like. Because it can, you can get a beautiful image.

  According to the fifth aspect of the present invention, it is possible to acquire information on a plurality of types of black levels by using a configuration of an imaging sensor that is currently widely used, and image processing (image processing ( By performing black level adjustment etc., a beautiful image can be obtained.

  According to the sixth aspect of the invention, the black level of the image data can be accurately adjusted according to the temperature state of the image sensor, and a beautiful image can be obtained regardless of the temperature change.

  According to the seventh aspect of the invention, the black level of the image data can be accurately adjusted according to the exposure time, and a beautiful image can be obtained regardless of the length of the exposure time.

  According to the eighth aspect of the invention, the black level of the image data can be accurately adjusted according to the set sensitivity, and a beautiful image can be obtained regardless of the degree of sensitivity setting.

  According to the ninth aspect of the present invention, the clamping operation can be performed by using the configuration of the general-purpose imaging sensor as it is, and the complexity of the supply of the timing pulse can be avoided, so that the cost can be reduced. it can.

  According to the tenth aspect of the present invention, it is possible to accurately adjust the black level of image data.

  According to the eleventh aspect of the present invention, offset correction between fields can be accurately performed in multi-field readout, so that a clear image can be obtained in an imaging apparatus employing a multi-field readout scheme.

  According to the twelfth aspect of the present invention, since the black level offset between fields in multi-field readout can be corrected, a clear image that does not cause streak noise or the like is obtained in an imaging apparatus that employs the multi-field readout scheme. be able to.

Hereinafter, an embodiment of an imaging apparatus according to the present invention will be described in detail using a digital camera as an example.
(Explanation of camera structure)
1A and 1B are external views of a digital camera 1 according to an embodiment of the imaging apparatus of the present invention, in which FIG. 1A is a front view of the digital camera, FIG. 1B is a top view, and FIG. And (d) show rear views, respectively.

  In these drawings, a photographing lens 200 is provided on the front side of the camera body 100, and an electronic viewfinder window 402 disposed on the upper side of the camera body 100 and a lower side of the electronic viewfinder window 402. A display monitor 304 such as an LCD (Liquid Crystal Display) is provided. In the playback mode (PLAY mode), the electronic viewfinder window 402 and the display monitor 304 play back and display a recorded image recorded on a built-in recording medium. In the recording mode (REC mode), the video is displayed while waiting for shooting. An electronic image (live view image) of the photographed subject is displayed. In these displays, either the electronic viewfinder window 402 or the display monitor 304 can be selected and displayed.

  On the upper surface side of the camera body 100, a shutter button 101 for instructing shooting, a shooting mode switching switch 102 for switching between the playback mode (PLAY mode) and the recording mode (REC mode), and an electronic A monitor enlargement switch 103 for enlarging and displaying an electronic image to be displayed on the viewfinder window 402 and the display monitor 304, and a sensitivity changeover switch 104 for setting the sensitivity of the image sensor 303 described later are provided.

  The shutter button 101 is a push-down switch that can be operated in a “half-pressed state” that is pressed halfway and further operated in a “full-pressed state”. When the shutter button 101 is half-pressed (S1) in the still image shooting mode, a preparatory operation for capturing a still image of the subject (preparation operations such as setting of exposure control values and focus adjustment) is executed, and the shutter button 101 is operated. When is fully pressed (S2), a photographing operation (a series of operations for exposing the imaging sensor, performing predetermined image processing on the image signal obtained by the exposure, and recording the image signal on a memory card or the like) is performed. The half-pressing operation of the shutter button 101 is detected by turning on a switch S1 (not shown), and the full pressing operation of the shutter button 101 is detected by turning on a switch S2 (not shown).

  On the rear side of the camera body 100, there is provided a main switch 105 including a slide switch that also functions as a power ON / OFF and a display changeover switch between an electronic viewfinder and a monitor. Further, on the right side of the main switch 105, a switch switch 106 that functions as a switch for moving a recorded image in the playback mode and a zoom switch for the photographing lens 200 is disposed.

  FIG. 2 is a cross-sectional view showing a schematic structure of the digital camera 1. The digital camera 1 according to the present embodiment basically includes a box-shaped camera body 100, a photographing lens 200 provided in the camera body 100, and an imaging unit 300 provided on the back side inside the camera body 100. And an electronic viewfinder 400 disposed on the upper side inside the camera body 100, and a temperature sensor 41 (temperature detection means).

  The photographic lens 200 is for taking reflected light (incident light) A from a subject under a light source (not shown) into an imaging surface in the camera body 100, and is arranged on the front side of the camera body 100. The photographic lens 200 includes a photographic optical system 201 composed of a plurality of lens groups, and an optical diaphragm 202 that is interposed in the photographic optical system 201 and regulates the amount of incident light. The photographic optical system 201 and the optical diaphragm 202 are mirrors. The cylinder 203 is held at a predetermined position.

  The camera body 100 forms a dark box for capturing a subject light image with the photographing lens 200, photoelectrically converts the subject light image into an image signal with a photoelectric conversion element, and performs predetermined signal processing on the image signal to perform image processing. It is recorded on a recording medium such as a memory or a memory card, and the recorded image is reproduced. Further, as described above, the display monitor 304 such as an LCD is provided on the back side of the camera body 100.

  An imaging unit 300 is built in the camera body 100. The imaging unit 300 includes an imaging sensor 303 having a photoelectric conversion element unit that generates a signal charge according to the amount of received light and a predetermined optical black unit. Contains. The image sensor 303 is disposed on the optical axis B of the taking lens 200 and in the vicinity of the back surface of the camera body 100.

  As this imaging sensor 303, for example, a plurality of photoelectric conversion elements composed of photodiodes or the like are two-dimensionally arranged in a matrix, and the light receiving surface of each photoelectric conversion element has different spectral characteristics, for example, R (red), G A Bayer array CCD (Charge Coupled Device) color area sensor in which (green) and B (blue) color filters are arranged in a ratio of 1: 2: 1 can be used. As such a CCD color area sensor, a progressive scan (sequential scan) type CCD employing an interline transfer method, an interlace scan (interlaced scan) type CCD used in combination with a mechanical shutter, or the like can be used. The imaging sensor 303 converts the light image of the subject formed by the photographing optical system 201 into analog electrical signals (image signals) of R (red), G (green), and B (blue) color components, and R , G, B image signals are output. A signal obtained by the image sensor 303 is read out by a multi-field reading method (described later based on FIG. 5). Note that an optical low-pass filter 305 having a predetermined thickness for removing moire from the image sensor 303 is disposed on the front surface of the image sensor 303.

  In addition to the above-described photoelectric conversion element unit, the image sensor 303 includes an optical black unit for obtaining output information of a plurality of types of black levels and a dummy pixel unit formed in the horizontal transfer unit. . The optical black portion is provided in the peripheral portion of the imaging area in the imaging sensor 303, and the dummy pixel portion is provided in the horizontal transfer portion of the imaging sensor 303. The optical black portion and the dummy pixel portion will be described in detail later.

  The electronic viewfinder 400 guides the subject light image to the eyepiece so that the image actually captured from the eyepiece can be confirmed. The user can confirm the prism 401 and the formed subject image. An electronic viewfinder window 402 serving as an eyepiece for performing the operation, a transmission-type small liquid crystal monitor 403 disposed below the prism 401, and a light source 404 for illuminating the small liquid crystal monitor 403. The small liquid crystal monitor 403 can display an image (live view image) captured by video, and the prism 401 is configured to reflect the display image of the small liquid crystal monitor 403 and guide it to the electronic viewfinder window 402.

  The temperature sensor 41 is for detecting the internal temperature of the camera body 100, particularly the temperature of the image sensor 303. The temperature information of the image sensor 303 detected by the temperature sensor 41 is used as information for adjusting the black level of the image data. If the image sensor 303 is provided with a temperature detection terminal, the output of the temperature detection terminal may be used instead of using the temperature sensor 41.

(Description of the electrical configuration of the digital camera)
Next, an electrical configuration of the digital camera 1 according to the present embodiment will be described. FIG. 3 is a block diagram of an imaging process performed by the digital camera 1 according to the present embodiment. In the figure, the same members as those shown in FIGS. 1 and 2 are denoted by the same reference numerals. The digital camera 1 includes a camera control unit (a camera control CPU (Central Processing Unit) 500, an operation unit 501, a shutter / aperture driver 502, a zoom / focus motor driver 503, a timing generator 504, an AFE (analog front end) 505, an image. A processing unit (image processing CPU) 600, an image memory 700, and a memory card 800 are provided.

  The camera control unit 500 includes a CPU (Central Processing Unit) and the like, and centrally controls the shooting operation of the digital camera 1. That is, the camera control unit 500 controls the shutter / aperture driver 502, the zoom / focus motor driver 503, and the timing generator 504 on the basis of the instruction signal given from the operation unit 501, and causes the AFE 505 and the image processing unit 600 to perform shooting. Then, the display monitor 304 and the small liquid crystal monitor 403 are controlled to perform predetermined image processing on the photographed image, and then the photographed image is recorded on the memory card 800 or displayed on the display monitor 304 or the small liquid crystal monitor 403. I will let you.

  Further, the camera control unit 500 extracts an image signal included in a predetermined photometry area set in advance on the screen from the image signal captured by the image sensor 303 during shooting standby in the electronic finder mode, and the image signal Is used to calculate the exposure control value at the time of shooting, and using this calculation result and a preset program diagram, the aperture value of the optical aperture 202 (see FIG. 2) and the charge accumulation time of the image sensor 303 ( Exposure time (shutter speed)).

  In addition, the camera control unit 500 outputs from the optical black unit and dummy pixel unit (simulated black level pixel unit) having different output characteristics in accordance with the temperature information, exposure time information, or setting sensitivity information of the image sensor 303. Using the information, control is performed to perform appropriate black level correction on the image data. That is, the camera control unit 500 configures an image processing unit in cooperation with an image processing unit 600 described later, and executes processing for adjusting the black level of image data. This will be described in detail later with reference to FIGS.

  The operation unit 501 is a switch for inputting operation information of various operation buttons provided on the camera body 100 to the camera control unit 500. The operation unit 501 includes the shutter button 101, the shooting mode changeover switch 102, the main switch 105, the switch corresponding to the push switch group 106, and the like.

  The shutter / aperture driver 502 controls the driving of the optical aperture 202 in the photographic lens 200. The shutter / aperture driver 502 sets the aperture amount of the optical aperture 202 to a predetermined aperture amount based on the aperture value input from the camera control unit 500. It also performs shutter drive control.

  The zoom / focus motor driver 503 controls the zoom for adjusting the focal length of the photographing optical system 201 of the photographing lens 200 and the driving of the focus motor for driving the focus lens. The zoom / focus motor driver 503 controls the drive operation of the zoom / focus motor based on the focal length setting signal input from the camera control unit 500.

  The timing generator 504 generates timing pulses for controlling the photographing operation of the image sensor 303 (charge accumulation based on exposure, reading of accumulated charge, etc.), the operation of the AFE 505, and the like. For example, the timing generator 504 generates a predetermined timing pulse (vertical transfer pulse, horizontal transfer pulse, charge sweeping pulse, etc.) based on the imaging control signal from the camera control unit 500 and outputs it to the imaging sensor 303 for live view. For example, a frame image is captured every 1/30 (seconds) and sequentially output to the AFE 505. As a result, at the time of shooting, charges are accumulated in conjunction with the exposure operation of the image sensor 303 (that is, the subject light image is photoelectrically converted into an image signal), and the accumulated charges are converted into voltage and output to the AFE 505. Then, after predetermined signal processing is performed by the AFE 505 and the image processing unit 600, the signal is recorded on the memory card 800. On the other hand, each frame image in shooting standby is displayed on the small liquid crystal monitor 403 if it is in the electronic finder mode after predetermined image processing is performed by the AFE 505 and the image processing unit 600.

  In addition, the timing generator 504 generates and transmits a sampling signal for noise removal, an OB clamp timing signal for clamping the output signal (OB potential) of the optical black portion, and the like to the AFE 505. Furthermore, the imaging sensor according to the present embodiment uses an imaging sensor 303 having an optical black portion and a dummy pixel portion having different output characteristics, as will be described later with reference to FIG. A black level detection timing signal for detecting each signal output from the part as a black level signal is generated.

  The AFE 505 performs predetermined signal processing on the image signal output from the image sensor 303 (analog signal group received by each pixel of the CCD area sensor), converts the image signal into a digital signal, and outputs the digital signal. The AFE 505 includes a CDS circuit (correlated double sampling circuit) 506, an AGC circuit 507, and an A / D conversion circuit 508. The AFE 505 includes other functional units, which will be described later with reference to FIG.

  The CDS circuit 506 reduces reset noise included in the analog image signal. That is, the reference voltage and the signal voltage detected immediately after the reset operation by the reset gate provided in the image sensor 303 are sampled, and the difference between the two is obtained to obtain the signal voltage. The AGC circuit 507 corrects the level of the analog image signal. The A / D conversion circuit 508 converts an analog image signal into, for example, a 14-bit digital image signal (image data).

The image processing unit 600 performs predetermined signal processing on the image data input from the AFE 505 to create an image file, reproduces and displays the digital image after the signal processing on the display monitor 304, and displays it on the memory card 800. Controls recording. The image data captured by the digital image processing unit 600 is temporarily written in the image memory 700 in synchronization with the reading of the image sensor 303. Thereafter, the image data written in the image memory 700 is accessed to access the digital image. Processing is performed in each block of the processing unit 600. The image processing unit 600 includes a black level correction unit 601, a pixel interpolation unit 602, a resolution conversion unit 603, a white balance control unit 604, a shading correction unit 605, a gamma correction unit 606, an image compression / decompression unit 607, and a video encoder 608. A memory card driver 609 is provided.

  The black level correction unit 601 corrects the black level of each R, G, B digital image signal input after A / D conversion by the AFE 505 (A / D conversion circuit 508) to a reference black level. It is. Specifically, the black level correction unit 601 corrects the black level (dark signal level) of the digital image signal input from the AFE 505 to a reference value (for example, 0 at the digital signal level after A / D conversion). Therefore, when the image signal level of the digital image signal input from the AFE 505 is SD1, and the dark signal level of the digital image signal is SD2, the calculation of SD1-SD2 is performed.

  As described in detail later, the black level correction unit 601 according to the present embodiment is characterized in that the dark signal level SD2 is appropriately adjusted according to the temperature, exposure time, set sensitivity, or the like of the image sensor 303. For example, when the output signal of the image sensor 303 is read out by a multi-field readout method, the dark signal level SD2 is adjusted between fields, and the optimum dark signal level SD2 is given to each field image output, It functions to remove the black level offset between fields and prevent the occurrence of horizontal streak noise. As a black level correction method by the black level correction unit 601, a method of performing correction for each readout line of the image sensor 303 or an average value of black level correction values for one entire readout field is calculated and the readout is performed. A method of performing uniform correction using the average value for field image output can be employed.

  The pixel interpolation unit 602 interpolates data of pixel positions that are insufficient in the frame image for each of R, G, and B color components. The pixel interpolation unit 602 masks image data constituting the frame image with a predetermined filter pattern for a G color component frame image having pixels up to a high band, and then uses a median (intermediate value) filter. Of the pixel data that actually exists around the pixel position to be interpolated, an average value of the pixel data from which the maximum value and the minimum value are removed is calculated, and the average value is interpolated as pixel data at the pixel position. For the R and B color components, after the image data constituting the frame image is masked with a predetermined filter pattern, the average value of the pixel data actually existing around the pixel position to be interpolated is calculated, and the average value is calculated. Is interpolated as pixel data at the pixel position.

  The resolution conversion unit 603 converts the resolution to the set number of recorded image pixels. In other words, the resolution of the image data after pixel interpolation is converted to a predetermined number of recorded image pixels by performing compression or thinning in the horizontal or vertical direction. In addition, when generating an image for monitor display, the resolution conversion unit 603 thins out horizontal pixels, and the like, for display on the LCD display monitor 304 or the small liquid crystal monitor 403 of the electronic viewfinder 400. A low resolution image is created.

  The white balance control unit 604 performs level correction on the image data of the R and B color components based on the WB adjustment data set by the camera control unit 500, respectively, so that the white balance of the pixel-interpolated image data ( WB) for adjustment. That is, the white balance control unit 604 specifies a portion that is originally estimated to be white from the luminance, saturation data, and the like in the photographic subject, the average of the R, G, and B color components of the portion, and G / The R ratio and the G / B ratio are obtained, and the levels are corrected as R and B correction gains.

  The shading correction unit 605 performs signal processing for removing image density (brightness) unevenness. Specifically, the shading correction unit 605 performs a process for compensating for a decrease in the amount of light and the like by changing a gain (amplification factor) for each pixel output of the imaging sensor 303 according to unevenness.

  The gamma correction unit 606 corrects the gradation characteristics of the WB-adjusted image data to the gradation characteristics of the display monitor 304 or an externally output monitor television. That is, the gamma correction unit 606 performs nonlinear conversion and offset adjustment for the level of image data for each color component using a predetermined gamma characteristic. The converted / adjusted image data is stored in the image memory 700.

  The image compression / decompression unit 607 compresses image data constituting a captured image to be recorded on the memory card 800, or displays an image to be reproduced and displayed from the memory card 800 on the display monitor 304 or the small liquid crystal monitor 403 for electronic viewfinder. The image data to be configured is expanded. The reason why the compression processing is performed at the time of recording is to increase the recording capacity of the memory card 800 by reducing the number of captured image data to be recorded on the memory card 800.

  When the display monitor 304 and the small liquid crystal monitor 403 are driven based on the NTSC or PAL video signal, the video encoder 608 converts the image data to be displayed on the display monitor 304 and the small liquid crystal monitor 403 to the NTSC or PAL system. Is output to the display monitor 304 and the small liquid crystal monitor 403.

  The memory card driver 609 performs processing for writing the image data compressed by the image compression / decompression unit 607 to the memory card 800.

  The image memory 700 is a memory that temporarily stores the image data in order to perform predetermined digital image processing on the image data. The image memory 700 has a capacity capable of storing at least three images, and the captured image is stored in a state of being separated into R, G, and B color components. The memory card 800 is a memory for recording and storing the image data compressed as described above, and is a medium that can externally store the image data by exchanging it.

  In such a configuration, when previewing in the recording mode, the frame image of each frame after gamma correction (for example, a low resolution image of 640 × 240 pixels) stored in the image memory 700 is read to the video encoder 608. It is converted into an NTSC system or PAL system image signal and output as a field image to the display monitor 304 or the small liquid crystal monitor 403. At the time of recording the captured image, the image data of the captured image after gamma correction is read from the image memory 700, and the image data is compressed to the recording resolution set by the image compression / decompression unit 607, for example, using the JPEG method. Is done. The compressed image data is recorded on the memory card 800 via the memory card driver 609. At the time of recording the image, a screen nail image (VGA; graphic image of 640 × 480 pixels) for reproduction display is created by linking with the image having the designated resolution described above, and also recorded on the memory card 800. It is desirable to keep it.

  Further, when the captured image is reproduced, the captured image (compressed image) recorded on the memory card 800 is read out to the image compression / decompression unit 607 via the memory card driver 609 and decompressed to the original image size. After that, the video encoder 608 converts the image signal into an NTSC or PAL image signal and outputs it to the display monitor 304 or the small liquid crystal monitor 403. In this case, it is preferable to display the above-described screen nail image (VGA) because a relatively high-speed reproduced image can be displayed.

(Detailed explanation about black level correction)
The above is a general description of the digital camera 1. In this embodiment, each field image data read out in multiple fields according to temperature information, exposure time information, setting sensitivity information, or the like of the image sensor 303. On the other hand, it is characterized in that the optimum black level correction is performed. Hereinafter, the configuration related to this feature point will be described in detail.

  FIG. 4 is a block diagram schematically showing an electrical configuration of the image sensor 303 (interline transfer type CCD color area sensor) used in the present embodiment. The imaging sensor 303 includes an imaging area unit 3031 mounted on a semiconductor substrate 3030. The imaging area unit 3031 is arranged in a matrix and converts incident light into signal charges having a charge amount corresponding to the amount of light. A plurality of sensor units 303S (photoelectric conversion element units) to be accumulated, a readout gate unit 303G serving as a gate for reading out signal charges accumulated in the sensor unit 303S, and a readout gate provided for each vertical column of the sensor units 303S. A vertical transfer unit (vertical CCD) 303V that vertically transfers the signal charge read out through the unit 303G. A horizontal transfer unit (horizontal CCD) 303H to which signal charges corresponding to one line are sequentially transferred from the vertical transfer unit 303V is disposed below the imaging area unit 3031.

  The imaging sensor 303 has a vertical overflow drain structure, and the semiconductor substrate 3030 is provided with an overflow drain terminal 3051, a vertical transfer pulse input terminal 3052, and a horizontal transfer pulse input terminal 3053. Here, the timing generator 504 supplies a charge sweeping pulse to the overflow drain terminal 3051 to sweep out excess signal charges accumulated in the sensor unit 303S. On the other hand, a charge read pulse is applied to the read gate unit 303G to move the accumulated signal charge to the vertical transfer unit 303V.

  Further, the timing generator 504 supplies, for example, four-phase vertical transfer pulses φV1, φV2, φV3, and φV4 to the vertical transfer pulse input terminal 3052, and the signal charges transferred from the sensor unit 303S are horizontally transferred from the vertical transfer unit 303V. For example, two-phase horizontal transfer pulses φH1 and φH2 are supplied to the horizontal transfer pulse input terminal 3053 to be transferred to the transfer unit 303H, and a charge-voltage conversion unit 303T provided at the transfer destination side end of the horizontal transfer unit 303H. The signal charge is transferred to (floating diffusion portion). The signal charges that have been horizontally transferred in this way are sequentially converted into voltage signals by the charge-voltage converter 303T and sent to the AFE 505. A bias voltage Vsub can be applied to the semiconductor substrate 3030, and the saturation signal charge amount of the sensor unit 303S is determined by the voltage value of the bias voltage Vsub.

  The signal charge of the image sensor 303 is read by a multi-field reading method. FIG. 5 is a schematic diagram illustrating an example of a signal charge readout method in the imaging sensor 303. Here, a three-field readout method in which the signal charge (pixel signal) of the sensor unit 303S is transferred in three portions is illustrated. is doing. In this case, reading is performed separately in units of three pixels in the vertical direction. That is, as shown in FIG. 5A, the pixel signals of the second line, the fifth line, the eighth line,... Are read in the first field. Similarly, as shown in FIG. 5B, each pixel signal of the third line, the sixth line, the ninth line,... Is read in the second field, and as shown in FIG. In the third field, the pixel signals of the first line, the fourth line, the seventh line,... Are read out.

  Each field image read out in this way is digitally converted through the AFE 505, and then temporarily stored in the image memory 700, and the first to third field images are synthesized at the time of image recording, etc. A frame image is generated. According to the image sensor 303 of such a multi-field readout method, the amount of charge transferred by one readout (one-field readout) is reduced, and the area of the vertical transfer unit 303V can be reduced correspondingly. In addition, it is possible to increase the sensitivity by increasing the area of the sensor portion 303S.

  In the present embodiment, an image sensor 303 provided with an optical black portion for acquiring a black level signal and a dummy pixel portion is used. FIG. 6 is a schematic diagram for explaining an arrangement relationship between the optical black portion and the dummy pixel portion. As shown in FIG. 6, the imaging sensor 303 includes an imaging area 31 that is arranged in the sensor main body 303 </ b> A (substantially coincides with the imaging area unit 3031) and outputs an effective pixel signal, and a periphery of the imaging area 31. An optical black (OB) portion 32 formed in the portion and a dummy pixel portion 33 (simulated black level pixel portion) provided near the exit of the horizontal transfer portion 303H are provided.

  The OB portion 32 is provided so as to surround the peripheral portion of the imaging area 31 as shown in FIG. The OB unit 32 includes a lower vertical OB unit 321 and an upper vertical OB unit 322 in which all lines are optically shielded parts, and a front OB unit 323 configured by using both horizontal ends of each line as optically shielded parts. The rear OB unit 324 is included. Note that the output signals of the front OB unit 323 and the rear OB unit 324 are read out in a manner positioned at the beginning and end of the line output when one line is read out.

  The front OB part 323 (first optical black part) and the rear OB part 324 (second optical black part) have different OB structures. In other words, the front OB unit 323 has a structure having the sensor unit 303S shown in FIG. 4 (a structure having a photoelectric conversion element structure), whereas the rear OB unit 324 is not provided with the sensor unit 303S (photoelectric unit). A structure having no conversion element structure), and an OB portion composed of a CCD or the like. Thereafter, an output signal from the OB unit 324 is clamped as an OB potential indicating a black level potential in an OB clamp circuit 514 (see FIG. 7) described later. That is, the color level of the image data is specified by treating the signal level output from the area of the rear OB unit 324 as “black”. As described above, the output signal of the OB unit 324 that does not have a photoelectric conversion element structure is clamped due to an improper potential due to the influence of a pixel defect (whiteout or blackout of the sensor unit 303S). This is to deter.

  As described above, since the front OB portion 323 and the rear OB portion 324 have different OB structures, the respective output characteristics differ as a result. As described above, the black level of the effective pixel signal (image data) read from the imaging area 31 varies depending on the temperature state of the imaging sensor 303 and the exposure time. That is, the output signal (black level signal) from the front OB unit 323 having the photoelectric conversion element structure varies depending on the temperature state and the exposure time as in the imaging area 31 (first output characteristic).

  On the other hand, the output signal from the rear OB unit 324 that does not have a photoelectric conversion element structure has little dependency on the temperature state and exposure time of the image sensor 303 described above (second output characteristic). This causes an offset between the black level of the effective pixel signal and the clamped OB potential. That is, even if the black level of the effective pixel signal output from the imaging area 31 varies depending on the temperature state of the imaging sensor 303 and the exposure time, the output signal from the rear OB unit 324 does not vary. This is because sampling is performed at a level different from the black level. In the case of multi-field reading, such a divergence may become remarkable between fields, and as described above, it causes a horizontal stripe noise.

  On the other hand, the output signal (output information) from the front OB unit 323 having the photoelectric conversion element structure varies equivalently to the effective pixel signal, so that there is a problem of defective pixels, but the temperature state of the image sensor 303 and the exposure. Depending on the time, it can be said that it is useful for black level adjustment. Therefore, in the present embodiment, in the black level correction in the black level correction unit 601 of the image processing unit 600, the output difference between the output signal from the front OB unit 323 and the output signal from the rear OB unit 324 is taken into consideration. The black level is adjusted between the fields. For example, the output difference is added (subtracted) to the effective pixel signal (image data) clamped based on the output signal from the post-OB unit 324, that is, the above-described dark signal level SD2 is added to the output difference. It is possible to perform correction based on the adjustment to eliminate the black level offset between the fields. Alternatively, it is possible to perform black level correction using the output signal itself from the previous OB unit 323 as the dark signal level SD2.

  Next, the dummy pixel unit 33 uses a part of the CCD constituting the horizontal transfer unit 303H, and the output signal from the dummy pixel unit 33 can be treated as a simulated black level signal. That is, since the horizontal transfer unit 303H is not provided with a photoelectric conversion element structure, when the output signal (output signal before signal charge transfer) is extracted, it is treated that a signal close to the black level is always output. Because it can.

  This dummy pixel unit 33 is different from the front OB unit 323 and the rear OB unit 324 in the possibility of noise generation when strong light is incident. That is, since the front OB part 323 and the rear OB part 324 having the photoelectric conversion element structure are adjacent to the imaging area 31, they are exposed to light in the case of taking a picture in a backlight state although they are shielded from light. May end up. Further, although the rear OB unit 324 does not have a photoelectric conversion element structure, a noisy output may occur when strong light hits it.

  On the other hand, since the dummy pixel unit 33 is formed in the horizontal transfer unit 303 </ b> H, no strong light is incident on the image sensor 303. This can be said to indicate the usefulness of using the output signal of the dummy pixel unit 33 in a case where the image is taken in a backlit state. That is, when the black level adjustment is performed based on the output signals of the front OB unit 323 and the rear OB unit 324 at the time of backlight photographing, the black level may be lost due to light exposure or noise. If the output signal is used as a reference, there is no such concern.

  Therefore, in this embodiment, in the black level correction in the black level correction unit 601 of the image processing unit 600, the output difference between the output signal from the dummy pixel unit 33 and the output signal from the rear OB unit 324 is taken into consideration. The black level is adjusted between the fields. For example, when it is detected from the luminance histogram or the like that the backlight is in the backlit state, the output difference is added to the effective pixel signal (image data) clamped based on the output signal from the rear OB unit 324 ( By subtracting, that is, it is possible to perform correction for adjusting the black level by adjusting the dark signal level SD2 based on the output difference. Alternatively, the black signal can be corrected by setting the output signal itself from the dummy pixel unit 33 as the dark signal level SD2.

  Next, the configuration of a portion deeply related to black level correction will be described. FIG. 7 is a block diagram showing circuit configurations of a part of the AFE 505 and the image processing unit 600 that are deeply related to black level correction. As shown in FIG. 7, the AFE 505 is provided with a clamp unit 511 in addition to the CDS circuit 506, the AGC circuit 507, and the A / D conversion circuit 508 described above. The clamp unit 511 clamps the signal voltage output from the charge-voltage conversion unit 303T of the image sensor 303 in accordance with the OB clamp timing signal supplied from the timing generator 504. The first switch 512 (SW1) A D / A conversion circuit 513 and an OB clamp circuit 514 are included. In addition, the image processing unit 600 includes a second switch 611 (SW2) and a black level data holding unit 612.

  The first switch 512 is a switch circuit that turns on a feedback loop that extracts an output signal of the A / D conversion circuit 508 and sends it to the OB clamp circuit 514 when an OB clamp timing signal is given from the timing generator 504. The D / A conversion circuit 513 is a conversion circuit that returns the output signal of the A / D conversion circuit 508 to an analog signal again. The OB clamp circuit 514 clamps an analog signal voltage corresponding to an output signal (in this embodiment, an output signal from the rear OB unit 324) in a state where the first switch 512 is turned on, and this is used as a black level signal in the current state. As feedback to the CDS circuit 506. In this way, in the AFE 505, the process of fixing the black level based on the output signal from the rear OB unit 324 is executed at a cycle in which the OB clamp timing signal is given.

  In addition to this, a black level detection timing signal is generated from the timing generator 504, and based on this, processing for recording output signals from the front OB unit 323, the rear OB unit 324 and the dummy pixel unit 33 is performed. That is, the second switch 611 is a switch circuit that is turned on in response to the black level detection timing signal and causes the black level data holding unit 612 to capture the image data input to the image processing unit 600. The black level data holding unit 612 is made of a memory member such as a RAM, and temporarily stores data input when the second switch 611 is ON.

  The black level detection timing signal is given when the output signals from the front OB unit 323, the rear OB unit 324, and the dummy pixel unit 33 are input to the image processing unit 600. The three types of black level signals read from each part are stored. Three types of black level signals stored in the black level data holding unit 612 are appropriately used under the control of a black level correction control unit 54 described later, and the black level of the image data is adjusted by the black level correction unit 601. Is.

  FIG. 8 is a functional block diagram showing a functional configuration deeply related to the black level correction of the camera control unit 500. The camera control unit 500 is functionally provided with a clamp control unit 52, a black level recording control unit 53, a black level correction control unit 54, and a table storage unit 55 for black level correction. These functional units function when the CPU configuring the camera control unit 500 reads out a control program (firmware) stored in a ROM or the like provided in the digital camera 1 as appropriate.

  The clamp control unit 52 generates a control signal for causing the clamp unit 511 of the AFE 505 to clamp the output voltage (OB potential) of the rear OB unit 324 and supplies the control signal to the timing generator 504. The timing generator 504 receives the control signal and generates the OB clamp timing signal.

  The black level recording control unit 53 generates a control signal for causing the black level data holding unit 612 to store the above three types of black level signals by causing the second switch 611 to be turned on and off, and causing the timing generator 504 to Give. The timing generator 504 receives the control signal and generates the black level detection timing signal.

  The black level correction control unit 54 generates a control signal for causing the black level correction unit 601 of the image processing unit 600 to appropriately adjust the black level of the image data in accordance with the shooting situation and the like. A temperature information acquisition unit 541, an exposure time information acquisition unit 542, a sensitivity information acquisition unit 543, a backlight determination unit 544, and a black level correction formula setting unit 545.

  The temperature information acquisition unit 541 acquires temperature (estimation) information of the image sensor 303 from the temperature sensor 41 (temperature detection unit) arranged in the vicinity of the image sensor 303. Since the temperature information affects the dark current level of the sensor unit 303S (photoelectric conversion element unit) and the front OB unit 323 having the photoelectric conversion element structure, the temperature information is useful information in black level correction.

  The exposure time information acquisition unit 542, for example, an unillustrated automatic exposure control mechanism (AE; exposure time) that determines the exposure time of the image sensor 303 based on the exposure evaluation value calculated from the luminance of the subject acquired from the pre-photographed image, for example. The exposure time information is acquired from the control means. Since the exposure time information affects the amount of noise superimposed on the output signal of the front OB unit 323, it is useful information in black level correction. That is, when the exposure is performed for a long time, the amount of noise superimposed on the output signal of the front OB unit 323 having the photoelectric conversion element structure increases, and this output signal tends to be disadvantageous in terms of noise.

  The sensitivity information acquisition unit 543 acquires sensitivity setting information such as ISO sensitivity set by the sensitivity changeover switch 104 (sensitivity setting means; see FIG. 1B). This setting state of ISO sensitivity may also affect the exposure time and the like, and is thus acquired as information for black level correction.

  The backlight determination unit 544 determines whether or not the captured image is a backlight scene. For example, the backlight determination unit 544 divides a captured image into a plurality of regions, measures the luminance for each of the divided regions, and obtains the ratio of a bright region above a certain threshold and a dark region below the threshold to obtain a backlight scene. It is determined whether or not. The determination information by the backlight determination unit 544 can be used for selecting whether the output information of the front OB unit 323 or the output information of the dummy pixel unit 33 is used, for example.

  The black level correction formula setting unit 545 applies the black level correction unit 601 to the black level correction unit 601 based on information obtained from the temperature information acquisition unit 541, the exposure time information acquisition unit 542, the sensitivity information acquisition unit 543, and the backlight determination unit 544, respectively. Set the optimal black level correction formula for black level adjustment. That is, the optimum black level adjustment is performed on the image data by utilizing the three types of black level information stored in the black level data holding unit 612.

  This black level correction formula setting method varies depending on the characteristics of the image sensor 303 and is not uniquely determined, but can be performed based on the following conditions (a) to (d), for example. Here, output information from the dummy pixel unit 33 is first output information, output information from the front OB unit 323 is second output information, and output information from the rear OB unit 324 is third output information. It is assumed that the clamping operation is performed by the AFE 505 based on the third output information.

(A) When the temperature information of the image sensor 303 acquired by the temperature information acquisition unit 541 determines that the temperature of the image sensor 303 is rising (or lower) than the predetermined temperature range, The dark current level in the OB portion 324 is likely to be different from the dark current level in the imaging area 31, while the front OB portion 323 having a photoelectric conversion element structure is likely to be equivalent to the dark current level in the imaging area 31. Therefore, in this case, the second output information is used. For example, the black difference that the output difference between the third output information and the second output information (this output difference information is stored in the black level data holding unit 612) can be added to the effective pixel output in the black level correction unit 601. Set the level correction formula. On the other hand, when the temperature of the image sensor 303 is within a predetermined steady range, the third output information is used.

(B) When the exposure time acquired by the exposure time information acquisition unit 542 is longer than the predetermined time, the output signal of the front OB unit 323 becomes disadvantageous in terms of noise due to the long exposure of the imaging area 31, so that In the case of exposure, the third output information is used. On the other hand, in the case of short-time exposure, the first output information and / or the second output information is used as necessary in consideration of other conditions.

(C) When the set sensitivity acquired by the sensitivity information acquisition unit 543 is relatively low (for example, sensitivity equivalent to ISO 100), the shutter speed is relatively long and can be handled in the same manner as in the case of long exposure. Uses the third output information. On the other hand, when the set sensitivity is relatively high (for example, sensitivity equivalent to ISO 400), the shutter speed is relatively short and can be handled substantially the same as in the case of short-time exposure. In this case, the first output information and Alternatively, the second output information is used.

(D) If the backlight determining unit 544 determines that the scene is a backlight scene, the front OB unit 323 and the rear OB unit 324 may be exposed and the black level may be disturbed. In this case, there is no risk of exposure. The first output information from the dummy pixel unit 33 is used.

  Based on the above conditions, the black level correction formula setting unit 545 selects one or more of the first to third output information according to the shooting situation, and adjusts the black level according to the shooting situation. The optimal black level correction formula is set. As described above, a black level correction formula that adds the output difference between the third output information from the post-OB unit 324 used for clamping and the first and second output information may be used. A black level correction formula using a predetermined function or the like may be used. For example, the S / N ratio between fields can be compensated for the black level value X obtained by adding the output difference in consideration of the black level average value of one field and its standard deviation (S / N ratio). A coefficient a other than 1 may be multiplied (a · X). Alternatively, a predetermined offset correction value b other than 0 may be provided (X + b or a · X + b).

  The table storage unit 55 stores a reference output value difference (output difference) between the post-OB unit 324 whose output information is used for the clamping operation and the front-OB unit 323 or the dummy pixel unit 33 not used for the clamping operation. To do. Such difference data is stored in advance in the table storage unit 55 when the digital camera 1 is shipped from the factory. The difference data stored in the table storage unit 55 includes, for example, offset correction information (an initial reference output value difference when detecting an output difference between the third output information and the first and second output information). Used as correction information for deduction). This makes it possible to accurately adjust the black level of the image data.

(Description of digital camera operation)
Next, the operation of the digital camera 1 according to the present embodiment will be described. FIG. 9 is a flowchart showing a series of imaging processes of the digital camera 1. As shown in FIG. 9, when the power of the digital camera 1 is turned on (YES in step S1), the camera control unit 500 (see FIG. 3) determines whether or not the shooting mode is set (step S2). ). When the reproduction mode is set (NO in step S2), the camera control unit 500 performs a process of reproducing and displaying an image stored in the memory card 800 or the like on the display monitor 304 (step S3). On the other hand, when the shooting mode is set (YES in step S2), an image obtained by the imaging operation of the imaging sensor 303 is displayed as a live view image on the display monitor 304 or the small liquid crystal monitor 403 (step S4).

  Next, the camera control unit 500 determines whether or not the half-pressing operation (S1: ON) of the shutter button 101 has been performed (step S5). If the half-pressing operation has not been performed, the half-pressing operation is performed. Is waited for (NO at step S5). When the shutter button 101 is half-pressed (YES in step S5), the camera control unit 500 performs AE control for determining an exposure control value (shutter speed and aperture value) based on the luminance of the subject. At the same time (step S6), for example, AF control for adjusting the focus by the phase difference detection method is performed (step S7).

  When the AE control and the AF control are completed, the camera control unit 500 determines the operation state of the shutter button 101. That is, it is determined whether or not the half-press operation of the shutter button 101 has been released (step S8). If released (YES in step S8), the process returns to step S5. On the other hand, if not released (NO in step S8), it is determined whether or not the shutter button 101 is fully pressed (S2: ON) (step S9), and the shutter button 101 is fully pressed. If not (NO in step S9), the process returns to step S8.

  When the shutter button 101 is fully pressed (YES in step S9), the camera control unit 500 opens the mechanical shutter or the electronic shutter for a predetermined time, and uses the image sensor 303 with the exposure control value set in step S6. Performs an imaging operation (exposure operation) (step S10). Then, the camera control unit 500 starts reading out signal charges (image transfer) via the timing generator 504 (step S11).

  The read signal charge is subjected to charge-voltage conversion and input to the AFE 505. The AFE 505 performs analog signal processing and A / D conversion processing (step S12). In the AFE 505, a clamp process for fixing the black level is executed by the clamp unit 511 (see FIG. 7).

  The digital image data after A / D conversion is transferred to the image processing unit 600. This digital image data is temporarily stored in the image memory 700. Then, under the control of the black level correction control unit 54, the black level adjustment of the image data is performed by the black level correction unit 601 (step S13). Subsequently, the camera control unit 500 causes the pixel interpolation unit 602, the resolution conversion unit 603, the white balance control unit 604, the shading correction unit 605, and the gamma correction unit 606 to execute predetermined image processing (step S14).

  Thereafter, the image compression / decompression unit 607 performs compression processing on the image processed image data and records the image on the memory card 800 (step S15). Then, the camera control unit 500 repeats the processes from step S2 to S15 until the power of the digital camera 1 is turned off (NO in step S16). When the power is turned off (YES in step S16), the process is performed. End. The above is a series of imaging processing by the digital camera 1.

  Next, operations related to black level correction will be described. FIG. 10 is a flowchart showing the operation related to the black level correction in step S13, FIG. 11 is a flowchart showing the OB clamping operation, and FIG. 12 is a time chart showing the detection timing of the black level signal used in the black level correction. is there.

  As shown in FIG. 10, first, under the control of the clamp control unit 52 (see FIG. 8), the black level is set based on the output signal (OB potential) of the rear OB unit 324 by the clamp unit 511 (see FIG. 7) of the AFE 505. An OB clamping operation to be fixed is executed (step S21). The details of the OB clamp operation will be described with reference to FIG. 11. The clamp control unit 52 determines whether or not it is the OB clamp timing (step S211), and waits if it is not the OB clamp timing (NO in step S211). If it is OB clamp timing (YES in step S211), an OB clamp timing signal is given to the first switch 512 via the timing generator 504, and the first switch 512 is turned ON (step S212).

  The OB clamp timing signal is given in synchronization with the timing at which the output signal of the rear OB unit 324 is output from the A / D conversion circuit 508. As a result, the output signal of the rear OB unit 324 is extracted and taken into the clamp unit 511 (step S213). Then, the extracted output signal of the OB unit 324 is returned to an analog signal by the D / A conversion circuit 513 (step S214). The analog signal is input to the OB clamp circuit 514. The OB clamp circuit 514 clamps the analog signal voltage (step S215), and feeds it back to the CDS circuit 506 as a current black level signal (step S216). Such black level clamping is executed in a cycle in which the OB clamp timing signal is given.

Returning to FIG. 10, in parallel with the above-described black level clamping processing, the black level recording control unit 53 executes black level signal detection processing for black level correction. Referring to the time chart of FIG. 12, when an arbitrary n-th pixel line is used as a reference, a black level detection timing signal (sampling signal) is output from the timing generator 504 at time t 1 and output from the rear OB unit 324. The signal is output to the second switch 611 at the timing when the signal is output, whereby the output signal d _n−1 (third output information) of the OB unit 324 after the n−1 line is detected (step S22). That is, the output signal dn ₋₁ is recorded in the black level data holding unit 612. The time t1 is synchronized with the timing at which the OB clamp timing signal is given to the first switch 512.

Similarly, at time t2 the output signal a _n from the dummy pixel unit 33 is output, the black level detection timing signal is provided to the second switch 611, the output signal a _{n (first} output information) Black It is recorded in the level data holding unit 612 (step S23). Further, at the timing of time t3 when the output signal b _n from the previous OB unit 323 is output, the black level detection timing signal is given to the second switch 611, whereby the output signal b _n of the n-line front OB unit 323 is output. (Second output information) is recorded in the black level data holding unit 612 (step S24). Then, the black level value X output difference obtained by adding the output signal _{a n} or output signal _{b n} for the output signal _{d n-1} is also recorded to the black level data holding unit 612 (step S25).

From time t4 when the effective pixel output signal C _n is output on the _n- th line, the black level correction control unit 54 performs an operation for setting an arithmetic expression for black level correction (step S26). Here, as described above, the temperature information of the image sensor 303, the exposure time information, based on whether or not the sensitivity information and backlight, etc., the output signal d _n-1, the output signal a _n and the output signal b _n Either or a plurality of these are selected and a correction formula is set. That is, depending on the shooting conditions, etc., the output signal d _n-1, or used as the black level correction as one of the output signals a _n and the output signal b _n, the black level value X obtained by adding the output difference Or a correction equation (a · X + b) using the coefficient a and the offset correction value b is set. For example, when performing the black level correction for each line, using the black level correction formula is set as described above, black level adjustment with respect to the effective pixel output signal C _n of the n lines by the black level correction unit 601 Is performed (step S27).

After that, at the time t5 when the output of the effective pixel output signal C _n ends, the OB clamp timing signal and the black level detection timing signal are changed to the first switch 512 and the second switch for clamping the n + 1 line and correcting the black level. 611 (return to step S21 and step S22), the same operation is repeated. That is, the output signal _{d n} of the OB portion 324 after the clamping operation and n lines are recorded in the black level data holding unit 612 at time t5, also the output signal _{a n + 1} from the dummy pixel unit 33 in the n + 1 line at time t6, At time t7, the output signal b _{n + 1} of the previous OB unit 323 in the n + 1 line is recorded in the black level data holding unit 612, respectively. Then, the black level adjustment is performed on the effective pixel output signal C _{n + 1 of} the n + 1 line.

  FIG. 13 is a flowchart showing an example of the black level correction formula setting process in step S26. In setting the black level correction formula, various information of the digital camera 1 is acquired by the black level correction control unit 54. That is, temperature information of the image sensor 303 is acquired by the temperature information acquisition unit 541 (step S31), exposure time information is acquired by the exposure time information acquisition unit 542 (step S32), and setting sensitivity information is acquired by the sensitivity information acquisition unit 543. Is acquired (step S33).

  Then, the black level correction formula setting unit 545 performs predetermined matrix determination based on the acquired various information. Here, it is determined whether or not the set sensitivity is equivalent to ISO 100 and the shutter speed is 1 second or more (step S34), and whether or not the temperature t of the image sensor 303 is within a range of 30 ° <t <60 ° ( In the example shown in FIG.

  If “YES” in the step S34, it means that the exposure is long time, so the black level correction formula setting unit 545 does not have a photoelectric element conversion structure that is advantageous in terms of noise, and outputs the output signal of the rear OB unit 324 to the black level. It is determined to use for level correction (step S36). If “YES” in step S35, the temperature of the imaging sensor 303 is within the steady range, and there is not much divergence between the dark current level of the rear OB unit 324 and the dark current level of the imaging area 31. Similarly, it is determined to use the output signal of the rear OB unit 324 (step S36).

  On the other hand, if “NO” in step S34 and “NO” in step S35, use of the output signal of the rear OB unit 324 causes problems such as dark current level deviation. The level correction formula setting unit 545 determines that the output signal of the post-OB unit 324 is not used.

  Then, the determination result of whether or not it is a backlight scene by the backlight determination unit 544 is acquired (step S37). If it is a backlight scene (YES in step S37), the output of the dummy pixel unit 33 that is not likely to be exposed to light. A determination using the signal is made (step S38). On the other hand, if it is not a backlight scene (NO in step S37), a decision is made to use the output signal of the front OB unit 323 having a photoelectric conversion element structure equivalent to the dark current level of the imaging area 31 (step S39). Here, “use the output signal” of the dummy pixel unit 33 or the front OB unit 323 refers to, for example, the effective pixel output signal clamped based on the output signal of the rear OB unit 324 with respect to the rear OB unit 324. This means that the output difference between the output signal and the output signal of the dummy pixel unit 33 or the previous OB unit 323 is added. Note that the temperature range, ISO sensitivity, and shutter speed are examples, and may be set as appropriate according to the characteristics of the image sensor 303 and the like.

  According to the digital camera 1 described above, output information obtained from the dummy pixel unit 33 or obtained from the previous OB unit 323 is obtained based on output information obtained from the OB unit 324 after no error due to a pixel defect occurs. By using the output information together, black level adjustment (black level correction) between fields in multi-field readout can be performed accurately. That is, when the first to third field images are sequentially read out as shown in FIG. 5, even if the temperature of the image sensor 303 rises during the reading period, an accurate value corresponding to the temperature in each field is obtained. Since the black level correction is performed, no horizontal stripe noise occurs even if the first to third field images are combined.

  The embodiments of the present invention have been described above. However, such embodiments can be accompanied by additions and changes of various configurations without departing from the spirit of the present invention. For example, the following modified embodiments can be taken.

(1) In the above embodiment, three types of black level signals are acquired from the three parts of the rear OB unit 324 having the photoelectric conversion element structure, the front OB unit 323 not having the photoelectric conversion element structure, and the dummy pixel unit 33. As an example, a configuration in which only two types of black level signals are acquired from two parts of the rear OB unit 324 and the front OB unit 323 or from two parts of the rear OB unit 324 and the dummy pixel unit 33 is described. Also good. Further, the output signals of the lower vertical OB unit 321 and the upper vertical OB unit 322 may be used as appropriate.

(2) Further, the OB portion having the photoelectric conversion element structure may be the rear OB portion 324 instead of the front OB portion 323. Further, the output signal of the rear OB unit 324 that does not have a photoelectric conversion element structure may be used for black level clamping in the AFE 505.

(3) An example of setting the black level correction formula is illustrated in FIG. 13, but the most suitable correction condition varies depending on the type and characteristics of the image sensor 303, and unique setting may be difficult. Therefore, black level correction data using various conditions (temperature of the image sensor, exposure time, setting sensitivity, etc.) as parameters are stored in a standard table and stored in advance in a ROM or the like provided in the digital camera 1. The black level correction formula setting unit 545 may be configured to set the black level correction formula with reference to the standard table.

(4) In the present embodiment, the case of three-field reading has been illustrated, but the digital camera 1 adopting the two-field reading or the four-field reading method may be used. Of course, the present invention can also be applied to a digital camera 1 that does not employ multi-field readout.

(5) In the above embodiment, the digital camera 1 has been described as an example of the imaging device of the present invention. However, a digital video camera using various imaging sensors such as a CCD and a CMOS, a sensing device including an imaging unit, and the like. It can also be applied to. In the case of a CMOS sensor, a dummy pixel portion cannot be provided. In this case, black level correction may be performed based on the outputs of the front OB and the rear OB.

BRIEF DESCRIPTION OF THE DRAWINGS It is an external view of the digital camera concerning embodiment of this invention, Comprising: The same figure (a) is a front view of the said digital camera, (b) is a top view, (c) is a side view, (d) is a rear view. Respectively. It is sectional drawing which shows schematic structure of the digital camera which concerns on embodiment of this invention. It is an imaging processing block diagram by the digital camera which concerns on embodiment of this invention. It is a block diagram which shows schematic structure of the image sensor used by embodiment of this invention. It is a schematic diagram which shows an example of the read-out method of the signal charge in an imaging sensor. It is a schematic diagram for demonstrating the arrangement | positioning relationship of the optical black part and dummy pixel part of an imaging sensor. It is a block diagram which shows the circuit structure of a part of AFE and image processing part which is a part deeply related to black level correction. It is the functional block diagram which showed the functional structure deeply related to black level correction | amendment of a camera control part. It is a flowchart for demonstrating a series of imaging processes of the digital camera concerning embodiment of this invention. It is a flowchart for demonstrating the operation | movement relevant to black level correction | amendment. It is a flowchart for demonstrating OB clamp operation | movement. It is a time chart which shows the detection timing of the black level signal used by black level correction. It is a flowchart which shows an example of a black level correction type | formula setting process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 303 Imaging sensor 31 Imaging area (photoelectric conversion element part)
32 Optical black part 323 Front optical black part (front OB part / 1st optical black part)
324 Rear optical black part (rear OB part / second optical black part)
33 Dummy pixel part (simulated black level pixel part)
303H Horizontal transfer part 41 Temperature sensor (temperature detection means)
500 Camera control unit (part of image processing means)
504 Timing generator 505 Analog front end (AFE)
511 Clamping section 52 Clamp control section 53 Black level recording control section 54 Black level correction control section 600 Image processing section (part of image processing means)
601 Black level correction unit 612 Black level data holding unit

Claims (12)

An imaging sensor having a photoelectric conversion element section that generates a signal charge according to the amount of received light, and a predetermined optical black section;
An image processing apparatus comprising: image processing means capable of performing predetermined image processing based on output information from the optical black portion;
As the optical black part, a first optical black part having at least a first output characteristic and a second optical black part having a second output characteristic different from the first output characteristic are provided,
The imaging apparatus, wherein the image processing means is capable of acquiring output information from both the first optical black portion and the second optical black portion. An imaging sensor having a photoelectric conversion element unit that generates a signal charge according to the amount of received light, a predetermined optical black unit, and a simulated black level pixel unit capable of generating output information according to a black level;
An imaging apparatus comprising: image processing means capable of acquiring output information from both the optical black portion and the simulated black level pixel portion and performing predetermined image processing based on the output information. An imaging sensor including a photoelectric conversion element unit that generates a signal charge according to the amount of received light, a predetermined optical black unit, and a dummy pixel unit formed in a horizontal transfer unit;
An image processing apparatus comprising image processing means capable of performing predetermined image processing based on output information from the optical black portion and the dummy pixel portion,
The optical black part includes a front optical black part and a rear optical black part,
The imaging apparatus, wherein the image processing means is capable of acquiring output information from the dummy pixel portion and / or the front optical black portion and the rear optical black portion.   The imaging apparatus according to claim 3, wherein either the front optical black portion or the rear optical black portion is an optical black portion having a photoelectric conversion element structure.   In the image processing means, first output information and / or second output information obtained from the dummy pixel portion and / or front optical black portion, and third output information obtained from the rear optical black portion are image data. The imaging apparatus according to claim 3, wherein the imaging apparatus is usable as information for black level adjustment. Having temperature detection means capable of acquiring temperature information of the imaging sensor;
The image processing means selects one or more of the first output information to the third output information based on temperature information given from the temperature detection means, and uses it as information for adjusting the black level of the image data. The imaging apparatus according to claim 5, wherein the imaging apparatus is used. Exposure time control means for controlling the exposure time of the image sensor;
The image processing means selects one or more of the first output information to the third output information based on the exposure time information set by the exposure time control means, and adjusts the black level of the image data. The imaging apparatus according to claim 5, wherein the imaging apparatus is used as information on the image. Sensitivity setting means for setting the sensitivity of the imaging sensor;
The image processing unit selects any one or a plurality of the first output information to the third output information based on the sensitivity information set by the sensitivity setting unit, and information for adjusting the black level of the image data The imaging apparatus according to claim 5, wherein the imaging apparatus is used as an imaging apparatus.   5. The clamp operation for output information from the photoelectric conversion element unit is configured to be performed based on output information from an optical black unit that does not have a photoelectric conversion element structure. The imaging device described in 1. Storage means for storing a difference in reference output value between an optical black portion whose output information is used for a clamping operation and an optical black portion or a dummy pixel portion which is not used for a clamping operation;
The imaging apparatus according to claim 3, wherein the image processing unit uses the difference as offset correction information.   The imaging apparatus according to claim 1, wherein output information of the imaging sensor is read out by a multiple field readout method.   Output information of the image sensor is read by a multiple field readout method, and any one or a plurality of the first output information to the third output information is used as information for black level adjustment between fields. The imaging apparatus according to claim 5, wherein:

Images