JP2015104102A - Photographing apparatus and pixel signal processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem that there is no photographing apparatus suitable for suppressing deterioration of a reference black level signal caused by random noise in the past.SOLUTION: A photographing apparatus includes; an imaging device having an effective pixel region and an OPB pixel region; conversion means for A/D converting an OPB pixel signal outputted from a pixel in the OPB pixel region a plurality of times; addition means for adding the A/D converted OPB pixel signals for the plurality of times; and a reference black level signal acquisition means for calculating an average level of the OPB pixel signals added by the addition means and acquiring a reference black level signal for correcting a black level of an effective pixel signal outputted from a pixel in the effective pixel region on the basis of the calculated average level.

Description

  The present invention relates to a photographing apparatus for correcting a black level of a pixel signal of an effective pixel using a pixel signal output from an optical black (hereinafter referred to as “OPB”) pixel, and a black level of the pixel signal of the effective pixel. The present invention relates to a pixel signal processing method to be corrected.

  An imaging device such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor is mounted on the photographing apparatus. The output of each pixel of the image sensor potentially includes fixed pattern noise that is unavoidable.

  In order to remove this type of fixed pattern noise, a configuration in which OPB pixels are arranged outside the effective pixel region of the image sensor is known. In such a configuration, the pixel signal output from the OPB pixel is acquired as the reference black level signal, and the black level of the pixel signal of the effective pixel is corrected using the acquired reference black level signal.

  However, the number of OPB pixels that can be arranged is limited in order to achieve an increase in the size of the image sensor and an increase in the area of the effective pixel region. Even if the pixel signals of a small number of OPB pixels are averaged to obtain the reference black level signal, the influence of random noise included in the reference black level signal cannot be sufficiently suppressed. Therefore, Patent Documents 1 and 2 describe specific configuration examples that can suppress deterioration of the reference black level signal due to random noise.

  For example, random noise generated in the amplification transistor of the pixel is inversely proportional to the square root of the product of the capacitance, gate width, and gate length of the amplification transistor. In the imaging apparatus described in Patent Document 1, in order to suppress random noise included in the pixel signal of the OPB pixel, the capacity, gate width, and gate length of the amplification transistor of the OPB pixel are made larger than those of the effective pixel.

  The imaging apparatus described in Patent Document 2 detects and includes noise (including random noise) included in the pixel signal of the OPB pixel by reading the OPB pixel line a plurality of times and artificially increasing the OPB pixel line. The correction accuracy is improved.

JP 2013-145905 A JP 2006-25146 A

  However, in the configuration described in Patent Document 1, it is necessary not only to apply a light-shielding film to pixels assigned as OPB pixels, but also to change the structure of the pixels themselves with respect to the pixels assigned as effective pixels. Further, there is a drawback that it is difficult to increase the gate width and gate length as the pixel density increases as the number of pixels increases. The configuration described in Patent Document 2 requires a noise detection / correction circuit that detects noise and corrects the detected noise. Moreover, since it is necessary to read the OPB pixel line a plurality of times, there is a problem that the processing takes time.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a photographing apparatus and a pixel signal processing method suitable for suppressing deterioration of a reference black level signal due to random noise. .

  The imaging apparatus of the present embodiment includes an imaging device having an effective pixel area and an OPB pixel area, conversion means for A / D converting the OPB pixel signal output from the pixels in the OPB pixel area, and A / D conversion. An addition means for adding OPB pixel signals for a plurality of times, an average level of the OPB pixel signal added by the addition means, and an effective pixel output from a pixel in the effective pixel area based on the calculated average level Reference black level signal acquisition means for acquiring a reference black level signal for correcting the black level of the signal.

  The photographing apparatus may include a black level correction unit that corrects the black level of the effective pixel signal based on the reference black level signal acquired by the reference black level signal acquisition unit.

  The photographing apparatus may include a determination unit that determines whether or not a predetermined photographing condition is satisfied. In this configuration, when the predetermined photographing condition is not satisfied, the conversion unit A / D converts the OPB pixel signal only once. Further, the reference black level signal acquisition means acquires a reference black level signal based on the A / D converted one-time OPB pixel signal.

  The predetermined shooting conditions are, for example, the ISO sensitivity set in the image sensor exceeds a predetermined sensitivity, the temperature of the image sensor exceeds a predetermined temperature, the exposure time of the image sensor exceeds a predetermined time, At least one of the following.

  The conversion means may be configured to change the number of times the A / D conversion of the OPB pixel signal is performed according to at least one of the ISO sensitivity set in the image sensor, the temperature of the image sensor, and the exposure time of the image sensor.

  The pixel signal processing method according to the present embodiment is a method for processing a pixel signal output from an imaging device having an effective pixel region and an OPB pixel region. The OPB pixel signal output from a pixel in the OPB pixel region is A multiple times. A conversion step for / D conversion, an addition step for adding a plurality of A / D converted OPB pixel signals, and calculating an average level of the OPB pixel signals added in the addition step, and calculating the calculated average level And a reference black level signal acquisition step of acquiring a reference black level signal for correcting the black level of the effective pixel signal output from the pixel in the effective pixel region.

  According to the present embodiment, a photographing apparatus and a pixel signal processing method suitable for suppressing deterioration of a reference black level signal due to random noise are provided.

It is a block diagram which shows the structure of the imaging device of embodiment of this invention. It is a figure which shows schematically the structure of the imaging surface of the image pick-up element with which the imaging device of embodiment of this invention is equipped. It is a figure which shows the image generation flow by the imaging device of embodiment of this invention.

  Hereinafter, a photographing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following, a digital single lens reflex camera will be described as an embodiment of the present invention. Note that the photographing device is not limited to a digital single lens reflex camera, but includes, for example, a mirrorless single lens camera, a compact digital camera, a camcorder, a tablet terminal, a PHS (Personal Handy phone System), a smartphone, a feature phone, a portable game machine, and the like. It may be replaced with another form of device having

[Configuration of the photographing apparatus 1]
FIG. 1 is a block diagram showing a configuration of the photographing apparatus 1 of the present embodiment. As shown in FIG. 1, the photographing apparatus 1 includes a photographing lens system 10 (photographing lenses 11 and 12). A diaphragm 13 is disposed between the photographing lens 11 and the photographing lens 12. A mirror 14 is disposed behind the taking lens system 10. The mirror 14 is a half mirror, and is disposed in a posture in which the half mirror surface is about 45 ° with respect to the optical axis AX of the photographing lens system 10.

  A luminous flux from the subject (subject luminous flux) passes through the taking lens system 10 and enters the mirror 14. Behind the mirror 14, a focal plane shutter 15 and an image sensor 16 are arranged in this order from the mirror 14 side. Above the mirror 14, a diffusion plate (focus plate or focus plate) 18 and a pentaprism 17 are arranged in this order from the mirror 14 side.

  Part of the subject luminous flux incident on the mirror 14 is reflected by the mirror 14 and incident on the pentaprism 17 via the diffusion plate 18. The diffusion plate 18 is disposed at a position equivalent to the imaging surface of the imaging element 16. Therefore, the subject light flux that has passed through the photographic lens system 10 forms an image on the diffusion plate 18. The pentaprism 17 has a plurality of reflection surfaces, and forms an erect image by reflecting an object image formed and incident on the diffusion plate 18 by each reflection surface, and emits the image toward the eyepiece lens 19. The eyepiece 19 re-images the subject image formed on the diffusion plate 18 and erected by the pentaprism 17 into a virtual image suitable for the photographer's observation. Thereby, the photographer can observe the subject image by looking into the eyepiece lens 19.

  The operation member 32 includes various switches necessary for the user to operate the photographing apparatus 1, such as a power switch, a release switch, a photographing mode switch, and a zoom switch. When the user presses the power switch, power is supplied from the battery (not shown) to the various circuits of the photographing apparatus 1 through the power line. After supplying power, a CPU (Central Processing Unit) 31 accesses a predetermined memory area (not shown), reads out a control program, loads it into the work area, and executes the loaded control program, whereby the entire photographing apparatus 1 is executed. Control.

  The CPU 31 drives and controls the aperture 13 via the aperture drive circuit 22 so that an appropriate exposure is obtained based on a photometric value measured by a photometric sensor 26 such as a TTL (Through The Lens) exposure meter. The state display device 33 (for example, LCD (Liquid Crystal Display)) displays the shooting mode, the appropriate exposure time at that time, the F value, and the like.

  When the release switch is pressed halfway, the CPU 31 determines the position and positional relationship between the photographing lens 11 and the photographing lens 12 on the optical axis AX via the lens control circuit 21 based on the detection result of the AF (Autofocus) sensor 25. Control. Thereby, the focus state of the photographic lens system 10 is adjusted. Next, when the release switch is fully pressed, the CPU 31 controls the drive of the focal plane shutter 15 via the shutter drive circuit 24 and causes the mirror 14 to return quickly. That is, the CPU 31 raises the mirror 14 via the mirror driving circuit 23 only during the period immediately before the front film travel of the focal plane shutter 15 is started and immediately after the rear curtain travel, so that the optical axis AX of the photographing lens system 10 is The mirror 14 is retracted from the parallel optical path.

  The subject luminous flux that has passed through the photographic lens system 10 is imaged on the imaging surface of the imaging element 16 while the focal plane shutter 15 is open. The imaging element 16 is, for example, a CCD image sensor or a CMOS image sensor, and accumulates an optical image formed by each pixel on the imaging surface as a charge corresponding to the amount of light. The image sensor 16 converts the accumulated charge into a voltage (herein referred to as “pixel signal”) by a floating diffusion amplifier, and outputs the converted pixel signal to the A / D conversion circuit 27. The A / D conversion circuit 27 performs A / D conversion on the input pixel signal and outputs it to a DSP (Digital Signal Processor) 41.

  The DSP 41 controls the charge accumulation operation and the pixel signal readout operation of the image sensor 16 and performs predetermined signal processing on the pixel signal input from the A / D conversion circuit 27. Specifically, the DSP 41 performs predetermined signal processing such as color interpolation, matrix calculation, and Y / C separation on the pixel signal input from the A / D conversion circuit 27 to perform luminance signal Y, color difference signal Cb, Cr is generated and compressed in a predetermined format such as JPEG (Joint Photographic Experts Group). For example, the buffer memory 42 is used as a temporary storage location for processing data when the DSP 41 executes processing.

  A memory card 50 is detachably loaded in the card slot of the card interface 43. The DSP 41 can communicate with the memory card 50 via the card interface 43. The DSP 41 stores the generated compressed image signal (photographed image data) in the memory card 50 (or a built-in memory (not shown) provided in the photographing apparatus 1).

  Further, the DSP 41 performs predetermined signal processing on the signal after Y / C separation, and buffers it in a frame memory (not shown) in units of frames. The DSP 41 sweeps the buffered signal from each frame memory at a predetermined timing, converts it into a video signal of a predetermined format, and outputs it to the LCD control circuit 45 via the monitor interface 44. The LCD control circuit 45 modulates the liquid crystal based on the captured image data input from the DSP 41 and controls the backlight 47 to emit light. Thereby, the photographed image of the subject is displayed on the display screen of the LCD 46. The user can view a real-time through image with a proper brightness photographed with a proper focus through the display screen of the LCD 46.

[Pixel arrangement on the imaging surface]
FIG. 2 is a diagram schematically illustrating the configuration of the imaging surface of the imaging device 16. As shown in FIG. 2, the pixel area on the imaging surface of the image sensor 16 is roughly divided into an effective pixel area 16A in which effective pixels are arranged and an OPB pixel area 16B in which OPB pixels are arranged. The OPB pixel is obtained by coating a pixel having the same structure as the effective pixel arranged in the effective pixel region 16A with a light shielding film (for example, an aluminum light shielding film), and is completely shielded from external light. The OPB pixel area 16B is arranged outside the effective pixel area so as not to appear as a black spot in the captured image. In this embodiment, the OPB pixel region 16B is a region that surrounds the four sides of the rectangular effective pixel region 16A. However, in another embodiment, only one side, only two sides, or three sides of the effective pixel region 16A. It may be a region that touches only.

[Image generation flow]
The black level of the pixel signal of the effective pixel (hereinafter referred to as “effective pixel signal”) is corrected using a pixel signal output from the OPB pixel (hereinafter referred to as “OPB pixel signal”). The details of the image generation flow including the black level correction process will be described below. FIG. 3 shows an image generation flow by the photographing apparatus 1 of the present embodiment. The image generation flow shown in FIG. 3 is started when the release switch is fully pressed.

[S11 in FIG. 3 (Start of Exposure)]
In this processing step S11, exposure of the image sensor 16 is started under the set conditions. Conditions to be set include, for example, exposure time, F value, ISO sensitivity, and the like. These conditions are set automatically according to the AE (Automatic Exposure) program or according to the operation of the operation member 32 by the user.

[S12 in FIG. 3 (End of Exposure)]
In this process step S12, the exposure of the image sensor 16 under the set conditions is terminated.

[S13 in FIG. 3 (Determination of Shooting Conditions)]
In this processing step S13, it is determined whether or not a predetermined photographing condition is satisfied. The imaging conditions determined here include, for example, at least one of the following three conditions.
The ISO sensitivity set for the image sensor 16 exceeds the predetermined sensitivity.
-The temperature of the image sensor 16 exceeds a predetermined temperature.
The exposure time of the image sensor 16 exceeds a predetermined time.
The temperature of the image sensor 16 is detected based on the output level of the temperature sensor 28 attached to the substrate of the image sensor 16.

[S14 in FIG. 3 (multiple A / D conversions)]
Random noise increases as the ISO sensitivity of the image sensor 16 increases, the temperature of the image sensor 16 increases, or the exposure time of the image sensor 16 increases. For example, when the ISO sensitivity of the image sensor 16 is doubled, the random noise is also increased about twice.

  This processing step S14 is performed when it is determined in processing step S13 (determination of photographing conditions) in FIG. 3 that a predetermined photographing condition is satisfied (YES in processing step S13, for example, ISO sensitivity of 1600 or higher). To be executed. Specifically, in this processing step S14, the A / D conversion circuit 27 performs A / D conversion a plurality of times for each OPB pixel signal input from the image sensor 16, and then performs A for each effective pixel signal. / D conversion is performed once. The OPB pixel signals for each round subjected to A / D conversion a plurality of times are transferred to the buffer memory 42 via the DSP 41, and are added and buffered for each OPB pixel. Each effective pixel signal that has been A / D converted once is transferred to the buffer memory 42 via the DSP 41 and buffered.

[S15 in FIG. 3 (Acquisition of Reference Black Level Signal)]
In this processing step S15, a reference black level signal for correcting the black level of the effective pixel signal is acquired. Specifically, an average value (hereinafter, referred to as “average OPB pixel signal”) is calculated for each OPB pixel signal added and buffered. One representative value selected from all the calculated average OPB pixel signals or an average value of the average OPB pixel signals (a value obtained by adding and averaging all the average OPB pixel signals) is used as the reference black level signal. To be acquired.

  Here, in the A / D conversion circuit 27, a certain amount of rms (root mean square) noise (input conversion noise) is internally generated due to resistance noise and kT / C noise. The distribution of input conversion noise when the same DC level is A / D converted a plurality of times follows a normal distribution.

  Consider a case where the output levels of all OPB pixel signals are defined as X. In this case, the distribution of the OPB pixel signal after A / D conversion shows a normal distribution with an average X and a variance V due to the influence of noise during A / D conversion. Further, consider that A / D conversion is performed twice for each OPB pixel signal and added. If the average and variance of the first time are defined as X1 and V1, respectively, and the average and variance of the second time are defined as X2 and V2, respectively, the average and variance of the normal distribution when the first and second distributions are added are These are X1 + X2 and V1 + V2 due to their properties (reproducibility). Considering X1 = X2 = X and V1 = V2 = V, a normal distribution with an average of 2X and a variance of 2V is obtained.

  In general, a value obtained by taking the square root of the variance (standard deviation) is expressed as noise. Therefore, the noise when the first and second normal distributions are added is √ (2V), and the noise when averaged (divided by 2) is 1 / √ (2V). On the other hand, the output level becomes X by averaging (division by 2). That is, when the distributions are added and averaged, the output level of the OPB pixel signal does not change before and after (1 time), while the noise decreases (1 / √ (2) times). Noise decreases as the addition is repeated n times (1 / √ (n) times). Regarding the reproducibility of the normal distribution, for example, Non-Patent Document 1 (searched on November 13, 2013, Internet <URL: http://www43.atwiki.jp/actuary-seminar?cmd=upload&act=open&pageid= 13 & file = reproductive_property.pdf>).

  As described above, the random noise increases as the ISO sensitivity of the image sensor 16 increases, the temperature of the image sensor 16 increases, or the exposure time of the image sensor 16 increases. Therefore, the number of times A / D conversion is performed on the same OPB pixel signal is such that at least one condition adversely affects noise (the higher the ISO sensitivity, the higher the temperature of the image sensor 16, the longer the exposure time). You may increase it).

[S16 in FIG. 3 (Normal A / D Conversion)]
When it is determined in the processing step S13 (determination of shooting conditions) in FIG. 3 that the predetermined shooting conditions are not satisfied (S13: NO), it is considered that there is not much random noise. In this case, in this processing step S16, the A / D conversion circuit 27 performs A / D conversion on the pixel signal of each pixel (OPB pixel / effective pixel) input from the image sensor 16 in order to reduce the processing load. Do it only once. Each pixel signal subjected to A / D conversion once is transferred to the buffer memory 42 via the DSP 41 and buffered.

[S17 in FIG. 3 (Acquisition of Reference Black Level Signal)]
In this processing step S17, a reference black level signal for correcting the black level of the effective pixel signal is acquired. Specifically, one representative value selected from all OPB pixel signals buffered in the buffer memory 42 or an average value of OPB pixel signals (a value obtained by adding and averaging all OPB pixel signals) Is obtained as a reference black level signal.

  Note that the user can set whether or not to execute the processing step S13 (determination of imaging conditions) in FIG. When the execution of the determination process is set to OFF, the process step S13 (determination of the photographing condition) is not executed after the process step S12 (exposure end), and the process step S16 (normal A / D) is executed. (Conversion) and subsequent steps are sequentially executed.

[S18 in FIG. 3 (correction of effective pixel signal)]
In this processing step S18, the black level of the effective pixel signal is corrected based on the reference black level signal acquired in the processing step S15 or S17 (acquisition of the reference black level signal) in FIG.

[S19 in FIG. 3 (Generation and Storage of Captured Image Data)]
In this processing step S19, the DSP 41 generates photographic image data based on the effective pixel signal after the black level correction, and the generated photographic image data is stored in the memory card 50 (or a built-in memory (not shown) provided in the photographic device 1). Save to. Thereby, the image generation flow of FIG. 3 is completed.

  As described above, in the image generation flow of FIG. 3, A / D conversion is performed a plurality of times for the same OPB pixel signal, and the outputs of each time are added and averaged to be included in each OPB pixel signal. Noise is reduced. Therefore, a highly accurate reference black level signal is acquired. In addition, since the A / D conversion of each OPB pixel signal is only performed a plurality of times with respect to the conventional processing, an increase in processing time is suppressed as much as possible, and noise detection and detection noise correction are performed. No correction circuit is required. Further, it is not necessary to change the structure of the OPB pixel and the effective pixel itself. Since there is no need to increase the gate width and gate length of the OPB pixel, the disadvantage of increasing the pixel density accompanying the increase in the number of pixels can be avoided.

  The above is the description of the exemplary embodiments of the present invention. Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention. For example, the embodiment of the present application also includes an embodiment that is exemplarily specified in the specification or a combination of obvious embodiments and the like as appropriate.

  In the present embodiment, the image sensor 16, the A / D conversion circuit 27, and the buffer memory 42 are configured as separate elements. However, in another embodiment, the A / D conversion circuit 27 and the buffer memory 42 are configured as an image sensor. 16 may be mounted.

DESCRIPTION OF SYMBOLS 1 Shooting device 10 Shooting lens system 11, 12 Shooting lens 13 Diaphragm 14 Mirror 15 Focal plane shutter 16 Imaging element 17 Penta prism 18 Diffusion plate 19 Eyepiece 21 Lens control circuit 22 Diaphragm drive circuit 23 Mirror drive circuit 24 Shutter drive circuit 25 AF Sensor 26 Photometric sensor 27 A / D conversion circuit 28 Temperature sensor 31 CPU
32 Operation member 33 Status display device 41 DSP
42 Buffer memory 43 Card interface 44 Monitor interface 45 LCD control circuit 46 LCD
47 Backlight 50 Memory card

Claims (8)

An image sensor having an effective pixel area and an optical black (hereinafter referred to as “OPB”) pixel area;
Conversion means for A / D converting the OPB pixel signal output from the pixels in the OPB pixel region a plurality of times;
Adding means for adding a plurality of A / D converted OPB pixel signals;
A reference black level for calculating the average level of the OPB pixel signal added by the adding means and correcting the black level of the effective pixel signal output from the pixel in the effective pixel area based on the calculated average level. Reference black level signal acquisition means for acquiring a signal;
Comprising
Shooting device. Black level correction means for correcting the black level of the effective pixel signal based on the reference black level signal;
The imaging device according to claim 1. A determination means for determining whether or not a predetermined photographing condition is satisfied;
If the predetermined shooting condition is not satisfied,
The converting means includes
A / D-convert the OPB pixel signal only once,
The reference black level signal acquisition means includes
Obtaining the reference black level signal based on a single OPB pixel signal subjected to A / D conversion;
The imaging device according to claim 1 or 2. The predetermined shooting conditions are:
ISO sensitivity set for the image sensor exceeds a predetermined sensitivity,
The temperature of the image sensor exceeds a predetermined temperature,
The exposure time of the image sensor exceeds a predetermined time,
Including at least one of
The imaging device according to claim 3. The converting means includes
Changing the number of A / D conversions of the OPB pixel signal according to at least one of the ISO sensitivity set for the image sensor, the temperature of the image sensor, and the exposure time of the image sensor;
The imaging device according to any one of claims 1 to 4. In a pixel signal processing method for processing a pixel signal output from an imaging device having an effective pixel region and an OPB pixel region,
A conversion step of A / D converting the OPB pixel signal output from the pixels in the OPB pixel region a plurality of times;
An addition step of adding a plurality of A / D converted OPB pixel signals;
An average level of the OPB pixel signal added in the adding step is calculated, and a reference black for correcting the black level of the effective pixel signal output from the pixel in the effective pixel area based on the calculated average level. A reference black level signal acquisition step for acquiring a level signal;
including,
Pixel signal processing method. A black level correction step of correcting a black level of the effective pixel signal based on the reference black level signal,
The pixel signal processing method according to claim 6. A determination step of determining whether or not a predetermined shooting condition is satisfied,
If the predetermined shooting condition is not satisfied,
In the conversion step,
The OPB pixel signal is A / D converted only once,
In the reference black level signal acquisition step,
The reference black level signal is acquired based on a single OPB pixel signal subjected to A / D conversion.
The pixel signal processing method according to claim 6 or 7.

Images