JP2018160740A - Signal processing circuit, imaging device, and signal processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a signal processing circuit capable of suppressing occurrence of horizontal stripes.SOLUTION: A signal processing circuit inputs frame data including first line data having a plurality of pieces of first optical black pixel data and a plurality of pieces of first effective pixel data and second line data having a plurality of pieces of second optical black pixel data and a plurality of pieces of second effective pixel data includes first black level estimating means (201 to 203) that estimate a black level of the first line data based on the plurality of pieces of first optical black pixel data of the first line data and second black level estimating means (205 and 206) that estimate the black level of the second line data based on the plurality of pieces of second optical black pixel data of the second line data and the black level of the first line data estimated by the first black level estimating means.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a signal processing circuit, an imaging device, and a signal processing method.

  2. Description of the Related Art An imaging device that performs pupil division type focus detection using an image sensor in which a microlens is formed in each pixel is known. Patent Document 1 discloses an imaging apparatus capable of detecting a pupil division type phase difference. In Patent Document 1, one pixel of an image sensor has two photodiodes (hereinafter referred to as divided pixels), and each photodiode receives light that has passed through a different pupil of a photographing lens by one microlens. . In an imaging device having such divided pixels, imaging surface phase difference AF (autofocus) can be performed by comparing the waveforms of output signals from two photodiodes. Moreover, a normal captured image can be obtained by adding the output signals from the two photodiodes.

JP 2001-83407 A

  All pixels (addition state of two photodiode output signals) are read in order to obtain a normal captured image for recording. On the other hand, the imaging control for performing imaging pixel phase difference AF and recording image acquisition scanning by controlling division line readout for an arbitrary line for imaging surface phase difference AF is, for example, moving image shooting or This is effective for focusing when shooting a still image with a mirrorless camera. A normal photographing pixel acquisition line (hereinafter referred to as a normal row) read out in a state where two photodiode output signals are added, and a line including a signal readout in divided pixel units for focusing (hereinafter referred to as an AF row). is there. An offset step due to a difference in dark current and signal readout timing of the pixel circuit portion occurs between the normal row and the AF row. The offset step causes horizontal stripes and causes image quality degradation. Moreover, since the offset step has temperature characteristics, real-time detection and correction are required. In the imaging device, when the image sensor readout signal is digitized, signal processing, image processing, and recording, the recording data processing flow performs frame data processing with the divided pixel data removed in the middle, but offset There may be steps. In that case, offset step correction is required during the frame data processing.

  The objective of this invention is providing the signal processing circuit, imaging device, and signal processing method which can suppress generation | occurrence | production of a horizontal stripe.

  The signal processing circuit according to the present invention includes first line data including a plurality of first optical black pixel data and a plurality of first effective pixel data, a plurality of second optical black pixel data, and a plurality of first optical pixel data. A signal processing circuit for inputting frame data having second line data including two effective pixel data, wherein the first line data is based on a plurality of first optical black pixel data of the first line data. The first black level estimating means for estimating the black level of one line data; the plurality of second optical black pixel data of the second line data; and the first black level estimating means estimated by the first black level estimating means. Second black level estimating means for estimating the black level of the second line data based on the black level of the first line data.

  According to the present invention, generation of horizontal stripes can be suppressed.

It is a figure which shows the structural example of an imaging device. It is a figure which shows the structural example of an image pick-up element. It is the top view and sectional drawing which show the structural example of the pixel array of an image pick-up element. It is a timing chart which shows reading of one frame of an imaging device. 6 is a timing chart illustrating a method for driving the imaging apparatus. It is a timing chart which shows the drive method from which an imaging device differs. 6 is a timing chart illustrating a method for driving the imaging apparatus. It is a figure which shows the frame read-out data state of an image pick-up element. It is a figure which shows the frame state of a process of a clamp process circuit. It is a figure which shows the extraction black level of a detection area. It is a figure which shows the structural example of a clamp process circuit. It is a figure which shows the table of a representative black level and a temperature compensation coefficient. It is a flowchart of the temperature compensation process of CPU. It is a figure which shows the structural example of a filter circuit. It is a timing chart which shows the process of a filter circuit. It is a figure which shows the structural example of an averaging means. It is a timing chart which shows the process of an averaging means. It is a timing chart which shows the process of a clamp process circuit. It is a figure which shows the other structural example of a clamp process circuit.

  FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus 1 according to an embodiment of the present invention. The imaging device 1 can be applied to a smartphone, a tablet, an industrial camera, a medical camera, and the like in addition to a digital camera and a video camera. The imaging apparatus 1 includes a CPU 100, an SSG 101, an optical system 102, an imaging element 103, a signal processing circuit 104, an image processing circuit 105, and a ROM 106. The imaging apparatus 1 further includes an encoding processing unit 107, a recording processing unit 108, a recording medium 109, a display processing unit 110, a display device 111, a TG 112, and a defocus amount calculation unit 113. The optical system 102 includes a focus lens and a zoom lens, and forms an optical image on the image sensor 103. The image sensor 103 generates an image signal by photoelectric conversion, converts the image signal from analog to digital, and outputs a digital image signal.

  The signal processing circuit 104 corrects the image signal output from the image sensor 103 and acquires an evaluation value for focusing control. The signal processing circuit 104 includes an image data separation unit 1040, a clamp processing unit 1041, a shading correction unit 1042, a flaw correction unit 1043, a level matching unit 1044, a shading correction unit 1045, and a correlation calculation unit 1046. . The image data separation unit 1040 outputs the divided pixel signal as the parallax image to the level matching unit 1044 of the signal processing path for the imaging plane phase difference AF, and all the pixel signals to the clamp processing unit 1041 of the signal processing path of the recording system. Output. The signal processing path of the recording system includes a clamp processing unit (clamp processing circuit) 1041 that reproduces a black level, a shading correction unit 1042 that corrects shading, and a flaw correction unit 1043 that corrects pixel defects of the image sensor 103. The signal processing path for AF inputs left and right (upper and lower, but handles left and right in the configuration of the image sensor 103 in FIG. 3), and inputs level matching means 1044, shading correction means 1045, and correlation calculation means. 1046. The level matching unit 1044 is a unit that matches the black level of the parallax image pair, and is a unit that performs clamp processing for each of the left and right (or top and bottom) to make the pedestal level common. The shading correction unit 1045 performs each shading correction of the parallax image pair. The correlation calculation unit 1046 evaluates the correlation between the parallax images in any distance measurement frame. The correlation calculation unit 1046 performs correlation calculation in the distance measurement frame while shifting each of the parallax image areas. In addition, the correlation calculation unit 1046 also performs correlation value estimation (subpixel estimation) processing for a resolution of one pixel or less. The defocus amount calculation means 113 converts the correlation value between the pixels into lens extension amount information and feeds it back to the optical system 102. The defocus amount calculation unit 113 holds the adjustment value acquired in the previous step as a conversion coefficient for converting the correlation calculation result into the lens extension amount. The optical system 102 drives the focus lens in accordance with the lens extension amount to realize autofocus.

  The image processing circuit 105 includes a noise reduction (NR) 1050, an optical recovery processing unit 1051, and a development processing unit 1052. The noise reduction (NR) 1050 performs noise suppression processing on the image signal corrected by the scratch correction unit 1043. The position of noise reduction (NR) in the present embodiment is an example, and may be performed in advance as part of signal processing or may be performed on color data after development. The optical recovery processing unit 1051 recovers the deterioration in image quality due to diffraction. The development processing unit 1052 performs color data processing (synchronization, false color processing, gamma correction, etc.).

  The encoding processing unit 107 compresses the image signal output from the development processing unit 1052 for recording. The recording processing unit 108 records the compressed image signal on the recording medium 109. The recording medium 109 is a memory card or the like. The display processing unit 110 generates a display image based on the image signal output from the development processing unit 1052 and transfers the image to the display device 111. The display device 111 is a liquid crystal display (LCD) or the like, and displays an image.

  The ROM 106 stores a program. The CPU 100 controls the overall processing of the imaging apparatus 1 by executing a program stored in the ROM 106. An SSG (Synchronous Signal Generator) 101 is a synchronization signal generation unit that controls imaging timing. The SSG 101 outputs frame control timing to the signal processing circuit 104 and outputs a frame synchronization signal to the TG 112. A TG (Timing Generator) 112 is a timing generator and controls the drive timing of the image sensor 103. The SSG 101 and TG 112 are an example for the signal processing circuit 104 to process the transfer data of the image sensor 103 on the fly.

  FIG. 2 is a diagram illustrating a configuration example of the image sensor 103 of FIG. The image sensor 103 includes a unit pixel 10, a vertical scanning circuit 12, a horizontal scanning circuit 14, a column readout circuit 16, a memory 18, a ramp signal generator 20, and a counter 22. The plurality of unit pixels 10 are arranged in a two-dimensional matrix along the row direction and the column direction, and constitute a pixel array that is an imaging region. In FIG. 2, only the unit pixels 10 in 2 rows and 2 columns are shown for simplification of the drawing, but the number of unit pixels 10 arranged in the row direction and the column direction is not particularly limited. In the present specification, the row direction indicates the horizontal direction in the drawing, and the column direction indicates the vertical direction in the drawing. In the present embodiment, the row direction corresponds to the horizontal direction in the imaging apparatus 1, and the column direction corresponds to the vertical direction in the imaging apparatus 1.

  Each of the plurality of unit pixels 10 is a pupil division type focus detection pixel having a plurality of photoelectric conversion elements 24A and 24B each converting light into electric charge. Each of the plurality of unit pixels 10 includes a light receiving unit 24, transfer MOS transistors 26A and 26B, a reset MOS transistor 28, an amplification MOS transistor 30, and a selection MOS transistor 32. FIG. 2 shows an example of a specific circuit configuration for only one unit pixel 10 for simplification of the drawing. The light receiving unit 24 includes a plurality of photoelectric conversion elements 24A and 24B. The photoelectric conversion elements 24A and 24B are, for example, photodiodes, and are photoelectric conversion units that convert light into electric charges. The photoelectric conversion element 24A has an anode connected to the ground voltage line and a cathode connected to the source of the transfer MOS transistor 26A. The photoelectric conversion element 24B has an anode connected to the ground voltage line and a cathode connected to the source of the transfer MOS transistor 26B. The drains of the transfer MOS transistors 26 A and 26 B are connected to the source of the reset MOS transistor 28 and the gate of the amplification MOS transistor 30. The connection nodes of the drains of the transfer MOS transistors 26A and 26B, the source of the reset MOS transistor 28, and the gate of the amplification MOS transistor 30 constitute a floating diffusion region (hereinafter referred to as FD region) 34.

  The drains of the reset MOS transistor 28 and the amplification MOS transistor 30 are connected to the power supply voltage line. The source of the amplification MOS transistor 30 is connected to the drain of the selection MOS transistor 32. The transfer MOS transistors 26A and 26B, the reset MOS transistor 28, the amplification MOS transistor 30, and the selection MOS transistor 32 constitute an in-pixel readout circuit for reading out pixel signals based on the charges generated by the photoelectric conversion elements 24A and 24B. Note that the names of the source and the drain of the transistor may differ depending on the conductivity type of the transistor, the function of interest, and the like, and the above-described source and drain may be referred to as opposite names.

  FIGS. 3A and 3B are a plan view and a cross-sectional view illustrating a configuration example of a pixel array of the image sensor 103. FIG. 3A is a top view of the pixel array, and FIG. 3B is a cross-sectional view taken along the line A-A ′ of FIG. The pixel array shown in FIG. 2 is not particularly limited, but can be realized by, for example, the planar layout shown in FIG. A unit area surrounded by a dotted line in FIG. One microlens 38 is arranged on each unit pixel 10. As shown in FIG. 3B, a color filter 36 is disposed between the photoelectric conversion elements 24A, 24B and the microlens 38.

  In this specification, when the plurality of photoelectric conversion elements 24 </ b> A and 24 </ b> B included in one unit pixel 10 are collectively shown, the light receiving unit 24 is used. One micro lens 38 is disposed on one light receiving unit 24 and condenses the light beam on the light receiving unit 24. That is, one microlens 38 is provided corresponding to one light receiving unit 24. Incident light collected by one microlens 38 enters the photoelectric conversion elements 24A and 24B constituting the light receiving unit 24 of the unit pixel 10 provided corresponding to the microlens 38. The photoelectric conversion elements 24A and 24B constitute a pair of pupil division type focus detection pixels.

  In FIG. 2, a signal line TXA, a signal line TXB, a signal line RES, and a signal line SEL are arranged in each row of the unit pixels 10 so as to extend in the row direction. The signal line TXA is connected to the gates of the transfer MOS transistors 26A of the unit pixels 10 arranged in the row direction, and forms a signal line common to the unit pixels 10. The signal lines TXB are respectively connected to the gates of the transfer MOS transistors 26B of the unit pixels 10 arranged in the row direction, and form a common signal line for these unit pixels 10. The signal lines RES are respectively connected to the gates of the reset MOS transistors 28 of the unit pixels 10 arranged in the row direction, and form a common signal line for these unit pixels 10. The signal lines SEL are respectively connected to the gates of the selection MOS transistors 32 of the unit pixels 10 arranged in the row direction, and form a signal line common to the unit pixels 10. In FIG. 2, only signal lines TXA, TXB, RES, and SEL connected to one unit pixel 10 are shown for simplification of the drawing.

  The vertical scanning circuit 12 is for selecting a unit pixel 10 for each row based on a predetermined timing signal from the TG 110 and outputting a pixel signal from the unit pixel 10. The signal line TXA, signal line TXB, signal line RES, and signal line SEL are connected to the vertical scanning circuit 12. A transfer pulse signal PTXA for driving the transfer MOS transistor 26A is output from the vertical scanning circuit 12 to the signal line TXA. A transfer pulse signal PTXB for driving the transfer MOS transistor 26B is output from the vertical scanning circuit 12 to the signal line TXB. A reset pulse signal PRES for driving the reset MOS transistor 28 is output from the vertical scanning circuit 12 to the signal line RES. A selection pulse signal PSEL for driving the selection MOS transistor 32 is output from the vertical scanning circuit 12 to the signal line SEL. Here, when a high-level signal is applied to these signal lines, the corresponding transistor is assumed to be in a conductive state (on state). In addition, when a low-level signal is applied, the corresponding transistor is turned off (off state).

  In each column of the unit pixels 10, column signal lines 40 are arranged extending in the column direction. The column signal line 40 is connected to the source of each selection MOS transistor 32 of the unit pixels 10 arranged in the column direction, and forms a signal line common to the unit pixels 10. A column readout circuit 16 and a current source 42 are connected to the column signal line 40 of each column. The column readout circuit 16 is for processing a signal read out from the unit pixel 10. As shown in FIG. 2, the column readout circuit 16 includes an amplifier 44 and an analog / digital conversion unit 52.

  The amplifier 44 includes an operational amplifier (differential amplifier) 46, a switch 48, an input capacitor C0, and a load capacitor Cf, and amplifies the signal of the unit pixel 10. The inverting input terminal of the operational amplifier 46 is connected to the column signal line 40 via the input capacitor C0. A reference voltage is applied to the non-inverting input terminal of the operational amplifier 46. A load capacitor Cf and a switch 48 are connected in parallel between the inverting input terminal and the output terminal of the operational amplifier 46. The switch 48 is driven by a signal PC0R applied to the control node. Here, when the signal PC0R is at a high level, the switch 48 is turned on (on state), and when the signal PC0R is at a low level, the switch 48 is turned off (off state).

  The analog-to-digital converter 52 includes an operational amplifier 54 that forms a buffer circuit and an operational amplifier 56 that forms a comparator. The non-inverting input terminal of the operational amplifier 54 is connected to the output terminal of the operational amplifier 46. The inverting input terminal of the operational amplifier 54 is connected to the output terminal of the operational amplifier 54. The output terminal of the operational amplifier 54 is connected to the non-inverting input terminal of the operational amplifier 56. The inverting input terminal of the operational amplifier 56 is connected to the ramp signal generator 20. The memory 18 is connected to the output terminal of the operational amplifier 56. A horizontal scanning circuit 14 and a counter 22 are connected to the memory 18.

  The ramp signal generator 20 generates a ramp signal whose level increases with time. The operational amplifier 56 compares the pixel signal output from the operational amplifier 54 with the ramp signal output from the ramp signal generator 20, and inverts the output signal when the magnitude relationship between the pixel signal and the ramp signal is reversed. The counter 22 starts counting the counter value when the level change of the ramp signal starts. The memory 18 stores the counter value of the counter 22 when the output signal of the operational amplifier 56 is inverted. This counter value is a digital pixel signal. The horizontal scanning circuit 14 sequentially outputs the digital pixel signals of each column stored in the memory 18 to the outside.

  FIG. 4 is a timing chart showing reading of one frame of the imaging apparatus 1. In the driving method of the imaging apparatus 1, as shown in FIG. 4, shutter scanning and readout scanning are sequentially performed for each row of the imaging elements 103 with the vertical synchronization signal VD as a reference. In the shutter scanning, the vertical scanning circuit 12 sequentially resets the held charges of the photoelectric conversion elements 24A and 24B of the plurality of unit pixels 10 belonging to the corresponding row for each row. In the readout scanning, the vertical scanning circuit 12 sequentially reads out signals based on the accumulated charges of the photoelectric conversion elements 24A and 24B from the plurality of unit pixels 10 belonging to the corresponding row for each row. The time from the start of shutter scanning to the start of readout scanning is the accumulation period of signal charges in the photoelectric conversion elements 24A and 24B.

  FIG. 5 is a timing chart showing the driving method of the imaging apparatus 1, and more specifically shows the read operation of each row. In FIG. 5, the above-described shutter scanning operation generally corresponds to the period from time t1 to time t3. Further, the above-described readout scanning operation generally corresponds to a period from time t10 to time t15.

  First, in synchronization with the horizontal synchronization signal HD, at time t1, the signal PRES is at a high level, the signals PTXA and PTXB are at a high level, and the reset MOS transistor 28 and the transfer MOS transistors 26A and 26B are turned on. As a result, the charges accumulated in the photoelectric conversion elements 24A and 24B are discharged through the transfer MOS transistors 26A and 26B and the reset MOS transistor 28. That is, the reset operation of the photoelectric conversion elements 24A and 24B is performed. As described above, in the driving method of the present embodiment, the photoelectric conversion elements 24A and 24B are simultaneously reset by setting the signal PRES, the signal PTXA, and the signal PTXB to the high level as the electronic shutter pulse. When electronic shutter driving is performed, the unit pixel 10 is reset prior to readout scanning as described above.

  Next, at time t2, the signals PTXA and PTXB become low level, and the transfer MOS transistors 26A and 26B are turned off. At this timing, the reset operation of the photoelectric conversion elements 24A and 24B ends, and the charge accumulation period of the photoelectric conversion elements 24A and 24B starts. After the transfer MOS transistors 26A and 26B are turned off, at time t3, the signal PRES goes high and the reset MOS transistor 28 is turned off. This completes a series of shutter scanning operations.

  In this state, after the charge is accumulated in the photoelectric conversion elements 24A and 24B for a predetermined period, the signal based on the accumulated charges of the photoelectric conversion elements 24A and 24B is read from the unit pixel 10 for each row or for every plurality of rows. That is, reading scanning is performed. In synchronization with the horizontal synchronization signal HD, at time t4, the signal PRES is set to the high level, and the reset MOS transistor 28 is turned on. As a result, the FD region 34, which is also the input node of the amplification MOS transistor 30, is electrically connected to the power supply voltage line via the reset MOS transistor 28, and the input node of the amplification MOS transistor 30 is reset to the reset level potential. Similarly, at time t4, the signal PC0R is set to the high level, and the switch 48 is turned on. As a result, the output terminal and the inverting input terminal of the operational amplifier 46 are short-circuited to put the operational amplifier 46 in a buffer state.

  At time t5, the signal PSEL is set to the high level, and the selection MOS transistor 32 is turned on. As a result, the amplifying MOS transistor 30 is in a state where a bias current is supplied to the source from the current source 42 via the column signal line 40, and constitutes a source follower circuit. As a result, a signal (reset signal) when the input node of the amplification MOS transistor 30 is at the reset level is output to the column readout circuit 16 via the selection MOS transistor 32 and the column signal line 40. . The reset signal input to the column readout circuit 16 is input to the inverting input terminal of the operational amplifier 46 that is in a state of buffering the output of the reference voltage via the input capacitor C0.

  At time t6, the signal PRES goes low, and the reset MOS transistor 28 is turned off. Thereby, the reset operation of the input node of the amplification MOS transistor 30 is released.

  At time t7, the signal PC0R becomes low level, and the switch 48 is turned off. As a result, the load capacitance Cf is connected to the feedback path of the operational amplifier 46, and the reset signal amplified according to the gain determined by the ratio (C0 / Cf) of the input capacitance C0 to the load capacitance Cf is received. , And output from the output terminal of the operational amplifier 46.

  Next, at time t10, the signal PTXA becomes high level, and the transfer MOS transistor 26A is turned on. Thereby, the signal charge generated by the photoelectric conversion in the photoelectric conversion element 24 </ b> A during the charge accumulation period is transferred to the FD region 34. A pixel signal based on the potential of the input node of the amplification MOS transistor 30 corresponding to the amount of signal charge transferred from the photoelectric conversion element 24A is output to the column signal line 40 via the selection MOS transistor 32.

  After outputting the pixel signal from the unit pixel 10 to the column readout circuit 16 via the column signal line 40, the signal PTXA becomes low level at time t11. The operational amplifier 46 amplifies the pixel signal input from the unit pixel 10 via the input capacitor C0 according to the gain determined by the ratio (C0 / Cf) of the input capacitor C0 and the load capacitor Cf, and outputs an output terminal. Output from. This pixel signal is hereinafter referred to as “A signal”.

  Next, at time t14, the signal PTXB becomes high level, and the transfer MOS transistor 26B is turned on. As a result, the signal charges generated by the photoelectric conversion in the photoelectric conversion element 24B are transferred to the FD area 34, and as a result, the total signal charges of the photoelectric conversion elements 24A and 24B are accumulated in the FD area 34. Is done. A pixel signal based on the potential of the input node of the amplification MOS transistor 30 corresponding to the total amount of signal charges of the photoelectric conversion elements 24A and 24B is output to the column signal line 40 via the selection MOS transistor 32.

  After the pixel signal is output from the unit pixel 10 to the column readout circuit 16 via the column signal line 40, the signal PTXB becomes low level at time t15. The operational amplifier 46 amplifies the pixel signal input from the unit pixel 10 via the input capacitor C0 according to the gain determined by the ratio (C0 / Cf) of the input capacitor C0 and the load capacitor Cf, and outputs an output terminal. Output from. This pixel signal is hereinafter referred to as “A + B signal”. The A + B signal is data obtained by combining data based on the charges of the plurality of photoelectric conversion elements 24A and 24B.

  The reset signal, the A signal, and the A + B signal output from the amplifier 44 are converted into digital signals by the analog / digital conversion unit 52 and then stored in the memory 18. These digital signals in each column stored in the memory 18 are sequentially read according to a control signal from the horizontal scanning circuit 14. When the column readout circuit 16 does not have the analog-digital conversion unit 52, signals of each column are sequentially read out through an output amplifier or a buffer.

  As the focus detection signal, the A signal based on the signal charge of the photoelectric conversion element 24A read out in this way and the B signal based on the signal charge of the photoelectric conversion element 24B are used. The B signal is calculated by subtracting the A signal from the signal (A + B signal) based on the signal charges of the photoelectric conversion element 24A and the photoelectric conversion element 24B.

  However, as shown in FIG. 5, the time for acquiring the A signal (time t10 to time t11) is different from the time for acquiring the A + B signal (time t14 to time t15). In addition, the reset of the input node of the amplification MOS transistor 30 is performed after the signal charge of the photoelectric conversion element 24A is transferred to the FD region 34 until the signal charge of the photoelectric conversion elements 24A and 24B is transferred to the FD region 34. I will not. Therefore, there is a time difference dt between the exposure time of the photoelectric conversion elements 24A and 24B until the A signal is acquired and the exposure time of the photoelectric conversion elements 24A and 24B until the A + B signal is acquired (FIG. 5). Therefore, the dark current component of the FD region 34 increased during the time difference dt is superimposed on the A + B signal read from time t14 to time t15.

  The above is a description of the case where the focus detection signal (A signal) and the image formation signal (A + B signal) are acquired, but the two focus detection signals (A signal and B signal) are read independently. There is a case. In that case, the case where the A signal and the B signal acquired from the image sensor 103 are generated into a signal for image formation (A + B signal) later by digital signal processing, the case where the signal is accumulated in the state of charge in the FD region 34, and There are cases where both are mixed in line units to obtain a frame image. In that case, similar signal level correction is required. When reading two focus detection signals (A signal and B signal) independently, as shown in FIG. 6, for example, the A signal is acquired from the signal charge of the photoelectric conversion element 24A in the first one horizontal cycle. Then, the B signal is acquired from the signal charge of the photoelectric conversion element 24B in one horizontal cycle. By doing in this way, the reset state of the input node of the amplification MOS transistor 30 with respect to reading of the A signal and the B signal can be made equal. The times t4 to t11 in one horizontal cycle and the subsequent times t4 ′ to t11 ′ in one horizontal cycle in FIG. 6 are the same time for each HD that is the starting point. The A signal and the B signal can be read out respectively, and an image forming signal (A + B) can be acquired as digital signal processing after the image sensor 103. In that case, a means for synthesizing the A signal and the B signal is required in the image data separation unit 1040 and the like in the signal processing circuit 104, and the dark current component deteriorates twice when the processing is performed.

  The adverse effect of the dark current component when obtained by the readout control shown in FIGS. 5 and 6 causes, for example, horizontal stripes in the offset component clamping process, and reduces the image quality of the recorded image. In the present embodiment, the driving / control for the imaging plane phase difference AF is illustrated as a factor including lines with different black levels in the frame data, but the present invention can be applied even if it is due to other factors. .

  FIG. 7 is a timing chart showing an image forming signal readout driving method of the imaging apparatus 1 of the present embodiment when the signal charges of the photoelectric conversion elements 24A and 24B are simultaneously acquired in the FD region 34. The operation from time t1 to t7 is the same as the operation in FIG. At time t10, the signals PTXA and PTXB become high level, and the transfer MOS transistors 26A and 26B are turned on. At time t11, the signals PTXA and PTXB become low level, and the transfer MOS transistors 26A and 26B are turned off. Thereby, an image forming signal (A + B signal) can be obtained.

  Here, let us consider a case where a focus detection signal (A signal) is acquired while recording frame data is being imaged, and a recording operation and focus control are performed in parallel. At this time, in order to suppress the transfer band between the image sensor 103 and the signal processing circuit 104 in FIG. 1, a control for transferring the focus detection signal (A signal) only on an arbitrary line without transferring all lines is considered. FIG. 8 shows a data state of frame reading of the image sensor 103 in such a control state. FIG. 8 shows the data for one line read out in one cycle of the signal HD of FIG. 5 (FIG. 6, FIG. 7, etc.) in the horizontal lines 800, 801, 802, etc. Expressed as frame data.

  Data from the nth line (denoted <line n> in FIG. 8) to the n + 4th line (denoted <line n + 4> in FIG. 8) will be described. A line 800 in FIG. 8 is a readout line (hereinafter referred to as a normal line) of an image forming signal (A + B signal) acquired by the driving method illustrated in FIG. 7, and data in the line 800 is stored in the signal processing circuit 104. It is not used for the AF signal processing path. A line 801 in FIG. 8 is a focus detection signal (A signal) readout line (hereinafter referred to as a first parallax line) acquired by the driving method shown in FIG. 5 and passes through the AF signal processing path. Signal. A line 802 in FIG. 8 is a readout line (hereinafter referred to as a second parallax line) of an image forming signal (A + B signal) acquired by the driving method illustrated in FIG.

  The vertical scanning circuit 12 is a reading unit, and reads the unit pixel 10 corresponding to the normal line 800 and the unit pixel 10 corresponding to the second parallax line 802 by different driving methods. The vertical scanning circuit 12 reads out only the A + B signal obtained by combining the data based on the charges of the plurality of photoelectric conversion elements 24A and 24B in the unit pixel 10 corresponding to the normal line 800 by the driving method of FIG. On the other hand, in the unit pixel 10 corresponding to the second parallax line 802, the vertical scanning circuit 12 uses the driving method of FIG. 5 to generate an A signal based on the charges of some of the photoelectric conversion elements 24A and a plurality of photoelectric conversions. An A + B signal obtained by combining data based on the charges of the elements 24A and 24B is read out. The normal line 800 is first line data, and the second parallax line 802 is second line data.

  The image data separation unit 1040 differentially processes the A signal of the first parallax line 801 and the A + B signal of the second parallax line 802 (actually includes measures against the luminance saturation signal) to generate a B signal. Then, the A signal and the B signal are output toward the AF signal processing path. As the recording system image signal, the image data separation unit 1040 outputs the A + B signal of the normal line 800 and the A + B signal of the second parallax line 802 to the clamp processing circuit 1041 in FIG. At this time, the frame data received by the clamp processing circuit 1041 is as shown in FIG.

  FIG. 9 shows the state of a processing frame corresponding to the pixel array of the image sensor 103 and received by the clamp processing circuit 1041 of FIG. Frame data 900 is frame data received by the clamp processing circuit 1041. A region 901 is a detection region for black level estimation in an OB pixel region (reference pixel region) in which an OB pixel (reference pixel) optically shielded from the light receiving unit 24 is arranged. The OB pixel is an optical black pixel. A region 902 is a region of effective pixels as a recorded image that is not optically shielded from light. The plurality of unit pixels 10 are divided into OB pixels in the region 901 and effective pixels in the region 902. A line 800 is a line of image forming signal (A + B signal) data acquired as the normal line 800 in FIG. The OB pixel data (optical black pixel data) 810 is OB pixel (A + B signal) data in the detection area 901 for black level estimation when the normal line 800 is read. A line 802 is a line of image forming signal (A + B signal) data acquired as the second parallax line 802 in FIG. The OB pixel data 812 is OB pixel (A + B signal) data in the detection area 901 for black level estimation in the second parallax line 802.

  FIG. 10 shows an example of the extracted black level obtained as a one-dimensional mapping of the pixel signal level in the vertical direction in the detection area 901 of the frame data 900 of FIG. The target black level is a value that the correction result clamped by the clamp processing circuit 1041 is expected to approach. In FIG. 10, assuming the presence of shading in the vertical direction with respect to the frame data 900, the curve in the figure is illustrated as the OB pixel data (black level) in the detection area 901. The OB pixel data (black level) 812 of the second parallax line 802 has an offset step with respect to the OB pixel data (black level) 810 of the normal line 800. This offset step becomes a horizontal stripe in the actual recorded image as in the frame data 900 of FIG.

  FIG. 11 is a diagram illustrating a configuration example of the clamp processing circuit 1041. Hereinafter, a signal processing method of the clamp processing circuit 1041 will be described. The clamp processing circuit 1041 scans the detection area 901 for black level estimation, estimates the black level of each of the OB pixel data 810 and 812, and corrects the effective pixels in the effective area 902. The accumulator 201 includes an adder and a register, and accumulates values of a plurality of OB pixel data 810 on the normal line 800 in the detection area 901. The position recognition of the pixel of the OB pixel data 810 on the normal line 800 is in accordance with an instruction of an SSG (Synchronous Signal Generator) 210 in FIG. The SSG 210 counts the valid state of the input frame data and grasps the processing position of the frame input in the horizontal and vertical directions. By grasping the frame state, the SSG 210 generates and outputs an event and a status signal for changing the line process and the area process. The SSG 210 output signal lines other than the parameter mask state (AND elements 213 and 214 in FIG. 11) and the control instruction of the output switching selector 209 are not shown because the figure becomes complicated.

  The divider 202 normalizes by dividing the integration result value of the integrator 201 by the number of the plurality of OB pixel data 810 on the normal line 800. The division here can be realized by a right shift of the signal bit if the divisor is a power of two. The divider 202 divides the integration result of the integrator 201 for one line of the OB pixel data 810 to obtain a representative value of the black level estimated value for each line. In this embodiment, it is exemplified as black level estimation having shading in the vertical direction as shown in FIG. In this case, in view of vertical shading, it is desired to suppress the line average history within a certain range. Therefore, the configuration of the clamp processing circuit 1041 in FIG. 11 is acquired as a moving average value that does not prolong the influence of shading as the history. The circuit configuration is illustrated. The averaging means 203 takes a moving average of the black level representative value of one line obtained by the divider 202 with each line representative value until immediately before. That is, the averaging means 203 averages the data divided by the divider 202 with respect to a plurality of normal lines 800 in the frame data, and outputs an estimated black level sig_203. The accumulator 201, the divider 202, and the averaging means 203 are first black level estimation means, and estimate the black level sig_203 of the normal line 800.

  In the effective pixel data processing of the normal line 800 in the effective area 902 of FIG. 9, the following processing is performed. The subtractor 204 is a correction unit, and outputs the corrected effective pixel data of the normal line 800 by subtracting the black level sig_203 output from the averaging unit 203 from the effective pixel data sig_216 of the normal line 800. That is, the subtractor 204 corrects the effective pixel data sig_216 of the normal line 800 based on the black level sig_203 output from the averaging unit 203. The delay element 216 is a delay element for matching the input data and the correction parameter update timing, and includes a shift register or the like. The logical element 213 is an AND element that transmits the result value of the averaging means 203 to the subtractor 204 when the normal line 800 in the effective area 902 is instructed by the SSG 210 instruction, and masks it to 0 value in other cases. . The adder 208 adds the post setup value set in the register 212 as the pedestal level to the output value of the subtracter 204, and outputs the corrected effective pixel data sig_208 of the normal line 800.

  When data is input to the second parallax line 802, the SSG 210 outputs an instruction signal so that the processing of the filter circuit 206 proceeds. The filter circuit 206 exemplifies a cyclic (IIR) low-pass filter (LPF), and its time constant may be changed by register setting via the CPU 100. The subtractor 205 calculates a difference sig_205 between the OB pixel data 812 on the detection region 901 of the second parallax line 802 and the black level sig_203 of the OB pixel data 810 output from the averaging unit 203. Then, the subtracter 205 outputs the difference (offset step) sig_205 to the filter circuit 206. The filter circuit 206 receives the difference sig_205 calculated by the subtractor 205 and outputs the black level sig_206 of the second parallax line 802. Details of the filter circuit 206 will be described later with reference to FIG. The delay element 215, the subtractor 205, and the filter circuit 206 are black level estimation means, and estimate the black level sig_206 of the second parallax line 802.

  In the effective pixel data processing of the second parallax line 802 in the effective area 902, the following processing is performed. The subtractor 207 is a correction unit, and subtracts the black level sig_206 output from the filter circuit 206 from the effective pixel data sig_217 of the second parallax line 802 to correct the corrected effective pixel data sig_207 of the second parallax line 802. Is output. The subtractor 207 corrects the effective pixel data sig_217 of the second parallax line 802 based on the black level sig_206 output from the filter circuit 206. The logic element 214 transmits the output value of the filter circuit 206 to the subtractor 207 when processing the effective pixel data in the effective region 902 in the second parallax line 802, and performs an operation of masking to 0 value otherwise. It is an element. The delay element 217 is a delay element for matching the input data and the correction parameter update timing, and includes a shift register or the like. The delay element 215 delays input data and outputs the delayed input data to the subtracter 205. The delay amount of the delay element 217 is set equal to the sum of the delay amount of the delay element 215 and the delay amount of the filter circuit 206. Further, by making the delay amount of the delay element 217 equal to the delay amount of the delay element 216, the appearance position of the horizontal data becomes the same when switching between the normal line 800 and the second parallax line 802, and the frame data It is established as 900. In addition, the AND elements 213 and 214 controlled switching in the effective area 902. However, when OB pixel data is used in the clamp processing circuit 1041 and later, the AND elements 213 and 214 have values closest to the OB pixel data in the detection area 901. A configuration of the SSG 210 that applies correction may be adopted.

  The selector 209 is a selection unit and selects the corrected effective pixel data sig_208 of the normal line 800 or the corrected effective pixel data sig_207 of the second parallax line 802. The SSG 210 includes means for recognizing valid data and means for counting the recognition cycles (not shown). Then, the SSG 210 counts the input amount of the frame data 900, and outputs an instruction value 0 to the selector 209 at the timing of the normal line 800 and the instruction value at the timing of the second parallax line 802 according to the count value. 1 is output to the selector 209. The selector 209 selects the corrected effective pixel data sig_208 of the normal line 800 at the instruction value 0, and selects the corrected effective pixel data sig_207 of the second parallax line 802 at the instruction value 1 to obtain the output sig_209. . The register 211 is a storage unit that temporarily stores the result value of the filter circuit 206. The register 211 is used to update the estimated black level of the OB pixel data 812 by the filter circuit 206 when one frame is completed, and to write it back to the filter circuit 206 as an initial value at the start of the next frame scan. The register 211 can also update the initial value from the CPU 100.

  FIG. 18 is a timing chart showing processing of the clamp processing circuit 1041 of FIG. A signal sig_200 is an input image signal to the clamp processing circuit 1041, and is a transfer state of the frame data 900 of FIG. The signal sig_200 includes the OB pixel data 810 of the normal line 800, the image formation data of the normal line 800, the OB pixel data 812 of the second parallax line 802, and the image formation of the second parallax line 802 in the frame configuration of FIG. Includes data. The signal sig_201 is an output signal of the accumulator 201, and holds a value until the detection of the next line is started after the processing of the right end of the detection area 901. The signal sig_202 is an output signal of the divider 202, and indicates a result value obtained by dividing the signal sig_201 by a divisor determined in advance with the signal sig_201 as a dividend. The signals sig_201 and sig_202 are not referred to when the second parallax line 802 is scanned (handled as don't care). A signal sig_203 is an output signal of the averaging means 203, and indicates a result value obtained by calculating and outputting the average value of the vertical line of the result of the divider 202 every completion of the detection area 901 of an arbitrary line. The signal sig_203 is not updated when the second parallax line 802 is scanned.

  The signal sig_206 is an output signal of the filter circuit 206. The signal sig_206 is updated when the second parallax line 802 is scanned. Signals sig_216 and sig_217 are delay data of the signal sig_200 having the delay relationship as described above. The output timings of both signals are aligned, and the same signal state is obtained as shown in FIG. Signals sig_213 and sig_214 are status signals for correction parameter mask issued by the SSG 210. The signal sig_213 controls the state so that the image forming data of the normal line 800 is offset-corrected. Here, it is defined that the correction parameter is valid when the signal sig_213 is in the 1 value state, and the correction parameter is masked to 0 value when the signal sig_213 is in the 0 value state, and offset correction is not performed. Similarly, the signal sig_214 controls the state so that the image forming data of the second parallax line 802 is offset-corrected, and the correction parameter is masked to 0 value and offset correction is performed in the 0-value state, similarly to the signal sig_213. Do not do.

  The signal sig_208 is a correction result signal when the normal line 800 is processed, and the signal sig_207 is a correction result signal when the second parallax line 802 is processed. The signal sig_210 is a correction line status signal generated by the SSG 210. The selector 209 outputs the signal sig_207 as the signal sig_209 as scanning of the second parallax line 802 when the signal sig_210 is 1, and otherwise outputs the signal sig_208 as the signal sig_209. This signal sig_209 becomes an output signal of the clamp processing circuit 1041.

  The black level estimated value of the clamp processing circuit 1041 varies when the gain value of the amplifier 44 in FIG. 2 is changed. FIG. 12A shows the relationship between the gain value 1201 of the amplifier 44 and the representative black level 1202 at that time. The representative black level 1202 is an average value of a plurality of process values acquired in advance under an arbitrarily set temperature environment, and is stored in the ROM 106. When the gain value of the amplifier 44 is switched, the black level estimation value of the filter circuit 206 that has been stable across the frames requires a lapse of time according to the time constant of the circuit toward the new target value, and the level fluctuation To do. In order to shorten the unstable period corresponding to the time constant, a representative black level corresponding to the next set gain value of the amplifier 44 is written to the register 211 via the CPU 100.

  Here, the representative black level 1202 is a representative value at an arbitrary temperature (such as room temperature) set in advance. When the black level temperature characteristic due to dark current must be taken into account, if the representative black level 1202 that is different from the temperature of the image sensor 103 during actual operation is applied as it is, the temperature characteristic tracking of the image sensor 103 is followed. Does not fulfill. Therefore, temperature compensation coefficients 1204 to 1208 for the gain value 1203 of the amplifier 44 are provided as compensation parameters for following the temperature characteristics, which is a feature of this embodiment, as shown in FIG. Here, the inheritance of the temperature characteristic tracking to the gain value change of the amplifier 44 of the black level estimation value in the filter circuit 206 will be described.

  As described above, in this embodiment, the configuration of the clamp processing circuit 1041 and the black level estimation of the normal line 800 are completed by one frame, and the black level estimation of the second parallax line 802 is inherited by a plurality of frames. It is configured as follows. In the black level detection region 901, the appearance frequency of the second parallax line 802 is low, and the filter circuit 206 has an IIR filter configuration. The black level in FIG. 10 includes an offset dark current component and has a temperature characteristic (the black level increases as the temperature rises).

  FIG. 13 is a flowchart of temperature compensation processing performed by the CPU 100 in the continuous frame processing of the filter circuit 206. The temperature compensation process is performed as one process of the program stored in the ROM 106. In the present embodiment, the CPU 100 stores the estimated black level value of the filter circuit 206 in the register 211 at the end of the frame processing, reads it out at the start of the next frame, and resets it in the filter circuit 206. The updating of the register 211 and the writing back to the filter circuit 206 at this time may be performed by the SSG 210. The initial value storage / update for temperature compensation in the register 211 is executed by the CPU 100 between the update of the register 211 value by the SSG 210 and the writing back to the filter 206.

  In step S100, the CPU 100 determines whether or not there is a change in the gain value of the amplifier 44 in the next frame imaging. When it is determined that there is a change, the process proceeds to step S101, and when it is determined that there is no change. The temperature compensation process ends. In step S101, the CPU 100 reads the stored value of the register 211. Next, in step S102, the CPU 100 reads the representative black level 1202 for the gain value (gain value before change) 1201 used for the current frame processing from the table of FIG. Next, in step S103, the CPU 100 calculates the ratio ratio of the value of the register 211 and the representative black level 1202. Next, in step S104, the CPU 100 reads the representative black level 1202 corresponding to the next gain value (changed gain value) 1201 set in the amplifier 44 from the table of FIG.

  Next, in steps S105 to S115, the CPU 100 executes a temperature compensation coefficient determination process. FIG. 12 (c) shows a temperature change index 1209, which relates the deviation between the current temperature and the reference temperature based on the magnitude relationship of the ratio ratio calculated earlier, and represents the representative black level during frame processing with the next gain value. Is used to estimate the initial temperature compensation coefficient. In the present embodiment, the temperature change index 1209 is a process value acquired in the sensor manufacturing process or the sensor evaluation process. The numerical value itself is not fixed in view of process trends and individual differences in manufacturing. Here, the temperature change index 1209 is divided into five regions of div (A) to div (E), but this may also be matched to the actual situation.

  In step S105, the CPU 100 determines whether or not the ratio ratio is smaller than the temperature change index div (A). If it is determined that the ratio ratio is small, the process proceeds to step S106. The process proceeds to S107. In step S106, assuming that the reference black level is applied as it is, the CPU 100 sets the adjustment value to 1, and proceeds to step S116. The adjustment value is a value selected from the temperature compensation coefficient table of FIG. 12B, but is set to 1 without any special reference table only at step S106.

  In step S107, the CPU 100 determines whether or not the ratio ratio satisfies the condition of div (A) ≦ ratio <div (B) with respect to the temperature change index. If the CPU 100 determines that the condition is satisfied, the process proceeds to step S108. If the CPU 100 determines that the condition is not satisfied, the process proceeds to step S109. In step S108, the CPU 100 sets a value corresponding to the next gain value 1203 in the temperature compensation coefficient 1204 in FIG. 12B as the adjustment value, and proceeds to step S116.

  In step S109, the CPU 100 determines whether or not the ratio ratio satisfies the condition of div (B) ≦ ratio <div (C) with respect to the temperature change index. If the CPU 100 determines that the condition is satisfied, the CPU 100 proceeds to step S110. If the CPU 100 determines that the condition is not satisfied, the CPU 100 proceeds to step S111. In step S110, the CPU 100 sets a value corresponding to the next gain value 1203 in the temperature compensation coefficient 1205 in FIG. 12B as the adjustment value, and proceeds to step S116.

  In step S111, the CPU 100 determines whether or not the ratio ratio satisfies the condition of div (C) ≦ ratio <div (D) with respect to the temperature change index. If the CPU 100 determines that the condition is satisfied, the process proceeds to step S112. If the CPU 100 determines that the condition is not satisfied, the CPU 100 proceeds to step S113. In step S112, the CPU 100 sets a value corresponding to the next gain value 1203 in the temperature compensation coefficient 1206 of FIG. 12B as the adjustment value, and proceeds to step S116.

  In step S113, the CPU 100 determines whether or not the ratio ratio satisfies the condition of div (D) ≦ ratio <div (E) with respect to the temperature change index. If the CPU 100 determines that the condition is satisfied, the process proceeds to step S114. If the CPU 100 determines that the condition is not satisfied, the process proceeds to step S115. In step S114, the CPU 100 sets a value corresponding to the next gain value 1203 in the temperature compensation coefficient 1207 in FIG. 12B as the adjustment value, and proceeds to step S116. In step S115, since div (E) ≦ ratio, the CPU 100 sets a value corresponding to the next gain value 1203 in the temperature compensation coefficient 1208 in FIG. 12B as the adjustment value, and the process proceeds to step S116. To proceed.

  In step S116, the CPU 100 sets the representative black level 1202 corresponding to the next gain value 1201 of the amplifier 44 as the adjustment black level set in the filter circuit 206 for the next frame processing, and the adjustment value obtained in steps S105 to S115. The product of is calculated. That is, the CPU 100 is a control unit and adjusts the representative black level 1202 corresponding to the next gain value 1201 of the amplifier 44 with an adjustment value corresponding to the ratio ratio. Next, in step S117, the CPU 100 is a control unit, writes the black level of the calculation result in step S116 into the register 211, and ends the temperature compensation process. In the present embodiment, the above processing is performed every time one frame is imaged. In this embodiment, the temperature change is obtained by the ratio ratio between the representative black level 1202 and the value of the current register 211. However, when the temperature change is gradual and the change for each gain setting is small, the difference between the two is obtained. You can go.

  The accumulator 201 in FIG. 11 has a register configuration, and accumulates valid data in a designated area to a register value. Further, the configuration of the divider 202 in FIG. 11 is not limited. For example, if the number of pixels in the detection area 901 in the horizontal direction (the number of integrated pixels) is a power of 2, the divider 202 can be configured only by bit shift with respect to the signal. When a strict division is performed instead of a power of 2, a divider such as that shown in FIG. 3 of Japanese Patent Laid-Open No. 2013-142570 can be used. In such a case, the division processing in the synchronous circuit requires a plurality of cycles, and therefore, for example, timing adjustment is considered as a delay factor of the delay element illustrated in FIG.

The filter circuit 206 in FIG. 11 may have a general configuration as an IIR filter, but in this embodiment, the initialization includes the initialization by the value of the register 211, which will be described. FIG. 14 shows a configuration example of the filter circuit 206 of FIG. As a general expression of the IIR filter 206, an arbitrary n-th process for the input X and the output Y is expressed by the following expression (1). When the expression (1) is expanded, the following expression (2) is obtained. It is done. The input X is the difference sig_205, and the output Y is the black level sig_206.
Yn = A.Xn + (1-A) .Yn-1 (1)
Yn = A · (Xn−Yn−1) + Yn−1 (2)

  The filter circuit 206 in FIG. 14 is a circuit corresponding to Expression (2). An adder 303 in FIG. 14 is an adder for the addition of Expression (2). The multiplier 304 is a multiplier that uses the coefficient A as a gain value. The subtractor 305 is a subtracter for subtraction of Expression (2). The register 301 is composed of a flip-flop circuit. The register 301 is exemplified in the synchronous design by the clock. The logic element 302 is an AND gate for resetting the register 301. When the signal sig_306 becomes 1 value, the logic element 302 initializes the register 301 to 0 value in synchronization with the clock. The combinational logic 307 updates the register 301 with the value of the signal sig_211 when the signal sig_304 becomes one value. That is, the register 301 is updated to the value of the black level sig_211 of the register 211 when the signal sig_304 becomes one value. The combinational logic 306 updates the register 301 with the value calculated for the input sig_205 when the signal sig_306 = sig_304 = 0 and the signal sig_303 is one. Signals sig_205, sig_206, and sig_211 are equal to the signals in FIG. Signals sig_303, sig_304, and sig_306 are not shown in FIG. As described above, the filter circuit 206 outputs the black level sig_206 of the second parallax line 802 based on the difference sig_205 output from the subtractor 205 and the black level sig_211 stored in the register 211.

  FIG. 15 is a timing chart of the signals sig_205, sig_206, sig_2211, sig_303, sig_304, and sig_306. As described above, the filter circuit 206 in FIG. 14 is exemplified by a synchronous design using a clock. In the timing chart of FIG. 15, clock notation is omitted, but it is assumed that in-phase transfer synchronized with the clock is established for each signal.

  The signal sig_206 is an output signal of the register 301. The signal sig_206 is initialized (reset) at a clock cycle in which the signal sig_306 becomes one value. Further, the register 301 for storing the output value of the filter circuit 206 has a clock cycle in which the signal sig_304 becomes 1 value at the time of initialization and update of the register 211 (change of the gain value of the amplifier 44). Updated to the value of level sig — 211. The value of the signal sig_211 is an output value of the register 211 in FIG. The register 301 has a clock cycle when the signal sig_303 is 1 value, and is updated with a result value obtained by performing the calculation of the expression (2) between the value of the input signal sig_205 at that time and the current value of the register 301. .

  As described above, in the frame data 900, the number of normal lines 800 is larger than the number of second parallax lines 802. The accumulator 201, the divider 202, and the averaging means 203 output the black level sig_203 of the normal line 800 based on the plurality of normal lines 800 in the frame data 900. On the other hand, the delay element 215, the subtractor 205, and the filter circuit 206 output the black level sig_206 of the second parallax line 802 based on the second parallax line 802 of the plurality of frame data 900.

  The configuration of the averaging means 203 in FIG. 11 is not general and will be described with reference to FIG. FIG. 16 shows a configuration example of a digital circuit of the averaging means 203, and FIG. 17 shows a timing chart thereof. In this embodiment, the averaging means 203 is also designed to be synchronized with a clock common to the system (at least the clamp processing circuit 1041).

  The registers 501 to 508 have a shift register configuration, and update the black level estimated representative value when the detection processing for one line is completed when the detection processing for one line is completed. In this embodiment, the averaging means 203 is exemplified as a circuit configuration that calculates a moving average of 8 lines. A signal sig_202 in FIG. 16 is an output value of the divider 202 in FIG. A signal sig_203 in FIG. 16 is a vertical 8-line moving average value of the black level estimation value as an output of the averaging means 203.

  A signal sig_503 in FIG. 16 is an event signal after the completion of one line processing in the detection region 901, and indicates that the result value of the divider 202 is valid. A signal sig_504 in FIG. 16 is a counter value for selecting a result output of the shift registers 501 to 508 at the time of 8-line average calculation. A signal sig_505 in FIG. 16 is a status signal indicating a period during which the values of the shift registers 501 to 508 are integrated. A signal sig_506 in FIG. 16 is an event signal for updating the moving average result. The signal sig_507 is a reset signal that initializes the registers of the entire circuit. Signals sig_508 to sig_515 in FIG. 16 are holding values of the register values of the shift registers 501 to 508, respectively. A signal sig_516 in FIG. 16 is one selection result signal among the signals sig_508 to sig_515 of the shift register value. A signal sig_517 in FIG. 16 is an integration result value of the signals sig_508 to sig_515 of the shift register value. These signal values have a timing relationship as shown in FIG.

  Signals sig_503, sig_504, sig_505, sig_506, and sig_507 are not shown in FIG. 11, and are generated by the occurrence of an event from the state of the frame counter in the SSG 210 of FIG. Further, the reset signal sig_507 may generate a register (not shown) set value from the CPU 100 since it is sufficient to generate an event once before the start of frame processing.

  The signal sig_503 is an arbitrary one line completion event in the detection area 901 as described above. The signal sig_505 is a status signal for obtaining a period of 8 clock cycles after receiving the event of the sig_503. Further, while the signal sig_505 is in the one-value state, the counter value of the signal sig_504 is incremented by 1, and the output value selection switching of the selector 509 is performed. The counter (not shown) for generating the signal sig_504 and the generation logic of the sig_505 may be in the SSG 210 or in the clamp processing circuit 1041.

  The adder 510 in FIG. 16 is used to accumulate the values of the signals sig_508 to sig_515 in the register 511. The integration timing at this time is given by the signal sig_505 in FIG. 16 (see the signal sig_505 in FIG. 17). The register 511 is initialized (0 value) not only by the reset signal sig_507 of the entire circuit but also by the signal sig_503 (see the signal sig_517 in FIG. 17). The signal sig_517 of the integration result of the signals sig_508 to sig_515 stored in the register 511 is exemplified as a moving average for 8 lines in this embodiment. Therefore, the signal sig_517 is truncated to the lower 3 bits (“R3” in FIG. 16 indicates a right 3 bit shift) as a 1/8 operation, and is stored in the register 512 in FIG. 16 at the timing of the signal sig_506. (See signal sig_203 in FIG. 17).

  In the present embodiment, the filter circuit 206 is set to be implemented by an IIR filter. In this case, when the right end value of the OB pixel data 812 in the detection area 901 deviates from the line average (such as hitting a minute scratched pixel), the black level estimated value of the line is affected. A clamp processing circuit 1041 for avoiding such a case is shown in FIG. As shown in FIG. 19, the clamp processing circuit 1041 has a configuration in which the input of the filter circuit 206 is calculated by the difference between the black level estimated value of the normal line 800 and the line average of the second parallax line 802. . An integrator 219 in FIG. 19 is an integrator similar to the integrator 201 in FIG. A divider 220 in FIG. 19 is a divider similar to the divider 202 in FIG. The delay amount of the delay element 218 in FIG. 19 is set so that the sum of the delay amounts of the delay element 218, the integrator 219, the divider 220, and the filter circuit 206 is equal to the delay amount of the delay element 217 in FIG. Is done.

  According to the present embodiment, even in frame data processing including an offset step between the normal line 800 and the second parallax line 802, it is possible to follow the temperature characteristics of the offset step and perform stable offset correction. It is feasible. The clamp processing circuit 1041 estimates the black level sig_203 of the normal line 800 and the black level sig_206 of the second parallax line 802, respectively, and corrects the effective pixel data of the normal line 800 and the second parallax line 802. Thereby, generation | occurrence | production of a horizontal stripe can be suppressed. In addition, the clamp processing circuit 1041 can follow the temperature characteristics of the offset step by continuously performing the black level estimation of the second parallax line 802 based on signals obtained from many frames, and can stably Offset correction can be performed.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

DESCRIPTION OF SYMBOLS 1 Imaging device, 104 Signal processing circuit, 201 Accumulator, 202 Divider, 203 Averaging means, 204,207 Subtractor, 208 Adder, 211,212 Register, 213,214 Logic element (AND gate), 215,216 , 217 Delay element, 1041 Clamp processing means (clamp processing circuit)

Claims (17)

First line data including a plurality of first optical black pixel data and a plurality of first effective pixel data, a first line data including a plurality of second optical black pixel data and a plurality of second effective pixel data. A signal processing circuit for inputting frame data having two line data,
First black level estimation means for estimating a black level of the first line data based on a plurality of first optical black pixel data of the first line data;
Based on the plurality of second optical black pixel data of the second line data and the black level of the first line data estimated by the first black level estimation means, the second line data And a second black level estimating means for estimating the black level of the signal processing circuit.   Further, based on the black level estimated by the first black level estimating means, the plurality of first effective pixel data of the first line is corrected and estimated by the second black level estimating means. 2. The signal processing circuit according to claim 1, further comprising correction means for correcting a plurality of second effective pixel data of the second line based on a black level.   The second black level estimation means is a subtraction means for calculating a difference between the plurality of second optical black pixel data of the second line data and the black level estimated by the first black level estimation means. The signal processing circuit according to claim 1, further comprising:   4. The signal processing circuit according to claim 3, wherein the second black level estimating means includes a filter means for inputting a difference calculated by the subtracting means. The first black level estimation means includes:
Integrating means for integrating a plurality of first optical black pixel data of the first line;
Division means for dividing the data accumulated by the accumulation means by the number of the plurality of first optical black pixel data;
5. The signal processing circuit according to claim 1, further comprising: averaging means for averaging data divided by the dividing means for a plurality of first lines. 6. In the first line, the first effective pixel data corrected by the correcting unit is selected, and in the second line, the second effective pixel data corrected by the correcting unit is selected. Means,
The signal processing circuit according to claim 2, further comprising: a counting unit that counts an input amount of the frame data and controls selection of the selection unit according to the count value. Storage means for storing an initial black level or a black level output by the filter means;
The filter means initializes the output of the filter means to a black level stored in the storage means, and based on the output of the filter means and the difference calculated by the subtraction means, the second line The signal processing circuit according to claim 4, wherein a black level of data is estimated.   8. The signal processing circuit according to claim 1, wherein the frame data has a larger number of the first line data than a number of the second line data. 9. The first black level estimation means estimates a black level of the first line data based on a plurality of first line data in the frame data,
9. The signal processing circuit according to claim 8, wherein the second black level estimation means estimates a black level of the second line data based on second line data of the plurality of frame data. . A signal processing circuit according to claim 7;
A plurality of unit pixels for generating data corresponding to the frame data by photoelectric conversion;
Amplifying means for amplifying signals of the plurality of unit pixels,
The storage means stores a black level corresponding to the gain value of the amplification means when the gain value of the amplification means is changed;
The image pickup apparatus, wherein the filter means initializes the output of the filter means to a black level stored in the storage means when the gain value of the amplifying means is changed.   Further, when the gain value of the amplification unit is changed, the amplification unit is changed according to the black level stored in the storage unit and the black level according to the gain value before the amplification unit is changed. 11. The imaging apparatus according to claim 10, further comprising a control unit that adjusts a black level according to a later gain value and writes the adjusted black level in the storage unit.   The control means compensates according to the ratio between the black level stored in the storage means and the black level according to the gain value before the change of the amplification means, and the gain value after the change of the amplification means. 12. The imaging apparatus according to claim 11, wherein a coefficient is determined, and a black level that is a product of a black level corresponding to the gain value after the change of the amplification unit and the compensation coefficient is written in the storage unit. Each of the plurality of unit pixels has a plurality of photoelectric conversion means each for converting light into electric charge,
The plurality of first optical black pixel data, the plurality of first effective pixel data, the plurality of second optical black pixel data, and the plurality of second effective pixel data are converted into the plurality of photoelectric conversions. The data based on the charge of the means is synthesized data,
Furthermore, it has a read-out means which reads the said unit pixel corresponding to the said 1st line data, and the said unit pixel corresponding to the said 2nd line data by a different method. The imaging apparatus according to item 1.   In the unit pixel corresponding to the first line data, the reading unit reads only data obtained by combining data based on charges of the plurality of photoelectric conversion units, and the unit pixel corresponding to the second line data Then, data obtained by combining data based on charges of some of the plurality of photoelectric conversion means and data based on charges of the plurality of photoelectric conversion means is read out. Imaging device. First line data including a plurality of first optical black pixel data and a plurality of first effective pixel data, a first line data including a plurality of second optical black pixel data and a plurality of second effective pixel data. A signal processing circuit for inputting frame data having two line data,
First black level estimation means for estimating a black level of the first line data based on a plurality of first optical black pixel data of the first line data;
Second black level estimation means for estimating a black level of the second line data based on a plurality of second optical black pixel data of the second line data;
In the frame data, the number of the first line data is larger than the number of the second line data,
The first black level estimation means estimates a black level of the first line data based on a plurality of first line data in the frame data,
The signal processing circuit, wherein the second black level estimation means estimates a black level of the second line data based on a plurality of second line data of the frame data. First line data including a plurality of first optical black pixel data and a plurality of first effective pixel data, a first line data including a plurality of second optical black pixel data and a plurality of second effective pixel data. A signal processing method for processing frame data having two line data,
Estimating a black level of the first line data by first black level estimating means based on a plurality of first optical black pixel data of the first line data;
Based on a plurality of second optical black pixel data of the second line data and the estimated black level of the first line data by the second black level estimation means, the second line And a method of estimating a black level of data. First line data including a plurality of first optical black pixel data and a plurality of first effective pixel data, a first line data including a plurality of second optical black pixel data and a plurality of second effective pixel data. A signal processing method for processing frame data having two line data,
In the frame data, the number of the first line data is larger than the number of the second line data,
Estimating a black level of the first line data based on a plurality of first optical black pixel data of a plurality of first line data in the frame data by a first black level estimating means; When,
Estimating a black level of the second line data based on a plurality of second optical black pixel data of the second line data of the plurality of frame data by a second black level estimation means; A signal processing method comprising:

Images