JP2004159274A - Solid-state imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To eliminate the trouble that an S/N is lowered when a dynamic range is widened with respect to an output of an image sensor in a solid-state imaging unit. <P>SOLUTION: A low illuminance signal based on the long-time accumulation of photoelectric charges and a high illuminance signal based on the short-time accumulation of the photoelectric charges are extracted from a pixel part of the image sensor. Thus, the saturation of photoelectric charges in high illuminance are prevented to widen the dynamic range, and the high illuminance signal is integrated (cumulatively added) outside an imaging area to improve the S/N. Besides, the integration is performed with an analog+digital hybrid scheme to further improve the S/N. <P>COPYRIGHT: (C)2004,JPO

Description

ここで、Ｔは、飽和検出のしきい値である。 Ｔ＝ｒとしても良いが、Ｔ＝ｒ＋Δ（Δ＞０）とし、Ｔをｒよりやや大きい値にすることで、ｙ_ｉが負になるのを避けることができる。
【００２３】
またディジタル積分値Ｙ_ｉは、
Ｙ_ｉ＝Ｙ_ｉ−１＋Ｄ_ｉ
となる。 ここでＹ_０＝０である。
このような処理によれば、ｙ_ｉ＜Ｔを必ず満たし、アナログ加算器の出力が飽和することはない。
【００２４】
また、等価な信号振幅は、
【数２】

いま、Ｎ回積分する場合の、等価な最大信号振幅は、ｙ_Ｎの最大信号振幅が、ｒであるとすれば、
Ｚｍａｘ＝（Ｎ＋１）ｒ
となる。 これは、読み出される信号が、毎回最大値としてｘ_ｉ＝ｒ（１＋（１／Ｎ））となる場合に相当し、このようなハイブリッド積分処理により、ダイナミックレンジが、Ｚｍａｘ／ｘ_ｉ＝Ｎと、Ｎ倍に拡大されることになる。
【００２５】
このようなハイブリッド積分処理を、短時間蓄積により高照度側のダイナミックレンジ拡大を行う方法に応用すると、低照度信号から高照度信号に切り替わったときのＳＮＲのディップ量を小さくしながら、ダイナミックレンジ拡大が可能となる。 高照度用信号の蓄積時間をＴ_ＩＨとし、その読み出された信号をＮ回、ハイブリッド積分を行ったとき、信号はＮ倍、ショット雑音は√Ｎ倍になるので、ある照度レベルにおいて、低照度信号から高照度信号に切り替わったときのＳＮＲのディップ量は、２０ｌｏｇ_１０√（Ｎ×Ｔ_ＩＨ／Ｔ_ＩＬ）［ｄＢ］となる。 Ｔ_ＩＨ／Ｔ_ＩＬ＝１／２５６で、Ｎ＝３２とすれば、ＳＮＲディップは−９［ｄＢ］ですむ。 このような処理を行わなければ、−２４［ｄＢ］であり、この差は非常に大きい。
【００２６】
例えば、低照度用出力での最大のＳＮＲは、一般には５０ｄＢ前後である。 いま仮に５０ｄＢのＳＮＲのところで低照度から高照度に切り替わったとすると、ハイブリッド積分処理を行う場合は、ＳＮＲディップ点のＳＮＲは４１［ｄＢ］となるが、このような処理を行わない場合は、ＳＮＲディップ点のＳＮＲは、２６［ｄＢ］となる。 人の眼は、ＳＮＲが４０［ｄＢ］近くあれば、殆どの場合ノイズに気がつかないといわれており、ハイブリッド積分処理により殆ど問題のないレベルに抑えることができる。これに対して、２６［ｄＢ］のＳＮＲでは、人の眼にはっきりとわかる画質劣化を生じる。
【００２７】
次に、ハイブリッド積分を行う実際の回路を説明する。 図５にその回路図を示す。 メモリキャパシタを順次、バッファアンプに接続して、これに記憶された電圧とセンサから読み出された信号を加え、これを再びメモリキャパシタに書き込む機能を備えるとともに、アナログ加算器の出力が、しきい値を越えたかどうかを比較器で判断し、越えた場合は、ある一定電圧を引いた値をメモリキャパシタに書き込む。
【００２８】
具体的な動作を説明するための回路を図６に示す。 このメモリは、６つの動作状態をもち、そのときの各スイッチのオン（１）、オフ（０）を図７に、それぞれの状態における回路接続を、図８に示す。
【００２９】
（１） 最初の画素リセットレベルサンプル；
高照度信号を読み出してハイブリッド積分を行う際の最初の読み出しのときはアナログメモリにはなにも書こまれていないので、アナログメモリ出力との加算は行わず、画素部の信号をアナログメモリに記憶する動作となる。 そこで、４トランジスタ＋１フォトダイオードの画素回路をもつ画素部の電荷検出部をリセットし、これを読み出す。 これをＶｓ（Ｒ）とする。 この電圧を、アナログ加算器のキャパシタＣ_１にサンプルする。 このときＣ_２の一端は、参照電圧Ｖ_Ｒに接続しておく。
【００３０】
（２） 最初の画素信号レベルサンプル；
４トランジスタ＋１フォトダイオードの画素回路をもつ画素部の電荷検出部に、ＴＸを開いて蓄積した電荷を転送し、このときの電荷検出部の電圧を読み出す。
これをＶｓ（Ｓ）とする。 このときＣ_２は、アナログ加算器の入出力間に接続する。 Ｃ_１への入力の電圧がＶｓ（Ｒ）からＶｓ（Ｓ）に変化することにより、Ｃ_１（Ｖｓ（Ｒ）−Ｖｓ（Ｓ））に相当する電荷がＣ_２に転送される。 その結果、アナログ加算器の出力は、
【数３】

となる。 これにより、加算器の出力には画素部の固定パターンノイズとリセットノイズが除去され、かつアナログ加算器のオフセットにも影響されない電圧が現れる。 これをメモリキャパシタに記憶する。
【００３１】
（３）アナログメモリ読み出しと２回目以降の画素部リセットレベルサンプル；
２回目以降は、高照度信号を読み出してハイブリッド積分を行うので、メモリ出力との加算を行う。 そこで、４トランジスタ＋１フォトダイオードの画素回路をもつ画素部の電荷検出部をリセットしたときの読み出し電圧Ｖｓ（Ｒ）を、アナログ加算器のキャパシタＣ_１にサンプルする。 このときＣ_２の一端は、アナログメモリの読み出すバッファアンプの出力を接続しておく。 その電圧をＶ_Ｍとする。
【００３２】
（４） ２回目以降の画素部信号レベルサンプルとメモリ出力との加算；
画素の電荷検出部に、ＴＸを開いて蓄積した電荷を転送し、このときの電荷検出部の電圧を読み出す。 これをＶｓ（Ｓ）とする。 このときＣ_２は、アナログ加算器の入出力間に接続する。 Ｃ_１への入力の電圧がＶｓ（Ｒ）からＶｓ（Ｓ）に変化することにより、Ｃ_１（Ｖｓ（Ｒ）−Ｖｓ（Ｓ））に相当する電荷がＣ_２に転送される。 その結果、アナログ加算器の出力は、
【数４】

となる。 これにより、アナログメモリに記憶された電圧と、画素部から読み出され、固定パターンノイズとリセットノイズが除去された新たな信号が加算される。
その加算出力に対して、比較器を用いて、しきい値電圧Ｖ_Ｔよりも大きいかどうかを判断する。
【００３３】
（５） 参照信号減算のための第１の参照信号サンプル；
上の（４）の演算の結果、加算器の出力が、しきい値電圧よりも大きいときは、ある一定電圧を、その出力から減ずる。 これを行うために、Ｃ_１に、第１の参照電圧Ｖ_Ｒ１をサンプルする。 このとき、アナログ加算器用の増幅器の入出力間を短絡する必要があるが、（４）の処理の結果得られ、Ｃ_２に記憶されている信号を破壊しないようにするために、Ｃ_２の一端を開放しておく。
【００３４】
（６） 参照信号減算のための第２の参照信号サンプルとアナログメモリへの再書き込み；
Ｃ_２をアナログ加算器の入出力間に接続する。 比較器の出力Ｄが１の（つまり、アナログ加算器出力が、しきい値電圧よりも大きい）場合は、Ｃ_１の一端を第２の参照電圧Ｖ_Ｒ２に接続し、比較器の出力Ｄが０の場合は、Ｃ_１の一端を第１の参照電圧Ｖ_Ｒ１のままにしておく。 この結果、アナログ加算器の出力は、次のようになる。
【数５】

このように、比較器の出力に応じて一定電圧（Ｖ_Ｒ１−Ｖ_Ｒ２）を減ずる処理が行われる。
この値をアナログメモリ用キャパシタに記憶する。
【００３５】
比較器はそれ自体１ビットのＡ／Ｄ変換器であるが、この比較器を多ビットのＡ／Ｄ変換器で構成する場合には、アナログ加算器出力がしきい値電圧を上回る度合いに対応して、Ｄは１，２…ｎの値をとる。
【００３６】
以上をＮ回繰り返せば、短時間蓄積をした画素部の信号に対して、先に説明したハイブリッド積分を行うことができる。 アナログメモリから減算された信号量は、比較器の出力を積分することにより得られる。 比較器の出力の積分は、ディジタルメモリとディジタル加算器を用いて行うことができ、その機能記述は、図１に示したブロック図で十分と考えられるので、詳細な説明は省略する。
なお、ハイブリッド積分型メモリのアナログ部は、メモリ部のスイッチの数を減らした図９のような、幾つかの変形が可能である。
【００３７】
本発明のもう１つの重要な点は、画素部からの信号の読み出しタイミングである。 これは、必要とされるハイブリッド積分型メモリ部の回路規模と大きな関係がある。
【００３８】
図４は、高照度信号の蓄積時間と、フレーム周期Ｔ_Ｆとの比がＮ×Ｔ_ＩＨ／Ｔ_Ｆ＝１／４の場合である。 高照度信号の蓄積は、画素部での蓄積とメモリを用いた積分による信号蓄積の両方を用いるが、この期間は高速な信号読み出しがなされるため、電源ライン、ＧＮＤライン、イメージセンサの基板に大きなスイッチングノイズを発生する。 この期間に信号読み出しを重複して行うと読み出された信号にスイッチングノイズが混入する。 したがって、この期間と信号読み出しは、時間的にずらす必要がある。 その結果、図４のようなタイミングになる。
【００３９】
このことを満たしながら、信号読み出しに要する時間を２倍にし、必要な高照度信号積分用メモリの量を半分にすることができる読み出しタイミングを図１０に示す。 画素部の偶数番目の水平ライン（２ｎ）と奇数番目の水平ライン（２ｎ＋１）とで、図１０のように信号蓄積及び信号読み出しのタイミングを設定すれば、ノイズ混入を避けながら、積分用メモリ量と読み出し速度を半分にすることができる。
【００４０】
同様に、高照度信号の蓄積時間と、フレーム周期Ｔ_Ｆとの比がＮ×Ｔ_ＩＨ／Ｔ_Ｆ＝１／８の場合には、図１１のように信号蓄積と信号読み出しのタイミングを設定すれば、ノイズ混入をさけながら、（１／４の画素数分毎に積分して読み出すので）積分用メモリ量を画素数の１／４にでき、読み出し速度もこの動作を用いない場合に比べて１／４にすることができる。
一般に、高照度信号の蓄積時間と、フレーム周期Ｔ_Ｆとの比をＮ×Ｔ_ＩＨ／Ｔ_Ｆ＝１／Ｍにすることで、積分用メモリ量を画素数の２／Ｍに減らすことができる。
【００４１】
高照度信号と低照度信号を得るもう１つの方法として、これらを別々の画素で得る方法も考えられる。 この場合の画素配置の例を図１２に示す。 （ａ），（ｂ）は、高照度用の画素を少なくして、低照度領域の解像度をあまり下げないようにしたもので、高照度領域では、やや解像度が低下しても良い場合もあり、そのような場合に有効である。 また、（ａ）は高照度画素の幅を低照度画素の２画素分とし、かつ偶数行と奇数行とで半画素分ずらしたものである。 このような配置にすれば、偶数行の画素と奇数行の画素とから単純な補間演算（平均演算）を用いることにより、低照度画素に対応する位置における照度を高照度画素からも簡単に求めることができる。 半画素ずらしによらない場合は、複雑な補間演算となることが避けられない。 （ｃ），（ｄ）は、高照度用と低照度用で同じ解像度としたもので、（ｃ）はチェックパターンにしたもの、（ｄ）はライン毎に交互に配置するものである。 （ｃ）の方が、水平と垂直の解像度が同じにできる点で有利である。これ以外にも種々の変形が考えられるものであり、他の配置であってもこの出願による積分処理が適用可能であることはいうまでもない。
【００４２】
このように高照度用と低照度用で別の画素を用いると解像度が低下するが、現在技術的には、１００万画素を超えるイメージセンサを実現するのはそれほど困難ではなく、一方、解像度はそれほど高くなくて良いが、ダイナミックレンジは広くしたいというニーズはあると思われる。 故に、多画素化の技術をダイナミックレンジを広くするために用いるという方法も実用性は十分にある。
【００４３】
次にディジタル積分方式を説明する。 最も単純なディジタル積分方式の広ダイナミックレンジ・イメージセンサの構成を図１３に示す。 読み出された高照度用の信号をＡ／Ｄ変換し、画素毎に、あらかじめ記憶されたディジタルメモリの値と画素部から読み出された信号を加え、これを再びディジタルメモリに格納する。 これを繰り返す。 この場合には、十分なビット幅のディジタルメモリを用い、飽和を避けることができる。 ディジタルメモリとしては、図１１のようにバースト読み出しにより、高照度信号の蓄積時間と、フレーム周期Ｔ_Ｆとの比をＮ×Ｔ_ＩＨ／Ｔ_Ｆ＝１／Ｍにすることで、積分用メモリ量を画素数の２／Ｍに減らすことができる。 このバースト読み出しを用いることでメモリの量を減らしながら、ディジタル積分によりダイナミックレンジ拡大を行う方法を、ディジタル積分方式における一実施例として提案する。
【００４４】
ただし、現実には、このような単純な構成では、１つ問題がある。 Ａ／Ｄ変換器の分解能を十分高くしないと、Ａ／Ｄ変換器の発生する量子化雑音によりディジタル積分された信号のＳＮＲが低下することである。 例えば、高照度部の蓄積時間を１フレームの１／８とし、３２回高速読み出しを行ってディジタル積分を行う場合、低照度用信号の蓄積時間をＴ_ＩＬ、高照度用信号の蓄積時間をＴ_ＩＨとするとき、ある照度レベルにおいて、低照度信号から高照度信号に切り替わったとき、低照度信号の最大値を１Ｖとすれば、Ｔ_ＩＨ／Ｔ_ＩＬ＝１／２５６であるので、高照度信号の最小値は、４ｍＶである。
【００４５】
もし、高照度信号のＡ／Ｄ変換を１Ｖフルスケールで８ビットで行ったとすると、高照度信号の最小値は、１ＬＳＢ（最下位ビット）であり、３２回読み出しを行ってディジタル積分を行っても分解能は、５ビットである。 量子化雑音で決まるＳＮＲは、３２ｄＢ程度になる。 これを１０ビットＡ／Ｄ変換を行えば、高照度信号のＳＮＲの最小値は、４４ｄＢになり、問題のない値とすることができるが、１０ビットのＡ／Ｄ変換器をイメージセンサのカラムに並べて動作させ、１０ビット精度を確保するのは困難さがともなう。 特に、高照度信号は、この場合の例で、通常の３２倍の高速で読み出さないといけないため、高速かつ高精度のＡ／Ｄ変換器をカラムに並べて動作させる必要がある。
また、１０ビットＡ／Ｄ変換を行えたとしても、３２回ディジタル積分を行うと１つの画素値を記憶するのに１５ビットのメモリが必要になる。
これらの点を改善したディジタル積分方式におけるカラムに並べる単位回路を、図１４及び図１７に示す。
【００４６】
図１４は、Ａ／Ｄ変換を容易にし、かつＳＮＲを大きく低下させないようにするため、積分型Ａ／Ｄ変換器において複数の傾斜を発生するランプ信号発生器を用いて、多段階分解能のＡ／Ｄ変換を行うものである。 以下に動作を説明する。 積分型Ａ／Ｄ変換器は、１個の比較器、ランプ信号発生器とこれに同期して動作するカウンタ及びカウント値を記憶（ラッチ）するメモリからなる。 ０Ｖから徐々に電圧が上昇するランプ信号を発生させ、ランプ信号のスタートと同時にディジタル値がインクリメントするカウンタを動作させる。
【００４７】
このときランプ信号とＡ／Ｄ変換したい入力信号を比較器に与え、ランプ信号が、入力信号を超えた瞬間に比較器が”０”から”１”に変化する。 この比較器の出力によって、ラッチ回路にカウンタの値を記憶する。 この記憶されたディジタル値が、入力されたアナログ信号がＡ／Ｄ変換された値となる。 いまカウンタのクロックの周期をＴ_ｃ、入力電圧のフルスケールをＶ_ＦＳとし、Ｎビット分解能のＡ／Ｄ変換を行う場合、ランプ信号の傾斜ｓ＝ｄＶ／ｄｔを
【数６】

に選ぶ。 アナログ入力電圧をＶ_１のとき、レジスタに記憶されるカウント値をＮ_１とすると、
【数７】

となり、カウント値がＡ／Ｄ変換値になることがわかる。 １ＬＳＢ（最下位ビット）の電圧振幅は、１ＬＳＢ＝Ｖ_ＦＳ／２^Ｎである。
【００４８】
これを例えば１０ビット分解能で行うと最大で１０２４回のカウントが必要になり、高速Ａ／Ｄ変換が困難になる。 そこで、小さい入力信号のときは、高い分解能（１ＬＳＢあたりの電圧を小さく）で、大きな入力信号のときは低い分解能でＡ／Ｄ変換を行う。 これを多段分解能Ａ／Ｄ変換と呼ぶ。
第１段階では、最高の分解能のＡ／Ｄ変換を行う。 このときのランプ信号の傾斜をＳ_１とする。 第２段階は、Ｓ_２＝Ｍ_２×Ｓ_１（Ｍ_２は正の整数）、第３段階は、Ｓ_３＝Ｍ_３×Ｓ_１（Ｍ_３は正の整数）のように段階的に傾斜を大きくすることで、分解能を変化する。
【００４９】
実際の動作の様子を、図１５に示す。 入力信号がＡのときには、１カウントあたり１ＬＳＢの傾斜でランプ信号が変化し、第１段階で比較器の出力が”０”から”１”に変化し、カウンタ値６がデータラッチに記憶される。 同時に、ゾーンラッチに、領域Ａであることを示す１が記憶される。 同様に入力信号がＢのときは、１カウントあたり２ＬＳＢの傾斜でランプ信号が変化し、第２段階で比較器の出力が”０”から”１”に変化することで、カウント値７がデータラッチに記憶される。 同時にゾーンラッチには領域Ｂであることを示す２が記憶される。 このようにして、入力信号に応じて分解能が変化するＡ／Ｄ変換が行える。例えば、１０ビット積分型Ａ／Ｄ変換を行うためには、１０２４クロック分カウントする必要があるが、これを図１６のように多段階分解能とすることで、６４カウントに削減できる。
【００５０】
例えば、先に述べた高照度信号の蓄積時間を１／８とし、ディジタル積分の回数を３２回とする場合、低照度信号の最大値を１Ｖとすると、高照度信号の振幅の最小値は、４ｍＶに相当するが、図１６のようにすれば４ｍＶの振幅のときは、１ＬＳＢ＝１ｍＶの分解能でＡ／Ｄ変換されるので、ディジタル値は４に相当し、３２回積分したときのディジタル値は、１２８と７ｂｉｔ相当となり、量子化雑音によるＳ／Ｎは積分によるノイズ増加を考慮しても、３８ｄＢ程度あり、十分である。また、振幅１６ｍＶを少し越えるとＡＤＣ（Ａ／Ｄ変換器）の分解能は、４ＬＳＢ＝４ｍＶとなるが、ディジタル値は４を越え、３２回積分したときのディジタル値は１２８以上となる。 同様に、振幅が、６４ｍＶを越えると、１６ＬＳＢ＝１６ｍＶの分解能、２５６ｍＶを越えると６４ＬＳＢ＝６４ｍＶの分解能となるが、どの領域でも量子化雑音によるＳ／Ｎは、３８ｄＢ程度以上を確保できる。
【００５１】
複数の傾斜のランプ信号を用いるかわりに、イメージセンサの画素部からの信号を増幅するアンプの利得を多段階に変化することで、多段分解能Ａ／Ｄ変換を行うこともできる。 これを用いた、ディジタル積分方式におけるカラムに並べる単位回路を、図１７に示す。 この場合のアンプの利得の設定例を図１８に示す。
【００５２】
先に複数の傾斜のランプ信号は時間の経過により順次発生するものとして説明したが、同時に複数の傾斜のランプ信号を発生させ、かつ複数の比較器を備えることにより、並列動作を行えることは明らかである。 また、アンプの利得を多段階に変化させる構成においても、時系列的にアンプの利得を順次変化させることに代えて利得の異なるアンプを複数設け、それらのアンプの出力と一つの傾斜のランプ信号とを複数の比較器を用いて同時に判定することにより、並列動作を行えることも明らかである。
【００５３】
ここで、ディジタルメモリを節約するための浮動小数点ディジタルメモリと、ディジタル積分回路について説明する。 浮動小数点表現では、ディジタル値を指数部と仮数部を用いて表す。 例えば、１５ｂｉｔのディジタル値を、仮数部８ビット、指数部３ビットで表せば、１１ビットで記憶することができる。 この例で浮動小数点表現への変換を表すと図１９のようになる。 仮数部Ｙ_Ｍの［Ｘ／２］は、Ｘを２で割って８ビットに丸めることを意味する。 指数部Ｙ_Ｅは、０から７までの値をとり、３ビットで表すことができる。 また、浮動小数点から固定小数点への変換は、
【数８】

を計算する。 これは単に、Ｙ_ＭをＹ_Ｅビットだけ桁の移動（ビットシフト）を行うだけで算出できる。
【００５４】
また、イメージアレイ周辺に設けられたノイズキャンセル回路や増幅器は、必要性に応じて設けられるものであるが、図１では低照度信号用と高照度信号用とにそれぞれ設けた例が示されている。 しかし、低照度信号と高照度信号とは読出しのタイミングが異なるため、図２０のように低照度信号用と高照度信号用とは兼用することが可能である。 ディジタル積分型においては、図１３にＡ／Ｄ変換器を低照度信号用と高照度信号用とにそれぞれ設けた例が示されている。 しかし、この場合のＡ／Ｄ変換器も図２１のように低照度信号用と高照度信号用とが兼用できる。
【００５５】
さらに他の実施例について、ここで述べる。
以上述べたように読み出した信号を外部の回路で積分を行う場合の問題は、画像信号を記憶するためのメモリの量である。
画像信号を画素毎に積分処理をするためには、１フレームに相当するメモリを必要とすると考えるのが一般的である。しかし、これをイメージセンサ上に集積化すると大きな面積を占めることになり、製造コストが高くなる。
この実施例の特徴は、短時間蓄積した信号の読み出し方を工夫することで、非常に少ないメモリで積分を行うことができる点である。
【００５６】
簡単のため、垂直有効画素数４８０画素のイメージセンサで、高照度信号の蓄積をほぼ１水平周期の期間、信号蓄積し、これを２回ずつ読み出して積分する場合について、タイミングを図示したものを図２２に示す。
動作の基本は、１水平期間の間に、低照度信号の垂直読み出し、高照度信号の垂直読み出し及び積分、積分用メモリからの信号の水平方向読み出しを行うことである。ある１水平ラインに着目したタイミングを図２３に示す。また、広ダイナミックレンジ・イメージセンサのブロック図を、図２４に示す。
【００５７】
低照度信号をＬＡの期間蓄積し、ＬＶのタイミングで垂直方向に読み出し、カラムでノイズキャンセル処理を行ったあと、Ａ／Ｄ変換し、ディジタルメモリに記憶する。このとき、もともと入っていたディジタルメモリの値との演算は行わないため、ＭＵＸ（マルチプレクサ）で、加算器の出力ではなくＡ／Ｄ変換器の直接の出力をメモリに接続する。このとき、信号がＡ／Ｄ変換器のフルスケールを越えていたかどうかを表す飽和検出フラグも合わせてディジタルメモリに記憶しておく。この読み出し動作で画素はリセットされる。
【００５８】
次いで高照度用短時間蓄積の１回目をＨＡ（１）の期間に行う。この信号は、ＨＶ（１）のタイミングで垂直方向に読み出し、ノイズキャンセルとＡ／Ｄ変換を行う。
ディジタルメモリに記憶された飽和検出フラグをチェックし、もし低照度信号がＡ／Ｄ変換器のフルスケールを越えていたら、ディジタルメモリの内容を高照度用信号で書き換え、飽和検出フラグはそのまま記憶しておく。つぎに高照度用短時間蓄積の２回目をＨＡ（２）の期間に行う。この信号をＨＶ（２）のタイミングで垂直方向に読み出し、ノイズキャンセルとＡ／Ｄ変換を行う。ディジタルメモリに記憶された飽和検出フラグをチェックし、もし低照度信号がＡ／Ｄ変換器のフルスケールを越えていたら、ＨＡ（１）の蓄積で記憶された信号と加算し、再び、ディジタルメモリに格納する。続いて、ＭＲの期間にディジタルメモリに記憶された値を、水平方向走査を行って出力に読み出す。
【００５９】
本方式の大きな特徴として、積分のために必要な記憶容量が非常に少なくて済むことである。これは、高照度信号の積分が低照度信号、高照度信号の読み出しに同期して規則的かつシーケンシャルに行われるため、ある１水平ラインに関して、高照度信号の積分が、完了して、外部に読み出しを行った後、直ちに、その積分のために用いたメモリを開放し、別の水平ラインの積分用に利用できるようになるためである。
【００６０】
いま、垂直方向のライン数（水平ライン数）をＮ_Ｖ、水平方向の画素数をＮ_Ｈ、ダイナミックレンジ拡大率をＲ、高照度信号の積分回数をＮ_Ｉ、１水平ラインの時間をＴ_Ｈ、フレーム速度をＴ_Ｖとすると以下のような基本的な関係式が得られる。
【数９】

【００６１】
簡単のためブランキング期間はＮ_Ｖの中に含めている。ダイナミックレンジ拡大率は、
【数１０】

【００６２】
ここでＴ_ＡＬは低照度信号の蓄積時間、Ｔ_ＡＨは高照度信号の蓄積時間である。また、Ｔ_ＡＨをＴ_Ｈを基準にして表すと
【数１１】

ここでＮ_ＡＨは高照度信号の蓄積時間を水平ラインの時間で割ったもの、すなわち高照度信号蓄積時間とＴ_Ｈの比を表したものである。
【００６３】
今回の方式は、１フレームの時間を無駄なく、低照度信号と高照度信号の蓄積に１００％割り当てられ、次式がなりたつ。
【数１２】

つまり、１フレームの時間を低照度信号の蓄積と、Ｎ_Ｉ回の高照度信号の蓄積に割り当てる。
【００６４】
広ダイナミックレンジ・イメージセンサの設計上、Ｎ_Ｖ、Ｒ、Ｎ_Ｉが仕様として与えられるとする。これらに対して、イメージセンサの性能を決める幾つかのファクタについて計算式を求める。
まず式（３），（４）から、次式が得られる。
【数１３】

【００６５】
広ダイナミックレンジ・イメージセンサでは、蓄積時間の一部を高照度用に利用するため、低照度信号の感度がやや低下する。１フレームの時間蓄積する通常のイメージセンサと比較した場合の感度低下率をγとすると
【数１４】

となる。ダイナミックレンジ拡大率を十分大きくとり、積分回数を増やさなければ低下は少ない。
【００６６】
低照度領域から高照度領域への切替時のＳＮＲディップ量（ΔＳＮＲ）は、光ショットノイズのみを考慮すると
【数１５】

ＳＮＲ低下は、ダイナミックレンジを拡大するほど低下し、また積分回数を増やせば改善されることがわかる。
【００６７】
積分に必要なメモリ量Ｃ_Ｍは、
【数１６】

ここで、Ｃ_ＭＦは、１フルフレームを記憶するのに必要なメモリ量であり、
Ｃ_ＭＦ＝Ｎ_Ｂ×Ｎ_Ｈ×Ｎ_Ｖである。Ｎ_Ｂは、１画素あたりのビット幅である。メモリ量は、ダイナミックレンジを拡大する程、少なくて済み、ＳＮＲディップを犠牲にして、積分回数を減らせばメモリ量が節約できることになる。
【００６８】
画素数６４０×４８０画素の場合での計算例を図２５に示す。
図２５のように、１水平読み出し周期の８倍の期間蓄積し、これを４回積分する場合の１水平周期での処理の時間的内訳を、図２６に示す。
あるｎ行目の低照度用信号を最初に読み出し、Ａ／Ｄ変換等の処理を経て、ディジタルメモリに記憶する。ついで、その行から８行手前の信号を読み出し、Ａ／Ｄ変換等の処理を経て、もし、ｎ−８行目のディジタルメモリに書かれてある低照度用信号の飽和フラグがたっていれば、この信号で置き換える。次に、その行から１６行手前の信号を読み出し、Ａ／Ｄ変換等の処理を経て、もし、ｎ−１６行目のディジタルメモリに書かれてある低照度用信号の飽和フラグがたっていれば、既に、１回目の高照度用信号で置き換えられているため、この信号を、記憶されている高照度用信号に加えて、その結果を、当該行のディジタルメモリに再書き込みをする。
【００６９】
以下同様に、順次２４行手前の信号、３２行手前の信号を読み出し、同様な処理を行う。３２行目手前の信号を読み出し積分すれば、その行に関しては値が確定するので、メモリに格納されている信号を、最後の期間を使って、水平走査により読み出す。
【００７０】
拡張として、図２７のように、長時間蓄積した低照度信号の読み出し、比較的短時間蓄積した高照度信号の読み出しと積分（この場合８Ｈ期間の積分を４回）、その後、極短時間積分した超高照度信号の読み出しと積分（この場合２Ｈの期間の積分を２回）を１水平周期の間に行うことで、さらに高ダイナミックレンジ化を図ることも考えられる。この場合、フラグとしては、メモリに記憶されている信号が、低照度用信号であるか、高照度信号であるか、超高照度信号であるかを区別するためのフラグとして２ビット分必要である。
このように飽和フラグを用いることにより、低照度信号、高照度信号および超高照度信号から飽和していない適切な照度信号が選択される。
ここでは、超高照度信号についても積分を行うものとして説明したが、所望により積分なしで出力するものであってもよい。
【００７１】
次に、Ａ／Ｄ変換部の他の実施例について説明する。図２８に、その構成を示す。一点鎖線のなかは、カラムに並べる１チャネル分を示しており、その外の多傾斜ランプ信号発生器、多段分解能Ｇｒａｙコードカウンタ、制御回路は共通である。Ａ／Ｄ変換を容易にし、かつＳＮＲを大きく低下させないようにするため、積分型Ａ／Ｄ変換器において複数の傾斜を発生するランプ信号発生器を用いて、多段階分解能のＡ／Ｄ変換を行う。以下に動作を説明する。
【００７２】
積分型Ａ／Ｄ変換器は、１個の比較器、ランプ信号発生器とこれに同期して動作するカウンタ及びカウント値を記憶（ラッチ）するメモリからなる。０Ｖから徐々に電圧が上昇するランプ信号を発生させ、ランプ信号のスタートと同時にディジタル値がインクリメントするカウンタを動作させる。このときランプ信号とＡ／Ｄ変換したい入力信号を比較器に与え、ランプ信号が、入力信号を超えた瞬間に比較器が”０”から”１”に変化する。この比較器の出力によって、ラッチ回路にカウンタの値を記憶する。この記憶されたディジタル値が、入力されたアナログ信号がＡ／Ｄ変換された値となる。このカウンタとしては、ディジタルノイズがＡ／Ｄ変換特性に与える影響を減らすため、全てのコードで遷移するビットが１ビットだけであるグレイコードを用いるのがよい。
【００７３】
カウンタを用いた積分型Ａ／Ｄ変換器は、例えば１０ビット分解能で行うと最大で１０２４回のカウントが必要になり、高速Ａ／Ｄ変換が困難になる。そこで、小さい入力信号のときは、高い分解能（１ＬＳＢあたりの電圧を小さく）で、大きな入力信号のときは低い分解能でＡ／Ｄ変換を行う。これを多段分解能Ａ／Ｄ変換と呼ぶ。第１段階では、最高の分解能のＡ／Ｄ変換を行う。このときのランプ信号の傾斜をｓ_１とする。第２段階は、ｓ_２＝Ｍ_２×ｓ_１（Ｍ_２は正の整数）、第３段階は、ｓ_３＝Ｍ_３×ｓ_２（Ｍ_３は正の整数）のように段階的に傾斜を大きくすることで、分解能を変化させる。
【００７４】
図２９に多傾斜ランプ発生器の傾斜の設定例と対応するカウント値（ディジタルコード）の関係を示す。入力信号がＡのときには、１カウントあたり１ＬＳＢの傾斜でランプ信号が変化し、第１段階で比較器の出力が”０”から”１”に変化し、カウンタ値６がデータラッチに記憶される。同様に入力信号がＢのときは、１カウントあたり２ＬＳＢの傾斜でランプ信号が変化する。このとき、カウント値も１カウントあたり、２ずつ増えるようにする。また、入力信号がＣのときは、１カウントあたり、４ＬＳＢの傾斜でランプ信号が変化する。このときはカウント値が４ずつ増えるようにする。
【００７５】
これに対応するグレイコードの例を、図３０に示す。このように、ランプ信号の傾斜の変化を２のべき乗に選び、分解能に応じてグレイコードをビットシフトした特殊なグレイコード（図３０）を用いれば、どの傾斜の領域でＡ／Ｄ変換されたかを意識することなくＡ／Ｄ変換が行えるとともに、遷移するビットが１ビットだけであるというグレイコードがもつ性質をあわせもつことができる。
【００７６】
１０ビットＡ／Ｄ変換を行う場合の、多段分解能ＡＤＣの設定例を図３１に示す。
多段分解能を採用しない場合には１０２４クロック分カウントする必要があるが、これを図３１のように多段階分解能とすることで、１１２カウントに削減できる。
なお、低照度信号用のＡ／Ｄ変換と高照度信号用のＡ／Ｄ変換では、傾斜の設定を変更する。低照度の場合は、線形性の良いＡ／Ｄ変換を保証するため単一傾斜にすることも考えられる。
【発明の効果】
以上述べたように、この出願の構成によれば撮像素子から得られた信号処理においてダイナミックレンジの広い信号処理が可能となる。 また、高照度信号を積分することによりＳＮＲ（信号対ノイズ比）を高めることもできる。 さらに、特定の画素回路を対象とするものではないので、埋め込みフォトダイオードなどを用いた高画質な画素回路に適用できる。
【図面の簡単な説明】
【図１】広ダイナミックレンジ・イメージセンサの一実施例
【図２】４トランジスタ画素回路
【図３】３トランジスタ画素回路
【図４】低照度信号と高照度信号の蓄積と読み出しのタイミング
【図５】ハイブリッド積分型メモリのアナログ部の回路図
【図６】ハイブリッド積分型メモリのアナログ部の原理を示す回路図
【図７】ハイブリッド積分型メモリの６つの動作状態に対する各スイッチのオン・オフ
【図８】６つの動作状態に対する回路接続
【図９】ハイブリッド積分型メモリのアナログ部の回路図（スイッチ簡単化）
【図１０】メモリ量と読み出し速度を半減するためのタイミング（Ｎ×Ｔ_ＩＨ／Ｔ_Ｆ＝１／４の場合）
【図１１】メモリ量を１／４にするためのタイミング（Ｎ×Ｔ_ＩＨ／Ｔ_Ｆ＝１／８の場合）
【図１２】低照度と高照度信号を別々の画素で得る場合の画素配置（グレーが高照度用）
【図１３】ディジタル積分方式の広ダイナミックレンジ・イメージセンサのブロック図
【図１４】ディジタル積分方式広ダイナミックレンジ・イメージセンサブロック図（多傾斜ランプ発生器）
【図１５】ディジタル積分方式広ダイナミックレンジ・イメージセンサ（多傾斜ランプ発生器利用）の動作例
【図１６】多段階解像Ａ／Ｄ変換の設定例
【図１７】ディジタル積分方式広ダイナミックレンジ・イメージセンサブロック図（利得可変増幅器）
【図１８】多段分解能Ａ／Ｄ変換（利得可変）の設定例
【図１９】浮動小数点表現への変換
【図２０】ハイブリッド積分方式の広ダイナミックレンジ・イメージセンサの他の実施形態
【図２１】ディジタル積分方式の広ダイナミックレンジ・イメージセンサの他の実施形態
【図２２】広ダイナミックレンジ・イメージセンサの読み出しタイミング（ＬＡ：低照度信号蓄積、ＬＶ：低照度信号垂直読み出し期間、ＨＶ：高照度信号垂直読み出し期間、ＭＲ：メモリ信号水平読み出し期間）
【図２３】ある１水平ラインに着目したタイミング
【図２４】広ダイナミックレンジ・イメージセンサのブロック図
【図２５】広ダイナミックレンジ・イメージセンサの仕様の例
【図２６】１水平周期での処理（高照度用信号を８Ｈ期間蓄積し、４回外部積分する場合
【図２７】１水平周期での処理（高照度用信号を８Ｈ期間蓄積し、４回外部積分し、超高照度信号を２回外部積分する場合）
【図２８】多傾斜ランプ信号発生器と多段分解能グレイコード発生器によるＡ／Ｄ変換器
【図２９】多傾斜ランプ発生器の例と対応するカウント値
【図３０】多段分解能ＡＤＣ用グレイコード
【図３１】多段階分解能Ａ／Ｄ変換の設定例
【符号の説明】
１ アナログ加算器
２ アナログ減算器（一方の入力が反転されている加算器）
３ 比較器（１ビットＡ／Ｄ変換器）
４ ディジタル加算器
ＬＡ 低照度信号蓄積期間（画素部への光電荷蓄積）
ＬＲ 低照度信号読出期間（画素部からの読出し）
ＨＡ 高照度信号蓄積期間（画素部への光電荷蓄積，メモリへ読出し，積分）
ＨＲ 高照度信号読出期間（メモリからの読出し）
Ｔ フレーム周期
ＬＶ 低照度信号垂直読み出し期間
ＨＶ 高照度信号垂直読み出し期間
ＭＲ メモリ信号水平読み出し期間[0001]
[Industrial applications]
The present invention relates to a technique for expanding a dynamic range of a sensor in a solid-state imaging device to a high illuminance side, and a technique for optimizing a readout timing from a pixel to reduce a motion distortion.
[0002]
[Prior art]
There are many methods for expanding a dynamic range to a high illuminance side on a CMOS image sensor. As a representative example, (1) a comparator and a coefficient device are arranged in a pixel circuit, which is disclosed in Japanese Patent Application Laid-Open No. 7-274072, and the number of clocks required for the photodiode to reach a threshold value set from outside is determined. (2) A comparator and a coefficient unit are arranged in a pixel circuit, and a photodiode is externally set within one frame period, which is disclosed in Japanese Patent Application Laid-Open No. 2000-253320. (3) In the four-transistor pixel circuit disclosed in JP-A-2001-169184 and JP-A-2001-186414, the blooming charge of the photodiode is reduced from one vertical period. For a sufficiently short period, the method of storing in the floating diffusion, the difference between the signal level and the reset level, and (5) A method of obtaining from signals accumulated for a period sufficiently shorter than the direct period. A method of using a floating diffusion of a four-transistor type pixel circuit as a low-sensitivity photoelectric conversion element for high illuminance as described in "APS Image Sensor", (6) US Pat. No. 6,175,383, and USP. No. 6,369,737, U.S. Application Publication No. 2002/0027606, and the IEEE Journal of Solid State Circuits, Vol. 35, no. 5, “10000-Pixel, 120-dB Imager in TFA Technology”, a method of obtaining a pixel value from a storage time and a read signal level by a floating-point expression, (7) Japanese Patent Application Laid-Open No. 2000-83198 and There is a method disclosed in Japanese Unexamined Patent Application Publication No. 2002-77733, which uses a logarithmic compression type pixel circuit in which a signal voltage changes logarithmically with respect to an incident light amount.
(8) U.S. Patent No. 6,175,383, Orly Yadid-Pecht, Bedabrata Pain, Eric. R. Fossum (California Institute of Technology), '' '' Method and Apparatus of High Dynamic Range Image Sensor with Individual's U.S.A. S. Patent, 6175383, Field: 1996.1.17, Date of Patent: 2001.1.16, a method of combining a signal subjected to long-time accumulation and a signal subjected to short-time accumulation, (9 U.S. Pat. No. 6,369,737, David X. et al. D. Yand, Abbas El Gammal, Boyd Fowler (The Board of Trustees of the Leland Stanford Junior University), "Method and Apparatus for Converting A Low Dynamic Range Floating-Point Digital Representation", U. S. Patent, 6369737, Field: 1997.10.30, Date of Patent: 2002.4.9. , There is a method of combining signals of a plurality of accumulation times.
[Patent Document 1] Japanese Patent Application Laid-Open No. 7-274072
[Patent Document 2] JP-A-2000-253320
[Patent Document 3] JP-A-2001-169184
[Patent Document 4] JP-A-2001-186414
[Patent Document 5] JP-A-2000-083198
[Patent Document 6] JP-A-2002-077733
[Patent Document 7] US Patent Application Publication No. 2002/0027606
[Patent Document 8] US Pat. No. 6,175,383
[Patent Document 9] US Pat. No. 6,369,737
[Non-Patent Document 1] Ying (Michelle) Wang et al., The 2001 IEEE Works on Charge-Coupled Devices and Advanced Image Sensors, pp. 137-140, "A High-Speed Image Diagnostic Sensor,"
[Non-Patent Document 2] IEEE Journal of Solid State Circuits, Vol. 35, no. 5 "100000-Pixel, 120-dB Imager in TFA Technology"
[0003]
[Problems to be solved by the invention]
The problems of the conventional method are that the SNR (signal-to-noise ratio) of the low illuminance section is sacrificed in order to expand the dynamic range to the high illuminance area, and the low illuminance area is switched to the high illuminance area. In such a case, a decrease (dip) in the SNR is large, and the SNR is saturated in a portion having originally high illuminance despite the high SNR.
It will be saturated.
Further, in order to obtain a wide dynamic range characteristic, there is a problem that the circuit configuration and the scale become very large, and the cost becomes high.
In particular, the method (8) provides a wide dynamic range characteristic with relatively simple processing. However, the wider the dynamic range is, the more the SNR dip when switching from the low illuminance region to the high illuminance region becomes extremely large. There is a problem that becomes larger. This is because the accumulation time changes. The method (9) is a method in which a plurality of signals having different accumulation times are read out to reduce the SNR dip when the accumulation time is switched. The cost is high because memory is required for each minute. Further, the number of transistors in the pixel circuit is large, and the aperture ratio is reduced.
[0004]
The present invention can increase the SNR in the low illuminance region, thereby expanding the dynamic range to the low illuminance side, reducing the SNR dip when switching from low illuminance to high illuminance, and further improving the SNR in the high illuminance region. , The SNR can be increased to about 60 dB in a high illuminance region, and a high-quality image can be obtained.
[0005]
Summary of the Invention
Pixels for low illuminance and high illuminance are mixed in one imaging area, or one pixel is temporally switched and used for high illuminance and low illuminance. The signal is integrated using an analog memory integrated on the sensor, a hybrid memory for integration (analog + digital), or a digital memory stored in a floating-point representation after A / D conversion. As a result, a decrease in SNR when switching from low illuminance to high illuminance can be suppressed, and the SNR on the high illuminance side can be made higher than the conventional one (in the case of the hybrid integration method, an SNR of 60 dB at the maximum). ).
[0006]
In addition, since the same characteristics as the conventional sensor can be secured on the low illuminance side, for example, a pixel structure using four transistors and Pinned Photo Diode can be used, reset noise removal and low dark current can be realized, and extremely low illuminance can be achieved. The SNR can be greatly improved over the conventional one. The hybrid memory for integration on the high illuminance side detects the saturation of the output obtained by adding the analog memory and the input signal with a comparator, and stores the number of times of saturation detection in a digital memory. Signal integration becomes possible.
[0007]
In a method using a digital memory, the amount of memory can be reduced while securing a sufficient SNR by using a floating-point representation. When using a digital memory, there is a problem in that a high-speed, high-precision (10-bit) A / D converter array is required, but this may be solved by a multi-stage resolution A / D converter. is there. In this case, the point that the SNR in the high illuminance region is lower than that of the hybrid integration method remains as a problem, but the practicability is high.
[0008]
Another important feature is that the signal readout from the pixel portion of the image sensor is divided into N times and the burst readout is performed so that the readout for the low illuminance and the readout for the high illuminance are temporally separated. That is, the required amount of the high illuminance integration memory is reduced to 1 / N while reducing the crosstalk noise.
Further, by adopting a read timing different from the burst read, it is possible to reduce the motion distortion while reducing the amount of the high illuminance integration memory.
[0009]
【Example】
The present invention relates to a method and a circuit for extending a dynamic range of an image sensor to a high illuminance side. FIG. 1 shows a block diagram of the image sensor of the present invention. The pixel portion is not particularly limited in its circuit form and device structure, and includes a structure using four MOS transistors and one photodiode shown in FIG. 2 which is one of active pixel methods of a CMOS image sensor. Can be used.
[0010]
Most of the methods reported so far are applicable only to the three-transistor pixel circuit shown in FIG. The three-transistor method cannot remove random noise called reset noise generated when signal charges are initialized. Further, the dark current is large. On the other hand, in the method of FIG. 2, since reset noise can be removed, the dynamic range toward the low illuminance side can be expanded. The dynamic range expansion circuit according to the present invention can remove the reset noise as shown in FIG. 2 and use a pixel circuit that can obtain a low dark current by Pinned Photo Diode, so that the dynamic range can be expanded to the low illuminance side.
[0011]
The low illuminance signal output is read out by a vertical scan for each horizontal line to a noise canceling circuit and a signal amplifying circuit provided above the image sensor, through which horizontal scanning is performed and sequentially read out as a time series signal at the final output. . In order to expand the dynamic range to the high illuminance side, accumulation is performed in a pixel portion for a high illuminance in a short time, the data is read out at high speed, and read out after integration is performed for each pixel by a circuit external to the image array.
[0012]
FIG. 4 shows an example of time distribution of signal accumulation and signal reading in one frame. The signal of each pixel is accumulated for a long time and read out as a signal for low illuminance, and then accumulated for a short time and read out as a signal for high illuminance. When the illuminance of the imaging surface when the magnitude of the noise is equal to that of the photoelectrically converted signal is Lmin, and the illuminance of the imaging surface when the signal is saturated in the pixel portion is Lmax, the dynamic range is 20 log.₁₀(Lmax / Lmin) [dB], and the saturation signal amount is proportional to the product of Lmax and the accumulation time T. On the other hand, the amount of the saturation signal is limited by the structure and the material. That is, Lmax and the accumulation time T are in inverse proportion. Therefore, if the data is stored and read for a short time for the high illuminance signal, the dynamic range can be expanded to the high illuminance side.
[0013]
Now, in a normal image sensor, the signal accumulation time is one frame cycle T_F, But the accumulation time of the high-illuminance signal is T_IHThen, the dynamic range expansion amount by this method is 20 log₁₀(T_F/ T_IH) [DB]. For example, T_IHTo T_FIf it is 1/100, the dynamic range can be expanded toward the high illuminance side of 40 [dB]. The dynamic range expansion to the low illuminance side is not only to reduce the noise, but as described above, the dynamic range expansion circuit of the present invention that can use the four-transistor pixel without reset noise is used in the low illuminance side. Effective for expanding the dynamic range.
[0014]
The signals read as the low illuminance signal and the high illuminance signal are combined to obtain one image. The problem at this time is that the SNR is reduced when switching from the low illuminance region to the high illuminance region. The noise in this transition region is dominated by shot noise. Shot noise is proportional to the square root of the signal accumulation time. The signal amplitude is proportional to the accumulation time. Therefore, its SNR (signal to noise ratio) is proportional to the square root of the accumulation time.
[0015]
The accumulation time of the signal for low illumination is T_IL, The accumulation time of the high illuminance signal is T_IHThen, at a certain illuminance level, the dip amount of the SNR when switching from the low illuminance signal to the high illuminance signal is 20 log₁₀√ (T_IH/ T_IL) [DB]. T_IH/ T_ILAssuming = 1/256, a reduction (dip) in SNR of -24 [dB] occurs. Therefore, in this state, the dynamic range can be expanded, but the SNR must be sacrificed.
[0016]
Therefore, for the signal for high illuminance, it is an essential point of the present invention to improve the SNR by repeating integration using a memory provided outside the pixel unit and an adder for a signal obtained by performing short-time accumulation. is there. As the memory and the adder, an analog integration system, a digital integration system, and a hybrid integration system (analog + digital) can be considered.
[0017]
The analog integrator is already well-known, and the analog integration method is the same as the hybrid integration method described later except that the digital section is omitted. Since an improvement can be achieved, an analog integration method is proposed as one embodiment. Furthermore, a method for solving the problem in the analog integration method is proposed as a hybrid integration method. Hereinafter, an embodiment of the hybrid integration method will be described.
[0018]
As a problem of the analog integration method, since the signal amplitude that can be handled by the analog integrator and the analog memory is finite, a signal from a large pixel portion is saturated as it is. In order to prevent this from being saturated, saturation can be avoided by attenuating the signal before the analog integrator and then applied. However, in this case, the effect of the noise of the circuit increases, and the effect of improving the SNR is obtained. Less is.
[0019]
The digital integration method has the advantage that there is no signal deterioration in the storage circuit. However, if a high-resolution A / D converter is not used, the SNR dip when switching from low illuminance to high illuminance becomes large. The problem is that the number of transistors for memory increases. Solutions to these problems will be described later.
The hybrid integration method is a method that solves the disadvantages of the analog integration method and the digital integration method.
[0020]
As shown in FIG. 1, a function is used to determine whether or not the output of the analog integrator is saturated, using a comparator. If the output is saturated, a certain voltage is subtracted from the output of the analog integrator. Is provided. This is a method in which this is always performed at the time of repeated integration, and the number of saturations is counted by a digital memory and a digital adder, so that the signal is stored in both the analog memory and the digital value of the number of saturations. This avoids the problem of saturating an analog memory or an analog integrator, and does not generate quantization noise because of analog integration.
[0021]
The operation of this circuit will be described using equations. The signal read out i-th time from a certain pixel is x_iWhen performing analog integration while reading this N times and performing saturation detection processing, the output y of the analog adder_iAre i = 1,. . . , For N
y_i= Y_i-1+ X_i-D_i× r
It can be expressed as. Where y₀= 0. r means a value that is reduced from the output when saturation is detected.
[0022]
Di is an output of the comparator for determining whether or not saturation occurs, and is determined by the following equation.
(Equation 1)

Here, T is a threshold value for saturation detection. Although T = r may be set, T = r + Δ (Δ> 0), and by setting T to a value slightly larger than r, y_iCan be prevented from becoming negative.
[0023]
The digital integrated value Y_iIs
Y_i= Y_i-1+ D_i
Becomes Where Y₀= 0.
According to such processing, y_i<T is always satisfied, and the output of the analog adder does not saturate.
[0024]
Also, the equivalent signal amplitude is
(Equation 2)

Now, when integrating N times, the equivalent maximum signal amplitude is y_NIf the maximum signal amplitude of r is r, then
Zmax = (N + 1) r
Becomes This means that the signal to be read is x_i= R (1+ (1 / N)), and the dynamic range becomes Zmax / x_i= N, which is N times larger.
[0025]
When such a hybrid integration process is applied to a method of expanding the dynamic range on the high illuminance side by accumulating in a short time, the dynamic range can be expanded while reducing the SNR dip amount when switching from the low illuminance signal to the high illuminance signal. Becomes possible. The accumulation time of the high illuminance signal is T_IHWhen the hybrid integration is performed on the read signal N times, the signal becomes N times and the shot noise becomes √N times, so that at a certain illuminance level, the signal is switched from a low illuminance signal to a high illuminance signal. SNR dip amount is 20 log₁₀√ (N × T_IH/ T_IL) [DB]. T_IH/ T_ILIf N = 32 and N = 32, the SNR dip needs to be -9 [dB]. If such a process is not performed, it is -24 [dB], and this difference is very large.
[0026]
For example, the maximum SNR at low illuminance output is generally around 50 dB. Assuming that the illuminance is switched from low illuminance to high illuminance at the SNR of 50 dB, the SNR at the SNR dip point is 41 [dB] when performing the hybrid integration process. The SNR at the dip point is 26 [dB]. It is said that human eyes are almost unaware of noise when the SNR is close to 40 [dB], and the level can be suppressed to almost no problem by the hybrid integration process. On the other hand, when the SNR is 26 [dB], the image quality is clearly degraded by human eyes.
[0027]
Next, an actual circuit for performing hybrid integration will be described. FIG. 5 shows a circuit diagram thereof. The memory capacitors are sequentially connected to a buffer amplifier, a function of adding a voltage stored in the buffer capacitors and a signal read from the sensor, and writing the signals to the memory capacitors again, and an output of the analog adder is provided with a threshold. The comparator determines whether the value has exceeded the value. If the value exceeds the value, a value obtained by subtracting a certain voltage is written to the memory capacitor.
[0028]
FIG. 6 shows a circuit for explaining a specific operation. This memory has six operating states. FIG. 7 shows ON (1) and OFF (0) of each switch at that time, and FIG. 8 shows circuit connections in each state.
[0029]
(1) First pixel reset level sample;
At the time of the first readout when performing the hybrid integration by reading out the high illuminance signal, nothing is written in the analog memory, so the addition with the analog memory output is not performed and the signal of the pixel section is stored in the analog memory. Operation. Therefore, the charge detection section of the pixel section having the pixel circuit of 4 transistors + 1 photodiode is reset and read out. This is defined as Vs (R). This voltage is applied to the capacitor C of the analog adder.₁To sample. Then C₂Is connected to a reference voltage V_RConnect to.
[0030]
(2) First pixel signal level sample;
The TX is opened to transfer the accumulated charges to the charge detection section of the pixel section having the pixel circuit of 4 transistors + 1 photodiode, and the voltage of the charge detection section at this time is read.
This is defined as Vs (S). Then C₂Is connected between the input and output of the analog adder. C₁When the voltage at the input to Vs (R) changes from Vs (R) to Vs (S), C₁The charge corresponding to (Vs (R) -Vs (S)) is C₂Is forwarded to As a result, the output of the analog adder is
(Equation 3)

Becomes As a result, a voltage that removes the fixed pattern noise and the reset noise of the pixel portion and is not affected by the offset of the analog adder appears at the output of the adder. This is stored in the memory capacitor.
[0031]
(3) Analog memory reading and second and subsequent pixel section reset level samples;
After the second time, since the high illuminance signal is read out and hybrid integration is performed, addition with the memory output is performed. Therefore, the read voltage Vs (R) when the charge detection unit of the pixel unit having the pixel circuit of 4 transistors + 1 photodiodes is reset is set to the capacitor C of the analog adder.₁To sample. Then C₂Is connected to the output of the buffer amplifier to be read from the analog memory. The voltage is V_MAnd
[0032]
(4) Addition of the pixel portion signal level sample from the second time and the memory output;
Opening the TX and transferring the accumulated charges to the charge detection unit of the pixel, and reading the voltage of the charge detection unit at this time. This is defined as Vs (S). Then C₂Is connected between the input and output of the analog adder. C₁When the voltage at the input to Vs (R) changes from Vs (R) to Vs (S), C₁The charge corresponding to (Vs (R) -Vs (S)) is C₂Is forwarded to As a result, the output of the analog adder is
(Equation 4)

Becomes As a result, the voltage stored in the analog memory and the new signal read from the pixel unit and from which the fixed pattern noise and the reset noise have been removed are added.
A threshold voltage V is applied to the added output using a comparator._TTo determine if it is greater than
[0033]
(5) a first reference signal sample for reference signal subtraction;
As a result of the above operation (4), when the output of the adder is larger than the threshold voltage, a certain voltage is subtracted from the output. To do this, C₁First reference voltage V_R1To sample. At this time, it is necessary to short-circuit the input and output of the amplifier for the analog adder.₂In order not to destroy the signal stored in₂One end is open.
[0034]
(6) Second reference signal sample for reference signal subtraction and rewriting to analog memory;
C₂Is connected between the input and output of the analog adder. When the output D of the comparator is 1 (that is, the output of the analog adder is larger than the threshold voltage), C₁Of the second reference voltage V_R2And when the output D of the comparator is 0, C₁Of the first reference voltage V_R1Leave as is. As a result, the output of the analog adder is as follows.
(Equation 5)

Thus, a constant voltage (V_R1-V_R2) Is performed.
This value is stored in the capacitor for analog memory.
[0035]
Although the comparator itself is a 1-bit A / D converter, if this comparator is constituted by a multi-bit A / D converter, it corresponds to the degree that the output of the analog adder exceeds the threshold voltage. Then, D takes a value of 1, 2,... N.
[0036]
If the above is repeated N times, the above-described hybrid integration can be performed on the signal of the pixel portion that has accumulated for a short time. The signal amount subtracted from the analog memory is obtained by integrating the output of the comparator. The integration of the output of the comparator can be performed by using a digital memory and a digital adder, and its functional description is considered to be sufficient in the block diagram shown in FIG.
The analog section of the hybrid integration type memory can be modified in several ways as shown in FIG. 9 in which the number of switches in the memory section is reduced.
[0037]
Another important point of the present invention is the timing for reading signals from the pixel portion. This has a great relationship with the required circuit scale of the hybrid integration type memory unit.
[0038]
FIG. 4 shows the storage time of the high illuminance signal and the frame period T_FN × T_IH/ T_F= 1/4. The high illuminance signal is stored using both the storage in the pixel portion and the signal storage by integration using a memory. During this period, high-speed signal reading is performed, so that the power supply line, the GND line, and the image sensor substrate are used. Generates large switching noise. If signal reading is performed repeatedly during this period, switching noise is mixed in the read signal. Therefore, this period and the signal reading need to be shifted in time. As a result, the timing is as shown in FIG.
[0039]
FIG. 10 shows a read timing at which the time required for signal readout can be doubled while satisfying this, and the required amount of memory for high illuminance signal integration can be halved. If the timing of signal accumulation and signal reading is set between the even-numbered horizontal lines (2n) and the odd-numbered horizontal lines (2n + 1) in the pixel portion as shown in FIG. And the reading speed can be halved.
[0040]
Similarly, the accumulation time of the high illuminance signal and the frame period T_FN × T_IH/ T_FIn the case of = 1/8, if the timing of signal accumulation and signal reading is set as shown in FIG. 11, integration for noise (because integration is performed every 1/4 of the number of pixels) is performed while avoiding noise contamination. The memory amount can be reduced to 1/4 of the number of pixels, and the reading speed can be reduced to 1/4 of the case where this operation is not used.
Generally, the accumulation time of the high illuminance signal and the frame period T_FN × T_IH/ T_FBy setting = 1 / M, the amount of memory for integration can be reduced to 2 / M of the number of pixels.
[0041]
As another method of obtaining a high illuminance signal and a low illuminance signal, a method of obtaining these signals with separate pixels is also conceivable. FIG. 12 shows an example of the pixel arrangement in this case. (A) and (b) show a case in which the number of pixels for high illuminance is reduced so that the resolution in the low illuminance area is not significantly reduced. In the high illuminance area, the resolution may be slightly reduced. Is effective in such a case. (A) shows a case where the width of the high illuminance pixel is set to two pixels of the low illuminance pixel, and is shifted by half a pixel between the even-numbered row and the odd-numbered row. With this arrangement, the illuminance at the position corresponding to the low illuminance pixel can be easily obtained from the high illuminance pixel by using a simple interpolation operation (average operation) from the pixels in the even-numbered rows and the pixels in the odd-numbered rows. be able to. Unless half-pixel shift is used, complicated interpolation calculation cannot be avoided. (C) and (d) show the same resolution for high illuminance and low illuminance, (c) shows a check pattern, and (d) shows alternate arrangement for each line. (C) is advantageous in that the horizontal and vertical resolutions can be the same. Various other modifications are conceivable, and it goes without saying that the integration processing according to the present application can be applied to other arrangements.
[0042]
As described above, if different pixels are used for the high illuminance and the low illuminance, the resolution is reduced. However, technically, it is not so difficult to realize an image sensor having more than one million pixels. It doesn't have to be that high, but there seems to be a need for a wider dynamic range. Therefore, the method of using the technique of increasing the number of pixels to widen the dynamic range is sufficiently practical.
[0043]
Next, the digital integration method will be described. FIG. 13 shows the configuration of the simplest digital integration type wide dynamic range image sensor. A / D conversion is performed on the read signal for high illuminance, and the value of the digital memory stored in advance and the signal read from the pixel unit are added for each pixel, and the digital signal is stored in the digital memory again. Repeat this. In this case, saturation can be avoided by using a digital memory having a sufficient bit width. As the digital memory, the accumulation time of the high illuminance signal and the frame period T_FN × T_IH/ T_FBy setting = 1 / M, the amount of memory for integration can be reduced to 2 / M of the number of pixels. A method of expanding the dynamic range by digital integration while reducing the amount of memory by using this burst read is proposed as an embodiment of the digital integration method.
[0044]
However, in reality, there is one problem with such a simple configuration. If the resolution of the A / D converter is not sufficiently increased, the SNR of the digitally integrated signal is reduced due to the quantization noise generated by the A / D converter. For example, when the accumulation time of the high illuminance section is set to 1/8 of one frame and the digital integration is performed by performing the high-speed reading 32 times, the accumulation time of the low illuminance signal is set to T._IL, The accumulation time of the high illuminance signal is T_IHAt a certain illuminance level, when the low illuminance signal is switched to the high illuminance signal, if the maximum value of the low illuminance signal is 1 V, T_IH/ T_IL= 1/256, the minimum value of the high illuminance signal is 4 mV.
[0045]
If the A / D conversion of the high illuminance signal is performed at 1 V full scale and 8 bits, the minimum value of the high illuminance signal is 1 LSB (least significant bit), and the signal is read out 32 times to perform digital integration. The resolution is also 5 bits. The SNR determined by the quantization noise is about 32 dB. If this is subjected to 10-bit A / D conversion, the minimum value of the SNR of the high illuminance signal will be 44 dB, which can be a problem-free value. It is difficult to operate the devices side by side, and to secure 10-bit accuracy. In particular, in this case, the high illuminance signal must be read out at a speed 32 times higher than usual, so that it is necessary to operate a high-speed and high-precision A / D converter in a column.
Even if 10-bit A / D conversion can be performed, if digital integration is performed 32 times, a 15-bit memory is required to store one pixel value.
FIGS. 14 and 17 show unit circuits arranged in columns in the digital integration system in which these points are improved.
[0046]
FIG. 14 shows a multi-step resolution A using a ramp signal generator that generates a plurality of slopes in an integrating A / D converter in order to facilitate A / D conversion and not to significantly reduce SNR. / D conversion. The operation will be described below. The integrating A / D converter includes one comparator, a ramp signal generator, a counter that operates in synchronization with the comparator, and a memory that stores (latches) a count value. A ramp signal whose voltage gradually rises from 0 V is generated, and a counter for incrementing the digital value simultaneously with the start of the ramp signal is operated.
[0047]
At this time, a ramp signal and an input signal to be A / D-converted are given to the comparator, and the comparator changes from "0" to "1" at the moment when the ramp signal exceeds the input signal. The value of the counter is stored in the latch circuit by the output of the comparator. The stored digital value is a value obtained by A / D converting the input analog signal. Now, let the counter clock cycle be T_cAnd the full scale of the input voltage is V_FSWhen performing A / D conversion with N-bit resolution, the slope s = dV / dt of the ramp signal is
(Equation 6)

Choose Analog input voltage to V₁When the count value stored in the register is N₁Then
(Equation 7)

It can be seen that the count value becomes the A / D conversion value. The voltage amplitude of 1 LSB (least significant bit) is 1 LSB = V_FS/ 2^NIt is.
[0048]
If this is performed with, for example, 10-bit resolution, a maximum of 1024 counts are required, making high-speed A / D conversion difficult. Therefore, A / D conversion is performed at a high resolution (a small voltage per LSB) for a small input signal, and at a low resolution for a large input signal. This is called multi-stage resolution A / D conversion.
In the first stage, A / D conversion with the highest resolution is performed. The slope of the ramp signal at this time is S₁And The second stage is S₂= M₂× S₁(M₂Is a positive integer), and the third step is S₃= M₃× S₁(M₃(A positive integer), the resolution is changed by gradually increasing the slope.
[0049]
FIG. 15 shows the actual operation. When the input signal is A, the ramp signal changes at a slope of 1 LSB per count, the output of the comparator changes from "0" to "1" in the first stage, and the counter value 6 is stored in the data latch. . At the same time, 1 indicating the area A is stored in the zone latch. Similarly, when the input signal is B, the ramp signal changes at a slope of 2 LSBs per count, and the output of the comparator changes from "0" to "1" in the second stage, so that the count value 7 changes to the data. Stored in the latch. At the same time, 2 indicating the area B is stored in the zone latch. Thus, A / D conversion in which the resolution changes according to the input signal can be performed. For example, in order to perform 10-bit integration type A / D conversion, it is necessary to count 1024 clocks, but by using a multi-step resolution as shown in FIG. 16, the count can be reduced to 64 counts.
[0050]
For example, when the accumulation time of the high illuminance signal is set to 1/8 and the number of digital integrations is set to 32, assuming that the maximum value of the low illuminance signal is 1V, the minimum value of the amplitude of the high illuminance signal is: In FIG. 16, when the amplitude is 4 mV, the A / D conversion is performed with a resolution of 1 LSB = 1 mV, as shown in FIG. 16, so that the digital value corresponds to 4, and the digital value obtained by integrating 32 times. Is equivalent to 128 bits and 7 bits, and the S / N due to quantization noise is about 38 dB, which is sufficient even if noise increase due to integration is considered. When the amplitude slightly exceeds 16 mV, the resolution of the ADC (A / D converter) becomes 4 LSB = 4 mV, but the digital value exceeds 4, and the digital value after integration 32 times becomes 128 or more. Similarly, when the amplitude exceeds 64 mV, the resolution becomes 16 LSB = 16 mV, and when the amplitude exceeds 256 mV, the resolution becomes 64 LSB = 64 mV.
[0051]
Instead of using ramp signals having a plurality of inclinations, multistage resolution A / D conversion can be performed by changing the gain of an amplifier that amplifies a signal from the pixel portion of the image sensor in multiple stages. FIG. 17 shows a unit circuit arranged in a column in the digital integration system using this. FIG. 18 shows an example of setting the gain of the amplifier in this case.
[0052]
Although it has been described above that the ramp signals having a plurality of slopes are sequentially generated with the passage of time, it is apparent that the parallel operation can be performed by simultaneously generating the ramp signals having a plurality of slopes and including a plurality of comparators. It is. Also, in a configuration in which the gain of the amplifier is changed in multiple stages, a plurality of amplifiers having different gains are provided instead of sequentially changing the gain of the amplifier in time series, and the outputs of the amplifiers and the ramp signal having one slope are provided. It is also clear that the parallel operation can be performed by simultaneously determining by using a plurality of comparators.
[0053]
Here, a floating-point digital memory for saving digital memory and a digital integration circuit will be described. In floating-point notation, digital values are represented using exponents and mantissas. For example, if a 15-bit digital value is represented by a mantissa part 8 bits and an exponent part 3 bits, it can be stored by 11 bits. FIG. 19 shows the conversion to the floating-point representation in this example. Mantissa Y_M[X / 2] means that X is divided by 2 and rounded to 8 bits. Exponent part Y_ETakes a value from 0 to 7 and can be represented by 3 bits. The conversion from floating point to fixed point is
(Equation 8)

Is calculated. This is simply Y_MIs Y_EIt can be calculated simply by shifting the digit by a bit (bit shift).
[0054]
The noise canceling circuits and amplifiers provided around the image array are provided according to necessity. FIG. 1 shows an example in which the noise canceling circuit and the amplifier are provided for a low illuminance signal and a high illuminance signal, respectively. I have. However, since the read timing of the low illuminance signal is different from the read timing of the high illuminance signal, the low illuminance signal and the high illuminance signal can be shared as shown in FIG. In the digital integration type, FIG. 13 shows an example in which A / D converters are provided for a low illuminance signal and a high illuminance signal, respectively. However, the A / D converter in this case can also be used for the low illuminance signal and the high illuminance signal as shown in FIG.
[0055]
Yet another embodiment will now be described.
As described above, a problem in the case where the read signal is integrated by an external circuit is the amount of memory for storing the image signal.
In general, it is considered that a memory corresponding to one frame is required in order to integrate an image signal for each pixel. However, when this is integrated on an image sensor, it occupies a large area, and the manufacturing cost increases.
A feature of this embodiment is that integration can be performed with a very small amount of memory by devising a method of reading out signals accumulated for a short time.
[0056]
For the sake of simplicity, an image sensor with 480 vertical effective pixels is used to collect the high illuminance signal for approximately one horizontal period, accumulate the signal, read out the signal twice, and integrate the signal. As shown in FIG.
The basic operation is to perform vertical reading of a low illuminance signal, vertical reading and integration of a high illuminance signal, and horizontal reading of a signal from an integration memory during one horizontal period. FIG. 23 shows the timing focusing on a certain horizontal line. FIG. 24 is a block diagram of a wide dynamic range image sensor.
[0057]
The low illuminance signal is accumulated during the period of LA, read out in the vertical direction at the timing of LV, subjected to noise cancellation processing in the column, A / D converted, and stored in the digital memory. At this time, since the calculation with the originally stored value of the digital memory is not performed, the MUX (multiplexer) connects the direct output of the A / D converter instead of the output of the adder to the memory. At this time, a saturation detection flag indicating whether the signal has exceeded the full scale of the A / D converter is also stored in the digital memory. The pixels are reset by this read operation.
[0058]
Next, the first time of the short-time accumulation for high illuminance is performed in the period of HA (1). This signal is read out in the vertical direction at the timing of HV (1) to perform noise cancellation and A / D conversion.
Check the saturation detection flag stored in the digital memory, and if the low illuminance signal exceeds the full scale of the A / D converter, rewrite the contents of the digital memory with the high illuminance signal and store the saturation detection flag as it is. Keep it. Next, the second short-time accumulation for high illuminance is performed in the period of HA (2). This signal is read out in the vertical direction at the timing of HV (2), and noise cancellation and A / D conversion are performed. The saturation detection flag stored in the digital memory is checked, and if the low illuminance signal exceeds the full scale of the A / D converter, the signal is added to the signal stored in the storage of the HA (1), and the digital memory is again stored. To be stored. Subsequently, the value stored in the digital memory during the MR period is read out by performing horizontal scanning.
[0059]
A major feature of this method is that the storage capacity required for integration is very small. This is because the integration of the high illuminance signal is performed regularly and sequentially in synchronization with the reading of the low illuminance signal and the high illuminance signal. This is because the memory used for the integration is released immediately after the reading, and can be used for the integration of another horizontal line.
[0060]
Now, the number of lines in the vertical direction (the number of horizontal lines) is N_V, The number of pixels in the horizontal direction is N_H, The dynamic range expansion rate is R, and the number of integration of the high illuminance signal is N._I, The time of one horizontal line is T_H, Frame rate T_VThen, the following basic relational expression is obtained.
(Equation 9)

[0061]
Blanking period is N for simplicity_VIncluded in The dynamic range expansion rate is
(Equation 10)

[0062]
Where T_ALIs the accumulation time of the low illuminance signal, T_AHIs the accumulation time of the high illuminance signal. Also, T_AHTo T_HExpressing with reference to
[Equation 11]

Where N_AHIs the accumulation time of the high-intensity signal divided by the time of the horizontal line, that is, the accumulation time of the high-intensity signal and T_HThe ratio of
[0063]
In this method, 100% is allocated to accumulation of the low illuminance signal and the high illuminance signal without wasting the time of one frame, and the following equation is obtained.
(Equation 12)

That is, the time of one frame is determined by the accumulation of the low illuminance signal and the N_IAllocate high-intensity signal accumulation times.
[0064]
N for wide dynamic range image sensor design_V, R, N_IIs given as a specification. For these, calculation formulas are calculated for some factors that determine the performance of the image sensor.
First, the following expression is obtained from Expressions (3) and (4).
(Equation 13)

[0065]
In the wide dynamic range image sensor, since a part of the accumulation time is used for high illuminance, the sensitivity of the low illuminance signal is slightly reduced. Assuming that the sensitivity reduction rate is γ when compared with a normal image sensor that accumulates time for one frame,
[Equation 14]

Becomes The reduction is small unless the dynamic range expansion rate is set large enough and the number of integrations is increased.
[0066]
The SNR dip amount (ΔSNR) at the time of switching from the low illuminance region to the high illuminance region is determined by considering only the light shot noise.
[Equation 15]

It can be seen that the SNR reduction is reduced as the dynamic range is expanded, and is improved by increasing the number of integrations.
[0067]
Memory amount C required for integration_MIs
(Equation 16)

Where C_MFIs the amount of memory required to store one full frame,
C_MF= N_B× N_H× N_VIt is. N_BIs the bit width per pixel. The amount of memory can be reduced as the dynamic range is expanded. If the number of integrations is reduced at the expense of the SNR dip, the amount of memory can be saved.
[0068]
FIG. 25 shows a calculation example when the number of pixels is 640 × 480 pixels.
As shown in FIG. 25, FIG. 26 shows the temporal breakdown of the processing in one horizontal cycle in the case where accumulation is performed eight times as long as one horizontal read cycle and this is integrated four times.
First, the low illuminance signal in a certain n-th row is read out, stored in a digital memory through A / D conversion and the like. Next, the signal eight lines before the row is read out from that row, and after processing such as A / D conversion, if the saturation flag of the low illuminance signal written in the digital memory of the (n-8) th row is set, Replace with this signal. Next, the signal 16 rows before the row is read out from the row, and after processing such as A / D conversion, if the saturation flag of the low illuminance signal written in the digital memory on the (n-16) th row is on, Since this signal has already been replaced by the first high illuminance signal, this signal is added to the stored high illuminance signal, and the result is rewritten in the digital memory of the row.
[0069]
In the same manner, a signal before 24 lines and a signal before 32 lines are sequentially read out, and similar processing is performed. If the signal before the 32nd row is read and integrated, the value is determined for that row, so the signal stored in the memory is read out by horizontal scanning using the last period.
[0070]
As an extension, as shown in FIG. 27, the reading of the low illuminance signal accumulated for a long time, the reading and integration of the high illuminance signal accumulated for a relatively short time (in this case, 8H period integration is performed four times), and then the extremely short integration It is conceivable to further increase the dynamic range by performing the reading and integration of the ultra-high illuminance signal (in this case, integration in the 2H period twice) during one horizontal cycle. In this case, as a flag, two bits are required as a flag for distinguishing whether the signal stored in the memory is a low illuminance signal, a high illuminance signal, or an ultra-high illuminance signal. is there.
By using the saturation flag in this manner, an appropriate illuminance signal that is not saturated is selected from the low illuminance signal, the high illuminance signal, and the ultra-high illuminance signal.
Here, it has been described that the ultra-high illuminance signal is also integrated, but the signal may be output without integration if desired.
[0071]
Next, another embodiment of the A / D converter will be described. FIG. 28 shows the configuration. The dashed line indicates one channel arranged in a column, and the other multi-ramp ramp signal generator, multi-stage Gray code counter, and control circuit are common. In order to facilitate the A / D conversion and prevent the SNR from being significantly reduced, a multi-step resolution A / D conversion is performed by using a ramp signal generator that generates a plurality of slopes in an integrating A / D converter. Do. The operation will be described below.
[0072]
The integrating A / D converter includes one comparator, a ramp signal generator, a counter that operates in synchronization with the comparator, and a memory that stores (latches) a count value. A ramp signal whose voltage gradually rises from 0 V is generated, and a counter for incrementing the digital value simultaneously with the start of the ramp signal is operated. At this time, a ramp signal and an input signal to be A / D-converted are given to the comparator, and the comparator changes from "0" to "1" at the moment when the ramp signal exceeds the input signal. The value of the counter is stored in the latch circuit by the output of the comparator. The stored digital value is a value obtained by A / D converting the input analog signal. In order to reduce the influence of digital noise on the A / D conversion characteristics, a gray code in which only one bit transitions in all codes is preferably used as this counter.
[0073]
The integration type A / D converter using the counter requires, for example, a maximum of 1024 counts when performed at a resolution of 10 bits, which makes high-speed A / D conversion difficult. Therefore, A / D conversion is performed at a high resolution (a small voltage per LSB) for a small input signal, and at a low resolution for a large input signal. This is called multi-stage resolution A / D conversion. In the first stage, A / D conversion with the highest resolution is performed. The ramp signal slope at this time is s₁And The second stage is s₂= M₂× s₁(M₂Is a positive integer), and the third step is s₃= M₃× s₂(M₃(A positive integer), the resolution is changed by gradually increasing the slope.
[0074]
FIG. 29 shows the relationship between a setting example of the tilt of the multi-tilt ramp generator and the corresponding count value (digital code). When the input signal is A, the ramp signal changes at a slope of 1 LSB per count, the output of the comparator changes from "0" to "1" in the first stage, and the counter value 6 is stored in the data latch. . Similarly, when the input signal is B, the ramp signal changes at a slope of 2 LSBs per count. At this time, the count value is also increased by 2 per count. When the input signal is C, the ramp signal changes at a slope of 4 LSB per count. At this time, the count value is increased by four.
[0075]
FIG. 30 shows an example of the corresponding gray code. As described above, if the change in the slope of the ramp signal is selected to be a power of 2 and a special gray code (FIG. 30) obtained by bit-shifting the gray code according to the resolution is used, in which slope area is the A / D conversion performed? A / D conversion can be performed without being conscious of this, and the property of the Gray code that only one bit transitions can be provided.
[0076]
FIG. 31 shows an example of setting a multi-stage resolution ADC when performing 10-bit A / D conversion.
If multi-stage resolution is not adopted, it is necessary to count for 1024 clocks, but by using this as a multi-stage resolution as shown in FIG. 31, the count can be reduced to 112 counts.
In the A / D conversion for the low illuminance signal and the A / D conversion for the high illuminance signal, the setting of the inclination is changed. In the case of low illuminance, a single slope may be considered to guarantee A / D conversion with good linearity.
【The invention's effect】
As described above, according to the configuration of this application, signal processing with a wide dynamic range can be performed in signal processing obtained from an image sensor. In addition, the SNR (signal-to-noise ratio) can be increased by integrating the high illuminance signal. Furthermore, since it is not intended for a specific pixel circuit, it can be applied to a high-quality pixel circuit using an embedded photodiode or the like.
[Brief description of the drawings]
FIG. 1 shows an embodiment of a wide dynamic range image sensor.
FIG. 2 shows a four-transistor pixel circuit.
FIG. 3 shows a three-transistor pixel circuit.
FIG. 4 shows the timing of accumulation and readout of a low illuminance signal and a high illuminance signal.
FIG. 5 is a circuit diagram of an analog unit of the hybrid integration type memory;
FIG. 6 is a circuit diagram showing the principle of the analog section of the hybrid integration type memory;
FIG. 7 shows ON / OFF of each switch for six operation states of the hybrid integration type memory.
FIG. 8 is a circuit connection for six operation states.
FIG. 9 is a circuit diagram of an analog section of a hybrid integration type memory (simplification of a switch).
FIG. 10 shows a timing (N × T) for halving a memory amount and a reading speed._IH/ T_F= 1/4)
FIG. 11 shows a timing (N × T) for reducing the memory amount to １ ／._IH/ T_F= 1/8)
FIG. 12 is a diagram showing a pixel arrangement in which a low illuminance signal and a high illuminance signal are obtained by different pixels (gray is for high illuminance);
FIG. 13 is a block diagram of a digital integration type wide dynamic range image sensor.
FIG. 14 is a block diagram of a digital integration type wide dynamic range image sensor (multi-tilt ramp generator)
FIG. 15 shows an operation example of a digital integration type wide dynamic range image sensor (using a multi-ramp ramp generator).
FIG. 16 shows a setting example of multi-step resolution A / D conversion.
FIG. 17 is a block diagram of a digital integration type wide dynamic range image sensor (variable gain amplifier).
FIG. 18 shows a setting example of multi-stage resolution A / D conversion (variable gain).
FIG. 19: Conversion to floating-point representation
FIG. 20 shows another embodiment of a hybrid integration type wide dynamic range image sensor.
FIG. 21 is a digital integration type wide dynamic range image sensor according to another embodiment.
FIG. 22: Read timing of a wide dynamic range image sensor (LA: low illuminance signal accumulation, LV: low illuminance signal vertical read period, HV: high illuminance signal vertical read period, MR: memory signal horizontal read period)
FIG. 23 is a timing focused on one horizontal line.
FIG. 24 is a block diagram of a wide dynamic range image sensor.
FIG. 25 shows an example of specifications of a wide dynamic range image sensor.
FIG. 26 shows processing in one horizontal cycle (in the case where a signal for high illuminance is accumulated for 8H periods and externally integrated four times)
FIG. 27 shows processing in one horizontal cycle (in the case where a signal for high illuminance is accumulated for 8H, externally integrated four times, and externally integrated with an ultra-high illuminance signal twice).
FIG. 28: A / D converter using a multi-ramp ramp signal generator and a multi-resolution gray code generator
FIG. 29 shows an example of a multi-tilt ramp generator and corresponding count values
FIG. 30: Gray code for multi-stage resolution ADC
FIG. 31 shows a setting example of multi-step resolution A / D conversion.
[Explanation of symbols]
1 Analog adder
2 Analog subtractor (adder with one input inverted)
3 Comparator (1-bit A / D converter)
4 Digital adder
LA Low illuminance signal accumulation period (accumulation of photocharge in pixel section)
LR Low illuminance signal readout period (reading from pixel unit)
HA High illuminance signal accumulation period (photocharge accumulation in pixel unit, readout to memory, integration)
HR High illuminance signal reading period (reading from memory)
T frame period
LV Low illuminance signal vertical readout period
HV high illuminance signal vertical readout period
MR memory signal horizontal readout period

Claims (24)

In a photo-charge storage type solid-state imaging device, a take-out means for extracting photo-charges from a pixel after performing a long-time photo-charge storage for low illuminance, and a light-emitting device for performing a short-time photo-charge storage for high illuminance, and A solid-state imaging device comprising: a take-out unit that takes out electric charges; and a unit that integrates an output of the high-illuminance take-out unit for each pixel. In a photocharge storage type solid-state imaging device, a means for extracting photocharge from a pixel after performing long-time photocharge accumulation for low illuminance, and a method for performing short-time photocharge accumulation for high illuminance and then extracting photocharge from the pixel , A means for integrating the output of the high illuminance extraction means for each pixel, and a means for extracting photocharges from the pixels after performing a very short time photocharge accumulation for ultra-high illuminance. apparatus. In a photo-charge storage type solid-state imaging device, a take-out means for extracting photo-charges from a pixel after performing a long-time photo-charge storage for low illuminance, and a light-emitting device for performing a short-time photo-charge storage for high illuminance, and Extracting means for extracting electric charges, analog storing means for storing illuminance information for each pixel, analog adding means for adding an output of the analog storing means and an output of the extracting means for high illuminance, and analog adding means. Means for writing an output to the analog storage means. In a photo-charge storage type solid-state imaging device, a take-out means for extracting photo-charges from a pixel after performing a long-time photo-charge storage for low illuminance, and a light-emitting device for performing a short-time photo-charge storage for high illuminance, and Extracting means for extracting electric charges, analog storing means for storing illuminance information for each pixel, analog adding means for adding an output of the analog storing means and an output of the extracting means for high illuminance, and analog adding means. A / D which compares the output with a reference signal value and converts the degree of the analog amount exceeding the reference signal value into a digital amount in accordance with the analog amount exceeding the reference signal value when the output of the analog adding means exceeds the reference signal value. Conversion means, D / A conversion means for converting the output of the A / D conversion means from a digital quantity to an analog quantity, and subtracting the output of the D / A conversion means from the output of the analog addition means And that subtracting means consists of a means for writing the output of the subtraction means to the analog memory means, a solid-state imaging apparatus characterized by inhibited saturated analog memory means. The digital addition means further comprises a digital storage means and a digital addition means. The digital addition means adds the output of the A / D conversion means and the output of the digital storage means for each corresponding pixel. 5. The solid-state imaging device according to claim 4, wherein an output of the adding means is stored as illuminance information of a corresponding pixel. In a photo-charge storage type solid-state imaging device, a take-out means for extracting photo-charges from a pixel after performing a long-time photo-charge storage for low illuminance, and a light-emitting device for performing a short-time photo-charge storage for high illuminance, and Extraction means for extracting electric charges, A / D conversion means for analog-to-digital conversion of the output of the extraction means for high illuminance, digital storage means for storing illuminance information for each pixel, and the output of the digital storage means A solid-state imaging device comprising: digital addition means for adding the output of the A / D conversion means; and means for writing the output of the digital addition means to the digital storage means. Furthermore, a floating-point representation-fixed-point representation converter and a fixed-point representation-floating-point representation converter are provided, and the floating-point data stored in the digital storage means is changed to fixed-point representation to convert the A / D conversion means. 7. The solid-state imaging device according to claim 6, wherein the digital addition means adds the data in the fixed-point representation, which is an output, to the digital addition means, and converts the output of the digital addition means to a floating-point representation and stores the result in the digital storage means. . The A / D converter includes a comparator, a ramp signal generator that generates a ramp signal having a plurality of slopes, a counter and a clock generator that operate in synchronization with the operation of the ramp signal generator, The A / D conversion is performed by counting the number of clocks from the generation of the ramp signal to the inversion of the comparison result. When the level of the input signal is low, the digital signal is obtained by comparing the ramp signal with a small slope with the input signal. 7. The solid-state imaging device according to claim 6, wherein when the level of the input signal is high, a digital signal is obtained by comparing a ramp signal having a large inclination with the input signal, thereby providing a multi-step resolution. 9. The solid-state imaging device according to claim 8, wherein the ramp signal generated by the ramp signal generator has a piecewise linear characteristic and a gradient changes stepwise. Further, a gray code / binary code converter is provided, and the A / D conversion means performs an A / D conversion operation using the gray code, and converts the A / D converted to a binary code by the gray code / binary code converter. 7. The solid-state imaging device according to claim 6, wherein integration is performed by adding an output of a / D conversion unit to a content of said digital storage unit. The A / D conversion means includes an amplifier having a plurality of gains, a comparator, a ramp signal generator for generating a ramp signal having one slope, and a counter operating in synchronization with the operation of the ramp signal generator. A / D conversion is performed by counting the number of clocks from the generation of the ramp signal to the inversion of the comparison result. When the level of the input signal is low, high-gain amplification is performed. The digital signal is obtained by comparing the input signal and the ramp signal, and when the level of the input signal is high, the digital signal is obtained by comparing the input signal which has been amplified with low gain and the ramp signal, thereby providing a multi-step resolution. The solid-state imaging device according to claim 6, wherein: A timing control circuit for controlling so as to separate a reading period of the low illuminance signal, a reading / integration period of the high illuminance signal to the storage unit, and a reading period of the storage unit during one frame period. The solid-state imaging device according to claim 1, wherein: The lines of the pixels arranged two-dimensionally are divided into N groups, and in each group, during one frame period, a reading period of a low illuminance signal, a reading / integration period of a high illuminance signal to a storage means, and The readout period from the storage means is separated from the other groups, and the respective groups are shifted from each other so that all of the N groups are read out by reading N times. 12. The solid-state imaging device according to claim 1, further comprising a timing control circuit for controlling the required amount of storage capacity of the digital storage means to be 1 / N. The lines of the pixels arranged two-dimensionally are divided into N groups, and in each group, during one frame period, a reading period of a low illuminance signal, a reading / integration period of a high illuminance signal to a storage means, and The reading period from the storage means is separated from the other groups, and the period is shifted from the other groups, so that all of the N groups are read out by reading N times. The solid-state imaging device according to claim 1, further comprising a timing control circuit configured to control a read period and a read / integration period of the high illuminance signal so as not to coincide with each other. 2. The solid-state imaging device according to claim 1, wherein the pixels for low illuminance and the pixels for high illuminance are independently provided, and are arranged in a mixed manner in the imaging area. 2. The solid-state imaging device according to claim 1, wherein the pixels for low illuminance and the pixels for high illuminance are independently provided, and arranged so as to form a check pattern in the imaging area. The low illuminance pixel and the high illuminance pixel are provided independently, and the high illuminance pixel is halved in pixel density with respect to the low illuminance pixel, and the high illuminance pixel is half a pixel in even and odd lines. 2. The solid-state imaging device according to claim 1, wherein the pixels are shifted by an amount corresponding to each other. A step of obtaining a low illuminance signal from a pixel by long-time accumulation of photocharges; a step of obtaining a high illuminance signal from a pixel by short-time accumulation of photocharges; A solid-state imaging signal processing method. The solid-state imaging device according to claim 6, wherein the A / D conversion unit is used for both a low illuminance signal and a high illuminance signal. The solid-state imaging device according to claim 1, further comprising a noise cancellation circuit and / or an amplifier, and performing a noise cancellation process and / or an amplification process on a signal from the pixel. 21. The solid-state imaging device according to claim 20, wherein the noise cancellation circuit and / or the amplifier is used for both a low illuminance signal and a high illuminance signal. Vertical reading of low illuminance signals by long-time accumulation, vertical reading of multiple high illuminance signals by short-time accumulation and integration outside the imaging area, and horizontal direction for external output for one row of pixels in the horizontal direction The solid-state imaging device according to claim 1, wherein scanning is assigned in a time-sharing manner within one horizontal readout cycle. In the integration by reading out the high illuminance signal, in order to integrate N signals, processing in one horizontal read cycle is performed by vertical reading of the low illuminance signal by long-time accumulation for one row of pixels in a certain horizontal direction. , Vertical reading of the high illuminance signal in the short-time accumulation for each row of pixels of M rows, 2M rows,... N × M rows away from the row, and stores each row in the storage means. 23. The solid-state imaging device according to claim 22, further comprising control means for performing integration processing outside the imaging area by adding the stored value. In the horizontal readout for the external output, when performing vertical readout of the low illuminance signal due to long-term accumulation, compare the low illuminance signal with a certain threshold, to determine whether or not exceeds the threshold, as a result Is temporarily stored together with the value of the low illuminance signal, and when performing horizontal reading, if the low illuminance signal does not exceed the threshold, the low illuminance signal value is read out by horizontal scanning, and the threshold is read. If it exceeds, discard the low illuminance signal value, corresponding to the pixel of the discarded illuminance signal, the high illuminance signal obtained by integrating the high illuminance signal by short-time accumulation a plurality of times, 23. The solid-state imaging device according to claim 22, further comprising control means for reading out the illuminance signal of the pixel to the outside by horizontal scanning.

Images