JP3703424B2 - IMAGING DEVICE, ITS CONTROL METHOD, CONTROL PROGRAM, AND STORAGE MEDIUM - Google Patents

Description

(2)ΔＺT／Ｐ＋Δｇ＞０のとき

ＲT(i,j)の生成
(3)ΔＺT／（Ｐ×√２）＋Δr≦０のとき

(4)ΔＺT／（Ｐ×√２）＋Δr＞０のとき

ＢT(i,j)の生成
(5)ΔＺT／（Ｐ×√２）＋Δｂ≦０のとき

(6)ΔＺT／（Ｐ×√２）＋Δｂ＞０のとき

以上の処理によって求められた補間画像信号Ｇ２T(i,j)、ＲT(i,j)、ＢT(i,j)は次に第２段階の補正処理に用いられる。
【０１１６】
第２段階の補間処理は、各々が６００×８００画素の画像信号Ｇ１(i,j)と補間画像信号Ｇ２T(i,j)、ＲT(i,j)、ＢT(i,j)から、ＲＧＢがそれぞれ１２００×１６００画素の解像度となるＧ画像信号Ｇ'(m,n)、Ｒ画像信号Ｒ'(m,n)、Ｂ画像信号Ｂ'(m,n)を生成する。以下の式（１０）〜式（２１）は、データがない位置の画素出力を隣接する画素の出力を平均することによって生成するための演算を表す式である。
【０１１７】
Ｇ'(m,n)の生成
(1)ｍ：偶数 ｎ：奇数のとき
Ｇ'(m,n)＝Ｇ２T(m/2,(n+1)/2) …（１０）
(2)ｍ：奇数 ｎ：偶数のとき
Ｇ'(m,n)＝Ｇ１((m+1)/2,n/2) …（１１）
(3)ｍ：偶数 ｎ：偶数のとき

(4)ｍ：奇数 ｎ：奇数のとき

Ｒ'(m,n)の生成
(5)ｍ：偶数 ｎ：奇数のとき
Ｒ'(m,n)＝(ＲT(m/2,(n+1)/2)＋ＲT(m/2+1,(n+1)/2)／２…（１４）
(6)ｍ：奇数 ｎ：偶数のとき
Ｒ'(m,n)＝(ＲT((m+1)/2,n/2)＋ＲT((m+1)/2,n/2+1)／２…（１５）
(7)ｍ：偶数 ｎ：偶数のとき

(8)ｍ：奇数 ｎ：奇数のとき
Ｒ'(m,n)＝ＲT((m+1)/2,(n+1)/2) …（１７）
Ｂ'(m,n)の生成
(9)ｍ：偶数 ｎ：奇数のとき
Ｂ'(m,n)＝(ＢT(m/2,(n-1)/2)＋ＢT(m/2,(n-1)/2+1))／２…（１８）
(10)ｍ：奇数 ｎ：偶数のとき
Ｂ'(m,n)＝(ＢT((m-1)/2,n/2)＋ＢT((m-1)/2+1,n/2))／２…（１９）
(11)ｍ：偶数 ｎ：偶数のとき
Ｂ'(m,n)＝ＢT(m/2,n/2) …（２０）
(12)ｍ：奇数 ｎ：奇数のとき

以上のように、第１段階の補間処理で複数の撮像領域の出力画像のうちの少なくとも一つについて位置補正を行った後、第２段階の補間処理で複数の撮像領域の出力画像に基づく合成映像信号を形成する。
【０１１８】
Ｇ'(m,n)、Ｒ'(m,n)、Ｂ'(m,n)を用いたその後の輝度信号処理、色差信号処理は通常のデジタルカラーカメラでの処理に準じたものとなる。
【０１１９】
次に、カメラの動作を説明する。撮影時にはカメラ本体１０１の接続端子１１４を保護するために接点保護キャップを装着して使用する。接点保護キャップ２００をカメラ本体１０１に装着すると、カメラのグリップとして機能し、カメラを持ち易くする役割を果たす。
【０１２０】
まず、メインスイッチ１０５をオンとすると、各部に電源電圧が供給されて動作可能状態となる。続いて、メモリに画像信号を記録可能か否かが判定される。この際に、残り容量に応じて撮影可能記録枚数が表示部１５０に表示される。その表示を見た操作者は、撮影が可能であれば、被写界にカメラを向けてレリーズボタン１０６を押下する。
【０１２１】
レリーズボタン１０６を半分だけ押下すると、スイッチ１２１の第１段回路が閉成し、露光時間の算出が行なわれる。すべての撮影準備処理が終了すると、撮影可能となり、その表示が撮影者に報じられる。これにより、レリーズボタン１０６が終端まで押下されると、スイッチ１２１の第２段回路が閉成し、不図示の操作検出回路がシステム制御回路にその検出信号を送出する。その際に、あらかじめ算出された露光時間の経過をタイムカウントして、所定の露光時間が経過すると、固体撮像素子駆動回路にタイミング信号を供給する。これにより、固体撮像素子駆動回路は水平および垂直駆動信号を生成し、すべての撮像領域について露光された８００×６００画素のそれぞれを水平および垂直方向に順次読み出す。
【０１２２】
このとき、撮影者は接点保護キャップ２００を持つようにして右手の人差し指と親指でカメラ本体１０１を挟み込むようにして、レリーズ釦１０６を押下する。レリーズ釦１０６の軸の中心線Ｌ２上にレリーズ釦１０６と一体的に突起１０６ａを設け、さらに、裏蓋１２５上であって中心線Ｌ２を延長した位置に突起１２０を設けているので、撮影者は２つの突起１０６ａと１２０を頼りに、人差し指で突起１０６ａを、親指で突起１２０をそれぞれ押すようにレリーズ操作を行う。こうすることにより、図３に示した偶力１２９の発生を容易に防ぐことができ、ブレのない高画質の画像を撮像することができる。
【０１２３】
読み出されたそれぞれの画素は、Ａ／Ｄ変換器５００にて所定のビット値のデジタル信号に変換されて、画像処理系２０のＲＧＢ画像処理回路２１０に順次供給される。ＲＧＢ画像処理回路２１０では、これらをそれぞれホワイトバランス、ガンマ補正を施した状態にて画素の補間処理を行なって、ＹＣ処理回路２３０に供給する。
【０１２４】
ＹＣ処理回路２３０では、その高域輝度信号発生回路にて、ＲＧＢそれぞれの画素の高域輝度信号ＹＨを生成し、同様に、低域輝度信号発生回路にて低域輝度信号ＹＬをそれぞれ演算する。演算した結果の高域輝度信号ＹＨは、ローパス・フィルタを介して加算器に出力される。同様に、低域輝度信号ＹＬは、高域輝度信号ＹＨが減算されてローパス・フィルタを通って加算器に出力される。これにより、高域輝度信号ＹＨとその低域輝度信号との差ＹＬ−ＹＨが加算されて輝度信号Ｙが得られる。同様に、色差信号発生回路では、色差信号Ｒ−Ｙ，Ｂ−Ｙを求めて出力する。出力された色差信号Ｒ−Ｙ，Ｂ−Ｙは、それぞれローパス・フィルタを通った成分が記録処理回路３００に供給される。
【０１２５】
次に、ＹＣ信号を受けた記録処理回路３００は、それぞれの輝度信号Ｙおよび色差信号Ｒ−Ｙ，Ｂ−Ｙを所定の静止画圧縮方式にて圧縮して、順次メモリに記録する。
【０１２６】
メモリに記録された静止画像または動画像を表わす画像信号からそれぞれの画像を再生する場合には、再生ボタン９を押下すると操作検出回路４３０にてその操作を検出して、システム制御部４００に検出信号を供給する。これにより記録処理回路３００が駆動される。駆動された記録処理回路３００は、メモリから記録内容を読み取って、液晶モニタ４に画像を表示する。操作者は、所望の画像を選択ボタンなどの押下により選択する。
【０１２７】
（第２の実施形態）
第１の実施形態では撮像系１０（８９０）の温度変化に起因するレジストレーションずれを温度センサ１６５の出力に基づいて補正したが、レジストレーションずれは被写体の距離変化によっても生じる。第２の実施形態はこの被写体距離の変化に起因するレジストレーションずれを補正するものである。
【０１２８】
図２１は測距装置の出力をレジストレーションずれの補正に用いるための信号処理系を表す図である。第１の実施形態と同じ要素には同じ符号を付している。
【０１２９】
図２１に示すように、撮像装置は、撮像系１０と、画像処理手段であるところの画像処理系２０と、記録再生系３０と、制御系４０とを有する。さらに、撮像系１０は、撮影レンズ１００、絞り１１０および固体撮像素子１２０を含み、画像処理系２０は、Ａ／Ｄ変換器５００、ＲＧＢ画像処理回路４１０およびＹＣ処理回路２３０を含み、記録再生系３０は、記録処理回路３００および再生処理回路３１０を含み、制御系４０は、システム制御部４００、操作検出部４３０、測距装置４６５および固体撮像素子駆動回路４２０を含む。
【０１３０】
測距装置４６５は超音波やＬＥＤ光を被写体に投射して距離を出力するアクティブ型測距装置や、被写体の画像を利用し三角測量の原理に基づいて距離を出力するパッシブ型測距装置が適当である。
【０１３１】
撮像系１０は、物体からの光を絞り１１０と撮影レンズ１００を介して固体撮像素子１２０の撮像面に結像する光学処理系であり、被写体像を固体撮像素子１２０に露光する。前述のように、固体撮像素子１２０は、ＣＣＤやＣＭＯＳセンサなどの撮像デバイスが有効に適用され、固体撮像素子１２０の露光時間および露光間隔を制御することにより、連続した動画像を表わす画像信号、または一回の露光による静止画像を表わす画像信号を得ることができる。
【０１３２】
第１の実施形態と同じように固体撮像素子１２０は、各撮像領域毎に長辺方向に８００画素、短辺方向に６００画素を有し、合計１９２万の画素数を有する撮像デバイスが有効に適用されて、その前面には赤色（Ｒ）、緑色（Ｇ）、青色（Ｂ）の３原色の光学フィルターが所定の領域毎に配置されている。画素ピッチは縦横ともに１．５６μmである。
【０１３３】
図２１に表すように、固体撮像素子１２０から読み出された画像信号は、それぞれＡ／Ｄ変換器５００を介して画像処理系２０に供給される。Ａ／Ｄ変換器５００は、たとえば、露光した各画素の信号の振幅に応じた、たとえば１０ビットのデジタル信号に変換して出力する信号変換回路であり、以降の画像信号処理はデジタル処理にて実行される。
【０１３４】
画像処理系２０は、Ｒ，Ｇ，Ｂのデジタル信号から所望の形式の画像信号を得る信号処理回路であり、Ｒ，Ｇ，Ｂの色信号を輝度信号Ｙおよび色差信号（Ｒ−Ｙ），（Ｂ−Ｙ）にて表わされるＹＣ信号などに変換する。
【０１３５】
ＲＧＢ画像処理回路４１０は、Ａ／Ｄ変換器５００を介して固体撮像素子１２０から受けた８００×６００×４画素の画像信号を処理する信号処理回路であり、ホワイトバランス回路、ガンマ補正回路、補間演算による高解像度化を行う補間演算回路を有する。
【０１３６】
ＹＣ処理回路２３０は、輝度信号Ｙおよび色差信号Ｒ−Ｙ，Ｂ−Ｙを生成する信号処理回路である。高域輝度信号ＹＬを生成する高域輝度信号発生回路、低域輝度信号ＹＬを生成する低域輝度信号発生回路、および、色差信号Ｒ−Ｙ，Ｂ−Ｙを生成する色差信号発生回路で構成されている。輝度信号Ｙは高域輝度信号ＹＨと低域輝度信号ＹＬを合成することによって形成される。
【０１３７】
記録再生系３０は、メモリへの画像信号の出力と、液晶モニタ４への画像信号の出力とを行う処理系であり、メモリへの画像信号の書き込みおよび読み出し処理を行なう記録処理回路３００と、メモリから読み出した画像信号を再生して、モニタ出力とする再生処理回路３１０とを含む。より詳細には、記録処理回路３００は、静止画像および動画像を表わすＹＣ信号を所定の圧縮形式にて圧縮し、また、圧縮データを読み出した際に伸張する圧縮伸張回路を含んでいる。
【０１３８】
圧縮伸張回路は、信号処理のためのフレームメモリなどを有し、このフレームメモリに画像処理系２０からのＹＣ信号をフレーム毎に蓄積して、それぞれ複数のブロック毎に読み出して圧縮符号化する。圧縮符号化は、たとえば、ブロック毎の画像信号を２次元直交変換、正規化およびハフマン符号化することにより行なう。
【０１３９】
再生処理回路３１０は、輝度信号Ｙおよび色差信号Ｒ−Ｙ，Ｂ−Ｙをマトリックス変換してたとえばＲＧＢ信号に変換する回路である。再生処理回路３１０によって変換した信号は液晶モニタ４に出力され、可視画像が表示再生される。
【０１４０】
制御系４０は、外部操作に連動して撮像系１０、画像処理系２０、記録再生系３０をそれぞれ制御する各部の制御回路を含み、レリーズボタン６の押下を検出して、固体撮像素子１２０の駆動、ＲＧＢ画像処理回路４１０の動作、記録処理回路３００の圧縮処理などを制御する。具体的には、レリーズボタン６の操作を検出する操作検出回路４３０と、その検出信号に連動して各部を制御し、撮像の際のタイミング信号などを生成して出力するシステム制御部４００と、このシステム制御部４００の制御の下に固体撮像素子１２０を駆動する駆動信号を生成する固体撮像素子駆動回路４２０とを含む。
【０１４１】
さて、ＲＧＢ画像処理回路４１０での処理は次のようなものである。Ａ／Ｄ変換器５００を介してＲ，Ｇ，Ｂ領域毎に出力されたＲＧＢ信号に対して、まず、ＲＧＢ画像処理回路４１０内のホワイトバランス回路にてそれぞれ所定の白バランス調整を行ない、さらに、ガンマ補正回路にて所定のガンマ補正を行なう。ＲＧＢ画像処理回路４１０内の補間演算回路は、固体撮像素子１２０の画像信号に補間処理と歪曲補正を施すことによって１２００×１６００画素の解像度の画像信号をＲＧＢ毎に生成し、後段の高域輝度信号発生回路、低域輝度信号発生回路、色差信号発生回路に供給する。
【０１４２】
この補間処理は、さらに、被写体の距離によるレジストレーションずれを補正する第１段階の補間処理と、ＲＧＢの各画像信号を同一の解像度とした合成映像信号を形成するための第２段階の補間処理で構成される。
【０１４３】
これに続く歪曲補正は、公知の手法によって撮影光学系の歪曲収差を補正する演算処理である。この際、ＲＧＢ各物体像の倍率や歪曲は撮影レンズ１００の設定によって同一であるので、各物体像に対して一律の歪み補正を行えばよい。撮影光学系の歪曲収差を演算上で補正することにより、撮影レンズ１００の構成を他の光学収差補正のために最適化できる。
【０１４４】
さて、第１段階の補間処理の詳細は次に示すとおりである。簡単のため以後の説明では結像系を基準として被写体距離を考えることとする。撮像領域８２０ａと８２０ｄ間のレジストレーション変化量ΔＺSは実被写体距離と基準被写体距離Ｓtとの差ΔＳの関数として、式（２２）で表すことができる。差ΔＳは測距装置４６５の出力より得る。
【０１４５】
ΔＺS＝ｄo・Ｓ'・ΔＳ／｛Ｓt・（ΔＳ＋Ｓt）｝ …（２２）
撮像系の構成は第１の実施形態と同一であるとして、レンズ部の間隔ｄo＝１．９８３２［ｍｍ］、基準被写体距離Ｓt＝２３８０−１．４５＝２３７８．５５［ｍｍ］、結像系と撮像領域との間隔Ｓ'＝１．４５［ｍｍ］である。
【０１４６】
図２２は実被写体距離が無限に大きくなったときの光路を表す図である。各撮像領域は１．２４８ｍｍ×０．９３６ｍｍであって、これらは横方向に０．１５６ｍｍ、縦方向に０．４６８ｍｍの分離帯を隔てて位置している。隣り合う撮像領域の中心間隔は縦方向、横方向に１．４０４ｍｍ、また、対角方向については１．９８５６ｍｍである。
【０１４７】
撮像領域８２０ａと８２０ｄに注目して、基準被写体距離にある物体の像を、画素ずらしのために撮像領域間隔の１．９８５６ｍｍから０．５画素分の対角寸法を差し引いた１．９８４５ｍｍ間隔で、撮像部上に形成するものとする。こうすると、前述のように、撮影レンズ８００のレンズ部８００ａ、８００ｄの間隔を１．９８３２ｍｍに設定することになる。図において矢印８５５ａ、８５５ｄは、撮影レンズ８００のレンズ部８００ａ、８００ｄによる正のパワーを有する結像系を表す記号、矩形８５６ａ、８５６ｄは撮像領域８２０ａ、８２０ｄの範囲を表す記号、Ｌ８０１、Ｌ８０２は結像系８５５ａ、８５５ｄの光軸である。
【０１４８】
実被写体距離と基準被写体距離Ｓtとの差ΔＳは無限大である。ΔＳ→∞としたときの式（２２）の極限値は
【０１４９】
【数１】

であり、０．００１２［ｍｍ］のレジストレーションずれが生じることが分かる。１単位を１画素ピッチ(pxl)として表せば、画素ピッチＰを用いて、レジストレーション変化量はΔＺS／Ｐとなる。
【０１５０】
以上に示した被写体距離によるレジストレーションずれを補正する第１段階の実際の補間処理は次の通りである。第１の実施形態と同様に撮像領域８２０ａ、８２０ｂ、８２０ｃ、８２０ｄからの画像信号をそれぞれ、Ｇ１(i,j)、Ｒ(i,j)、Ｂ(i,j)、Ｇ２(i,j)とし、アドレスを図２０に示すように定める。
【０１５１】
結像系８５５ａ（レンズ部８００ａ）の画像信号Ｇ１(i,j)を基準として、Ｇ２(i,j)、Ｒ(i,j)、Ｂ(i,j)の補間画像信号Ｇ２S(i,j)、ＲS(i,j)、ＢS(i,j)を次に示す式（２４）から式（２９）に基づいて生成する。式（２４）から（２９）は、仮想位置の画素出力を両隣の実際に存在する画素出力から直線補間で生成するための式である。画素ピッチをＰとして、撮像領域８２０ａおよび撮像領域８２０ｄで受光面上の横方向ピッチａ＝Ｐ、縦方向ピッチｂ＝Ｐ、定数ｃ＝９００、正の整数ｈ＝１としたとき、これらは受光面内で横方向にａ×ｈ×ｃ、縦方向にｂ×ｃだけ離れた位置関係にあるので、温度変化や被写体距離変化に伴って生じるレジストレーションずれを極めて簡単な演算で補正することが可能である。
【０１５２】
なお、Ｒ画像のレジストレーションずれとＢ画像のレジストレーションずれは、結像系の光軸間距離の比からＧ画像間のレジストレーションずれの１／√２である。
【０１５３】
Ｇ２S(i,j)の生成
(1)ΔＺS≧０のとき

(2)ΔＺS＜０のとき

ＲS(i,j)の生成
(3)ΔＺS≧０のとき

(4)ΔＺS＜０のとき

ＢS(i,j)の生成
(5)ΔＺS≧０のとき

(6)ΔＺS＜０のとき

以上の処理によって求められた補間画像信号Ｇ２S(i,j)、ＲS(i,j)、ＢS(i,j)は、被写体距離に起因するレジストレーションずれが解消され、次に第２段階の補正処理に用いられる。
【０１５４】
（第３の実施形態）
第２の実施形態では、測距装置４６５の出力に基づいて被写体距離に起因するレジストレーションずれを補正した。第３の実施形態では測距装置４６５の出力を利用することなく、このレジストレーションずれを補正する。
【０１５５】
図２に示した使用者がカメラの状態をセットするためのスイッチ１０９をマクロ撮影モード設定スイッチとして用いる。次回の撮影を近距離の被写体に対して行う場合は、撮影者はマクロ撮影モード設定スイッチ１０９を押下する。操作検出回路４３０がマクロ撮影モード設定スイッチ１０９の押下を検出すると、例えば、基準被写体距離Ｓtとの差ΔＳを−１．１９［ｍ］とした補間処理を行う。すなわち、式（２２）のΔＳをΔＳ＝−１．１９としてΔＺSを算出し、式（２４）から式（２９）に基づいて補間処理を行う。
【０１５６】
このように構成すれば、測距装置を必要としないので、コストを低く抑えることができる。
また、これに加えて、カメラの小型化、薄型化に有利となる。
【０１５７】
（第４の実施形態）
第１の実施形態では温度センサ１６５の出力に基づいて温度変化に起因するレジストレーションずれを、第２の実施形態では測距装置４６５の出力に基づいて被写体距離に起因するレジストレーションずれを補正した。第４の実施形態では温度センサ１６５や測距装置４６５の出力を利用することなく、固体撮像素子の出力そのものを基にレジストレーションずれを補正する。このとき２つのＧ撮像領域の出力の斜め方向の相関を利用して、視差に起因する像のズレを検出する。このように構成することによって、温度変化に起因するレジストレーションずれと被写体距離に起因するレジストレーションずれ、さらには、撮影レンズの製造上の誤差によるレジストレーションずれをまとめて補正することができる。
【０１５８】
図２３はレジストレーションずれを検出するために用いる固体撮像素子上の画素を説明するための図である。簡単のため縦横の画素数をそれぞれ１／１００にして、物体像と撮像領域との位置関係を示している。
【０１５９】
図２３において、３２０ａ、３２０ｂ、３２０ｃ、３２０ｄは固体撮像素子８２０の４つの撮像領域である。説明のために撮像領域３２０ａ、３２０ｂ、３２０ｃ、３２０ｄの各々は画素を８×６個配列してなる。撮像領域３２０ａと撮像領域３２０ｄはＧ画像信号を、撮像領域３２０ｂはＲ画像信号を、撮像領域３２０ｃはＢ画像信号を出力する。撮像領域３２０ａと３２０ｄ内の各画素は白抜きの矩形で、撮像領域３２０ｂ内の各画素はハッチングを付した矩形で、撮像領域３２０ｃ内の各画素は黒い矩形で示している。
【０１６０】
また、各撮像領域間には横方向に１画素、縦方向に３画素に相当する寸法の分離帯が形成されている。したがって、各撮像領域の中心の距離は、横方向と縦方向について同一である。
【０１６１】
３５１ａ、３５１ｂ、３５１ｃ、３５１ｄは物体像である。画素ずらしのために、物体像３５１ａ、３５１ｂ、３５１ｃ、３５１ｄの中心３６０ａ、３６０ｂ、３６０ｃ、３６０ｄはそれぞれ撮像領域３２０ａ、３２０ｂ、３２０ｃ、３２０ｄの中心から撮像領域全体の中心３２０ｅの方向に１／４画素分オフセットさせている。
【０１６２】
この結果、被写界側の所定距離にある平面上に各撮像領域を逆投影すると、物体像の中心３６０ａ、３６０ｂ、３６０ｃ、３６０ｄの逆投影像は一つに重なり、撮像領域３２０ａ、３２０ｂ、３２０ｃ、３２０ｄの各画素の逆投影像はその中心が重なり合わないようにモザイク状に配列される。
【０１６３】
Ｌ３０１およびＬ３０２はレジストレーションずれを検出するために用いる画素列を表す線分である。撮像領域３２０ａと撮像領域３２０ｄは何れもＧ画像を出力するため、撮像領域３２０ａにあって線分Ｌ３０１の下に位置する画素列の信号と、撮像領域３２０ｄにあって線分Ｌ３０１の下に位置する画素列の信号とは類似した形となり、これらに対して相関演算を施すことによって、撮像領域３２０ａと撮像領域３２０ｄ間のレジストレーションずれを知ることができる。同様に、撮像領域３２０ａにあって線分Ｌ３０２の下に位置する画素列の信号と、撮像領域３２０ｄにあって線分Ｌ３０２の下に位置する画素列の信号とに相関演算を施すことによっても、撮像領域３２０ａと撮像領域３２０ｄ間のレジストレーションずれを知ることができる。
【０１６４】
ここでは２対の画像信号を用いて、それぞれレジストレーションずれを算出し、その結果を平均することによって検出精度を高める。
【０１６５】
図２４は撮像領域３２０ａにあって線分Ｌ３０１の下に位置する画素列の信号と撮像領域３２０ｄにあって線分Ｌ３０１の下に位置する画素列の信号とを表した図である。図において、黒丸で示した信号３７０は撮像領域３２０ａにあって線分Ｌ３０１の下に位置する画素列の信号、白丸で示した信号３７１は撮像領域３２０ｄにあって線分Ｌ３０１の下に位置する画素列の信号であり、特に３６１'は図２３の画素３６１の出力、３６２'は画素３６２の出力、３６３'は画素３６３の出力、３６４'は画素３６４の出力である。これらは全てＧ画像信号の一部である。すなわち、信号３７０と信号３７１は色特性において同一のスペクトル分布を有しているので、相関演算の結果は極めて高い精度で画像信号のシフトを表す。
【０１６６】
また、図２５は撮像領域３２０ａにあって線分Ｌ３０２の下に位置する画素列の信号と撮像領域３２０ｄにあって線分Ｌ３０２の下に位置する画素列の信号とを表した図である。図において、黒丸で示した信号３７２は撮像領域３２０ａにあって線分Ｌ３０２の下に位置する画素列の信号、白丸で示した信号３７３は撮像領域３２０ｄにあって線分Ｌ３０２の下に位置する画素列の信号であり、特に３６５'は図２３の画素３６５の出力、３６６'は画素３６６の出力、３６７'は画素３６７の出力、３６８'は画素３６８の出力である。これらもまた全てＧ画像信号の一部である。すなわち、信号３７２と信号３７３は色特性において同一のスペクトル分布を有している。
【０１６７】
図２４と図２５は何れも、温度変化なし、かつ被写体が基準被写体距離に位置した状態であって、画素ずらしを行っているために０．５画素分だけ信号が相対的に横ズレしている。
【０１６８】
この状態に対して温度変化が生じたり、あるいは被写体が基準被写体距離から離れると、信号のシフトが起こる。例えば、被写体が遠いときには、図２４と図２５において、信号３７０が矢印Ａ、信号３７１が矢印Ｂの方向に移動し、被写体が近いときには反対方向に移動する。この挙動は実被写体距帯と基準被写体距離Ｓtとの差ΔＳの関数である式（２２）で表される。
【０１６９】
色特性において同一のスペクトル分布を有した１対の信号間の相対的位置変化量は、相関演算を用いた公知の手法、例えば特公平５−８８４４５号公報に開示されている手法を用いて検出することができる。その出力から設計的に生じる信号のシフト量０．５を減じると、１単位を１画素ピッチ(pxl)として表したレジストレーション変化量ΔＺS／Ｐとなる。
【０１７０】
なお、通常、光学系の特性上、固体撮像素子８２０にはそのナイキスト周波数を上向る高周波成分が投影されている。したがって、物体のパターンによっては物体像の位相が信号の位相に完全には反映されないことがある。これを原因としたレジストレーション変化量ΔＺS／Ｐの検出誤差を減ずるため、線分Ｌ３０１の下に位置する画素列の信号から得たレジストレーション変化量と、これとは物体像のサンプリング位置が０．５画素分ずれた線分Ｌ３０２の下に位置する画素列の信号から得たレジストレーション変化量とを平均する。得られたレジストレーション変化量は精度が高く、これを用いることによって、式（２４）から式（２９）による第１段階の補間処理が理想的に行える。
【０１７１】
以上の処理によって求められた補間画像信号Ｇ２S(i,j)、ＲS(i,j)、ＢS(i,j)は次に第２段階の補正処理に用いられる。
【０１７２】
第２の実施形態とは異なって、この場合、補間画像信号Ｇ２S(i,j)、ＲS(i,j)、ＢS(i,j)は温度変化に起因するレジストレーションずれと被写体距離に起因するレジストレーションずれ、さらには、撮影レンズの製造上の誤差によるレジストレーションずれをまとめて補正したものとなる。
【０１７３】
続く第２段階の補間処理は、第１の実施形態に示したと同様の各々が６００×８００の画像信号Ｇ１(i,j)と補間画像信号Ｇ２T(i,j)、ＲT(i,j)、ＢＴ(i,j)から、ＲＧＢがそれぞれ１２００×１６００の解像度となるＧ画像信号Ｇ'(m,n)、Ｒ画像信号Ｒ'(m,n)、Ｂ画像信号Ｂ'(m,n)を生成するものである。第１の実施形態中の式（１０）から式（２１）内のＧ２T(i,j)、ＲT(i,j)、ＢT(i,j)をそれぞれＧ２S(i,j)、ＲS(i,j)、ＢS(i,j)と置き換えて用いる。
【０１７４】
以上のように、撮像素子は同一のスペクトル分布で形成された略同一視野の２つのＧ画像と、２つのＧ画像と略同一視野のＲ画像やＢ画像を出力するとともに、第１段階の補間処理にて、２つのＧ画像の間隔変化に基づいてＲ画像やＢ画像の位置の補正を行った後、第２段階の補間処理で、ＲＧＢの画像に基づく合成映像信号を形成する。
【０１７５】
また、特開平１０−２８９３１６号公報に開示されているようにエリアベースマッチング方を用いれば、画素毎にレジストレーションずれを求めることが出来る。これを利用して、レジストレーションずれを補正すると、被写体に奥行きがある場合でも画面全体に渡って色ずれがなく鮮鋭度が高い画像を得ることが可能である。この場合には、式（２４）から式（２９）のΔＺSを画素アドレスの関数として取り扱えばよい。
【０１７６】
図２６はレジストレーションずれを検出するために用いる固体撮像素子上の画素の他の設定を説明するための図である。
【０１７７】
図２６において、５２０ａ、５２０ｂ、５２０ｃ、５２０ｄは固体撮像素子８２０の４つの撮像領域である。ここでは説明のため撮像領域５２０ａ、５２０ｂ、５２０ｃ、５２０ｄの各々は画素を８×４個配列してなるが、実施には画素数を拡大して実用的な解像度を得る。撮像領域５２０ａと５２０ｄはＧ画像信号を、撮像領域５２０ｂはＲ画像信号を、撮像領域５２０ｃはＢ画像信号を出力する。撮像領域５２０ａと５２０ｄ内の画素は白抜きの矩形で、撮像領域５２０ｂ内の画素はハッチングを付した矩形で、撮像領域５２０ｃ内の画素は黒い矩形で示している。
【０１７８】
また、各撮像領域間には横方向に１画素、縦方向に０画素に相当する寸法の分離帯が形成されている。したがって、Ｇ画像を出力する２つの撮像領域の中心距離は、横方向に９画素、縦方向に４．５画素である。すなわち、撮像領域８２０ａおよび撮像領域８２０ｄで受光面上の横方向ピッチａ＝Ｐ、縦方向ピッチｂ＝Ｐ、定数ｃ＝４．５、正の整数ｂ＝２としたとき、これらは受光面内で横方向にａ×ｈ×ｃ、縦方向にｂ×ｃだけ離れた位置関係にある。このような関係を作ることにより、温度変化や被写体距離変化に伴って生じるレジストレーションずれが画素の配置されている方向に生じるので、極めて簡単な演算でこれを補正することが可能である。
【０１７９】
５５１ａ、５５１ｂ、５５１ｃ、５５１ｄは物体像である。画素ずらしのために、物体像５５１ａ、５５１ｂ、５５１ｃ、５５１ｄの中心５６０ａ、５６０ｂ、５６０ｃ、５６０ｄはそれぞれ撮像領域５２０ａ、５２０ｂ、５２０ｃ、５２０ｄの中心から撮像領域全体の中心５２０ｅに近づく方向に縦横ともに１／４画素分オフセットさせている。
【０１８０】
この結果、被写界側の所定距離にある平面上に各撮像領域を逆投影すると、物体像の中心５６０ａ、５６０ｂ、５６０ｃ、５６０ｄの逆投影像は一つに重なり、撮像領域５２０ａ、５２０ｂ、５２０ｃ、５２０ｄの各画素の逆投影像はその中心が重なり合わないようにモザイク状に配列される。
【０１８１】
Ｌ５０１およびＬ５０２はレジストレーションずれを検出するために用いる画素列を表す線分である。分かりやすくするために使用する画素にはハッチングを付して示した。撮像領域５２０ａと撮像領域５２０ｄは何れもＧ画像を出力するため、撮像領域５２０ａにあって線分Ｌ５０１の下に位置する画素列の信号と、撮像領域５２０ｄにあって線分Ｌ５０１の下に位置する画素列の信号とは類似した形となり、これらの相関をとることによって、撮像領域５２０ａと撮像領域５２０ｄ間のレジストレーションずれを知ることができる。同様に、撮像領域５２０ａにあって線分Ｌ５０２の下に位置する画素列の信号と、撮像領域５２０ｄにあって線分Ｌ５０２の下に位置する画素列の信号との相関をとることによっても、撮像領域５２０ａと撮像領域５２０ｄ間のレジストレーションずれを知ることができる。
【０１８２】
ここでは２対の画像信号を用いて、それぞれレジストレーションずれを算出し、その結果を平均することによって検出精度を高める。
【０１８３】
温度変化が生じたり、あるいは被写体が基準被写体距離から離れると、信号のシフトが起こる。この挙動は実被写体距離と基準被写体距離Ｓtとの差ΔＳの関数である式（２２）で表される。
【０１８４】
色特性において同一のスペクトル分布を有した１対の信号間の相対的位置変化量は、相関演算を用いた公知の手法、例えば特公平５−８８４４５号公報に開示されている手法を用いて検出することができる。その出力から設計的に生じる信号のシフト量０．５を減じると、１単位を１画素ピッチ(pxl)として表したレジストレーション変化量ΔＺS／Ｐとなる。ここで、使用している画素が１画素おきであることに注意しなければならない。得られたレジストレーション変化量を用いることによって、式（２４）から（２９）による第１段階の補間処理が可能となる。
【０１８５】
また、撮像領域のアスペクト比を縦横逆転させたときには、各撮像領域間には縦方向に１画素、横方向に０画素に相当する寸法の分離帯が形成されることとなり、２つのＧ画像を出力する撮像領域の中心の距離は、縦方向に９画素、横方向に４．５画素である。したがって、これら２つの撮像領域について、受光面上の横方向ピッチａ＝Ｐ、縦方向ピッチｂ＝Ｐ、定数ｃ＝４．５、正の整数ｈ＝２としたとき、受光面内で横方向にａ×ｃ、縦方向にｂ×ｈ×ｃだけ離れた位置関係となる。
【０１８６】
図２７はレジストレーションずれを検出するために用いる固体撮像素子上の画素のさらに他の設定を説明するための図である。この例では画素のピッチが横方向と縦方向で異なっている。
【０１８７】
図２７において、６２０ａ、６２０ｂ、６２０ｃ、６２０ｄは固体撮像素子８２０の４つの撮像領域である。ここでは説明のため撮像領域６２０ａ、６２０ｂ、６２０ｃ、６２０ｄの各々は画素を８×６個配列してなる。撮像領域６２０ａと６２０ｄはＧ画像信号を、撮像領域６２０ｂはＲ画像信号を、撮像領域６２０ｃはＢ画像信号を出力する。撮像領域６２０ａと６２０ｄ内の画素は白抜きの矩形で、撮像領域６２０ｂ内の画素はハッチングを付した矩形で、撮像領域６２０ｃ内の画素は黒い矩形で示している。
【０１８８】
また、各撮像領域間には横方向に１画素、縦方向に３画素に相当する寸法の分離帯が形成されている。したがって、各撮像領域の中心の距離は、横方向と縦方向に同一である。
【０１８９】
６５１ａ、６５１ｂ、６５１ｃ、６５１ｄは物体像である。画素ずらしのために、物体像６５１ａ、６５１ｂ、６５１ｃ、６５１ｄの中心６６０ａ、６６０ｂ、６６０ｃ、６６０ｄはそれぞれ撮像領域６２０ａ、６２０ｂ、６２０ｃ、６２０ｄの中心から撮像領域全体の中心６２０ｅの方向に１／４画素分オフセットさせている。
【０１９０】
この結果、被写界側の所定距離にある平面上に各撮像領域を逆投影すると、物体像の中心６６０ａ、６６０ｂ、６６０ｃ、６６０ｄの逆投影像は一つに重なり、撮像領域６２０ａ、６２０ｂ、６２０ｃ、６２０ｄの各画素の逆投影像はその中心が重なり合わないようにモザイク状に配列される。
【０１９１】
Ｌ６０１およびＬ６０２はレジストレーションずれを検出するために用いる画素列を表す線分である。撮像領域６２０ａと撮像領域６２０ｄは何れもＧ画像を出力するため、撮像領域６２０ａにあって線分Ｌ６０１の下に位置する画素列の信号と、撮像領域６２０ｄにあって線分６０１の下に位置する画素列の信号との相関をとることによって、撮像領域６２０ａと撮像領域６２０ｄ間のレジストレーションずれを知ることができる。同様に、撮像領域６２０ａにあって線分Ｌ６０２の下に位置する画素列の信号と、撮像領域６２０ｄにあって線分Ｌ６０２の下に位置する画素列の信号との相関をとることによっても、撮像領域６２０ａと撮像領域６２０ｄ間のレジストレーションずれを知ることができる。
【０１９２】
ここでは２対の画像信号を用いて、それぞれレジストレーションずれを算出し、その結果を平均することによって検出精度を高める。
【０１９３】
温度変化が生じたり、あるいは被写体が基準被写体距離から離れると、信号のシフトが起こる。この挙動は実被写体距離と基準被写体距離Ｓtとの差ΔＳの関数である式（２２）で表される。
【０１９４】
色特性において同一のスペクトル分布を有した１対の信号間の相対的位置変化量は、相関演算を用いた公知の手法、例えば特公平５−８８４４５号公報に開示されている手法を用いて検出することができる。その出力から設計的に生じる信号のシフト量０．５を減じると、１単位を１画素ピッチ(pxl)として表したレジストレーション変化量ΔＺS／Ｐとなる。得られたレジストレーション変化量を用いることによって、式（２４）から式（２９）による第１段階の補間処理が可能となる。
【０１９５】
【他の実施形態】
なお、本発明は、複数の機器（例えばホストコンピュータ、インタフェイス機器、リーダ、プリンタなど）から構成されるシステムに適用しても、一つの機器からなる装置（例えば、複写機、ファクシミリ装置など）に適用してもよい。
【０１９６】
また、本発明の目的は、前述した実施形態の機能を実現するソフトウェアのプログラムコードを記録した記憶媒体（または記録媒体）を、システムあるいは装置に供給し、そのシステムあるいは装置のコンピュータ（またはＣＰＵやＭＰＵ）が記憶媒体に格納されたプログラムコードを読み出し実行することによっても、達成されることは言うまでもない。この場合、記憶媒体から読み出されたプログラムコード自体が前述した実施形態の機能を実現することになり、そのプログラムコードを記憶した記憶媒体は本発明を構成することになる。また、コンピュータが読み出したプログラムコードを実行することにより、前述した実施形態の機能が実現されるだけでなく、そのプログラムコードの指示に基づき、コンピュータ上で稼働しているオペレーティングシステム（ＯＳ）などが実際の処理の一部または全部を行い、その処理によって前述した実施形態の機能が実現される場合も含まれることは言うまでもない。
【０１９７】
さらに、記憶媒体から読み出されたプログラムコードが、コンピュータに挿入された機能拡張カードやコンピュータに接続された機能拡張ユニットに備わるメモリに書込まれた後、そのプログラムコードの指示に基づき、その機能拡張カードや機能拡張ユニットに備わるＣＰＵなどが実際の処理の一部または全部を行い、その処理によって前述した実施形態の機能が実現される場合も含まれることは言うまでもない。
【０１９８】
以上説明したように、外光を異なる位置から取り込むための複数の開口と、前記複数の開口より取り込んだ外光をそれぞれ別個に受光し、該別個に受光する外光ごとに所定の色成分を抽出する複数の撮像手段と、該撮像手段の出力信号を処理する画像処理手段とを有する撮像装置において、前記画像処理手段は、前記複数の撮像手段の出力画像のうちの少なくとも一つについて位置補正を行った後に、前記複数の撮像手段の出力画像に基づく合成映像信号を形成することにより、次の効果が得られた。
【０１９９】
（１）小型デジタルカラーカメラ用の撮像装置に好適な画像の位置補正方法を得ることができた。
【０２００】
（２）この結果、色ずれや鮮鋭度の低下がない高精細な画像を簡単に得ることが可能となった。
【０２０１】
さらに、複数の撮像領域を備えた撮像素子と、該複数の撮像領域に対応した複数の結像系によって物体像を前記複数の撮像領域上に形成する撮影光学系と、前記撮像素子の出力信号を処理する画像処理手段と、温度計測手段とを有した撮像装置において、前記撮像素子は略同一視野の複数の画像を出力するとともに、前記画像処理手段は、前記複数の画像のうちの少なくとも一つについて前記温度計測手段の出力に応じた位置補正を行った後に、前記複数の画像に基づく合成映像信号を形成することにより、次の効果が得られた。
【０２０２】
（３）温度変化に起因するレジストレーションずれを補正するに好適な方法を得ることができた。
【０２０３】
さらに、複数の撮像領域を備えた撮像素子と、該複数の撮像領域に対応した複数の結像系によって物体像を前記複数の撮像領域上に形成する撮影光学系と、前記撮像素子の出力信号を処理する画像処理手段とを有した撮像装置において、前記撮像素子は略同一視野の複数の画像を出力するとともに、前記画像処理手段は、被写体距離情報に基づいて前記複数の画像のうちの少なくとも一つについて位置補正を行った後に、前記複数の画像に基づく合成映像信号を形成することにより、次の効果が得られた。
【０２０４】
（４）被写体距離変化に起因するレジストレーションずれを補正するに好適な方法を得ることができた。
【０２０５】
さらに、マクロ撮影モードが設定された際に、前記複数の画像のうちの少なくとも一つについて位置補正を行った後に、前記複数の画像に基づく合成映像信号を形成することにより、次の効果が得られた。
【０２０６】
（５）マクロ撮影モードが設定された際の被写体距離変化に起因するレジストレーションずれを補正するに好適な方法を得ることができた。
【０２０７】
（６）測距装置を省略することが可能となった。
【０２０８】
さらに、同一平面上に複数の撮像領域を備えた撮像素子と、該複数の撮像領域上に各々物体像を形成する撮影光学系と、前記撮像素子の出力信号を処理する画像処理手段とを有した撮像装置において、前記撮像素子は同一のスペクトル分布で形成された略同一視野の第１および第２の画像と、該第１および第２の画像とは異なるスペクトル分布で形成され第１および第２の画像と略同一視野の第３の画像を出力するとともに、前記画像処理手段は、前記出力信号の処理過程において、第１および第２の画像の間隔変化に基づいて第３の画像の位置の補正を行った後、前記第１、第２、第３の画像に基づく合成映像信号を形成することにより、次の効果が得られた。
【０２０９】
（７）固体撮像素子の出力そのものを基にレジストレーションずれを補正するに好適な方法を得ることができた。したがって、この構成によれば、温度センサや測距装置を必要としない。
【０２１０】
（８）副次的に、カメラの小型化や低コスト化が可能となった。
【０２１１】
（９）加えて、エリアベースマッチング方等によって画素毎のレジストレーションずれを検出し、画素毎にあるいは小領域毎にレジストレーションずれの補正を異ならせれば、様々な距離が混在する奥行きのある被写体であっても、画面全体に渡って高精細な画像を得ることができる。
【０２１２】
さらに、同一平面上にほぼ同一寸法の第１および第２の撮像領域を備えた撮像素子と、前記第１の撮像領域上に第１の物体像を、前記第２の撮像領域上に第２の物体像を形成する撮影光学系と、画像処理手段とを有する撮像装置において、前記第１および第２の撮像領域は、受光面上で横方向にａ、縦方向にｂのピッチで複数の画素が整列してなり、ｈを正の整数としたときに、受光面内で前記第１および第２の撮像領域は横方向にａ×ｈ×ｃ、縦方向にｂ×ｃ、あるいは、横方向にａ×ｃ、縦方向にｂ×ｈ×ｃだけ離れた位置関係にあるとともに、前記撮像素子は同一のスペクトル分布で形成された略同一視野を持つ第１および第２の画像を形成し、前記画像処理手段は前記第１および第２の画像に基づいた合成映像信号を生成することにより、次の効果が得られた。
【０２１３】
（１０）温度変化や被写体距離変化に伴って生じるレジストレーションずれを極めて簡単な演算で補正可能とするための撮像領域の配置を得ることができた。
【０２１４】
さらに、前記第１および第２の画像の間隔変化を前記出力信号の処理過程において補正し、前記第１および第２の画像に基づく合成映像信号を形成することにより、次の効果が得られた。
【０２１５】
（１１）レジストレーションずれを極めて簡単な演算で補正可能とするための撮像領域の配置を利用して、極めて高精細な映像信号を形成することができた。
【発明の効果】
以上説明したように、上記の実施形態によれば、ＲＧＢの画像のずれを良好に補正して合成することが可能となる。
【図面の簡単な説明】
【図１】デジタルカラーカメラの断面図である。
【図２】デジタルカラーカメラの裏面図である。
【図３】図２を左方から見た側面図である。
【図４】図２を右方から見た側面図である。
【図５】撮像系の詳細図である。
【図６】絞りの平面図である。
【図７】撮影レンズを光射出側から見た図である。
【図８】固体撮像素子の正面図である。
【図９】撮影レンズを光入射側から見た図である。
【図１０】光学フィルターの分光透過率特性を表す図である。
【図１１】マイクロレンズの作用を説明するための図である。
【図１２】レンズ部の断面図である。
【図１３】撮影レンズのレンズ部の間隔設定を説明するための図である。
【図１４】物体像と撮像領域との位置関係を説明するための図である。
【図１５】撮像領域を被写体上に投影したときの画素の位置関係を説明するための図である。
【図１６】ファインダーを構成する第１プリズムおよび第２プリズムの斜視図である。
【図１７】ファインダー系の断面図である。
【図１８】信号処理系のブロック図である。
【図１９】撮像系の要素が熱膨張に伴って位置変化した状態を説明するための図である。
【図２０】撮像領域からの画像信号のアドレスを説明するための図である。
【図２１】測距装置の出力をレジストレーションずれの補正に用いるための信号処理系を表す図である。
【図２２】実被写体距離が無限に大きくなったときの光路を表す図である。
【図２３】レジストレーションずれを検出するために用いる固体撮像素子上の画素を説明するための図である。
【図２４】２つの撮像領域の画素列の信号を表した図である。
【図２５】２つの撮像領域の画素列の信号を表した図である。
【図２６】レジストレーションずれを検出するために用いる固体撮像素子上の画素を説明するための図である。
【図２７】レジストレーションずれを検出するために用いる固体撮像素子上の画素を説明するための図である。
【符号の説明】
１０１ カメラ本体
８００ 撮影レンズ
８００ａ，８００ｂ，８００ｃ，８００ｄ 撮影レンズのレンズ部
８２０ 固体撮像素子
２１０ ＲＧＢ画像処理回路[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus used as a digital electronic still camera, a video movie camera, and the like, a control method thereof, a control program, and a storage medium.
[0002]
[Prior art]
In a digital color camera, a scene image is exposed to a solid-state image sensor such as a CCD or CMOS sensor for a desired time in conjunction with the release button being pressed, and the resulting single-screen image signal is converted to a digital signal. Then, predetermined processing such as YC processing is performed to obtain an image signal of a predetermined format. A digital image signal representing a captured image is recorded in a semiconductor memory for each image. The recorded image signal is read out at any time, either continuously or continuously, reproduced as a signal that can be displayed or printed, and output to a monitor or the like for display.
[0003]
In addition, the applicant of the present application has previously proposed a technique for generating an RGB image using a trinocular optical system or a quadruple optical system, and synthesizing these to obtain a video signal. This technique is extremely effective in realizing a thin imaging system.
[0004]
[Problems to be solved by the invention]
However, the method using the above-described trinocular optical system or quadruple optical system has a problem in that RGB images may be displaced due to the parallax of the optical system.
[0005]
The RGB image misalignment causes a color misalignment of the output image and a decrease in sharpness, which is not preferable.
Japanese Patent Laid-Open No. 6-6680 and Japanese Patent Laid-Open No. 8-116490 provide a technique for obtaining a composite image by performing position correction based on a pair of images having parallax, but individually obtaining RGB images. It is not a thing.
[0006]
Accordingly, the present invention has been made in view of the above-described problems, and an object of the present invention is to provide an imaging apparatus capable of satisfactorily correcting and synthesizing image deviation, a control method thereof, a control program, and a storage medium. That is.
[0010]
[Means for Solving the Problems]
  In order to solve the above-mentioned problems and achieve the purpose,An image pickup apparatus according to the present invention processes an image pickup device having a plurality of image pickup areas on the same plane, a photographing optical system that forms object images on the plurality of image pickup areas, and an output signal of the image pickup element. Image processing means, wherein the imaging device is formed with the same spectral distribution of the first and second images having substantially the same field of view, and the first and second images are formed with different spectral distributions. While outputting a third image having substantially the same field of view as the first and second images, the image processing means, in the process of processing the output signal,Based on at least one of temperature change, subject distance change, manufacturing optical system manufacturing errorOf the first and second imagesThe imaging position on the image sensorAfter correcting the position of the third image based on the interval change, a composite video signal based on the first, second, and third images is formed.
[0011]
  In addition, an imaging apparatus according to the present invention includes an imaging device having first and second imaging regions having substantially the same dimensions on the same plane, a first object image on the first imaging region, and the first object image on the first imaging region. An imaging optical system for forming a second object image on the two imaging areas, and image processing means for processing an output signal of the imaging element, wherein the first and second imaging areas are on a light receiving surface. In the light receiving surface, the first and second imaging regions are a in the horizontal direction when a plurality of pixels are aligned at a pitch of a in the horizontal direction and b in the vertical direction. Xh × c, b × c in the vertical direction, a × c in the horizontal direction, and b × h × c in the vertical direction, and the imaging elements were formed with the same spectral distribution Forming first and second images having substantially the same field of view;A process of processing the output signal based on at least one of a change in temperature, a change in subject distance, a manufacturing error of the photographing optical system, and a change in the interval between the imaging positions of the first and second images on the image sensor. To generate a composite video signal based on the first and second imagesIt is characterized by that.
[0013]
  The image pickup apparatus control method according to the present invention includes an image pickup device having a plurality of image pickup regions on the same plane, a photographing optical system that forms object images on the plurality of image pickup regions, and the image pickup device. An image pickup apparatus control method for controlling an image pickup apparatus including an image processing means for processing an output signal, wherein the image processing means is substantially formed by the same spectral distribution output from the image pickup device. For the first and second images having the same field of view, and the third image having a spectral distribution different from that of the first and second images and having substantially the same field of view as the first and second images, the output signal In the process ofBased on at least one of temperature change, subject distance change, manufacturing optical system manufacturing errorOf the first and second imagesThe imaging position on the image sensorAfter the position of the third image is corrected based on the interval change, a composite video signal based on the first, second, and third images is formed.
[0014]
  According to another aspect of the present invention, there is provided a method for controlling an imaging apparatus, comprising: an imaging element having first and second imaging areas having substantially the same dimensions on the same plane; and a first object image on the first imaging area. An imaging apparatus control method for controlling an imaging apparatus, comprising: an imaging optical system that forms a second object image on the second imaging area; and an image processing unit that processes an output signal of the imaging element. In the first and second imaging regions, when a plurality of pixels are aligned at a pitch of a in the horizontal direction and b in the vertical direction on the light receiving surface, and h is a positive integer, Within the light receiving surface, the first and second imaging regions are separated by a × h × c in the horizontal direction, b × c in the vertical direction, or a × c in the horizontal direction, and b × h × c in the vertical direction. In the positional relationship, the image processing meansThe first and second images having substantially the same field of view formed by the same spectral distribution formed by the imaging device based on at least one of a temperature change, a change in subject distance, and a manufacturing error of the photographing optical system. A change in the interval of the imaging position on the image sensor is corrected in the process of the output signal, and a composite video signal based on the first and second images is formed.It is characterized by that.
[0018]
  The control program according to the present invention includes an imaging device having a plurality of imaging regions on the same plane, a photographing optical system that forms object images on the plurality of imaging regions, and an output signal of the imaging device. A control program for controlling an imaging apparatus including an image processing unit to process, wherein the image processing unit includes first and substantially identical visual fields formed by the same spectral distribution output from the imaging device. In the process of processing the output signal for a second image and a third image formed with a different spectral distribution from the first and second images and having substantially the same field of view as the first and second images,Based on at least one of temperature change, subject distance change, manufacturing optical system manufacturing errorOf the first and second imagesThe imaging position on the image sensorA code for a step of forming a composite video signal based on the first, second, and third images after correcting the position of the third image based on a change in the interval; .
[0019]
  In addition, the control program according to the present invention includes an imaging device having first and second imaging areas having substantially the same dimensions on the same plane, a first object image on the first imaging area, and the first object image. A control program for controlling an image pickup apparatus comprising: a photographing optical system that forms a second object image on two image pickup regions; and an image processing means that processes an output signal of the image pickup device. The first and second imaging regions have a plurality of pixels aligned at a pitch of a in the horizontal direction and b in the vertical direction on the light receiving surface, and when the h is a positive integer, The first and second imaging regions are in a positional relationship of a × h × c in the horizontal direction, b × c in the vertical direction, or a × c in the horizontal direction and b × h × c in the vertical direction, A control program is provided in the image processing means.The first and second images having substantially the same field of view formed by the same spectral distribution formed by the imaging device based on at least one of a temperature change, a change in subject distance, and a manufacturing error of the photographing optical system. A change in the interval of the imaging position on the image sensor is corrected in the process of the output signal, and a composite video signal based on the first and second images is formed.It is characterized by that.
[0024]
  In addition, an imaging apparatus according to the present invention includes an imaging device having first and second imaging regions having substantially the same dimensions on the same plane, a first object image on the first imaging region, and the first object image on the first imaging region. An imaging optical system that forms a second object image on the two imaging areas, and a signal processing device that processes an output signal of the imaging element, wherein the first and second imaging areas are on a light receiving surface In the light receiving surface, the first and second imaging regions are a in the horizontal direction when a plurality of pixels are aligned at a pitch of a in the horizontal direction and b in the vertical direction. Xh × c, b × c in the vertical direction, a × c in the horizontal direction, and b × h × c in the vertical direction, and the imaging elements were formed with the same spectral distribution Forming a first image and a second image having substantially the same field of view;Processing of the output signal to change the interval between the image formation positions of the first and second images on the image sensor based on at least one of temperature change, subject distance change, and manufacturing error of the photographing optical system Correcting in the process to form a composite video signal based on the first and second imagesIt is characterized by that.
[0034]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.
[0035]
(First embodiment)
2, 3, and 4 are views showing the appearance of the digital color camera according to the first embodiment of the present invention. FIG. 2 is a rear view of the camera, and FIG. 3 is a left side of FIG. 4 is a side view of the camera viewed from the right side of FIG.
[0036]
2, 3, and 4, 101 is a card-type camera body, 105 is a main switch, 106 is a release button (see also FIG. 1), and 107, 108, and 109 are for the user to set the camera state. The switch 150 is a display section for the remaining number of images that can be taken. Reference numeral 111 denotes a viewfinder eyepiece window from which object light incident on the viewfinder exits. 114 is a standardized connection terminal for connecting to an external computer or the like to transmit and receive data, 200 is a contact protection cap that also serves as a grip, and 120 is coaxial with the release button 106 disposed on the front of the camera. The formed protrusion 890 is an imaging system located inside. The contact protection cap 200 is made of a soft resin or rubber.
[0037]
The camera body 101 may be the same size as the PC card and attached to a personal computer. In this case, the length is 85.6 mm, the width is 54.0 mm, the thickness is 3.3 mm (PC card standard Type 1), or the thickness is 5.0 mm (PC card standard Type 2).
[0038]
FIG. 1 is a cross-sectional view of a digital color camera, taken along a plane that passes through the release button 106, the imaging system 890, and the viewfinder eyepiece window 111. In the figure, 123 is a housing for holding each component of the camera, 125 is a back cover, 890 is an imaging system, 121 is a switch that is turned on when the release button 106 is pressed, and 124 projects the release button 106. A coil spring that biases in the direction. The switch 121 includes a first stage circuit that closes when the release button 106 is pressed halfway, and a second stage circuit that closes when the release button 106 is pressed down to the end.
[0039]
Reference numerals 112 and 113 denote first and second prisms forming a finder optical system. The first and second prisms 112 and 113 are made of a transparent material such as acrylic resin, and both have the same refractive index. Moreover, it is solid so that the light beam goes straight through the inside. A region 113b subjected to light-shielding printing is formed around the object light exit surface 113a of the second prism 113 to limit the passage range of the viewfinder exit light. In addition, as shown in the drawing, this printing region extends to the portion facing the side surface of the second prism 113 and the object light exit surface 113a.
[0040]
The imaging system 890 is configured by attaching a protective glass 160, a photographic lens 800, a sensor substrate 161, and relay members 163 and 164 for adjusting the sensor position to the housing 123. On the sensor substrate 161, a solid-state image sensor 820, a sensor cover glass 162, and a temperature sensor 165 as temperature measuring means are attached, and a diaphragm 810, which will be described later, is adhered to the photographing lens 800. The relay members 163 and 164 are movably fitted in the through-holes 123a and 123b of the housing, and adjusted so that the positional relationship between the photographing lens 800 and the solid-state imaging device 820 is appropriate, and then the sensor substrate 161 and the housing. 123 is adhered and fixed.
[0041]
Furthermore, the protective glass 160 and the sensor cover glass 162 are printed for light shielding in areas other than the effective portion in order to reduce the incidence of light from outside the imaging range to the solid-state imaging device 820 as much as possible. . 162a and 162b shown in the figure are print areas. Further, a non-printing area is provided with a permeable coating in order to avoid the occurrence of ghosts.
[0042]
Details of the imaging system 890 will be described.
[0043]
FIG. 5 is a detailed view of the imaging system 890. The basic elements of the photographing optical system are a photographing lens 800, a diaphragm 810, and a solid-state image sensor 820. The imaging system 890 includes four optical systems for separately obtaining a green (G) image signal, a red (R) image signal, and a blue (B) image signal.
[0044]
The assumed object distance is several meters, which is very large compared to the optical path length of the imaging system. Therefore, if the incident surface is aplanatic with respect to the assumed object distance, the incident surface is a concave surface with a very small curvature. Replaced.
[0045]
As shown in FIG. 7 as viewed from the light emitting side, the photographing lens 800 has four lens portions 800a, 800b, 800c, and 800d, each of which is constituted by a ring-shaped spherical surface. An infrared cut filter having a low transmittance in the wavelength region of 670 nm or more is formed on the lens portions 800a, 800b, 800c, and 800d, and a light-shielding film is formed on the flat portion 800f shown by hatching. ing.
[0046]
Each of the four lens portions 800a, 800b, 800c, and 800d is an imaging system, and as will be described later, the lens portions 800a and 800d are for a green (G) image signal, and the lens portion 800b is a red (R) image signal. The lens portion 800c is for blue (B) image signals. The focal lengths at the respective representative wavelengths of RGB are all 1.45 mm.
[0047]
In order to suppress the high frequency component of the object image above the Nyquist frequency determined by the pixel pitch of the solid-state imaging device 820 and increase the response on the low frequency side, the light incident surface 800e of the photographing lens 800 has transmittances indicated by 854a and 854b. A distribution area is provided. This is called apodization, and is a technique for obtaining a desired MTF by giving a characteristic that has the highest transmittance at the center of the aperture and decreases toward the periphery.
[0048]
The diaphragm 810 has four circular openings 810a, 810b, 810c, and 810d as shown in FIG. The object light incident on the light incident surface 800e of the photographic lens 800 from each of these is emitted from the four lens portions 800a, 800b, 800c, and 800d, and forms four object images on the imaging surface of the solid-state imaging device 820. . The diaphragm 810, the light incident surface 800e, and the imaging surface of the solid-state imaging device 820 are arranged in parallel. The aperture 810 and the four lens portions 800a, 800b, 800c, and 800d are set in a positional relationship that satisfies the condition of Zinken-Sommer, that is, a positional relationship that excludes coma and astigmatism at the same time.
[0049]
In addition, the field curvature is favorably corrected by dividing the lens portions 800a, 800b, 800c, and 800d into an annular shape. That is, an image surface formed by one spherical surface is a spherical surface represented by the curvature of Petzpearl, and the image surface is flattened by connecting a plurality of these. As shown in FIG. 12, which is a sectional view of the lens unit, the center position PA of the spherical surface of each annular zone is all the same from the condition for preventing coma and astigmatism, and further in such a form. If the lens portions 800a, 800b, 800c, and 800d are divided, the distortion of the object image generated in each annular zone is completely the same, and overall high MTF characteristics can be obtained. What is necessary is just to correct the distortion left in this case by arithmetic processing. If the distortion generated in each lens unit is the same, the correction process can be simplified.
[0050]
The radius of the ring-shaped spherical surface is set so as to increase in a differential series from the center ring zone to the periphery, and the amount of increase is mλ / (n−1). Here, λ is a representative wavelength of an image formed by each lens unit, n is a refractive index of the photographing lens 800 with respect to the representative wavelength, and m is a positive constant. With this configuration, the optical path length difference between the light beams passing through the adjacent annular zones is mλ and the emitted light has the same phase, and functions as a diffractive optical element when the number of annular zones is increased by increasing the number of divided lens portions. .
[0051]
In addition, in order to suppress the flare generated in the step portion of the annular zone as much as possible, a step parallel to the principal ray is provided as shown in the figure. Since the lens portions 800a, 800b, 800c, and 800d are separated from the pupil, the flare suppression effect by such a configuration is great.
[0052]
FIG. 8 is a front view of the solid-state image sensor 820. The solid-state imaging device 820 includes four imaging regions 820a, 820b, 820c, and 820d corresponding to the four object images to be formed. Although the drawing is simplified, each of the imaging regions 820a, 820b, 820c, and 820d is a 1.248 mm × 0.936 mm region in which 800 × 600 pixels having a vertical and horizontal pitch P of 1.56 μm are arranged. And the diagonal dimension of each imaging area is 1.56 mm. A separation band of 0.156 mm in the horizontal direction and 0.468 mm in the vertical direction is formed between the imaging regions. Accordingly, the distance between the centers of the respective imaging regions is the same in the horizontal direction and the vertical direction, and is 1.404 mm.
[0053]
That is, in the imaging area 820a and the imaging area 820d, when the horizontal pitch a = P, the vertical pitch b = P, the constant c = 900, and the positive integer h = 1 on the light receiving surface, The positions are a × h × c in the direction and b × c apart in the vertical direction. By creating such a relationship, it is possible to correct a registration shift caused by a temperature change or a subject distance change by a very simple calculation. The registration shift is an inconsistency of object image sampling positions that occur between imaging systems having different received light spectrum distributions such as an R imaging system / G imaging system / B imaging system in a multi-plate color camera or the like.
[0054]
851a, 851b, 851c, and 851d in the figure are image circles in which object images are formed. The image circles 851a, 851b, 851c, and 851d are shaped like the aperture opening, although the illuminance decreases at the periphery due to the effects of the print areas 162a and 162b provided on the protective glass 160 and the sensor cover glass 162. The circular shape is determined by the size of the exit-side spherical surface of the lens 800. Therefore, overlapping portions are generated in the image circles 851a, 851b, 851c, and 851d.
[0055]
Returning to FIG. 5, hatched portions 852 a and 852 b in the region sandwiched between the stop 810 and the photographing lens 800 are optical filters formed on the light incident surface 800 e of the photographing lens 800. As shown in FIG. 9 when the photographing lens 800 is viewed from the light incident side, the optical filters 852a, 852b, 852c, and 852d are formed in a range that completely includes the aperture openings 810a, 810b, 810c, and 810d.
[0056]
The optical filters 852a and 852d have a spectral transmittance characteristic that mainly transmits green as indicated by G in FIG. 10, the optical filter 852b has a spectral transmittance characteristic that mainly transmits red, indicated by R, and The optical filter 852c has a spectral transmittance characteristic indicated by B, which mainly transmits blue light. That is, these are primary color filters. The object image formed in the image circles 851a and 851d is a green light component and the object formed in the image circle 851b as a product of the characteristics of the infrared cut filters formed in the lens portions 800a, 800b, 800c, and 800d. The image is a red light component, and the object image formed on the image circle 851c is a blue light component.
[0057]
If substantially the same focal length is set for each wavelength of the representative wavelength of each spectral distribution in each imaging system, a color image in which chromatic aberration is corrected can be obtained by combining these image signals. Ordinarily, achromatism for removing chromatic aberration requires a combination of at least two lenses having different dispersions. On the other hand, there is a cost reduction effect due to the fact that each imaging system has a single structure. Furthermore, the effect of thinning the imaging system is great.
[0058]
On the other hand, optical filters are also formed on the four imaging regions 820a, 820b, 820c, and 820d of the solid-state imaging device 820. The spectral transmittance characteristics of the imaging areas 820a and 820d are indicated by G in FIG. 10, the spectral transmittance characteristics of the imaging area 820b are indicated by R in FIG. 10, and the spectral transmittance characteristics of the imaging area 820c are shown in FIG. This is indicated by B. That is, the imaging areas 820a and 820d are sensitive to green light (G), the imaging area 820b is sensitive to red light (R), and the imaging area 820c is sensitive to blue light (B).
[0059]
Since the received light spectrum distribution of each imaging region is given as the product of the spectral transmittance of the pupil and the imaging region, the combination of the imaging system pupil and the imaging region is almost selected by the wavelength region even if there are overlapping image circles. .
[0060]
Further, a micro lens 821 is formed for each light receiving portion (for example, 822a, 822b) of each pixel on the imaging regions 820a, 820b, 820c, 820d. The micro lens 821 has an eccentric arrangement with respect to the light receiving portion of the solid-state imaging device 820, and the amount of eccentricity is set to be zero at the center of each imaging region 820a, 820b, 820c, 820d, and to increase toward the periphery. Yes. Further, the eccentric direction is a direction of a line segment connecting the center point of each of the imaging regions 820a, 820b, 820c, and 820d and each light receiving unit.
[0061]
FIG. 11 is a diagram for explaining the action of the microlens, and is a cross-sectional view showing an enlarged view of the light receiving portions 822a and 822b in the positions where the imaging region 820a and the imaging region 820b are adjacent to each other. The micro lens 821a is decentered upward in the drawing with respect to the light receiving portion 822a, while the micro lens 821b is decentered in the lower portion of the drawing with respect to the light receiving portion 822b. As a result, the luminous flux incident on the light receiving portion 822a is limited to the area indicated by hatching as 823a, and the luminous flux incident on the light receiving portion 822b is limited to the area indicated by hatching as 823b.
[0062]
The luminous flux regions 823a and 823b are inclined in opposite directions, and are directed toward the lens portions 800a and 800b, respectively. Therefore, if the amount of eccentricity of the microlens is appropriately selected, only the light beam emitted from the specific pupil enters each imaging region. That is, the object light that has passed through the aperture 810a is mainly photoelectrically converted in the imaging region 820a, and the object light that has passed through the aperture 810b is mainly photoelectrically converted in the imaging region 820b and passed through the aperture 810c. The amount of eccentricity can be set so that light is mainly photoelectrically converted in the imaging region 820c, and further, object light that has passed through the aperture 810d of the diaphragm is mainly photoelectrically converted in the imaging region 820d.
[0063]
In addition to the method for selectively assigning pupils to each imaging region using the wavelength range described above, the method for selectively assigning pupils to each imaging region using a microlens is also applied. In addition, by providing printing areas on the protective glass 160 and the sensor cover glass 162, it is possible to reliably prevent crosstalk between wavelengths while allowing overlap of image circles. That is, the object light that has passed through the aperture 810a is photoelectrically converted in the imaging region 820a, the object light that has passed through the aperture 810b is photoelectrically converted in the imaging region 820b, and the object light that has passed through the aperture 810c is captured in the imaging region. The object light that has undergone photoelectric conversion at 820c and has passed through the aperture 810d of the diaphragm is photoelectrically converted at the imaging region 820d. Therefore, the imaging areas 820a and 820d output G image signals, the imaging area 820b outputs R image signals, and the imaging area 820c outputs B image signals.
[0064]
In an image processing system (not shown), a plurality of imaging regions of the solid-state imaging device 820 each form a color image based on selective photoelectric conversion output obtained from one of a plurality of object images. At this time, distortion of each imaging system is corrected in calculation, and signal processing for forming a color image is performed on the basis of the G image signal including the peak wavelength 555 nm of the relative visibility. Since the G object image is formed in the two imaging regions 820a and 820d, the number of pixels is twice that of the R image signal and the B image signal, and a particularly high-definition image can be obtained in a wavelength range with high visibility. Can be done. At this time, a method of pixel shifting that increases the resolution with a small number of pixels is used by shifting the object images on the imaging regions 820a and 820d of the solid-state imaging device by ½ pixel vertically and horizontally. As shown in FIG. 8, the object image centers 860a, 860b, 860c, and 860d that are also the centers of the image circles are respectively set in the directions of arrows 861a, 861b, 861c, and 861d from the centers of the imaging regions 820a, 820b, 820c, and 820d. An offset of / 4 pixel is formed to constitute a ½ pixel shift as a whole. Here, the lengths of the arrows 861a, 861b, 861c, and 861d are not shown to represent the offset amount.
[0065]
In comparison with an imaging system that uses a single photographic lens, if the pixel pitch of the solid-state image sensor is fixed, a Bayer array system in which an RGB color filter is formed with 2 × 2 pixels as a set on the solid-state image sensor. In comparison, this method has an object image size of 1 / √4. As a result, the focal length of the photographic lens is reduced to approximately 1 / √4 = ½. Therefore, it is extremely advantageous for making the camera thinner.
[0066]
Next, the positional relationship between the photographing lens and the imaging region will be described. As described above, each imaging area is 1.248 mm × 0.936 mm, and these are located with a separation band of 0.156 mm in the horizontal direction and 0.468 mm in the vertical direction. The center interval between adjacent imaging regions is 1.404 mm in the vertical and horizontal directions, and 1.9856 mm in the diagonal direction.
[0067]
Focusing on the imaging areas 820a and 820d, an object image at a reference object distance of 2.38 m is obtained by subtracting a diagonal size of 0.5 pixels from an imaging area interval of 1.9856 mm for pixel shifting. It shall be formed on an image pick-up part at an interval of 9845 mm. As a result, as shown in FIG. 13, the distance between the lens portions 800a and 800d of the photographing lens 800 is set to 1.9832 mm. In the figure, arrows 855a and 855d are symbols representing an imaging system having positive power by the lens portions 801a and 800d of the photographing lens 800, rectangles 856a and 856d are symbols representing the ranges of the imaging regions 820a and 820d, and L801 and L802 are symbols. This is the optical axis of the imaging systems 855a and 855d. Since the light incident surface 800e of the photographic lens 800 is a plane, and the lens portions 800a and 800d which are light exit surfaces are Fresnel lenses made of concentric spherical surfaces, a straight line passing through the spherical center and perpendicular to the light incident surface is formed. It becomes the optical axis.
[0068]
Next, for simplicity, the positional relationship between the object image and the imaging region and the positional relationship of the pixels when projected onto the subject will be described with the number of vertical and horizontal pixels set to 1/100. 14 and 15 are explanatory diagrams thereof.
[0069]
First, in FIG. 14, 320 a, 320 b, 320 c, and 320 d are four imaging regions of the solid-state imaging device 820. Here, for explanation, each of the imaging regions 320a, 320b, 320c, and 320d is formed by arranging 8 × 6 pixels. The imaging areas 320a and 320d output G image signals, the imaging area 320b outputs R image signals, and the imaging area 320c outputs B image signals. The pixels in the imaging regions 320a and 320d are white rectangles, the pixels in the imaging region 320b are hatched rectangles, and the pixels in the imaging region 320c are black rectangles.
[0070]
In addition, a separation band having a size corresponding to one pixel in the horizontal direction and three pixels in the vertical direction is formed between the imaging regions. Therefore, the center distance of the imaging region for outputting the G image is the same in the horizontal direction and the vertical direction.
[0071]
Reference numerals 351a, 351b, 351c, and 351d are object images. For pixel shifting, the centers 360a, 360b, 360c, and 360d of the object images 351a, 351b, 351c, and 351d are each ¼ from the centers of the imaging regions 320a, 320b, 320c, and 320d toward the center 320e of the entire imaging region. The pixel is offset.
[0072]
As a result, when each imaging region is back-projected onto a plane at a predetermined distance on the object side, the result is as shown in FIG. Also on the object side, the backprojected images of the pixels in the imaging regions 320a and 320d are white rectangles 362a, and the backprojected image of the pixels in the imaging region 320b are hatched rectangles 362b in the imaging region 320c. The backprojected image of the pixel is indicated by a black rectangle 362c.
[0073]
The back-projected images of the object image centers 360a, 360b, 360c, and 360d overlap as a point 361, and the pixels of the imaging regions 320a, 320b, 320c, and 320d are back-projected so that the centers do not overlap. The white rectangles output the G image signal, the hatched rectangles output the R image signal, and the blacked rectangles output the R image signal. As a result, the image pickup device having a Bayer array color filter on the subject. Sampling equivalent to is performed.
[0074]
Next, the finder system will be described. This viewfinder device is thinned by utilizing the property that light is totally reflected at the interface between a medium having a high refractive index and a medium having a low refractive index. Here, a configuration when used in the air will be described.
[0075]
FIG. 16 is a perspective view of the first prism 112 and the second prism 113 constituting the viewfinder. The first prism 112 has four surfaces 112c, 112d, 112e, and 112f at positions facing the surface 112a (see FIG. 17), and object light incident from the surface 112a is emitted from the surfaces 112c, 112d, 112e, and 112f. . The surfaces 112a, 112c, 112d, 112e, and 112f are all flat surfaces.
[0076]
On the other hand, the second prism 113 includes surfaces 113c, 113d, 113e, and 113f at positions facing the surfaces 112c, 112d, 112e, and 112f of the first prism 112. The object light incident from the surfaces 113c, 113d, 113e, and 113f exits from the surface 113a. The surfaces 112c, 112d, 112e, 112f of the first prism 112 and the surfaces 113c, 113d, 113e, 113f of the second prism 113 are opposed to each other with a slight air gap. Therefore, the surfaces 113c, 113d, 113e, and 113f of the second prism 113 are also flat surfaces.
[0077]
In addition, since it is necessary to observe an object with an eye close to the finder, the finder system should not have refractive power. Therefore, since the object light incident surface 112a of the first prism 112 is a flat surface, the object light exit surface 113a of the second prism 113 is also a flat surface. Moreover, these are parallel surfaces. Furthermore, since the imaging system 890 and the signal processing system obtain a rectangular image as a comprehensive process including distortion correction in calculation, the observation visual field that can be seen through the viewfinder needs to be rectangular. Therefore, the optically effective surfaces of the first prism 112 and the second prism 113 are both symmetrical in the vertical and horizontal directions. The line of intersection of the two symmetry planes is the finder optical axis L1.
[0078]
FIG. 17 is a diagram for explaining the role of the surfaces facing each other with an air gap. The finder system is configured by combining the first prism 112 and the second prism 113 in a predetermined positional relationship, and a state where the optical path is reversely traced from the position of the observer's eye in the main cross section is viewed from above.
[0079]
In the figure, the point P1 is a point farthest from the finder that can overlook the entire observation field when the pupil of the observer's eye is infinitely narrowed, and is a so-called eye point.
[0080]
Consider a light ray 130 that emits a point P1 and has an angle slightly exceeding the viewfinder field angle ω. The light ray 130 is refracted by the surface 113a of the second prism 113 and reaches the surface 113c. The inclination angle of the surface 113c is set so that the incident angle β of the light beam corresponding to the finder field angle ω becomes a critical angle. Therefore, the incident angle of the light beam 130 on the incident surface 113c slightly exceeds the critical angle. As a result, the light cannot be emitted from the surface 113c and is totally reflected. A side surface of the second prism 113 has a print area 113b for light shielding, and the light beam 130 is absorbed here. Therefore, from the observer, the subject is not seen in the direction of the light beam 130, and becomes a dark part indicating that the object is outside the object scene.
[0081]
Next, consider a light ray 131 that emits a point P1 and has an angle slightly smaller than the viewfinder field angle ω. The light beam 131 is refracted by the surface 113a of the second prism 113 and reaches the surface 113c. As described above, the inclination angle of the surface 113c is set so that the incident angle β of the light beam corresponding to the finder field angle ω becomes a critical angle. The incident angle of the light ray 131 on the surface 113c is slightly smaller than the critical angle. The light ray 131 exits from the surface 113c, passes through a slight air gap, and then enters the surface 112c of the first prism 112. Since the surface 113c and the surface 112c opposite to the surface 113c have the same shape, the traveling direction of the light beam in the first prism 112 is the same as the traveling direction in the second prism 113.
[0082]
For light rays reaching the first prism 112, the overall characteristics of the first prism 112 and the second prism 113 are equivalent to a parallel plate. As a result, the light beam 131 exits from the surface 112a with an angle equal to the incident angle to the surface 113a. That is, the viewing angle β is equal to the viewfinder field angle ω. Accordingly, the observer can see the subject in the direction of the light beam 131 and can recognize the object scene. The optical paths of the light beams 130 and 131 shown above represent that the finder field can be limited using the critical angle, that is, a clear finder field profile can be obtained.
[0083]
As described above, since the first prism 112 and the second prism 113 have a plane-symmetrical shape, the optical path shown in FIG. 17 is also folded with respect to the finder optical axis L1. Furthermore, the finder field of view is restricted by the same principle in the relationship between the surfaces 112e and 112f of the first prism 112 and the surfaces 113e and 113f of the second prism 113, respectively. In the above, for the sake of simplicity, the light beam is traced in reverse from the position of the observer's eye. However, if the light path is considered in the direction in which the light emitted from the subject travels, the first prism 112 from within the observation field can be obtained from the reversibility of the light beam. This is equivalent to the fact that the object light incident on the object light incident surface 112a passes through the air gap, and the object light incident on the object light incident surface 112a of the first prism 112 from outside the observation field does not pass through the air gap. Therefore, as a general finder characteristic, a substantially rectangular finder field can be obtained from the position of the point P1.
[0084]
Next, a schematic configuration of the signal processing system will be described.
[0085]
FIG. 18 is a block diagram of the signal processing system. This camera is a single-plate digital color camera using a solid-state image sensor 120 such as a CCD or CMOS sensor, and drives the solid-state image sensor 120 continuously or once to generate an image signal representing a moving image or a still image. obtain. Here, the solid-state imaging device 120 is a type of imaging device that converts exposed light into an electrical signal for each pixel, accumulates charges corresponding to the light amount, and reads the charges.
[0086]
Note that only the portions directly related to the present invention are shown in the drawings, and illustration and description of portions not directly related to the present invention are omitted.
[0087]
As shown in FIG. 18, the imaging apparatus includes an imaging system 10, an image processing system 20 that is an image processing unit, a recording / reproducing system 30, and a control system 40. Further, the imaging system 10 includes a photographing lens 100, a diaphragm 110, and a solid-state imaging device 120. The image processing system 20 includes an A / D converter 500, an RGB image processing circuit 210, and a YC processing circuit 230, and a recording / reproducing system. 30 includes a recording processing circuit 300 and a reproduction processing circuit 310, and the control system 40 includes a system control unit 400, an operation detection unit 430, a temperature sensor 165, and a solid-state image sensor driving circuit 420.
[0088]
The imaging system 10 is an optical processing system that forms an image of light from an object on the imaging surface of the solid-state imaging device 120 through the diaphragm 110 and the photographing lens 100, and exposes the subject image to the solid-state imaging device 120. As described above, an imaging device such as a CCD or a CMOS sensor is effectively applied to the solid-state imaging device 120, and an image signal representing a continuous moving image by controlling the exposure time and the exposure interval of the solid-state imaging device 120, Alternatively, an image signal representing a still image by one exposure can be obtained.
[0089]
As described above, the solid-state imaging device 120 is an imaging device that has 800 pixels in the long side direction and 600 pixels in the short side direction for each imaging region, and has a total of 19.2 million pixels. Optical filters of three primary colors (R), green (G), and blue (B) are arranged for each predetermined region.
[0090]
Image signals read from the solid-state imaging device 120 are supplied to the image processing system 20 via the A / D converter 500, respectively. The A / D converter 500 is, for example, a signal conversion circuit that converts and outputs, for example, a 10-bit digital signal according to the amplitude of the signal of each exposed pixel, and the subsequent image signal processing is performed by digital processing. Executed.
[0091]
The image processing system 20 is a signal processing circuit that obtains an image signal of a desired format from R, G, and B digital signals. The R, G, and B color signals are converted into a luminance signal Y and a color difference signal (RY), It is converted into a YC signal represented by (BY).
[0092]
The RGB image processing circuit 210 is a signal processing circuit that processes an image signal of 800 × 600 × 4 pixels received from the solid-state imaging device 120 via the A / D converter 500, and includes a white balance circuit, a gamma correction circuit, and an interpolation. It has an interpolation calculation circuit for increasing the resolution by calculation.
[0093]
The YC processing circuit 230 is a signal processing circuit that generates the luminance signal Y and the color difference signals RY and BY. A high-frequency luminance signal generation circuit that generates a high-frequency luminance signal YH, a low-frequency luminance signal generation circuit that generates a low-frequency luminance signal YL, and a color difference signal generation circuit that generates color difference signals RY and BY Has been. The luminance signal Y is formed by combining the high frequency luminance signal YH and the low frequency luminance signal YL.
[0094]
The recording / reproducing system 30 is a processing system that outputs an image signal to the memory and outputs an image signal to the liquid crystal monitor 4, and a recording processing circuit 300 that performs writing and reading processing of the image signal to and from the memory; And a reproduction processing circuit 310 that reproduces an image signal read from the memory and outputs it as a monitor output. More specifically, the recording processing circuit 300 includes a compression / expansion circuit that compresses YC signals representing still images and moving images in a predetermined compression format and expands the compressed data when it is read out.
[0095]
The compression / decompression circuit has a frame memory or the like for signal processing. The YC signal from the image processing system 20 is stored in this frame memory for each frame, and is read and compressed and encoded for each of a plurality of blocks. The compression coding is performed, for example, by performing two-dimensional orthogonal transformation, normalization, and Huffman coding on the image signal for each block.
[0096]
The reproduction processing circuit 310 is a circuit that converts the luminance signal Y and the color difference signals RY and BY into, for example, RGB signals by matrix conversion. The signal converted by the reproduction processing circuit 310 is output to the liquid crystal monitor 4 and a visible image is displayed and reproduced.
[0097]
The control system 40 includes control circuits of respective units that control the imaging system 10, the image processing system 20, and the recording / reproducing system 30 in response to an external operation. The control system 40 detects the pressing of the release button 106 to detect the solid-state imaging device 120. It controls driving, operation of the RGB image processing circuit 210, compression processing of the recording processing circuit 300, and the like. Specifically, an operation detection circuit 430 that detects the operation of the release button 106, a system control unit 400 that controls each unit in conjunction with the detection signal, generates and outputs a timing signal at the time of imaging, and the like. A solid-state image sensor drive circuit 420 that generates a drive signal for driving the solid-state image sensor 120 under the control of the system control unit 400.
[0098]
The processing in the RGB image processing circuit 210 is as follows. The RGB signal output for each of the R, G, and B regions via the A / D converter 500 is first subjected to predetermined white balance adjustment in the white balance circuit in the RGB image processing circuit 210, and further The gamma correction circuit performs a predetermined gamma correction. The interpolation calculation circuit in the RGB image processing circuit 210 generates an image signal having a resolution of 1200 × 1600 for each RGB by performing interpolation processing and distortion correction on the image signal of the solid-state imaging device 120, and the subsequent high-frequency luminance signal This is supplied to a generation circuit, a low luminance signal generation circuit, and a color difference signal generation circuit.
[0099]
This interpolation process further includes a first-stage interpolation process for correcting a relative movement of an object image due to expansion and contraction of the photographing lens due to a temperature change and a registration shift due to a manufacturing error of the photographing lens. This is composed of a second-stage interpolation process for forming a composite video signal in which the RGB image signals have the same resolution.
[0100]
Subsequent distortion correction is a calculation process for correcting distortion aberration of the photographing optical system by a known method. At this time, since the magnification and distortion of each RGB object image are the same depending on the setting of the photographing lens 100, uniform distortion correction may be performed on each object image. By correcting the distortion of the photographic optical system by calculation, the configuration of the photographic lens 100 can be optimized for correcting other optical aberrations.
[0101]
The details of the first stage interpolation process are as follows.
[0102]
Both the object image interval and the imaging region interval vary depending on the temperature change of the imaging system 10. Assuming that the linear expansion coefficient of the solid-state imaging device 120 is αS, the linear expansion coefficient of the taking lens 100 is αL, the temperature change amount is ΔT, and the distance between the lens portions is do, the imaging system 10 has a very small imaging magnification. The registration change amount ΔZT between the regions 820a and 820d can be expressed by Expression (1) as the difference between the extension of the photographing lens and the extension of the solid-state imaging device.
[0103]
ΔZT = do × (αL−αS) × ΔT (1)
Here, αS = 0.26 × 10^-Five, ΔT = 20 °, do = 1.9832 [mm] as the distance between the lens portions forming the two G object images, and αL = 1.2 × 10 when the photographing lens 100 is made of low-melting glass.^-FiveΔZT is calculated as 0.00037 [mm]. This is an interval change amount between the two G object images and also an interval change amount between the R object image and the B object image. The temperature change amount ΔT is obtained by the temperature sensor 165.
[0104]
FIG. 19 is a diagram showing this state, and shows a state in which the same elements as those in FIG. 13 have been changed in position due to thermal expansion. For the sake of simplicity, the drawing shows the imaging system 855a as a reference. The imaging system 855a corresponds to the lens unit 800a in the photographing lens 800 shown in FIG. 7, and the imaging system 855d corresponds to the lens unit 800d.
[0105]
Registration shift due to movement of the object image due to expansion or contraction of the photographing lens occurs in a direction connecting the optical axes of the two imaging systems. When the imaging system 855a and the imaging system 855d are considered, these registrations are registered. The deviation occurs in a direction parallel to the paper surface of FIG. The reason why this interpolation processing is performed before distortion correction is that when distortion correction is performed, the direction of registration deviation is not parallel to the paper surface, and interpolation cannot be realized with simple calculations.
[0106]
When a temperature difference of ΔT [° C.] occurs as compared with the time of manufacturing, the optical axis distance do of the imaging system 855a and the imaging system 855d undergoes a dimensional change of do × αL × ΔT. Since the imaging system 10 has an extremely small imaging magnification, the amount of movement of the object image on the imaging region 856d may be considered as do × αL × ΔT. On the other hand, the solid-state image sensor 120 also undergoes some thermal deformation. The fact that the imaging magnification is extremely small is also used here, and the amount of change can be expressed as do × αS × ΔT. Therefore, the registration change amount ΔZT is expressed as a difference between them, and becomes do × (αL−αS) × ΔT as described above.
[0107]
here,
KT = do × (αL−αS) (2)
Then, the registration change amount ΔZT is a product of a constant and a temperature difference.
ΔZT = KT × ΔT (3)
It is expressed. KT is a registration temperature coefficient of the G image. If one unit is expressed as one pixel pitch (pxl), the registration change amount is ΔZT / P using the pixel pitch P.
[0108]
Registration shift due to temperature change occurs between all imaging systems.
[0109]
Further, since the registration deviation due to the manufacturing error of the photographing lens can be made relatively small, only the component in the same direction as the registration deviation due to the expansion and contraction of the photographing lens is considered for the sake of simplicity.
[0110]
The registration shift due to the manufacturing error of the photographing lens is based on the positional relationship between the imaging region 820a and the object image formed in the image circle 851a.
Δr (pxl): manufacturing error in the positional relationship between the imaging region 820b and the object image formed in the image circle 851b
Δb (pxl): manufacturing error in the positional relationship between the imaging region 820c and the object image formed in the image circle 851c
Δg (pxl): manufacturing error in the positional relationship between the imaging region 820d and the object image formed in the image circle 851d
It is defined as One pixel pitch (pxl) is one unit.
[0111]
The actual interpolation processing in the first stage for correcting the registration deviation due to the temperature change and the manufacturing error described above is as follows. The image signals from the imaging areas 820a, 820b, 820c, and 820d are G1 (i, j), R (i, j), B (i, j), and G2 (i, j), respectively, and the addresses are shown in FIG. Determine as shown.
[0112]
With reference to the image signal G1 (i, j) of the imaging system 855a (lens unit 800a), the interpolated image signal G2T (i, j) of G2 (i, j), R (i, j), B (i, j) is used as a reference. j), RT (i, j), and BT (i, j) are generated based on the following equations (4) to (9). Expressions (4) to (9) are expressions for generating the pixel output at the virtual position by linear interpolation from the pixel outputs actually present on both sides. Depending on the sign of ΔZT / P + Δg, ΔZT / (P × √2) + Δr, ΔZT / (P × √2) + Δb, different arithmetic expressions are used.
[0113]
When the horizontal pitch a = P, the vertical pitch b = P, the constant c = 900, and the positive integer h = 1 on the light receiving surface in the image pickup region 820a and the image pickup region 820d, these are set in the horizontal direction within the light receiving surface. Since the positional relationship is a × h × c and b × c apart in the vertical direction, the pixels are always arranged in the direction in which the registration shift occurs, and the registration shift caused by the temperature change is extremely simple. It is possible to correct by calculation.
[0114]
The registration temperature coefficient of the R image and the registration temperature coefficient of the B image are 1 / √2 of the registration temperature coefficient KT between the G images from the ratio of the distance between the optical axes of the imaging system. That is, with respect to the distance from the lens unit 800a to the lens unit 800d in FIG. 7, the distance from the lens unit 800a to the lens unit 800b and the distance to the lens unit 800c are 1 / √2, and as a result, Registration deviation is also 1 / √2.
[0115]
Generation of G2T (i, j)
(1) When ΔZT / P + Δg ≦ 0

  (2) When ΔZT / P + Δg> 0

Generation of RT (i, j)
(3) When ΔZT / (P × √2) + Δr ≦ 0

  (4) When ΔZT / (P × √2) + Δr> 0

Generation of BT (i, j)
(5) When ΔZT / (P × √2) + Δb ≦ 0

  (6) When ΔZT / (P × √2) + Δb> 0

The interpolated image signals G2T (i, j), RT (i, j), and BT (i, j) obtained by the above processing are then used for the second stage correction processing.
[0116]
The second stage of interpolation processing is based on the image signal G1 (i, j) of 600 × 800 pixels and the interpolated image signals G2T (i, j), RT (i, j), BT (i, j), RGB Generates a G image signal G ′ (m, n), an R image signal R ′ (m, n), and a B image signal B ′ (m, n) each having a resolution of 1200 × 1600 pixels. The following formulas (10) to (21) are formulas representing operations for generating pixel outputs at positions where there is no data by averaging the outputs of adjacent pixels.
[0117]
Generation of G '(m, n)
(1) m: even n: odd
G ′ (m, n) = G2T (m / 2, (n + 1) / 2) (10)
(2) m: odd number n: even number
G ′ (m, n) = G1 ((m + 1) / 2, n / 2) (11)
(3) m: even n: even

  (4) m: odd number n: odd number

Generation of R '(m, n)
(5) m: Even n: When odd
R ′ (m, n) = (RT (m / 2, (n + 1) / 2) + RT (m / 2 + 1, (n + 1) / 2) / 2 (14)
(6) m: odd number n: even number
R ′ (m, n) = (RT ((m + 1) / 2, n / 2) + RT ((m + 1) / 2, n / 2 + 1) / 2 (15)
(7) m: even n: even

  (8) m: odd n: when odd
R ′ (m, n) = RT ((m + 1) / 2, (n + 1) / 2) (17)
Generation of B '(m, n)
(9) m: Even n: When odd
B '(m, n) = (BT (m / 2, (n-1) / 2) + BT (m / 2, (n-1) / 2 + 1)) / 2 (18)
(10) m: odd number n: even number
B ′ (m, n) = (BT ((m−1) / 2, n / 2) + BT ((m−1) / 2 + 1, n / 2)) / 2 (19)
(11) m: even n: even
B ′ (m, n) = BT (m / 2, n / 2) (20)
(12) m: odd n: odd

As described above, after performing position correction on at least one of the output images of the plurality of imaging regions in the first stage interpolation processing, the composition based on the output images of the plurality of imaging regions is performed in the second stage interpolation processing. Form a video signal.
[0118]
Subsequent luminance signal processing and color difference signal processing using G ′ (m, n), R ′ (m, n), and B ′ (m, n) are in accordance with processing in a normal digital color camera. .
[0119]
Next, the operation of the camera will be described. At the time of shooting, a contact protection cap is attached and used to protect the connection terminal 114 of the camera body 101. When the contact protection cap 200 is attached to the camera body 101, it functions as a camera grip and plays a role of making it easier to hold the camera.
[0120]
First, when the main switch 105 is turned on, the power supply voltage is supplied to each part and the operation is enabled. Subsequently, it is determined whether or not an image signal can be recorded in the memory. At this time, the number of recordable images is displayed on the display unit 150 according to the remaining capacity. If the operator who has seen the display can take a picture, he points the camera at the object scene and presses the release button 106.
[0121]
When the release button 106 is pressed halfway, the first stage circuit of the switch 121 is closed and the exposure time is calculated. When all the shooting preparation processes are completed, shooting is possible and the display is reported to the photographer. Thus, when the release button 106 is pressed down to the end, the second stage circuit of the switch 121 is closed, and an operation detection circuit (not shown) sends its detection signal to the system control circuit. At that time, the elapsed time calculated in advance is counted, and when a predetermined exposure time elapses, a timing signal is supplied to the solid-state image sensor driving circuit. As a result, the solid-state imaging device driving circuit generates horizontal and vertical driving signals, and sequentially reads out 800 × 600 pixels exposed in all imaging regions in the horizontal and vertical directions.
[0122]
At this time, the photographer presses the release button 106 so as to hold the contact protection cap 200 and sandwich the camera body 101 with the index finger and thumb of the right hand. Since the projection 106a is provided integrally with the release button 106 on the center line L2 of the axis of the release button 106, and the projection 120 is provided on the back cover 125 at a position extending the center line L2, the photographer Relies on the two protrusions 106a and 120 to perform the release operation so that the protrusion 106a is pushed with the index finger and the protrusion 120 is pushed with the thumb. By doing so, the generation of the couple 129 shown in FIG. 3 can be easily prevented, and a high-quality image without blur can be taken.
[0123]
Each read pixel is converted into a digital signal having a predetermined bit value by the A / D converter 500 and sequentially supplied to the RGB image processing circuit 210 of the image processing system 20. The RGB image processing circuit 210 performs pixel interpolation processing in a state where white balance and gamma correction are performed, respectively, and supplies the result to the YC processing circuit 230.
[0124]
In the YC processing circuit 230, the high-frequency luminance signal generation circuit generates a high-frequency luminance signal YH for each pixel of RGB, and similarly, the low-frequency luminance signal generation circuit calculates the low-frequency luminance signal YL. . The calculated high frequency luminance signal YH is output to the adder via a low-pass filter. Similarly, the low-frequency luminance signal YL is subtracted from the high-frequency luminance signal YH and is output to the adder through the low-pass filter. Thereby, the difference signal YL-YH between the high frequency luminance signal YH and the low frequency luminance signal is added to obtain the luminance signal Y. Similarly, the color difference signal generation circuit obtains and outputs the color difference signals RY and BY. The output color difference signals RY and BY are supplied to the recording processing circuit 300 after passing through a low-pass filter.
[0125]
Next, the recording processing circuit 300 that has received the YC signal compresses each luminance signal Y and the color difference signals RY and BY by a predetermined still image compression method, and sequentially records them in the memory.
[0126]
When each image is reproduced from an image signal representing a still image or a moving image recorded in the memory, when the reproduction button 9 is pressed, the operation is detected by the operation detection circuit 430 and detected by the system control unit 400. Supply signal. As a result, the recording processing circuit 300 is driven. The driven recording processing circuit 300 reads the recorded contents from the memory and displays an image on the liquid crystal monitor 4. The operator selects a desired image by pressing a selection button or the like.
[0127]
(Second Embodiment)
In the first embodiment, the registration shift due to the temperature change of the imaging system 10 (890) is corrected based on the output of the temperature sensor 165. However, the registration shift also occurs due to a change in the distance of the subject. The second embodiment corrects the registration deviation caused by the change in the subject distance.
[0128]
FIG. 21 is a diagram showing a signal processing system for using the output of the distance measuring device for correction of registration deviation. The same elements as those in the first embodiment are denoted by the same reference numerals.
[0129]
As shown in FIG. 21, the imaging apparatus includes an imaging system 10, an image processing system 20 that is an image processing unit, a recording / reproducing system 30, and a control system 40. Furthermore, the imaging system 10 includes a photographing lens 100, a diaphragm 110, and a solid-state imaging device 120. The image processing system 20 includes an A / D converter 500, an RGB image processing circuit 410, and a YC processing circuit 230, and a recording / reproducing system. 30 includes a recording processing circuit 300 and a reproduction processing circuit 310, and the control system 40 includes a system control unit 400, an operation detection unit 430, a distance measuring device 465, and a solid-state image sensor driving circuit 420.
[0130]
The distance measuring device 465 is an active distance measuring device that outputs a distance by projecting ultrasonic waves or LED light onto a subject, or a passive distance measuring device that outputs a distance based on the principle of triangulation using an image of the subject. Is appropriate.
[0131]
The imaging system 10 is an optical processing system that forms an image of light from an object on the imaging surface of the solid-state imaging device 120 through the diaphragm 110 and the photographing lens 100, and exposes the subject image to the solid-state imaging device 120. As described above, an imaging device such as a CCD or a CMOS sensor is effectively applied to the solid-state imaging device 120, and an image signal representing a continuous moving image by controlling the exposure time and the exposure interval of the solid-state imaging device 120, Alternatively, an image signal representing a still image by one exposure can be obtained.
[0132]
As in the first embodiment, the solid-state imaging device 120 has an image pickup device that has 800 pixels in the long side direction and 600 pixels in the short side direction for each image pickup region, and has a total of 19.2 million pixels. As applied, optical filters of the three primary colors of red (R), green (G), and blue (B) are arranged for each predetermined area on the front surface. The pixel pitch is 1.56 μm both vertically and horizontally.
[0133]
As shown in FIG. 21, the image signals read from the solid-state imaging device 120 are supplied to the image processing system 20 via the A / D converter 500, respectively. The A / D converter 500 is, for example, a signal conversion circuit that converts and outputs, for example, a 10-bit digital signal corresponding to the amplitude of the signal of each exposed pixel, and the subsequent image signal processing is performed by digital processing. Executed.
[0134]
The image processing system 20 is a signal processing circuit that obtains an image signal of a desired format from R, G, and B digital signals. The R, G, and B color signals are converted into a luminance signal Y and a color difference signal (RY), It is converted into a YC signal represented by (BY).
[0135]
The RGB image processing circuit 410 is a signal processing circuit that processes an image signal of 800 × 600 × 4 pixels received from the solid-state imaging device 120 via the A / D converter 500, and includes a white balance circuit, a gamma correction circuit, and an interpolation. It has an interpolation calculation circuit for increasing the resolution by calculation.
[0136]
The YC processing circuit 230 is a signal processing circuit that generates the luminance signal Y and the color difference signals RY and BY. A high-frequency luminance signal generation circuit that generates a high-frequency luminance signal YL, a low-frequency luminance signal generation circuit that generates a low-frequency luminance signal YL, and a color difference signal generation circuit that generates color difference signals RY and BY Has been. The luminance signal Y is formed by combining the high frequency luminance signal YH and the low frequency luminance signal YL.
[0137]
The recording / reproducing system 30 is a processing system that outputs an image signal to the memory and outputs an image signal to the liquid crystal monitor 4, and a recording processing circuit 300 that performs writing and reading processing of the image signal to and from the memory; And a reproduction processing circuit 310 that reproduces an image signal read from the memory and outputs it as a monitor output. More specifically, the recording processing circuit 300 includes a compression / expansion circuit that compresses YC signals representing still images and moving images in a predetermined compression format and expands the compressed data when it is read out.
[0138]
The compression / decompression circuit has a frame memory or the like for signal processing. The YC signal from the image processing system 20 is stored in this frame memory for each frame, and is read and compressed and encoded for each of a plurality of blocks. The compression coding is performed, for example, by performing two-dimensional orthogonal transformation, normalization, and Huffman coding on the image signal for each block.
[0139]
The reproduction processing circuit 310 is a circuit that converts the luminance signal Y and the color difference signals RY and BY into, for example, RGB signals by matrix conversion. The signal converted by the reproduction processing circuit 310 is output to the liquid crystal monitor 4 and a visible image is displayed and reproduced.
[0140]
The control system 40 includes control circuits for respective units that control the imaging system 10, the image processing system 20, and the recording / reproducing system 30 in conjunction with external operations. The control system 40 detects pressing of the release button 6, and It controls driving, operation of the RGB image processing circuit 410, compression processing of the recording processing circuit 300, and the like. Specifically, an operation detection circuit 430 that detects the operation of the release button 6, a system control unit 400 that controls each unit in conjunction with the detection signal, generates and outputs a timing signal at the time of imaging, and the like. A solid-state image sensor drive circuit 420 that generates a drive signal for driving the solid-state image sensor 120 under the control of the system control unit 400.
[0141]
The processing in the RGB image processing circuit 410 is as follows. The RGB signal output for each of the R, G, and B regions via the A / D converter 500 is first subjected to predetermined white balance adjustment by the white balance circuit in the RGB image processing circuit 410, and further, The gamma correction circuit performs a predetermined gamma correction. The interpolation calculation circuit in the RGB image processing circuit 410 generates an image signal with a resolution of 1200 × 1600 pixels for each RGB by performing interpolation processing and distortion correction on the image signal of the solid-state imaging device 120, and the subsequent high-frequency brightness The signal is supplied to a signal generation circuit, a low luminance signal generation circuit, and a color difference signal generation circuit.
[0142]
This interpolation process further includes a first-stage interpolation process for correcting registration deviation due to the distance of the subject, and a second-stage interpolation process for forming a composite video signal having the same resolution for each of the RGB image signals. Consists of.
[0143]
Subsequent distortion correction is a calculation process for correcting distortion aberration of the photographing optical system by a known method. At this time, since the magnification and distortion of each RGB object image are the same depending on the setting of the photographing lens 100, uniform distortion correction may be performed on each object image. By correcting the distortion of the photographic optical system by calculation, the configuration of the photographic lens 100 can be optimized for correcting other optical aberrations.
[0144]
The details of the first stage interpolation process are as follows. For simplicity, in the following description, the subject distance is considered with reference to the imaging system. The registration change amount ΔZs between the imaging regions 820a and 820d can be expressed by Expression (22) as a function of the difference ΔS between the actual subject distance and the reference subject distance St. The difference ΔS is obtained from the output of the distance measuring device 465.
[0145]
ΔZ S = do · S ′ · ΔS / {St · (ΔS + St)} (22)
Assuming that the configuration of the imaging system is the same as that of the first embodiment, the lens unit interval do = 1.9832 [mm], the reference subject distance St = 2380-1.45 = 2378.55 [mm], and the imaging system And the imaging region is S ′ = 1.45 [mm].
[0146]
FIG. 22 is a diagram showing an optical path when the actual subject distance becomes infinitely large. Each imaging area is 1.248 mm × 0.936 mm, and these are located with a separation band of 0.156 mm in the horizontal direction and 0.468 mm in the vertical direction. The center interval between adjacent imaging regions is 1.404 mm in the vertical and horizontal directions, and 1.9856 mm in the diagonal direction.
[0147]
Paying attention to the imaging areas 820a and 820d, the image of the object at the reference subject distance is 1.9845 mm intervals obtained by subtracting the diagonal dimension of 0.5 pixels from the imaging area interval 1.9856 mm for pixel shifting. Suppose that it forms on an image pick-up part. In this case, as described above, the distance between the lens portions 800a and 800d of the photographing lens 800 is set to 1.9832 mm. In the figure, arrows 855a and 855d are symbols representing an imaging system having positive power by the lens portions 800a and 800d of the photographing lens 800, rectangles 856a and 856d are symbols representing the ranges of the imaging regions 820a and 820d, and L801 and L802 are symbols. This is the optical axis of the imaging systems 855a and 855d.
[0148]
The difference ΔS between the actual subject distance and the reference subject distance St is infinite. The limit value of equation (22) when ΔS → ∞ is
[0149]
[Expression 1]

It can be seen that a registration deviation of 0.0012 [mm] occurs. If one unit is expressed as one pixel pitch (pxl), using the pixel pitch P, the registration change amount is ΔZS / P.
[0150]
The actual interpolation processing in the first stage for correcting the registration deviation due to the subject distance as described above is as follows. Similar to the first embodiment, the image signals from the imaging regions 820a, 820b, 820c, and 820d are respectively G1 (i, j), R (i, j), B (i, j), and G2 (i, j ) And addresses are determined as shown in FIG.
[0151]
Using the image signal G1 (i, j) of the imaging system 855a (lens unit 800a) as a reference, the interpolated image signal G2S (i, j) of G2 (i, j), R (i, j), B (i, j) is used. j), RS (i, j), and BS (i, j) are generated based on the following equation (24) to equation (29). Expressions (24) to (29) are expressions for generating the pixel output at the virtual position by linear interpolation from the pixel outputs actually existing on both sides. When the pixel pitch is P and the horizontal pitch a = P, the vertical pitch b = P, the constant c = 900, and the positive integer h = 1 on the light receiving surface in the imaging region 820a and the imaging region 820d, these are received. Since the positional relationship is a × h × c in the horizontal direction and b × c in the vertical direction in the plane, it is possible to correct a registration shift caused by a temperature change or a subject distance change by a very simple calculation. Is possible.
[0152]
Note that the registration deviation of the R image and the registration deviation of the B image are 1 / √2 of the registration deviation between the G images from the ratio of the distance between the optical axes of the imaging system.
[0153]
Generation of G2S (i, j)
(1) When ΔZS ≧ 0

  (2) When ΔZS <0

Generation of RS (i, j)
(3) When ΔZS ≧ 0

  (4) When ΔZS <0

Generation of BS (i, j)
(5) When ΔZS ≧ 0

  (6) When ΔZS <0

The interpolated image signals G2S (i, j), RS (i, j), and BS (i, j) obtained by the above processing eliminate the registration deviation caused by the subject distance, and then the second stage. Used for correction processing.
[0154]
(Third embodiment)
In the second embodiment, the registration deviation due to the subject distance is corrected based on the output of the distance measuring device 465. In the third embodiment, this registration shift is corrected without using the output of the distance measuring device 465.
[0155]
The switch 109 shown in FIG. 2 for setting the camera state is used as a macro shooting mode setting switch. When the next shooting is performed on a subject at a short distance, the photographer presses the macro shooting mode setting switch 109. When the operation detection circuit 430 detects that the macro shooting mode setting switch 109 is pressed, for example, an interpolation process is performed in which the difference ΔS from the reference subject distance St is set to −1.19 [m]. That is, ΔZs is calculated by setting ΔS in equation (22) to ΔS = −1.19, and interpolation processing is performed based on equations (24) to (29).
[0156]
If constituted in this way, since a distance measuring device is not required, the cost can be kept low.
In addition to this, it is advantageous for reducing the size and thickness of the camera.
[0157]
(Fourth embodiment)
In the first embodiment, the registration deviation caused by the temperature change is corrected based on the output of the temperature sensor 165, and in the second embodiment, the registration deviation caused by the subject distance is corrected based on the output of the distance measuring device 465. . In the fourth embodiment, registration deviation is corrected based on the output of the solid-state imaging device itself without using the output of the temperature sensor 165 or the distance measuring device 465. At this time, an image shift caused by the parallax is detected using the correlation in the oblique direction between the outputs of the two G imaging regions. With this configuration, it is possible to collectively correct a registration error caused by a temperature change, a registration error caused by a subject distance, and a registration error caused by a manufacturing error of the photographing lens.
[0158]
FIG. 23 is a diagram for explaining pixels on a solid-state imaging device used for detecting registration deviation. For simplicity, the positional relationship between the object image and the imaging region is shown by setting the number of vertical and horizontal pixels to 1/100.
[0159]
In FIG. 23, 320a, 320b, 320c, and 320d are four imaging regions of the solid-state imaging device 820. For the sake of explanation, each of the imaging regions 320a, 320b, 320c, and 320d is formed by arranging 8 × 6 pixels. The imaging area 320a and the imaging area 320d output a G image signal, the imaging area 320b outputs an R image signal, and the imaging area 320c outputs a B image signal. Each pixel in the imaging areas 320a and 320d is a white rectangle, each pixel in the imaging area 320b is a hatched rectangle, and each pixel in the imaging area 320c is a black rectangle.
[0160]
In addition, a separation band having a size corresponding to one pixel in the horizontal direction and three pixels in the vertical direction is formed between the imaging regions. Therefore, the distance between the centers of the respective imaging regions is the same in the horizontal direction and the vertical direction.
[0161]
Reference numerals 351a, 351b, 351c, and 351d are object images. For pixel shifting, the centers 360a, 360b, 360c, and 360d of the object images 351a, 351b, 351c, and 351d are each ¼ from the centers of the imaging regions 320a, 320b, 320c, and 320d toward the center 320e of the entire imaging region. The pixel is offset.
[0162]
As a result, when each imaging region is back-projected onto a plane at a predetermined distance on the object side, the back-projected images at the centers 360a, 360b, 360c, and 360d of the object image overlap, and the imaging regions 320a, 320b, The backprojected images of the pixels 320c and 320d are arranged in a mosaic so that the centers do not overlap.
[0163]
L301 and L302 are line segments representing pixel columns used for detecting registration deviation. Since both the imaging area 320a and the imaging area 320d output a G image, a signal of a pixel column located in the imaging area 320a and below the line segment L301 and a signal in the imaging area 320d and below the line segment L301 are displayed. The signal of the pixel column to be processed has a similar form, and by performing a correlation operation on these signals, a registration shift between the imaging region 320a and the imaging region 320d can be known. Similarly, by performing a correlation operation on the signal of the pixel column located below the line segment L302 in the imaging region 320a and the signal of the pixel column located below the line segment L302 in the imaging region 320d. Thus, it is possible to know a registration shift between the imaging region 320a and the imaging region 320d.
[0164]
Here, registration accuracy is calculated by using two pairs of image signals, and the detection accuracy is increased by averaging the results.
[0165]
FIG. 24 is a diagram illustrating a signal of a pixel column located in the imaging region 320a and below the line segment L301 and a signal of a pixel column located in the imaging region 320d and below the line segment L301. In the drawing, a signal 370 indicated by a black circle is a signal of a pixel column located in the imaging region 320a and under the line segment L301, and a signal 371 indicated by a white circle is in the imaging region 320d and below the line segment L301. Specifically, 361 ′ is an output of the pixel 361 in FIG. 23, 362 ′ is an output of the pixel 362, 363 ′ is an output of the pixel 363, and 364 ′ is an output of the pixel 364. These are all part of the G image signal. That is, since the signal 370 and the signal 371 have the same spectral distribution in color characteristics, the result of the correlation calculation represents the shift of the image signal with extremely high accuracy.
[0166]
FIG. 25 is a diagram showing a signal of a pixel column located under the line segment L302 in the imaging region 320a and a signal of a pixel column located under the line segment L302 in the imaging region 320d. In the figure, a signal 372 indicated by a black circle is a signal of a pixel column located in the imaging region 320a and below the line segment L302, and a signal 373 indicated by a white circle is located in the imaging region 320d and below the line segment L302. Specifically, reference numeral 365 ′ denotes an output of the pixel 365 in FIG. 23, 366 ′ denotes an output of the pixel 366, 367 ′ denotes an output of the pixel 367, and 368 ′ denotes an output of the pixel 368. These are all part of the G image signal. That is, the signal 372 and the signal 373 have the same spectral distribution in color characteristics.
[0167]
Both FIG. 24 and FIG. 25 show a state in which there is no temperature change and the subject is located at the reference subject distance, and the signal is relatively shifted by 0.5 pixels due to pixel shifting. Yes.
[0168]
When the temperature changes with respect to this state or the subject moves away from the reference subject distance, a signal shift occurs. For example, in FIGS. 24 and 25, the signal 370 moves in the direction of arrow A and the signal 371 moves in the direction of arrow B when the subject is far away, and moves in the opposite direction when the subject is near. This behavior is expressed by Expression (22), which is a function of the difference ΔS between the actual subject distance zone and the reference subject distance St.
[0169]
The relative positional change amount between a pair of signals having the same spectral distribution in color characteristics is detected using a known method using a correlation operation, for example, a method disclosed in Japanese Patent Publication No. 5-88445. can do. When the shift amount 0.5 of the signal generated in design from the output is subtracted, the registration change amount ΔZ S / P is expressed by expressing one unit as one pixel pitch (pxl).
[0170]
Normally, due to the characteristics of the optical system, the solid-state image sensor 820 projects a high frequency component that exceeds the Nyquist frequency. Therefore, depending on the object pattern, the phase of the object image may not be completely reflected in the phase of the signal. In order to reduce the detection error of the registration change amount ΔZS / P caused by this, the registration change amount obtained from the signal of the pixel column located below the line segment L301 and the sampling position of the object image are 0. The registration change amount obtained from the signal of the pixel column located under the line segment L302 shifted by 5 pixels is averaged. The obtained registration change amount has high accuracy, and by using this, the first-stage interpolation processing according to the equations (24) to (29) can be ideally performed.
[0171]
The interpolated image signals G2S (i, j), RS (i, j), and BS (i, j) obtained by the above processing are then used for the second stage correction processing.
[0172]
Unlike the second embodiment, in this case, the interpolated image signals G2S (i, j), RS (i, j), and BS (i, j) are caused by registration deviation and subject distance due to temperature change. The registration deviation due to the manufacturing lens manufacturing error and the registration deviation due to the manufacturing lens are corrected together.
[0173]
In the subsequent second-stage interpolation processing, each of the image signals G1 (i, j) of 600 × 800 and the interpolated image signals G2T (i, j) and RT (i, j) are similar to those shown in the first embodiment. , BT (i, j), G image signal G ′ (m, n), R image signal R ′ (m, n), and B image signal B ′ (m, n) each having a resolution of 1200 × 1600 RGB. ). G2T (i, j), RT (i, j), and BT (i, j) in the expressions (10) to (21) in the first embodiment are respectively changed to G2S (i, j) and RS (i , j) and BS (i, j).
[0174]
As described above, the imaging device outputs two G images having substantially the same field of view formed with the same spectral distribution, and R and B images having substantially the same field of view as the two G images, and the first-stage interpolation. After correcting the positions of the R image and the B image based on the change in the interval between the two G images in the process, a composite video signal based on the RGB image is formed by the second stage interpolation process.
[0175]
Further, if an area-based matching method is used as disclosed in Japanese Patent Laid-Open No. 10-289316, registration deviation can be obtained for each pixel. By using this to correct the registration shift, it is possible to obtain an image with high sharpness and no color shift over the entire screen even when the subject has a depth. In this case, ΔZ S in the equations (24) to (29) may be handled as a function of the pixel address.
[0176]
FIG. 26 is a diagram for explaining another setting of pixels on the solid-state imaging device used for detecting registration deviation.
[0177]
In FIG. 26, 520a, 520b, 520c, and 520d are four imaging regions of the solid-state imaging device 820. Here, for explanation, each of the imaging regions 520a, 520b, 520c, and 520d is formed by arranging 8 × 4 pixels. However, in practice, the number of pixels is increased to obtain a practical resolution. The imaging areas 520a and 520d output G image signals, the imaging area 520b outputs R image signals, and the imaging area 520c outputs B image signals. The pixels in the imaging areas 520a and 520d are white rectangles, the pixels in the imaging area 520b are hatched rectangles, and the pixels in the imaging area 520c are black rectangles.
[0178]
A separation band having a size corresponding to 1 pixel in the horizontal direction and 0 pixel in the vertical direction is formed between the imaging regions. Therefore, the center distance between the two imaging regions that output the G image is 9 pixels in the horizontal direction and 4.5 pixels in the vertical direction. That is, when the horizontal pitch a = P, the vertical pitch b = P, the constant c = 4.5, and the positive integer b = 2 on the light receiving surface in the image pickup region 820a and the image pickup region 820d, these are within the light receiving surface. In the positional relationship, a × h × c in the horizontal direction and b × c in the vertical direction. By creating such a relationship, a registration shift caused by a change in temperature or a change in subject distance occurs in the direction in which the pixels are arranged. This can be corrected by a very simple calculation.
[0179]
Reference numerals 551a, 551b, 551c, and 551d are object images. For pixel shifting, the centers 560a, 560b, 560c, and 560d of the object images 551a, 551b, 551c, and 551d are both vertically and horizontally in the direction approaching the center 520e of the entire imaging region from the centers of the imaging regions 520a, 520b, 520c, and 520d, respectively. It is offset by 1/4 pixel.
[0180]
As a result, when each imaging region is back-projected onto a plane at a predetermined distance on the object side, the back-projected images at the centers 560a, 560b, 560c, and 560d of the object image overlap, and the imaging regions 520a, 520b, The backprojected images of the pixels 520c and 520d are arranged in a mosaic so that the centers do not overlap.
[0181]
L501 and L502 are line segments representing pixel columns used to detect registration deviation. The pixels used for easy understanding are shown with hatching. Since both the imaging area 520a and the imaging area 520d output a G image, the signal of the pixel column located in the imaging area 520a and below the line segment L501 and the signal in the imaging area 520d and below the line segment L501. The signal of the pixel column to be processed has a similar shape, and by taking these correlations, it is possible to know the registration deviation between the imaging region 520a and the imaging region 520d. Similarly, by correlating the signal of the pixel column located in the imaging region 520a and below the line segment L502 with the signal of the pixel column located in the imaging region 520d and below the line segment L502, A registration shift between the imaging area 520a and the imaging area 520d can be known.
[0182]
Here, registration accuracy is calculated by using two pairs of image signals, and the detection accuracy is increased by averaging the results.
[0183]
When the temperature changes or the subject moves away from the reference subject distance, a signal shift occurs. This behavior is expressed by Expression (22), which is a function of the difference ΔS between the actual subject distance and the reference subject distance St.
[0184]
The relative positional change amount between a pair of signals having the same spectral distribution in color characteristics is detected using a known method using a correlation operation, for example, a method disclosed in Japanese Patent Publication No. 5-88445. can do. When the shift amount 0.5 of the signal generated in design from the output is subtracted, the registration change amount ΔZ S / P is expressed by expressing one unit as one pixel pitch (pxl). Here, it should be noted that every other pixel used is used. By using the obtained registration change amount, the first-stage interpolation processing according to the equations (24) to (29) becomes possible.
[0185]
Further, when the aspect ratio of the imaging area is reversed vertically and horizontally, a separation band having a size corresponding to 1 pixel in the vertical direction and 0 pixel in the horizontal direction is formed between the imaging areas. The distance of the center of the imaging area to be output is 9 pixels in the vertical direction and 4.5 pixels in the horizontal direction. Accordingly, with respect to these two imaging regions, when the horizontal pitch a = P, the vertical pitch b = P, the constant c = 4.5, and the positive integer h = 2 on the light receiving surface, The positional relationship is a × c and b × h × c apart in the vertical direction.
[0186]
FIG. 27 is a diagram for explaining still another setting of pixels on the solid-state imaging device used for detecting registration deviation. In this example, the pixel pitch is different between the horizontal direction and the vertical direction.
[0187]
In FIG. 27, reference numerals 620a, 620b, 620c, and 620d denote four imaging regions of the solid-state imaging device 820. Here, for the sake of explanation, each of the imaging regions 620a, 620b, 620c, and 620d is formed by arranging 8 × 6 pixels. The imaging areas 620a and 620d output G image signals, the imaging area 620b outputs R image signals, and the imaging area 620c outputs B image signals. The pixels in the imaging areas 620a and 620d are white rectangles, the pixels in the imaging area 620b are hatched rectangles, and the pixels in the imaging area 620c are black rectangles.
[0188]
In addition, a separation band having a size corresponding to one pixel in the horizontal direction and three pixels in the vertical direction is formed between the imaging regions. Therefore, the distance between the centers of the respective imaging regions is the same in the horizontal direction and the vertical direction.
[0189]
Reference numerals 651a, 651b, 651c, and 651d are object images. For pixel shifting, the centers 660a, 660b, 660c, and 660d of the object images 651a, 651b, 651c, and 651d are 1/4 from the centers of the imaging regions 620a, 620b, 620c, and 620d to the center 620e of the entire imaging region. The pixel is offset.
[0190]
As a result, when each imaging region is back-projected onto a plane at a predetermined distance on the object side, the back-projected images of the object image centers 660a, 660b, 660c, 660d are overlapped, and the imaging regions 620a, 620b, The backprojected images of the pixels 620c and 620d are arranged in a mosaic so that the centers do not overlap.
[0191]
L601 and L602 are line segments representing pixel columns used for detecting registration deviation. Since both the imaging area 620a and the imaging area 620d output a G image, the pixel column signal located in the imaging area 620a and below the line segment L601 and the signal in the imaging area 620d and below the line segment 601 are located. By taking a correlation with the signal of the pixel column to be registered, it is possible to know a registration shift between the imaging region 620a and the imaging region 620d. Similarly, by correlating the signal of the pixel column located in the imaging region 620a and below the line segment L602 with the signal of the pixel column located in the imaging region 620d and below the line segment L602, A registration shift between the imaging area 620a and the imaging area 620d can be known.
[0192]
Here, registration accuracy is calculated by using two pairs of image signals, and the detection accuracy is increased by averaging the results.
[0193]
When the temperature changes or the subject moves away from the reference subject distance, a signal shift occurs. This behavior is expressed by Expression (22), which is a function of the difference ΔS between the actual subject distance and the reference subject distance St.
[0194]
The relative positional change amount between a pair of signals having the same spectral distribution in color characteristics is detected using a known method using a correlation operation, for example, a method disclosed in Japanese Patent Publication No. 5-88445. can do. When the shift amount 0.5 of the signal generated in design from the output is subtracted, the registration change amount ΔZ S / P is expressed by expressing one unit as one pixel pitch (pxl). By using the obtained registration change amount, the first-stage interpolation processing according to the equations (24) to (29) becomes possible.
[0195]
[Other Embodiments]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.
[0196]
Another object of the present invention is to supply a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0197]
Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.
[0198]
As described above, a plurality of apertures for capturing external light from different positions and external light captured from the plurality of apertures are separately received, and a predetermined color component is obtained for each external light received separately. In the imaging apparatus having a plurality of imaging means to be extracted and an image processing means for processing an output signal of the imaging means, the image processing means corrects the position of at least one of the output images of the plurality of imaging means. After performing the above, the following effects were obtained by forming a composite video signal based on the output images of the plurality of imaging means.
[0199]
(1) An image position correction method suitable for an image pickup apparatus for a small digital color camera could be obtained.
[0200]
(2) As a result, it has become possible to easily obtain a high-definition image with no color misregistration or sharpness reduction.
[0201]
Furthermore, an imaging device having a plurality of imaging regions, a photographing optical system for forming an object image on the plurality of imaging regions by a plurality of imaging systems corresponding to the plurality of imaging regions, and an output signal of the imaging device In the imaging apparatus having the image processing means for processing and the temperature measurement means, the imaging device outputs a plurality of images having substantially the same field of view, and the image processing means outputs at least one of the plurality of images. After performing position correction according to the output of the temperature measuring means for one, the following effect was obtained by forming a composite video signal based on the plurality of images.
[0202]
(3) A method suitable for correcting a registration shift caused by a temperature change could be obtained.
[0203]
Furthermore, an imaging device having a plurality of imaging regions, a photographing optical system for forming an object image on the plurality of imaging regions by a plurality of imaging systems corresponding to the plurality of imaging regions, and an output signal of the imaging device An image processing unit that processes the image, the image sensor outputs a plurality of images having substantially the same field of view, and the image processing unit outputs at least one of the plurality of images based on subject distance information. The following effects were obtained by forming a composite video signal based on the plurality of images after performing position correction for one.
[0204]
(4) A method suitable for correcting a registration shift caused by a change in subject distance could be obtained.
[0205]
Further, when the macro shooting mode is set, after performing position correction on at least one of the plurality of images, the following effect is obtained by forming a composite video signal based on the plurality of images. It was.
[0206]
(5) A method suitable for correcting a registration shift caused by a change in the subject distance when the macro shooting mode is set can be obtained.
[0207]
(6) The distance measuring device can be omitted.
[0208]
In addition, the image sensor includes a plurality of imaging regions on the same plane, a photographing optical system that forms object images on the plurality of imaging regions, and an image processing unit that processes an output signal of the imaging device. In the imaging apparatus described above, the imaging element is formed with the first and second images having substantially the same visual field formed with the same spectral distribution, and the first and second images having different spectral distributions from the first and second images. A third image having substantially the same field of view as the second image, and the image processing means determines the position of the third image based on a change in the interval between the first and second images in the process of the output signal. After performing the above correction, the following effects were obtained by forming a composite video signal based on the first, second, and third images.
[0209]
(7) A method suitable for correcting the registration deviation based on the output itself of the solid-state imaging device could be obtained. Therefore, according to this configuration, no temperature sensor or distance measuring device is required.
[0210]
(8) Secondary, the camera can be reduced in size and cost.
[0211]
(9) In addition, if registration deviation for each pixel is detected by area-based matching or the like, and correction of registration deviation differs for each pixel or for each small area, a subject with a depth in which various distances are mixed Even so, a high-definition image can be obtained over the entire screen.
[0212]
Furthermore, an imaging device having first and second imaging areas having substantially the same dimensions on the same plane, a first object image on the first imaging area, and a second on the second imaging area. In the imaging apparatus having the imaging optical system for forming the object image and the image processing means, the first and second imaging areas have a plurality of pitches on the light receiving surface at a pitch of a in the horizontal direction and b in the vertical direction. When the pixels are aligned and h is a positive integer, the first and second imaging regions in the light receiving surface are a × h × c in the horizontal direction, b × c in the vertical direction, or horizontal The image sensor forms a first image and a second image having substantially the same field of view formed with the same spectral distribution, with a positional relationship of a × c in the direction and b × h × c in the vertical direction. The image processing means generates a composite video signal based on the first and second images. The following effect was obtained.
[0213]
(10) It was possible to obtain the arrangement of the imaging areas so that the registration shift caused by the temperature change or the subject distance change can be corrected by an extremely simple calculation.
[0214]
Furthermore, the following effects were obtained by correcting the change in the interval between the first and second images in the process of processing the output signal and forming a composite video signal based on the first and second images. .
[0215]
(11) An extremely high-definition video signal can be formed by using the arrangement of the imaging area so that the registration shift can be corrected by an extremely simple calculation.
【The invention's effect】
As described above, according to the above-described embodiment, it is possible to satisfactorily correct and synthesize RGB image shifts.
[Brief description of the drawings]
FIG. 1 is a cross-sectional view of a digital color camera.
FIG. 2 is a rear view of the digital color camera.
FIG. 3 is a side view of FIG. 2 viewed from the left.
4 is a side view of FIG. 2 viewed from the right side.
FIG. 5 is a detailed view of an imaging system.
FIG. 6 is a plan view of a diaphragm.
FIG. 7 is a view of a photographic lens as viewed from the light exit side.
FIG. 8 is a front view of a solid-state image sensor.
FIG. 9 is a diagram of a photographic lens viewed from the light incident side.
FIG. 10 is a diagram illustrating spectral transmittance characteristics of an optical filter.
FIG. 11 is a diagram for explaining the action of a microlens.
FIG. 12 is a cross-sectional view of a lens unit.
FIG. 13 is a diagram for explaining the interval setting of the lens portion of the photographic lens.
FIG. 14 is a diagram for explaining a positional relationship between an object image and an imaging region.
FIG. 15 is a diagram for explaining a positional relationship of pixels when an imaging region is projected onto a subject.
FIG. 16 is a perspective view of a first prism and a second prism constituting the finder.
FIG. 17 is a cross-sectional view of a finder system.
FIG. 18 is a block diagram of a signal processing system.
FIG. 19 is a diagram for explaining a state in which the position of an imaging system element has changed due to thermal expansion.
FIG. 20 is a diagram for explaining an address of an image signal from an imaging region.
FIG. 21 is a diagram illustrating a signal processing system for using the output of the distance measuring device to correct registration deviation.
FIG. 22 is a diagram illustrating an optical path when an actual subject distance becomes infinitely large.
FIG. 23 is a diagram for explaining a pixel on a solid-state image sensor used for detecting a registration shift.
FIG. 24 is a diagram illustrating signals of pixel columns in two imaging regions.
FIG. 25 is a diagram illustrating signals of pixel columns in two imaging regions.
FIG. 26 is a diagram for explaining a pixel on a solid-state image sensor used for detecting a registration shift.
FIG. 27 is a diagram for explaining a pixel on a solid-state imaging device used for detecting a registration shift.
[Explanation of symbols]
101 Camera body
800 shooting lens
800a, 800b, 800c, 800d Lens part of photographing lens
820 Solid-state image sensor
210 RGB image processing circuit

Claims (7)

An imaging device having a plurality of imaging areas on the same plane;
A photographing optical system for forming object images on the plurality of imaging regions,
Image processing means for processing the output signal of the image sensor,
The imaging device has first and second images having substantially the same field of view formed with the same spectral distribution, and the first and second images formed with a spectral distribution different from the first and second images. outputs the third image of substantially the same field of view, the image processing means, in the process of the output signal, a temperature change, a change in object distance, the second based on at least one of the manufacturing error of the imaging optical system Based on the first, second, and third images after correcting the position of the third image based on the interval change of the imaging positions of the first and second images on the image sensor. An imaging apparatus characterized by forming a composite video signal. An image sensor comprising first and second imaging regions of substantially the same dimensions on the same plane;
A photographing optical system for forming a first object image on the first imaging region and a second object image on the second imaging region;
Image processing means for processing the output signal of the image sensor,
In the first and second imaging regions, a plurality of pixels are aligned at a pitch of a in the horizontal direction and b in the vertical direction on the light receiving surface, and when h is a positive integer, The first and second imaging regions are in a positional relationship of a × h × c in the horizontal direction, b × c in the vertical direction, or a × c in the horizontal direction and b × h × c in the vertical direction. In addition, the image pickup device forms first and second images having substantially the same field of view formed with the same spectral distribution, and the image processing means changes temperature, changes in subject distance, manufacture of the photographing optical system. A change in the interval between the imaging positions of the first and second images on the image sensor based on at least one of the errors is corrected in the process of processing the output signal, and the synthesis is based on the first and second images. An image pickup apparatus for forming a video signal. An imaging device comprising an imaging device having a plurality of imaging regions on the same plane, a photographing optical system for forming object images on the plurality of imaging regions, and an image processing means for processing an output signal of the imaging device. An imaging device control method for controlling an apparatus,
In the image processing means, the first and second images having substantially the same field of view formed with the same spectral distribution output from the image sensor, and the first and second images are formed with different spectral distributions. For the third image having substantially the same field of view as the first and second images, in the process of processing the output signal, the first image based on at least one of temperature change, subject distance change, and manufacturing optical system manufacturing error . After correcting the position of the third image based on the change in the interval between the imaging positions of the first and second images on the image sensor , the first, second, and third images A control method for an imaging apparatus, characterized in that a composite video signal is formed. An image sensor having first and second imaging areas having substantially the same dimensions on the same plane, a first object image on the first imaging area, and a second object on the second imaging area An imaging apparatus control method for controlling an imaging apparatus including an imaging optical system that forms an image and an image processing unit that processes an output signal of the imaging element,
In the first and second imaging regions, a plurality of pixels are aligned at a pitch of a in the horizontal direction and b in the vertical direction on the light receiving surface, and when h is a positive integer, The first and second imaging regions are in a positional relationship of a × h × c in the horizontal direction, b × c in the vertical direction, or a × c in the horizontal direction and b × h × c in the vertical direction. The image processing means has a substantially identical field of view formed with the same spectral distribution formed by the imaging device based on at least one of temperature change, subject distance change, and manufacturing error of the photographing optical system . A change in the interval between the image formation positions of the first and second images on the image sensor is corrected in the process of the output signal to form a composite video signal based on the first and second images. Control method for imaging apparatus. An imaging device comprising: an imaging device having a plurality of imaging regions on the same plane; a photographing optical system that forms object images on the plurality of imaging regions; and an image processing unit that processes an output signal of the imaging device. A control program for controlling a device,
In the image processing means, the first and second images having substantially the same field of view formed with the same spectral distribution output from the image sensor, and the first and second images are formed with different spectral distributions. For the third image having substantially the same field of view as the first and second images, in the process of processing the output signal, the first image based on at least one of a temperature change, a subject distance change, and a manufacturing error of the photographing optical system . After correcting the position of the third image based on an interval change of the imaging position of the first and second images on the image sensor , the first, second, and third images A control program comprising a process code for forming a composite video signal based thereon. An image sensor having first and second imaging areas having substantially the same dimensions on the same plane, a first object image on the first imaging area, and a second object on the second imaging area A control program for controlling an imaging apparatus comprising an imaging optical system for forming an image and an image processing means for processing an output signal of the imaging element,
In the first and second imaging regions, a plurality of pixels are arranged at a pitch of a in the horizontal direction and b in the vertical direction on the light receiving surface, and when h is a positive integer, The first and second imaging regions have a positional relationship of a × h × c in the horizontal direction, b × c in the vertical direction, or a × c in the horizontal direction and b × h × c in the vertical direction. The control program is substantially formed in the image processing means with the same spectral distribution formed by the imaging device based on at least one of temperature change, subject distance change, and manufacturing error of the photographing optical system. A change in the interval between the image forming positions of the first and second images having the same field of view on the image sensor is corrected in the process of processing the output signal to form a composite video signal based on the first and second images. A control program characterized by causing An image sensor comprising first and second imaging regions of substantially the same dimensions on the same plane;
A photographing optical system for forming a first object image on the first imaging region and a second object image on the second imaging region;
A signal processing device for processing an output signal of the image sensor;
In the first and second imaging regions, a plurality of pixels are aligned at a pitch of a in the horizontal direction and b in the vertical direction on the light receiving surface, and when h is a positive integer, The first and second imaging regions are in a positional relationship of a × h × c in the horizontal direction, b × c in the vertical direction, or a × c in the horizontal direction and b × h × c in the vertical direction. In addition, the image sensor forms first and second images having substantially the same field of view formed with the same spectral distribution, and the signal processing device is configured to change the temperature, change the subject distance, and manufacture the photographing optical system. Based on at least one of the errors, a change in the interval between the image formation positions of the first and second images on the image sensor is corrected in the process of processing the output signal, and based on the first and second images An imaging apparatus characterized by forming a composite video signal.

Images