JP4286058B2 - Image processing device - Google Patents

Description

【００４２】
【数２】

と示すことができる。ここでＫは、撮影光学系Ｏの焦点距離ｆ＝１２．５ｍｍ、Ｆ＝１．４、かつ許容錯乱円直径δ＝０．０２ｍｍ、のとき、約１．００４５と計算できる。
【００４３】
式（１）の右側の等式は左側の等式に基づくもので、１枚めのＣＣＤ１１の撮影光学系Ｏからの距離Ｘ１が決まれば、図３のような被写界深度の隣接条件を満足するｎ枚めのＣＣＤｎの距離ｘｎを等比級数演算により求めることができることを示している。
【００４４】
ここで、図１のようにＣＣＤが３枚であり、１枚めのＣＣＤ（Ｆ）１１が無限遠被写体に合焦するよう配置されている、すなわちＣＣＤ（Ｆ）１１の撮影光学系Ｏからの距離ｘ１がｘ１＝ｆとなるよう配置されている場合、ＣＣＤ（Ｆ）１１とＣＣＤ（Ｍ）１２、およびＣＣＤ（Ｍ）１２とＣＣＤ（Ｎ）１３がそれぞれ図３の条件を満足する時、ＣＣＤ（Ｍ）１２の距離ｘ２はｆ＊Ｋ、ＣＣＤ（Ｎ）１３の距離ｘ３はｆ＊Ｋ＊Ｋで求められる。たとえば、上記の撮影光学系Ｏの焦点距離ｆ＝１２．５ｍｍ、Ｆ＝１．４、かつ許容錯乱円直径δ＝０．０２ｍｍ、のとき、ＣＣＤ（Ｆ）１１の距離ｘ１＝１２．５００ｍｍ、ＣＣＤ（Ｍ）１２の距離ｘ２＝１２．５５６ｍｍ、ＣＣＤ（Ｎ）１３の距離ｘ３＝１２．６１３ｍｍと求めることができる。
【００４５】
また、図３を観察すると、近距離側（図の右側）に配置したＣＣＤほど、そのＣＣＤの被写界深度範囲に対応する像側の距離が長いことが判る。したがって、上述のようにｘ１＝ｆとしてＣＣＤ１１〜１３の位置を決定し、その間隔を保ったまま、撮影光学系Ｏを移動（合焦調節）したとしても、各ＣＣＤが受け持つ被写界深度範囲は重畳こそすれ、不連続は生じないことがわかる。すなわち、上記の例においては、ＣＣＤ（Ｆ）１１の距離ｘ１＝１２．５００ｍｍ、ＣＣＤ（Ｍ）１２の距離ｘ２＝１２．５５６ｍｍ、ＣＣＤ（Ｎ）１３の距離ｘ３＝１２．６１３ｍｍの配置状態において、ＣＣＤ（Ｆ）１１〜ＣＣＤ（Ｍ）１２の配置間隔０．０５６ｍｍ、ＣＣＤ（Ｍ）１２〜ＣＣＤ（Ｎ）１３の配置間隔０．０５７ｍｍのままにしておいても、焦点距離ｆ＝１２．５ｍｍ、Ｆ＝１．４、許容錯乱円直径δ＝０．０２ｍｍの各条件が変化しない間は撮影光学系Ｏでどのような距離の被写体をフォーカスしようと、被写界深度範囲に不連続は生じない（ＣＣＤ（Ｆ）１１が無限遠合焦、つまりｘ１＝ｆの時のみ各被写界深度範囲が隣接、他の合焦状態では各被写界深度範囲が重畳する）。
【００４６】
＜ＣＣＤ位置制御についての考察その２＞
上記の式（１）、（２）式の計算例から明らかなように、３枚程度のＣＣＤを配置する場合、各ＣＣＤが隣りのＣＣＤとの距離を図３の条件を満足するように求めた１枚目と２枚目、および２枚目と３枚目のＣＣＤの配置間隔には大きな差が無いことがわかる。
【００４７】
そこで、ＣＣＤの配置間隔を制御する場合、全てのＣＣＤを等距離で配置することも考えられる。その場合、上述のように図３において近距離側（図の右側）に配置したＣＣＤほど、そのＣＣＤの被写界深度範囲に対応する像側の距離が長いことから、ＣＣＤ配置ピッチ（以下、このように３つ配置されたＣＣＤの等間隔の距離に関しては必要に応じ用語「ピッチ」を用いる）としては、最小値を求めれば良い。
【００４８】
ここで、ＣＣＤ１１〜１３の間隔を等間隔とする場合、その等間隔はどのように決定すれば良いかを考察する。
【００４９】
図３において、３角形ＭＳＮと３角形ＵＴＮは合同であるから２枚のＣＣＤの間隔（配置距離）ｐは線分ＬＮの長さｈの２倍である。そして図３において３角形ＰＱＭと３角形ＭＬＮは相似であるから、線分ＬＮの長さｈは
【００５０】
【数３】

である。また、撮影光学系Ｏの口径Ａはその焦点距離ｆと開口絞り値Ｆで表現すればＡ＝ｆ／Ｆであるから、したがってｐは
【００５１】
【数４】

と表せる。なお、式（１）との関係において、ｌ’＝ｘｎ−１、かつｐはｘｎとｘｎ−１の差であるから、式（４）は、式（１）の左側の等式の両辺からｘｎ−１を減じ、Ｋを式（２）で置換することによっても導くことができる。
【００５２】
上記の式（４）は、図３のような隣接する被写界深度を有する２枚のＣＣＤの間隔ｐの一般表現と見ることができる。特に、式（４）はｌ’の１次関数であり、ｐはｌ’が増大するほど、すなわち被写体距離が近距離であるほど大きな値となることが判る。
【００５３】
ここで、被写体距離に拘らず一定の等距離ピッチでＣＣＤを用いる場合、その間隔ｐはどのような値であるべきかを考えると、式（４）から明らかなように被写体が近距離にある条件、すなわち比較的大きな像距離ｌ’を用いて求めた間隔ｐは、それよりも被写体が遠距離である場合にはその被写体距離で必要な間隔ｐよりも過大となり、連続すべき２つの被写界深度範囲が不連続となってしまうため、不適当であることがわかる。すなわち、被写体距離に拘らず一定の等距離ピッチのＣＣＤを用いるには、その間隔は式（４）のｐが取り得る最小値でなければならない。
【００５４】
式（４）が最小値をとるのは、像距離ｌ’、すなわち像点Ｌに対応する被写体の距離が無限大である場合で、この時像点Ｌは撮影光学系Ｏの後側焦点に一致するから、式（４）のｌ’をｌ’＝ｆで置換することによってこの光学系において採用すべきＣＣＤの等距離ピッチｐｃｃｄは
【００５５】
【数５】

と求めることができる。一般のカメラ設計仕様においては、最も遠距離を受け持つＣＣＤ（図１のＣＣＤ１１（Ｆ））は後焦点位置まで相対移動できるように設計するのが普通であると考えられ、ｌ’＝ｆの条件を用いて式（５）を導くのは至極妥当なものといえる。
【００５６】
さて、式（５）で示されるＣＣＤピッチｐｃｃｄは、撮影光学系Ｏの焦点距離ｆおよび開放Ｆ値（さらに許容錯乱円直径δ、しかしＣＣＤを交換するなどそのサイズが変更となる場合を除けばこの値には定数を用いるのが普通である）によって決定される。したがって、レンズ交換ないしズーミングによって撮影光学系Ｏの焦点距離ｆおよび開放Ｆ値のいずれかに変更が生じた場合は、ＣＣＤ１１〜１３の距離が式（５）により求められる等距離ピッチｐｃｃｄになるように移動すればよい。
【００５７】
δ＝０．０２（ｍｍ）の場合、８．０ｍｍ、１２．５ｍｍ、２０．０ｍｍの各焦点距離ｆ、および１．４、２．８、５．６の各開放Ｆ値における式（５）のｐｃｃｄの計算結果（ｍｍ）は、たとえば
【００５８】
【表１】

のようになる。ここで、上の表１を観察すると、ｐｃｃｄの値は開放絞り値Ｆに大きく依存し、焦点距離ｆの影響は少ないことが判る。このことは、式（５）を観察すると、右辺分母全体と分子のｆにより形成される定数項ｆ／（ｆ−δＦ）が常に１よりも大きく、かつ１にごく近い値となることからも明らかである。したがって、近似計算においては、ＣＣＤピッチｐｃｃｄは
【００５９】
【数６】

としても実用上は充分であるといえる。この式（６）のＣＣＤピッチｐｃｃｄは式（５）の値に極めて近く、しかも必ず式（５）の値よりも小さな値となるので、２つの被写界深度範囲が不連続となる問題も生じない（表１の２δＦの値を参照）。
【００６０】
すなわち、最も簡便な設計仕様においては（等距離の）ＣＣＤピッチｐｃｃｄは式（６）により決定すればよく、その場合、撮影光学系Ｏの開放Ｆ値のみによりＣＣＤピッチｐｃｃｄを決定できる（たとえばレンズ交換を行なっても、交換前と後のレンズの開放Ｆ値が同一であればＣＣＤピッチｐｃｃｄは変更する必要がない）ため、制御のためのハードウェアおよびソフトウェアはより簡単安価なものとなる。なお、式（６）は、いわゆる無限遠の焦点深度２δＦと同一であり、開放Ｆ値および許容錯乱円直径δのみにより決定される。
【００６１】
なお、式（５）のｐｃｃｄをＣＣＤ間隔として採用した場合、図３の像点Ｗの像距離（以下ｌｗ’）は像点Ｌの像距離ｌ’と式（５）のｐｃｃｄの和として求めることができ、また像距離ｌ’およびｌｗ’から、（１／ｌ）＋（１／ｌ’）＝（１／ｆ）の関係を用いて像距離ｌ’およびｌｗ’それぞれに対応する被写体距離ｌおよびｌｗを算出することができる。ちなみにｆ＝１２．５（ｍｍ）、Ｆ＝１．４、δ＝０．０２（ｍｍ）の場合（ＣＣＤピッチｐｃｃｄ＝０．０５６１ｍｍ）、被写体距離の差（ｌ−ｌｗ）すなわち、２枚のＣＣＤに結像する被写体距離の差は、ｌ＝１００（ｍ）において（ｌ−ｌｗ）＝９７．３ｍ、ｌ＝５（ｍ）において（ｌ−ｌｗ）＝３２０１ｍｍ、ｌ＝１（ｍ）において（ｌ−ｌｗ）＝２５９ｍｍ、ｌ＝７００（ｍｍ）において（ｌ−ｌｗ）＝１３６ｍｍ、ｌ＝５００（ｍｍ）において（ｌ−ｌｗ）＝７２ｍｍとなり、これらの数値は図２に示した結像範囲の構成と概ね一致していることが判る。
【００６２】
以上のように、撮影光学系の作動条件（焦点距離ｆ、許容錯乱円直径δ、撮影絞りＦ）が変化しない間は、撮像素子の形成する被写界深度範囲が隣接またはわずかに重畳する条件（式（１）および式（２））を満足するようＣＣＤの配置間隔を制御する、あるいは各撮像素子の配置間隔がほぼ等距離となる（式（５）あるいは式（６））よう各撮像素子の位置を制御することにより、撮影光学系Ｏを移動するだけで風景撮影〜口腔内撮影において、３つのＣＣＤで全ての被写界深度を連続的にカバーして撮影を行なえることが判る。
【００６３】
なお、式（１）および式（２）による条件は、撮影光学系Ｏよりも遠い側（近距離被写体側）のＣＣＤの間隔が大きくなる。つまり、式（５）あるいは式（６）による等距離配置よりも撮影光学系Ｏよりも遠い側（近距離被写体側）においてより広い被写界深度を確保できる利点があり、一方、式（５）あるいは式（６）による等距離配置の場合はＣＣＤ位置の制御がより容易になる、という利点がある。
【００６４】
＜ＣＣＤ配置間隔の制御＞
レンズ交換やズーミングによりレンズの設定（焦点距離、開放絞り）条件が変化した場合は、式（１）および（２）、あるいは式（５）または式（６）に基いてＣＣＤ配置間隔を制御する必要がある。
【００６５】
以下では、撮影光学系Ｏの焦点距離に応じてどのような制御を行なえば良いかについて制御の一例を示す。
【００６６】
ここでは、式（５）または式（６）の条件（ＣＣＤ等距離配置）に基いてＣＣＤ配置間隔を等距離に制御する例を示すが、下記と同様の制御は式（１）および（２）の条件を用いる場合でも遠距離側と近距離側のＣＣＤの間隔の変更量が異なるだけで同様に適用できる。
【００６７】
さて、もし、多少厳密な制御を行なうのであれば、式（５）に基づくＣＣＤ配置ピッチを制御する、つまり上述の表１に示したレンズの場合であれば、最も右側のカラムの数値に基づきＣＣＤ配置ピッチを制御する。すなわち、式（６）に基づく制御では、レンズの設定条件のうち、焦点距離ｆおよび開放絞り値Ｆのいずれかが変更となった場合にはこれに応じてＣＣＤ配置ピッチを制御することになる。この式（５）に基づく制御によれば、後述の絞り値のみを用いる制御に比して３枚のＣＣＤによって少し広い被写界深度範囲をカバーできる利点があるが、以下では、式（６）に基く絞り値のみを用いるより簡便な制御を例示する。
【００６８】
図１においては、図１のＣＰＵ３０がレンズ交換やズーミングによる撮影光学系Ｏの焦点距離の変化を検出できるよう、レンズ検出手段２８を設けてあるので、ＣＰＵ３０はレンズ検出手段２８の出力に基いてＣＣＤの位置を制御することができる。
【００６９】
図１にはＣＣＤの等間隔配置制御を行なうための最も簡略な構成を示してある。すなわち、図１では、ＣＣＤ１２（Ｍ）を固定とし、このＣＣＤ１２（Ｍ）を基準として、ＣＣＤ１１（Ｆ）とＣＣＤ１３（Ｎ）をＣＣＤ１２（Ｍ）に近づけたり、離したりできるようにしてある。
【００７０】
図１のＣＣＤ１１〜１３の位置は、撮影光学系Ｏが大きな開口絞り値（Ｌ）のもの（「暗い」レンズ。たとえばＦ＝４．０やＦ＝５．６など）である場合を示しており、このときのＣＣＤ１１〜１２の像面距離（上述のＣＣＤ配置ピッチｄ）はｄｍｆｌ、ＣＣＤ１２〜１３の像面距離（上述のＣＣＤ配置ピッチｄ）はｄｍｎｌである。
【００７１】
撮影光学系Ｏが中程度の開口絞り値（Ｍ）のもの（「暗い」レンズ。たとえばＦ＝１．４〜Ｆ＝２．０程度）である場合は、ＣＣＤ１１〜１２の像面距離（上述のＣＣＤ配置ピッチｄ）はより短いｄｍｆｍに、ＣＣＤ１２〜１３の像面距離（上述のＣＣＤ配置ピッチｄ）はより短いｄｍｎｍに調節すればよい。
【００７２】
また、撮影光学系Ｏが小さな開口絞り値（Ｓ）のもの（「明るい」レンズ。たとえばＦ＝１．０〜Ｆ＝１．４程度）である場合を示しており、このときのＣＣＤ１１〜１２の像面距離（上述のＣＣＤ配置ピッチｄ）はさらに短いｄｍｆｓに、ＣＣＤ１２〜１３の像面距離（上述のＣＣＤ配置ピッチｄ）はさらに短いｄｍｎｓに調節すればよい。
【００７３】
このようなＣＣＤ間隔の制御は、ＣＣＤ位置制御手段２１によりＣＣＤ１１（Ｆ）を、ＣＣＤ位置制御手段２３によりＣＣＤ１３（Ｎ）によって移動させ、ＣＣＤ１２（Ｍ）は固定位置に置いておけばよい。このようにＣＣＤ１２（Ｍ）は固定位置に置く場合は、ＣＣＤ１２（Ｍ）のためのＣＣＤ位置制御手段２２（破線）は当然ながら不要である。ＣＣＤ位置制御手段２１および２３は、ＣＰＵ３０の指令に応じて図示のＣＣＤ間隔ｄｍｆｌ、ｄｍｎｌ、ｄｍｆｍ、ｄｍｎｍ、ｄｍｆｓ、およびｄｍｎｓを形成するよう動作するソレノイドやモータなどの駆動手段を用いて構成することができる（あるいは、機械的な精度を充分確保できるのであれば、純機械的な機構を介してユーザの操作力のみを駆動源として動作するようＣＣＤ位置制御手段２１および２３を構成することもできる）。
【００７４】
以上のようにして、本実施形態によれば、ＣＣＤの移動制御をなるべく行なうことなく実用上充分なＮＭＦ画像合成を行なえる画像処理装置を提供することができる。すなわち、本実施形態によれば、固定焦点距離の撮影光学系ＯではＣＣＤの移動制御は不要、レンズ交換ないしズーミングが行なわれ焦点距離が変更された時のみ行なえばよく、現実の風景撮影から至近距離の撮影において、実用上問題ない連続した被写界深度を得ることができ、良好な条件でＣＣＤ１１〜１３から得た１１、１２、１３の遠距離（Ｆ：Ｆａｒ）、中距離（Ｍ：Ｍｉｄｄｌｅ）、近距離（Ｎ：Ｎｅａｒ）を合成処理することができ、極めて高画質の画像を出力することができる。
【００７５】
本実施形態では、ＣＣＤの移動制御を行なう場合でも、レンズ交換ないしズーミングに応じて、各撮像素子の形成する被写界深度範囲が隣接またはわずかに重畳する条件を満足するよう、また、特に簡便な仕様においては等間隔の位置にＣＣＤを移動すれば足りるので、ＣＰＵ３０の制御は極めて簡単になる。あるいは、ＣＰＵ３０のような制御手段を用いなくても、レンズ交換ないしズーミングに連動する機械的な機構のみによって、ＣＣＤの移動制御を行なうことも極めて容易であり、ＣＣＤの移動のために必要なハードウェアあるいはソフトウェアのコストは極めて小さくて済む。
【００７６】
また、図１の構成では、３つのＣＣＤのうち、１つを固定し、他の２つのみを移動する、という極めて簡単安価な構成により不要、レンズ交換ないしズーミングに応じて必要なＣＣＤの移動制御を行なうことができる。
【００７７】
なお、図１では説明を容易にするため、ＣＣＤ（Ｍ）１２を固定とし、ＣＣＤ（Ｆ）１１とＣＣＤ（Ｎ）１３を移動させる構成を示したが、いずれのＣＣＤを固定とするかは任意である。たとえば、通常のレンズ構成では、撮影光学系Ｏは近距離側で被写体方向に前進し、遠距離側でＣＣＤの方向に後退し所定の無限遠位置で停止するよう動作させるので、ＣＣＤ（Ｆ）１１を固定し、ＣＣＤ（Ｍ）１２とＣＣＤ（Ｎ）１３を移動させるような構成とすれば、焦点距離の異なる各レンズにおいて距離環（フォーカス手段２７）の設計を簡略に行なえるようになる、と考えられる。
【００７８】
＜ＣＣＤ移動のための構成＞
以上では、特に簡略な構成として、撮影光学系の特定の作動条件に適した等距離の相互間隔をもって複数のＣＣＤ１１〜１３を配置すること、また、撮影光学系の作動条件の変更に応じて等距離の相互間隔を保ったまま各ＣＣＤを異なる位置に移動させる構成を示した。
【００７９】
以下では、特にこれら複数のＣＣＤを撮影光学系の作動条件の変更に応じて等距離の相互間隔を保ったまま移動させる構成に適したより具体的なメカニズムの例を示す。
【００８０】
図４はＣＣＤ１１’（Ｆ）、１２（Ｍ）、１３（Ｎ）を有するＮＭＦ画像合成システムの構成を示している。図４では、図１と同一ないし相当する部材には同一符号を用い、これらに関する説明は図１のものと同様であるものとして以下では説明を省略する。
【００８１】
図４が図１と異なっているのは、ハーフミラーＭ１の反射方向をハーフミラーＭ２（あるいはミラー３）の反射方向と直角に相対するように反転させ、ＣＣＤ（Ｆ）１１’を撮影光学系Ｏに関してＣＣＤ（Ｍ）１２、ＣＣＤ（Ｎ）１３とは反対側に配置した点である。なお、図４では、図１との差異を明確にするために、図１のＣＣＤ（Ｆ）１１の配置を破線で示してある。
【００８２】
また、ＣＣＤ（Ｍ）１２は図１と同様に撮影光学系Ｏに対しては固定位置に配置する。
【００８３】
さらに、ＣＣＤ（Ｆ）１１’はＣＣＤ（Ｎ）１３と結合部材４１を用いて機械的に一体に結合されている。結合部材４１は、たとえば金属（あるいはプラスチックなどの任意の材質）の一枚板のような部材であり、この結合部材４１上にＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３をリジッドに固定してある。
【００８４】
そして、ＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３は、結合部材４１を介して撮影光学系の作動条件の変更に応じてＣＣＤ位置制御手段２３によりＳ（小絞り）またはＬ（大絞り）側に移動される。なお、このときの結合部材４１の移動方向（Ｓ／Ｌ）はハーフミラーＭ１〜ＣＣＤ（Ｆ）１１’間の光軸と平行とする。
【００８５】
当然ながら、このような構成ではＣＣＤ（Ｆ）１１’とＣＣＤ（Ｎ）１３を機械的に一体に結合しているため、図１の構成では必要であったＣＣＤ位置制御手段２１が不要となり、撮影光学系の作動条件の変更に際してはＣＣＤ位置制御手段２３のみを用いて上述のＣＣＤ間隔ｄｍｆｌ、ｄｍｎｌ、ｄｍｆｍ、ｄｍｎｍ、ｄｍｆｓ、およびｄｍｎｓを形成するよう制御することができる。
【００８６】
すなわち、図４においては、ＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３をＳ（小絞り）方向に移動した場合、結合部材４１による移動方向は同一であるが、上記のミラー配置により、光軸上ではＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３はいずれもＣＣＤ（Ｍ）１２に近付くように動くことになる。もちろん、このときのＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３の移動距離は同一である。逆にＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３をＬ（大絞り）方向に移動した場合は、光軸上ではＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３はいずれもＣＣＤ（Ｍ）１２から等距離分づつ離れるよう動く。
【００８７】
図４のような構成を採用することにより、撮影光学系の作動条件の変更に応じて等距離の相互間隔を保ったまま容易に移動させることができる。図４の構成では、２つのＣＣＤを移動させるのに単一の駆動手段のみを用いた極めて簡単安価なハードウェアしか必要としない、という利点がある。
【００８８】
＜プリズムを用いた構成例＞
図４に基づくより具体的な構成例を図５に示す。図５の構成は、カラーテレビカメラなどにおいて用いられているプリズムを利用したものである。
【００８９】
図５のプリズム５１は部分透過の反射界面５２、５３、５４を有し、主光軸（Ｍ）に沿って図の左側の撮影光学系Ｏ（図５では不図示）から入射された撮影光をＦ、Ｍ、Ｎの各方向に出射し、ＣＣＤ１１’（Ｆ）、１２（Ｍ）、１３（Ｎ）（図４中の各ＣＣＤと同じもの）に入射する。各ＣＣＤ１１’（Ｆ）、１２（Ｍ）、１３（Ｎ）に対する出射光の強度はそれぞれ同等となるように反射界面５２、５３、５４の透過／反射率が定められている。
【００９０】
プリズム５１でカラー撮影のための分光を行なうには、ＣＣＤとの間にカラーフィルタを配置するが、本発明のようなＮＭＦ画像合成システムでは必要ない。
【００９１】
この種のプリズムは、カラーテレビカメラなどにおける用途があり、比較的容易に入手することができ、ＮＭＦ画像合成システムで必要な光路分割手段としては好適なものであるが、出射光の強度を制御するなどの目的で各出射面の角度は図示のような特殊な角度になっている。たとえば、図示の構成では、出射方向Ｆは主光軸（Ｍ）に対して４７°出射方向Ｎは主光軸（Ｍ）に対して７０°の構成となっている。
【００９２】
しかしながら、このような構成においても、図４と同等のＣＣＤ位置制御を行なうことができる。すなわち、図５においてもＣＣＤ（Ｆ）１１’とＣＣＤ（Ｎ）１３は結合部材４１を用いて機械的に一体に結合されており、ＣＣＤ位置制御手段２３（不図示）により直線Ｒに沿ってＳ（小絞り）またはＬ（大絞り）側に移動される。
【００９３】
この直線Ｒは、出射方向Ｆと出射方向Ｎを２等分する直線Ｂと９０°交差するようにとられている。この直線Ｒに沿ってＳ（小絞り）またはＬ（大絞り）側に移動される。ＣＣＤ（Ｆ）１１’とＣＣＤ（Ｎ）１３を移動させる構成においては、ＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３への出射方向ＦおよびＮが直線Ｒとなす角度は３１．５°（９０°−５８．５°＝（７０°＋４７°）／２）となり、結合部材４１を移動した時のＣＣＤ（Ｆ）１１’およびＣＣＤ（Ｎ）１３の直線ＦおよびＮ上における移動量がまったく等しくなるように制御できる。
【００９４】
以上のように、図４と同等の構成は図５のような特殊な出射方向を有するプリズムによる光路分割手段を用いる場合においても部材の配置方向を選択するだけで同様に達成することができる。図５の構成は、カラー撮影用途の入手が比較的容易な分光プリズムを利用しており、カスタムメイドの高価な構成を用いなくてもＮＭＦ撮影システムを実現することができる、という利点がある。
【００９５】
【発明の効果】
以上の説明から明らかなように、本発明によれば、単一の撮影光学系により結像された画像を撮影する３つの撮像素子を有し、前記各撮像素子により撮影された画像データを合成した画像を得る画像処理装置において、撮影光学系から前記各撮像素子に撮影光を導入する光路分割手段を設けるとともに、前記撮影光学系の開放絞りの値、または前記撮影光学系の焦点距離の値に変化が生じた場合に、前記３つの撮像素子のうち特定の１つの位置を固定し、残りの撮像素子の２つを機械的に結合し、この機械的に結合した２つの撮像素子を同一方向に移動した時、前記各撮像素子が等距離の相互間隔を保ったまま前記各撮像素子を異なる位置に移動させるよう前記光路分割手段の光路分割方向を定める構成を採用しているので、撮像素子の移動制御をできるだけ行なうことなく、極めて簡単安価なハードウェアおよび制御機構ないし制御ソフトウェアにより、実用上充分なＮＭＦ画像合成を行なえる画像処理装置を提供することができ、特に、単一の駆動手段を用いて機械的に結合した２つの撮像素子を同一方向に移動させるだけで、撮影光学系の特定の作動条件に適した等距離の相互間隔による撮像素子の配置を達成することができる、という優れた効果が得られる。
【図面の簡単な説明】
【図１】本発明を採用したＮＭＦ画像合成システムの要部の構成を示したブロック図である。
【図２】本発明の画像合成システムの原理を示した説明図である。
【図３】本発明の画像合成システムにおけるＣＣＤ配置条件を示した説明図である。
【図４】本発明を採用した異なるＮＭＦ画像合成システムの要部の構成を示したブロック図である。
【図５】３色分光用プリズムを流用して光路分割を行なう構成におけるＣＣＤ移動制御を示した説明図である。
【符号の説明】
１１〜１３ ＣＣＤ
１４ ビデオＩ／Ｆ
２１、２２、２３ ＣＣＤ位置制御手段
２７ フォーカス手段
２８ レンズ検出手段
３０ ＣＰＵ
３１ 操作部
３２ プログラムメモリ
３３ ネットワークＩ／Ｆ
３５ メモリ
４１ 結合部材
５１ プリズム
Ｍ１、Ｍ２ ハーフミラー
Ｍ３ ミラー[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus having a plurality of imaging elements that capture images formed by a single imaging optical system, and obtaining an image obtained by synthesizing a plurality of image data captured by the plurality of imaging elements. Is.
[0002]
[Prior art]
A technique is known in which images having different shooting distances are combined to obtain an image in which all subjects from a distant view to a close view are in focus (for example, Patent Document 1 below).
[0003]
In this type of image processing apparatus, for example, different positions with respect to the photographing optical system so as to have depths of field at short distance (N: Near), medium distance (M: Middle), and far distance (F: Far), respectively. N, near distance, M (medium distance), and F (long distance) obtained from the respective image sensors, respectively, are arranged at short distance, medium distance, and long distance image sensors. The image is analyzed, and an appropriate part is extracted from each image based on the sharpness of the image and synthesized. Hereinafter, such an imaging system is referred to as an NMF image synthesis system.
[0004]
The applicant has already proposed an image processing apparatus having manual adjustment means for the operator to independently adjust the distances between the imaging optical system and the plurality of imaging elements as the configuration of the NMF image synthesis system (see below). Patent Document 2).
[0005]
[Patent Document 1]
JP-A-10-262176 (FIG. 1)
[Patent Document 2]
Japanese Patent Application No. 2002-50607 (FIG. 1)
[0006]
[Problems to be solved by the invention]
The invention described in Patent Document 2 proposes a configuration in which the distances between the photographing optical system and the plurality of image sensors are independently adjusted. Since the distances of subjects at short distance (hereinafter referred to as N), medium distance (hereinafter referred to as M), and long distance (hereinafter referred to as F) are not constant, each of the image sensors can be precisely adjusted in the configuration as in Patent Document 2. There is an advantage that. In particular, it is necessary to use a wide aperture and to capture multiple subjects that cannot be placed within the depth of field with a normal camera such as a distant mountain and a close-in mouth. It can meet the special demands for photography.
[0007]
However, in the configuration as in Patent Document 2, means for moving all the image sensors is necessary, the hardware configuration is complicated, and it is extremely difficult if the operator manually moves the image sensor. Is difficult and requires skill. Further, even if the positions of all the image sensors are controlled by a control means such as a computer, in that case, the program for controlling the positions of the image sensors is at least a short distance (N), a medium distance (M), and a long distance. It is necessary to determine which of the subjects in (F) is the target, and the configuration becomes extremely complicated.
[0008]
Therefore, a system that can perform practically sufficient NMF image composition for a subject in a common sense range from a distant view to a close range without performing movement control of the image sensor as much as possible is desired. For example, when the photographing lens is a fixed type with a single focal length and the photographing aperture does not change, it is convenient if photographing can be performed without performing any movement control of the image sensor.
[0009]
In addition, in a configuration in which the focal length is changed as in the case where the photographing aperture is adjusted or the photographing lens is interchangeable or zoomed, the position of the image sensor is changed when the specific focal length or the open F-number is changed by lens replacement or zoom setting. However, for example, in a state where a specific focal length is selected, it is convenient if shooting can be performed without further movement control of the image sensor. Absent.
[0010]
An object of the present invention is to provide an image processing apparatus that can perform practically sufficient NMF image synthesis without performing movement control of an image sensor as much as possible.
[0011]
[Means for Solving the Problems]
In order to solve the above problems, according to the present invention, an image formed by a single imaging optical system is captured. Three The image sensor, Photographed by each image sensor In an image processing apparatus for obtaining an image obtained by combining image data, an optical path dividing unit for introducing photographing light from a photographing optical system to each of the imaging elements is provided, When a change occurs in the value of the open aperture of the photographing optical system or the value of the focal length of the photographing optical system, Above Three The position of a specific one of the image sensors is fixed, and the remaining image sensors Two When the two image pickup devices mechanically connected are moved in the same direction, the image pickup devices are moved to different positions while maintaining an equal distance from each other. The configuration for determining the optical path dividing direction of the optical path dividing means is adopted.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0013]
FIG. 1 shows the configuration of an NMF image composition system employing the present invention. In FIG. 1, symbol O is a photographing optical system, and half mirrors (or beam splitters) M1 and M2 and mirrors (or prisms) M3 are arranged on the optical axis, and a long distance (F) and a medium distance (F). The CCDs 11, 12, and 13 are arranged at different positions with respect to the photographing optical system O so as to have the depth of field at M) and the short distance (N), respectively. Although not shown in FIG. 1, a photographing aperture is disposed in the optical path between the photographing optical system O and each CCD as necessary.
[0014]
The outputs of the CCDs 11 to 13 are input to an image processing system for synthesizing images via a video I / F (interface) 14.
[0015]
In FIG. 1, a mirror M3 composed of a total reflection mirror is shown for easy understanding of the distance from the optical system to the CCD, but this mirror M3 is not always necessary, and can be omitted in many cases in an actual optical system. The transmission ratios of the half mirrors M1 and M2 are set so that M1 = 66% and M2 = 50%, and the light quantity reaching each CCD does not cause an imbalance.
[0016]
In such a configuration, by setting the distances df, dm, and dn between the CCDs 11 to 13 and the photographing optical system O, the photographing distances (focusing distances) of the N, M, and F images, and N, M, and F, respectively. Each depth of field of F is determined. In the present embodiment, the distances between the CCDs 11 to 13 and the photographing optical system O are df, dm, and dn.
[0017]
The present invention intends to provide a configuration in which the position control (control of df, dm, dn) of the CCDs 11 to 13 is minimized.
[0018]
The determination of the distances df, dm, and dn between the CCDs 11 to 13 and the photographing optical system O, and the position control of the CCDs 11 to 13 (control of df, dm, and dn) will be described in detail later.
[0019]
The photographic optical system O can move in the n direction when focusing on a subject at a short distance by the focusing means 27, and can be moved in the f direction when focusing on a subject at a long distance. The simplest configuration of the focusing means 27 is that the operator manually moves the photographing optical system O via a mechanism such as a helicoid or a cam. Of course, the focus means 27 may be driven by a motor or the like, or may be focused using a known autofocus control means. As a detector for autofocus control, an optical sensor and an ultrasonic sensor (not shown) provided separately may be used, and a configuration using the outputs of the CCDs 11 to 13 may be considered.
[0020]
In the configuration of FIG. 1, lens detection means 28 is provided. The lens detection means 28 is at least a photographic optical system O currently mounted.
·Focal length
Open F value (or current aperture value)
Thus, it is configured to detect information regarding the current lens configuration. Alternatively, the lens detection means 28 may be capable of detecting lens model information and the like.
[0021]
Of the above, “focal length” is the focal length of the lens of the photographing optical system O, not “subject distance”. In this specification, “focal length” is used to indicate only the focal length of the lens of the photographing optical system O, and “subject distance” is used to indicate the distance of a focused subject.
[0022]
The lens detection means 28 is not necessary if the photographing optical system O is a fixed lens with a single focal length fixed to the casing. However, in the case of an interchangeable or zoom lens, the lens detection means 28 can be replaced by lens replacement or zooming. Since the configuration information such as the focal length and the open F value changes, the information is transmitted to the CPU 30 described later.
[0023]
In the case of an interchangeable lens, the position information such as a notch provided with the lens detection means 28 at the specific position of each lens can be detected by using a limit switch or a magnetic sensor. In the case of a zoom lens, the current focal length may be detected by detecting the rotation position information of the zoom ring of the lens. Further, in the case of a system in which an interchangeable and zoom lens can be mounted, a detection mechanism for both of them can be provided.
[0024]
Similarly, regarding the open F value, position information such as a notch provided at a specific position of each lens can be detected by the lens detection means 28 using a limit switch, a magnetic sensor, or the like. Conventional detection means for detecting the position of the aperture ring can be used to detect the current aperture value.
[0025]
The outputs of the CCDs 11 to 13 are input to the image processing apparatus on the right side of FIG. 1 via the video I / F 14. The process of taking a composite image from each of the CCDs 11 to 13 is performed by operating a shutter button (not shown).
[0026]
The image processing apparatus on the right side of the video I / F 14 can also be configured using hardware such as a PC (personal computer). The image processing apparatus includes a CPU 30 that performs image processing described later, an operation unit 31 that includes an operation unit such as a shutter button, a program memory 32 that stores image processing described later as a program of the CPU 30, and a memory used as a work area for image processing. 35.
[0027]
The CPU 30 inputs the image data of the long distance (F: Far), medium distance (M: Middle), and short distance (N: Near) of the CCDs 11, 12, 13 input from the video I / F 14 and stored in the memory 35. On the other hand, by performing predetermined image processing, portions with high definition are extracted from the N, M, and F images, respectively, and an image with high definition is synthesized. This compositing process itself is not a subject of the present invention, and image processing convenient for those skilled in the art may be appropriately performed.
[0028]
In FIG. 1, a network I / F 33 is provided to share the obtained image data with other devices. The I / F system of the network I / F 33 is arbitrary such as Ethernet (trade name).
[0029]
Next, a method for determining the positions of the CCDs 11 to 13, that is, the distances df, dm, and dn between the CCDs 11 to 13 and the photographing optical system O will be described.
[0030]
In the present embodiment, the positions of the CCDs 11 to 13 are fixed while the focal length and the photographing aperture (F number) of the photographing optical system O are constant, and the object of a close distance from a distant view is only moved by the photographing optical system O. Make it possible to shoot.
[0031]
Also, between CCDs 11-13 Arrangement interval (hereinafter simply referred to as “interval”) Considering the ease of control, we will also consider a configuration that controls to be equally spaced. The positions of the CCDs 11 to 13 are fixed, or their arrangement interval This is because the device design and the configuration of the control circuit of the CCDs 11 to 13 can be made extremely simple and inexpensive.
[0032]
FIG. 2 is a diagram for explaining an outline of a design concept for determining the distances df, dm, and dn between the CCDs 11 to 13 and the photographing optical system O in the image composition system of the present invention.
[0033]
In FIG. 2, a subject close to a distant view and the photographing optical system O are placed at a fixed position, and three CCDs 11 to 13 are arranged behind the photographing optical system O. Assuming that the distance d1 between the CCDs 11 and 12 and the distance d2 between the CCDs 12 and 13 are substantially equal, the depth of field range of these CCDs 11 to 13 is narrow for a short-distance subject and wide for a long-distance subject. Become.
[0034]
Although FIG. 2 is not exact, the correspondence between the subject and its image can vary depending on the focal length of the photographic optical system O used. However, in a general lens having a focal length of several to several tens of mm, as shown in the figure. Each of the CCDs 11 to 13 centers on a subject such as a distant mountain, a middle tree, and a close-up person at a long distance, and a subject (back teeth, middle teeth, anterior teeth) at a close distance such as in the oral cavity. Covers the depth of field range.
[0035]
In FIG. 2, if the focal length of the photographing optical system O and the F value of the photographing aperture are determined, the distance d1 between the CCDs 11 and 12 and the distance d2 between the CCDs 12 and 13 may be controlled to be constant. It also shows that it is almost possible to handle practical shooting situations.
[0036]
That is, in practical use, it is extremely rare to take a sharp image of a close-up oral cavity and a distant mountain within the same screen. A camera is used to selectively photograph a landscape including a person or a subject (back teeth, middle teeth, anterior teeth) in the oral cavity at a close distance.
[0037]
Therefore, in order to perform satisfactory NMF image composition in each of these landscape (far-distance) photographing and intraoral (short-distance) photographing, there is no discontinuity in the depth of field range handled by the CCDs 11 to 13. That is, it is only necessary to control so that the depth of field ranges are adjacent or slightly overlapped.
[0038]
The consideration about the arrangement position thru | or the arrangement | positioning space | interval of CCD11-13 is shown below.
[0039]
<Consideration about CCD position control # 1>
FIG. 3 shows image points L and W of two subjects formed by a photographing optical system O (having a focal length f and a photographing aperture value F) with an aperture A, and an allowable circle of confusion diameter δ before and after the image points L and W, respectively. 2 illustrates a depth-of-field range that satisfies the above. The two depth-of-field ranges relating to the image points L and W in FIG. 3 are in a special state adjacent to each other without being superposed. Therefore, if a CCD is placed at each of the image points L and W, these two images The CCD covers the range from point R to point V, and a sharp image satisfying the allowable circle of confusion diameter δ can be taken within this range.
[0040]
Here, CCDn-1 at the position of the image point L is placed, the next CCDn is placed at the position of the image point W, and the distances of the CCDn-1 and the CCDn from the photographing optical system O are xn-1 and xn. . Since the triangle MSN and the triangle UTN are congruent and the triangle MSN and the triangle PP′N are similar, the distances xn−1 and xn between the CCDn−1 and the CCDn are
[0041]
[Expression 1]

[0042]
[Expression 2]

Can be shown. Here, K can be calculated as about 1.0045 when the focal length f of the photographing optical system O = 12.5 mm, F = 1.4, and the allowable circle of confusion diameter δ = 0.02 mm.
[0043]
The equation on the right side of equation (1) is based on the equation on the left side, and if the distance X1 from the photographing optical system O of the first CCD 11 is determined, the adjacent condition of the depth of field as shown in FIG. It shows that the distance xn of the satisfactory nth CCDn can be obtained by geometric series calculation.
[0044]
Here, as shown in FIG. 1, there are three CCDs, and the first CCD (F) 11 is arranged so as to focus on a subject at infinity, that is, from the photographing optical system O of the CCD (F) 11. When the distance x1 is set so that x1 = f, the CCD (F) 11 and the CCD (M) 12 and the CCD (M) 12 and the CCD (N) 13 satisfy the conditions shown in FIG. The distance x2 of the CCD (M) 12 is obtained by f * K, and the distance x3 of the CCD (N) 13 is obtained by f * K * K. For example, when the focal length f of the photographing optical system O is 12.5 mm, F is 1.4, and the allowable circle of confusion circle δ is 0.02 mm, the distance x1 of the CCD (F) 11 is 12.500 mm, The distance x2 of the CCD (M) 12 is 12.556 mm and the distance x3 of the CCD (N) 13 is 12.613 mm.
[0045]
Further, when observing FIG. 3, it can be seen that the CCD arranged on the short distance side (right side in the figure) has a longer distance on the image side corresponding to the depth of field range of the CCD. Therefore, even if the positions of the CCDs 11 to 13 are determined with x1 = f as described above and the photographing optical system O is moved (focus adjustment) while maintaining the interval, the depth of field range that each CCD takes charge of. It can be seen that there is no discontinuity due to superposition. That is, in the above example, in the arrangement state where the distance x1 of the CCD (F) 11 is 12.500 mm, the distance x2 of the CCD (M) 12 is 12.556 mm, and the distance x3 of the CCD (N) 13 is 12.613 mm. , Arrangement of CCD (F) 11 to CCD (M) 12 interval 0.056 mm, CCD (M) 12 to CCD (N) 13 arrangement interval Even if 0.057 mm is left as it is, the photographic optical system O does not change what the focal length f = 12.5 mm, F = 1.4, and the allowable confusion circle diameter δ = 0.02 mm. When focusing on an object at a distance, there is no discontinuity in the depth of field range (when the CCD (F) 11 is in focus at infinity, that is, when x1 = f, each depth of field range is adjacent, In the focus state, each depth of field range is superimposed).
[0046]
<Consideration on CCD position control # 2>
As is clear from the calculation examples of the above formulas (1) and (2), when about three CCDs are arranged, the distance between each CCD and the adjacent CCD is obtained so as to satisfy the conditions of FIG. 1st and 2nd CCDs and 2nd and 3rd CCDs interval It can be seen that there is no big difference.
[0047]
So CCD Arrangement interval When controlling the above, it is conceivable to arrange all CCDs at equal distances. In this case, as described above, the CCD arranged on the short distance side (right side in the figure) in FIG. 3 has a longer distance on the image side corresponding to the depth of field range of the CCD. (Hereinafter, the term “pitch” is used as necessary with respect to the equally spaced distances of the three CCDs arranged in this way.) For this, the minimum value may be obtained.
[0048]
Here, when the intervals between the CCDs 11 to 13 are set to be equal intervals, how to determine the equal intervals is considered.
[0049]
In FIG. 3, since the triangular MSN and the triangular UTN are congruent, interval (Arrangement distance) p is twice the length h of the line segment LN. In FIG. 3, since the triangle PQM and the triangle MLN are similar, the length h of the line segment LN is
[0050]
[Equation 3]

It is. Further, since the aperture A of the photographing optical system O can be expressed by its focal length f and aperture stop value F, A = f / F.
[0051]
[Expression 4]

It can be expressed. In the relationship with the equation (1), since l ′ = xn−1 and p is the difference between xn and xn−1, the equation (4) is obtained from both sides of the equation on the left side of the equation (1). It can also be derived by subtracting xn-1 and replacing K with equation (2).
[0052]
The above equation (4) is obtained by using two CCDs having adjacent depths of field as shown in FIG. interval It can be seen as a general expression of p. In particular, equation (4) is a linear function of l ′, and it can be seen that p increases as l ′ increases, that is, the closer the subject distance is, the greater the value.
[0053]
Here, constant equidistance pitch regardless of subject distance so When using a CCD, interval Considering what value p should be, it was found using the condition that the subject is close, as is clear from equation (4), that is, a relatively large image distance l ′. interval p is necessary for the subject distance when the subject is farther than that. interval Since it becomes larger than p and the two depth-of-field ranges to be continuous become discontinuous, it can be seen that this is inappropriate. That is, to use a CCD with a constant equidistance pitch regardless of the subject distance, interval Must be the minimum possible value of p in equation (4).
[0054]
The expression (4) takes the minimum value when the image distance l ′, that is, the distance of the subject corresponding to the image point L is infinite. At this time, the image point L is at the rear focal point of the photographing optical system O. Therefore, by replacing l ′ in equation (4) with l ′ = f, the equidistant pitch pccd of the CCD to be employed in this optical system is
[0055]
[Equation 5]

It can be asked. In general camera design specifications, it is considered that the CCD having the longest distance (CCD 11 (F) in FIG. 1) is usually designed so as to be relatively movable to the back focal position, and the condition of l ′ = f It can be said that it is extremely reasonable to derive Equation (5) using
[0056]
Now, the CCD pitch pccd represented by the equation (5) is not limited to the focal length f and the open F value (further allowable confusion circle diameter δ, but the size of the CCD is changed, such as when the CCD is replaced). This value is usually a constant. Therefore, when either the focal length f or the open F value of the photographing optical system O is changed due to lens replacement or zooming, the distance between the CCDs 11 to 13 becomes the equidistant pitch pccd obtained by the equation (5). Move to.
[0057]
When δ = 0.02 (mm), Equations (5) at respective focal lengths f of 8.0 mm, 12.5 mm, and 20.0 mm and open F values of 1.4, 2.8, and 5.6 The calculation result (mm) of pccd of
[0058]
[Table 1]

become that way. Here, observing Table 1 above, it can be seen that the value of pccd greatly depends on the maximum aperture value F and the influence of the focal length f is small. This is because the constant term f / (f−δF) formed by the entire right side denominator and f of the numerator is always larger than 1 and very close to 1 when the equation (5) is observed. it is obvious. Therefore, in the approximate calculation, the CCD pitch pccd is
[0059]
[Formula 6]

However, it can be said that it is sufficient for practical use. Since the CCD pitch pccd in the equation (6) is very close to the value in the equation (5) and is always smaller than the value in the equation (5), there is a problem that the two depth of field ranges are discontinuous. Does not occur (see 2δF value in Table 1).
[0060]
That is, in the simplest design specification (Equal distance) The CCD pitch pccd may be determined by equation (6). In that case, the CCD pitch pccd can be determined only by the open F value of the photographing optical system O (for example, even if the lens is replaced, the lens opening before and after the replacement is opened). Since it is not necessary to change the CCD pitch pccd if the F values are the same, the hardware and software for control are simpler and less expensive. Equation (6) is the same as the so-called infinity depth of focus 2δF, and is determined only by the open F value and the allowable confusion circle diameter δ.
[0061]
Note that the pccd of the formula (5) is the CCD interval 3, the image distance (hereinafter referred to as lw ′) of the image point W in FIG. 3 can be obtained as the sum of the image distance l ′ of the image point L and pccd in Expression (5), and the image distances l ′ and lw ′, The subject distances l and lw corresponding to the image distances l ′ and lw ′ can be calculated using the relationship (1 / l) + (1 / l ′) = (1 / f). Incidentally, when f = 12.5 (mm), F = 1.4, and δ = 0.02 (mm) (CCD pitch pccd = 0.0561 mm), the difference in subject distance (l−lw), that is, two sheets The difference in object distance formed on the CCD is (l−lw) = 97.3 m at l = 100 (m), (l−lw) = 3201 mm at l = 5 (m), and l = 1 (m). When (l-lw) = 259 mm, l = 700 (mm), (l-lw) = 136 mm, and l = 500 (mm), (l-lw) = 72 mm. These numerical values are the imaging shown in FIG. It can be seen that it is almost consistent with the composition of the range.
[0062]
As described above, while the operating conditions of the imaging optical system (focal length f, allowable confusion circle diameter δ, imaging aperture F) do not change, the depth of field range formed by the imaging element is adjacent or slightly overlapped. CCD to satisfy (Formula (1) and Formula (2)) Arrangement interval Or by controlling the position of each image sensor so that the arrangement intervals of the image sensors are substantially equidistant (equation (5) or equation (6)). It can be seen that in the photographing to the intraoral photographing, it is possible to continuously cover all the depth of field with three CCDs.
[0063]
Note that the conditions according to the equations (1) and (2) are as follows: CCD on the far side (short-distance subject side) from the photographing optical system O Interval Becomes larger. That is, there is an advantage that a wider depth of field can be secured on the side farther from the photographing optical system O (short-distance subject side) than the equidistant arrangement according to the formula (5) or the formula (6). ) Or the equidistant arrangement according to the equation (6) has an advantage that the control of the CCD position becomes easier.
[0064]
<CCD arrangement interval Control>
If the lens settings (focal length, open aperture) change due to lens replacement or zooming, the CCD placement is based on Equations (1) and (2) or Equation (5) or Equation (6). interval Need to control.
[0065]
In the following, an example of control regarding what control should be performed according to the focal length of the photographing optical system O will be described.
[0066]
Here, the CCD placement is based on the condition of Equation (5) or Equation (6) (CCD equidistant placement). interval Is controlled at the same distance, but control similar to the following is performed on the long-distance side and the short-distance side CCD even when the conditions of equations (1) and (2) are used. Interval It can be similarly applied only by changing the amount of change.
[0067]
If somewhat strict control is performed, the CCD arrangement pitch based on the formula (5) is controlled, that is, in the case of the lens shown in Table 1 above, based on the value in the rightmost column. Control the CCD placement pitch. That is, in the control based on Expression (6), when either the focal length f or the open aperture value F is changed among the lens setting conditions, the CCD arrangement pitch is controlled accordingly. . According to the control based on the equation (5), there is an advantage that a slightly wider depth of field range can be covered by three CCDs as compared with the control using only an aperture value described later. ) Illustrates simpler control using only the aperture value based on ().
[0068]
In FIG. 1, since the lens detection means 28 is provided so that the CPU 30 of FIG. 1 can detect a change in the focal length of the photographing optical system O due to lens replacement or zooming, the CPU 30 is based on the output of the lens detection means 28. The position of the CCD can be controlled.
[0069]
Figure 1 shows the CCD Equally spaced The simplest configuration for performing the control is shown. That is, in FIG. 1, the CCD 12 (M) is fixed, and the CCD 11 (F) and the CCD 13 (N) can be brought close to or separated from the CCD 12 (M) with the CCD 12 (M) as a reference.
[0070]
The positions of the CCDs 11 to 13 in FIG. 1 indicate the case where the photographing optical system O has a large aperture stop value (L) (“dark” lens, for example, F = 4.0, F = 5.6, etc.). At this time, the image plane distance (the above-mentioned CCD arrangement pitch d) of the CCDs 11 to 12 is dmfl, and the image plane distance (the above-mentioned CCD arrangement pitch d) of the CCDs 12 to 13 is dmnl.
[0071]
When the photographing optical system O has a medium aperture stop value (M) (“dark” lens. For example, F = 1.4 to F = 2.0), the image plane distance of the CCDs 11 to 12 (described above). The CCD arrangement pitch d) may be adjusted to a shorter dmfm, and the image plane distance of the CCDs 12 to 13 (the above-mentioned CCD arrangement pitch d) may be adjusted to a shorter dmnm.
[0072]
Further, the case where the photographing optical system O has a small aperture stop value (S) (“bright” lens. For example, F = 1.0 to F = 1.4) is shown, and the CCDs 11 to 12 at this time are shown. The image plane distance (the above-described CCD arrangement pitch d) may be adjusted to a shorter dmfs, and the image plane distances of the CCDs 12 to 13 (the above-mentioned CCD arrangement pitch d) may be adjusted to a further shorter dmns.
[0073]
For such control of the CCD interval, the CCD 11 (F) is moved by the CCD position control means 21 and the CCD 13 (N) is moved by the CCD position control means 23, and the CCD 12 (M) is kept at a fixed position. Thus, when the CCD 12 (M) is placed at a fixed position, the CCD position control means 22 (broken line) for the CCD 12 (M) is naturally not necessary. The CCD position control means 21 and 23 are configured using driving means such as a solenoid or a motor that operates to form the illustrated CCD intervals dmfl, dmnl, dmfm, dmnm, dmfs, and dmns in accordance with a command from the CPU 30. (Or if sufficient mechanical accuracy can be ensured, the CCD position control means 21 and 23 can be configured to operate using only the user's operating force as a drive source via a purely mechanical mechanism. ).
[0074]
As described above, according to the present embodiment, it is possible to provide an image processing apparatus capable of performing practically sufficient NMF image synthesis without performing CCD movement control as much as possible. In other words, according to the present embodiment, in the photographing optical system O having a fixed focal length, the CCD movement control is unnecessary, and it is sufficient to perform only when the focal length is changed by exchanging lenses or zooming, and is close to the actual landscape photographing. In distance photography, a continuous depth of field with no practical problems can be obtained, and 11, 12, and 13 obtained from CCDs 11 to 13 under good conditions (F: Far), medium distance (M: (Middle) and short distance (N: Near) can be synthesized, and an extremely high quality image can be output.
[0075]
In the present embodiment, even when the movement control of the CCD is performed, it is particularly easy to satisfy the condition that the depth-of-field range formed by each image sensor is adjacent or slightly overlapped according to lens exchange or zooming. In such a specification, it is sufficient to move the CCD to positions at equal intervals, so that the control of the CPU 30 becomes extremely simple. Alternatively, it is extremely easy to control the movement of the CCD only by a mechanical mechanism that is linked to lens exchange or zooming without using a control means such as the CPU 30, and the hardware necessary for the movement of the CCD. The cost of hardware or software is very small.
[0076]
Further, in the configuration of FIG. 1, one of the three CCDs is fixed, and only the other two are moved. This is not necessary due to a very simple and inexpensive configuration, and the necessary CCD movement according to lens exchange or zooming. Control can be performed.
[0077]
For ease of explanation, FIG. 1 shows a configuration in which the CCD (M) 12 is fixed and the CCD (F) 11 and the CCD (N) 13 are moved. Which CCD is fixed? Is optional. For example, in a normal lens configuration, the photographing optical system O moves forward toward the subject on the short distance side, moves backward toward the CCD on the long distance side, and stops at a predetermined infinity position. 11 is fixed and the CCD (M) 12 and the CCD (N) 13 are moved, the distance ring (focusing means 27) can be designed easily for each lens having a different focal length. ,it is conceivable that.
[0078]
<Configuration for CCD movement>
In the above, as a particularly simple configuration, a plurality of CCDs 11 to 13 are arranged at equal distances suitable for specific operating conditions of the photographing optical system, and according to changes in the operating conditions of the photographing optical system, etc. A configuration has been shown in which each CCD is moved to a different position while maintaining the mutual distance.
[0079]
In the following, an example of a more specific mechanism suitable for a configuration in which the plurality of CCDs are moved while maintaining an equal distance from each other in accordance with a change in operating conditions of the photographing optical system will be described.
[0080]
FIG. 4 shows the configuration of an NMF image synthesis system having CCDs 11 ′ (F), 12 (M), and 13 (N). In FIG. 4, the same reference numerals are used for members that are the same as or correspond to those in FIG. 1, and the description thereof is the same as that in FIG.
[0081]
FIG. 4 is different from FIG. 1 in that the reflection direction of the half mirror M1 is reversed so as to be perpendicular to the reflection direction of the half mirror M2 (or mirror 3), and the CCD (F) 11 ′ is taken as a photographing optical system. It is that it is arranged on the side opposite to the CCD (M) 12 and the CCD (N) 13 with respect to O. In FIG. 4, in order to clarify the difference from FIG. 1, the arrangement of the CCD (F) 11 in FIG. 1 is indicated by a broken line.
[0082]
The CCD (M) 12 is arranged at a fixed position with respect to the photographing optical system O as in FIG.
[0083]
Further, the CCD (F) 11 ′ is mechanically coupled integrally with the CCD (N) 13 and the coupling member 41. The coupling member 41 is a member such as a single plate of metal (or an arbitrary material such as plastic), for example, and the CCD (F) 11 ′ and the CCD (N) 13 are rigidly fixed on the coupling member 41. It is.
[0084]
The CCD (F) 11 ′ and the CCD (N) 13 are connected to the S (small aperture) or L (large aperture) by the CCD position control means 23 according to the change of the operating condition of the photographing optical system via the coupling member 41. Moved to the side. In this case, the moving direction (S / L) of the coupling member 41 is parallel to the optical axis between the half mirrors M1 to CCD (F) 11 ′.
[0085]
Naturally, in such a configuration, the CCD (F) 11 ′ and the CCD (N) 13 are mechanically coupled together, so that the CCD position control means 21 required in the configuration of FIG. When changing the operating conditions of the photographing optical system, it is possible to control to form the above-mentioned CCD intervals dmfl, dmnl, dmfm, dmnm, dmfs, and dmns using only the CCD position control means 23.
[0086]
That is, in FIG. 4, when the CCD (F) 11 ′ and the CCD (N) 13 are moved in the S (small aperture) direction, the movement direction by the coupling member 41 is the same. On the axis, both the CCD (F) 11 ′ and the CCD (N) 13 move so as to approach the CCD (M) 12. Of course, the moving distances of the CCD (F) 11 ′ and the CCD (N) 13 at this time are the same. Conversely, when the CCD (F) 11 ′ and the CCD (N) 13 are moved in the L (large aperture) direction, the CCD (F) 11 ′ and the CCD (N) 13 are both CCD (M) on the optical axis. Move away from twelve equidistant distances.
[0087]
By adopting the configuration as shown in FIG. 4, it can be easily moved while keeping the same distance from each other according to the change of the operating condition of the photographing optical system. The configuration of FIG. 4 has the advantage that only very simple and inexpensive hardware using only a single drive means is required to move the two CCDs.
[0088]
<Configuration example using prism>
A more specific configuration example based on FIG. 4 is shown in FIG. The configuration of FIG. 5 utilizes a prism used in a color television camera or the like.
[0089]
The prism 51 in FIG. 5 has reflective interfaces 52, 53, and 54 that are partially transmissive, and photographic light incident from a photographic optical system O (not shown in FIG. 5) on the left side of the drawing along the main optical axis (M). Are emitted in the directions of F, M, and N, and enter the CCDs 11 ′ (F), 12 (M), and 13 (N) (the same as the CCDs in FIG. 4). The transmission / reflectances of the reflective interfaces 52, 53, and 54 are determined so that the intensity of the emitted light with respect to each of the CCDs 11 ′ (F), 12 (M), and 13 (N) is equal.
[0090]
In order to perform spectrum for color photographing with the prism 51, a color filter is disposed between the CCD and the CCD, but this is not necessary in the NMF image composition system as in the present invention.
[0091]
This type of prism has applications in color television cameras and the like, and can be obtained relatively easily, and is suitable as an optical path splitting means necessary for an NMF image synthesis system, but controls the intensity of emitted light. For the purpose of, for example, the angle of each exit surface is a special angle as shown in the figure. For example, in the illustrated configuration, the emission direction F is 47 ° with respect to the main optical axis (M), and the emission direction N is 70 ° with respect to the main optical axis (M).
[0092]
However, even in such a configuration, CCD position control equivalent to that in FIG. 4 can be performed. That is, also in FIG. 5, the CCD (F) 11 ′ and the CCD (N) 13 are mechanically coupled together using the coupling member 41, and along the straight line R by the CCD position control means 23 (not shown). It is moved to the S (small aperture) or L (large aperture) side.
[0093]
This straight line R is taken to intersect 90 degrees with a straight line B that bisects the emission direction F and the emission direction N. It is moved along the straight line R to the S (small aperture) or L (large aperture) side. In the configuration in which the CCD (F) 11 ′ and the CCD (N) 13 are moved, the angle between the emission directions F and N to the CCD (F) 11 ′ and the CCD (N) 13 and the straight line R is 31.5 ° ( 90 ° −58.5 ° = (70 ° + 47 °) / 2), and the movement amount of the CCD (F) 11 ′ and the CCD (N) 13 on the straight lines F and N when the connecting member 41 is moved is completely zero. It can be controlled to be equal.
[0094]
As described above, the same configuration as that of FIG. 4 can be achieved in the same manner by simply selecting the arrangement direction of the members even when the optical path dividing means using the prism having a special emission direction as shown in FIG. 5 is used. The configuration of FIG. 5 uses a spectral prism that is relatively easy to obtain for color imaging applications, and has the advantage that an NMF imaging system can be realized without using a custom-made expensive configuration.
[0095]
【The invention's effect】
As is clear from the above description, according to the present invention, an image formed by a single photographing optical system is photographed. Three The image sensor, Photographed by each image sensor In an image processing apparatus for obtaining an image obtained by combining image data, an optical path dividing unit for introducing photographing light from a photographing optical system to each of the imaging elements is provided, When a change occurs in the value of the open aperture of the photographing optical system or the value of the focal length of the photographing optical system, Above Three The position of a specific one of the image sensors is fixed, and the remaining image sensors Two When the two image pickup devices mechanically connected are moved in the same direction, the image pickup devices are moved to different positions while keeping the same distance from each other. Since the configuration for determining the optical path splitting direction of the optical path splitting means is adopted, practically sufficient NMF image synthesis can be achieved by performing extremely simple and inexpensive hardware and control mechanism or control software without performing movement control of the image sensor as much as possible. In particular, it is possible to provide a specific operating condition of the imaging optical system simply by moving two image sensors mechanically coupled using a single driving means in the same direction. An excellent effect is obtained that it is possible to achieve the arrangement of the image pickup elements at suitable equidistant mutual intervals.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a main part of an NMF image synthesis system employing the present invention.
FIG. 2 is an explanatory diagram showing the principle of the image composition system of the present invention.
FIG. 3 is an explanatory diagram showing CCD arrangement conditions in the image composition system of the present invention.
FIG. 4 is a block diagram showing a configuration of a main part of a different NMF image synthesis system employing the present invention.
FIG. 5 is an explanatory diagram showing CCD movement control in a configuration in which an optical path is split using a three-color spectroscopic prism.
[Explanation of symbols]
11-13 CCD
14 Video I / F
21, 22, 23 CCD position control means
27 Focusing means
28 Lens detection means
30 CPU
31 Operation unit
32 program memory
33 Network I / F
35 memory
41 coupling member
51 prism
M1, M2 half mirror
M3 mirror

Claims (2)

In an image processing apparatus that has three imaging elements that capture an image formed by a single imaging optical system and obtains an image obtained by combining image data captured by each imaging element ,
While providing optical path dividing means for introducing photographing light from the photographing optical system to each of the image sensors,
When a change occurs in the value of the open aperture of the photographing optical system or the value of the focal length of the photographing optical system , one specific position among the three image sensors is fixed, and the remaining image sensors When two are mechanically coupled and the two mechanically coupled image sensors are moved in the same direction,
2. An image processing apparatus according to claim 1, wherein an optical path dividing direction of the optical path dividing means is determined so that the image pickup elements are moved to different positions while maintaining an equal distance between the image pickup elements. Has a detection means for detecting a change of the photographing optical system exchange, or the photographing open aperture value of the optical system caused by zooming or the value of the focal length of the photographing optical system, said imaging optical by said detecting means 2. The image pickup device according to claim 1, wherein when the change in the value of the open aperture of the system or the change in the value of the focal length of the photographing optical system is detected, the respective image sensors are moved to mutually equidistant positions. The image processing apparatus described.

Images