JP3727704B2 - Multi-directional image input device - Google Patents

Description

【００６１】
従って、数種類の角度方向φに対してピーク座標を検出し、それらをサインカーブで回帰近似することにより、ｄを推定することができる。なお、対物レンズ５，６のＮ．Ａ．によりピーク検出が可能な角度方向φは、次式が満足される範囲に限られるので、その範囲内でいくつかの角度方向φを選択すれば良い。
【００６２】
【数２】

【００６３】
本実施形態によれば、原点を検出する際に２Ｄ投影画像からピーク座標を検出するので、３Ｄ観測像から直接ピーク座標を検出するよりもピーク検出が簡便に行なえる。また複数の投影画像から検出されたピーク値から原点を推測するため、検出誤差をキャンセルでき、最も確からしい原点の位置を推測することが可能になる。
【００６４】
本実施形態は図７と同様な構成で以下に示すように動作させても良い。まず光軸と平行な方向への２Ｄ投影画像からＸ−Ｙ平面内の原点を検出する。次に、メモリ５０３または５０４に記録されている３Ｄ観測像から検出されたＸ−Ｙ原点を通り、かつ回転面に平行な画像を読みだす。読み出された２Ｄ画像は所定の角度φだけ回転された後にＺ方向に累積加算され、１Ｄ平行投影像が作成される。ピーク検出はこの１Ｄプロフィールについて行なわれる。以上のように構成すれば、ピーク検出は１Ｄ上の検索になるためより簡便に行なえ、しかも精度良く原点が検出できる。
【００６５】
次に図９には、本発明による第４の実施形態としての多方向画像入力装置の構成例を示し説明する。
【００６６】
この多方向画像入力装置は、多光軸画像入力系において微小試料に対する各方向からの入力画像間の相関を利用することにより、各３Ｄ観測像空間どうしの相対位置関係を求める装置である。
【００６７】
図９に示す本実施形態のプロセッサ６００以外の他の構成は、図１に示した第１実施形態と同様に構成されているものとする。即ち、顕微鏡１，２により合焦点面を移動されながら入力された微小試料の画像は、プロセッサ６０内のＡ／Ｄ変換器６１，６２によりディジタル画像信号に変換され、メモリ６３，６４に記録される。
【００６８】
次にＦＦＴが実行されるように構成されたＤＳＰ６５において、メモリ６３，６４の各々には、記録された合焦面の異なる画像から構成される３Ｄ観測像に対して３Ｄフーリエ変換が行なわれ、再びメモリ６３，６４に記録される。続いて積和演算器６６において、前記メモリ６３，６４に記録された各３Ｄフーリエ像を用いて、所定の積和演算が行なわれ、その結果は、メモリ６４に記録される。メモリ６４に再記録された３Ｄフーリエ像に対してＤＳＰ６５では、３Ｄ逆フーリエ変換が実行され、その結果は、メモリ６３に記録される。
【００６９】
以上の構成動作により、各３Ｄ観測像間の３Ｄ相関が計算される。ピーク検出器６７では、メモリ６３に記録された３Ｄ相関像からピーク座標が検出され、座標値はメモリ６８に記録される。メモリ６８に記録された３Ｄ相関ピーク座標は各入力光学系による観測像間の相対位置関係として観測モード時に利用される。またプロセッサ６０内には第１の実施形態のコントローラ１７，１８，１９に相当するコントローラ６９，７０，７１が設けられる。またプロセッサ６０全体の制御はＣＰＵ７２で行なわれ、各構成要素間の信号伝達は内部バス７３を介して行なわれる。
【００７０】
この様に構成された第４の実施形態の作用について説明する。本実施形態は、次式で定義される３Ｄ観測像間の相関を計算する作用を有する。
【００７１】
【数３】

【００７２】
ただし、ｇ1 (r) ，ｇ2 (r) は各々多光軸画像入力系における第１、第２の３Ｄ観測像である。またベクトルｒ，ｔは共に３Ｄ直交座標空間で定義される座標ベクトルである。微小試料を用いる場合、（３）式は以下のように展開できる。
【００７３】
【数４】

【００７４】
ただし、ｈ1 (r) ，ｈ2 (r) は各々多光軸画像入力系における第１、第２の３Ｄレスポンス関数、δ(r) は微小試料を表すデルタ関数、ｒs は第１、第２の３Ｄ観測像間の相対的位置ずれを表す３Ｄベクトル、および* は３Ｄコンボリューションのオペレータである。（４）式を計算すると３Ｄ相関像ｓ(t) において点ｒs の位置にピークが出現するので、これを検出することにより相対的位置関係が分かる。本実施形態では３Ｄ相関を求める際に、以下に示すように３Ｄフーリエ変換を用いた計算を行なっている。（３）式は次式のように書き換えることができる。
【００７５】
【数５】

【００７６】
ただし、Ｆ_３Ｄ｛｝は３Ｄフーリエ変換のオペレータ、Ｆ^＊ _{３ Ｄ}｛｝は３Ｄフーリエ変換および複素共約のオペレータ、Ｆ_３Ｄ ^−１｛｝は３Ｄ逆フーリエ変換のオペレータである。３Ｄ観測像ｇ１ (r) ，ｇ２ (r) は３Ｄフーリエ変換像Ｇ１ (u) ，Ｇ２ (u) （但し、ｕは３Ｄ直交空間の空間周波数座標）を以下のように書き改める。
【００７７】
【数６】

【００７８】
この場合、３Ｄフーリエ空間で行なう演算は次式で表される。
【００７９】
【数７】

【００８０】
前記積和演算器６６では、２つの３Ｄフーリエ像に対して（７）式に基づく演算を実行する。このようにして算出された２つの３Ｄ観測像の相関像より、ピーク座標を検出することにより相対的位置関係を求めることができる。
【００８１】
ところで、多光軸光学結像系の各光学系のＮ．Ａ．が非常に大きい場合でなければ、各３Ｄ観測像間の相関は弱く、任意の試料に対して常に精度良く相関ピークを検出するのは難しい。
【００８２】
ところが本実施形態によれば、理想的な点物体として扱える微小試料を用いるために光学結像系本来の結像特性を表す３Ｄ観測像を得ることができ、最も理想的な３Ｄ相関像を求めることができる。つまり、各光学結像系の間にわずかでも相関性があれば、微小試料の３Ｄ観測像を用いて相関ピークを検出することが可能になる。
【００８３】
よって、相関法によれば第１〜第３の実施形態に示したような各３Ｄ観測空間に共通な座標を求めるような操作は必要なく、３Ｄ観測像間の相対的位置関係が直接求められることになるため、記録すべき情報が少なくて済むと同時に位置合わせの処理が簡便になる。
【００８４】
次に図１０には、本発明による第５の実施形態としての多方向画像入力装置の構成例を示し説明する。
【００８５】
この多方向画像入力装置は、蛍光色素溶液に対物レンズで絞ったレーザ光を照射することにより、試料空間に微小試料を設置したのと同様の効果をもたらすように構成したものである。
【００８６】
この多方向画像入力装置において、顕微鏡８１，８２はそれぞれ対物レンズ８３，８４を有し、第１の実施形態と同様に、試料空間で光軸が直交するように配置され、顕微鏡８１，８２も同様に合焦面制御器やＴＶカメラも設けられるがここでは図示を省略する。
【００８７】
試料空間にはキャリブレーションモード時に容器８５が設置され、中には蛍光色素溶液８６が注入される。またレーザ光学系が以下のように構成される。
【００８８】
レーザ８７から発せられるレーザ光は、レーザビームエキスパンダ８８により所望のビームサイズに変換され、ミラー８９により顕微鏡８１内に導入される。顕微鏡８１内にはミラー９０が設置され、対物レンズ８３の結像光学系を利用してレーザ光が容器８５内で収斂されるように構成されている。顕微鏡８１には図示しない合焦面制御器が設けられているが、顕微鏡８１内にレーザ光が導かれる際は、その合焦面制御器により顕微鏡８１の合焦面は３Ｄ観測画像空間上で定義される光軸方向の座標の中央に設定される。この時、顕微鏡８２では合焦面を移動させながら複数の画像が入力され、プロセッサ８０に転送される。プロセッサ８０は第１の実施形態のプロセッサ４と同様に構成され、顕微鏡８２を利用して入力された３Ｄ観測像からピーク座標が検出される。
【００８９】
なお、観測モード時では容器８５の代りに試料が溶液と共に注入された容器８５と同じ仕様の別の容器が設置され、内部の試料が観測される。また、対物レンズ８３，８４は水浸レンズで構成され、両対物レンズと容器８５とを全て液体中に浸すような新たな容器を設けるように構成しても良い。その場合、容器８５は周囲の液体と屈折率が近い物質で構成される。
【００９０】
次に、本実施形態の作用について説明する。
【００９１】
本実施形態では、顕微鏡８１の光学結像系利用して容器８５内に作成したレーザ光スポットにより蛍光色素溶液８６を励起することにより、微小試料を試料空間内に設置したのと同様の作用をもたらす。レーザ光は空間的コヒーレンスに優れているため、理想的なスポット光を作成することができる。レーザ光スポットは顕微鏡８１の３Ｄ観測空間上の原点に対応する位置に出現するため、その位置を顕微鏡８２の３Ｄ観測像から検出することにより、両光学結像系の相対位置関係を知ることができる。
【００９２】
従って、本実施形態によれば、容器８５を試料空間に設置するという簡単な方法により試料空間に微小試料を設置したのと同様の効果を得ることかできるため、装置構成や操作が簡便になる。
【００９３】
次に第６の実施形態の多方向画像入力装置について説明する。
【００９４】
この第６の実施形態は、蛍光色素溶液に対物レンズで絞ったレーザ光を照射するように構成されるが、２光子励起現象を利用するため、赤外域の強パルス光を発生させる機種が用いられる。レーザパルス光が連続的に照射される間、顕微鏡８２では、十分なコントラストの画像が得られるまでＴＶカメラのシャッタは解放され、その後に画像信号がプロセッサ８０に転送される。
【００９５】
この第６の実施形態の作用について説明する。
【００９６】
本実施形態は、レーザ光源に蛍光物質の励起波長の２倍の波長のパルスレーザ光を用いることにより、蛍光色素の２光子励起現象を利用するものである。２光子励起による蛍光は通常極めて観測しにくいが、強レーザパルス光を収斂させることにより作成された高エネルギーな局所空間においては観測が可能な蛍光が発生されるようになる。そのような２光子励起蛍光が発生されるのは極めて微小な空間に限られるため、光学結像系の焦点深度より遥かに狭い範囲で発光する理想的な微小点光源が得られることになる。
【００９７】
従って、本実施形態によれば、理想的な微小蛍光点を観測できるため、精度に優れた位置合わせが行なえる。
【００９８】
次に図１１には、本発明による第７の実施形態としての多方向画像入力装置の構成例を示し説明する。
【００９９】
この多方向画像入力装置は、観測すべき試料と位置合わせに用いる微小試料を混在させた溶液を用いるように構成したものである。
【０１００】
この多方向画像入力装置は、顕微鏡９１，９２は、前述した第１の実施形態と同様に試料空間で光軸が直交するように配置される。なお、顕微鏡９１，９２には、第１の実施形態と同様に、対物レンズ、合焦面制御器、ＴＶカメラが設けられるが、図示を省略する。
【０１０１】
まず、試料空間には容器９３が設置され、中には適当な密度で微小蛍光球９４と共に観測すべき試料９５が注入されている。なお、微小蛍光球９４と試料９５とは、異なる波長領域に蛍光スペクトルを有する２種類の蛍光色素でそれぞれ染色されている。また顕微鏡９１の光路上には、透過波長領域の異なる２種類の色フィルタ９６，９７が設置され、必要に応じてどちらかが使用される。これらの色フィルタ９６，９７の切り替えは、フィルタ変更部９８により行なわれる。
【０１０２】
このような構成においては、キャリブレーションモード時に、微小蛍光球９４の蛍光波長を選択的に透過させる色フィルタ９６が光路上に設置され、顕微鏡９１により微小蛍光球９４の画像が入力される。また観測モード時には、試料９５の蛍光波長を選択的に透過させる色フィルタ９７が光路上に設置され、試料９５が観測される。
【０１０３】
以上のような構成は図１１では図示は省略するが、顕微鏡９２においても同様に設けられている。各顕微鏡９１，９２で入力された微小蛍光球９４の画像はプロセッサ９９に入力され、第１の実施形態と同様な処理が行なわれることによりピーク座標が検出される。ただし、本実施形態では画像内に撮像されている複数の微小蛍光球９４の画像の各々についてピーク座標を検出し、それらの位置座標を全て共通のアドレスで認識できるような３Ｄ観測像空間が定義される。なお、プロセッサ９９内には内部バス１００上のフィルタ変更部９８を制御するためのコントローラ１０１が設けられているが、それ以外の構成は第１の実施形態のプロセッサ９９と同様に構成される。
【０１０４】
次に、本実施形態の作用について説明する。
【０１０５】
本実施形態は、キャリブレーションモードと観測モードとで試料空間上での構成を変更させることなく、色フィルタ９６，９７を変更することでキャリブレーションモード時に微小蛍光球９４の画像が選択的に入力されるように構成されたものである。
【０１０６】
従って、本実施形態によれば、試料空間における構成や操作をより簡便にすることができる。またキャリブレーションモードと観測モードとの時間差を短くすることができるため、より正確な位置合わせが実現できる。また複数の微小蛍光球９４についての位置合わせを利用できるため、精度を向上できるという効果もある。
【０１０７】
以上の実施形態について説明したが、本明細書には以下のような発明も含まれている。
【０１０８】
（１） 異なる方向から観察するための空間における画像入力の範囲内に対象物を結像するように配置された複数の光学系と、この複数の光学系の各々の像面に配置された複数の撮像素子とを有する多方向画像入力装置において、 画像入力時に対象物が存在する前記画像入力の範囲内に配置し、観察する際の光学的な基準設定が行える形状の標準試料と、
前記複数の光学系の物体空間における焦点の合った物体面の位置（合焦位置）を移動させる合焦位置制御手段と、
この合焦位置制御手段により前記光学系の合焦位置を所定の物体空間に設定した状態で前記撮像素子が取得した前記標準試料の像情報及び、該像情報を取得する際の合焦位置の情報を対応させて記録する標準試料像記録手段と、
前記合焦位置制御手段により光学系の合焦位置を移動させながら前記標準試料記録手段に記録された異なる合焦位置に対応する複数の標準試料の像より構成される標準試料観測像空間から、前記複数の光学系の各々に対応するこの標準試料観測像空間どうしの位置関係を検出する位置関係検出手段と、
を具備することを特徴とする多方向画像入力装置。
【０１０９】
この発明に関する実施の形態は、第１、３、４の実施形態が対応する。
【０１１０】
本発明によれば、観察する際に光学的な特性が得られる形状の標準試料の像を入力することにより、光学結像系の空間特性を直接反映した像を推定することが可能になり、多方向画像入力装置における各入力画像間の位置合わせに必要な像情報を得ることができる。従って、本発明に多方向画像入力装置における各光学結像系の開口が小さいために各入力画像間の相関がとりにくい場合にも適用できると共に、対象画像の空間周波数特性にも依存しない、適用範囲が広くしかも簡便で実用上有用な位置合わせ方法を提供することができる。
【０１１１】
（２） 前記（１）項に記載の多方向画像入力装置において、
前記位置関係検出手段は、
複数の光学系の各々に対応する標準試料観測像空間に対して、観測した標準試料の像の位置を原点とする絶対番地座標により、標準試料観測像空間どうしの位置関係を検出することを特徴とする。
【０１１２】
（３） 前記（２）項に記載の多方向画像入力装置において、
前記位置関係検出手段は、
複数の光学系の各々に対応する標準試料観測像空間から、標準試料に対して、最も焦点の合った位置をピーク位置として検出する機能を有することを特徴とする。
【０１１３】
（４） 前記（３）項に記載の多方向画像入力装置において、
前記位置関係検出手段は、前記複数の光学系の各々に対し、
前記標準試料像記録手段に記録された異なる合焦位置に対応する複数の標準試料の像情報を加算する像加算手段と、
加算された像情報から画素濃度値のピ―ク位置を検出する第１のピーク位置検出手段と、
この第１のピーク位置検出手段により検出されたピーク位置における異なる合焦位置に対応する複数の標準試料の像の画素濃度値から、ピーク位置を検出する第２のピーク位置検出手段とを有し、
前記複数の光学系の配置位置と、前記第２のピーク検出手段が求めた各光学系に対応するピーク位置との相対的な位置関係を求め、前記第１及び第２のピーク位置検出手段が求めた各光学系に対するピーク位置が同じ位置にあると判断することによって、前記複数の光学系の各々に対応する前記標準試料観測像空間どうしの位置関係を求めることを特徴とする多方向画像入力装置。
【０１１４】
（５） 前記（４）項に記載の多方向画像入力装置において、
前記位置関係検出手段内の第１のピーク位置検出手段と第２のピーク位置検出手段は、同一に構成し使用する、若しくは１つのピーク位置検出手段を切り換えて使用することを特徴とする。
【０１１５】
（６） 前記（３）項に記載の多方向画像入力装置において、
前記位置関係検出手段は、
前記標準試料像情報記録手段に記録された異なる合焦位置に対応する複数の標準試料の像を用いて、光軸方向に対して所定の角度方向に平行投影像を生成する平行投影手段と、
平行投影像からピーク位置を検出するピーク位置検出手段と、
ピーク位置を利用して標準試料の位置を推定する位置推定手段と、
を具備することを特徴とする。
【０１１６】
この発明に関する実施の形態は、第１実施形態が対応する。
【０１１７】
本発明によれば、標準試料の像より多方向画像入力装置における各入力画像間の位置合わせに有効な指標となる像情報（例えばピーク座標）を検出することにより、複数の光学系の各々に対応する標準試料観測像空間の座標を容易に設定することができる。そして、そのような操作を同一の標準試料像に対して複数の画像入力系の各々で行なうことにより、各標準試料観測像空間について求めた座標を共通の座標として扱うことが可能になり、異方向からの入力画像間の位置合わせが実現される。
【０１１８】
（７） 前記（１）項に記載の多方向画像入力装置において、
前記位置関係検出手段は、
複数の光学系の各々に対する標準試料観測像空間どうしの相関を検出することにより、標準試料観測像空間どうしの位置関係を検出することを特徴とする。
【０１１９】
（８） 前記（７）項に記載の多方向画像入力装置において、
前記位置関係検出手段は、
複数の光学系の各々に対する標準試料観測像空間に対してフーリエ変換を行なうフーリエ変換手段と、
フーリエ変換された複数の光学系の各々に対する標準試料観測像空間から、フーリエ空間で定義される相互相関演算を行なう相互相関演算手段と、
相互相関演算手段により算出されたフーリエ空間における相互相関像を逆フーリエ変換する逆フーリエ変換手段と、
逆フーリエ変換された相互相関像からピーク位置を検出するピーク位置検出手段と、
を具備することを特徴とする。
【０１２０】
（９） 前記（８）項に記載の多方向画像入力装置において、
前記位置関係検出手段内のフーリエ変換手段と逆フーリエ変換手段は、同一の手段により構成されることを特徴とする。
【０１２１】
この発明に関する実施の形態は、第４実施形態が対応する。
【０１２２】
本発明によれば、あらかじめ形状の分かっている標準試料の像を入力することにより、光学結像系の空間特性を直接反映した像を推定することが可能になり、最も理想的な標準試料観測像空間どうしの相関像を求めることができる。相関法によれば、標準試料観測像間の相対的位置関係が直接求められることになるため、記録すべき情報が少なくて済むと同時に位置合わせの処理が簡便になる。
【０１２３】
（１０） 前記（１）項に記載の多方向画像入力装置において、
前記標準試料は、前記標準試料像記録手段に記録された像の１画素に相当する物体空間の大きさより小さい径の物体で構成されることを特徴とする。
【０１２４】
この発明に関する実施の形態は、第１実施形態が対応する。
【０１２５】
本発明によれば、標準試料に入力画像の１画素に相当する大きさよりも小さい微小物体を用いることにより、光学結像系の空間特性を直接反映した像を入力することができる。従って、本発明に関わる処理を行なうために最も理想的な入力像を得ることができる。
【０１２６】
（１１） 前記（１）項に記載の多方向画像入力装置において、
前記標準試料は、観察の際の該多方向画像入力装置の光学系の特性が得られる断面形状であることを特徴とする。
【０１２７】
（１２） 前記（１）項に記載の多方向画像入力装置において、
前記標準試料は、２つの光軸を含む平面又はそれぞれの光軸を含む平面の内の標準試料の断面が円形であり、球、円錐、円柱の物体であることを特徴とする。
この発明に関する実施の形態は、第１実施形態が対応する。
【０１２８】
本発明によれば、球形の物体は３Ｄ空間の全方向に対して十分な帯域を有すると考えられるので、微小試料を用いた場合の入力画像と相似で大きさが拡大された画像が観測されることになり、標準試料としての作用効果が得られる。加えて試料は必ずしも微小である必要はないので、取り扱いが簡便になり、実用上有利である。
【０１２９】
（１３） 前記（１）項に記載の多方向画像入力装置において、
標準試料は、先鋭な物体の先端であって、この物体の径が標準試料像記録手段に記録された像の１画素に相当する物体空間の大きさより小さい範囲である
ことを特徴とする。
【０１３０】
この発明に関する実施の形態は、第１実施形態が対応する。
【０１３１】
本発明によれば、標準試料として最も理想的な微小物体を用いるために先鋭な物体の先端を利用するものであり、標準試料自体を微小にする必要がないにも関わらず、微小物体を用いたのと同等の作用効果を得ることができる。従って、取り扱いが簡便であると共に最も理想的な標準試料像を入力できる。
【０１３２】
（１４） 異なる方向から観察するための空間における画像入力の範囲内に対象物を結像するように配置された複数の光学系と、この複数の光学系の各々の像面に配置された複数の撮像素子とを有する多方向画像入力装置において、 画像入力時に対象物が存在する前記画像入力の範囲内に配置し、観察する際の光学的な基準設定が行える形状の標準試料と、
前記複数の光学系の各々に設けられ、物体空間における焦点の合った物体面の位置（合焦位置）を移動させる合焦位置制御手段と、
この合焦位置制御手段により前記光学系の合焦位置を所定の物体空間に設定した状態で前記撮像素子が取得した前記標準試料の像情報及び、該像情報を取得する際の合焦位置の情報を対応させて記録する標準試料像記録手段と、
前記合焦位置制御手段により光学系の合焦位置を移動させながら、前記標準試料像記録手段に記録された異なる合焦位置に対応する複数の標準試料の像より構成される標準試料観測像空間から、前記複数の光学系の各々に対応するこの標準試料観測像空間同志の位置関係を検出する位置関係検出手段と、
前記位置関係検出手段により検出された複数の光学系の各々に対応する標準試料観測像空間どうしの位置関係に基づき、複数の光学系の各々の空間位置を制御する光学系位置制御手段と、
を具備することを特徴とする多方向画像入力装置。
【０１３３】
この発明に関する実施の形態は、第２実施形態が対応する。
【０１３４】
本発明は、位置関係検出手段により検出された複数の光学系の各々に対応する標準試料観測像空間どうしの位置関係に基づき、標準試料観測像空間において原点が中心に一致するように複数の光学系の各々の空間位置を微調整するように構成されたものである。
【０１３５】
従って本発明によれば、各顕微鏡からの入力画像を相互利用する際に最も大きな画角サイズを利用できる。つまり、各顕微鏡からの入力画像を位置合わせする際に入力画像のアドレスをシフトさせる必要が無くなり、再構成画像が入力画像よりも小さくなることが無い。
【０１３６】
（１５） 異なる方向から対象物を結像するように配置された複数の光学系と、この複数の光学系の各々の像面に配置された複数の撮像素子とを具備する多方向画像入力装置において、
画像入力時に対象物が存在する略位置に配置される蛍光物質の入った容器と、
この容器内の蛍光物質を励起する光を発生させる光源手段と、
複数の光学系の内の一つである第１の光学系を利用して、光源手段により発せられた励起光を容器内で収斂させるように励起光を第１の光学系に導く光学手段と、
複数の光学系の各々に設けられ、物体空間における焦点の合った物体面の位置（合焦位置）を移動させる合焦位置制御手段と、
合焦位置制御手段により光学系の合焦位置を所定の物体空間に設定した状態において撮像素子により取得された蛍光像とこの蛍光像を撮像する際の合焦位置の情報とを対応させて記録する蛍光像記録手段と、
複数の光学系の内の一つであり第１の光学系とは別の第２の光学系において、合焦位置制御手段により光学系の合焦位置を移動させながら蛍光像記録手段に記録された異なる合焦位置に対応する複数の蛍光像より構成される蛍光観測像空間から、第１及び第２の蛍光観測像空間どうしの位置関係を検出する位置関係検出手段と、を具備することを特徴とする。
【０１３７】
この発明に関する実施の形態は、第５実施形態が対応する。
【０１３８】
本発明は、多方向画像入力装置における各光学結像系を利用して収斂させたレーザ光を試料空間に設置した蛍光物質に照射することにより、試料空間に微小試料を設置したのと同様の効果をもたらすように構成したものである。従って本発明によれば、蛍光物質を注入した容器を試料空間に設置するという簡単な方法により試料空間に微小試料を設置したのと同様の効果を得ることができるため、装置構成や操作が簡便になる。
【０１３９】
（１６） 前記（３）項に記載の多方向画像入力装置において、
光源手段は、容器内の蛍光物質の励起波長の２倍の波長のパルスレーザ光を発生させるレーザ光源手段である
ことを特徴とする。
【０１４０】
この発明に関する実施の形態は、第６実施形態が対応する。
【０１４１】
本発明は、多方向画像入力装置における各光学結像系を利用して収斂させたレーザ光を試料空間に設置した蛍光物質に照射する際に、レーザ光源に蛍光物質の励起波長の２倍の波長のパルスレーザ光を用いることにより２光子励起現象を起こさせ、極めて小さい蛍光スポットを実現するものである。従って本発明によれば、蛍光物質を注入した容器を試料空間に設置するという簡単な方法により試料空間に微小試料を設置したのと同様の効果を得ることができるため、装置構成や操作が簡便になると共に、理想的な微小蛍光点を観測できるため、精度に優れた位置合わせが行なえる。
【０１４２】
（１７） 異なる方向から対象物を結像するように配置された複数の光学系と、この複数の光学系の各々の像面に配置された複数の撮像素子とを具備する多方向画像入力装置において、
蛍光を発生する対象物と共に、対象物の蛍光波長とは異なる波長の蛍光を発生する標準試料が注入される容器と、
対象物の蛍光波長を透過させる第１の色フィルタ及び標準試料の蛍光波長を透過させる第２の色フィルタと、
複数の光学系の各々に設けられ、第１及び第２の色フィルタのどちらかを光学系の光路上に設置する色フィルタ変更手段と、
複数の光学系の各々に設けられ、物体空間における焦点の合った物体面の位置（合焦位置）を移動させる合焦位置制御手段と、
第２の色フィルタが光路上に設置されると共に合焦位置制御手段により光学系の合焦位置を所定の物体空間に設定した状態において撮像素子により取得された標準試料の像とこの像を撮像する際の合焦位置の情報とを対応させて記録する標準試料像記録手段と、
前記合焦位置制御手段により光学系の合焦位置を移動させながら標準試料像記録手段に記録された異なる合焦位置に対応する複数の標準試料の像より構成される標準試料観測像空間から、複数の光学系の各々に対応するこの標準試料観測像空間どうしの位置関係を検出する位置関係検出手段と、を具備することを特徴とする。
【０１４３】
この発明に関する実施の形態は、第７実施形態が対応する。
【０１４４】
本発明は、観測すべき蛍光試料と位置合わせに用いる対象物の蛍光波長とは異なる波長の蛍光を発生する標準試料を混在させた溶液を用い、標準試料の蛍光波長を選択的に透過させる色フィルタを用いて標準試料像のみを入力できるように構成したものである。従って本発明によれば、試料空間における構成や操作をより簡便にすることができる。また位置合わせのために標準試料像を入力する作業と対象物体の観測作業との時間差を短くすることがきるため、より正確な位置合わせが実現できる。
【０１４５】
【発明の効果】
以上詳述したように本発明によれば、各光学結像系の開口が小さいために各入力画像間の相関がとりにくい場合にも適用できると共に、対象画像の空間周波数特性にも依存しない、適用範囲が広くしかも簡便で実用上有用な位置合わせ可能な多方向画像入力装置を提供することができる。
【図面の簡単な説明】
【図１】本発明による第１の実施形態としての多方向画像入力装置の構成例を示す図である。
【図２】図１に示した標準試料設置部の詳細な構成例を示す図である。
【図３】本実施形態に用いる先端が鋭利な試料の先端部を疑似的な微小試料の外観を示す図である。
【図４】本実施形態における試料空間と３Ｄ観測画像空間との概念を示す図である。
【図５】本実施形態において微小試料の３Ｄ像からピーク座標を検出するための手法について説明するための図である。
【図６】本発明による第２の実施形態としての多方向画像入力装置の構成例を示す図である。
【図７】本発明による第３の実施形態としての多方向画像入力装置の構成例を示す図である。
【図８】第３の実施形態における多方向画像入力装置の作用について説明するための図である。
【図９】本発明による第４の実施形態としての多方向画像入力装置の構成例を示す図である。
【図１０】本発明による第５の実施形態としての多方向画像入力装置の構成例を示す図である。
【図１１】本発明による第７の実施形態としての多方向画像入力装置の構成例を示す図である。
【符号の説明】
１，２…顕微鏡
３…標準試料設置部
４…プロセッサ
５，６…対物レンズ
７，８…合焦面制御器
９，１０…ＴＶカメラ
１２，１３…Ａ／Ｄ変換器
１４，１５，２３，２５…メモリ
１６…累積加算器
１７，１８，１９…コントローラ
２０…微小試料
２１…内部バス
２２…ＣＰＵ
２４…ピーク検出器[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a multidirectional image input apparatus configured to input an image of a target object from a plurality of different directions.
[0002]
[Prior art]
In general, in order to reconstruct a tomographic image or a three-dimensional image (3D image) parallel to the optical axis using an image input device having an optical imaging system, an image of an object is input from a plurality of different directions. Technology is known. In such an apparatus, it is necessary to align each input image within the tomographic plane.
[0003]
For example, P.J.Shaw, D.A.Agard, Y.Hiraoka and J.W.Sedat, “Tilted view reconstruction in optical microscopy”, Biophys. J.55 (1989) pp. 101-110. In a microscope provided with a rotating sample holder, an image set composed of a plurality of input images is input twice from directions different from each other by 90 ° while moving the in-focus position in the optical axis direction. There has been proposed a system for calculating a phase correlation between input image sets of a target object when aligning in a dimensional space.
[0004]
[Problems to be solved by the invention]
The above-described conventional technology, in principle, has a sufficiently large aperture (Numerical Aperture: NA) of the objective lens of the microscope, and even if the optical axis is set in two orthogonal directions, the three-dimensional optical transfer characteristics in each direction can be obtained. It can be used under conditions such as large overlap on the spatial frequency.
[0005]
However, this prior art does not describe the case where the aperture of the optical imaging system is small and the correlation between the input images is difficult to obtain, or the alignment that can be applied without depending on the spatial frequency characteristics of the target image.
[0006]
Accordingly, an object of the present invention is to provide a multi-directional image input device that has a wide application range, is simple, and can be used practically.
[0007]
[Means for Solving the Problems]
  A plurality of optical systems arranged to form an image of an object within an image input range in a space for observation from different directions, and a plurality of image sensors arranged on the image planes of the plurality of optical systems In a multi-directional image input device having an object, the object is located within the range of the image input when the image is inputMarkedA quasi-sample, a focus position control means for moving a focused object surface position (focus position) in the object space of the plurality of optical systems, and a focus position of the optical system by the focus position control means Standard sample image recording means for recording the image information of the standard sample acquired by the imaging device in a state where the image sensor is set in a predetermined object space and the information on the in-focus position at the time of acquiring the image information, The standard sample while moving the focus position of the optical system by the focus position control meansimageThe positional relationship between the standard sample observation image space corresponding to each of the plurality of optical systems from the standard sample observation image space composed of the images of the plurality of standard samples corresponding to different in-focus positions recorded in the recording means. A positional relationship detecting means for detectingThe positional relationship detection means uses, for each of the plurality of optical systems, an XY plane using image information of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image recording means. First peak position detecting means for detecting the peak position of the pixel density value, and pixel densities of a plurality of standard sample images corresponding to different in-focus positions at the peak position detected by the first peak position detecting means Second peak position detection means for detecting a peak position in the Z direction from the value, the arrangement position of the plurality of optical systems, and a peak corresponding to each optical system obtained by the second peak position detection means Corresponding to each of the plurality of optical systems by obtaining a relative positional relationship with the position and determining that the peak positions for the respective optical systems obtained by the first and second peak position detecting means are at the same position. To do Determining the positional relationship between the quasi-sample observation image space each otherA multidirectional image input device is provided.
  In addition, a plurality of optical systems arranged to form an image of an object within an image input range in a space for observation from different directions, and a plurality of optical systems arranged on the respective image planes of the plurality of optical systems In a multidirectional image input device having an image sensor, a standard sample arranged within the image input range in which an object is present at the time of image input, and a focused object plane in the object space of the plurality of optical systems A focus position control means for moving the position (focus position), and the standard sample acquired by the imaging device in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means. Standard sample image recording means for recording image information and information on a focus position at the time of acquiring the image information, and the standard sample while moving the focus position of the optical system by the focus position control means Record on recording means A position for detecting a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems from a standard sample observation image space composed of images of a plurality of standard samples corresponding to different focused positions. A plurality of standard sample images corresponding to different in-focus positions recorded in the standard sample image recording unit for each of the plurality of optical systems. The first peak position detecting means for detecting the peak position of the pixel density value from the added image information, and the peak position detected by the first peak position detecting means is different. Second peak position detecting means for detecting a peak position from pixel density values of images of a plurality of standard samples corresponding to in-focus positions, the arrangement positions of the plurality of optical systems, and the second peak Detection means By calculating the relative positional relationship with the peak position corresponding to each optical system, and determining that the peak position for each optical system determined by the first and second peak position detecting means is at the same position Provided is a multidirectional image input device for obtaining a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems.
[0008]
  In a multidirectional image input device comprising a plurality of optical systems arranged to form an image of an object from different directions, and a plurality of imaging elements arranged on each image plane of the plurality of optical systems, A container containing a fluorescent material arranged at a substantially position where an object is present when inputting an image, light source means for generating light for exciting the fluorescent material in the container, and one of the plurality of optical systems. An optical unit that guides the excitation light to the first optical system so that the excitation light emitted by the light source unit is converged in the container using a first optical system; A focusing position control means provided in each for moving a position (focusing position) of a focused object plane in the object space; and a focusing position of the optical system is set to a predetermined object space by the focusing position control means. In the state set to A fluorescence image recording means for recording the fluorescence image acquired in association with information on a focus position when the fluorescence image is captured, and one of the plurality of optical systems, the first optical In a second optical system different from the system, a plurality of fluorescent images corresponding to different in-focus positions recorded in the fluorescent image recording means while moving the in-focus position of the optical system by the in-focus position control means A positional relationship detection means for detecting a positional relationship between the fluorescence observation image spaces respectively corresponding to the first and second optical systems from a fluorescence observation image space constituted by: For each of the first and second optical systems, the peak position of the pixel density value in the XY plane is detected using a plurality of fluorescent images corresponding to different in-focus positions recorded in the fluorescent image recording means. First peak position detecting means for performing the first peak detection Second peak position detecting means for detecting a peak position in the Z direction from pixel density values of a plurality of fluorescent images corresponding to different in-focus positions at the peak position detected by the center position detecting means. And a relative positional relationship between the arrangement position of the second optical system and the peak position corresponding to each optical system obtained by the second peak position detection means, and the first and second peak position detection means The positional relationship between the fluorescence observation image spaces corresponding to each of the plurality of optical systems is obtained by determining that the obtained peak positions for the respective optical systems are at the same position.A multidirectional image input device is provided.
  Furthermore, in a multidirectional image input device comprising a plurality of optical systems arranged so as to form an image of an object from different directions and a plurality of imaging elements arranged on the respective image planes of the plurality of optical systems. A container into which a standard sample that generates fluorescence having a wavelength different from the fluorescence wavelength of the target is injected together with a target that generates fluorescence, a first color filter that transmits the fluorescence wavelength of the target, and a standard sample A second color filter that transmits the fluorescence wavelength; a color filter changing unit that is provided in each of the plurality of optical systems, and that sets one of the first and second color filters on an optical path of the optical system; In-focus position control means that is provided in each of the plurality of optical systems and moves the position (in-focus position) of the focused object surface in the object space, and the second color filter are installed on the optical path. The in-focus position control hand The standard sample that records the image of the standard sample acquired by the image sensor in correspondence with the information on the focus position when the image is captured in a state where the in-focus position of the optical system is set in a predetermined object space A standard sample comprising image recording means and images of a plurality of standard samples corresponding to different focus positions recorded in the standard sample image recording means while moving the focus position of the optical system by the focus position control means A positional relationship detection means for detecting a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems from the observation image space, and the positional relationship detection means includes the plurality of optical systems. A first peak position for detecting a peak position of a pixel density value in the XY plane using image information of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image recording means for each. Detection means A second peak position detecting means for detecting a peak position in the Z direction from pixel density values of a plurality of standard sample images corresponding to different in-focus positions at the peak position detected by the first peak position detecting means; And determining a relative positional relationship between the arrangement positions of the plurality of optical systems and the peak positions corresponding to the respective optical systems obtained by the second peak position detecting means, and the first and second peaks. A multidirectional image input device for obtaining a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems by determining that the peak positions for the respective optical systems obtained by the position detection means are at the same position. I will provide a.
[0009]
  ThisThe multi-directional image input device configured as follows: By inputting an image of a standard sample with a shape that satisfies the optical standard setting when observing or obtains optical characteristics when observing, An image that directly reflects the spatial characteristics of the optical imaging system can be estimated, and image information necessary for alignment between the input images in the multidirectional image input apparatus can be obtained. The present invention can also be applied to the case where the correlation between the input images is difficult to obtain due to the small aperture of each optical imaging system in the multidirectional image input apparatus, and does not depend on the spatial frequency characteristics of the target image, and is applicable. Is wide.
  In addition, the multidirectional image input device sets a micro sample in the sample space by irradiating a fluorescent material set in the sample space with a laser beam converged by using each optical imaging system in the multidirectional image input device. It is configured to bring about the same effect as that. In addition, a color filter that selectively transmits the fluorescence wavelength of the standard sample using a solution in which a standard sample that generates fluorescence having a wavelength different from the fluorescence wavelength of the target object used for alignment with the fluorescence sample to be observed is mixed. Only the standard sample image is input.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0011]
FIG. 1 shows an example of the configuration of a multidirectional image input apparatus as a first embodiment according to the present invention. This apparatus is roughly composed of two microscopes 1, a microscope 2, a standard sample placement unit 3, and a processor 4.
[0012]
The microscopes 1 and 2 are configured in the same manner so that their optical axes intersect at right angles on an object space (hereinafter referred to as a sample space) in which an arbitrary sample is placed. The microscopes 1 and 2 are, as main constituent parts, objective lenses 5 and 6 and focusing surface controllers 7 and 8 for driving and controlling the positions of focused object surfaces (focusing surfaces) in the optical axis direction. And TV cameras 9 and 10 for taking an image of the sample.
[0013]
It is assumed that the actual size in the sample space corresponding to the inter-pixel distance of the input image is known by measuring in advance using a standard scale. This is necessary to define a common 3D scale for the observation image space of each of the microscopes 1 and 2. That is, when the z coordinate is defined in the optical axis direction in the observation image space by the microscope 1, the z coordinate in the observation image space by the microscope 1 corresponds to the horizontal direction in the imaging plane.
[0014]
Therefore, in order to define a common 3D scale, it is necessary to know the actual size in the image and the actual movement distance of the focal plane.
[0015]
As an example of the configuration of the focusing surface controllers 7 and 8, for example, a stepping motor is incorporated, and the objective lenses 5 and 6 are moved according to the number of pulses generated by the controller 17 and 18 of the stepping motor in the processor 4. Each distance is controlled and set to a focal plane position.
[0016]
Therefore, the processor 4 can always grasp the position of the focal plane corresponding to each of the plurality of inputted images. An encoder for detecting the positions of the objective lenses 5 and 6 in the optical axis direction may be provided in the focusing plane controllers 7 and 8 so that the detected encoded signal is fed back to the processor 4.
[0017]
With the configuration as described above, the x, y, and z scales of the 3D observation image space configured by arranging a plurality of images input while moving the focal plane are input to the input images of the microscopes 1 and 2, respectively. On the other hand, it is described as a common scale. In addition, what is necessary to mutually use the images input by the microscopes 1 and 2 is information on the origin common to the microscope image input systems.
[0018]
In the present embodiment, the origin of the 3D observation image space is determined by a configuration using a micro standard sample as described below.
[0019]
In FIG. 2, the detailed structural example of the said standard sample installation part 3 is shown.
[0020]
In the standard sample setting unit 3, a holder 31 for holding the micro sample 20 is placed on the sample space near the intersection of the optical axes of the objective lenses 5 and 6 (microscopes 1 and 2) by a holder rotation control device 32 as necessary. The drive control is performed so that the micro sample 20 is placed at the position.
[0021]
The operation of the standard sample placement unit 3 has a “calibration mode” for obtaining position information and an “observation mode” for observing a target sample. In the calibration mode, the micro sample 20 is placed in the sample space, and the image is input simultaneously by the microscopes 1 and 2.
[0022]
The configuration operation in the calibration mode related to the present invention will be described below.
[0023]
In each of the microscopes 1 and 2, the focusing surface controllers 7 and 8 capture images of a plurality of micro samples 20 corresponding to different focusing surfaces while moving the focusing surfaces, and image signals are sent to the processor 4. Entered.
[0024]
In the processor 4, the image signals are digitally converted by the A / D converters 12 and 13, respectively, and recorded in the memories 14 and 15. In this way, a plurality of images with different focal planes are recorded in the memories 14 and 15.
[0025]
Next, the addition between the plurality of images recorded in the memory 14 is executed by the cumulative adder 16, and the result is recorded in the memory 23 through the memory internal bus 21 as one image. Two-dimensional peak coordinates are detected from the added image recorded in the memory 23 by the peak detector 24, and the coordinate values are recorded in the memory 25.
[0026]
The density peak in the added image appears corresponding to the position of the standard sample 20 in the 3D space. Subsequently, image density values corresponding to the peak coordinates are sequentially read from each image recorded in the memory 14 and recorded in the memory 23 as a one-dimensional profile. The peak detector 24 detects peak coordinates from the one-dimensional profile recorded in the memory 23, and the coordinate values are recorded in the memory 25.
[0027]
The operation described above is performed in the same manner for the image recorded in the memory 15, and as a result, the peak coordinates are detected for each of the 3D observation images for the micro sample 20 input from both the microscopes 1 and 2. It is defined as the common origin of the 3D observation space. Note that the memory 25 also records scale information in the x, y, and z directions obtained in advance. It is assumed that the image information of these standard samples and the information on the in-focus position when the image is captured (taken) are stored in the memory in correspondence with each other.
[0028]
The origin of the 3D observation image and the x, y, z scale information obtained in this way in each microscope image input system are used for alignment between the input images from each microscope in the observation mode. In the processor 4, controllers 17 and 18 that drive and control the focusing surface controllers 7 and 8 and a controller 19 that drives and controls the standard sample placement unit 3 are incorporated. The entire processor 4 is controlled by the CPU 22, and signal transmission between the components is performed via the internal bus 21.
[0029]
As the micro sample 20, an object smaller than the size in the sample space corresponding to one pixel of the digital image recorded in the memories 14 and 15 is used. As an example of the micro sample 20, the cross section may be circular in the direction along the optical axis, and a micro metal sphere, a cone, a cylinder, or the like may be used.
[0030]
When the microscopes 1 and 2 are fluorescent microscopes, microspheres (fluorescent beads) infiltrated with a fluorescent dye may be used.
[0031]
On the other hand, instead of the micro sample 20, a spherical object that is not so large as compared with the imaging range may be used. Since this spherical object is considered to have a sufficient band in all directions of the 3D space, an image enlarged in size similar to the input image when a micro sample is used will be observed. The peak can be detected. Moreover, you may comprise so that the front-end | tip part of a sample with a sharp tip as shown in FIG. 3 may be used as a pseudo micro sample. As an example of a sample having a sharp tip, a microneedle or the like created by a semiconductor process may be used.
[0032]
Next, the operation of the multi-directional image input apparatus configured as described above will be described.
[0033]
This multi-directional image input device has an operation of setting the origin of observation image space coordinates by inputting a minute sample 3D image and detecting peak coordinates.
[0034]
FIG. 4 shows the concept of the sample space and the 3D observation image space.
[0035]
In general, the optical imaging system is subjected to a strong band limitation in the optical axis direction, but in the case of an optical imaging system having a large aperture (NA) as used in an optical microscope, the depth of focus is relatively shallow. Therefore, although the 3D image of the micro sample extends in the optical axis direction longer than the imaging plane direction, a density peak in the added image appears at a position where the micro sample exists.
[0036]
Therefore, it is possible to set the origin of coordinates in the observation image space by detecting such a peak of the 3D image. In a multi-optical axis microscope as shown in this embodiment, if a micro sample placed in a sample space is observed from each microscope, and the origin of each observation space is set, images input by each microscope are common to each other. It can be handled in 3D space.
[0037]
With reference to FIG. 5, a method for detecting peak coordinates from a 3D image of a minute sample in the present embodiment will be described.
[0038]
First, in order to detect a peak on the XY plane parallel to the imaging plane, a plurality of input images are cumulatively added in the Z direction while moving the focal plane, and a 2D search is performed in the cumulative addition image. To detect the peak position. Thus, the origin with respect to the X and Y coordinates can be detected. Next, peak coordinates are detected from a one-dimensional profile in the Z direction created by reading out pixel density values corresponding to peak coordinates on the XY plane from each input image. In this way, the origin in the Z direction can be detected.
[0039]
As described above, according to the first embodiment, a 3D image that directly reflects the optical imaging system 3D spatial characteristics can be input by using a micro sample or an object equivalent thereto as a standard sample. The origin can be easily set by detecting the peak of the 3D observation image.
[0040]
By performing such an operation on each of the plurality of image input systems with respect to the same standard sample image, it becomes possible to treat the origin obtained for each 3D observation image space as a common origin, and from different directions. Alignment between input images is realized. Further, since the inter-pixel distance and the position of the focal plane in each input image are grasped in the processor 4, the common 3D coordinates for each 3D observation image space are defined by these and the position information of the origin. .
[0041]
Furthermore, according to the peak coordinate detection method shown in the present embodiment, the peak detection may be performed by searching for 2D and 1D once, which is simpler than searching for the peak coordinates directly from the 3D observation space. That is, it is not necessary to directly search a 3D image composed of a large number of pixels, and peak coordinates are detected by a simple method.
[0042]
Next, FIG. 6 shows a configuration example of a multidirectional image input apparatus as a second embodiment according to the present invention and will be described. 6 that are the same as those in the first embodiment shown in FIG. 1 are assigned the same reference numerals, and descriptions thereof are omitted.
[0043]
In the present embodiment, the microscope 1 is configured to be held by an xyz automatic stage 34. In the processor 4, a controller 35 is provided on the internal bus 21, and an xyz automatic stage 34 is driven by a control signal from the controller 35 to finely adjust the spatial position of the microscope 1.
[0044]
The operation of the second embodiment configured as described above will be described.
[0045]
The present embodiment has an effect of finely adjusting the position of the microscope 1 so that the origin of the 3D observation image space detected by the same configuration operation as that of the first embodiment is located at the center of the 3D observation image space.
[0046]
First, in the same manner as in the first embodiment, when the origin on the 3D observation image space in the microscope 1 is observed, the distances in the x, y, and z directions between the center of the 3D observation image space and the origin are obtained. The xyz automatic stage 34 is driven and controlled according to the distance information. Then, the standard sample image is input again, and the origin of the 3D observation image space is detected.
[0047]
By repeating the above operation several times, the origin and center coincide with each other in the 3D observation image space.
[0048]
According to the present embodiment, since the condition in which the origin coincides with the center in the 3D observation image space is obtained, the largest field angle size can be used when the input images from the respective microscopes are mutually used in the observation mode. That is, it is not necessary to shift the address of the input image when aligning the input image from each microscope, and the reconstructed image does not become smaller than the input image.
[0049]
In the present embodiment, an xyz automatic stage may also be provided for the microscope 2 so that both microscopes can be adjusted in the same manner.
[0050]
Next, FIG. 7 shows an example of the configuration of a multidirectional image input apparatus as a third embodiment according to the present invention.
[0051]
As in the first embodiment, the present embodiment relates to a method for estimating the origin from the parallel projection image when obtaining the origin on the 3D observation image space from the input image of the minute sample. FIG. 7 shows the configuration of the processor 40 of this embodiment, but other configurations such as a microscope are the same as those of the first embodiment.
[0052]
In this multidirectional image input device, as in the first embodiment, an image of a micro sample input while moving the focal plane by the microscopes 1 and 2 is input by A / D converters 41 and 42 in the processor 40. Are converted into digital image signals and recorded in the memories 43 and 44. A 3D observation image composed of a plurality of images with different focal planes recorded in the memory 43 is rotated by a predetermined angle φ by the image rotation unit 45. Note that the image rotation unit 45 includes a ROM that stores read addresses and interpolation coefficients for rotation operation of the angle φ, a product-sum calculation device, and the like, and performs rotation processing while performing 3D observation image interpolation processing. It is configured to be.
[0053]
Then, the 3D observation image rotated by a predetermined angle φ is recorded in the memory 46, and then cumulatively added in the Z-direction by the cumulative adder 47 to be two-dimensional (2D) projected in the memory 48 with respect to the angle φ direction. An image is recorded in the memory 48. A peak position is detected by a peak detector 49 from the 2D projection image recorded in the memory 48. Such an operation is repeated for different rotation angles φ, and the peak coordinates for each angular direction are detected.
[0054]
Next, the detected peak coordinates are sent to the sine curve regression analyzer 50. As a result of a predetermined analysis, the origin coordinates are detected and recorded in the memory 51. The above operation is similarly performed on the 3D observation image recorded by the microscope 2 recorded in the memory 44.
[0055]
In the processor 40, controllers 52, 53, and 54 corresponding to the controllers 17, 18, and 19 of the first embodiment are provided. The entire control of the processor 40 is performed by the CPU 55, and signal transmission between each component is performed via the internal bus 56.
[0056]
Next, the operation of the multidirectional image input apparatus configured as described above will be described with reference to the conceptual diagram of FIG. In the present embodiment, the origin is detected from the 3D observation image of the minute sample by the following action.
[0057]
First, a projection image 1 shown in FIG. 8 is created by projecting a 3D observation image in parallel in the optical axis direction, and the origin in the XY plane is detected.
[0058]
Next, it is assumed that the 3D observation image is projected in the direction of the angle φ around the center of the 3D observation space with respect to the optical axis to create the projection image 2 shown in FIG. Although FIG. 8 describes that the image is projected in the angle direction φ, according to the configuration shown in FIG. 7, the image itself is rotated by the angle −φ and then projected in the Z direction. This is realized.
[0059]
However, it is clear that the same projection image can be obtained in either case. Assuming that the position where the micro sample is actually deviated from the center of the 3D observation space by the distance d in the optical axis direction, the deviation from the center of the peak coordinates in the projected image is expressed by the following equation.
[0060]
[Expression 1]

[0061]
Therefore, it is possible to estimate d by detecting peak coordinates with respect to several kinds of angular directions φ and performing regression approximation with sine curves. Note that the N.I. A. The angle direction φ in which the peak can be detected is limited to a range in which the following expression is satisfied, and therefore several angle directions φ may be selected within the range.
[0062]
[Expression 2]

[0063]
According to the present embodiment, since the peak coordinates are detected from the 2D projection image when the origin is detected, the peak detection can be performed more easily than the direct detection of the peak coordinates from the 3D observation image. In addition, since the origin is estimated from the peak values detected from a plurality of projection images, the detection error can be canceled and the most probable origin position can be estimated.
[0064]
The present embodiment may be operated as shown below with the same configuration as in FIG. First, the origin in the XY plane is detected from a 2D projection image in a direction parallel to the optical axis. Next, an image passing through the XY origin detected from the 3D observation image recorded in the memory 503 or 504 and parallel to the rotation plane is read out. The read 2D image is rotated by a predetermined angle φ and then cumulatively added in the Z direction to create a 1D parallel projection image. Peak detection is performed on this 1D profile. If comprised as mentioned above, since the peak detection is a search on 1D, it can be performed more easily and the origin can be detected with high accuracy.
[0065]
Next, FIG. 9 shows a configuration example of a multidirectional image input apparatus as a fourth embodiment according to the present invention.
[0066]
This multidirectional image input device is a device that obtains a relative positional relationship between 3D observation image spaces by using a correlation between input images from various directions with respect to a minute sample in a multi-optical axis image input system.
[0067]
It is assumed that the configuration other than the processor 600 of the present embodiment shown in FIG. 9 is the same as that of the first embodiment shown in FIG. That is, the image of the micro sample input while moving the focal plane by the microscopes 1 and 2 is converted into a digital image signal by the A / D converters 61 and 62 in the processor 60 and recorded in the memories 63 and 64. The
[0068]
Next, in the DSP 65 configured to perform FFT, each of the memories 63 and 64 performs 3D Fourier transform on a 3D observation image composed of recorded images with different focal planes. It is recorded in the memories 63 and 64 again. Subsequently, a product-sum operation unit 66 performs a predetermined product-sum operation using each 3D Fourier image recorded in the memories 63 and 64, and the result is recorded in the memory 64. The DSP 65 performs a 3D inverse Fourier transform on the 3D Fourier image re-recorded in the memory 64, and the result is recorded in the memory 63.
[0069]
With the above configuration operation, the 3D correlation between the 3D observation images is calculated. The peak detector 67 detects the peak coordinates from the 3D correlation image recorded in the memory 63, and the coordinate values are recorded in the memory 68. The 3D correlation peak coordinates recorded in the memory 68 are used in the observation mode as the relative positional relationship between the observation images by the respective input optical systems. In the processor 60, controllers 69, 70, and 71 corresponding to the controllers 17, 18, and 19 of the first embodiment are provided. The entire processor 60 is controlled by the CPU 72, and signal transmission between each component is performed via the internal bus 73.
[0070]
The operation of the fourth embodiment configured as described above will be described. This embodiment has an effect of calculating a correlation between 3D observation images defined by the following equation.
[0071]
[Equation 3]

[0072]
Here, g1 (r) and g2 (r) are the first and second 3D observation images in the multi-optical axis image input system, respectively. The vectors r and t are both coordinate vectors defined in the 3D orthogonal coordinate space. When using a small sample, the expression (3) can be developed as follows.
[0073]
[Expression 4]

[0074]
Here, h1 (r) and h2 (r) are first and second 3D response functions in the multi-optical axis image input system, δ (r) is a delta function representing a minute sample, and rs is first and second, respectively. A 3D vector representing a relative displacement between 3D observation images, and * is an operator of 3D convolution. When the equation (4) is calculated, a peak appears at the position of the point rs in the 3D correlation image s (t). By detecting this, the relative positional relationship can be known. In this embodiment, when obtaining 3D correlation, calculation using 3D Fourier transform is performed as shown below. Equation (3) can be rewritten as:
[0075]
[Equation 5]

[0076]
However, F_3D{} Is a 3D Fourier transform operator, F^* _{3 D}{} Is an operator of 3D Fourier transform and complex co-contract, F_3D ^-1{} Is an operator of 3D inverse Fourier transform. The 3D observation images g1 (r) and g2 (r) are rewritten as follows: 3D Fourier transform images G1 (u) and G2 (u) (where u is the spatial frequency coordinate of the 3D orthogonal space).
[0077]
[Formula 6]

[0078]
In this case, the calculation performed in the 3D Fourier space is expressed by the following equation.
[0079]
[Expression 7]

[0080]
The product-sum calculator 66 executes a calculation based on the equation (7) for two 3D Fourier images. The relative positional relationship can be obtained by detecting the peak coordinates from the correlation images of the two 3D observation images calculated in this way.
[0081]
By the way, N. of each optical system of the multi-optical axis optical imaging system. A. If is not very large, the correlation between the respective 3D observation images is weak, and it is difficult to always detect the correlation peak with high accuracy for an arbitrary sample.
[0082]
However, according to the present embodiment, since a small sample that can be handled as an ideal point object is used, a 3D observation image representing the original imaging characteristics of the optical imaging system can be obtained, and the most ideal 3D correlation image is obtained. be able to. That is, if there is even a slight correlation between the optical imaging systems, it is possible to detect a correlation peak using a 3D observation image of a minute sample.
[0083]
Therefore, according to the correlation method, an operation for obtaining coordinates common to each 3D observation space as shown in the first to third embodiments is not necessary, and a relative positional relationship between 3D observation images can be directly obtained. Therefore, the amount of information to be recorded can be reduced, and at the same time, the alignment process is simplified.
[0084]
Next, FIG. 10 shows a configuration example of a multidirectional image input apparatus as a fifth embodiment according to the present invention.
[0085]
This multi-directional image input device is configured to produce the same effect as a micro sample placed in a sample space by irradiating a fluorescent dye solution with laser light focused by an objective lens.
[0086]
In this multidirectional image input device, the microscopes 81 and 82 have objective lenses 83 and 84, respectively, and are arranged so that the optical axes are orthogonal to each other in the sample space, as in the first embodiment. Similarly, a focusing surface controller and a TV camera are also provided, but illustration is omitted here.
[0087]
A container 85 is installed in the sample space in the calibration mode, and a fluorescent dye solution 86 is injected therein. The laser optical system is configured as follows.
[0088]
Laser light emitted from the laser 87 is converted into a desired beam size by a laser beam expander 88 and introduced into the microscope 81 by a mirror 89. A mirror 90 is installed in the microscope 81 and is configured such that laser light is converged in the container 85 using the imaging optical system of the objective lens 83. The microscope 81 is provided with a focusing surface controller (not shown). When laser light is guided into the microscope 81, the focusing surface controller controls the focusing surface of the microscope 81 in the 3D observation image space. It is set to the center of the defined coordinate in the optical axis direction. At this time, the microscope 82 receives a plurality of images while moving the focal plane, and transfers the images to the processor 80. The processor 80 is configured similarly to the processor 4 of the first embodiment, and detects the peak coordinates from the 3D observation image input using the microscope 82.
[0089]
In the observation mode, another container having the same specifications as the container 85 into which the sample is injected together with the solution is installed instead of the container 85, and the internal sample is observed. The objective lenses 83 and 84 may be water immersion lenses, and a new container that immerses both the objective lens and the container 85 in the liquid may be provided. In that case, the container 85 is made of a material having a refractive index close to that of the surrounding liquid.
[0090]
Next, the operation of this embodiment will be described.
[0091]
In the present embodiment, the fluorescent dye solution 86 is excited by a laser beam spot created in the container 85 by using the optical imaging system of the microscope 81, so that the same effect as that in which a micro sample is placed in the sample space is obtained. Bring. Since laser light is excellent in spatial coherence, an ideal spot light can be created. Since the laser light spot appears at a position corresponding to the origin on the 3D observation space of the microscope 81, the relative positional relationship between the two optical imaging systems can be known by detecting the position from the 3D observation image of the microscope 82. it can.
[0092]
Therefore, according to the present embodiment, it is possible to obtain the same effect as when a micro sample is placed in the sample space by a simple method of placing the container 85 in the sample space. .
[0093]
Next, a multidirectional image input apparatus according to a sixth embodiment will be described.
[0094]
The sixth embodiment is configured to irradiate a fluorescent dye solution with laser light focused by an objective lens, but uses a two-photon excitation phenomenon, so a model that generates intense pulsed light in the infrared region is used. It is done. While the laser pulse light is continuously irradiated, in the microscope 82, the shutter of the TV camera is released until an image with sufficient contrast is obtained, and then the image signal is transferred to the processor 80.
[0095]
The operation of the sixth embodiment will be described.
[0096]
In this embodiment, a two-photon excitation phenomenon of a fluorescent dye is used by using a pulsed laser beam having a wavelength twice the excitation wavelength of a fluorescent substance as a laser light source. Fluorescence due to two-photon excitation is usually extremely difficult to observe, but observable fluorescence is generated in a high-energy local space created by converging strong laser pulse light. Since such two-photon excitation fluorescence is generated only in a very small space, an ideal minute point light source that emits light in a range much narrower than the focal depth of the optical imaging system can be obtained.
[0097]
Therefore, according to the present embodiment, an ideal minute fluorescent spot can be observed, so that alignment with excellent accuracy can be performed.
[0098]
Next, FIG. 11 shows a configuration example of a multidirectional image input apparatus according to the seventh embodiment of the present invention.
[0099]
This multi-directional image input device is configured to use a solution in which a sample to be observed and a micro sample used for alignment are mixed.
[0100]
In this multidirectional image input apparatus, the microscopes 91 and 92 are arranged so that the optical axes are orthogonal to each other in the sample space, as in the first embodiment. The microscopes 91 and 92 are provided with an objective lens, a focusing surface controller, and a TV camera as in the first embodiment, but are not shown.
[0101]
First, a container 93 is installed in the sample space, and a sample 95 to be observed is injected together with the micro fluorescent sphere 94 at an appropriate density. Note that the micro fluorescent sphere 94 and the sample 95 are respectively stained with two types of fluorescent dyes having fluorescence spectra in different wavelength regions. On the optical path of the microscope 91, two types of color filters 96 and 97 having different transmission wavelength regions are installed, and one of them is used as necessary. Switching between these color filters 96 and 97 is performed by a filter changing unit 98.
[0102]
In such a configuration, in the calibration mode, the color filter 96 that selectively transmits the fluorescence wavelength of the micro fluorescent sphere 94 is installed on the optical path, and an image of the micro fluorescent sphere 94 is input by the microscope 91. In the observation mode, a color filter 97 that selectively transmits the fluorescence wavelength of the sample 95 is installed on the optical path, and the sample 95 is observed.
[0103]
The configuration as described above is omitted in FIG. 11, but is similarly provided in the microscope 92. The images of the micro fluorescent spheres 94 input by the microscopes 91 and 92 are input to the processor 99, and the peak coordinates are detected by performing the same processing as in the first embodiment. However, in this embodiment, a 3D observation image space is defined in which the peak coordinates are detected for each of the images of the plurality of micro fluorescent spheres 94 captured in the image, and all of the position coordinates can be recognized by a common address. Is done. Note that a controller 101 for controlling the filter changing unit 98 on the internal bus 100 is provided in the processor 99, but the rest of the configuration is the same as that of the processor 99 of the first embodiment.
[0104]
Next, the operation of this embodiment will be described.
[0105]
In the present embodiment, the image of the micro fluorescent sphere 94 is selectively input in the calibration mode by changing the color filters 96 and 97 without changing the configuration in the sample space between the calibration mode and the observation mode. It is comprised so that it may be.
[0106]
Therefore, according to the present embodiment, the configuration and operation in the sample space can be simplified. In addition, since the time difference between the calibration mode and the observation mode can be shortened, more accurate alignment can be realized. Further, since the alignment for the plurality of micro fluorescent spheres 94 can be used, there is an effect that the accuracy can be improved.
[0107]
Although the above embodiments have been described, the present invention includes the following inventions.
[0108]
(1) A plurality of optical systems arranged to form an image of an object within an image input range in a space for observation from different directions, and a plurality of optical systems arranged on the image planes of the plurality of optical systems. A multi-directional image input device having an imaging element, a standard sample having a shape that can be placed within the image input range where an object is present at the time of image input and can be optically set for observation,
An in-focus position control means for moving a position (in-focus position) of a focused object plane in the object space of the plurality of optical systems;
The image information of the standard sample acquired by the imaging device in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means, and the focus position when the image information is acquired A standard sample image recording means for recording information correspondingly;
From a standard sample observation image space composed of a plurality of standard sample images corresponding to different focus positions recorded in the standard sample recording means while moving the focus position of the optical system by the focus position control means, A positional relationship detection means for detecting a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems;
A multi-directional image input device.
[0109]
Embodiments relating to the present invention correspond to the first, third, and fourth embodiments.
[0110]
According to the present invention, it is possible to estimate an image that directly reflects the spatial characteristics of the optical imaging system by inputting an image of a standard sample having a shape that provides optical characteristics during observation. Image information necessary for alignment between the input images in the multidirectional image input apparatus can be obtained. Accordingly, the present invention can be applied to the case where the correlation between the input images is difficult due to the small aperture of each optical imaging system in the multi-directional image input apparatus, and it does not depend on the spatial frequency characteristics of the target image. It is possible to provide a positioning method which is wide and simple and practically useful.
[0111]
(2) In the multidirectional image input device according to (1),
The positional relationship detecting means includes
Detects the positional relationship between the standard sample observation image spaces by using the absolute address coordinates with the origin of the position of the observed standard sample image for the standard sample observation image space corresponding to each of the multiple optical systems. And
[0112]
(3) In the multidirectional image input device according to (2),
The positional relationship detecting means includes
It has a function of detecting the most focused position as a peak position with respect to the standard sample from the standard sample observation image space corresponding to each of the plurality of optical systems.
[0113]
(4) In the multidirectional image input device according to (3),
The positional relationship detection means for each of the plurality of optical systems
Image adding means for adding image information of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image recording means;
First peak position detecting means for detecting a peak position of the pixel density value from the added image information;
Second peak position detecting means for detecting peak positions from pixel density values of a plurality of standard sample images corresponding to different in-focus positions at the peak positions detected by the first peak position detecting means; ,
A relative positional relationship between the arrangement positions of the plurality of optical systems and the peak positions corresponding to the respective optical systems obtained by the second peak detecting means is obtained, and the first and second peak position detecting means are provided. A multidirectional image input characterized in that a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems is obtained by determining that the obtained peak positions for the respective optical systems are at the same position. apparatus.
[0114]
(5) In the multidirectional image input device according to (4),
The first peak position detection means and the second peak position detection means in the positional relationship detection means are configured and used in the same way, or one peak position detection means is switched and used.
[0115]
(6) In the multidirectional image input device according to (3),
The positional relationship detecting means includes
Parallel projection means for generating parallel projection images in a predetermined angular direction with respect to the optical axis direction, using images of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image information recording means;
A peak position detecting means for detecting a peak position from a parallel projection image;
Position estimation means for estimating the position of the standard sample using the peak position;
It is characterized by comprising.
[0116]
The first embodiment corresponds to the embodiment relating to the present invention.
[0117]
According to the present invention, image information (for example, peak coordinates) that is an effective index for alignment between input images in a multidirectional image input device is detected from an image of a standard sample, so that each of a plurality of optical systems can be detected. The coordinates of the corresponding standard sample observation image space can be easily set. By performing such operations on the same standard sample image in each of the plurality of image input systems, the coordinates obtained for each standard sample observation image space can be handled as common coordinates. Alignment between input images from the direction is realized.
[0118]
(7) In the multidirectional image input device according to (1),
The positional relationship detecting means includes
A positional relationship between the standard sample observation image spaces is detected by detecting a correlation between the standard sample observation image spaces for each of the plurality of optical systems.
[0119]
(8) In the multidirectional image input device according to (7),
The positional relationship detecting means includes
Fourier transform means for performing Fourier transform on the standard sample observation image space for each of a plurality of optical systems;
A cross-correlation calculating means for performing a cross-correlation calculation defined in Fourier space from a standard sample observation image space for each of a plurality of optical systems subjected to Fourier transform;
An inverse Fourier transform means for performing an inverse Fourier transform on the cross-correlation image in the Fourier space calculated by the cross-correlation calculating means;
A peak position detecting means for detecting a peak position from the cross-correlation image subjected to inverse Fourier transform;
It is characterized by comprising.
[0120]
(9) In the multidirectional image input device according to (8),
The Fourier transform means and the inverse Fourier transform means in the positional relationship detection means are constituted by the same means.
[0121]
The fourth embodiment corresponds to the embodiment relating to the present invention.
[0122]
According to the present invention, by inputting an image of a standard sample whose shape is known in advance, it is possible to estimate an image that directly reflects the spatial characteristics of the optical imaging system, and the most ideal standard sample observation Correlation images between image spaces can be obtained. According to the correlation method, since the relative positional relationship between the standard sample observation images is directly obtained, the amount of information to be recorded can be reduced and the alignment process can be simplified.
[0123]
(10) In the multidirectional image input device according to (1),
The standard sample is composed of an object having a diameter smaller than the size of the object space corresponding to one pixel of an image recorded in the standard sample image recording means.
[0124]
The first embodiment corresponds to the embodiment relating to the present invention.
[0125]
According to the present invention, it is possible to input an image that directly reflects the spatial characteristics of the optical imaging system by using a small object smaller than the size corresponding to one pixel of the input image as the standard sample. Therefore, it is possible to obtain the most ideal input image for performing processing related to the present invention.
[0126]
(11) In the multidirectional image input device according to (1),
The standard sample is characterized in that it has a cross-sectional shape capable of obtaining the characteristics of the optical system of the multidirectional image input apparatus during observation.
[0127]
(12) In the multidirectional image input device according to (1),
The standard sample is characterized in that a cross section of the standard sample in a plane including two optical axes or a plane including the respective optical axes is circular, and is a sphere, a cone, or a cylindrical object.
The first embodiment corresponds to the embodiment relating to the present invention.
[0128]
According to the present invention, since a spherical object is considered to have a sufficient band in all directions of the 3D space, an image enlarged in size similar to the input image when a minute sample is used is observed. Therefore, the effect as a standard sample can be obtained. In addition, since the sample does not necessarily need to be very small, handling becomes simple and is practically advantageous.
[0129]
(13) In the multidirectional image input device according to (1),
The standard sample is the tip of a sharp object, and the diameter of the object is in a range smaller than the size of the object space corresponding to one pixel of the image recorded in the standard sample image recording means.
It is characterized by that.
[0130]
The first embodiment corresponds to the embodiment relating to the present invention.
[0131]
According to the present invention, in order to use the most ideal minute object as the standard sample, the tip of the sharp object is used. Even though the standard sample itself does not need to be minute, the minute object is used. It is possible to obtain the same operational effects as those obtained. Therefore, handling is simple and the most ideal standard sample image can be input.
[0132]
(14) A plurality of optical systems arranged to form an image of an object within an image input range in a space for observation from different directions, and a plurality of optical systems arranged on the image planes of the plurality of optical systems. A multi-directional image input device having an imaging element, a standard sample having a shape that can be placed within the image input range where an object is present at the time of image input and can be optically set for observation,
In-focus position control means that is provided in each of the plurality of optical systems and moves the position of the object surface in focus in the object space (in-focus position);
The image information of the standard sample acquired by the imaging device in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means, and the focus position when the image information is acquired A standard sample image recording means for recording information correspondingly;
A standard sample observation image space composed of images of a plurality of standard samples corresponding to different focus positions recorded in the standard sample image recording means while moving the focus position of the optical system by the focus position control means. From the positional relationship detection means for detecting the positional relationship of the standard sample observation image space corresponding to each of the plurality of optical systems,
Optical system position control means for controlling the spatial position of each of the plurality of optical systems based on the positional relation between the standard sample observation image spaces corresponding to each of the plurality of optical systems detected by the positional relation detection means;
A multi-directional image input device.
[0133]
The embodiment relating to this invention corresponds to the second embodiment.
[0134]
The present invention is based on the positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems detected by the positional relationship detection means, so that the plurality of optical elements are arranged such that the origin coincides with the center in the standard sample observation image space. The system is configured to finely adjust the spatial position of each system.
[0135]
Therefore, according to the present invention, the largest field angle size can be used when the input images from the microscopes are mutually used. That is, it is not necessary to shift the address of the input image when aligning the input image from each microscope, and the reconstructed image does not become smaller than the input image.
[0136]
(15) A multi-directional image input device including a plurality of optical systems arranged to form an image of an object from different directions and a plurality of image sensors arranged on the image planes of the plurality of optical systems. In
A container containing a fluorescent substance that is placed at a substantially position where an object is present when inputting an image;
Light source means for generating light for exciting the fluorescent substance in the container;
Optical means for guiding the excitation light to the first optical system so that the excitation light emitted by the light source means is converged in the container by using the first optical system which is one of the plurality of optical systems; ,
In-focus position control means that is provided in each of the plurality of optical systems and moves the position (focus position) of the focused object surface in the object space;
Recording the fluorescence image acquired by the imaging element in correspondence with the information on the focus position when capturing the fluorescence image in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means Fluorescent image recording means for
In the second optical system that is one of the plurality of optical systems and is different from the first optical system, the image is recorded on the fluorescent image recording means while the in-focus position of the optical system is moved by the in-focus position control means. A positional relationship detecting means for detecting a positional relationship between the first and second fluorescence observation image spaces from a fluorescence observation image space composed of a plurality of fluorescence images corresponding to different in-focus positions. Features.
[0137]
The fifth embodiment corresponds to the embodiment relating to the present invention.
[0138]
The present invention is similar to the case where a micro sample is placed in a sample space by irradiating a fluorescent material placed in the sample space with a laser beam converged using each optical imaging system in the multidirectional image input device. It is configured to bring about an effect. Therefore, according to the present invention, it is possible to obtain the same effect as that of installing a micro sample in the sample space by a simple method of installing a container into which the fluorescent material is injected in the sample space. become.
[0139]
(16) In the multidirectional image input device according to (3),
The light source means is a laser light source means for generating pulsed laser light having a wavelength twice the excitation wavelength of the fluorescent substance in the container.
It is characterized by that.
[0140]
The sixth embodiment corresponds to the embodiment relating to the present invention.
[0141]
In the present invention, when irradiating a fluorescent material placed in a sample space with a laser beam converged by using each optical imaging system in a multidirectional image input device, the laser light source has twice the excitation wavelength of the fluorescent material. By using a pulse laser beam having a wavelength, a two-photon excitation phenomenon is caused to realize an extremely small fluorescent spot. Therefore, according to the present invention, it is possible to obtain the same effect as that of installing a micro sample in the sample space by a simple method of installing a container into which the fluorescent material is injected in the sample space. At the same time, since an ideal minute fluorescent spot can be observed, positioning with excellent accuracy can be performed.
[0142]
(17) A multidirectional image input device including a plurality of optical systems arranged to form an image of an object from different directions and a plurality of imaging elements arranged on the image planes of the plurality of optical systems. In
A container into which a standard sample that generates fluorescence having a wavelength different from the fluorescence wavelength of the target is injected together with the target that generates fluorescence,
A first color filter that transmits the fluorescence wavelength of the object and a second color filter that transmits the fluorescence wavelength of the standard sample;
A color filter changing unit that is provided in each of the plurality of optical systems and installs one of the first and second color filters on the optical path of the optical system;
In-focus position control means that is provided in each of the plurality of optical systems and moves the position (focus position) of the focused object surface in the object space;
An image of the standard sample acquired by the imaging device and the image obtained in a state where the second color filter is installed on the optical path and the in-focus position of the optical system is set in a predetermined object space by the in-focus position control means. Standard sample image recording means for recording information corresponding to the in-focus position when
From a standard sample observation image space composed of a plurality of standard sample images corresponding to different focus positions recorded in the standard sample image recording means while moving the focus position of the optical system by the focus position control means, And a positional relationship detection means for detecting the positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems.
[0143]
The seventh embodiment corresponds to the embodiment relating to the present invention.
[0144]
The present invention uses a solution in which a standard sample that emits fluorescence having a wavelength different from the fluorescence wavelength of an object used for alignment with the fluorescent sample to be observed is mixed, and selectively transmits the fluorescence wavelength of the standard sample. In this configuration, only a standard sample image can be input using a filter. Therefore, according to the present invention, the configuration and operation in the sample space can be simplified. Further, since the time difference between the operation of inputting the standard sample image for the alignment and the operation of observing the target object can be shortened, more accurate alignment can be realized.
[0145]
【The invention's effect】
As described above in detail, according to the present invention, it can be applied even when the correlation between the input images is difficult because the aperture of each optical imaging system is small, and it does not depend on the spatial frequency characteristics of the target image. It is possible to provide a multi-directional image input apparatus that has a wide application range, is simple, and can be used practically.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration example of a multidirectional image input apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram showing a detailed configuration example of a standard sample setting unit shown in FIG.
FIG. 3 is a diagram showing the appearance of a pseudo micro sample at the tip of a sample having a sharp tip used in the present embodiment.
FIG. 4 is a diagram showing a concept of a sample space and a 3D observation image space in the present embodiment.
FIG. 5 is a diagram for explaining a technique for detecting peak coordinates from a 3D image of a minute sample in the present embodiment.
FIG. 6 is a diagram illustrating a configuration example of a multidirectional image input apparatus according to a second embodiment of the present invention.
FIG. 7 is a diagram showing a configuration example of a multidirectional image input apparatus as a third embodiment according to the present invention.
FIG. 8 is a diagram for explaining the operation of a multidirectional image input device according to a third embodiment.
FIG. 9 is a diagram showing a configuration example of a multidirectional image input apparatus as a fourth embodiment according to the present invention.
FIG. 10 is a diagram showing a configuration example of a multidirectional image input apparatus as a fifth embodiment according to the present invention.
FIG. 11 is a diagram illustrating a configuration example of a multidirectional image input apparatus according to a seventh embodiment of the present invention.
[Explanation of symbols]
1, 2, ... Microscope
3 ... Standard sample installation section
4 ... Processor
5, 6 ... Objective lens
7,8 ... Focus surface controller
9, 10 ... TV camera
12, 13 ... A / D converter
14, 15, 23, 25 ... memory
16 ... Cumulative adder
17, 18, 19 ... Controller
20 ... micro sample
21 ... Internal bus
22 ... CPU
24 ... Peak detector

Claims (9)

A plurality of optical systems arranged to form an image of an object within an image input range in a space for observation from different directions, and a plurality of image sensors arranged on the image planes of the plurality of optical systems In a multidirectional image input device having
A standard sample placed within the image input object is present at the time of image input,
An in-focus position control means for moving a position (in-focus position) of a focused object plane in the object space of the plurality of optical systems;
The image information of the standard sample acquired by the imaging device in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means, and the focus position when the image information is acquired A standard sample image recording means for recording information correspondingly;
From a standard sample observation image space composed of images of a plurality of standard samples corresponding to different focus positions recorded in the standard sample image recording means while moving the focus position of the optical system by the focus position control means. A positional relationship detecting means for detecting a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems ,
The positional relationship detecting means uses, for each of the plurality of optical systems, pixels on an XY plane using image information of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image recording means. A first peak position detecting means for detecting a peak position of the density value, and pixel density values of images of a plurality of standard samples corresponding to different in-focus positions at the peak position detected by the first peak position detecting means; Second peak position detection means for detecting a peak position in the Z direction, and an arrangement position of the plurality of optical systems, and a peak position corresponding to each optical system obtained by the second peak position detection means, By obtaining a relative positional relationship and determining that the peak position for each optical system obtained by the first and second peak position detecting means is at the same position, the plurality of optical systems corresponding to each of the plurality of optical systems. standard Multidirectional image input apparatus and obtains the positional relationship between the charge observation image space each other. A plurality of optical systems arranged to form an image of an object within an image input range in a space for observation from different directions, and a plurality of image sensors arranged on the image planes of the plurality of optical systems In a multidirectional image input device having
A standard sample arranged within the range of the image input in which the object is present at the time of image input;
An in-focus position control means for moving a position (in-focus position) of a focused object plane in the object space of the plurality of optical systems;
The image information of the standard sample acquired by the imaging device in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means, and the focus position when the image information is acquired A standard sample image recording means for recording information correspondingly;
From a standard sample observation image space composed of images of a plurality of standard samples corresponding to different focus positions recorded in the standard sample image recording means while moving the focus position of the optical system by the focus position control means. A positional relationship detecting means for detecting a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems,
The positional relationship detection means for each of the plurality of optical systems
Image adding means for adding image information of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image recording means;
First peak position detecting means for detecting a peak position of the pixel density value from the added image information;
Second peak position detecting means for detecting peak positions from pixel density values of a plurality of standard sample images corresponding to different in-focus positions at the peak positions detected by the first peak position detecting means; ,
A relative positional relationship between the arrangement positions of the plurality of optical systems and the peak positions corresponding to the respective optical systems obtained by the second peak detecting means is obtained, and the first and second peak position detecting means are provided. A multidirectional image input characterized in that a positional relationship between the standard sample observation image spaces corresponding to each of the plurality of optical systems is obtained by determining that the obtained peak positions for the respective optical systems are at the same position. apparatus. In the multi-directional image input device according to claim 1 or 2,
The focusing position control means is provided in each of the plurality of optical systems,
Optical system position control means for controlling the spatial position of each of the plurality of optical systems based on the positional relation between the standard sample observation image spaces corresponding to each of the plurality of optical systems detected by the positional relation detection means. A multi-directional image input device. In the multi-directional image input device according to any one of claims 1 to 3,
The multi-directional image input apparatus, wherein the standard sample is smaller than a size corresponding to one pixel of the image sensor in an object space of the plurality of optical systems. In the multi-directional image input device according to any one of claims 1 to 3,
The multi-directional image input device, wherein the standard sample is a microsphere infiltrated with a fluorescent dye. In the multi-directional image input device according to any one of claims 1 to 3,
The multi-directional image input apparatus according to claim 1, wherein the standard sample is a spherical object that is not so large as compared with an imaging range. In the multi-directional image input device according to any one of claims 1 to 3,
The multi-directional image input device, wherein the standard sample is a tip portion of a sample having a sharp tip. In a multidirectional image input device comprising a plurality of optical systems arranged to form an image of an object from different directions and a plurality of imaging elements arranged on the image planes of the plurality of optical systems,
A container containing a fluorescent material arranged at a substantially position where an object is present when inputting an image, and a light source means for generating light for exciting the fluorescent material in the container,
Using the first optical system that is one of the plurality of optical systems, the excitation light is caused to converge on the first optical system so that the excitation light emitted by the light source means is converged in the container. Optical means to guide;
In-focus position control means that is provided in each of the plurality of optical systems and moves the position of the object surface in focus in the object space (in-focus position);
Correspondence between the fluorescence image acquired by the imaging device and the information on the focus position when capturing the fluorescence image in a state where the focus position of the optical system is set in a predetermined object space by the focus position control means Fluorescent image recording means for recording,
In the second optical system that is one of the plurality of optical systems and is different from the first optical system, the in-focus position of the optical system is moved by the in-focus position control means while the fluorescent image is moved. The positional relationship between the fluorescence observation image spaces corresponding to the first and second optical systems is detected from the fluorescence observation image space composed of a plurality of fluorescence images corresponding to different in-focus positions recorded in the recording means. a positional relationship detecting means for the provided,
The positional relationship detecting means uses a plurality of fluorescent images corresponding to different in-focus positions recorded in the fluorescent image recording means for each of the first and second optical systems, so that the pixel density on the XY plane First peak position detecting means for detecting the peak position of the value, and pixel density values of a plurality of fluorescent images corresponding to different in-focus positions at the peak position detected by the first peak position detecting means in the Z direction. Second peak position detecting means for detecting the peak position, and the arrangement positions of the first and second optical systems, and the peak positions corresponding to the respective optical systems obtained by the second peak position detecting means, obtains the relative positional relationship, the peak positions for each optical system in which the first and second peak position detecting means is required by determining the in the same position, corresponding to each of the plurality of optical systems The fluorescence observation image sky Multidirectional image input apparatus characterized by obtaining the what was positional relationship. In a multidirectional image input device comprising a plurality of optical systems arranged to form an image of an object from different directions and a plurality of imaging elements arranged on the image planes of the plurality of optical systems,
A container into which a standard sample that generates fluorescence having a wavelength different from the fluorescence wavelength of the target is injected together with the target that generates fluorescence,
A first color filter that transmits the fluorescence wavelength of the object and a second color filter that transmits the fluorescence wavelength of the standard sample;
A color filter changing means provided in each of a plurality of optical systems, wherein one of the first and second color filters is installed on an optical path of the optical system;
In-focus position control means that is provided in each of the plurality of optical systems and moves the position of the object surface in focus in the object space (in-focus position);
An image of a standard sample acquired by the imaging element in a state where the second color filter is installed on the optical path and the focus position of the optical system is set in a predetermined object space by the focus position control means. A standard sample image recording means for recording information corresponding to the in-focus position at the time of capturing an image, and
From a standard sample observation image space composed of a plurality of standard sample images corresponding to different focus positions recorded in the standard sample image recording means while moving the focus position of the optical system by the focus position control means, A positional relationship detecting means for detecting a positional relationship between the standard sample observation image spaces corresponding to each of a plurality of optical systems ,
The positional relationship detecting means uses, for each of the plurality of optical systems, pixels on an XY plane using image information of a plurality of standard samples corresponding to different in-focus positions recorded in the standard sample image recording means. A first peak position detecting means for detecting a peak position of the density value, and pixel density values of images of a plurality of standard samples corresponding to different in-focus positions at the peak position detected by the first peak position detecting means; Second peak position detection means for detecting a peak position in the Z direction, and an arrangement position of the plurality of optical systems, and a peak position corresponding to each optical system obtained by the second peak position detection means, By obtaining a relative positional relationship and determining that the peak position for each optical system obtained by the first and second peak position detecting means is at the same position, the plurality of optical systems corresponding to each of the plurality of optical systems. standard Multidirectional image input apparatus and obtains the positional relationship between the charge observation image space each other.

Images