JP4334839B2 - Endoscope observation device - Google Patents

Description

である。
また、座標系をＸ、Ｙ、Ｚ方向に対して（ｘ、ｙ、ｚ）倍するマトリクスは、
【式２】

である。
また、座標系をｘ軸の周囲をａ（ラジアン）だけ回転させるマトリクスは、
【式３】

である。
【００７４】
そして、超音波画像データの座標系からＣＴ画像データの座標系への変換マトリクスをＭとすると、超音波画像データの座標系で内視鏡２の視線方向ベクトルＶ（ｘ、ｙ、ｚ、）をＣＴ画像データの座標系における視線方向ベクトルＶ’（ｘ’、ｙ’、ｚ’）に変換するには、
Ｖ’＝Ｍ×Ｖ ・・・（式４）
で求めることができる。
このことにより、マトリクス演算部５５は、ＣＴボリュームと超音波ボリューム生成部５２からの超音波ボリュームとの関係を算出することができる。
【００７５】
次に、観測装置本体６Ｂは、マトリクス演算部５５で算出した変換マトリクスに基づき、位置姿勢抽出部５３で抽出した内視鏡２又は生検鉗子８の３次元的位置姿勢から位置姿勢算出部５６で予め得たＣＴ画像データの座標系における内視鏡２又は生検鉗子８の位置姿勢を算出する（ステップＳ６５）。
【００７６】
次に、観測装置本体６Ｂは、位置姿勢算出部５６で算出したＣＴ画像データの座標系における内視鏡２又は生検鉗子８の位置姿勢をＣＴ画像合成部５７で予め得たＣＴ画像に合成してこのＣＴ合成画像をモニタ７に出力する。そして、モニタ７は、この表示面に表示されているＣＴ画像の代わりにＣＴ合成画像が表示される（ステップＳ６６）。
【００７７】
ここで、ＣＴ合成画像は、図１６に示すように内視鏡２又は生検鉗子８の位置姿勢がＸ記号６１と→６２で表示される。
尚、図１６に示すＣＴ合成画像は、多断面再構成ＭＰＲ（ Multi Planar Reconstrrution ）画像である。また、ＣＴ合成画像は、図１７に示すようなＭＩＰ（ Maximum Intensity Projection or Minimum Intensity Projection ）画像上に内視鏡２又は生検鉗子８の位置姿勢を合成するようにしても良い。
術者は、このモニタ７に表示されるＣＴ合成画像により、内視鏡２又は生検鉗子８の位置姿勢を確認可能である。
【００７８】
次に、生検について説明する。
生検鉗子８は、上述したように上記第１の実施の形態で説明したのと同様なものを使用する。そして、術者は、観測装置本体６Ｂによる図１８に示す生検のフローチャートに従って、生検を行う。
【００７９】
観測装置本体６Ｂは、内視鏡装置４（ＣＣＵ３）から観察画像（内視鏡画像）を内視鏡画像入力部１１で入力される（ステップＳ７１）。尚、このとき、観測装置本体６Ｂは、上記第１の実施の形態で説明したのと同様に観察画像（内視鏡画像）に対して画像補正部１４による光学特性の補正処理が行われている。
次に、観測装置本体６Ｂは、上記第１の実施の形態で説明したのと同様に観察画像（内視鏡画像）から鉗子識別部２３で生検鉗子８を識別する。
【００８０】
そして、観測装置本体６Ｂは、鉗子識別部２３で識別された生検鉗子８の種類に基づき、鉗子挿入長判定部２４ｂで、入力された超音波画像データから生検鉗子８の挿入長を判定する（ステップＳ７２）。尚、鉗子挿入長判定部２４ｂは、上記第１の実施の形態で説明したのと同様に観察画像（内視鏡画像）から生検鉗子８の色、又は目盛りを識別して生検鉗子８の挿入長を判定するようにしても良い。
【００８１】
そして、観測装置本体６Ｂは、鉗子挿入長判定部２４で判定した生検鉗子８の挿入長をＣＴ合成画像上に表示する（ステップＳ７３）。
また、生検鉗子８の生検カップ８ａがターゲット３１に近づき、内視鏡２の観察範囲から生検カップ８ａが外れて観察画像（内視鏡画像）から生検カップ８ａが見えなくなったとき、術者は、観測装置本体６Ｂのフロントパネルに設けた切換釦を押下操作して、上記第１の実施の形態で説明したのと同様に観察画像をＶＢＳ画像に切り換え、算出した生検鉗子８の挿入長に応じて更新されるＶＢＳ画像を表示させる。尚、観測装置本体６Ｂは、切換釦を再度、押下操作されると、ＶＢＳ画像から観察画像（内視鏡画像）に切り換えるようになっている。
【００８２】
そして、観測装置本体６Ｂは、上記第１の実施の形態で説明したのと同様に生検鉗子８の先端（生検カップ８ａ）の明暗（明るさ）を検出することで、生検鉗子８の生検カップ８ａがターゲット３１に到達したか否かを判断し、ターゲット３１に到達したと判断したときその結果をモニタ７に出力してＶＢＳ画像に重畳する。
【００８３】
そして、観測装置本体６Ｂは、術者による生検が終了するまで上記ステップＳ７４までを繰り返す。そして、術者は、モニタ７の表示面に表示されるＶＢＳ画像及びＣＴ合成画像を参照しながら、生検鉗子８を操作して生体組織を採取し、生検が終了する。
【００８４】
この結果、本第２の実施の形態の内視鏡観測装置１Ｂは、上記第１の実施の形態と同様な効果を得ることに加え、生検（生体検査）時にＣＴ撮影を行うことが無いので操作性が向上する。
【００８５】
尚、内視鏡観測装置は、図１９に示すように体外式超音波装置５０の代わりに内視鏡２の処置具挿通用チャンネルに挿入可能な超音波プローブを用いて構成しても良い。
図１９に示すように内視鏡観測装置１Ｃは、内視鏡２の処置具挿通用チャンネルに挿入して、体腔内から被検体の超音波断層像を撮像して超音波３次元画像（以下、超音波画像）データを得る超音波プローブ７０と、この超音波プローブ７０で得た超音波画像データから内視鏡２の位置姿勢を抽出し、予め得たＣＴ画像データの座標系における内視鏡２の位置姿勢を算出してＣＴ画像上に合成する処理を行う観測装置本体６Ｃとを有して構成されている。
【００８６】
前記超音波プローブ７０は、超音波照射・検出を連続的に回転させつつ被検体内を体軸方向に連続送りすることにより、被検体の３次元領域について螺旋状の連続スキャン（ヘリカルスキャン： helical scan ）を行い、３次元領域の連続するスライスの断層像から、３次元超音波画像データを生成するように構成されている。
【００８７】
前記観測装置本体６Ｃは、上記第１の実施の形態で説明したのと同様な前記内視鏡画像入力部１１と、前記内視鏡機種検出部１２と、前記内視鏡情報記録部１３と、前記画像補正部１４と、前記ＣＴ画像入力部１７と、前記切換部２１と、前記ＶＢＳ画像記録部２２と、前記鉗子識別部２３と、前記鉗子挿入長判定部２４と、前記画像参照部２５と、前記光検出部２６と、前記生検判定部２７とを有して構成される。
また、前記観測装置本体６Ｃは、上記第１の実施の形態で説明したのと同様な前記ＣＴボリューム生成部５４と、前記超音波ボリューム生成部５２と、前記マトリクス演算部５５とを有して構成される。
【００８８】
更に、前記観測装置本体６Ｃは、前記超音波プローブ７０と接続し、超音波断層画像を取得する超音波画像入力部５１ｃと、前記超音波ボリューム生成部５２で構築された超音波ボリュームを画像処理し、超音波画像データの座標系における内視鏡２の３次元的位置姿勢を抽出する位置姿勢抽出部５３ｃと、前記マトリクス演算部５５で算出した変換マトリクスに基づき、前記位置姿勢抽出部５３ｃで抽出した内視鏡２の３次元的位置姿勢から予め得たＣＴ画像データの座標系における内視鏡２の位置姿勢を算出する位置姿勢算出部５６ｃと、この位置姿勢算出部５６ｃで算出したＣＴ画像データの座標系における内視鏡２の位置姿勢をＣＴ画像上に合成し、モニタ７に出力するＣＴ画像合成部５７ｃとを有して構成される。
【００８９】
このように構成される内視鏡観測装置１Ｃは、上記第１の実施の形態で説明したの同様にナビユニットによるナビゲーションでターゲット近傍まで内視鏡２の挿入部先端部を導かれる。そして、内視鏡観測装置１Ｃは、内視鏡２の処置具挿通用チャンネルに超音波プローブ７０を挿入されて、図２０に示す生検位置姿勢算出のフローチャートに従って、超音波プローブ７０で得た超音波画像データから内視鏡２を抽出し、この内視鏡２の位置姿勢を算出して予め得たＣＴ画像上に合成する生検位置姿勢算出処理を行う。
【００９０】
先ず、観測装置本体６Ｃは、上記第２の実施の形態で説明したのと同様にＣＴ装置５のＷＳから予め得たターゲット３１までのＣＴ画像データをＣＴ画像入力部１７で入力され、この入力されたＣＴ画像データから３次元的なＣＴボリュームをＣＴボリューム生成部５４で生成する（ステップＳ６１’）。
【００９１】
次に、観測装置本体６Ｃは、図示しない入力装置により予め入力されたスキャン範囲を超音波プローブ７０に対して設定する（ステップＳ６７）。
そして、観測装置本体６Ｃは、超音波プローブ７０からの超音波画像データを超音波画像入力部５１ｃで入力され、３次元的な超音波ボリュームを超音波ボリューム生成部５２で生成する（ステップＳ６２’）。
【００９２】
次に、観測装置本体６Ｃは、上記第２の実施の形態で説明したのと同様に超音波ボリューム生成部５２で生成された超音波ボリュームを位置姿勢抽出部５３ｃで画像処理し、超音波画像データの座標系における内視鏡２の３次元的位置姿勢を抽出する（ステップＳ６３’）。
【００９３】
そして、観測装置本体６Ｃは、上記第２の実施の形態で説明したのと同様に超音波ボリューム生成部５２で生成された超音波ボリュームと、ＣＴボリューム生成部５４で生成されたＣＴボリュームとをマトリクス演算部５５でボリュームマッチングすることで、超音波座標系→ＣＴ座標系への変換マトリクスを算出する（ステップＳ６４’）。
【００９４】
次に、観測装置本体６Ｃは、マトリクス演算部５５で算出した変換マトリクスに基づき、位置姿勢抽出部５３ｃで抽出した内視鏡２の３次元的位置姿勢から位置姿勢算出部５６ｃで予め得たＣＴ画像データの座標系における内視鏡２の位置姿勢を算出する（ステップＳ６５’）。
【００９５】
次に、観測装置本体６Ｃは、位置姿勢算出部５６ｃで算出したＣＴ画像データの座標系における内視鏡２の位置姿勢をＣＴ画像合成部５７ｃで予め得たＣＴ画像に合成してこのＣＴ合成画像をモニタ７に出力する。そして、モニタ７は、この表示面に表示されているＣＴ画像の代わりにＣＴ合成画像が表示される（ステップＳ６６’）。
術者は、このモニタ７に表示されるＣＴ合成画像により、内視鏡２の位置姿勢を確認可能である。
【００９６】
次に、生検について説明する。
術者は、超音波プローブ７０を内視鏡２から引き抜き、代わりに生検鉗子８を挿入する。ここで、生検鉗子８は、上述したように上記第１の実施の形態で説明したのと同様なものを使用する。そして、術者は、観測装置本体６Ｃによる図２１に示す生検のフローチャートに従って、生検を行う。
観測装置本体６Ｃは、内視鏡装置４（ＣＣＵ３）から観察画像（内視鏡画像）を内視鏡画像入力部１１で入力される（ステップＳ７１’）。
【００９７】
次に、観測装置本体６Ｃは、上記第１の実施の形態で説明したのと同様に観察画像（内視鏡画像）から鉗子識別部２３で生検鉗子８を識別し（ステップＳ３２’）、鉗子挿入長判定部２４で、入力された観察画像（内視鏡画像）上から生検鉗子８の色又は目盛りを識別し（ステップＳ３３’）、この識別した結果に基づき、生検鉗子８の挿入長を判定する（ステップＳ３４’）。 ◎
そして、観測装置本体６Ｃは、鉗子挿入長判定部２４で判定した生検鉗子８の挿入長をＣＴ合成画像上に表示する（ステップＳ７３’）。
【００９８】
また、生検鉗子８の生検カップ８ａがターゲット３１に近づき、内視鏡２の観察範囲から生検カップ８ａが外れて観察画像（内視鏡画像）から生検カップ８ａが見えなくなったとき、術者は、観測装置本体６Ｃのフロントパネルに設けた切換釦を押下操作して、上記第１の実施の形態で説明したのと同様に観察画像をＶＢＳ画像に切り換え、算出した生検鉗子８の挿入長に応じて更新されるＶＢＳ画像を表示させる。
【００９９】
そして、観測装置本体６Ｃは、上記第１の実施の形態で説明したのと同様に生検鉗子８の先端（生検カップ８ａ）の明暗（明るさ）を検出することで、生検鉗子８の生検カップ８ａがターゲット３１に到達したか否かを判断し、ターゲット３１に到達したと判断したときその結果をモニタ７に出力してＶＢＳ画像に重畳する。
【０１００】
そして、観測装置本体６Ｃは、術者による生検が終了する（ステップＳ７４’）まで上記ステップＳ７４’までを繰り返す。そして、術者は、モニタ７の表示面に表示されるＶＢＳ画像及びＣＴ合成画像を参照しながら、生検鉗子８を操作して生体組織を採取し、生検が終了する。
【０１０１】
この結果、本変形例の内視鏡観測装置１Ｃは、上記第２の実施の形態と同様な効果を得ることに加え、超音波プローブ７０を内視鏡２に挿入することで、体腔内からの超音波画像データを得られ、より詳細な内視鏡２の位置姿勢を得ることができる。
尚、本発明は、以上述べた実施の形態のみに限定されるものではなく、発明の要旨を逸脱しない範囲で種々変形実施可能である。
【０１０２】
［付記］
（付記項１） 病変近傍まで達してこの病変近傍を観察する内視鏡を有し、この内視鏡で得た病変近傍の観察画像を得る内視鏡観測装置において、
前記内視鏡を用いて前記病変近傍に指標を配置する指標配置手段と、
前記指標配置手段により前記指標を配置した状態で、前記病変近傍と前記内視鏡とを含む領域の断層画像データを得る断層画像データ取得手段と、
前記断層画像データ取得手段で得た断層画像データに基づき、病変部と前記内視鏡との相対位置を特定する相対位置特定手段と、
を具備したことを特徴とする内視鏡観測装置。
【０１０３】
（付記項２） 病変近傍まで達してこの病変近傍を観察する内視鏡を有し、この内視鏡で得た病変近傍の観察画像を得る内視鏡観測装置において、
前記内視鏡を用いて前記病変近傍に指標を配置する指標配置手段と、
前記指標配置手段により前記指標を配置した状態で、前記病変近傍と前記内視鏡と前記指標とを含む領域の断層画像データを取得する断層画像データ取得手段と、
前記断層画像データ取得手段で得た断層画像データに基づき、病変部と前記内視鏡との相対位置を特定する相対位置特定手段と、
を具備したことを特徴とする内視鏡観測装置。
【０１０４】
（付記項３） 前記相対位置特定手段は、前記断層画像データから得た前記病変部の中心点と、前記内視鏡のベクトルとで前記病変部位と前記内視鏡との相対位置を特定することを特徴とする付記項１に記載の内視鏡観測装置。
【０１０５】
（付記項４） 前記相対位置特定手段は、前記断層画像データから得た前記病変部の中心点と、前記内視鏡の挿入部先端部の中心点と、前記指標の中心点とで前記病変部位と前記内視鏡との相対位置を特定することを特徴とする付記項２に記載の内視鏡観測装置。
【０１０６】
（付記項５） 病変近傍まで達してこの病変近傍を観察する内視鏡を有し、この内視鏡で得た病変近傍の観察画像を得る内視鏡観測装置において、
前記病変近傍の断層画像データを積層した第１の３次元データを取得する第１の３次元データ取得手段と、
前記病変近傍の超音波断層画像データを得て第２の３次元データを取得する第２の３次元データ取得手段と、
前記第１の３次元データ取得手段で得た前記第１の３次元データと前記第２の３次元データ取得手段で得た前記第２の３次元データとの相関を求める第１の相関手段と、
前記第２の３次元データ取得手段で得た前記第２の３次元データと前記内視鏡との相関を求める第２の相関手段と、
前記第１の相関手段及び前記第２の相関手段で得た相関関係に基づき、前記内視鏡の位置と前記第１の３次元データとを関連付ける関連付け手段と、
を具備したことを特徴とする内視鏡観測装置。
【０１０７】
【発明の効果】
以上説明したように本発明によれば、体腔内管路の末梢部における生診率（生体検査診断率）の向上化が可能な内視鏡観測装置を実現できる。
【図面の簡単な説明】
【図１】本発明の第１の実施の形態の内視鏡観測装置を示す全体構成図
【図２】図１の内視鏡観測装置で実行される生検メインのフローチャート
【図３】ランドマークを気管支のターゲット近傍の粘膜に貼り付けた際のモニタ表示例
【図４】図３の生検位置確認のフローチャート
【図５】観察画像（内視鏡画像）にターゲットが合成された合成画像のモニタ表示例
【図６】図３の対象の位置関係算出のフローチャート、図７は目盛りが形成されている生検鉗子を示す概略図
【図７】目盛りが形成されている生検鉗子を示す概略図
【図８】長さに応じて着色されている生検鉗子を示す概略図
【図９】図３の生検のフローチャート
【図１０】形状の異なるランドマークを最低２つターゲット近傍に貼り付けた際のモニタ表示
【図１１】変形例の生検位置確認のフローチャート
【図１２】本発明の第２の実施の形態の内視鏡観測装置を示す全体構成図
【図１３】図１２の内視鏡観測装置で実行される生検メインのフローチャート
【図１４】図１３の生検位置姿勢算出のフローチャート
【図１５】ＣＴボリュームと超音波ボリュームとのボリュームマッチングを示す概念図
【図１６】内視鏡又は生検鉗子の位置姿勢が合成されたＣＴ合成画像の表示例
【図１７】ＣＴ画像としてのＭＩＰ画像を示す表示例
【図１８】図３の生検のフローチャート
【図１９】変形例の内視鏡観測装置を示す全体構成図
【図２０】図１９の内視鏡観測装置で実行される生検位置姿勢算出のフローチャート
【図２１】図１９の内視鏡観測装置で実行される生検のフローチャート
【符号の説明】
１…内視鏡観測装置
２…内視鏡
３…ＣＣＵ
４…内視鏡装置
５…ＣＴ装置
６…観測装置本体
７…モニタ
８…生検鉗子（指標配置手段）
１０…ランドマーク
１１…内視鏡画像入力部
１２…内視鏡機種検出部
１３…内視鏡情報記録部
１４…画像補正部
１５…内視鏡側対象識別部
１６…内視鏡側位置算出部
１７…ＣＴ画像入力部（断層画像データ取得手段）
１８…ＣＴ側対象識別部
１９…ＣＴ側位置算出部（相対位置特定手段）
２０…画像合成部
２１…切換部
２２…ＶＢＳ画像記録部
２３…鉗子識別部
２４…鉗子挿入長判定部
２５…画像参照部
２６…光検出部
２７…生検判定部
３１…ターゲット
４１…光ファイバ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an endoscope observation apparatus, and more particularly to an endoscope observation apparatus used when performing a biopsy (biological examination) or the like in a body duct such as a bronchi.
[0002]
[Prior art]
In recent years, endoscopes have been widely used. An endoscope uses a treatment tool inserted into a treatment instrument insertion channel by observing an organ in a body cavity by inserting an elongated insertion portion into the body cavity, and using a treatment tool inserted into a treatment instrument insertion channel as necessary. Various treatments such as inspection) can be performed.
[0003]
In recent years, endoscopic examination has been performed to diagnose an affected area using three-dimensional image data obtained by capturing a tomographic image of a subject using, for example, a CT (Computed Tomography) apparatus.
[0004]
An endoscope observation apparatus used for such an endoscopy, for example, uses an endoscope (insertion unit thereof) up to a target site in a subject's internal duct by using a navigation image created based on the three-dimensional image data. ) Is guided.
The endoscope observation apparatus collects a sample of a biological tissue using a biopsy forceps or the like inserted through a treatment instrument insertion channel.
[0005]
[Problems to be solved by the invention]
However, in the above conventional endoscope observation apparatus, for example, at the time of biopsy (biological examination) in the bronchus, it is difficult for the insertion portion of the endoscope to be inserted to the peripheral portion of the lung. In this case, the conventional endoscope observation apparatus can reach the vicinity of the target site in the peripheral part of the lung, but it is difficult to reach the target site.
For this reason, the conventional endoscope observation apparatus performs a biopsy (biological examination) using the biopsy forceps or the like in a blind state in the vicinity of a target site on an endoscope observation image.
[0006]
The present invention has been made in view of the above-described problems, and provides an endoscope observation apparatus capable of improving a biopsy rate (biological examination diagnosis rate) in a peripheral portion of a body cavity duct.
[0007]
[Means for Solving the Problems]
  Of the present inventionFirst endoscope observation apparatusHas an endoscope that reaches the vicinity of the lesion and observes the vicinity of the lesion, and in an endoscope observation apparatus that obtains an observation image of the vicinity of the lesion obtained by the endoscope, the lesion using the endoscope An index placement means for placing an index in the vicinity, and the vicinity of the lesion and the endoscope in a state where the index is placed by the index placement meansThe indicator andTomographic image data of the area includinggetTomographic image data acquisition means and tomographic image data obtained by the tomographic image data acquisition meansBased on the centroid position information of the lesion and the centroid position information of the index obtained from the above, and the gaze direction information of the endoscope,The distal end of the endoscope reaching the vicinity of the lesionPerspectiveAnd relative position specifying means for specifying the lesion position when viewed fromFeatures.
  Of the present inventionSecond endoscope observation deviceHas an endoscope that reaches the vicinity of the lesion and observes the vicinity of the lesion, and in an endoscope observation apparatus that obtains an observation image of the vicinity of the lesion obtained by the endoscope, the lesion using the endoscope Index placement means for placing a marker in the vicinity, and tomographic image data acquisition for obtaining tomographic image data of a region including the vicinity of the lesion, the endoscope, and the marker in a state where the marker is placed by the marker placement means And relative position specifying means for specifying a lesion position when viewed from the distal end portion of the endoscope that has reached the vicinity of the lesion, based on the tomographic image data obtained by the tomographic image data acquisition means.The relative position specifying means specifies the lesion position based on the center of gravity of the lesion obtained from the tomographic image data, the center of gravity of the distal end portion of the endoscope, and the center of gravity of the index. To do.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
FIGS. 1 to 11 relate to a first embodiment of the present invention, FIG. 1 is an overall configuration diagram showing an endoscope observation apparatus according to the first embodiment of the present invention, and FIG. 2 is an internal view of FIG. FIG. 3 is an example of a monitor display when a landmark is attached to a mucous membrane near a bronchial target, FIG. 4 is a flowchart of biopsy position confirmation in FIG. 3, and FIG. Is a monitor display example of a composite image in which a target is combined with an observation image (endoscopic image), FIG. 6 is a flowchart for calculating the positional relationship of the target in FIG. 3, and FIG. 7 shows a biopsy forceps in which a scale is formed. FIG. 7A is a schematic view showing a biopsy forceps when the biopsy cup is closed, and FIG. 7B is a schematic view showing the biopsy forceps when the biopsy cup is opened. FIG. 8 is a schematic view showing biopsy forceps colored according to length. ) Is a schematic view showing the biopsy forceps when the biopsy cup is closed, FIG. 8B is a schematic view showing the biopsy forceps when the biopsy cup is open, and FIG. 9 is a biopsy of FIG. FIG. 10 is a monitor display example when at least two landmarks having different shapes are pasted in the vicinity of the target, and FIG. 11 is a flowchart of biopsy position confirmation in a modified example.
[0009]
In the present embodiment, a case where the present invention is applied to a bronchoscope will be described.
[0010]
As shown in FIG. 1, an endoscope observation apparatus 1 having a first embodiment of the present invention includes a bronchial endoscope (hereinafter simply referred to as an endoscope) 2 that can be inserted into a bronchus, and the endoscope. An endoscope apparatus 4 including a camera control unit (hereinafter referred to as CCU) 3 that generates a standard video signal by performing signal processing on an endoscopic image obtained by imaging means 2 (not shown); A CT apparatus 5 that obtains three-dimensional image (hereinafter referred to as CT image) data by capturing a tomographic image of the target, the position of the target (target part) extracted from the CT image data obtained by the CT apparatus 5, and the target (target) Observation that obtains the positional relationship with the endoscope 2 inserted in the vicinity of the part) and synthesizes the target (target part) with the endoscope image (hereinafter referred to as observation image) signal-processed by the CCU 3 From the main body 6 and the observation main body 6 It is mainly a monitor 7 for displaying the formed image.
[0011]
The endoscope 2 has an elongated and flexible insertion portion (not shown). Further, the endoscope 2 inserts a treatment tool such as a biopsy forceps 8 from a treatment tool insertion port (not shown), and a channel opening formed at the distal end portion of the insertion portion via a treatment tool insertion channel (not shown). The distal end side of the treatment tool protrudes from the base and can be biopsied.
[0012]
The endoscope 2 is provided with a light guide (not shown) that transmits illumination light. The light guide transmits illumination light from a light source (not shown). Illumination light transmitted from the light guide illuminates a subject such as an affected part from a distal end surface of an illumination window (not shown) fixed to the distal end portion of the insertion portion.
[0013]
For the illuminated subject, a subject image is taken into the endoscope 2 from an observation window (not shown) provided adjacent to the illumination window. The captured subject image is picked up by an image pickup device (not shown), subjected to photoelectric conversion, and converted into an image pickup signal. The imaging signal is transmitted to a signal cable (not shown) and output to the CCU 3.
[0014]
The CCU 3 processes an imaging signal from the imaging device of the endoscope 2 to generate a standard video signal, outputs the video signal to the monitor 7, and displays the display surface of the monitor 7. The observation image is displayed on the screen.
[0015]
Although not shown, the endoscope observation apparatus 1 is a navigation unit (hereinafter referred to as a navigation) that constructs a virtual endoscope image (hereinafter referred to as a VBS image) generated based on CT image data obtained in advance by the CT apparatus 5. Unit). Then, the navigation unit links the generated VBS image to the position of the endoscope 2 (the distal end portion of the insertion portion) and displays it on the monitor 7, and the endoscope 2 (the distal end portion of the insertion portion thereof) up to the vicinity of the target. ) Navigation.
[0016]
While the endoscope 2 is guided by a VBS image displayed on the monitor 7 by the navigation unit, for example, the endoscope 2 (the distal end portion of the insertion portion) reaches the vicinity of the target in the bronchus. ing. The endoscope observation apparatus 1 performs endoscope observation / treatment (biopsy or treatment) in the vicinity of the target.
[0017]
Here, the endoscope observation apparatus 1 is configured to collect a sample of a biological tissue with a biopsy forceps 8 or the like in order to confirm the target by biopsy (biological examination). .
[0018]
At this time, as will be described later, the endoscope observation apparatus 1 attaches a landmark 10 formed of a metal foil having biosafety such as gold or platinum to the vicinity of the target with a biopsy forceps 8 that is an indicator placement means. In addition (see FIG. 3), CT imaging is performed by the CT apparatus 5 near the target. The landmark 10 is formed of a metal foil capable of CT imaging, so that it can be easily distinguished from the organ in the body cavity and can be identified.
The endoscope observation apparatus 1 obtains the positional relationship between the landmark 10, the target, and the distal end of the insertion portion of the endoscope 2 from the CT image data, and synthesizes the target on the observation image to confirm the biopsy position. It is like that.
[0019]
Next, a detailed configuration of the observation apparatus body 6 will be described.
The observation apparatus main body 6 is connected to the endoscope apparatus 4 and an endoscope image input unit 11 that acquires an observation image, and an endoscope that detects an ID stored and held in the endoscope 2 being used. Based on the ID detected by the endoscope model detection unit 12, the endoscope information recording unit 13 that outputs the optical characteristics of the endoscope 2 recorded in advance based on the ID detected by the endoscope model detection unit 12, and the endoscope information recording Based on the optical characteristics from the unit 13, an image correction unit 14 that corrects distortions such as an angle of view and distortion of the observation image input by the endoscope image input unit 11, and an observation corrected by the image correction unit 14 An endoscope-side object identifying unit 15 for identifying the landmark 10 from the image; an endoscope-side position calculating unit 16 for calculating the positional relationship of the targets identified by the endoscope-side object identifying unit 15; Connected with the CT apparatus 5 as image data acquisition means The CT image input unit 17 that acquires CT image data from the WS (workstation) of the CT apparatus 5, and the target, landmark 10, and endoscope 2 of the CT image data input from the CT image input unit 17 The positional relationship between the CT-side object identifying unit 18 that identifies the distal end of the insertion unit, and the target, landmark 10, and distal end of the insertion unit of the endoscope 2 that are identified by the CT-side target identifying unit 18 as relative position specifying means. The calculated CT side position calculation unit 19, the positional relationship between the target of the CT image data calculated by the CT side position calculation unit 19, the landmark 10, and the endoscope 2 and the endoscope side position calculation unit 16 The position of the target at the viewpoint of the distal end portion of the insertion portion of the endoscope 2 is calculated from the positional relationship of the landmarks 10 in the observed image, and is synthesized on the observed image and the synthesized image is converted into the previous image. Constructed and an image synthesizing unit 20 to be output to the monitor 7.
[0020]
The observation apparatus body 6 is turned on by pressing a switching button provided on a front panel (not shown), and a switching unit 21 that switches an image displayed on the monitor 7 from an observation image to a VBS image, and the CT apparatus 5 in advance. A VBS image up to the target is generated based on the CT image data obtained in the above, and the recorded VBS image is recorded on the observation image input from the endoscope image input unit 11 via the switching unit 21 and the switching unit 21. Based on the type of the biopsy forceps 8 identified by the forceps identification unit 23 and the type of the biopsy forceps 8 identified by the forceps identification unit 23, the color or scale (see FIG. 8 and FIG. 9), the forceps insertion length determination unit 24 for determining the insertion length of the biopsy forceps 8 and the insertion length of the biopsy forceps 8 determined by the forceps insertion length determination unit 24 In the VBS image recording unit 22 Load the VBS image formed recording, configured to have an image reference unit 25 to be output to the monitor 7.
[0021]
The switching unit 21 is configured to detect and automatically turn on when the biopsy forceps 8 protrude from the distal end of the insertion unit from the observation image input by the endoscope image input unit 11. You may do it.
The observation device body 6 includes a light detection unit 26 such as a photodiode that detects the light and darkness (brightness) of the tip of the optical fiber connected to the biopsy forceps 8, and the light and darkness (brightness) detected by the light detection unit 26. ) To determine whether or not the tip of the biopsy forceps 8 has reached and contacted the target, and has a biopsy determination unit 27 that superimposes the result on the VBS image.
[0022]
The endoscope observation apparatus 1 configured as described above is used for endoscopic observation / treatment (biopsy or treatment) such as bronchial diseases.
First, the operator inserts the insertion portion of the endoscope 2 into the body cavity of the patient orally or nasally, and moves the endoscope 2 to a predetermined position that the operator understands, for example, the upper end (the larynx) of the trachea. Advance the tip of the insertion section. At this time, the surgeon inserts the insertion portion while viewing the observation image obtained by the endoscope 2 displayed on the monitor 7.
[0023]
Then, when the surgeon advances the distal end portion of the insertion portion of the endoscope 2 to a predetermined position, the navigation unit is started from here, and the target to the target is observed while referring to the VBS image displayed on the monitor 7. The insertion part of the endoscope 2 is inserted to the vicinity.
[0024]
When the distal end portion of the insertion portion of the endoscope 2 reaches the vicinity of the target, the surgeon performs a biopsy (biological examination) according to the biopsy main flowchart shown in FIG.
First, the operator uses the biopsy forceps 8 or the like inserted through the treatment instrument insertion channel of the endoscope 2 while viewing the observation image, for example, as shown in FIG. Affixing to the mucous membrane (step S1). At this time, although not shown, the monitor 7 displays only the observation image (endoscopic image) on the display surface.
[0025]
Next, the surgeon performs CT imaging of the vicinity of the target with the CT apparatus 5 in this state (a state in which the endoscope 2 is inserted) (step S2).
Then, the surgeon confirms the biopsy position according to the biopsy position confirmation flowchart shown in FIG. 4 described later by the observation apparatus body 6 (step S3). When the biopsy position is confirmed, the surgeon performs a biopsy according to the biopsy flowchart shown in FIG. 9 described later by the observation apparatus body 6 (step S4), and the biopsy is completed.
[0026]
Next, a flowchart of biopsy position confirmation shown in FIG. 4 will be described.
The observation apparatus main body 6 includes the position of the target extracted from the CT image data obtained in step S2 of the biopsy main in FIG. 2 and the distal end of the insertion portion of the endoscope 2 inserted in the vicinity of the target (target site). The biopsy position confirmation process which synthesize | combines a target with an observation image is calculated | required.
[0027]
First, the observation apparatus body 6 receives CT image data from the WS of the CT apparatus 5 via the CT image input unit 17 (step S11).
Next, the observation apparatus main body 6 identifies the target, the landmark 10, and the distal end portion of the insertion portion of the endoscope 2 from the CT image data input from the CT image input unit 17 with the CT-side object identification unit 18. (Step S12).
[0028]
Next, the observation apparatus body 6 calculates the positional relationship between the target identified by the CT-side object identifying unit 18, the landmark 10, and the distal end of the insertion portion of the endoscope 2 by the CT-side position calculating unit 19 (Step S <b> 13). ). Here, the CT side position calculation unit 19 calculates according to the flowchart of the positional relationship of the object shown in FIG.
[0029]
Next, the observation apparatus body 6 detects the ID stored and held in the endoscope 2 in use by the endoscope model detection unit 12 (step S14), and the ID detected by the endoscope model detection unit 12 Based on the above, the optical characteristics of the endoscope 2 recorded in advance by the endoscope information recording unit 13 are output to the image correcting unit 14.
[0030]
Next, the observation apparatus body 6 receives an observation image (endoscopic image) from the endoscope apparatus 4 (CCU3) through the endoscope image input unit 11 (step S15).
The observation apparatus body 6 includes an image correction unit 14 configured to display an observation image (endoscope image) input by the endoscope image input unit 11 based on the optical characteristics from the endoscope information recording unit 13. Optical characteristics (distortion) such as corners and distortion are corrected (step S16).
[0031]
Next, the observation apparatus body 6 identifies the landmark 10 as a target from the observation image (endoscopic image) corrected by the image correction unit 14 by using the endoscope-side target identification unit 15 (step S17). The positional relationship of the landmark 10 identified by the endoscope side target identifying unit 15 is calculated by the endoscope side position calculating unit 16 (step S18). Here, the endoscope side position calculation unit 16 calculates in accordance with the flowchart of the positional relationship of the object shown in FIG. 6 described later, similarly to the CT side position calculation unit 19.
[0032]
Then, the observation apparatus body 6 includes the positional relationship between the target of the CT image data calculated by the CT side position calculation unit 19, the landmark 10, and the endoscope 2 and the observation image calculated by the endoscope side position calculation unit 16. The position of the target at the viewpoint from the distal end portion of the insertion portion of the endoscope 2 is calculated from the positional relationship of the landmark 10 and synthesized on the observation image (endoscopic image) (step S19). The composite image is output to the monitor 7.
[0033]
As shown in FIG. 5, the monitor 7 displays a composite image in which the target 31 is semitransparently combined with the observation image (endoscopic image). The surgeon can confirm the biopsy position from the composite image displayed on the monitor 7. Here, in the composite image, if the position of the distal end portion of the insertion portion of the endoscope 2 is shifted due to camera shake or the like, the position of the target 31 becomes inaccurate, but one landmark 10 may be pasted.
[0034]
Next, with reference to FIG. 6, a flowchart for calculating the positional relationship of the target performed by the CT side position calculation unit 19 or the endoscope side position calculation unit 16 will be described. Here, for convenience, the CT side position calculation unit 19 will be described as an example.
The CT side position calculation unit 19 extracts the contours of the target 31, the landmark 10, and the distal end portion of the insertion portion of the endoscope 2 from the CT image data (step S21).
[0035]
Then, the CT side position calculation unit 19 identifies the target 31, the landmark 10, and the insertion portion distal end portion of the endoscope 2 from the extracted contour (step S22).
Then, the CT-side position calculation unit 19 calculates the center of gravity (position) and direction of the identified target 31, landmark 10, and distal end of the insertion portion of the endoscope 2 (step S23). The CT-side position calculation unit 19 calculates the direction of the distal end portion of the insertion portion of the endoscope 2 from the center of gravity of the distal end portion of the insertion portion of the endoscope 2 and its proximal end side, and the endoscope 2 The tip of the insertion part is obtained as a vector.
[0036]
Thereby, the CT side position calculation unit 19 can calculate the positional relationship between the target 31, the landmark 10, and the distal end portion of the insertion portion of the endoscope 2.
The endoscope side position calculation unit 16 is substantially the same as the CT side position calculation unit 19 described above, and only calculates the positional relationship of the landmark 10 as a target.
[0037]
Next, biopsy will be described.
First, the biopsy forceps 8 used in the present embodiment will be described with reference to FIGS.
In the biopsy forceps 8 used in this embodiment, a light incident end face 41a of an optical fiber 41 for detecting reflected light from a target 31 is attached to a biopsy cup 8a. The proximal end side of the optical fiber 41 is connected to the observation apparatus main body 6, and the light from the light emitting end face of the optical fiber 41 is received by the light detection unit 26 (see FIG. 1).
[0038]
Further, the biopsy forceps 8 is, for example, formed with a scale as shown in FIG. 7 or colored according to the length as shown in FIG. 8 so as to know the insertion length.
7A and 8A show the biopsy forceps 8 when the biopsy cup 8a is closed, and FIGS. 7B and 8B show the biopsy cup 8a opened. The biopsy forceps 8 is shown.
[0039]
Using such a biopsy forceps 8, the operator inserts the biopsy forceps 8 from the treatment instrument insertion port, and projects the distal end side from the channel opening at the distal end of the insertion section through the treatment instrument insertion channel. The surgeon performs biopsy according to the biopsy flowchart shown in FIG.
[0040]
The observation apparatus body 6 receives an observation image (endoscopic image) from the endoscope apparatus 4 (CCU3) through the endoscope image input unit 11 (step S31). At this time, in the observation apparatus main body 6, as described above, the optical characteristic correction processing by the image correction unit 14 is performed on the observation image (endoscopic image).
[0041]
Next, the observation apparatus body 6 identifies the biopsy forceps 8 by the forceps identification unit 23 from the observation image (endoscopic image) input via the switching unit 21 (step S32). At this time, in the observation apparatus main body 6, as described above, the positional relationship calculation processing of the biopsy forceps 8 by the endoscope side position calculation unit 16 is performed from the observation image (endoscopic image).
[0042]
Based on the type of biopsy forceps 8 identified by the forceps identification unit 23, the observation device body 6 determines the biopsy forceps 8 from the input observation image (endoscopic image) by the forceps insertion length determination unit 24. Is identified (step S33), and the insertion length of the biopsy forceps 8 is determined based on the identified result (step S34).
[0043]
Here, the biopsy cup 8a of the biopsy forceps 8 approaches the target 31, the biopsy cup 8a is removed from the observation range of the endoscope 2, and the biopsy cup 8a is not visible from the observation image (endoscopic image). At this time, the surgeon depresses a switching button provided on the front panel of the observation apparatus body 6.
[0044]
Then, the observation apparatus body 6 receives the switching signal of the switching button, and the switching unit 21 switches the image displayed on the monitor 7 from the observation image to the VBS image. Note that the observation device body 6 switches from the VBS image to the observation image (endoscopic image) when the switching button is pressed again.
[0045]
Then, in the observation apparatus main body 6, the VBS image recorded in the VBS image recording unit 22 is read and updated by the image reference unit 25 according to the calculated insertion length of the biopsy forceps 8 (step S <b> 35). 7 is output. That is, the VBS image displayed on the display surface of the monitor 7 is a viewpoint from the tip of the biopsy forceps 8 (biopsy cup 8a) and changes according to the insertion length.
[0046]
Then, the observation apparatus body 6 uses the biopsy determination unit 27 to determine whether or not the output of the light detection unit 26 that detects the brightness (brightness) of the tip (biopsy cup 8a) of the biopsy forceps 8 is equal to or less than a threshold value. Determination is made (step S36).
[0047]
When the output of the light detection unit 26 is equal to or less than the threshold value, the biopsy determination unit 27 determines that the biopsy cup 8a of the biopsy forceps 8 has reached the target 31, and outputs the result to the monitor 7 to output VBS. Superimpose on the image.
On the other hand, when the output of the light detection unit 26 exceeds the threshold value, the biopsy determination unit 27 determines that the biopsy cup 8 a of the biopsy forceps 8 has not reached the target 31.
[0048]
Then, the observation apparatus body 6 repeats the steps from S31 to S35 until the output (brightness / darkness) of the light detection unit 26 becomes equal to or less than the threshold value. Then, the surgeon operates the biopsy forceps 8 to collect the biological tissue while referring to the VBS image displayed on the display surface of the monitor 7, and the biopsy is completed.
[0049]
Accordingly, the endoscope observation apparatus is difficult to insert the distal end portion of the insertion portion of the endoscope 2, that is, exists in the peripheral portion of the body cavity duct that is out of the observation range (observation image) of the endoscope 2. The biopsy forceps 8 can be made to reach the target 31 that is present, and biopsy can be performed.
As a result, the endoscope observation apparatus 1 according to the present embodiment has an effect that the biopsy rate (biological examination diagnosis rate) in the peripheral part of the body cavity duct is improved.
[0050]
The endoscope observation apparatus 1 corrects a positional deviation of the distal end portion of the insertion portion of the endoscope 2 at the time of CT imaging and biopsy position confirmation by installing a plurality of individually identifiable landmarks 10. It may be configured as possible.
In this case, as shown in FIG. 10, the surgeon attaches at least two landmarks 10 having different shapes in the vicinity of the target. The shape of the landmark 10 may be any shape as long as it can be distinguished from each other on the CT image data.
[0051]
And the observation apparatus main body 6 performs the biopsy position confirmation process according to the flowchart of biopsy position confirmation shown in FIG.
Here, the confirmation process of the biopsy position is performed by always detecting the positional relationship between the two landmarks 10 on the observation image (endoscopic image), so that the deviation amount of the distal end portion of the insertion portion of the endoscope 2 is detected. Are sequentially detected and the composite image is updated. The rest is the same as that described in the first embodiment.
[0052]
First, the observation apparatus body 6 receives CT image data from the WS of the CT apparatus 5 via the CT image input unit 17 (step S11 ').
Next, the observation apparatus body 6 sets the target 31, two landmarks 10, and the insertion portion distal end portion of the endoscope 2 from the CT image data input from the CT image input unit 17 to the CT side target identification unit 18. (Step S12 ′).
[0053]
Next, the observation apparatus main body 6 calculates the positional relationship between the target 31, the two landmarks 10 identified by the CT-side object identification unit 18, and the distal end of the insertion unit of the endoscope 2 by the CT-side position calculation unit 19. (Step S13 ′).
Next, the observation apparatus body 6 detects the ID stored and held in the endoscope 2 in use by the endoscope model detection unit 12 (step S14 ′) and detects the ID by the endoscope model detection unit 12. Based on the ID, the endoscope information recording unit 13 outputs the optical characteristics of the endoscope 2 recorded in advance to the image correction unit 14.
[0054]
Next, the observation apparatus body 6 receives an observation image (endoscopic image) from the endoscope apparatus 4 (CCU3) through the endoscope image input unit 11 (step S15 ').
The observation apparatus body 6 includes an image correction unit 14 configured to display an observation image (endoscope image) input by the endoscope image input unit 11 based on the optical characteristics from the endoscope information recording unit 13. Optical characteristics (distortion) such as corners and distortion are corrected (step S16 ′).
[0055]
Next, the observation apparatus body 6 identifies the two landmarks 10 as targets from the observation image (endoscopic image) corrected by the image correction unit 14 by the endoscope-side target identification unit 15 (step S17 ′). Then, the positional relationship between the two landmarks 10 identified by the endoscope-side object identifying unit 15 is calculated by the endoscope-side position calculating unit 16 (step S18 ′).
[0056]
The observation apparatus main body 6 is calculated by the positional relationship between the target 31, the two landmarks 10, and the endoscope 2 of the CT image data calculated by the CT side position calculation unit 19 and the endoscope side position calculation unit 16. The position of the target 31 at the viewpoint from the distal end portion of the insertion portion of the endoscope 2 is calculated from the positional relationship between the two landmarks 10 of the observed image and synthesized on the observation image (endoscopic image) (step In step S19 ′, the composite image is output to the monitor 7. And the observation apparatus main body 6 repeats a process from said step S15 'until biopsy completion (step S40).
[0057]
As a result, the endoscope observation apparatus 1 according to the present modification is configured so that the observation apparatus body 6 always detects the positional relationship between the two landmarks 10 on the observation image (endoscope image). It is possible to sequentially detect the shift amount of the distal end portion of the insertion portion and update the composite image.
[0058]
(Second Embodiment)
FIGS. 12 to 21 relate to a second embodiment of the present invention, FIG. 12 is an overall configuration diagram showing an endoscope observation apparatus of the second embodiment of the present invention, and FIG. 13 is an internal view of FIG. FIG. 14 is a flowchart of biopsy position and orientation calculation in FIG. 13, FIG. 15 is a conceptual diagram showing volume matching between a CT volume and an ultrasonic volume, and FIG. FIG. 17 is a display example showing a MIP image as a CT image, FIG. 18 is a flowchart of the biopsy in FIG. 3, and FIG. 19 is a modified example. FIG. 20 is a flowchart of the biopsy position / orientation calculation executed by the endoscope observation apparatus in FIG. 19, and FIG. 21 is a raw view executed by the endoscope observation apparatus in FIG. It is an inspection flowchart.
[0059]
In the second embodiment, the position and orientation (position and inclination) of the endoscope 2 or the biopsy forceps 8 are obtained from ultrasonic three-dimensional image data obtained by imaging an ultrasonic tomogram during biopsy (biological examination). Is configured to calculate. Since other configurations are the same as those of the first embodiment, description thereof will be omitted, and the same components will be described with the same reference numerals.
[0060]
That is, as shown in FIG. 12, the endoscope observation apparatus 1B according to the second embodiment picks up an ultrasonic tomographic image of a subject from outside the body to obtain ultrasonic three-dimensional image (hereinafter referred to as ultrasonic image) data. And the position and orientation of the endoscope 2 or biopsy forceps 8 from the ultrasonic image data obtained by the extracorporeal ultrasonic device 50, and the coordinate system of CT image data obtained in advance. And an observation apparatus body 6B that performs processing for calculating the position and orientation of the endoscope 2 or the biopsy forceps 8 and synthesizing the position and orientation on the CT image.
[0061]
The endoscope observation apparatus 1B is connected to the navigation unit in the same manner as described in the first embodiment, and is configured to perform navigation of the insertion portion of the endoscope 2 to the vicinity of the target. Yes. The endoscope observation apparatus 1B performs endoscope observation / treatment (biopsy or treatment) in the vicinity of the target.
[0062]
At this time, as will be described later, the endoscope observation apparatus 1 </ b> B performs ultrasonic imaging of the vicinity of the target with the external ultrasonic apparatus 50. Then, the endoscope observation apparatus 1B extracts the position and orientation of the endoscope 2 or the biopsy forceps 8 from the ultrasonic image data obtained by the extracorporeal ultrasonic apparatus 50, and in the coordinate system of the CT image data obtained in advance. The position and orientation of the endoscope 2 or biopsy forceps 8 are calculated and synthesized on the CT image so that the biopsy position and orientation can be confirmed.
[0063]
Next, a detailed configuration of the observation apparatus body 6B will be described.
The observation apparatus main body 6B includes the endoscope image input unit 11, the endoscope model detection unit 12, the endoscope information recording unit 13, and the like described in the first embodiment. The image correction unit 14, the CT image input unit 17, the switching unit 21, the VBS image recording unit 22, the forceps identification unit 23, the image reference unit 25, and the light detection unit 26. The biopsy determination unit 27 is configured.
[0064]
Further, the observation apparatus main body 6B is connected to the extracorporeal ultrasonic apparatus 50 as a tomographic image data acquisition means, and an ultrasonic image input unit 51 for acquiring an ultrasonic tomographic image, and an input by the ultrasonic image input unit 51. An ultrasonic volume generation unit 52 for constructing a three-dimensional ultrasonic volume from the obtained ultrasonic image data, and image processing is performed on the ultrasonic volume constructed by the ultrasonic volume generation unit 52, and coordinates of the ultrasonic image data are obtained. A position / orientation extraction unit 53 that extracts a three-dimensional position / orientation of the endoscope 2 or the biopsy forceps 8 in the system, and three-dimensional data from the CT image data obtained in advance by the CT image input unit 17 up to the target 31. CT volume generation unit 54 for constructing a typical CT volume, the CT volume from the CT volume generation unit 54, and the ultrasonic volume generation unit 52 Based on the matrix calculation unit 55 that calculates a conversion matrix from an ultrasonic coordinate system to a CT coordinate system as a relationship with the ultrasonic volume, and the position and orientation based on the conversion matrix calculated by the matrix calculation unit 55 as a relative position specifying unit Position and orientation calculation for calculating the position and orientation of the endoscope 2 or biopsy forceps 8 in the coordinate system of CT image data obtained in advance from the three-dimensional position and orientation of the endoscope 2 or biopsy forceps 8 extracted by the extraction unit 53 Unit 56 and the CT image composition unit 57 for synthesizing the position and orientation of the endoscope 2 or biopsy forceps 8 in the coordinate system of the CT image data calculated by the position and orientation calculation unit 56 on the CT image and outputting them to the monitor 7 And is configured.
In the present embodiment, the forceps insertion length determination unit 24b determines the insertion length of the biopsy forceps 8 from the ultrasound image data input by the ultrasound image input unit 51.
[0065]
In the endoscope observation apparatus 1B configured as described above, the distal end portion of the insertion portion of the endoscope 2 is guided to the vicinity of the target by navigation by the navigation unit as described in the first embodiment. Then, the surgeon performs a biopsy (biological examination) according to the main biopsy flowchart shown in FIG.
[0066]
First, the operator operates the extracorporeal ultrasonic device 50 with the endoscope 2 inserted, and performs ultrasound imaging while performing the biopsy position / posture calculation flowchart shown in FIG. Accordingly, the biopsy position and orientation are confirmed (step S51). At this time, although not shown, the monitor 7 displays, for example, two images, that is, an observation image (endoscopic image) and CT image data obtained in advance in parallel on the display surface. While confirming this biopsy position and orientation, the surgeon performs a biopsy according to a biopsy flowchart shown in FIG. 18 described later by the observation apparatus body 6B (step S52), and the biopsy is completed.
[0067]
Next, a flowchart of biopsy position / posture calculation shown in FIG. 14 will be described.
The observation apparatus body 6B extracts the endoscope 2 or the biopsy forceps 8 from the ultrasonic image data obtained by the external ultrasonic apparatus 50, and calculates the position and orientation of the endoscope 2 or the biopsy forceps 8. A biopsy position / orientation calculation process is performed on a CT image obtained in advance.
[0068]
First, the CT image input unit 17 inputs CT image data from the WS of the CT apparatus 5 to the target 31 obtained in advance in the observation apparatus body 6B. Then, the observation apparatus body 6B generates a three-dimensional CT volume from the CT image data input from the CT image input unit 17 using the CT volume generation unit 54 (step S61).
[0069]
Next, in the observation apparatus body 6 </ b> B, ultrasonic image data from the external ultrasonic apparatus 50 is input by the ultrasonic image input unit 51. Then, the observation apparatus body 6B generates a three-dimensional ultrasonic volume from the ultrasonic image data input from the ultrasonic image input unit 51 using the ultrasonic volume generation unit 52 (step S62).
[0070]
Next, the observation apparatus body 6B performs image processing on the ultrasonic volume generated by the ultrasonic volume generation unit 52 by the position and orientation extraction unit 53, and the endoscope 2 or the biopsy forceps 8 in the coordinate system of the ultrasonic image data. Are extracted (step S63).
[0071]
Then, the observation apparatus body 6B performs volume matching between the ultrasonic volume generated by the ultrasonic volume generation unit 52 and the CT volume generated by the CT volume generation unit 54 as shown in FIG. Thus, a conversion matrix from the ultrasonic coordinate system to the CT coordinate system is calculated, and the relationship between the CT volume and the ultrasonic volume from the ultrasonic volume generation unit 52 is calculated (step S64).
[0072]
Here, the conversion matrix is a matrix for converting coordinates from the ultrasonic coordinate system to the CT coordinate system, and includes rotation, scale, and translation components.
The rotation and scale can be represented by a 3 × 3 matrix, but here, it is represented by a 4 × 4 matrix including translation.
[0073]
A matrix that translates the coordinate system by (x, y, z) is
[Formula 1]

It is.
In addition, a matrix for multiplying the coordinate system by (x, y, z) in the X, Y, and Z directions is
[Formula 2]

It is.
Also, the matrix that rotates the coordinate system around the x axis by a (radians) is
[Formula 3]

It is.
[0074]
If the conversion matrix from the coordinate system of the ultrasonic image data to the coordinate system of the CT image data is M, the line-of-sight direction vector V (x, y, z,) of the endoscope 2 in the coordinate system of the ultrasonic image data. Is converted into a line-of-sight direction vector V ′ (x ′, y ′, z ′) in the coordinate system of CT image data.
V ′ = M × V (Formula 4)
Can be obtained.
Accordingly, the matrix calculation unit 55 can calculate the relationship between the CT volume and the ultrasonic volume from the ultrasonic volume generation unit 52.
[0075]
Next, the observation apparatus main body 6 </ b> B is based on the conversion matrix calculated by the matrix calculation unit 55, and the position / orientation calculation unit 56 from the three-dimensional position / orientation of the endoscope 2 or the biopsy forceps 8 extracted by the position / orientation extraction unit 53. The position / orientation of the endoscope 2 or the biopsy forceps 8 in the coordinate system of the CT image data obtained in advance is calculated (step S65).
[0076]
Next, the observation apparatus body 6B synthesizes the position and orientation of the endoscope 2 or the biopsy forceps 8 in the coordinate system of the CT image data calculated by the position and orientation calculation unit 56 with the CT image obtained in advance by the CT image synthesis unit 57. Then, this CT composite image is output to the monitor 7. The monitor 7 displays a CT composite image instead of the CT image displayed on the display surface (step S66).
[0077]
Here, in the CT composite image, the position and orientation of the endoscope 2 or the biopsy forceps 8 are displayed as X symbols 61 and → 62 as shown in FIG.
The CT composite image shown in FIG. 16 is a multi-section reconstruction MPR (Multi Planar Reconstrrution) image. Further, the CT synthesized image may be obtained by synthesizing the position and orientation of the endoscope 2 or the biopsy forceps 8 on a MIP (Maximum Intensity Projection or Minimum Intensity Projection) image as shown in FIG.
The operator can confirm the position and orientation of the endoscope 2 or the biopsy forceps 8 from the CT composite image displayed on the monitor 7.
[0078]
Next, biopsy will be described.
As described above, the biopsy forceps 8 is the same as that described in the first embodiment. Then, the surgeon performs a biopsy according to the biopsy flowchart shown in FIG. 18 by the observation apparatus body 6B.
[0079]
The observation apparatus body 6B receives an observation image (endoscopic image) from the endoscope apparatus 4 (CCU3) through the endoscope image input unit 11 (step S71). At this time, in the observation apparatus body 6B, the optical characteristic correction processing by the image correction unit 14 is performed on the observation image (endoscopic image) as described in the first embodiment. Yes.
Next, the observation device main body 6B identifies the biopsy forceps 8 by the forceps identification unit 23 from the observation image (endoscopic image) in the same manner as described in the first embodiment.
[0080]
The observation device body 6B determines the insertion length of the biopsy forceps 8 from the input ultrasonic image data by the forceps insertion length determination unit 24b based on the type of the biopsy forceps 8 identified by the forceps identification unit 23. (Step S72). The forceps insertion length determination unit 24b identifies the color or scale of the biopsy forceps 8 from the observation image (endoscopic image) in the same manner as described in the first embodiment, and biopsy forceps 8 The insertion length may be determined.
[0081]
Then, the observation apparatus body 6B displays the insertion length of the biopsy forceps 8 determined by the forceps insertion length determination unit 24 on the CT composite image (step S73).
Further, when the biopsy cup 8a of the biopsy forceps 8 approaches the target 31, the biopsy cup 8a is removed from the observation range of the endoscope 2, and the biopsy cup 8a becomes invisible from the observation image (endoscopic image). The surgeon depresses a switching button provided on the front panel of the observation apparatus main body 6B to switch the observation image to the VBS image and calculates the biopsy forceps in the same manner as described in the first embodiment. The VBS image updated according to the insertion length of 8 is displayed. Note that the observation apparatus body 6B switches from the VBS image to the observation image (endoscopic image) when the switching button is pressed again.
[0082]
Then, the observation device main body 6B detects the brightness (brightness) of the tip (biopsy cup 8a) of the biopsy forceps 8 in the same manner as described in the first embodiment, so that the biopsy forceps 8 It is determined whether the biopsy cup 8a has reached the target 31, and when it is determined that the target 31 has been reached, the result is output to the monitor 7 and superimposed on the VBS image.
[0083]
And the observation apparatus main body 6B repeats to said step S74 until the biopsy by an operator is complete | finished. Then, the surgeon operates the biopsy forceps 8 while referring to the VBS image and the CT composite image displayed on the display surface of the monitor 7, and the biopsy is completed.
[0084]
As a result, the endoscope observation apparatus 1B according to the second embodiment obtains the same effect as that of the first embodiment, and does not perform CT imaging during a biopsy (biological examination). Therefore, operability is improved.
[0085]
The endoscope observation apparatus may be configured using an ultrasonic probe that can be inserted into the treatment instrument insertion channel of the endoscope 2 instead of the extracorporeal ultrasonic apparatus 50 as shown in FIG.
As shown in FIG. 19, the endoscope observation apparatus 1C is inserted into a treatment instrument insertion channel of the endoscope 2, picks up an ultrasonic tomographic image of the subject from inside the body cavity, and obtains an ultrasonic three-dimensional image (hereinafter referred to as an ultrasonic tomographic image). , Ultrasonic image) The ultrasonic probe 70 for obtaining data, and the position and orientation of the endoscope 2 are extracted from the ultrasonic image data obtained by the ultrasonic probe 70, and the endoscope in the coordinate system of the CT image data obtained in advance An observation apparatus main body 6C that performs a process of calculating the position and orientation of the mirror 2 and combining it on the CT image is configured.
[0086]
The ultrasonic probe 70 continuously feeds the inside of the subject in the body axis direction while continuously rotating and irradiating and detecting ultrasonic waves, so that a helical continuous scan (helical scan: helical) is performed on the three-dimensional region of the subject. scan) to generate 3D ultrasonic image data from tomographic images of successive slices of the 3D region.
[0087]
The observation apparatus main body 6C includes the endoscope image input unit 11, the endoscope model detection unit 12, the endoscope information recording unit 13, and the like described in the first embodiment. The image correction unit 14, the CT image input unit 17, the switching unit 21, the VBS image recording unit 22, the forceps identification unit 23, the forceps insertion length determination unit 24, and the image reference unit 25, the light detection unit 26, and the biopsy determination unit 27.
The observation apparatus main body 6C includes the CT volume generation unit 54, the ultrasonic volume generation unit 52, and the matrix calculation unit 55 similar to those described in the first embodiment. Composed.
[0088]
Further, the observation apparatus main body 6C is connected to the ultrasonic probe 70, and performs image processing on the ultrasonic volume constructed by the ultrasonic image input unit 51c for acquiring an ultrasonic tomographic image and the ultrasonic volume generation unit 52. The position and orientation extraction unit 53 c extracts the three-dimensional position and orientation of the endoscope 2 in the coordinate system of the ultrasonic image data, and the position and orientation extraction unit 53 c based on the conversion matrix calculated by the matrix calculation unit 55. A position / orientation calculation unit 56c for calculating the position / orientation of the endoscope 2 in the coordinate system of CT image data obtained in advance from the extracted three-dimensional position / orientation of the endoscope 2, and a CT calculated by the position / orientation calculation unit 56c. A CT image synthesizing unit 57 c that synthesizes the position and orientation of the endoscope 2 in the coordinate system of the image data on the CT image and outputs it to the monitor 7 is configured.
[0089]
In the endoscope observation apparatus 1C configured as described above, the distal end portion of the insertion portion of the endoscope 2 is guided to the vicinity of the target by navigation by the navigation unit as described in the first embodiment. The endoscope observation apparatus 1C was obtained with the ultrasonic probe 70 in accordance with the biopsy position / posture calculation flowchart shown in FIG. 20 with the ultrasonic probe 70 inserted into the treatment instrument insertion channel of the endoscope 2. A biopsy position / orientation calculation process is performed in which the endoscope 2 is extracted from the ultrasound image data, the position / orientation of the endoscope 2 is calculated and synthesized on a CT image obtained in advance.
[0090]
First, in the observation apparatus body 6C, CT image data from the WS of the CT apparatus 5 to the target 31 obtained in advance is input by the CT image input unit 17 in the same manner as described in the second embodiment. A three-dimensional CT volume is generated from the CT image data thus obtained by the CT volume generation unit 54 (step S61 ′).
[0091]
Next, the observation apparatus body 6C sets a scan range input in advance by an input device (not shown) for the ultrasonic probe 70 (step S67).
The observation apparatus main body 6C receives the ultrasonic image data from the ultrasonic probe 70 by the ultrasonic image input unit 51c, and generates a three-dimensional ultrasonic volume by the ultrasonic volume generation unit 52 (step S62 ′). ).
[0092]
Next, in the same manner as described in the second embodiment, the observation apparatus main body 6C performs image processing on the ultrasonic volume generated by the ultrasonic volume generation unit 52 by the position / orientation extraction unit 53c, and generates an ultrasonic image. The three-dimensional position and orientation of the endoscope 2 in the data coordinate system are extracted (step S63 ').
[0093]
Then, the observation apparatus main body 6C uses the ultrasonic volume generated by the ultrasonic volume generation unit 52 and the CT volume generated by the CT volume generation unit 54 in the same manner as described in the second embodiment. The matrix calculation unit 55 performs volume matching to calculate a conversion matrix from the ultrasonic coordinate system to the CT coordinate system (step S64 ′).
[0094]
Next, the observation apparatus main body 6C uses the CT obtained in advance by the position / orientation calculation unit 56c from the three-dimensional position / orientation of the endoscope 2 extracted by the position / orientation extraction unit 53c based on the conversion matrix calculated by the matrix calculation unit 55. The position and orientation of the endoscope 2 in the coordinate system of the image data are calculated (step S65 ′).
[0095]
Next, the observation apparatus main body 6C synthesizes the position and orientation of the endoscope 2 in the coordinate system of the CT image data calculated by the position and orientation calculation unit 56c with the CT image obtained in advance by the CT image synthesis unit 57c and performs this CT synthesis. The image is output to the monitor 7. The monitor 7 displays a CT composite image instead of the CT image displayed on the display surface (step S66 ').
The surgeon can confirm the position and orientation of the endoscope 2 from the CT composite image displayed on the monitor 7.
[0096]
Next, biopsy will be described.
The operator pulls out the ultrasonic probe 70 from the endoscope 2 and inserts the biopsy forceps 8 instead. Here, the biopsy forceps 8 is the same as described in the first embodiment as described above. Then, the surgeon performs biopsy according to the biopsy flowchart shown in FIG.
The observation apparatus body 6C receives an observation image (endoscopic image) from the endoscope apparatus 4 (CCU3) through the endoscope image input unit 11 (step S71 ').
[0097]
Next, the observation device main body 6C identifies the biopsy forceps 8 with the forceps identification unit 23 from the observation image (endoscopic image) in the same manner as described in the first embodiment (step S32 ′). The forceps insertion length determination unit 24 identifies the color or scale of the biopsy forceps 8 from the input observation image (endoscopic image) (step S33 ′), and based on the identified result, the biopsy forceps 8 The insertion length is determined (step S34 ′). ◎
Then, the observation apparatus body 6C displays the insertion length of the biopsy forceps 8 determined by the forceps insertion length determination unit 24 on the CT composite image (step S73 ').
[0098]
Further, when the biopsy cup 8a of the biopsy forceps 8 approaches the target 31, the biopsy cup 8a is detached from the observation range of the endoscope 2, and the biopsy cup 8a becomes invisible from the observation image (endoscopic image). The surgeon depresses a switching button provided on the front panel of the observation apparatus main body 6C to switch the observation image to the VBS image and calculates the biopsy forceps as described in the first embodiment. The VBS image updated according to the insertion length of 8 is displayed.
[0099]
Then, the observation device main body 6C detects the brightness (brightness) of the tip (biopsy cup 8a) of the biopsy forceps 8 in the same manner as described in the first embodiment, so that the biopsy forceps 8 It is determined whether the biopsy cup 8a has reached the target 31, and when it is determined that the target 31 has been reached, the result is output to the monitor 7 and superimposed on the VBS image.
[0100]
Then, the observation apparatus body 6C repeats the process up to step S74 'until the biopsy by the operator is completed (step S74'). Then, the surgeon operates the biopsy forceps 8 while referring to the VBS image and the CT composite image displayed on the display surface of the monitor 7, and the biopsy is completed.
[0101]
As a result, the endoscope observation apparatus 1C according to the present modified example obtains the same effect as that of the second embodiment, and also inserts the ultrasonic probe 70 into the endoscope 2 so that the endoscope observation apparatus 1C Thus, it is possible to obtain a more detailed position and orientation of the endoscope 2.
It should be noted that the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the invention.
[0102]
[Appendix]
(Supplementary Item 1) An endoscope observation apparatus that has an endoscope that reaches the vicinity of a lesion and observes the vicinity of the lesion, and obtains an observation image of the vicinity of the lesion obtained by the endoscope.
Index placement means for placing an index in the vicinity of the lesion using the endoscope;
Tomographic image data acquisition means for obtaining tomographic image data of an area including the vicinity of the lesion and the endoscope in a state where the index is arranged by the index arrangement means;
Based on the tomographic image data obtained by the tomographic image data acquisition means, relative position specifying means for specifying the relative position between the lesioned part and the endoscope;
An endoscope observation apparatus comprising:
[0103]
(Additional Item 2) An endoscope observation apparatus that has an endoscope that reaches the vicinity of a lesion and observes the vicinity of the lesion, and obtains an observation image of the vicinity of the lesion obtained by the endoscope.
Index placement means for placing an index in the vicinity of the lesion using the endoscope;
A tomographic image data acquisition unit that acquires tomographic image data of a region including the vicinity of the lesion, the endoscope, and the index in a state where the index is disposed by the index placement unit;
Based on the tomographic image data obtained by the tomographic image data acquisition means, relative position specifying means for specifying the relative position between the lesioned part and the endoscope;
An endoscope observation apparatus comprising:
[0104]
(Additional Item 3) The relative position specifying unit specifies a relative position between the lesion site and the endoscope based on a center point of the lesion part obtained from the tomographic image data and a vector of the endoscope. The endoscope observation apparatus according to Additional Item 1, wherein the endoscope observation apparatus is characterized in that
[0105]
(Additional Item 4) The relative position specifying means includes the lesion at a center point of the lesion part obtained from the tomographic image data, a center point of a distal end part of the insertion part of the endoscope, and a center point of the index The endoscope observation apparatus according to Additional Item 2, wherein a relative position between a part and the endoscope is specified.
[0106]
(Additional Item 5) In an endoscope observation apparatus that has an endoscope that reaches the vicinity of a lesion and observes the vicinity of the lesion, and obtains an observation image of the vicinity of the lesion obtained by the endoscope.
First three-dimensional data acquisition means for acquiring first three-dimensional data obtained by laminating tomographic image data in the vicinity of the lesion;
Second 3D data acquisition means for acquiring ultrasonic tomographic image data in the vicinity of the lesion and acquiring second 3D data;
First correlation means for obtaining a correlation between the first three-dimensional data obtained by the first three-dimensional data acquisition means and the second three-dimensional data obtained by the second three-dimensional data acquisition means; ,
Second correlation means for obtaining a correlation between the second three-dimensional data obtained by the second three-dimensional data acquisition means and the endoscope;
Association means for associating the position of the endoscope with the first three-dimensional data based on the correlation obtained by the first correlation means and the second correlation means;
An endoscope observation apparatus comprising:
[0107]
【The invention's effect】
As described above, according to the present invention, it is possible to realize an endoscope observation apparatus capable of improving the biopsy rate (biological examination diagnosis rate) in the peripheral part of the body cavity duct.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram showing an endoscope observation apparatus according to a first embodiment of the present invention.
FIG. 2 is a flowchart of a main biopsy performed by the endoscope observation apparatus in FIG.
[Fig. 3] Example of monitor display when a landmark is attached to the mucous membrane near the target of the bronchi
FIG. 4 is a flowchart of biopsy position confirmation in FIG.
FIG. 5 is a monitor display example of a composite image in which a target is combined with an observation image (endoscope image).
6 is a flowchart for calculating the positional relationship of the target in FIG. 3, and FIG. 7 is a schematic diagram showing a biopsy forceps in which a scale is formed.
FIG. 7 is a schematic view showing a biopsy forceps in which a scale is formed.
FIG. 8 is a schematic diagram showing biopsy forceps colored according to length.
FIG. 9 is a biopsy flowchart of FIG.
FIG. 10: Monitor display when two or more landmarks with different shapes are pasted near the target
FIG. 11 is a flowchart of confirmation of a biopsy position according to a modified example.
FIG. 12 is an overall configuration diagram showing an endoscope observation apparatus according to a second embodiment of the present invention.
FIG. 13 is a flowchart of a main biopsy performed by the endoscope observation apparatus in FIG.
FIG. 14 is a flowchart of biopsy position / posture calculation in FIG. 13;
FIG. 15 is a conceptual diagram showing volume matching between a CT volume and an ultrasonic volume.
FIG. 16 is a display example of a CT composite image in which the position and orientation of an endoscope or biopsy forceps are combined
FIG. 17 is a display example showing a MIP image as a CT image.
FIG. 18 is a flowchart of the biopsy in FIG.
FIG. 19 is an overall configuration diagram showing an endoscope observation apparatus according to a modification.
FIG. 20 is a flowchart of biopsy position / orientation calculation executed by the endoscope observation apparatus in FIG. 19;
FIG. 21 is a flowchart of a biopsy performed by the endoscope observation apparatus in FIG.
[Explanation of symbols]
1. Endoscope observation device
2. Endoscope
3 ... CCU
4. Endoscope device
5 ... CT equipment
6 ... Main body of observation device
7 ... Monitor
8 ... Biopsy forceps (index placement means)
10 ... Landmark
11 ... Endoscopic image input unit
12 ... Endoscope model detector
13. Endoscope information recording unit
14: Image correction unit
15 ... Endoscope-side object identification unit
16: Endoscope side position calculation unit
17 ... CT image input unit (tomographic image data acquisition means)
18 ... CT side object identification part
19 ... CT side position calculation unit (relative position specifying means)
20: Image composition unit
21 ... switching part
22 ... VBS image recording unit
23 ... Forceps identification part
24 ... Forceps insertion length determination unit
25 ... Image reference part
26. Photodetector
27 ... Biopsy determination unit
31 ... Target
41 ... Optical fiber

Claims (3)

In an endoscope observation apparatus that has an endoscope that reaches the vicinity of the lesion and observes the vicinity of the lesion, and obtains an observation image of the vicinity of the lesion obtained by the endoscope,
Index placement means for placing an index in the vicinity of the lesion using the endoscope;
A tomographic image data acquisition unit that acquires tomographic image data of a region including the vicinity of the lesion, the endoscope, and the index in a state where the index is disposed by the index placement unit;
Based on the centroid position information of the lesion and the centroid position information of the index obtained from the tomographic image data obtained by the tomographic image data acquisition means, and the gaze direction information of the endoscope, the vicinity of the lesion is reached. A relative position specifying means for specifying a lesion position when viewed from the viewpoint of the distal end of the endoscope;
An endoscope observation apparatus comprising: In an endoscope observation apparatus that has an endoscope that reaches the vicinity of the lesion and observes the vicinity of the lesion, and obtains an observation image of the vicinity of the lesion obtained by the endoscope,
Index placement means for placing an index in the vicinity of the lesion using the endoscope;
A tomographic image data acquisition unit that acquires tomographic image data of a region including the vicinity of the lesion, the endoscope, and the index in a state where the index is disposed by the index placement unit;
Based on the tomographic image data obtained by the tomographic image data acquisition means, a relative position specifying means for specifying a lesion position when viewed from the distal end portion of the endoscope that has reached the vicinity of the lesion;
Equipped with,
The relative position specifying means specifies the lesion position based on the center of gravity of the lesion obtained from the tomographic image data, the center of gravity of the tip of the endoscope, and the center of gravity of the index. Endoscope observation device. The endoscope observation apparatus according to claim 1, further comprising a combining unit that generates a combined image by combining the observation image and the lesion position .

Images