JP4018300B2 - Image processing device - Google Patents

Description

ここで、Ｉは光源強度を表す。
ｋ_a 、ｋ_d 、ｋ_s はそれぞれＣＧで通常用いる環境光、拡散光、鏡面反射光成分の割合を表す。
ｄは投影面からボクセルまでの距離を表し、
ｋ₁ およびｋ₂ は投影面に近いものほど明るく表示するデプスコーディング（ｄｅｐｔｈ ｃｏｄｉｎｇ）法のパラメータである。
Ｌはボクセルから光源への単位方向ベクトル、
Ｈは正反射光を求めるためのベクトルで、レイの方向である視線方向ベクトルＶにより次式で求められる。
Ｈ＝（Ｖ＋Ｌ）／｜Ｖ＋Ｌ｜ ・・・（３）
ボリューム内のあるボクセルｆ（ｉ，ｊ，ｋ）の法線ベクトルＮは次式（４）で求められる。
Ｎ＝▽ｆ（ｉ，ｊ，ｋ）／｜▽ｆ（ｉ，ｊ，ｋ）｜ ・・・（４）
ここで、▽ｆ（ｉ，ｊ，ｋ）は、このボクセル値のｘ、ｙ、ｚ方向の勾配として次式（５）のように求められる。

各ボクセルには色属性値を与えることもでき、その場合には、光源の色特性とボクセルの色値の積として、色を含めた輝度値を決定する。色を扱う場合にはＲ、Ｇ、Ｂの３原色で計算する方法があるが、より厳密には可視光領域でのスペクトル特性を考慮する必要もある。
【００１６】
次に、不透明度の決定の仕方について説明する。
先に述べたように、不透明度αとしてボクセルの値そのものを用いることもできるが、ボクセル値に加えて隣接するボクセルの値からの勾配に基づいてαを決定する方法もある。勾配値を考慮するのは、異なる成分との境界を強調することができるからである。
いま、成分の境界となるボクセル値ｆと不透明度αが図１６のように与えられているとする。このとき、任意のボクセル値ｆをもつボクセルの不透明度αは、勾配と図から補間された不透明度の積として次式（６）で与えられる。

（ｆ_n ＜ｆ＜ｆ_n+1 のとき。それ以外のときはα＝０）
【００１７】
以上のようなボリウムレンダリング処理を用いると、従来のようなポリゴンに変換するという中間データ変換がないので、三次元のデータを正確に見ることができるという特長がある。
しかも、ポリゴンに変換すると、あるパラメータが１つの値のとき（例えば、温度がパラメータである場合、例えば２０°Ｃのとき）のポリゴンは得られるものの、それ以外の値（温度）についての情報はこのポリゴンレンダリング処理では欠落してしまうが、ボリウムレンダリング処理を用いると操作者の見たい温度を表示装置を見ながら例えば１０°Ｃから１２°Ｃというように指定しながら見てゆくことが可能になる。
【００１８】
ただ、このボリウムレンダリング処理の難点は処理に時間がかかることであり、入出力装置を使いながら対話形式によって指定することは従来はとても考えられないことであった。
本出願人は、ボリウムレンダリング処理が膨大なデータを処理するものであるが、その処理内容のほとんどが実は「繰り返し処理」であることに着目し、これに高速処理を可能とするＭＭＸテクノロジを採用すれば、対話形式による指定が可能であることに至った。
そこで、次に、本発明のもう１つの基礎となっている高速処理を可能とするＭＭＸテクノロジについて説明する。
【００１９】
ＭＭＸテクノロジとは、元々はＩｎｔｅｌ社が開発した「マルチメディアや通信分野における処理を高速化する技術」である。つまり、従来のＰｅｎｔｉｕｍ（以下、「Ｐｅｎｔｉｕｍ」と言う。）では負荷が重かった処理や不可能だった処理を、特にマルチメディアデータを扱う場合において、より高速に実行することが可能になった技術のことである。
その中核になるデータ処理手法が、Ｍ．Ｊ．Ｆｌｙｎｎ氏が提唱した「ＳＩＭＤ」（Single Instruction-stream Multiple Data-stream ）である。
ＳＩＭＤはその言葉どおり、ひとつの命令で複数のデータを処理する手法である。複数データを一度に処理できれば実行速度が速くなることは分かるが、それが、マルチメディアに結び付くのは「マルチメディア処理」のほとんどが実は「繰り返し処理」だからである。画像データにエフェクト処理を行ったり、オーディオデータを再生したり、圧縮されたファイルを解凍したり、これらはいずれも元データに対して同じ作業を延々と繰り返す処理になる。
たとえば、元データがＤ１からＤ１００まで１００個の要素で構成されているとき、まずＤ１に対して処理（演算）を行い、次にＤ２、Ｄ３と順番に実行する。ひとつずつ実行すると１００回の演算が必要になる。ところがこれをＳＩＭＤを使って１度に４つのデータを処理できる場合には、まずＤ１〜Ｄ４を取り出して演算し、次にＤ５〜Ｄ８取り出して処理することができるので、全部で２５回の演算で済むことになる。
【００２０】
信号処理によく使われる演算を例にとって説明する。
図１７のように、２種類のデータ列（ａ，ｂ）を最初から８つ互いにペアにして掛け合わせ、その和を求める処理を行なうとする。
▲１▼ 単純に演算処理の部分だけに注目すると、ＭＭＸテクノロジがない従来手法の場合だと、データ読み込みに１６回、掛け算とシフトを８回、足し算を７回実行することになる。
▲２▼ ところが、ＭＭＸテクノロジでは、データ読み込みに４回、掛け算２回、シフト２回、足し算１回で済むことになる。
このように、▲１▼では３９個、▲２▼では９個となり、その差は圧倒的に大きいうえ、さらにＭＭＸ命令の多くは１クロックで済むものが多いので、実行時間の差はさらに大きくなる。
そこで、本発明は、このように、従来数時間かかったボリュームレンダリングにＳＩＭＤ手法を用いるとわずか数秒といった短時間で処理できることに着目したものである。
【００２１】
以下、本発明の実施の形態を図面に基づき詳細に説明する。
図１は本発明の第一の実施の形態に係る画像処理システムの構成図である。
本システムは、コンピュータ本体１とディスプレイ２、コンピュータマウス３、キーボード４から構成される。ディスプレイ２は二次元画像を出力し、コンピュータマウス３は二次元の位置・動き入力とボタンによる入力が可能である。
操作者はディスプレイ２に表示された画面を見ながらコンピュータマウス３とキーボード４を使って、指示を与える。
図２は、本発明の実施の形態に係る装置の構成図で、ＣＰＵ２１は数値計算や画像処理の制御といった種々の処理を行なう。ＲＡＭ２２は三次元格子データ等を格納する。ＲＯＭ２３は処理プログラムや既知データを格納する。ここに本発明が使用するボリウムレンダリング処理プログラム２３１とＳＩＭＤ技術を応用した命令などを利用した三次元高速処理プログラム２３２も格納されている。本発明はＲＡＭやＲＯＭに限定されるものではなく、半導体記憶装置以外の記憶装置を用いてもよいことはいうまでもない。表示装置２４は図１に示したディスプレー２で、ＣＲＴやカラー液晶表示装置などが使用される。入力装置２５は図１に示したコンピュータマウス３とキーボード４である。
ＣＰＵは１個しか例示してないが、高速化を進めるためにボリウムレンダリング処理専用のＣＰＵや三次元高速処理専用のＣＰＵを別途用いることもある。
また、三次元格子データ等を外部とやり取りするためには、図示しないがパラレルＩ／ＯやシリアルＩ／Ｏがメインバスに接続される。
図３は、本発明による三次元座標指定方法により、三次元空間中に任意の線分を設定する手順のフロー図である。
図４は、本実施の形態を実現するための機能を、データの流れと共に図示したものである。コンピュータ本体１には、あらかじめ縦方向、横方向、高さ方向それぞれに大きさを持つ三次元データ集合が図４のＢ３に示されるデータ記憶に入力されており、個々のデータは、一つの数値からなるスカラデータであるものとする。
【００２２】
図３において、ステップＳ１では、図４のＢ４とＢ５に示される変数に値を設定する。設定する値は、これまでの操作や条件設定などにより決定する。Ｂ４には三次元図法を表現するのに必要な視点位置、視点角度、視野角などの一連の情報が記憶され、Ｂ５には本実施の形態においては線分を表現するための二つの三次元座標が記憶される。
【００２３】
ステップＳ２では、コンピュータ本体１のハードウェア・ソフトウェアによって実現される三次元画像表示機能（図４のＢ６）により、先ず三次元データ集合から三次元画像が生成される。三次元画像の生成には、Ｂ３のデータとＢ４の三次元作画パラメータが使用される。
本実施の形態においては、三次元画像は図５に示すようなものとなる。
【００２４】
ステップＳ３では、Ｂ５に記憶されている座標情報を表現するための図形を作成する。この図形は、三次元画像の作成と同様な図法によって作成されるものであり、本実施の形態においては図６に示すようなものとなる。ここで、Ｐ１とＰ２は設定の対象となる二つの三次元座標を表現した図形であり、Ｌは操作者が線分を認識できるようにＰ１とＰ２をつないだ直線図形である。
【００２５】
ステップＳ４では、ステップＳ２とＳ３で作成した二つの画像を重ね合わせてディスプレイ２上に表示する。本実施の形態においては、図７のようなものとなる。
【００２６】
ステップＳ５では、コンピュータマウス３またはキーボード４による操作者入力があるまで待機する。
ステップＳ６では、操作者の入力が座標設定作業の終了を指示するものかどうかを判断し、もしそうであれば全体の処理を終了する。
ステップＳ７では、操作者の入力が三次元描画パラメータの変更を指示するものかどうかを判断し、もしそうであればステップＳ８の処理へ分岐する。
ステップＳ８では、三次元描画変数Ｂ４の内容を操作者の指示に従って更新し、ステップＳ２へループする。
【００２７】
ステップＳ９では、操作者の指示が三次元座標の変更であると判断し、まず変更の指示が二つある点のどちらに対するものであるかを決定する。本実施の形態においては、コンピュータマウス３のボタンが最初に押された時点で、Ｐ１とＰ２のうちマウスポインタに近い方が変更対象であると決定する。
【００２８】
ステップＳ１０では、変更対象である点が動くことのできる面（Ｍとする）を決定する。本実施の形態においては、対象点は、三次元画像の視点位置から現在の座標に引いた直線（Ｎとする）と直交する方向に移動できるものとする。従って、この場合、面Ｍとしては直線Ｎと直交する平面となる。この様子を図８に示す。このような条件により、点は操作者から見て奥行き方向には動かないことになり、縦横に動かしている操作と対象点の三次元移動が直感的に一致し、理解し易くなる。
【００２９】
ステップＳ１１では、面Ｍを使用して補助画像を作成する。
本実施の形態の補助画像としては面Ｍによって三次元画像を切断した画像Ａと面Ｍによって三次元データをサンプリングした画像Ｂの二種類を作成する。
画像Ａを図９に、画像Ｂを図１０に示す。
【００３０】
ステップＳ１２では、ステップＳ１１で作成した補助画像を元々の表示画像に組み合わせてディスプレイ２に表示する。操作者は元々の表示画像と画像Ａと画像Ｂを、選択し、好みの強さで重ね合わせてディスプレイ２に表示させることができる。図１１に画像Ａだけを選択した表示例と、図１２に画像Ａと画像Ｂを重ね合わせ表示例を示す。
ステップＳ１３では、コンピュータマウス３またはキーボード４による操作者の入力があるまで待機する。
ステップＳ１４では、操作者の入力が現在実行中の座標変更の終了を指示するものであるかどうかを判断する。もし終了の指示があれば、処理はステップＳ４にループする。
【００３１】
ステップＳ１５では、座標変更の内容を解釈する。操作者の入力は対象点の三次元空間における移動ベクトルに変換される。
ステップＳ１６では、ステップＳ１５で計算された移動ベクトルに基づき、座標変数Ｂ５の記憶内容を変更する。
ステップＳ１７では、更新された座標変数Ｂ５の内容に基づき、新たな座標表示図形を作成する。即ち、Ｐ１、Ｐ２、Ｌのそれぞれの図形を新しい位置に描画する。
ステップＳ１８では、ステップＳ１７で作成された座標表示図形を三次元画像に重ね合わせてディスプレイ２に表示し、ステップＳ１２にループする。
【００３２】
以上のように、本発明は、以下の各機能の組み合わせからなるものである。
また、これらの各機能は、操作者に対話的操作を可能にするために十分な速度で、変化する角度などのパラメータに応じて連続的に動作し続けることができるものである。
（１） 経時的に変化するパラメータにあわせて、三次元画像を連続的に生成し続けることができる高速な三次元表示機能。
（２） 操作の対象となる点の座標を、数値として記憶する記憶機能。
（３） 前記（２）項の記憶機能によって記憶されている座標点と、関連する点・直線・図形・立体などを、（１）項の三次元表示と同様の投影図法により任意の角度で明確に描出することのできる図形表示機能。
（４） 前記（１）・（２）各項により生成された二つの画像を重ね合わせて表示する重ね合わせ表示機能。
（５） 操作者が、投影図法や角度などを前記（１）・（２）各項の機能に対して指示する機能指示機能。
（６） 操作者が、前記（４）項により生成された画像上において、操作の対象となる点の移動を視覚的・対話的・連続的に指示する点移動指示機能。
（７） 前項によって指示された画像上の移動を、その時点の三次元画像の投影図法・角度・設定された制約条件などを元に、三次元的な移動方向・移動距離に変換し、それを用いて前記（２）項の機能により記憶されている三次元座標を変更する三次元座標変更機能。
（８） 操作者が前記（６）項の機能を用いて対象となる点の移動を指示する際には、次の（ａ）・（ｂ）・（ｃ）の三種類の補助画像を、独立に表示・非表示が選択できるものである。
（ａ） 対象となる点が移動可能な全ての点集合から構成される、曲面・平面・直線・などの三次元物体を、図形として三次元画像に重ね合わせて表示することができる三次元物体重ね合わせ表示機能。
（ｂ） 同じ三次元物体を用いて三次元データ集合からデータをサンプリングしたものを、三次元画像と同じ投影図法・角度により三次元画像に重ね合わせて表示するデータサンプリング重ね合わせ表示機能。
（ｃ） 同じ三次元物体を用いて三次元データ集合に対して、切断・強調・変形など何らかの加工を施し、データに基づいている三次元画像表示も連動して変化する加工機能。
【００３３】
従来装置では二次元画像で２点間の距離を測定することは可能であっても三次元画像で２点間の距離を測定することは時間がかかり過ぎて不可能であったため、そういうこと自体考えさえもしなかったのであるが、本発明によればそれが可能となってしまう。すなわち、通常の二次元表示装置とマウスを使用し、本発明による高速三次元画像生成を用いて、二次元表示装置に三次元空間画像を表示し、マウスで三次元空間内の点を指定して、回転・拡大・移動等を高速に行なうことにより、１画面の２点の投影だけでは分からなかった奥行きが回転をさせることにより位置を再指定でき、さらに回転させて位置を指定することを数回繰り返すだけで急速に収束して２点間の距離を正確に測定することが可能となる。
【００３４】
【発明の効果】
上述のように、従来装置では考えさえもしなかった、通常の二次元表示装置を使っての例えば２点間の距離の測定が、本発明によれば、高速三次元画像生成機能により図形の回転・拡大・移動等が高速に行なわれるようになるので、１画面の２点の投影だけでは分からなかった奥行きを高速回転により即座に知ることができ、位置を再指定することを数回繰り返すことで２点間の距離を正確に測定することができるようになる。
以上は回転の例であるが、拡大・移動についても同様のことが言える。
【図面の簡単な説明】
【図１】本発明の実施の形態に係るシステム構成図である。
【図２】本発明の実施の形態に係る装置の構成図である。
【図３】本発明の実施の形態における処理フロー図である。
【図４】本発明の実施の形態におけるデータの流れを示すブロック図である。
【図５】Ｂ６により生成される三次元画像の例である。
【図６】Ｂ７により生成される座標表示図形の例である。
【図７】Ｂ８により生成される補助画像のない場合の重ね合わせ画像の例である。
【図８】本発明の実施の形態における、座標設定対象点の動ける面Ｍと視点との関係を示す図である。
【図９】Ｂ１０により生成される補助画像の例として、面Ｍによって三次元画像を切断して作成した画像Ａを示す。
【図１０】Ｂ１０により生成される補助画像の例として、面Ｍによって三次元データをサンプリングして作成した画像Ｂを示す。
【図１１】Ｂ８により生成される補助画像ＡとＢの場合の重ね合わせ画像の例で、元々の表示画像と画像Ａだけを選択した表示例を示す。
【図１２】Ｂ８により生成される補助画像ＡとＢの場合の重ね合わせ画像の例で、元々の表示画像と画像Ａおよび画像Ｂを重ね合わせた表示例を示す。
【図１３】ボリウムレンダリングの概念を示す図である。
【図１４】レイキャステイングにおけるサンプリングを示す図である。
【図１５】ボリウムレンダリングの処理手順を示す図である。
【図１６】勾配値を考慮した不透明度αを説明する図である。
【図１７】あるデータ処理の手法を説明する図で、（ａ）は従来の手法、（ｂ）はＭＭＸテクノロジの手法を示す図である。
【符号の説明】
１ コンピュータ本体
２、２４ ディスプレイ
３ コンピュータマウス
４ キーボード
２１ ＣＰＵ
２２ ＲＡＭ
２３ ＲＯＭ
２３１ ボリウムレンダリング処理プログラム
２３２ 三次元高速処理プログラム
２４ 表示装置
２５ 入力装置[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus having a function of displaying a data set arranged in a three-dimensional space as a three-dimensional image, and more particularly to a user interface of various analysis applications based on images.
[0002]
[Prior art]
Data arranged in a three-dimensional space represented by this kind of numerical calculation data and medical imaging device data known in the past tend to be very large in number, and humans directly examine the numerical values of each data. Is often impossible in practice. Therefore, it is generally performed in many fields to express three-dimensional data as a three-dimensional image that can be easily recognized visually by humans.
The three-dimensional image generated in this way is not only sensuously viewed as a video, but is also used as important information for performing various analyzes according to the purpose of each field. There are many methods for analysis, but it is common to know the three-dimensional position accurately, measure the distance in the three-dimensional space, and calculate the volume of the space delimited by certain conditions In addition to the methods used in the field, there are many analysis methods specially devised for each field.
[0003]
[Problems to be solved by the invention]
When performing analysis using 3D spatial information using 3D images, the designation / change operation of position / shape / trajectory is specified / changed for one or more 3D coordinates using some input device. It is common to perform as a combination of operations.
However, an output device that displays a 3D image can only handle 2D information, and the input device can only specify 2D coordinates. It is difficult to specify the depth. Although it is possible to specify by direct numerical input of each coordinate, the value of the intuitive operation utilizing the image is completely lost, so the value as an image display analysis system has been greatly impaired.
In addition, there are three-dimensional input / output devices using special machines and devices, but they are very special and have many problems in terms of economy, reliability, ease of use, future availability, etc. It was hard to say that it reached a general and practical system.
Such difficulty in designating the three-dimensional coordinates requires a long learning period for the operator and places a great mental and time burden on the operation.
Furthermore, the contents intended by the operator are often not accurately recognized by the system, which increases the risk of damaging the most essential value of the system, namely the authenticity of the analysis results.
[0004]
The present invention solves the above-mentioned problems by enabling the operation of specifying point coordinates in a three-dimensional space to be performed simply, quickly and accurately using a generally available input / output device. It is.
[0005]
[Means for Solving the Problems]
  In order to achieve the above object, a three-dimensional image high-speed display device according to the present invention is provided.IsA CPU that performs various processes such as numerical calculation and image processing control, a first storage device that stores three-dimensional lattice data, a second storage device that stores processing programs and known data, a display device, and an input In the three-dimensional image display apparatus provided with the apparatus, a volume rendering processing program and a three-dimensional high-speed processing program are stored in the ROM.
  An image processing apparatus according to the present invention is an image processing apparatus that performs three-dimensional image processing using volume data, a coordinate storage unit that stores current coordinates of a three-dimensional point, and an operator's input, When moving the coordinates of the point that is the change target, information necessary to express the three-dimensional projection of the movable surface of the point that is the change target and the current coordinates of the point that is the change target A surface determining means (implemented by the CPU) for determining based on the above, a first image generating means for generating a first image obtained by cutting the volume data by the surface, and a second sampled volume data on the surface A second image creating means for creating a first image, a superimposed image creating means for creating a superimposed image by superimposing the first image and the second image, and displaying the superimposed image. It includes a Display means.
[0006]
  Also,According to the present inventionThree-dimensional image high-speed display deviceIsIt is characterized by having the following functions.
(1) A high-speed three-dimensional display function capable of continuously generating a three-dimensional image according to a parameter that changes over time.
(2) A storage function for storing the coordinates of points to be operated as numerical values.
(3) The coordinate points stored by the storage function of the item (2) and related points, straight lines, figures, solids, etc. can be displayed at an arbitrary angle by the same projection projection as the three-dimensional display of the item (1). Graphic display function that can be drawn clearly.
(4) A superposition display function for superposing and displaying the two images generated by the items (1) and (2).
(5) A function instruction function for an operator to instruct the projection projection, the angle, and the like to the functions of the items (1) and (2).
(6) A point movement instruction function for the operator to visually, interactively, and continuously instruct the movement of the point to be operated on the image generated in the item (4).
(7) The movement on the image instructed in the previous section is converted into a three-dimensional movement direction and distance based on the projection method, angle, and set constraints of the three-dimensional image at that time. A three-dimensional coordinate changing function for changing the three-dimensional coordinates stored by the function of the item (2). It is characterized by.
[0007]
  Also,According to the present inventionThe three-dimensional image high-speed display device further includes at least one of the following functions (a) to (c), and the operator moves the target point using the function described in (6) above. When the instruction is given, the three types of auxiliary images can be independently displayed or hidden.
(A) A three-dimensional object that can be displayed by superimposing a three-dimensional object such as a curved surface, a plane, a straight line, etc. composed of all point sets that can move the target point as a figure on a three-dimensional image. Overlay display function.
(B) A data sampling superposition display function for displaying data sampled from a three-dimensional data set using the same three-dimensional object and superposing it on the three-dimensional image with the same projection projection method and angle as the three-dimensional image.
(C) A processing function in which a 3D data set is subjected to some processing such as cutting, emphasis, and deformation using the same 3D object, and the 3D image display based on the data is changed in conjunction with the processing.
[0008]
  Also,According to the present inventionSpecifying coordinates in the original space interactively on a 3D imageLaw is,According to the present inventionUsing a high-speed 3D image display device, the data set obtained as 3D lattice point data is converted into a 3D image using the high-speed 3D image generation function, and the 3D image is displayed on the display. The coordinate points displayed graphically by the same projection method as the three-dimensional image are superimposed and displayed using the two-dimensional image displayed on the display as a visual guide. By interactively, continuously and intuitively grasping the current coordinate settings while continuously changing the original image creation parameters, the coordinates in the 3D space can be easily and accurately used while using the 2D input / output device. It is characterized by specifying.
[0009]
  Also,According to the present inventionThe invention further provides an auxiliary image that serves as a visual guide for changing the coordinates using the three-dimensional lattice point data, the three-dimensional image creation parameters at that time and the set coordinate values being changed while the coordinate setting values are being changed. More visual information is displayed by adding the function to generate the images continuously or interactively and superimposing the auxiliary images or replacing them with the original display images. The feature is that the coordinate point can be changed accurately and quickly according to the position.
[0010]
With the above configuration, accurate 3D coordinate point specification can be performed quickly and accurately on a 3D image that is easy to understand intuitively for the operator, while using 2D input devices and output devices that are widely used in general. It becomes possible.
The operator creates a three-dimensional image in which the coordinate point to be specified is more characteristically expressed by interactive specification such as an angle and a drawing condition, and displays an operation target figure displayed superimposed on the image. It can be easily installed at the position and shape you want.
While repeatedly performing the specified operation, the operator can check the 3D image and the superimposed image of the figure interactively and continuously at various angles and drawing conditions, and can always confirm whether there is a mistake visually. .
In addition, since the auxiliary image at the time of position change of the present invention makes it very easy to grasp the three-dimensional spatial position of the coordinate point on the three-dimensional image, including the depth, the number of position change operations and the coordinate point position Both the time required for the entire designation can be greatly reduced.
Further, since it becomes very easy to confirm whether or not the coordinate point after the position change has been accurately set to the three-dimensional position intended by the operator, the accuracy of position designation is greatly improved.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Before describing the embodiment of the present invention in detail, the volume rendering used by the present invention, which can directly convert a huge amount of three-dimensional lattice point data into three-dimensional data without converting it into conventional polygons, and Similarly, the MMX technology employed by the present invention will be described.
[0012]
First, volume rendering will be described.
In volume rendering, if X-ray CT (Computed Tomography) is used, the inside of the human body can be cut and displayed as a tomographic image. This tomographic image represents the amount of X-ray absorption at each point on the tomographic plane. By storing the tomographic images obtained by moving the tomographic planes little by little, the human body can be divided vertically, horizontally, and back and forth at regular intervals. Data can be obtained. Such data is called volume data, and depending on the measurement method, it can represent the concentration and density of a specific substance distributed in a certain space (volume).
When displaying a skeleton based on volume data, for example, in the conventional CG technique, it is necessary to first determine the shape of the bone surface and then apply a rendering algorithm such as ray tracing. However, depending on the location, the bone has a very complex shape, and the X-ray absorption rate may change smoothly between the bone and the adjacent tissue, so that bones, muscles, and internal organs are clearly separated. Is difficult.
In such a case, a method of drawing a tissue directly from volume data without determining the shape of the surface of the bone or the like is called volume rendering. As an application of volume rendering to other than medical images, it is used when clouds are displayed using the concentration distribution of water vapor in the atmosphere.
[0013]
The volume rendering method is as follows (reference: “computer graphics” (page 204 and later) issued by the Foundation for Image Information Education, supervised by the Technical CG Standard Textbook Editorial Committee).
In volume rendering, feature values distributed in a three-dimensional space are sampled at regular intervals along an arbitrary line of sight (ray), and the values are added to produce a semitransparent image. To do.
Sampling a volume along a ray is called ray casting. The concept of volume rendering is shown in FIG. Rays are emitted from each pixel on the projection plane and sample voxels in the volume at regular intervals.
In volume rendering, the inside is visualized by performing translucent display, which is realized by giving opacity α to each voxel. Although the opacity α may use the voxel value itself, a method for determining α based on the gradient from the value of the adjacent voxel has been proposed.
By increasing α of the voxel corresponding to the object surface to be displayed, it becomes possible to draw the target with emphasis close to being opaque. In ray casting, the product of the brightness value of each voxel and α is added sequentially. When the sum of α becomes 1 or when the ray exits the target volume, the processing for that pixel is terminated and added. The result is displayed as the value of that pixel.
The luminance value and opacity at the current sample position are i and α, respectively, and the integrated luminance value incident at this point is I_inThen, the integrated luminance value I when passing through this sample point is_outIs represented by the following formula (1).
I_out= I_in(1-α) + iα (1)
In general, the luminance value and opacity at the sampling point in ray casting can be obtained from the values of each of the eight adjacent voxels by linear interpolation as shown in FIG.
[0014]
A general volume rendering processing procedure is shown in FIG.
For the three-dimensional volume data obtained from the CT image and simulation results of various physical quantity distributions, first, noise removal, image enhancement, and the like are executed in the preprocessing. Here, three-dimensional image processing for the entire volume may be performed. Next, the luminance value and opacity of each voxel are determined, and the final luminance value for each pixel is obtained by ray casting. The determination of these values is important in volume rendering.
[0015]
Therefore, how to determine the luminance value and the opacity will be described.
First, regarding the luminance value, the luminance value i of each voxel is calculated from the von shading model as shown in Equation (2) using the normal vector N estimated from the gradient from the value of the adjacent voxel. The

Here, I represents the light source intensity.
k_a, K_d, K_sRespectively represent the ratio of ambient light, diffused light, and specular reflection light components normally used in CG.
d represents the distance from the projection plane to the voxel;
k₁And k₂Is a parameter of a depth coding method in which the closer to the projection plane is displayed brighter.
L is the unit direction vector from the voxel to the light source,
H is a vector for obtaining specularly reflected light, and is obtained from the line-of-sight direction vector V, which is the ray direction, by
H = (V + L) / | V + L | (3)
A normal vector N of a certain voxel f (i, j, k) in the volume is obtained by the following equation (4).
N = ▽ f (i, j, k) / | ▽ f (i, j, k) | (4)
Here, ▽ f (i, j, k) is obtained as the gradient of the voxel value in the x, y, and z directions as shown in the following equation (5).

A color attribute value can also be given to each voxel. In this case, the luminance value including the color is determined as the product of the color characteristic of the light source and the color value of the voxel. In the case of handling colors, there is a method of calculating with three primary colors of R, G, and B, but more strictly, it is necessary to consider the spectral characteristics in the visible light region.
[0016]
Next, how to determine the opacity will be described.
As described above, the value of the voxel itself can be used as the opacity α, but there is a method of determining α based on the gradient from the value of the adjacent voxel in addition to the voxel value. The gradient value is taken into account because the boundary between different components can be emphasized.
Now, it is assumed that the voxel value f and the opacity α which are the boundaries of the components are given as shown in FIG. At this time, the opacity α of a voxel having an arbitrary voxel value f is given by the following equation (6) as a product of the gradient and the opacity interpolated from the figure.

(F_n<F <f_{n + 1}When. Otherwise, α = 0)
[0017]
When the volume rendering process as described above is used, there is no intermediate data conversion such as conversion to a polygon as in the prior art, and there is a feature that three-dimensional data can be accurately viewed.
Moreover, when converted into polygons, polygons can be obtained when a certain parameter has a single value (for example, when temperature is a parameter, for example, at 20 ° C), but information about other values (temperature) is Although this polygon rendering process is missing, if the volume rendering process is used, it is possible to see the temperature desired by the operator while designating the display device, for example, 10 ° C to 12 ° C. Become.
[0018]
However, the difficulty of this volume rendering process is that it takes time to process, and it has been unthinkable to specify it interactively using an input / output device.
The applicant of the present invention pays attention to the fact that the volume rendering process processes a large amount of data, but most of the processing content is actually "repetitive processing" and adopts MMX technology that enables high-speed processing. In this way, it was possible to specify interactively.
Therefore, the MMX technology that enables high-speed processing, which is another basis of the present invention, will be described next.
[0019]
The MMX technology is “technology for speeding up processing in the multimedia and communication fields” originally developed by Intel. In other words, a technology that makes it possible to execute processing that was heavy or impossible with conventional Pentium (hereinafter referred to as “Pentium”) at a higher speed, particularly when handling multimedia data. That is.
The core data processing method is M.M. J. et al. “SIMD” (Single Instruction-stream Multiple Data-stream) proposed by Mr. Flynn.
SIMD is a technique for processing a plurality of data with a single command, as the word suggests. It can be seen that if multiple data can be processed at once, the execution speed will be faster, but the reason why it is linked to multimedia is that most “multimedia processing” is actually “repetitive processing”. The effect processing is performed on the image data, the audio data is reproduced, and the compressed file is decompressed. In any case, the same operation is repeatedly performed on the original data.
For example, when the original data is composed of 100 elements from D1 to D100, first, processing (calculation) is performed on D1, and then D2 and D3 are executed in order. If executed one by one, 100 operations are required. However, when four data can be processed at one time using SIMD, D1 to D4 can be first extracted and then calculated, and then D5 to D8 can be extracted and processed. Will be enough.
[0020]
An explanation will be given by taking an example of operations often used for signal processing.
As shown in FIG. 17, it is assumed that eight types of data strings (a, b) are first paired and multiplied to obtain the sum.
{Circle over (1)} Focusing only on the arithmetic processing part, in the case of the conventional method without MMX technology, 16 times for data reading, 8 times multiplication and shifting, and 7 times addition are executed.
(2) However, with MMX technology, data reading requires four times, two multiplications, two shifts, and one addition.
Thus, 39 in (1) and 9 in (2), the difference is overwhelmingly large, and many MMX instructions only require one clock, so the difference in execution time is even greater. Become.
Thus, the present invention focuses on the fact that processing can be performed in a short time of only a few seconds when the SIMD method is used for volume rendering that has conventionally taken several hours.
[0021]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a configuration diagram of an image processing system according to the first embodiment of the present invention.
This system includes a computer main body 1, a display 2, a computer mouse 3, and a keyboard 4. The display 2 outputs a two-dimensional image, and the computer mouse 3 can input a two-dimensional position / motion and a button.
The operator gives an instruction using the computer mouse 3 and the keyboard 4 while viewing the screen displayed on the display 2.
FIG. 2 is a block diagram of the apparatus according to the embodiment of the present invention. The CPU 21 performs various processes such as numerical calculation and image processing control. The RAM 22 stores three-dimensional lattice data and the like. The ROM 23 stores processing programs and known data. A volume rendering processing program 231 used by the present invention and a three-dimensional high-speed processing program 232 using instructions applying SIMD technology are also stored here. The present invention is not limited to the RAM and the ROM, and it goes without saying that a storage device other than the semiconductor storage device may be used. The display device 24 is the display 2 shown in FIG. 1, and a CRT, a color liquid crystal display device or the like is used. The input device 25 is the computer mouse 3 and the keyboard 4 shown in FIG.
Although only one CPU is illustrated, a CPU dedicated to volume rendering processing or a CPU dedicated to three-dimensional high-speed processing may be used separately in order to increase the speed.
In order to exchange 3D grid data and the like with the outside, parallel I / O and serial I / O are connected to the main bus (not shown).
FIG. 3 is a flowchart of a procedure for setting an arbitrary line segment in the three-dimensional space by the three-dimensional coordinate designating method according to the present invention.
FIG. 4 illustrates functions for realizing the present embodiment together with a data flow. In the computer main body 1, a three-dimensional data set having a size in each of the vertical direction, the horizontal direction, and the height direction is input in advance into the data storage indicated by B3 in FIG. It is assumed to be scalar data consisting of
[0022]
In FIG. 3, in step S1, values are set in the variables indicated by B4 and B5 in FIG. The value to be set is determined by the previous operations and condition settings. B4 stores a series of information such as a viewpoint position, a viewpoint angle, and a viewing angle necessary for expressing a three-dimensional projection, and B5 stores two three-dimensional information for expressing a line segment in the present embodiment. The coordinates are stored.
[0023]
In step S2, a 3D image is first generated from the 3D data set by the 3D image display function (B6 in FIG. 4) realized by the hardware and software of the computer main body 1. B3 data and B4 three-dimensional drawing parameters are used to generate a three-dimensional image.
In the present embodiment, the three-dimensional image is as shown in FIG.
[0024]
In step S3, a graphic for expressing the coordinate information stored in B5 is created. This figure is created by the same projection method as that for creating a three-dimensional image, and in the present embodiment, it is as shown in FIG. Here, P1 and P2 are figures that express two three-dimensional coordinates to be set, and L is a straight line figure that connects P1 and P2 so that the operator can recognize the line segment.
[0025]
In step S4, the two images created in steps S2 and S3 are superimposed and displayed on the display 2. In the present embodiment, it is as shown in FIG.
[0026]
In step S5, the process waits until there is an operator input from the computer mouse 3 or the keyboard 4.
In step S6, it is determined whether or not the operator's input is an instruction to end the coordinate setting operation. If so, the entire process is ended.
In step S7, it is determined whether or not the operator's input is an instruction to change the three-dimensional drawing parameter. If so, the process branches to step S8.
In step S8, the contents of the three-dimensional drawing variable B4 are updated according to the operator's instruction, and the process loops to step S2.
[0027]
In step S9, it is determined that the operator's instruction is to change the three-dimensional coordinates, and first, it is determined which of the two points the change instruction is for. In the present embodiment, when the button of the computer mouse 3 is first pressed, it is determined that one of P1 and P2 that is closer to the mouse pointer is the object to be changed.
[0028]
In step S10, a surface (M) on which the point to be changed can move is determined. In the present embodiment, it is assumed that the target point can move in a direction orthogonal to a straight line (N) drawn from the viewpoint position of the three-dimensional image to the current coordinates. Therefore, in this case, the surface M is a plane orthogonal to the straight line N. This is shown in FIG. Under such conditions, the point does not move in the depth direction when viewed from the operator, and the operation moving vertically and horizontally and the three-dimensional movement of the target point intuitively match, making it easy to understand.
[0029]
In step S11, an auxiliary image is created using the surface M.
Two types of images are created as auxiliary images in the present embodiment: an image A obtained by cutting a three-dimensional image by the surface M and an image B obtained by sampling three-dimensional data by the surface M.
Image A is shown in FIG. 9, and image B is shown in FIG.
[0030]
In step S12, the auxiliary image created in step S11 is combined with the original display image and displayed on the display 2. The operator can select the original display image, the image A, and the image B, and display them on the display 2 by superimposing them with a desired strength. FIG. 11 shows a display example in which only image A is selected, and FIG. 12 shows a display example in which image A and image B are superimposed.
In step S13, the process waits until there is an operator input from the computer mouse 3 or the keyboard 4.
In step S14, it is determined whether or not the operator's input is an instruction to end the coordinate change currently being executed. If there is an instruction to end, the process loops to step S4.
[0031]
In step S15, the contents of the coordinate change are interpreted. The operator's input is converted into a movement vector in the three-dimensional space of the target point.
In step S16, the stored content of coordinate variable B5 is changed based on the movement vector calculated in step S15.
In step S17, a new coordinate display graphic is created based on the updated content of the coordinate variable B5. That is, the respective figures P1, P2, and L are drawn at new positions.
In step S18, the coordinate display figure created in step S17 is superimposed on the three-dimensional image and displayed on the display 2, and the process loops to step S12.
[0032]
As described above, the present invention is a combination of the following functions.
Each of these functions can continue to operate continuously according to parameters such as a changing angle at a speed sufficient to allow an operator to perform an interactive operation.
(1) A high-speed three-dimensional display function capable of continuously generating a three-dimensional image according to a parameter that changes over time.
(2) A storage function for storing the coordinates of points to be operated as numerical values.
(3) The coordinate points stored by the storage function of the item (2) and related points, straight lines, figures, solids, etc. can be displayed at an arbitrary angle by the same projection projection as the three-dimensional display of the item (1). Graphic display function that can be drawn clearly.
(4) A superposition display function for superposing and displaying the two images generated by the items (1) and (2).
(5) A function instruction function for an operator to instruct the projection projection, the angle, and the like to the functions of the items (1) and (2).
(6) A point movement instruction function for the operator to visually, interactively, and continuously instruct the movement of the point to be operated on the image generated in the item (4).
(7) The movement on the image instructed in the previous section is converted into a three-dimensional movement direction and distance based on the projection method, angle, and set constraints of the three-dimensional image at that time. A three-dimensional coordinate changing function for changing the three-dimensional coordinates stored by the function of the item (2).
(8) When the operator uses the function of item (6) to instruct the movement of the target point, the following three types of auxiliary images (a), (b), and (c) are used: Display / hide can be selected independently.
(A) A three-dimensional object that can be displayed by superimposing a three-dimensional object such as a curved surface, a plane, a straight line, etc. composed of all point sets that can move the target point as a figure on a three-dimensional image. Overlay display function.
(B) A data sampling superposition display function for displaying data sampled from a three-dimensional data set using the same three-dimensional object and superposing it on the three-dimensional image with the same projection projection method and angle as the three-dimensional image.
(C) A processing function in which a 3D data set is subjected to some processing such as cutting, emphasis, and deformation using the same 3D object, and the 3D image display based on the data is changed in conjunction with the processing.
[0033]
In the conventional apparatus, although it is possible to measure the distance between two points with a two-dimensional image, it is too much time to measure the distance between two points with a three-dimensional image. I didn't even think about it, but according to the present invention it is possible. That is, using a normal two-dimensional display device and a mouse, using the high-speed three-dimensional image generation according to the present invention, a three-dimensional space image is displayed on the two-dimensional display device, and a point in the three-dimensional space is designated with the mouse. By rotating, enlarging, moving, etc. at high speed, the depth can be re-designated by rotating the depth that could not be found by only projecting two points on one screen, and the position can be further designated by rotating it. It is possible to measure the distance between two points accurately by converging rapidly only by repeating several times.
[0034]
【The invention's effect】
As described above, for example, the distance between two points using a normal two-dimensional display device, which was not even considered in the conventional device, can be measured by a high-speed three-dimensional image generation function according to the present invention.・ Because enlargement and movement are performed at high speed, the depth that could not be found by only projecting two points on one screen can be immediately known by high-speed rotation, and re-specifying the position several times Thus, the distance between the two points can be accurately measured.
The above is an example of rotation, but the same can be said for enlargement and movement.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram according to an embodiment of the present invention.
FIG. 2 is a configuration diagram of an apparatus according to an embodiment of the present invention.
FIG. 3 is a processing flow diagram according to the embodiment of the present invention.
FIG. 4 is a block diagram showing a data flow in the embodiment of the present invention.
FIG. 5 is an example of a three-dimensional image generated by B6.
FIG. 6 is an example of a coordinate display graphic generated by B7.
FIG. 7 is an example of a superimposed image when there is no auxiliary image generated by B8.
FIG. 8 is a diagram showing a relationship between a moving surface M of a coordinate setting target point and a viewpoint according to the embodiment of the present invention.
FIG. 9 shows an image A created by cutting a three-dimensional image along a plane M as an example of an auxiliary image generated by B10.
FIG. 10 shows an image B created by sampling three-dimensional data using a plane M as an example of an auxiliary image generated by B10.
FIG. 11 is an example of a superimposed image in the case of auxiliary images A and B generated by B8, and shows a display example in which only the original display image and image A are selected.
FIG. 12 is an example of a superimposed image in the case of auxiliary images A and B generated by B8, and shows a display example in which the original display image and images A and B are superimposed.
FIG. 13 is a diagram showing the concept of volume rendering.
FIG. 14 is a diagram illustrating sampling in ray casting.
FIG. 15 is a diagram illustrating a processing procedure of volume rendering.
FIG. 16 is a diagram illustrating opacity α in consideration of a gradient value.
17A and 17B are diagrams for explaining a certain data processing method, in which FIG. 17A shows a conventional method, and FIG. 17B shows a method of MMX technology.
[Explanation of symbols]
1 Computer body
2, 24 display
3 Computer mouse
4 Keyboard
21 CPU
22 RAM
23 ROM
231 Volume rendering program
232 Three-dimensional high-speed processing program
24 display devices
25 Input device

Claims (3)

An image processing apparatus that performs three-dimensional image processing using volume data,
Coordinate storage means for storing the current coordinates of the three-dimensional point;
When the coordinates of the point to be changed are moved according to the input from the operator, the surface on which the point to be changed can move is the information necessary for expressing the 3D projection and the change target. Surface determining means for determining based on the current coordinates of the point ;
First image creating means for creating a first image obtained by cutting the volume data by the surface;
Second image creation means for creating a second image obtained by sampling the volume data on the surface;
A superimposed image creating means for creating a superimposed image in which the first image and the second image are superimposed;
Display means for displaying the superimposed image;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The image processing apparatus, wherein the surface is a plane orthogonal to a straight line drawn from a viewpoint position to a current coordinate of the point to be changed . The image processing apparatus according to claim 1,
The coordinate storage means stores current coordinates of a plurality of points that can be moved in accordance with an operator's input,
The point to be changed, the image processing apparatus is a point selected from the plurality of points.

Images