JP4174985B2 - Display device and computer program for difference between two three-dimensional data - Google Patents

Description

【００５６】
但し、
Ｋ ：３次元データの構成点の個数
ｄk ：３次元データの構成点と標準モデルの表面との距離
Ｗsc：偏倍安定化のウエイトパラメータ
Ｓ0 ：初期スケール
Ｓi ：各方向の偏倍量（但し、Ｓ3 は奥行き方向である）
αi ：標準モデルの各方向の回転
ｔi ：標準モデルの各方向への移動量
ここで、標準モデルＤＳ上の構成点は次の（２）式にしたがって移動し、それにともなって、３次元データＤＴの構成点と標準モデルＤＳの表面との間の距離ｄk が変化する。
【００５７】
【数２】

【００５８】
〔局所的概略位置合わせ〕
上に述べた全体的概略位置合わせを自動で行った場合に、それがうまく合わなかったときに、手動で合わせることとなるが、ここに述べる局所的概略位置合わせは、手動での位置合わせの際にできるだけ簡単に行うための手法である。なお、自動でうまくいかなかった分は一旦リセットし、初めから手動でやり直す。
【００５９】
局所的概略位置合わせでは、３次元データＤＴ上の特徴的な線または点と、標準モデルＤＳ上の特徴的な線または点とを対応づけ、それらの距離を最小にするように標準モデルＤＳの位置、方向、およびサイズを変更する。なお、線と線とを対応付けた場合は、一方の線上の点とその点から他方の線上へ降ろした垂線のうち最短となる点とを特徴点とし、線上でこれらの点を複数点取得するものとする。
【００６０】
すなわち、３次元データＤＴ上の特徴点とそれに対応する標準モデルＤＳ上の特徴点との距離に対して、次の（４）式に示すエネルギー関数Ｅ（ｓｉ，αｉ，ｔｉ）が最小となるように、標準モデルＤＳのｔｉ，αｉ，ｓｉを導く。
【００６１】
【数３】

【００６２】
但し、
ｋ ：対応する特徴点の個数
Ｍk ：位置合わせ後の標準モデル上の特徴点
ｘ ：位置合わせ前の標準モデル上の特徴点
Ｃk ：３次元データ上の特徴点
Ｓi ：標準モデルの各方向の偏倍量
αi ：標準モデルの各方向の回転
ｔi ：標準モデルの各方向への移動量
〔輪郭・特徴点の抽出〕
３次元データＤＴまたは２次元画像ＦＴ上に、輪郭および特徴点を抽出する（＃１４）。標準モデルＤＳについての輪郭ＲＫおよび特徴点ＴＴを予め抽出しておいた場合には、それらと同じ位置に配置されるべき輪郭および特徴点を、３次元データＤＴ上に、またはそれに対応する２次元画像上に配置する（図１５参照）。
【００６３】
標準モデルＤＳについての輪郭ＲＫおよび特徴点ＴＴが予め抽出されていない場合には、３次元データＤＴ上または２次元画像上への配置と合わせて標準モデルＤＳ上でも指定する。
〔データ削減〕
次に、計算量および誤差を削減するために、３次元データＤＴについてデータの削減を行い、必要且つ信頼性の高いデータのみを取り出す（＃１５）。データの削減を行うことによって、元の３次元データＤＴの形状を崩すことなく、計算量を減らすことができる。
【００６４】
データの削減に当たって、例えば、対象物の領域外のデータを除外し、不要なデータを除く。例えば、２次元画像ＦＴから顔の領域を判別し、その領域に対応した３次元データＤＴのみを残す。あるいは、対象物と背景との間の距離の相違を用いて領域を判別する。また、概略位置合わせの情報を用いて、顔の領域を抽出するなどの各種の方法がある。また、３次元データＤＴに信頼性データＤＲがある場合には、信頼性の高いもののみを残す。近隣にデータが多い場合はそのデータを間引き、密度を平均化する。
〔変形〕
標準モデルＤＳ全体の変形が行われる（＃１６）。ここでは、３次元データＤＴの各構成点と標準モデルＤＳの面との間の距離に関連して定義されたエネルギー関数ｅ1 を用いるとともに、それに加えて、標準モデルＤＳの特徴点と３次元データＤＴに対して指定された特徴点との間の距離に関連して定義されるエネルギー関数ｅ3 、標準モデルＤＳの輪郭と３次元データＤＴに対して指定された輪郭との間の距離に関連して定義されるエネルギー関数ｅ2 、および、過剰な変形を回避するために定義されたエネルギー関数ｅs を用い、それらを総合したエネルギー関数ｅを評価し、総合のエネルギー関数ｅが最小となるように標準モデルＤＳの面を変形させる。
【００６５】
なお、総合のエネルギー関数ｅとして、ｅ１，ｅ２，ｅ３，ｅｓの４つの関数を用いるのが一番望ましいが、ｅ１〜ｅ３のうち任意の２つだけを用いることも可能である。
【００６６】
次に、各エネルギー関数について簡単に説明する。
〔標準モデルと３次元データとの距離〕
図３において、３次元データＤＴを構成する点群の１つが点Ｐｋで示されている。標準モデルＤＳの面Ｓにおいて、点Ｐｋに最も近い点がＱｋで示されている。点Ｑｋは、点Ｐｋから面Ｓに垂線を下ろしたときの交点である。ここでは、点Ｐｋと点Ｑｋとの距離（差分ＳＢ）が評価される。
【００６７】
すなわち、３次元データＤＴの各点と標準モデルＤＳの面との差分エネルギーｅ1 は、データ削減後の３次元データＤＴ上の点Ｐｋと、それを標準モデルＤＳの面Ｓ上に投影した点Ｑｋとの二乗距離を用いて、次の（７）式によって算出される。
【００６８】
【数４】

【００６９】
但し、
Ｔ1A：制御点群
Ｐk ：削減後の３次元データの構成点
Ｑk ：構成点からモデル表面への投影点
Ｋ ：削減後の構成点の個数
ｄ^k ：構成点からモデル表面への投影方向，
ｄ^k ＝（Ｑk-Ｐk ）／｜Ｑk-Ｐk ｜
ρk ：構成点Ｐk の信頼性
w(ρk)：信頼性関数，w(ρk)＝１／（α＋ρk ）ⁿ
Ｗ ：Σw(ρk)
Ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
〔標準モデル上の輪郭と計測データ上の輪郭との距離〕
ここでは、３次元データＤＴ上に指定された輪郭ＲＫ、または２次元画像ＦＴ上に指定された輪郭ＲＫと、標準モデルＤＳ上の輪郭ＲＫとの距離が評価される。
【００７０】
計測データの輪郭ＲＫが３次元データＤＴ上に指定される場合は、３次元データＤＴの輪郭ＲＫ上の点から標準モデルＤＳ上の対応する輪郭ＲＫへ垂線を降ろし、その垂線のうち最短のものを距離とする。なお、輪郭ＲＫ上では複数の点を指定する。
【００７１】
計測データの輪郭ＲＫが２次元画像ＦＴ上に指定される場合は、２次元画像ＦＴを撮影したカメラについてのカメラパラメータを用い、標準モデルＤＳの輪郭ＲＫを２次元画像ＦＴ上に投影する。２次元画像ＦＴの輪郭ＲＫ上の点から、標準モデルＤＳの対応する輪郭ＲＫへ垂線を降ろし、その垂線のうち最短のものを距離とする。なお、輪郭ＲＫ上では複数の点を指定する。
【００７２】
計測データの輪郭ＲＫが３次元データＤＴ上に指定される場合に、標準モデルＤＳの輪郭ＲＫ毎の差分エネルギーｅ2 は、それらの距離の二乗和を用いて次の（８）式によって計算される。
【００７３】
【数５】

【００７４】
但し、
Ｔ2A：制御点群
ｐk ：３次元データ上の輪郭点
ｑk ：３次元データ上の輪郭点から対応するモデル輪郭への垂足点
ｎ ：１つのモデル輪郭に対応が付けられている３次元データの輪郭点数
ｄ^k ：計測データの輪郭点から対応するモデル輪郭線への投影方向，
ｄ^k ＝（ｑk-ｐk ）／｜ｑk-ｐk ｜
ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
〔標準モデル上の特徴点と対応した計測データ上の特徴点との距離〕
計測データ上に特徴点ＴＴを設定することにより、３次元データＤＴ上に指定された特徴点ＴＴ、または２次元画像ＦＴ上に指定された特徴点ＴＴと、標準モデルＤＳ上の特徴点との距離が評価される。
【００７５】
３次元データＤＴ上の特徴点ＴＴと標準モデルＤＳ上の特徴点ＴＴとの差分エネルギーｅ3 は、対応する特徴点ＴＴの二乗距離を用いて次の（１０）式によって計算される。
〔過剰な変形を回避するための安定化エネルギー〕
上に述べた差分のエネルギーに加え、過剰な変形を回避するための安定化エネルギーｅs が導入される。
【００７６】
すなわち、変形に用いられる制御点の間が、図１６に示す仮想バネ(elastic bar) ＫＢによってつながれているものとする。仮想バネＫＢの制約に基づいて、標準モデルＤＳの面Ｓの形状の安定化のための安定化エネルギーｅs が定義される。
【００７７】
なお、仮想バネは必ずしも制御点間に張られている必要はない。制御点と仮想バネとの関係が明確であればよい。
図１６において、フィッティング対象である標準モデルＤＳの面Ｓの一部が示されている。面Ｓは、制御点群Ｕ＝｜ｕｉ，ｉ＝１…Ｎ｜で形成されている。隣接する制御点間には、仮想バネＫＢが配置されている。仮想バネＫＢは、制御点間に引っ張り力による拘束を与え、面Ｓの異常変形を防ぐ働きをする。
【００７８】
つまり、隣接する制御点ｕの間隔が大きくなった場合に、それに応じて仮想バネＫＢによる引っ張り力が大きくなる。例えば、点Ｑｋが点Ｐｋに近づく場合に、その移動にともなって制御点ｕの間隔が大きくなると、仮想バネＫＢによる引っ張り力が増大する。点Ｑｋが移動しても制御点ｕの間隔が変わらなければ、つまり制御点ｕ間の相対位置関係に変化がなければ、仮想バネＫＢによる引っ張り力は変化しない。仮想バネＫＢによる引っ張り力を面Ｓの全体について平均化したものを、安定化エネルギーｅs として定義する。したがって、面Ｓの一部が突出して変形した場合に安定化エネルギーｅs は増大する。面Ｓの全体が平均して移動すれば安定化エネルギーｅs は零である。
【００７９】
安定化エネルギーｅs は、仮想バネＫＢの変形の状態により、次の（１１）式により求められる。
【００８０】
【数６】

【００８１】
但し、
ＴsA：制御点群
Ｕ〜m,Ｖ〜m ：仮想バネの端点（制御点）の初期値
Ｕm,Ｖm ：変形後の仮想バネの端点
Ｌ0m：初期状態の仮想バネの長さ，
Ｌ0m＝｜Ｕ〜m −Ｖ〜m ｜
Ｍ ：仮想バネの本数
ｃ ：バネ係数
Ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
したがって、バネ係数ｃを大きくすると、仮想バネＫＢは硬くなって変形し難くなる。
【００８２】
このような安定化エネルギー関数ｅs を導入することにより、面Ｓの形状変化に一定の拘束を設けることとなり、面Ｓの過度の変形を防ぐことができる。
〔総合のエネルギー関数〕
上に述べたように、各エネルギー関数ｅ1,ｅ2,ｅ3,ｅ4 について、それぞれ制御点群Ｔ1A, Ｔ2A, Ｔ3A, ＴsAが用いられる。ここでは、これらの制御点群Ｔ1A〜ＴsAは同じであるが、後述するように互いに異ならせることができる。これら制御点群ＴＡを用いて標準モデルＤＳの変形を行い、次の（１２）式に示す総合エネルギー関数ｅ（ＴＡ）を最小にする制御点群ＴＡを求める。
【００８３】
【数７】

【００８４】
但し、
ｅ1(Ｔ1A）: ３次元データの構成点とモデル表面との差分エネルギー
e2^s (T2A）: モデル輪郭毎の計測データ上の輪郭との差分エネルギー
ｅ3(Ｔ3A）: 計測データの特徴点とモデル上の特徴点との差分エネルギー
ｅS(ＴSA）: 過剰な変形を回避するための安定化エネルギー
ｗi, c : それぞれのエネルギーのウエイトパラメータ
ＴＡ＝Ｔ1A＝Ｔ2A＝Ｔ3A＝ＴSA
〔繰り返し変形〕
実際には繰り返し変形を行う（＃１７）。つまり、制御点を動かして繰り返して変形を行う。ｎ回目の変形後の総合エネルギー関数をｅn(ＴＡ）とすると、次の（１３）式の条件が満たされたときに、総合エネルギー関数ｅn （ＴＡ）が収束したと判断する。
【００８５】
【数８】

【００８６】
さて、ここで、変形処理の全体的な流れを図１０に沿って説明する。まず、計測データと標準モデルＤＳとの間で対応する点の組みを作成する（図３のＰｋとＱｋ）（＃２１）。
【００８７】
面Ｓを変形し（＃２２）、変形後の総合エネルギー関数ｅn(ＴＡ）を計算する（＃２３）。総合エネルギー関数ｅn(ＴＡ）が収束するまで（＃２４でイエス）、処理を繰り返す。
【００８８】
総合エネルギー関数ｅn(ＴＡ）の収束を判定する方法として、上に述べたように総合エネルギー関数ｅn(ＴＡ）が所定の値よりも小さくなったときを収束とする方法、前回の計算と比較べた変化の割合が所定値以下となったときに収束とする方法など、公知の方法を用いることが可能である。
〔差分表示〕
ステップ＃１７までの処理によってかなり精密な３次元モデルＭＬを得ることができる。しかし、ステップ＃１７までの処理は対象物全体を対象として行われているので、局所的に特徴を有する部分については必ずしも精密なフィッティングがなされているとは限らない。そこで、対象物のうち特徴のある部分またはステップ＃１７までの処理で十分なフィッティングがなされていない部分を３次元データＤＴから抽出し、抽出した３次元データＤＴのみを用いて、さらに標準モデルＤＳのフィッティングを行う。
【００８９】
そのような部分を抽出するために、ステップ＃１７までの処理によって得られた３次元モデルＭＬと、対象物の３次元データＤＴとの差分ＳＢの状態を差分表示画面ＨＧ１として表示する。ユーザは、差分表示画面ＨＧ１を見ることによって、差分ＳＢの状態を知り、さらにフィッティング処理を行う必要のある部分を抽出することができる。また、さらにフィッティング処理を行う必要のある部分をモデリングプログラムＰＲによって自動的に認識させ、以下の処理を自動的に続行させることも可能である。
【００９０】
図１１において、２つの３次元データの差分ＳＢを求め（＃３１）、各領域ＥＡについての代表値ＤＨを求める（＃３２）。
求められた代表値ＤＨの大きさに応じて、各領域ＥＡの表示色ＣＬを決定する（＃３３）。
【００９１】
決定した表示色ＣＬによってそれぞれの領域ＥＡを表示する（＃３４）。
〔部分領域変形〕
図１７は部分領域ＢＲＹの抽出の例を示す図、図１８（Ａ）（Ｂ）（Ｃ）は部分領域ＢＲＹを用いて標準モデルＤＳの変形を行う方法の例を説明する図である。
【００９２】
図１７に示すように、３次元データＤＴから部分領域ＢＲＹを抽出する。各部分領域ＢＲＹａ、ＢＲＹｂ、ＢＲＹｃは、それぞれ、対象物の目およびその周辺、口およびその周辺、鼻およびその周辺を、３次元データＤＴより抽出して得られた３次元データである。また、各部分領域ＢＲＹは、必要に応じて、データ量を減らすためにデータ削減が行われる。データ削減の処理は、例えば、ステップ＃１５で行ったデータ削減と同様の手順で行う。後述する部分領域変形では、フィッティングの領域が部分領域ＢＲＹに限定されるので、（６）式の右辺をステップ＃１５のときよりも０に近くしてデータ削減率を低くし、これによって部分領域ＢＲＹの精密性を維持した場合であってもフィッティング処理に多くの時間を要さない。従って、部分領域ＢＲＹの範囲の広さ、要求する精密性、および処理時間などを考慮して、部分領域ＢＲＹごとにデータ削減率を設定することができる。部分領域ＢＲＹの抽出は、ステップ＃１２の処理のとき、つまり、３次元データＤＴを得たときに予め行っておいてもよいし、ステップ＃１７までの処理が完了した後に行ってもよい。また、上のステップ＃１９で述べた方法を用いることが可能である。
【００９３】
図１８に示すように、標準モデルＤＳ１ａ、ＤＳ１ｂは、標準モデルＤＳ１の部分領域変形の過程を示すものである。
ステップ＃１９においては、ステップ＃１７までの処理で得られた図１８（Ａ）の標準モデルＤＳ１を、図１８（Ｂ）に示すように部分領域ＢＲＹａを用いてフィッティングし、標準モデルＤＳ１ａを得る。フィッティングは、ステップ＃１６で行ったように、変形ないし繰り返し変形と同様の手順で行われ、（７）式の制御点群Ｔ1A、点Ｐk 、点Ｑk 、または（１３）式のしきい値εなどについて、当該部分領域ＢＲＹに応じて好適な値を用いればよい。
【００９４】
それぞれの部分領域ＢＲＹについて、順次、部分領域変形を行う（＃２０でイエス、＃１９）。各部分領域ＢＲＹごとに、制御点群などをそれぞれ異ならせてフィッティングを行う。これにより、各部分領域ＢＲＹごとに、標準モデルＤＳ１をさらに緻密にフィッティングさせることができる。
【００９５】
例えば、図１８（Ｂ）に示す標準モデルＤＳ１ａを得た後、図１８（Ｃ）に示すように部分領域ＢＲＹｂを用いてフィッティングし、標準モデルＤＳ１ｂ（３次元モデルＭＬ）を得る。
【００９６】
なお、各部分領域ＢＲＹは、他の部分領域ＢＲＹの一部と重なり、または他の部分領域ＢＲＹに含まれていてもよい。例えば、図１７に示すように、部分領域ＢＲＹａの一部と部分領域ＢＲＹｃの一部とが重なっていてもよいし、部分領域ＢＲＹｂについて部分領域変形を行った後に、上唇のみを新たな部分領域ＢＲＹとして抽出し、部分領域変形を行ってもよい。
【００９７】
上に述べた実施形態によると、全体的に標準モデルＤＳのフィッティングを行った後、部分領域ＢＲＹを用いて部分的に標準モデルＤＳのフィッティングを行うので、目または口もとなどの局部について局部的な異常変形を起こすことなく、それらをよりよく一致させることができる。したがって、対象物により一層近い３次元モデルＭＬを生成することができる。
【００９８】
上に述べた実施形態において、領域ＥＡの大きさ、個数、または形状、しきい値ＴＨの値または単位、表示色ＣＬの種類、代表値ＤＨの求め方などは、上に述べた以外の種々のものとすることができる。その他、モデリング装置１の構成、回路、処理内容、処理順序、処理タイミング、係数の設定などは、本発明の趣旨に沿って適宜変更することができる。
【００９９】
【発明の効果】
本発明によると、２つの３次元データの差分がどのようになっているかを容易に把握することができる。
【０１００】
請求項２、５、６、７の発明によると、どの部分を加工しまたは変形すればよいのかが分かり易い。
請求項３、７の発明によると、標準モデルを人の頭部の計測データにフィッティングして３次元モデルを生成する場合に、目または口もとなどの局部について、局部的な異常変形を起こすことなく、よりよく一致させることができる。
【図面の簡単な説明】
【図１】本発明の実施形態に係るモデリング装置を示すブロック図である。
【図２】ある標準モデルに定義された領域の例を示す図である。
【図３】３次元モデルと３次元データとの差分を説明するための図である。
【図４】 各領域に設定されたしきい値の例を示す図である。
【図５】 しきい値の範囲に対して設定された表示色の例を示す図である。
【図６】差分表示画面の例を示す図である。
【図７】３次元データの各点に対して色を設定した場合の差分表示画面の例を示す図である。
【図８】歯車製品についての差分表示画面の例を示す図である。
【図９】 モデリング装置の全体の処理の流れを示すフローチャートである。
【図１０】変形処理を示すフローチャートである。
【図１１】差分表示処理を示すフローチャートである。
【図１２】標準モデルの例を示す図である。
【図１３】対象物から３次元データを取得する様子を示す図である。
【図１４】 概略の位置合わせの様子を示す図である。
【図１５】輪郭および特徴点の抽出処理の様子を示す図である。
【図１６】標準モデルの異常変形を防ぐための仮想バネを説明するための図である。
【図１７】部分領域の抽出の例を示す図である。
【図１８】部分領域を用いて標準モデルの変形を行う方法の例を説明する図である。
【符号の説明】
１ モデリング装置（差分の表示装置）
１０ 処理装置
１３ ディスプレイ装置（表示する手段）
ＤＴ ３次元データ
ＢＲＹ 部分領域
ＤＳ，ＤＳ１，ＤＳ１ａ 標準モデル（３次元データ）
ＤＳ１ｂ 標準モデル（３次元データ）
ＭＬ ３次元モデル（３次元データ）
ＳＢ 差分
ＥＡ 領域
ＤＨ 代表値
ＣＬ 表示色
ＰＲ モデリングプログラム[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a display device for a difference between two three-dimensional data and a computer program.
[0002]
[Prior art]
Conventionally, in order to create a three-dimensional model of an object, a standard model for the object is prepared, and the standard model is deformed (fitted) according to the three-dimensional data obtained by actually measuring the object. ) Has been proposed (Japanese Patent Laid-Open No. 5-81377).
[0003]
When performing fitting, in order to generate an accurate three-dimensional model with as little data processing amount as possible, it is possible to know the degree of error (difference) between the three-dimensional model obtained by fitting and the three-dimensional data of the object. desirable. For this purpose, the difference between them is displayed in color.
[0004]
Further, in order to inspect the finished accuracy of the manufactured article based on the CAD data, the manufactured article is measured three-dimensionally, the obtained measurement data is compared with the CAD data, and their error (difference) is compared. It is proposed to display the image by color (Japanese Patent Laid-Open No. 11-195054).
[0005]
[Problems to be solved by the invention]
However, according to the conventional display method, the display color is determined for the difference of each point of the three-dimensional data, so that a plurality of different colors are displayed in a narrow range due to the difference of each difference. It becomes. Therefore, when the user looks at it, a plurality of colors are mixed, and it is often difficult to grasp how the difference is.
[0006]
In addition, it is necessary to further process and deform the parts of the article and the three-dimensional model to increase the accuracy, but in the conventional display method, which part should be processed or deformed? Is difficult to understand.
[0007]
The present invention has been made in view of the above-described problems, and an object of the present invention is to make it easy to grasp the difference between two pieces of three-dimensional data.
[0008]
[Means for Solving the Problems]
  An apparatus according to the present invention is a display device for a difference between two three-dimensional data, on the surface related to the two three-dimensional data or one of the three-dimensional data.A means for obtaining a difference between opposite points of the two three-dimensional data for each region composed of a plurality of points, and for each regionMeans for obtaining a representative value for the difference;The display color of the areaI was askedAboveDepending on the size of the representative valueDecideMeans for determining.
[0009]
  Preferably, beforeDecisionThe determining means includes means for setting one or a plurality of threshold values to be variable for each region, and determines a display color for each range defined by the set threshold values.
[0010]
  According to another aspect of the present invention, there is provided a device for displaying a difference between two three-dimensional data, wherein the two three-dimensional data or a plurality of regions defined on a surface related to any one of the three-dimensional data For each, a means for obtaining a representative value for the difference between the two three-dimensional data, a means for determining the display color of the area in accordance with the magnitude of the obtained representative value, and each area by the determined display color Means for determining, and the means for determining the display color of the area has means for variably setting one or a plurality of threshold values for each area, depending on the set threshold value The display color is determined for each specified range.
  Preferably, each of the devices includes means for correcting any three-dimensional data in units of regions. Alternatively, the means for obtaining the representative value may obtain the representative value of the difference obtained by the means for obtaining the difference by calculating one of a simple average, a square average, or a weighted average of the difference. Good.
  A computer program according to the present invention is a computer program for controlling a computer that displays a difference between two three-dimensional data, and is related to the two three-dimensional data or one of the three-dimensional data. A process for obtaining a difference between opposing points of the two three-dimensional data for each area composed of a plurality of points on the surface; a process for obtaining a representative value for the obtained difference for each area; And a process of determining a display color of the area according to the size of the representative value obtained for the area.
[0011]
In the present invention, display colors include achromatic colors such as white, gray, and black. The concept of color includes density.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment in which a display device according to the present invention is applied to a modeling device 1 that generates a three-dimensional model of a human head will be described. The modeling apparatus 1 generates a three-dimensional model by deforming (fitting) a standard model created in advance to fit measurement data (three-dimensional data or a two-dimensional image) on a human head.
[0013]
1 is a block diagram showing a modeling apparatus 1 according to an embodiment of the present invention, FIG. 2 is a diagram showing an example of an area EA defined in a certain standard model DS, and FIG. 3 is a diagram showing a 3D model ML and 3D data DT. FIG. 4 is a diagram illustrating an example of the threshold value TH set for each area EA, and FIG. 5 is an example of the display color CL set for the range of the threshold value TH. FIG. 6 is a diagram showing an example of the difference display screen HG1, FIG. 7 is a diagram showing an example of the difference display screen HG1j when colors are set for each point of the three-dimensional data DT, and FIG. It is a figure which shows the example of the difference display screen HG2 about a product.
[0014]
As shown in FIG. 1, the modeling device 1 includes a processing device 10, a magnetic disk device 11, a medium drive device 12, a display device 13, a keyboard 14, a mouse 15, and a three-dimensional measuring device 16.
[0015]
The processing device 10 includes a CPU, a RAM, a ROM, a video RAM, an input / output port, various controllers, and the like. Various functions are realized on the processing device 10 by the CPU executing programs stored in the RAM and ROM.
[0016]
The magnetic disk device 11 includes an OS (Operating System), a modeling program PR for generating a three-dimensional model ML, other programs, a standard model (standard model data) DS, three-dimensional data (three-dimensional measurement data) DT, A two-dimensional image (two-dimensional measurement data) FT, a generated three-dimensional model ML, and other data are stored. These programs and data are loaded into the RAM of the processing device 10 at appropriate times.
[0017]
The modeling program PR includes programs for measurement processing, rough alignment, data reduction processing, deformation processing, partial region selection processing, difference display processing, and other processing.
[0018]
The medium drive device 12 accesses a recording medium such as a CD-ROM (CD), a floppy disk FD, or a magneto-optical disk, and reads / writes data or programs. An appropriate drive device is used according to the type of the recording medium. The modeling program PR described above can be installed from these recording media. Standard model DS, three-dimensional data DT, two-dimensional image FT, and the like can also be input via a recording medium.
[0019]
The display surface HG of the display device 13 includes the above-described various data, the three-dimensional model ML generated by the modeling program PR, the generated or generating three-dimensional model, and the three-dimensional data of the object. A difference display screen showing the difference, other data (image) or screen is displayed.
[0020]
The keyboard 14 and the mouse 15 are used for inputting data or giving commands to the processing device 10.
The three-dimensional measuring device 16 is for obtaining three-dimensional data DT of an object by, for example, a light cutting method. It is also possible to obtain the three-dimensional data DT directly by the three-dimensional measuring device 16, and the calculation is performed by the processing device 10 or the like based on the data output from the three-dimensional measuring device 16 and indirectly three-dimensionally. It is also possible to obtain data DT.
[0021]
Simultaneously with the three-dimensional data DT, it is also possible to acquire a two-dimensional image FT on the same line of sight for the same object as necessary. As such a three-dimensional measuring device 16, for example, a known device disclosed in JP-A-10-206132 can be used.
[0022]
As another known method for acquiring the three-dimensional data DT of the object, there is a method using a plurality of cameras arranged with parallax with respect to the object. From a plurality of images having parallax obtained from these cameras, three-dimensional data DT can be obtained by calculation using a stereoscopic photography method.
[0023]
In particular, in the present embodiment, as described above, the difference display screen showing the difference SB between the two three-dimensional data (that is, the standard model DS or the three-dimensional model ML and the three-dimensional data DT of the object) in color. HG1 can be displayed on the display surface HG of the display device 13.
[0024]
In generating the difference display screen HG1, a plurality of areas EA1, EA2, EA3,... Are defined in advance on the surface of the structured standard model DS.
In defining the area EA, the required accuracy of each part in the standard model DS is taken into consideration, and one area EA is defined by a part that requires the same degree of accuracy.
[0025]
In the example shown in FIG. 2, the standard model DS is a human head, the area around the mouth that requires high accuracy in order to greatly affect the facial expression is the area EA1, the area around the eyes is the area EA3, EA4, An area around the nose that requires a certain degree of accuracy is defined as an area EA2, and another area that requires low accuracy is defined as an area EA5.
[0026]
In addition, the area EA may be defined for each part unit, processing unit, or correction unit.
For each of these areas EA, a representative value DH for the difference SB between the standard model DS and the three-dimensional data DT is obtained. As the representative value DH, for example, an average value or a root mean square value in each area EA of the difference SB of each three-dimensional data is used.
[0027]
The difference SB is obtained, for example, as the distance between one point on one three-dimensional data and one point on the other three-dimensional data. Or it calculates | requires as a distance from one point on one three-dimensional data to the surface on the other three-dimensional data.
[0028]
For example, in the example shown in FIG. 3, a perpendicular line is drawn from one point on the three-dimensional data DT to the surface of the deformed three-dimensional model ML, and the length of the perpendicular line is defined as a difference SB.
In addition, one or a plurality of threshold values TH are set to be variable for each area EA in advance for each area EA.
[0029]
That is, in the example shown in FIG. 4, two threshold values TH1 and TH2 are set for each area EA. For the most accurate areas EA1, 3, and 4, the threshold value 1 is “0.1” and the threshold value 2 is “0.3”. For the medium accuracy area EA2, the threshold value 1 is “0.3” and the threshold value 2 is “0.5”. Since no threshold value is set for the low-precision area EA5, the threshold values “0.5” and “1.0” set as default values are used in this case.
[0030]
In setting the threshold value TH, the value can be determined based on experiments and experiences. Further, when generating a difference display screen of an article manufactured based on CAD data, the threshold value TH may be calculated from an article manufacturing tolerance or an allowable tolerance. A default value may be applied to the area EA in which the threshold value TH is not set.
[0031]
A preset display color CL is assigned to each range defined by the set threshold value TH.
In the example shown in FIG. 5, the display color CL is “blue” when the threshold value is 1 or less, and the display color CL is “yellow” when the threshold value is 1 or 2 and the threshold value is 2 or more. The display color CL is “red”. These colors “blue”, “yellow”, and “red” may be other various colors. Instead of the color, a code for specifying the color may be used. Further, it may be data of the color itself.
[0032]
Based on these settings, the display color CL of each area EA is determined. The difference display screen HG1 displays each area EA with the determined display color CL. An example is shown in FIG.
[0033]
As shown in FIG. 6, in the three-dimensional model ML (or standard model DS), one display color CL is assigned to each area EA and displayed.
In the example of FIG. 6, it can be seen that the representative values DH of the eye areas EA3 and EA4 are not less than the threshold value 2 and the difference SB is large. Therefore, it is determined that further fitting is necessary for the areas EA3 and EA4. It can be seen that the nose area EA2 has a representative value DH equal to or less than the threshold value 1, and the difference SB is sufficiently small. Therefore, it is determined that the area EA2 is sufficiently accurate and the fitting may be terminated.
[0034]
Thus, in the three-dimensional model ML, the user can easily understand the difference SB from the three-dimensional data DT. Further, the modeling program PR can automatically recognize the state of the difference SB. Therefore, according to the state of the difference SB, it is possible to determine whether to continue the fitting or end the fitting.
[0035]
Further, when determining whether or not to continue the fitting, such a determination can be made in units of each area EA. When fitting is necessary, fitting is performed only for the necessary area EA. Therefore, in that case, unnecessary fitting is avoided, and an accurate three-dimensional model ML can be generated with a small amount of data processing.
[0036]
Further, as described above, the definition of the area EA may be performed for the standard model DS, and does not have to be performed for the three-dimensional data DT of the object. Therefore, the definition of the area EA is easy. When the area EA is defined for the standard model DS, the area EA can be determined in accordance with the required accuracy of each part.
[0037]
On the other hand, when the color is set for each point of the three-dimensional data DT without defining the area EA, the color of each point is mixed and the state of the difference SB cannot be grasped. . Further, since the same threshold value TH is set for each point, it is not possible to clearly distinguish a portion that requires accuracy from a portion that does not.
[0038]
In FIG. 8, an area EA is defined for the difference SB between the gear product manufactured based on the CAD data and the measurement data obtained by performing the three-dimensional measurement of the gear product, as in FIG. The display color CL corresponding to the representative value DH is displayed.
[0039]
In the example of FIG. 8, for the tooth surface of the gear product, the representative value DH is smaller than the threshold value TH1, and it is determined that the accuracy is sufficient. However, for the end face of the gear product, the representative value DH is larger than the threshold value TH2, and it is determined that the thickness of the gear product needs to be further processed to increase the accuracy.
[0040]
In addition, when the difference SB between the design drawing (CAD data) of the industrial product and the actual product measurement data is displayed as in the example illustrated in FIG. 8, if the root mean square is taken as the representative value DH, the sign of the difference SB is positive or negative. Can not be distinguished from each other, a simple average or a weighted average may be used instead of a square average.
[0041]
The display color CL includes achromatic colors such as white, gray, and black. The colors are determined by, for example, the densities of the three primary colors, and therefore different colors have different colors. That is, the concept of color includes density.
[0042]
For example, by changing the gray scale density as the display color CL, the tendency of the difference SB can be easily grasped.
The modeling apparatus 1 can be configured using a personal computer or a workstation. The programs and data described above can also be obtained by receiving via the network NW.
[0043]
Next, an overall processing flow of the modeling apparatus 1 will be described with reference to a flowchart.
9 is a flowchart showing the overall processing flow of the modeling apparatus 1, FIG. 10 is a flowchart showing the deformation process, FIG. 11 is a flowchart showing the difference display process, FIG. 12 is an example of the standard model DS1, and FIG. FIGS. 14A and 14B are diagrams illustrating a state of acquiring the three-dimensional data DT from the object, FIGS. 14A and 14B are diagrams illustrating a schematic alignment state, and FIGS. FIG. 16 is a diagram for explaining a virtual spring for preventing abnormal deformation of the standard model DS.
[Preparation of standard model]
In FIG. 9, first, a standard model DS for an object is prepared (# 11). In this embodiment, since the object is a human head, it is most similar to the head of the object from among a plurality of standard model groups with various sizes and shapes for the entire circumference of the head. A standard model DS1 is prepared.
[0044]
The standard model DS may be either a three-dimensional shape model defined by polygons or a three-dimensional shape model defined by free-form surfaces. In the case of a three-dimensional shape model defined by polygons, the surface shape is determined by the three-dimensional coordinates of the vertices of each polygon. In the case of a three-dimensional shape model defined by a free-form surface, the shape of the surface is determined by the function that defines the surface and the coordinates of each control point.
[0045]
In the case of a three-dimensional shape model defined by polygons, the vertices of each polygon are described as “component points”.
In addition, when fitting the standard model DS, a point used to deform the standard model is referred to as a “control point”. The positional relationship between the control points and the constituent points of the polygon is arbitrary, and the control points may be set on the surface of the polygon or may be set away from the surface of the polygon. One control point is associated with a plurality of component points (about 3 to 100), and the associated component point moves in accordance with the movement of the control point. When fitting the standard model DS, the entire standard model DS is deformed by moving these control points. Even when the three-dimensional shape model is defined by a free-form surface, the arrangement of control points used for fitting is arbitrary.
[0046]
Control points are arranged at a high density in portions having fine shapes such as the corners of the eyes and lips, and in portions having rapid shape changes such as the nose and lips. The other portions are uniformly arranged.
[0047]
In the standard model DS, a characteristic contour RK and a characteristic point TT viewed from a certain direction are set. As the contour RK, for example, a wrinkle line, a nose line, a lip line, or a chin line is set for the eyes, nose, mouth, or jaw. As the feature point TT, for example, there is an actual characteristic part such as the end of the eye or mouth, the top of the nose, the lower end of the chin, or the middle of them, but there is no feature. A part that is easy to specify is selected.
[0048]
In the standard model DS1 shown in FIG. 12, chin lines, lip lines, and wrinkle lines are set as contours RK1 to RK3. As can be seen in FIG. 12, the contour RK1 is a portion that becomes an edge when the standard model DS1 is viewed from a certain direction. In the standard model DS1 shown in FIG. 12, only a part of the set feature points TT is actually shown in the drawing.
[0049]
As described above, a plurality of areas EA1, EA2, EA3... Are defined for the standard model DS. A threshold value TH is set for each area EA. The display color CL for the range based on the threshold value TH is also set.
[Acquisition of 3D data]
Next, three-dimensional measurement of the object is performed to obtain three-dimensional data DT (# 12). At that time, a two-dimensional image FT of the object is also acquired at the same time. Further, the reliability data DR indicating the reliability of each point of the three-dimensional data DT or information for obtaining the reliability data DR is acquired as necessary.
[0050]
For example, as shown in FIG. 13, the head of a person who is the object is measured (captured) using a three-dimensional measuring device 16. Thereby, the three-dimensional data DT and the two-dimensional image FT are acquired.
[0051]
Note that the three-dimensional data DT and / or the two-dimensional image FT obtained by measuring the object may be referred to as “measurement data”. Either the preparation of the standard model DS or the acquisition of the three-dimensional data DT may be performed first or in parallel.
[Rough alignment]
Approximate alignment between the standard model DS and the three-dimensional data DT is performed (# 13). In this process, the orientation, size, and position of the standard model DS are changed so that the standard model DS and the three-dimensional data DT substantially match. At this time, the size of each standard model DS can be matched to the three-dimensional data DT by scaling the standard model DS individually to an arbitrary magnification in each of the X, Y, and Z directions.
[0052]
For example, as shown in FIG. 14A, by rotating the standard model DS and scaling in each direction with respect to the three-dimensional data DT, the three-dimensional data DT as shown in FIG. And a standard model DSa of approximately the same size can be obtained. In addition, in order to make it intelligible, the thing which is not adjusting the position on drawing is shown.
[Overall approximate alignment]
In the overall approximate alignment, the position, direction, and size of the standard model DS are changed so that the distance between the three-dimensional data DT and the standard model DS is minimized.
[0053]
That is, si, αi, ti that minimizes the energy function e (si, αi, ti) shown in the following equation (1) is derived.
Note that f (si, αi, ti) is an energy function defined in relation to the distance between the three-dimensional data DT and the standard model DS. g (si) is a stabilization energy function for avoiding excessive deformation.
[0054]
In addition, using the two-dimensional image FT acquired at the same time when the three-dimensional measuring device 16 acquires the three-dimensional data DT, using the pattern matching on the two-dimensional image FT, the initial values of position, direction, and size are set. May be given.
[0055]
[Expression 1]

[0056]
However,
K: Number of constituent points of 3D data
dk: distance between the constituent points of the three-dimensional data and the surface of the standard model
Wsc: Weight parameter for partial magnification stabilization
S0: Initial scale
Si: Amount of magnification in each direction (where S3 is the depth direction)
αi: Rotation in each direction of the standard model
t i: Amount of movement of the standard model in each direction
Here, the constituent points on the standard model DS move according to the following equation (2), and accordingly, the distance dk between the constituent points of the three-dimensional data DT and the surface of the standard model DS changes.
[0057]
[Expression 2]

[0058]
[Local approximate alignment]
If the overall rough alignment described above is performed automatically and it does not fit well, it will be manually aligned. However, the local rough alignment described here is a manual alignment. It is a technique to do as easily as possible. In addition, the part which did not work automatically is reset once, and it starts it manually from the beginning.
[0059]
In the local approximate alignment, the characteristic line or point on the three-dimensional data DT and the characteristic line or point on the standard model DS are associated with each other so that the distance between them is minimized. Change position, direction, and size. When lines are associated with each other, the feature point is the point on one line and the shortest of the perpendiculars drawn from that point onto the other line, and multiple points are obtained on the line. It shall be.
[0060]
That is, the energy function E (si, αi, ti) shown in the following equation (4) is minimized with respect to the distance between the feature point on the three-dimensional data DT and the corresponding feature point on the standard model DS. Thus, ti, αi, and si of the standard model DS are derived.
[0061]
[Equation 3]

[0062]
However,
k: number of corresponding feature points
Mk: Feature point on the standard model after alignment
x: Feature point on the standard model before alignment
Ck: Feature point on 3D data
Si: Deviation amount in each direction of the standard model
αi: Rotation in each direction of the standard model
t i: Amount of movement of the standard model in each direction
[Extraction of contour and feature points]
Outlines and feature points are extracted on the three-dimensional data DT or the two-dimensional image FT (# 14). When the contour RK and the feature point TT for the standard model DS have been extracted in advance, the contour and the feature point to be arranged at the same position as those are displayed on the three-dimensional data DT or the corresponding two-dimensional It arrange | positions on an image (refer FIG. 15).
[0063]
When the contour RK and the feature point TT are not extracted in advance for the standard model DS, they are also specified on the standard model DS together with the arrangement on the three-dimensional data DT or the two-dimensional image.
[Data reduction]
Next, in order to reduce the calculation amount and the error, data is reduced for the three-dimensional data DT, and only necessary and reliable data is extracted (# 15). By reducing the data, the calculation amount can be reduced without destroying the shape of the original three-dimensional data DT.
[0064]
In reducing the data, for example, data outside the object area is excluded and unnecessary data is excluded. For example, the face area is determined from the two-dimensional image FT, and only the three-dimensional data DT corresponding to the area is left. Alternatively, the region is determined using the difference in distance between the object and the background. In addition, there are various methods such as extracting a face region using the information on the approximate alignment. If the three-dimensional data DT includes the reliability data DR, only the highly reliable data is left. If there are many data in the neighborhood, the data is thinned out and the density is averaged.
[Deformation]
The entire standard model DS is deformed (# 16). Here, the energy function e1 defined in relation to the distance between each constituent point of the three-dimensional data DT and the surface of the standard model DS is used, and in addition, the feature points of the standard model DS and the three-dimensional data are used. Energy function e3 defined in relation to the distance between the feature points specified for DT, related to the distance between the contour of the standard model DS and the contour specified for the three-dimensional data DT. The energy function e2 defined in order to avoid excessive deformation and the energy function es defined to avoid excessive deformation are used to evaluate the combined energy function e so that the total energy function e is minimized. The surface of the model DS is deformed.
[0065]
It is most preferable to use four functions e1, e2, e3, and es as the total energy function e, but it is also possible to use only two of e1 to e3.
[0066]
Next, each energy function will be briefly described.
[Distance between standard model and 3D data]
In FIG. 3, one of the point groups constituting the three-dimensional data DT is indicated by a point Pk. On the surface S of the standard model DS, the point closest to the point Pk is indicated by Qk. The point Qk is an intersection when a perpendicular is drawn from the point Pk to the surface S. Here, the distance (difference SB) between the point Pk and the point Qk is evaluated.
[0067]
That is, the difference energy e1 between each point of the three-dimensional data DT and the surface of the standard model DS is a point Pk on the three-dimensional data DT after data reduction and a point Qk obtained by projecting the point Pk on the surface S of the standard model DS. Is calculated by the following equation (7).
[0068]
[Expression 4]

[0069]
However,
T1A: Control point group
Pk: Composition point of 3D data after reduction
Qk: Projection point from component point to model surface
K: Number of component points after reduction
d^k: Projection direction from component point to model surface,
d^k= (Qk-Pk) / | Qk-Pk |
ρk: Reliability of the component point Pk
w (ρk): reliability function, w (ρk) = 1 / (α + ρk)ⁿ
W: Σw (ρk)
L: Adjustment scale for handling various energies in the same unit
[Distance between contour on standard model and contour on measurement data]
Here, the distance between the contour RK designated on the three-dimensional data DT or the contour RK designated on the two-dimensional image FT and the contour RK on the standard model DS is evaluated.
[0070]
When the contour RK of the measurement data is specified on the three-dimensional data DT, a perpendicular is dropped from a point on the contour RK of the three-dimensional data DT to the corresponding contour RK on the standard model DS, and the shortest of the perpendiculars Is the distance. A plurality of points are designated on the contour RK.
[0071]
When the contour RK of the measurement data is designated on the two-dimensional image FT, the contour RK of the standard model DS is projected on the two-dimensional image FT using the camera parameters for the camera that captured the two-dimensional image FT. A perpendicular is dropped from a point on the contour RK of the two-dimensional image FT to the corresponding contour RK of the standard model DS, and the shortest one of the perpendiculars is taken as a distance. A plurality of points are designated on the contour RK.
[0072]
When the contour RK of the measurement data is specified on the three-dimensional data DT, the difference energy e2 for each contour RK of the standard model DS is calculated by the following equation (8) using the sum of squares of the distances. .
[0073]
[Equation 5]

[0074]
However,
T2A: Control point group
pk: contour point on 3D data
qk: droop point from the contour point on the 3D data to the corresponding model contour
n: Number of contour points of 3D data associated with one model contour
d^k: Projection direction from contour point of measurement data to corresponding model contour line,
d^k= (Qk-pk) / | qk-pk |
l: Adjustment scale for handling various energies in the same unit
[Distance between feature points on standard model and feature points on measurement data]
By setting the feature point TT on the measurement data, the feature point TT specified on the three-dimensional data DT or the feature point TT specified on the two-dimensional image FT and the feature point on the standard model DS The distance is evaluated.
[0075]
The difference energy e3 between the feature point TT on the three-dimensional data DT and the feature point TT on the standard model DS is calculated by the following equation (10) using the square distance of the corresponding feature point TT.
[Stabilization energy to avoid excessive deformation]
In addition to the differential energy described above, a stabilization energy es is introduced to avoid excessive deformation.
[0076]
That is, it is assumed that the control points used for the deformation are connected by a virtual spring (elastic bar) KB shown in FIG. Based on the constraint of the virtual spring KB, a stabilization energy es for stabilizing the shape of the surface S of the standard model DS is defined.
[0077]
The virtual spring does not necessarily have to be stretched between the control points. It is sufficient if the relationship between the control point and the virtual spring is clear.
In FIG. 16, a part of the surface S of the standard model DS to be fitted is shown. The surface S is formed by the control point group U = | ui, i = 1... N | A virtual spring KB is disposed between adjacent control points. The virtual spring KB acts to prevent abnormal deformation of the surface S by giving a restraint between the control points by a pulling force.
[0078]
That is, when the interval between adjacent control points u increases, the pulling force by the virtual spring KB increases accordingly. For example, when the point Qk approaches the point Pk, if the distance between the control points u increases with the movement, the pulling force by the virtual spring KB increases. Even if the point Qk moves, if the distance between the control points u does not change, that is, if the relative positional relationship between the control points u does not change, the pulling force by the virtual spring KB does not change. A value obtained by averaging the pulling force by the virtual spring KB over the entire surface S is defined as a stabilization energy es. Therefore, the stabilization energy es increases when a part of the surface S protrudes and deforms. If the entire surface S moves on average, the stabilization energy es is zero.
[0079]
The stabilization energy es is obtained by the following equation (11) depending on the deformation state of the virtual spring KB.
[0080]
[Formula 6]

[0081]
However,
TsA: Control point group
U to m, V to m: initial value of the end point (control point) of the virtual spring
Um, Vm: End point of virtual spring after deformation
L0m: length of the virtual spring in the initial state,
L0m = | U ~ m -V ~ m |
M: Number of virtual springs
c: Spring coefficient
L: Adjustment scale for handling various energies in the same unit
Therefore, when the spring coefficient c is increased, the virtual spring KB becomes hard and is not easily deformed.
[0082]
By introducing such a stabilization energy function es, a constant constraint is provided for the shape change of the surface S, and excessive deformation of the surface S can be prevented.
[Total energy function]
As described above, the control point groups T1A, T2A, T3A, and TsA are used for the energy functions e1, e2, e3, and e4, respectively. Here, these control point groups T1A to TsA are the same, but can be different from each other as will be described later. Using these control point groups TA, the standard model DS is deformed to obtain a control point group TA that minimizes the total energy function e (TA) shown in the following equation (12).
[0083]
[Expression 7]

[0084]
However,
e1 (T1A): Differential energy between the constituent points of 3D data and the model surface
e2^s(T2A): Difference energy from the contour on the measurement data for each model contour
e3 (T3A): Difference energy between feature points of measurement data and feature points on the model
eS (TSA): Stabilization energy to avoid excessive deformation
wi, c: Weight parameter of each energy
TA = T1A = T2A = T3A = TSA
[Repeated deformation]
Actually, the deformation is repeatedly performed (# 17). That is, the control point is moved and repeatedly deformed. Assuming that the total energy function after the n-th deformation is en (TA), it is determined that the total energy function en (TA) has converged when the condition of the following equation (13) is satisfied.
[0085]
[Equation 8]

[0086]
Now, the overall flow of the deformation process will be described with reference to FIG. First, a pair of corresponding points is created between the measurement data and the standard model DS (Pk and Qk in FIG. 3) (# 21).
[0087]
The surface S is deformed (# 22), and the total energy function en (TA) after deformation is calculated (# 23). The process is repeated until the total energy function en (TA) converges (Yes in # 24).
[0088]
As a method of judging the convergence of the total energy function en (TA), as described above, the method of convergence when the total energy function en (TA) becomes smaller than a predetermined value, and the previous calculation were compared. It is possible to use a known method such as a method of converging when the rate of change becomes a predetermined value or less.
[Difference display]
A fairly precise three-dimensional model ML can be obtained by the processing up to step # 17. However, since the processing up to step # 17 is performed for the entire target object, a precise fitting is not necessarily performed for a locally characteristic portion. Therefore, a characteristic part of the object or a part that has not been sufficiently fitted in the processing up to step # 17 is extracted from the three-dimensional data DT, and only the extracted three-dimensional data DT is used to further extract the standard model DS. Perform fitting.
[0089]
In order to extract such a part, the state of the difference SB between the three-dimensional model ML obtained by the processing up to step # 17 and the three-dimensional data DT of the object is displayed as a difference display screen HG1. The user can know the state of the difference SB by looking at the difference display screen HG1, and can further extract a portion that needs to be fitted. Further, it is possible to automatically recognize a portion that needs to be fitted by the modeling program PR, and to automatically continue the following processing.
[0090]
In FIG. 11, a difference SB between two three-dimensional data is obtained (# 31), and a representative value DH for each area EA is obtained (# 32).
The display color CL of each area EA is determined in accordance with the obtained representative value DH (# 33).
[0091]
Each area EA is displayed with the determined display color CL (# 34).
(Partial area deformation)
FIG. 17 is a diagram illustrating an example of extraction of a partial region BRY, and FIGS. 18A, 18B, and 18C are diagrams illustrating an example of a method for deforming the standard model DS using the partial region BRY.
[0092]
As shown in FIG. 17, the partial region BRY is extracted from the three-dimensional data DT. Each of the partial regions BRYa, BRYb, and BRYc is three-dimensional data obtained by extracting the eyes and the periphery thereof, the mouth and the periphery thereof, the nose and the periphery thereof from the three-dimensional data DT, respectively. Each partial area BRY is subjected to data reduction in order to reduce the data amount as necessary. The data reduction process is performed in the same procedure as the data reduction performed in step # 15, for example. In the partial area modification described later, since the fitting area is limited to the partial area BRY, the right side of the equation (6) is made closer to 0 than in step # 15 to lower the data reduction rate, thereby reducing the partial area. Even when the precision of BRY is maintained, the fitting process does not require much time. Accordingly, the data reduction rate can be set for each partial area BRY in consideration of the range of the partial area BRY, the required precision, the processing time, and the like. The extraction of the partial area BRY may be performed in advance in the process of step # 12, that is, when the three-dimensional data DT is obtained, or may be performed after the process up to step # 17 is completed. Further, the method described in step # 19 above can be used.
[0093]
As shown in FIG. 18, the standard models DS1a and DS1b show the process of partial area deformation of the standard model DS1.
In step # 19, the standard model DS1 of FIG. 18A obtained by the processing up to step # 17 is fitted using the partial region BRYa as shown in FIG. 18B to obtain the standard model DS1a. . The fitting is performed in the same procedure as the deformation or repetitive deformation as in step # 16, and the control point group T1A, the point Pk, the point Qk in the equation (7), or the threshold value ε in the equation (13). For example, a suitable value may be used in accordance with the partial region BRY.
[0094]
For each partial region BRY, partial region deformation is sequentially performed (Yes in # 20, # 19). For each partial region BRY, fitting is performed with different control point groups. As a result, the standard model DS1 can be fitted more precisely for each partial region BRY.
[0095]
For example, after obtaining the standard model DS1a shown in FIG. 18B, fitting is performed using the partial region BRYb as shown in FIG. 18C to obtain the standard model DS1b (three-dimensional model ML).
[0096]
Each partial region BRY may overlap with a part of another partial region BRY, or may be included in another partial region BRY. For example, as shown in FIG. 17, a part of the partial area BRYa and a part of the partial area BRYc may overlap, or after the partial area deformation is performed on the partial area BRYb, only the upper lip is replaced with a new partial area. It may be extracted as BRY and subjected to partial region deformation.
[0097]
According to the above-described embodiment, since the standard model DS is partially fitted using the partial region BRY after the standard model DS is entirely fitted, the local area such as the eyes or the mouth is localized. They can be better matched without causing abnormal deformation. Therefore, a three-dimensional model ML that is closer to the object can be generated.
[0098]
In the embodiment described above, the size, number, or shape of the area EA, the value or unit of the threshold value TH, the type of the display color CL, the method of obtaining the representative value DH, and the like can be various Can be. In addition, the configuration, circuit, processing content, processing order, processing timing, coefficient setting, and the like of the modeling apparatus 1 can be appropriately changed in accordance with the spirit of the present invention.
[0099]
【The invention's effect】
According to the present invention, it is possible to easily grasp the difference between two pieces of three-dimensional data.
[0100]
  Claim 2,5, 6, 7According to this invention, it is easy to understand which part should be processed or deformed.
  Claim3, 7According to the invention, when a three-dimensional model is generated by fitting a standard model to measurement data of a human head, a local part such as an eye or a mouth is matched better without causing local abnormal deformation. be able to.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a modeling apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram showing an example of regions defined in a certain standard model.
FIG. 3 is a diagram for explaining a difference between a three-dimensional model and three-dimensional data.
FIG. 4 is a diagram illustrating an example of threshold values set in each region.
FIG. 5 is a diagram illustrating an example of display colors set for a threshold range.
FIG. 6 is a diagram illustrating an example of a difference display screen.
FIG. 7 is a diagram illustrating an example of a difference display screen when a color is set for each point of three-dimensional data.
FIG. 8 is a diagram illustrating an example of a difference display screen for a gear product.
FIG. 9 is a flowchart showing the overall processing flow of the modeling apparatus.
FIG. 10 is a flowchart showing a deformation process.
FIG. 11 is a flowchart showing a difference display process.
FIG. 12 is a diagram illustrating an example of a standard model.
FIG. 13 is a diagram showing how three-dimensional data is acquired from an object.
FIG. 14 is a diagram showing a state of rough alignment.
FIG. 15 is a diagram illustrating a state of outline and feature point extraction processing;
FIG. 16 is a diagram for explaining a virtual spring for preventing abnormal deformation of a standard model;
FIG. 17 is a diagram illustrating an example of partial area extraction;
FIG. 18 is a diagram for explaining an example of a method for deforming a standard model using a partial region.
[Explanation of symbols]
1 Modeling device (difference display device)
10 Processing device
13. Display device (means for displaying)
DT 3D data
BRY partial area
DS, DS1, DS1a Standard model (3D data)
DS1b standard model (3D data)
ML 3D model (3D data)
SB difference
EA area
DH typical value
CL display color
PR modeling program

Claims (8)

A display device for the difference between two three-dimensional data,
Means for obtaining a difference between opposite points of the two three-dimensional data, for each region composed of a plurality of points on a surface related to the two three-dimensional data or any of the three-dimensional data;
Means for obtaining a representative value for the obtained difference for each region;
Means for determining the display color of the area according to the size of the representative value obtained for the area;
A display device for the difference between two pieces of three-dimensional data, characterized by comprising: The means for determining is
Means for variably setting one or more threshold values for each region;
Determine the display color for each range defined by the set threshold.
The display device of a difference between two three-dimensional data according to claim 1. A means for correcting any three-dimensional data in units of each region;
The display device of a difference between two three-dimensional data according to claim 1 or 2. The means for obtaining the representative value obtains the representative value of the difference obtained by the means for obtaining the difference by calculating one of a simple average, a square average, or a weighted average of the difference.
The display apparatus of the difference of the two three-dimensional data in any one of Claims 1 thru | or 3. A display device for the difference between two three-dimensional data,
Means for obtaining a representative value for a difference between two three-dimensional data for each of a plurality of regions defined on a surface related to the two three-dimensional data or one of the three-dimensional data;
Means for determining the display color of the area according to the size of the obtained representative value;
Means for displaying each area according to the determined display color;
Have
The means for determining the display color of the area is:
Means for variably setting one or more threshold values for each region;
Determine the display color for each range defined by the set threshold.
An apparatus for displaying a difference between two three-dimensional data. A display device for the difference between two three-dimensional data,
Means for obtaining a difference between two pieces of three-dimensional data for each of a plurality of regions defined on a surface related to the two pieces of three-dimensional data or any of the three-dimensional data;
Means for obtaining a representative value for the obtained difference for each region by calculating either a simple average, a square average, or a weighted average of the difference;
Means for determining the display color of the area according to the size of the obtained representative value;
Means for displaying each area according to the determined display color;
Have
The means for determining the display color of the area is:
Means for variably setting one or more threshold values for each region;
Determine the display color for each range defined by the set threshold.
An apparatus for displaying a difference between two three-dimensional data. A means for correcting any three-dimensional data in units of each region;
The difference display apparatus of the two-dimensional data of Claim 5 or Claim 6. A computer program for controlling a computer that displays a difference between two three-dimensional data,
Means for calculating a difference between opposing points of the two three-dimensional data, for each region composed of a plurality of points on a surface related to the two three-dimensional data or any of the three-dimensional data ;
Means for calculating a representative value for the calculated difference for each region ;
Causing the computer to function as means for determining the display color of the area according to the size of the representative value calculated for the area ;
A computer program characterized by the above.

Images