JP4838411B2 - Generation method of shape model - Google Patents

Description

【００４４】
但し、
Ｋ ：３次元データの構成点の個数
ｄk ：３次元データの構成点と標準モデルの表面との距離
Ｗsc：偏倍安定化のウエイトパラメータ
Ｓ0 ：初期スケール
Ｓi ：各方向の偏倍量（但し、Ｓ3 は奥行き方向である）
αi ：標準モデルの各方向の回転
ｔi ：標準モデルの各方向への移動量
ここで、標準モデルＤＳ上の構成点は次の（２）式にしたがって移動し、それにともなって、３次元データＤＴの構成点と標準モデルＤＳの表面との間の距離ｄk が変化する。
【００４５】
【数２】

【００４６】
また、３次元計測装置１６による計測（撮影）が一方向からに限られる場合に、奥行き方向（Ｚ方向）の偏倍量が正確に得られない場合がある。その場合には、３次元データＤＴの形状と標準モデルＤＳの形状に大きな違いはないとみなし、次の（３）式に示すように、Ｘ，Ｙ方向の偏倍量（Ｓ1,Ｓ2 ）によってＺ方向の偏倍量（Ｓ3 ）を補正する方法も考えられる。
【００４７】
【数３】

【００４８】
但し、
γ ：視線方向ｘ3 変形分へのウエイトパラメータ
〔局所的概略位置合わせ〕
上に述べた全体的概略位置合わせを自動で行った場合に、それがうまく合わなかったときに、手動で合わせることとなるが、ここに述べる局所的概略位置合わせは、手動での位置合わせの際にできるだけ簡単に行うための手法である。なお、自動でうまくいかなかった分は一旦リセットし、初めから手動でやり直す。
【００４９】
局所的概略位置合わせでは、３次元データＤＴ上の特徴的な線または点と、標準モデルＤＳ上の特徴的な線または点とを対応づけ、それらの距離を最小にするように標準モデルＤＳの位置、方向、およびサイズを変更する。なお、線と線とを対応付けた場合は、一方の線上の点とその点から他方の線上へ降ろした垂線のうち最短となる点とを特徴点とし、線上でこれらの点を複数点取得するものとする。
【００５０】
すなわち、３次元データＤＴ上の特徴点とそれに対応する標準モデルＤＳ上の特徴点との距離に対して、次の（４）式に示すエネルギー関数Ｅ（ｓｉ，αｉ，ｔｉ）が最小となるように、標準モデルＤＳのｔｉ，αｉ，ｓｉを導く。
【００５１】
【数４】

【００５２】
但し、
ｋ ：対応する特徴点の個数
Ｍk ：位置合わせ後の標準モデル上の特徴点
ｘ ：位置合わせ前の標準モデル上の特徴点
Ｃk ：３次元データ上の特徴点
Ｓi ：標準モデルの各方向の偏倍量
αi ：標準モデルの各方向の回転
ｔi ：標準モデルの各方向への移動量
また、３次元計測装置１６による計測（撮影）が一方向からに限られる場合に、奥行き方向（Ｚ方向）のスケールが正確に得られない場合がある。その場合には、上に述べた全体的概略位置合わせの場合と同様に、次の（５）式を用いてＺ方向のスケールを補正する方法が考えられる。
【００５３】
【数５】

【００５４】
〔輪郭・特徴点の抽出〕
３次元データＤＴまたは２次元画像ＦＴ上に、輪郭および特徴点を抽出する（＃１４）。標準モデルＤＳについての輪郭ＲＫおよび特徴点ＴＴを予め抽出しておいた場合には、それらと同じ位置に配置されるべき輪郭および特徴点を、３次元データＤＴ上に、またはそれに対応する２次元画像上に配置する（図７参照）。
【００５５】
標準モデルＤＳについての輪郭ＲＫおよび特徴点ＴＴが予め抽出されていない場合には、３次元データＤＴ上または２次元画像上への配置と合わせて標準モデルＤＳ上でも指定する。
〔データ削減〕
次に、計算量および誤差を削減するために、３次元データＤＴについてデータの削減を行い、必要且つ信頼性の高いデータのみを取り出す（＃１５）。データの削減を行うことによって、元の３次元データＤＴの形状を崩すことなく、計算量を減らすことができる。
【００５６】
データの削減に当たって、例えば、対象物の領域外のデータを除外し、不要なデータを除く。例えば、２次元画像ＦＴから顔の領域を判別し、その領域に対応した３次元データＤＴのみを残す。あるいは、対象物と背景との間の距離の相違を用いて領域を判別する。また、概略位置合わせの情報を用いて、顔の領域を抽出するなどの各種の方法がある。また、３次元データＤＴに信頼性データＤＲがある場合には、信頼性の高いもののみを残す。近隣にデータが多い場合はそのデータを間引き、密度を平均化する。
【００５７】
データを間引いて密度を平均化する場合は、次の（６）式で示される条件を満たす３次元データＤＴのみを採用する。
【００５８】
【数６】

【００５９】
但し、
Ｐk ：構成点
Ｐ〜r ：既に採用された構成点
Ｒ_det (x) ：構成点ｘの周囲の密度を表す関数
上の（６）式によると、注目されているデータＰｋについて、それまでに採用されて残っているデータＰｒとの間の距離が一定以上であれば、そのデータＰｋを採用する。
〔変形〕
標準モデルＤＳの変形が行われる（＃１６）。ここでは、３次元データＤＴの各構成点と標準モデルＤＳの面との間の距離に関連して定義されたエネルギー関数ｅ1 を用いるとともに、それに加えて、標準モデルＤＳの特徴点と３次元データＤＴに対して指定された特徴点との間の距離に関連して定義されるエネルギー関数ｅ3 、標準モデルＤＳの輪郭と３次元データＤＴに対して指定された輪郭との間の距離に関連して定義されるエネルギー関数ｅ2 、および、過剰な変形を回避するために定義されたエネルギー関数ｅs を用い、それらを総合したエネルギー関数ｅを評価し、総合のエネルギー関数ｅが最小となるように標準モデルＤＳの面を変形させる。
【００６０】
なお、総合のエネルギー関数ｅとして、ｅ１，ｅ２，ｅ３，ｅｓの４つの関数を用いるのが一番望ましいが、ｅ１〜ｅ３のうち任意の２つだけを用いることも可能である。
【００６１】
次に、各エネルギー関数について順次説明する。
〔標準モデルと３次元データとの距離〕
図８において、３次元データＤＴを構成する点群の１つが点Ｐｋで示されている。標準モデルＤＳの面Ｓにおいて、点Ｐｋに最も近い点がＱｋで示されている。点Ｑｋは、点Ｐｋから面Ｓに垂線を下ろしたときの交点である。ここでは、点Ｐｋと点Ｑｋとの距離が評価される。
【００６２】
すなわち、３次元データＤＴの各点と標準モデルＤＳの面との差分エネルギーｅ1 は、データ削減後の３次元データＤＴ上の点Ｐｋと、それを標準モデルＤＳの面Ｓ上に投影した点Ｑｋとの二乗距離を用いて、次の（７）式によって算出される。
【００６３】
【数７】

【００６４】
但し、
Ｔ1A：制御点群
Ｐk ：削減後の３次元データの構成点
Ｑk ：構成点からモデル表面への投影点
Ｋ ：削減後の構成点の個数
ｄ^k ：構成点からモデル表面への投影方向，
ｄ^k ＝（Ｑk-Ｐk ）／｜Ｑk-Ｐk ｜
ρk ：構成点Ｐk の信頼性
w(ρk)：信頼性関数，w(ρk)＝１／（α＋ρk ）ⁿ
Ｗ ：Σw(ρk)
Ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
〔標準モデル上の輪郭と計測データ上の輪郭との距離〕
ここでは、３次元データＤＴ上に指定された輪郭ＲＫ、または２次元画像ＦＴ上に指定された輪郭ＲＫと、標準モデルＤＳ上の輪郭ＲＫとの距離が評価される。
【００６５】
計測データの輪郭ＲＫが３次元データＤＴ上に指定される場合は、３次元データＤＴの輪郭ＲＫ上の点から標準モデルＤＳ上の対応する輪郭ＲＫへ垂線を降ろし、その垂線のうち最短のものを距離とする。なお、輪郭ＲＫ上では複数の点を指定する。
【００６６】
計測データの輪郭ＲＫが２次元画像ＦＴ上に指定される場合は、２次元画像ＦＴを撮影したカメラについてのカメラパラメータを用い、標準モデルＤＳの輪郭ＲＫを２次元画像ＦＴ上に投影する。２次元画像ＦＴの輪郭ＲＫ上の点から、標準モデルＤＳの対応する輪郭ＲＫへ垂線を降ろし、その垂線のうち最短のものを距離とする。なお、輪郭ＲＫ上では複数の点を指定する。
【００６７】
計測データの輪郭ＲＫが３次元データＤＴ上に指定される場合に、標準モデルＤＳの輪郭ＲＫ毎の差分エネルギーｅ2 は、それらの距離の二乗和を用いて次の（８）式によって計算される。
【００６８】
【数８】

【００６９】
但し、
Ｔ2A：制御点群
ｐk ：３次元データ上の輪郭点
ｑk ：３次元データ上の輪郭点から対応するモデル輪郭への垂足点
ｎ ：１つのモデル輪郭に対応が付けられている３次元データの輪郭点数
ｄ^k ：計測データの輪郭点から対応するモデル輪郭線への投影方向，
ｄ^k ＝（ｑk-ｐk ）／｜ｑk-ｐk ｜
ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
計測データの輪郭ＲＫが２次元画像ＦＴ上に指定される場合に、標準モデルＤＳの輪郭ＲＫ毎の差分エネルギーｅ2 ’は次の（９）式によって計算される。
【００７０】
【数９】

【００７１】
但し、
Ｔ2A：制御点群
ｐk ：２次元画像上の輪郭点
ｑk ：２次元画像上の輪郭点から２次元画像上に投影された対応するモデル輪郭への垂足点
ｎ ：１つのモデル輪郭に対応が付けられている計測データの輪郭点数
ｄ^k ：２次元画像上の輪郭点から対応するモデル輪郭への投影方向，
ｄ^k ＝（ｑk-ｐk ）／｜ｑk-ｐk ｜
ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
計測データの輪郭ＲＫを２次元画像ＦＴ上で指定する理由は、例えば３次元データＤＴがあいまいな場合に、３次元データＤＴ上に輪郭ＲＫを指定すると輪郭ＲＫそのものが不正確となるからである。したがって、それに代えて２次元画像ＦＴを用いて輪郭ＲＫを抽出するのである。
〔標準モデル上の特徴点と対応した計測データ上の特徴点との距離〕
計測データ上に特徴点ＴＴを設定することにより、３次元データＤＴ上に指定された特徴点ＴＴ、または２次元画像ＦＴ上に指定された特徴点ＴＴと、標準モデルＤＳ上の特徴点との距離が評価される。
【００７２】
３次元データＤＴ上の特徴点ＴＴと標準モデルＤＳ上の特徴点ＴＴとの差分エネルギーｅ3 は、対応する特徴点ＴＴの二乗距離を用いて次の（１０）式によって計算される。
【００７３】
なお、２次元画像ＦＴ上に特徴点ＴＴを指定した場合は、カメラパラメータを用いて標準モデルＤＳの特徴点ＴＴを２次元画像ＦＴ上に投影し、２次元画像ＦＴ上での差分エネルギーを計算する。
【００７４】
【数１０】

【００７５】
但し、
Ｔ3A：制御点群
Ｆk ：計測データの特徴点
Ｇk ：計測データの特徴点に対応する標準モデル上の特徴点
Ｎ ：計測データの特徴点と標準モデル上の特徴点との対応数
Ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
〔過剰な変形を回避するための安定化エネルギー〕
上に述べた差分のエネルギーに加え、過剰な変形を回避するための安定化エネルギーｅs が導入される。
【００７６】
すなわち、変形に用いられる制御点の間が、図９に示す仮想バネ(elastic bar) ＫＢによってつながれているものとする。仮想バネＫＢの制約に基づいて、標準モデルＤＳの面Ｓの形状の安定化のための安定化エネルギーｅs が定義される。
【００７７】
なお、仮想バネは必ずしも制御点間に張られている必要はない。制御点と仮想バネとの関係が明確であればよい。
図９において、フィッティング対象である標準モデルＤＳの面Ｓの一部が示されている。面Ｓは、制御点群Ｕ＝｜ｕｉ，ｉ＝１…Ｎ｜で形成されている。隣接する制御点間には、仮想バネＫＢが配置されている。仮想バネＫＢは、制御点間に引っ張り力による拘束を与え、面Ｓの異常変形を防ぐ働きをする。
【００７８】
つまり、隣接する制御点ｕの間隔が大きくなった場合に、それに応じて仮想バネＫＢによる引っ張り力が大きくなる。例えば、点Ｑｋが点Ｐｋに近づく場合に、その移動にともなって制御点ｕの間隔が大きくなると、仮想バネＫＢによる引っ張り力が増大する。点Ｑｋが移動しても制御点ｕの間隔が変わらなければ、つまり制御点ｕ間の相対位置関係に変化がなければ、仮想バネＫＢによる引っ張り力は変化しない。仮想バネＫＢによる引っ張り力を面Ｓの全体について平均化したものを、安定化エネルギーｅs として定義する。したがって、面Ｓの一部が突出して変形した場合に安定化エネルギーｅs は増大する。面Ｓの全体が平均して移動すれば安定化エネルギーｅs は零である。
【００７９】
安定化エネルギーｅs は、仮想バネＫＢの変形の状態により、次の（１１）式により求められる。
【００８０】
【数１１】

【００８１】
但し、
ＴsA：制御点群
Ｕ〜m,Ｖ〜m ：仮想バネの端点（制御点）の初期値
Ｕm,Ｖm ：変形後の仮想バネの端点
Ｌ0m：初期状態の仮想バネの長さ，
Ｌ0m＝｜Ｕ〜m −Ｖ〜m ｜
Ｍ ：仮想バネの本数
ｃ ：バネ係数
Ｌ ：種々のエネルギーを同一単位で扱うための調整用スケール
したがって、バネ係数ｃを大きくすると、仮想バネＫＢは硬くなって変形し難くなる。
【００８２】
このような安定化エネルギー関数ｅs を導入することにより、面Ｓの形状変化に一定の拘束を設けることとなり、面Ｓの過度の変形を防ぐことができる。
〔総合のエネルギー関数〕
上に述べたように、各エネルギー関数ｅ1,ｅ2,ｅ3,ｅ4 について、それぞれ制御点群Ｔ1A, Ｔ2A, Ｔ3A, ＴsAが用いられる。ここでは、これらの制御点群Ｔ1A〜ＴsAは同じであるが、後述するように互いに異ならせることができる。これら制御点群ＴＡを用いて標準モデルＤＳの変形を行い、次の（１２）式に示す総合エネルギー関数ｅ（ＴＡ）を最小にする制御点群ＴＡを求める。
【００８３】
【数１２】

【００８４】
但し、
ｅ1(Ｔ1A）: ３次元データの構成点とモデル表面との差分エネルギー
e2^s (T2A）: モデル輪郭毎の計測データ上の輪郭との差分エネルギー
ｅ3(Ｔ3A）: 計測データの特徴点とモデル上の特徴点との差分エネルギー
ｅS(ＴSA）: 過剰な変形を回避するための安定化エネルギー
ｗi, c : それぞれのエネルギーのウエイトパラメータ
ＴＡ＝Ｔ1A＝Ｔ2A＝Ｔ3A＝ＴSA
〔繰り返し変形〕
実際には繰り返し変形を行う（＃１７）。つまり、制御点を動かして繰り返して変形を行う。ｎ回目の変形後の総合エネルギー関数をｅn(ＴＡ）とすると、次の（１３）式の条件が満たされたときに、総合エネルギー関数ｅn （ＴＡ）が収束したと判断する。
【００８５】
【数１３】

【００８６】
さて、ここで、変形処理の全体的な流れを図３に沿って説明する。まず、計測データと標準モデルＤＳとの間で対応する点の組みを作成する（図８のＰｋとＱｋ）（＃２１）。
【００８７】
面Ｓを変形し（＃２２）、変形後の総合エネルギー関数ｅn(ＴＡ）を計算する（＃２３）。総合エネルギー関数ｅn(ＴＡ）が収束するまで（＃２４でイエス）、処理を繰り返す。
【００８８】
総合エネルギー関数ｅn(ＴＡ）の収束を判定する方法として、上に述べたように総合エネルギー関数ｅn(ＴＡ）が所定の値よりも小さくなったときを収束とする方法、前回の計算と比較べた変化の割合が所定値以下となったときに収束とする方法など、公知の方法を用いることが可能である。
〔異なる制御点の使用〕
上に述べた（１２）式では、フィッティング対象（３次元データＤＴの構成点、輪郭ＲＫ、特徴点ＴＴ）がそれぞれ異なるエネルギー関数ｅ1 〜ｅ4 について、同じ制御点群を使用したが、ここに示す例は、フィッティング対象毎、すなわちエネルギー関数毎に異なる制御点群を用いる。つまり、制御点群Ｔ1A, Ｔ2A, Ｔ3A, ＴsAを互いに異ならせる。
【００８９】
この場合には、総合エネルギー関数ｅ（ＴＡ）として上に示した（１２）式を用いることができる。但し、そこに用いられる制御点群Ｔ1A, Ｔ2A, Ｔ3A, ＴSAは、互いに異なっており、次に示す関係にある。
【００９０】
ＴＡ⊃Ｔ1A
ＴＡ⊃Ｔ2A
ＴＡ⊃Ｔ3A
ＴＡ＝ＴSA
すなわち、上に述べたように、特徴点は点であるので、特徴点同士のエネルギーに対しては、局所的な動きになってしまう。例えば、３次元データＤＴと標準モデルＤＳとの目の位置を合わせようとするときに、特徴点ＴＴが設定された部分のみが強く引っ張られ、いびつに変形する可能性がある。そのような場合に、全体的な動きとなるようにするのが好ましい。
【００９１】
一方、３次元データＤＴの構成点に対しては、目の横などは細かく動いてほしい。しかし、少数の制御点しか用いない場合には、構成点は細かく移動しない。
そこで、３次元データＤＴの構成点については多数の制御点を用い、特徴点については少数の制御点を用いる。輪郭ＲＫについてはその中間の量とする。
【００９２】
例えば、輪郭ＲＫが急激に変化する部分については、制御点を細かくする。安定化エネルギーは全ての制御点に対してかける。このような制御点の選択は、標準モデルＤＳを準備する際に行う。
【００９３】
なお、制御点群Ｔ1A, Ｔ2A, Ｔ3A, ＴsAは互いに異なるのであるが、各制御点群に含まれる制御点は、互いに共通に用いられるものもある。
〔信頼性に応じたウエイトの変更〕
上に述べた（１２）式では、各情報についての信頼性が同等であるとして総合エネルギー関数ｅ（ＴＡ）を評価したが、ここに示す例は、それぞれの情報の信頼性に応じて重みを変更する。これによって、様々な情報の中からより信頼性の高い情報に重きを置いて判定することができる。
【００９４】
なお、それぞれの情報の信頼性は、３次元計測時、または輪郭・特徴点の自動抽出時に得られるものとする。
この場合には、次の（１４）式に示す総合エネルギー関数ｅ（ＴＡ）を最小にする制御点群ＴＡを求める。
【００９５】
【数１４】

【００９６】
但し、
ρi ：各エネルギー関数ｅi （ＴＡ）に関する情報の信頼性
Ｗ (ρi)：信頼性関数
ＴＡ＝Ｔ1A＝Ｔ2A＝Ｔ3A＝ＴSA
〔繰り返し変形〕
上に述べたステップ＃１６の変形処理と同じ処理を、解像度または制御点密度の異なる標準モデルＤＳを用いて繰り返す（＃１７）。
【００９７】
すなわち、解像度を変更して行う場合には、まず、第１ステップで、低解像度の標準モデル（簡易モデル）ＤＳＫを用いて上のステップ＃１６の変形処理を行う。次に、第２ステップで、第１ステップで得られた情報を適用して、高解像度の標準モデルＤＳＥを用いてステップ＃１６の変形処理と同じ処理を行う。
【００９８】
また、制御点密度を変更して行う場合には、まず、第１ステップで、標準モデルＤＳＦから制御点を間引いて生成した低密度の標準モデル（低密度型モデル）ＤＳＬを用いて上のステップ＃１６の変形処理を行い、これによって大まかなフィッティングを行う。次に、第２ステップで、高密度の標準モデルＤＳＦを用いてステップ＃１６の変形処理と同じ処理を行い、精密なフィッティングを行う。なお、第１ステップと第２ステップとで、総合エネルギー関数の収束条件を異ならせればよい。
【００９９】
これによって、演算時間を短縮することができ、全体形状を合わせつつ、局所的な形状も精度よく再現でき、より精度の高い３次元モデルＭＬを得ることができる。次に、各処理内容を詳しく説明する。
（１） 簡易モデルを用いる繰り返し変形
簡易モデルＤＳＫとして、例えば図１１に示す簡易モデルＤＳＫ１が用いられる。
【０１００】
図１１において、簡易モデルＤＳＫ１は、図４に示す標準モデルＤＳから構成点を間引いて減少させ、ポリゴンを粗くしたものである。構成点を間引くことによって解像度が低下し、軽量となる。例えば、標準モデルＤＳの構成点が１万個程度あった場合に、簡易モデルＤＳＫの構成点は数百個程度である。したがって、簡易モデルＤＳＫを、低解像度モデルということがある。これに対して、標準モデルＤＳを詳細モデルということがある。なお、簡易モデルＤＳＫでは制御点の個数も減少する。
【０１０１】
例えば、標準モデルＤＳの構成点が１万個程度あった場合に、簡易モデルＤＳＫの構成点は数百個程度である。
簡易モデルＤＳＫ１には、特徴点ＴＴおよび制御点ｕが設定されている。簡易モデルＤＳＫ１の特徴点ＴＴは、標準モデルＤＳの特徴点ＴＴと対応がとれており、変形情報の受渡しに用いられる。但し、簡易モデルＤＳＫ１の特徴点ＴＴの個数は、標準モデルＤＳの特徴点ＴＴの個数と必ずしも同一ではなく、通常、簡易モデルＤＳＫ１の特徴点ＴＴの個数の方が数が少ない。
【０１０２】
したがって、変形処理の第１ステップにおいて、簡易モデルＤＳＫ１を用いて変形を行う。第１ステップで変形した後の特徴点ＴＴの位置情報に基づいて、第２ステップにおいて標準モデルＤＳを予め変形し、変形された標準モデルＤＳに対して変形処理を行う。
（２） 低密度型モデルを用いる繰り返し変形
低密度型モデルＤＳＬとして、例えば図１２に示す低密度型モデルＤＳＬ１が用いられる。
【０１０３】
図１２において、低密度型モデルＤＳＬ１は、図４に示す標準モデルＤＳから制御点ｕを間引いてその密度を減少させたものである。構成点の個数は標準モデルＤＳと同じである。
【０１０４】
したがって、変形処理の第１ステップにおいて、標準モデルＤＳから制御点を間引いて生成した低密度型モデルＤＳＬを用いて変形を行う。第１ステップで変形した後の構成点の位置情報に基づいて、第２ステップにおいて標準モデルＤＳを予め変形し、変形された標準モデルＤＳに対して変形処理を行う。
【０１０５】
このように、第１ステップで、粗い制御点群ＴＡａを用いて大まかな形状を合致させる。その後の第２ステップで、全体的に制御点密度を増大し、または特に精緻な合致を必要とする部分の制御点を増大した細かな制御点群ＴＡｂを用い、精密な形状を合致させる。これによって演算時間を短縮することができる。最初から多くの制御点を用いて変形を行うと、計算時間を無駄に多く費やす可能性が高い。
【０１０６】
また、第１ステップでは大体の形状しか合わないが、見当違いの形状になる可能性は少ない。因みに、最初から標準モデルＤＳを用いて精密な変形を行うと、見当違いの形状に変形される可能性が大きくなる。
【０１０７】
上に述べた実施形態において、モデリング装置１の構成、回路、処理内容、処理順序、処理タイミング、係数の設定などは、本発明の趣旨に沿って適宜変更することができる。
【０１０８】
【発明の効果】
本発明によると、演算時間を短縮することができ、全体形状を合わせつつ、局所的な形状も精度よく再現することが可能である。
【図面の簡単な説明】
【図１】本発明に係るモデリング装置を示すブロック図である。
【図２】モデリング装置の全体の処理の流れを示すフローチャートである。
【図３】変形処理を示すフローチャートである。
【図４】標準モデルの例を示す図である。
【図５】対象物から３次元データを取得する様子を示す図である。
【図６】 概略の位置合わせの様子を示す図である。
【図７】輪郭および特徴点の抽出処理の様子を示す図である。
【図８】標準モデルの面と３次元データの点とを模式的に示す図である。
【図９】標準モデルの異常変形を防ぐための仮想バネを説明するための図である。
【図１０】対象物の３次元データおよび信頼性データを取得する方法の例を説明する図である。
【図１１】簡易モデルの例を示す図である。
【図１２】低密度型モデルの例を示す図である。
【符号の説明】
１ モデリング装置
１０ 処理装置
ＭＬ ３次元モデル（形状モデル）
ＤＳ 標準モデル
ＤＳＫ 簡易モデル
ＤＳＬ 低密度型モデル
ＲＫ 輪郭
ＴＴ 特徴点
ＴＡ 制御点群
ＰＲ モデリングプログラム[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method for generating a shape model, and is used, for example, for generating a three-dimensional model in the field of computer graphics.
[0002]
[Prior art]
In recent years, 3D CG (3D computer graphics) technology is often used for movies and games. In the three-dimensional CG, a three-dimensional model and a light are arranged and moved in a virtual three-dimensional space, so that the degree of freedom of expression is high.
[0003]
Conventionally, a non-contact type three-dimensional measuring apparatus using a light cutting method or the like has been put into practical use, and by performing measurement using this, three-dimensional data of an object can be created relatively easily. However, in order to use the three-dimensional data obtained by the measurement as it is for the three-dimensional CG, there are various problems such as complicated processing for reducing the amount of data by thinning out the obtained data.
[0004]
In order to deal with this problem, a method has been proposed in which a standard model of an object is prepared and the standard model is deformed in accordance with measured three-dimensional data (Japanese Patent Laid-Open No. 5-81377).
[0005]
In this conventional method, three-dimensional shape information of three-dimensional data obtained by measurement, that is, a point group existing in three dimensions is used as a fitting target, and the surface of the standard model is fitted to the three-dimensional point group.
[0006]
[Problems to be solved by the invention]
However, in the conventional method described above, when the surface of the standard model is deformed in accordance with the measured three-dimensional point group, the amount of data is large, so that a long time is required for the calculation.
[0007]
Also, if the number of three-dimensional point groups is reduced in order to reduce the amount of data, the accuracy of the generated three-dimensional model is reduced, and it is difficult to express, for example, a fine facial expression of a person. If the number of three-dimensional point groups is excessively large, there is a possibility that local deformation will occur when fitting a standard model, and there is a possibility that the entire shape will be damaged.
[0008]
The present invention has been made in view of the above-described problems, and provides a shape model generation method that can reduce the calculation time and can accurately reproduce a local shape while matching the overall shape. Objective.
[0009]
  The method according to the present invention is a method of generating a shape model by fitting a standard model based on measurement data, the step of preparing the standard model, and a feature point common to the standard model. Preparing a simple model with reduced resolution from the model, transforming the simple model based on the measurement data, and converting the feature points of the standard model corresponding to the feature points of the deformed simple model to the simple model Deforming the standard model by moving in the same manner as when the model is deformed, and the deformed standard modelShapeThe step of deforming based on the measurement data.
[0011]
  The standard model is a model defined from a plurality of constituent points, and the simple model is obtained by thinning out the constituent points from the standard model.
  In another embodiment of the present invention, a standard model in which constituent points constituting a model and control points for deforming the model are defined is based on measurement data.fittingA first model for preparing the standard model, a second step for preparing a low-density model having control points having a density lower than that of the standard model, and the low-density model. A third step of deforming the model based on the measurement data, and moving the constituent points of the standard model corresponding to the deformed constituent points of the low-density model in the same manner as when transforming the low-density model. Accordingly, the method includes a fourth step of deforming the standard model, and a fifth step of deforming the deformed standard model based on the measurement data.
[0012]
  Low density modelIs the control pointStandard modelThe control point is thinned out from the control point.SaidFirst3Step andSaidFirst5The conditions for performing the deformation may differ depending on the step.
[0013]
In the present invention, the fitting refers to a deformation process of a standard model or a series of processes including it. In fitting, for example, three-dimensional data or a two-dimensional image obtained by measuring an object is used.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block diagram showing a modeling apparatus 1 according to the present invention.
In the present embodiment, a standard model created in advance is deformed (fitted) based on measurement data (three-dimensional data or a two-dimensional image) about a human head, thereby obtaining a three-dimensional model of the human head. An example of generation will be described.
[0015]
As shown in FIG. 1, the modeling device 1 includes a processing device 10, a magnetic disk device 11, a medium drive device 12, a display device 13, a keyboard 14, a mouse 15, and a three-dimensional measuring device 16.
[0016]
The processing device 10 includes a CPU, a RAM, a ROM, a video RAM, an input / output port, various controllers, and the like. Various functions are realized on the processing device 10 by the CPU executing programs stored in the RAM and ROM.
[0017]
The magnetic disk device 11 includes an OS (Operating System), a modeling program PR for generating a three-dimensional model ML, other programs, a standard model (standard model data) DS, three-dimensional data (three-dimensional measurement data) DT, The reliability data DR indicating the reliability of the three-dimensional data DT, the two-dimensional image (two-dimensional measurement data) FT, the generated three-dimensional model ML, and other data are stored. These programs and data are loaded into the RAM of the processing device 10 at appropriate times.
[0018]
Note that the modeling program PR includes programs for measurement processing, rough alignment, data reduction processing, deformation processing, partial region selection processing, and other processing.
[0019]
The medium drive device 12 accesses a recording medium such as a CD-ROM (CD), a floppy disk FD, or a magneto-optical disk, and reads / writes data or programs. An appropriate drive device is used according to the type of the recording medium. The modeling program PR described above can be installed from these recording media. The standard model DS, the three-dimensional data DT, the reliability data DR, the two-dimensional image FT, and the like can also be input via the recording medium.
[0020]
Various data described above, the three-dimensional model ML generated by the modeling program PR, and other data (images) are displayed on the display surface HG of the display device 13.
[0021]
The keyboard 14 and the mouse 15 are used for inputting data or giving commands to the processing device 10.
The three-dimensional measuring device 16 is for obtaining three-dimensional data DT of an object by, for example, a light cutting method. It is also possible to obtain the three-dimensional data DT directly by the three-dimensional measuring device 16, and the calculation is performed by the processing device 10 or the like based on the data output from the three-dimensional measuring device 16 and indirectly three-dimensionally. It is also possible to obtain data DT.
[0022]
Simultaneously with the three-dimensional data DT, it is also possible to acquire a two-dimensional image FT on the same line of sight for the same object as necessary. As such a three-dimensional measuring device 16, for example, a known device disclosed in JP-A-10-206132 can be used.
[0023]
As another known method for acquiring the three-dimensional data DT of the object, there is a method using a plurality of cameras arranged with parallax with respect to the object. From a plurality of images having parallax obtained from these cameras, three-dimensional data DT can be obtained by calculation using a stereoscopic photography method.
[0024]
In this method, for example, by using three cameras, data for determining the reliability of each point of the three-dimensional data DT can be acquired simultaneously.
That is, an object is photographed by multi-viewing with three cameras, and three images are obtained. For these three images, the corresponding points are searched for. Based on the corresponding points for the two images, the three-dimensional data DT is obtained by a known calculation. The other image is used to obtain the reliability data DR.
[0025]
For example, as shown in FIG. 10, three cameras A, B, and C are used to photograph the object Q to obtain three images FA, FB, and FC. For each image FA, FB, FC, the respective image planes (un, vn) are shown (n = 1, 2, 3). A point QP on the object Q in the three-dimensional space M is projected onto points PA, PB, and PC on each image plane.
[0026]
Here, if the correspondence between the points PA, PB, and PC is obtained, the three-dimensional space M ′ can be reconstructed from these correspondences. In the three-dimensional space M ′, a point QP ′ corresponding to the points PA and PB is obtained. Ideally, the point QP 'on the reconstructed three-dimensional space M' is represented by the image plane (u_Three, V_Three) And the point QP on the original three-dimensional space M are converted into the image plane (u_Three, V_Three) It should match the point PC projected above.
[0027]
However, since it is difficult to accurately determine the projection transformation and it is difficult to accurately determine the correspondence, they usually do not match. Therefore, the deviation between the point PC ′ and the point PC is taken as an error and used as an index of reliability.
[0028]
For example, the error between the point PC ′ and the point PC is indicated by the number of shifted pixels. If the point PC ′ and the point PC are on the same pixel, the error is “0”. If it is shifted by one pixel, the error is “1”. If it is shifted by two pixels, the error is “2”. This shifted pixel number can be used as the reliability data DR.
[0029]
As another method for determining the reliability data DR, for example, a method disclosed in Japanese Patent Application Laid-Open No. 61-125686 and other known methods can be used.
The modeling apparatus 1 can be configured using a personal computer or a workstation. The programs and data described above can also be obtained by receiving via the network NW.
[0030]
Next, an overall processing flow of the modeling apparatus 1 will be described with reference to a flowchart.
2 is a flowchart showing the overall processing flow of the modeling apparatus 1, FIG. 3 is a flowchart showing the deformation processing, FIG. 4 is a diagram showing an example of the standard model DS1, and FIG. 5 is to acquire three-dimensional data DT from the object. FIGS. 6A and 6B are diagrams showing a schematic alignment, FIG. 7 is a diagram showing a contour and feature point extraction process, and FIG. 8 is a diagram showing the surface S of the standard model DS. FIG. 9 is a diagram schematically illustrating a point P of the three-dimensional data DT, FIG. 9 is a diagram for explaining a virtual spring for preventing abnormal deformation of the standard model DS, and FIG. 10 is a three-dimensional data DT and reliability of an object. It is a figure explaining the example of the method of acquiring data DR.
[Preparation of standard model]
In FIG. 2, first, a standard model DS for an object is prepared (# 11). In this embodiment, since the object is a human head, it is most similar to the head of the object from among a plurality of standard model groups with various sizes and shapes for the entire circumference of the head. A standard model DS1 is prepared.
[0031]
The standard model DS may be either a three-dimensional shape model defined by polygons or a three-dimensional shape model defined by free-form surfaces. In the case of a three-dimensional shape model defined by polygons, the surface shape is determined by the three-dimensional coordinates of the vertices of each polygon. In the case of a three-dimensional shape model defined by a free-form surface, the shape of the surface is determined by the function that defines the surface and the coordinates of each control point.
[0032]
In the case of a three-dimensional shape model defined by polygons, the vertices of each polygon are described as “component points”.
In addition, when fitting the standard model DS, a point used to deform the standard model is referred to as a “control point”. The positional relationship between the control points and the constituent points of the polygon is arbitrary, and the control points may be set on the surface of the polygon or may be set away from the surface of the polygon. One control point is associated with a plurality of component points (about 3 to 100), and the associated component point moves in accordance with the movement of the control point. When fitting the standard model DS, the entire standard model DS is deformed by moving these control points. Even when the three-dimensional shape model is defined by a free-form surface, the arrangement of control points used for fitting is arbitrary.
[0033]
Control points are arranged at a high density in portions having fine shapes such as the corners of the eyes and lips, and in portions having rapid shape changes such as the nose and lips. The other portions are uniformly arranged.
[0034]
In the standard model DS, a characteristic contour RK and a characteristic point TT viewed from a certain direction are set. As the contour RK, for example, a wrinkle line, a nose line, a lip line, or a chin line is set for the eyes, nose, mouth, or jaw. As the feature point TT, for example, there is an actual characteristic part such as the end of the eye or mouth, the top of the nose, the lower end of the chin, or the middle of them, but there is no feature. A part that is easy to specify is selected.
[0035]
In the standard model DS1 shown in FIG. 4, chin lines, lip lines, and heel lines are set as contours RK1 to RK3. As can be seen in FIG. 4, the contour RK1 is a portion that becomes an edge when the standard model DS1 is viewed from a certain direction. In the standard model DS1 shown in FIG. 4, only a part of the set feature points TT is actually shown in the figure.
[0036]
As the standard model DS, there are cases where two types are used: a standard model DSE with a large number of component points (that is, high resolution) and a simple model DSK with a small number of component points (that is, low resolution). . In addition, although the number of constituent points is the same, there are cases where two types are used: a standard model DSL with high density control points and a low density model DSL with low density control points. The simplified model DSK and the low density model DSL are generated by thinning out the configuration points or control points from the standard model DSE or DSF, respectively.
[Acquisition of 3D data]
Next, three-dimensional measurement of the object is performed to obtain three-dimensional data DT (# 12). At that time, a two-dimensional image FT of the object is also acquired at the same time. Further, as described above, the reliability data DR indicating the reliability of each point of the three-dimensional data DT or information for obtaining the reliability data DR is acquired as necessary.
[0037]
For example, as shown in FIG. 5, a three-dimensional measuring device 16 is used to measure (capture) a human head that is an object. Thereby, the three-dimensional data DT and the two-dimensional image FT are acquired.
[0038]
Note that the three-dimensional data DT and / or the two-dimensional image FT obtained by measuring the object may be referred to as “measurement data”. Either the preparation of the standard model DS or the acquisition of the three-dimensional data DT may be performed first or in parallel.
[Rough alignment]
Approximate alignment between the standard model DS and the three-dimensional data DT is performed (# 13). In this process, the orientation, size, and position of the standard model DS are changed so that the standard model DS and the three-dimensional data DT substantially match. At this time, the size of each standard model DS can be matched to the three-dimensional data DT by scaling the standard model DS individually to an arbitrary magnification in each of the X, Y, and Z directions.
[0039]
For example, as shown in FIG. 6A, by rotating the standard model DS and scaling in each direction with respect to the three-dimensional data DT as shown in FIG. And a standard model DSa of approximately the same size can be obtained. In addition, in order to make it intelligible, the thing which is not adjusting the position on drawing is shown.
[0040]
As a method of rough alignment, as described below, there are two methods: (1) overall rough alignment and (2) local rough alignment.
Among these methods, the method (1) can be automatically performed. In the method (2), it is difficult to automatically extract feature points in the method, and therefore, it is necessary to perform a part manually. In the fitting after the rough alignment, the standard model DS is basically locally deformed. Therefore, the method (1) is suitable when importance is attached to the shape, like an animation. The method (2) is preferable when the position is desired to match the shape. Further, if the user is accustomed to performing feature point extraction, it is effective to use the method (2) in order to shorten the processing time.
[Overall approximate alignment]
In the overall approximate alignment, the position, direction, and size of the standard model DS are changed so that the distance between the three-dimensional data DT and the standard model DS is minimized.
[0041]
That is, si, αi, ti that minimizes the energy function e (si, αi, ti) shown in the following equation (1) is derived.
Note that f (si, αi, ti) is an energy function defined in relation to the distance between the three-dimensional data DT and the standard model DS. g (si) is a stabilization energy function for avoiding excessive deformation.
[0042]
In addition, using the two-dimensional image FT acquired at the same time when the three-dimensional measuring device 16 acquires the three-dimensional data DT, using the pattern matching on the two-dimensional image FT, the initial values of position, direction, and size are set. May be given.
[0043]
[Expression 1]

[0044]
However,
K: Number of constituent points of 3D data
dk: distance between the constituent points of the three-dimensional data and the surface of the standard model
Wsc: Weight parameter for partial magnification stabilization
S0: Initial scale
Si: Amount of magnification in each direction (where S3 is the depth direction)
αi: Rotation in each direction of the standard model
t i: Amount of movement of the standard model in each direction
Here, the constituent points on the standard model DS move according to the following equation (2), and accordingly, the distance dk between the constituent points of the three-dimensional data DT and the surface of the standard model DS changes.
[0045]
[Expression 2]

[0046]
In addition, when the measurement (photographing) by the three-dimensional measuring device 16 is limited from one direction, the amount of magnification in the depth direction (Z direction) may not be obtained accurately. In that case, it is considered that there is no significant difference between the shape of the three-dimensional data DT and the shape of the standard model DS, and as shown in the following equation (3), it depends on the amount of magnification (S1, S2) in the X and Y directions. A method of correcting the amount of magnification (S3) in the Z direction is also conceivable.
[0047]
[Equation 3]

[0048]
However,
γ: Weight parameter for gaze direction x3 deformation
[Local approximate alignment]
If the overall rough alignment described above is performed automatically and it does not fit well, it will be manually aligned. However, the local rough alignment described here is a manual alignment. It is a technique to do as easily as possible. In addition, the part which did not work automatically is reset once, and it starts it manually from the beginning.
[0049]
In the local approximate alignment, the characteristic line or point on the three-dimensional data DT and the characteristic line or point on the standard model DS are associated with each other so that the distance between them is minimized. Change position, direction, and size. When lines are associated with each other, the feature point is the point on one line and the shortest of the perpendiculars drawn from that point onto the other line, and multiple points are obtained on the line. It shall be.
[0050]
That is, the energy function E (si, αi, ti) shown in the following equation (4) is minimized with respect to the distance between the feature point on the three-dimensional data DT and the corresponding feature point on the standard model DS. Thus, ti, αi, and si of the standard model DS are derived.
[0051]
[Expression 4]

[0052]
However,
k: number of corresponding feature points
Mk: Feature point on the standard model after alignment
x: Feature point on the standard model before alignment
Ck: Feature point on 3D data
Si: Deviation amount in each direction of the standard model
αi: Rotation in each direction of the standard model
t i: Amount of movement of the standard model in each direction
Further, when the measurement (photographing) by the three-dimensional measuring device 16 is limited from one direction, the scale in the depth direction (Z direction) may not be obtained accurately. In that case, a method of correcting the scale in the Z direction using the following equation (5) is conceivable as in the case of the overall rough alignment described above.
[0053]
[Equation 5]

[0054]
[Extraction of contour and feature points]
Outlines and feature points are extracted on the three-dimensional data DT or the two-dimensional image FT (# 14). When the contour RK and the feature point TT for the standard model DS have been extracted in advance, the contour and the feature point to be arranged at the same position as those are displayed on the three-dimensional data DT or the corresponding two-dimensional It arrange | positions on an image (refer FIG. 7).
[0055]
When the contour RK and the feature point TT are not extracted in advance for the standard model DS, they are also specified on the standard model DS together with the arrangement on the three-dimensional data DT or the two-dimensional image.
[Data reduction]
Next, in order to reduce the calculation amount and the error, data is reduced for the three-dimensional data DT, and only necessary and reliable data is extracted (# 15). By reducing the data, the calculation amount can be reduced without destroying the shape of the original three-dimensional data DT.
[0056]
In reducing the data, for example, data outside the object area is excluded and unnecessary data is excluded. For example, the face area is determined from the two-dimensional image FT, and only the three-dimensional data DT corresponding to the area is left. Alternatively, the region is determined using the difference in distance between the object and the background. In addition, there are various methods such as extracting a face region using the information on the approximate alignment. If the three-dimensional data DT includes the reliability data DR, only the highly reliable data is left. If there are many data in the neighborhood, the data is thinned out and the density is averaged.
[0057]
When the density is averaged by thinning out data, only the three-dimensional data DT that satisfies the condition expressed by the following equation (6) is employed.
[0058]
[Formula 6]

[0059]
However,
Pk: Composition point
P to r: configuration points already adopted
R_det(x): a function representing the density around the component point x
According to the above formula (6), if the distance between the data Pk of interest and the remaining data Pr that has been adopted so far is equal to or greater than a certain value, the data Pk is adopted.
[Deformation]
The standard model DS is deformed (# 16). Here, the energy function e1 defined in relation to the distance between each constituent point of the three-dimensional data DT and the surface of the standard model DS is used, and in addition, the feature points of the standard model DS and the three-dimensional data are used. Energy function e3 defined in relation to the distance between the feature points specified for DT, related to the distance between the contour of the standard model DS and the contour specified for the three-dimensional data DT. The energy function e2 defined in order to avoid excessive deformation and the energy function es defined to avoid excessive deformation are used to evaluate the combined energy function e so that the total energy function e is minimized. The surface of the model DS is deformed.
[0060]
It is most preferable to use four functions e1, e2, e3, and es as the total energy function e, but it is also possible to use only two of e1 to e3.
[0061]
Next, each energy function will be described sequentially.
[Distance between standard model and 3D data]
In FIG. 8, one of the point groups constituting the three-dimensional data DT is indicated by a point Pk. On the surface S of the standard model DS, the point closest to the point Pk is indicated by Qk. The point Qk is an intersection when a perpendicular is drawn from the point Pk to the surface S. Here, the distance between the point Pk and the point Qk is evaluated.
[0062]
That is, the difference energy e1 between each point of the three-dimensional data DT and the surface of the standard model DS is a point Pk on the three-dimensional data DT after data reduction and a point Qk obtained by projecting the point Pk on the surface S of the standard model DS. Is calculated by the following equation (7).
[0063]
[Expression 7]

[0064]
However,
T1A: Control point group
Pk: Composition point of 3D data after reduction
Qk: Projection point from component point to model surface
K: Number of component points after reduction
d^k: Projection direction from component point to model surface,
d^k= (Qk-Pk) / | Qk-Pk |
ρk: Reliability of the component point Pk
w (ρk): reliability function, w (ρk) = 1 / (α + ρk)ⁿ
W: Σw (ρk)
L: Adjustment scale for handling various energies in the same unit
[Distance between contour on standard model and contour on measurement data]
Here, the distance between the contour RK designated on the three-dimensional data DT or the contour RK designated on the two-dimensional image FT and the contour RK on the standard model DS is evaluated.
[0065]
When the contour RK of the measurement data is specified on the three-dimensional data DT, a perpendicular is dropped from a point on the contour RK of the three-dimensional data DT to the corresponding contour RK on the standard model DS, and the shortest of the perpendiculars Is the distance. A plurality of points are designated on the contour RK.
[0066]
When the contour RK of the measurement data is designated on the two-dimensional image FT, the contour RK of the standard model DS is projected on the two-dimensional image FT using the camera parameters for the camera that captured the two-dimensional image FT. A perpendicular is dropped from a point on the contour RK of the two-dimensional image FT to the corresponding contour RK of the standard model DS, and the shortest one of the perpendiculars is taken as a distance. A plurality of points are designated on the contour RK.
[0067]
When the contour RK of the measurement data is specified on the three-dimensional data DT, the difference energy e2 for each contour RK of the standard model DS is calculated by the following equation (8) using the sum of squares of the distances. .
[0068]
[Equation 8]

[0069]
However,
T2A: Control point group
pk: contour point on 3D data
qk: droop point from the contour point on the 3D data to the corresponding model contour
n: Number of contour points of 3D data associated with one model contour
d^k: Projection direction from contour point of measurement data to corresponding model contour line,
d^k= (Qk-pk) / | qk-pk |
l: Adjustment scale for handling various energies in the same unit
When the contour RK of the measurement data is designated on the two-dimensional image FT, the difference energy e2 'for each contour RK of the standard model DS is calculated by the following equation (9).
[0070]
[Equation 9]

[0071]
However,
T2A: Control point group
pk: contour point on 2D image
qk: droop point from the contour point on the 2D image to the corresponding model contour projected onto the 2D image
n: Number of contour points of measurement data associated with one model contour
d^k: Projection direction from the contour point on the 2D image to the corresponding model contour,
d^k= (Qk-pk) / | qk-pk |
l: Adjustment scale for handling various energies in the same unit
The reason why the contour RK of the measurement data is designated on the two-dimensional image FT is that, for example, when the three-dimensional data DT is ambiguous, if the contour RK is designated on the three-dimensional data DT, the contour RK itself becomes inaccurate. . Therefore, the contour RK is extracted using the two-dimensional image FT instead.
[Distance between feature points on standard model and feature points on measurement data]
By setting the feature point TT on the measurement data, the feature point TT specified on the three-dimensional data DT or the feature point TT specified on the two-dimensional image FT and the feature point on the standard model DS The distance is evaluated.
[0072]
The difference energy e3 between the feature point TT on the three-dimensional data DT and the feature point TT on the standard model DS is calculated by the following equation (10) using the square distance of the corresponding feature point TT.
[0073]
When the feature point TT is specified on the two-dimensional image FT, the feature point TT of the standard model DS is projected on the two-dimensional image FT using the camera parameters, and the difference energy on the two-dimensional image FT is calculated. To do.
[0074]
[Expression 10]

[0075]
However,
T3A: Control point group
Fk: Features of measurement data
Gk: Feature point on standard model corresponding to feature point of measurement data
N: Number of correspondences between feature points of measurement data and feature points on standard model
L: Adjustment scale for handling various energies in the same unit
[Stabilization energy to avoid excessive deformation]
In addition to the differential energy described above, a stabilization energy es is introduced to avoid excessive deformation.
[0076]
That is, it is assumed that the control points used for the deformation are connected by the virtual spring (elastic bar) KB shown in FIG. Based on the constraint of the virtual spring KB, a stabilization energy es for stabilizing the shape of the surface S of the standard model DS is defined.
[0077]
The virtual spring does not necessarily have to be stretched between the control points. It is sufficient if the relationship between the control point and the virtual spring is clear.
In FIG. 9, a part of the surface S of the standard model DS to be fitted is shown. The surface S is formed by the control point group U = | ui, i = 1... N | A virtual spring KB is disposed between adjacent control points. The virtual spring KB acts to prevent abnormal deformation of the surface S by giving a restraint between the control points by a pulling force.
[0078]
That is, when the interval between adjacent control points u increases, the pulling force by the virtual spring KB increases accordingly. For example, when the point Qk approaches the point Pk, if the distance between the control points u increases with the movement, the pulling force by the virtual spring KB increases. Even if the point Qk moves, if the distance between the control points u does not change, that is, if the relative positional relationship between the control points u does not change, the pulling force by the virtual spring KB does not change. A value obtained by averaging the pulling force by the virtual spring KB over the entire surface S is defined as a stabilization energy es. Therefore, the stabilization energy es increases when a part of the surface S protrudes and deforms. If the entire surface S moves on average, the stabilization energy es is zero.
[0079]
The stabilization energy es is obtained by the following equation (11) depending on the deformation state of the virtual spring KB.
[0080]
[Expression 11]

[0081]
However,
TsA: Control point group
U to m, V to m: initial value of the end point (control point) of the virtual spring
Um, Vm: End point of virtual spring after deformation
L0m: length of the virtual spring in the initial state,
L0m = | U ~ m -V ~ m |
M: Number of virtual springs
c: Spring coefficient
L: Adjustment scale for handling various energies in the same unit
Therefore, when the spring coefficient c is increased, the virtual spring KB becomes hard and is not easily deformed.
[0082]
By introducing such a stabilization energy function es, a constant constraint is provided for the shape change of the surface S, and excessive deformation of the surface S can be prevented.
[Total energy function]
As described above, the control point groups T1A, T2A, T3A, and TsA are used for the energy functions e1, e2, e3, and e4, respectively. Here, these control point groups T1A to TsA are the same, but can be different from each other as will be described later. Using these control point groups TA, the standard model DS is deformed to obtain a control point group TA that minimizes the total energy function e (TA) shown in the following equation (12).
[0083]
[Expression 12]

[0084]
However,
e1 (T1A): Differential energy between the constituent points of 3D data and the model surface
e2^s(T2A): Difference energy from the contour on the measurement data for each model contour
e3 (T3A): Difference energy between feature points of measurement data and feature points on the model
eS (TSA): Stabilization energy to avoid excessive deformation
wi, c: Weight parameter of each energy
TA = T1A = T2A = T3A = TSA
[Repeated deformation]
Actually, the deformation is repeatedly performed (# 17). That is, the control point is moved and repeatedly deformed. Assuming that the total energy function after the n-th deformation is en (TA), it is determined that the total energy function en (TA) has converged when the condition of the following equation (13) is satisfied.
[0085]
[Formula 13]

[0086]
Now, the overall flow of the deformation process will be described with reference to FIG. First, a pair of corresponding points is created between the measurement data and the standard model DS (Pk and Qk in FIG. 8) (# 21).
[0087]
The surface S is deformed (# 22), and the total energy function en (TA) after deformation is calculated (# 23). The process is repeated until the total energy function en (TA) converges (Yes in # 24).
[0088]
As a method of judging the convergence of the total energy function en (TA), as described above, the method of convergence when the total energy function en (TA) becomes smaller than a predetermined value, and the previous calculation were compared. It is possible to use a known method such as a method of converging when the rate of change becomes a predetermined value or less.
[Use of different control points]
In the equation (12) described above, the same control point group is used for the energy functions e1 to e4 having different fitting objects (composition points, contours RK, and feature points TT of the three-dimensional data DT). The example uses a different control point group for each fitting object, that is, for each energy function. That is, the control point groups T1A, T2A, T3A, TsA are made different from each other.
[0089]
In this case, the above equation (12) can be used as the total energy function e (TA). However, the control point groups T1A, T2A, T3A, and TSA used there are different from each other and have the following relationship.
[0090]
TA⊃T1A
TA⊃T2A
TA⊃T3A
TA = TSA
That is, as described above, since the feature points are points, local motion occurs with respect to the energy between the feature points. For example, when trying to match the positions of the eyes of the three-dimensional data DT and the standard model DS, there is a possibility that only the portion where the feature point TT is set is strongly pulled and deformed into an irregular shape. In such a case, it is preferable to make the movement as a whole.
[0091]
On the other hand, for the constituent points of the three-dimensional data DT, I want the side of the eyes to move finely. However, when only a small number of control points are used, the constituent points do not move finely.
Therefore, a large number of control points are used for the constituent points of the three-dimensional data DT, and a small number of control points are used for the feature points. The contour RK is an intermediate amount.
[0092]
For example, for a portion where the contour RK changes rapidly, the control point is made fine. Stabilization energy is applied to all control points. Such control point selection is performed when the standard model DS is prepared.
[0093]
Although the control point groups T1A, T2A, T3A, and TsA are different from each other, some of the control points included in each control point group are commonly used.
[Change of weight according to reliability]
In the above equation (12), the total energy function e (TA) is evaluated on the assumption that the reliability of each information is equivalent. However, in the example shown here, the weight is set according to the reliability of each information. change. Accordingly, it is possible to make a determination by placing emphasis on more reliable information among various types of information.
[0094]
In addition, the reliability of each information shall be acquired at the time of three-dimensional measurement or the automatic extraction of an outline and a feature point.
In this case, a control point group TA that minimizes the total energy function e (TA) shown in the following equation (14) is obtained.
[0095]
[Expression 14]

[0096]
However,
ρi: Reliability of information on each energy function ei (TA)
W (ρi): Reliability function
TA = T1A = T2A = T3A = TSA
[Repeated deformation]
The same process as the deformation process of step # 16 described above is repeated using a standard model DS having a different resolution or control point density (# 17).
[0097]
That is, when the resolution is changed, first, in the first step, the deformation process of the above step # 16 is performed using the low resolution standard model (simple model) DSK. Next, in the second step, the information obtained in the first step is applied and the same process as the deformation process in step # 16 is performed using the high-resolution standard model DSE.
[0098]
When the control point density is changed, first, in the first step, the upper step is performed using the low-density standard model (low-density model) DSL generated by thinning out the control points from the standard model DSF. The deformation process of # 16 is performed, and rough fitting is performed by this. Next, in the second step, using the high-density standard model DSF, the same processing as the deformation processing in step # 16 is performed, and precise fitting is performed. In addition, what is necessary is just to change the convergence conditions of a total energy function in a 1st step and a 2nd step.
[0099]
As a result, the calculation time can be shortened, the local shape can be accurately reproduced while matching the overall shape, and a more accurate three-dimensional model ML can be obtained. Next, each processing content will be described in detail.
(1) Repeated deformation using a simple model
As the simple model DSK, for example, a simple model DSK1 shown in FIG. 11 is used.
[0100]
In FIG. 11, the simplified model DSK1 is obtained by thinning the polygons by thinning out the configuration points from the standard model DS shown in FIG. By thinning out the constituent points, the resolution is reduced and the weight is reduced. For example, when there are about 10,000 constituent points of the standard model DS, there are about several hundred constituent points of the simplified model DSK. Therefore, the simplified model DSK is sometimes referred to as a low resolution model. On the other hand, the standard model DS may be referred to as a detailed model. In the simple model DSK, the number of control points also decreases.
[0101]
For example, when there are about 10,000 constituent points of the standard model DS, there are about several hundred constituent points of the simplified model DSK.
A feature point TT and a control point u are set in the simple model DSK1. The feature point TT of the simple model DSK1 corresponds to the feature point TT of the standard model DS, and is used for delivery of deformation information. However, the number of feature points TT of the simplified model DSK1 is not necessarily the same as the number of feature points TT of the standard model DS, and usually the number of feature points TT of the simplified model DSK1 is smaller.
[0102]
Therefore, in the first step of the deformation process, deformation is performed using the simple model DSK1. Based on the position information of the feature point TT after being deformed in the first step, the standard model DS is deformed in advance in the second step, and deformation processing is performed on the deformed standard model DS.
(2) Repetitive deformation using low-density model
As the low density model DSL, for example, a low density model DSL1 shown in FIG. 12 is used.
[0103]
In FIG. 12, the low-density model DSL1 is obtained by thinning the control points u from the standard model DS shown in FIG. 4 to reduce its density. The number of constituent points is the same as that of the standard model DS.
[0104]
Therefore, in the first step of the deformation process, deformation is performed using the low-density model DSL generated by thinning out control points from the standard model DS. Based on the position information of the component points after the deformation in the first step, the standard model DS is deformed in advance in the second step, and the deformation process is performed on the deformed standard model DS.
[0105]
Thus, in the first step, the rough shape is matched using the rough control point group TAa. In the subsequent second step, a precise shape is matched by using a fine control point group TAb in which the control point density is increased as a whole, or the control points in a portion requiring particularly precise matching are increased. Thereby, the calculation time can be shortened. If deformation is performed using many control points from the beginning, there is a high possibility that a lot of calculation time is wasted.
[0106]
In addition, although only a rough shape is matched in the first step, there is little possibility of a misregistered shape. Incidentally, if the standard model DS is used for precise deformation from the beginning, there is a high possibility that it will be deformed into an incorrect shape.
[0107]
In the embodiment described above, the configuration, circuit, processing content, processing order, processing timing, coefficient setting, and the like of the modeling apparatus 1 can be appropriately changed in accordance with the spirit of the present invention.
[0108]
【The invention's effect】
According to the present invention, the calculation time can be shortened, and the local shape can be accurately reproduced while matching the overall shape.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a modeling apparatus according to the present invention.
FIG. 2 is a flowchart showing an overall processing flow of the modeling apparatus.
FIG. 3 is a flowchart showing a deformation process.
FIG. 4 is a diagram illustrating an example of a standard model.
FIG. 5 is a diagram showing how three-dimensional data is acquired from an object.
FIG. 6 is a diagram showing a state of rough alignment.
FIG. 7 is a diagram illustrating a state of outline and feature point extraction processing;
FIG. 8 is a diagram schematically showing a surface of a standard model and points of three-dimensional data.
FIG. 9 is a diagram for explaining a virtual spring for preventing abnormal deformation of a standard model.
FIG. 10 is a diagram illustrating an example of a method for acquiring three-dimensional data and reliability data of an object.
FIG. 11 is a diagram illustrating an example of a simple model.
FIG. 12 is a diagram showing an example of a low-density model.
[Explanation of symbols]
1 Modeling device
10 Processing device
ML 3D model (shape model)
DS standard model
DSK simple model
DSL low density model
RK contour
TT feature point
TA control point cloud
PR modeling program

Claims (5)

A method for generating a shape model by fitting a standard model based on measurement data,
Preparing the standard model;
Preparing a simple model having common feature points with the standard model and having a resolution reduced from the standard model;
Transforming the simple model based on the measurement data;
Transforming the standard model by moving the feature points of the standard model corresponding to the transformed feature points of the simple model in the same manner as when transforming the simple model;
Deforming the deformed shape of the standard model based on the measurement data;
A shape model generating method characterized by comprising: The standard model is a model defined from a plurality of constituent points, and the simple model is a thinned out part of the standard model.
The shape model generation method according to claim 1. A method of generating a shape model by fitting a standard model, in which component points constituting a model and control points for deforming the model are defined, based on measurement data,
A first step of preparing the standard model;
A second step of preparing a low density model having control points of lower density than the standard model;
A third step of deforming the low-density model based on the measurement data;
A fourth step of deforming the standard model by moving the constituent points of the standard model corresponding to the deformed constituent points of the low-density model in the same manner as the deformation of the low-density model;
A fifth step of deforming the deformed standard model based on the measurement data;
A shape model generating method characterized by comprising: The control point of the low-density model is the control point of the standard model thinned out.
The method for generating a shape model according to claim 3. The conditions for performing the deformation in the third step and the fifth step are different.
The method for generating a shape model according to claim 3 or 4.

Images