JP4229398B2 - Three-dimensional modeling program, three-dimensional modeling control program, three-dimensional modeling data transmission program, recording medium, and three-dimensional modeling method - Google Patents

Description

【０００７】
すべての頂点がｋ個の辺を持つ（頂点の次数がｋであるという）ものをｋ正則メッシュ(k-regular meshes)または単に正則メッシュと呼び、頂点が任意の数の次数を持つものを非正則メッシュと呼ぶ。またほとんどすべての頂点が正則であるものを正則メッシュと区別して半正則メッシュ(semi-regular meshes)と呼ぶことがあるが、本明細書では３Ｄメッシュの境界部分以外のすべての頂点が正則となる場合、単に正則メッシュと呼ぶことにする。図１９は正則メッシュと非正則メッシュとを例示する。図１９（Ａ）に示されるように、頂点と三角形とからなる３Ｄメッシュにおいて、すべて（またはほとんどすべて）の頂点がｋ個の三角形（または辺）を持つ場合、その３Ｄメッシュは正則性を持つという。一方、図１９（Ｂ）に示されるように、すべて（またはほとんどすべて）の頂点がｋ個の三角形（または辺）を持たない場合、その３Ｄメッシュは非正則性を持つという。３Ｄメッシュは３次元空間上に配置された頂点を多角形で結ぶことにより物体の表面を近似した３Ｄモデルである。
【０００８】
次に、代表的な従来のモデリング手法について説明する。非特許文献１に記載されているように、「Virtualized Reality」という 米国カーネギーメロン大学で開発された３Ｄ動画生成システムがある。この３Ｄ動画生成システムは複数の距離画像(Range Image)を用いて３Ｄメッシュを得る手法を用いている。この距離画像とは、３Ｄ空間内にある平面（画像）を仮定し、その平面から物体までの距離を画素値として持たせた画像である。しかし、上述の３Ｄ動画生成システムでは、正則性を実現することができないという問題があった。さらに処理の実時間性も実現することができず、処理に時間をかければかけるほど画質（解像度）が良くなる特性であるプログレッシブ性(漸進性)も実現することができないという問題があった。
【０００９】
他の従来のモデリング手法として、非特許文献２に記載されているような京都大学の松山教授が開発したモデリングシステムとして実時間性を持つ３Ｄ動画生成システムがある。この３Ｄ動画生成システムは視体積交差法を用いている。視体積交差法は２次元画像から３Ｄ空間を再構成する方法の１つであり、同一時刻にカメラで撮像した複数枚の画像を利用して物体の位置および周囲形状を求める方法である。この３Ｄ動画生成システムは実時間性を持つものの、正則性およびプログレッシブ性は持たないという問題があった。
【００１０】
他のモデリング手法として、特許文献１に記載されている方法があるが、当該方法は実時間性は有するものの、正則性およびプログレッシブ性等を考慮したものではないという問題があった。
【００１１】
次に、メッシュ化および三角形化に関する手法について説明する。非特許文献３に記載されているように、良く知られている三角形化の手法としてDelauney法がある。Delauney法は３点を通る円の中に他の点が存在しない場合、その３点で三角形を構成するということをすべての頂点について行なうという方法である。しかしDelauney法は処理計算量が多いため実時間性がなく、また正則性は約束されないという問題があった。
【００１２】
他のメッシュ化および三角形化に関する手法として、非特許文献４に記載されているように、非正則メッシュを正則メッシュへ変換するＭＡＰＳという著名な手法がある。ＭＡＰＳは既にモデリング済みの非正則メッシュを対象としている。ＭＡＰＳは処理計算量が多く、実時間性がないという問題があった。
【００１３】
他のメッシュ化および三角形化に関する手法として、特許文献１に記載されている方法があるが、当該方法は鋭角特徴を再現するものであり、正則性、プログレッシブ性および実時間性等を考慮したものではないという問題があった。
【００１４】
【発明が解決しようとする課題】
何らかの方法で取得された３Ｄメッシュに対する各種の「３Ｄメッシュ処理」技術は種々存在し、スムージング、特徴抽出、簡略化／詳細化および符号化等の技術は多数提案されてきた。３Ｄメッシュは３Ｄ空間上でランダムにサンプリングされた点を様々な方法で接続することによりポリゴンを構築している。このようにランダムにサンプリングを行なっているため、例えば通常のディジタル信号における等間隔でサンプリングされたグリッドおよび画素等のような概念はない。このため等間隔の時系列データとして処理することが困難となり、３Ｄメッシュ処理を複雑なものにしている。このため従来から、頂点同士が不規則に接続されたメッシュ(非正則メッシュ)をパラメータ化し、正則メッシュへ変換することにより、３Ｄメッシュの取り扱いを容易にしようという試みが盛んに行われてきた。しかしいずれも正則性およびプログレッシブ性を持たないという問題があった。特に、非正則３Ｄメッシュをモデリングした後、正則メッシュに変換するという二つの手順を踏んでいるため、処理計算量が非常に多くなる。この結果、短時間で正則３Ｄメッシュを得たい場合、または動きのある物体（動的物体）に対する動的な正則３Ｄメッシュを実時間で生成したい場合に不都合となるという問題があった。
【００１５】
従来の多くの３Ｄモデリング手法は、静止した物体（静的物体）の形状を取得することを目的とするか、またはディスク上に保存されたビデオ信号列を用いてオフラインで３Ｄアニメーションを取得することを目的としている。動的物体の３Ｄアニメーションの取得について、カメラから送られるビデオ信号列を実時間で処理し、モデリングする手法もいくつか提案されてきた。しかし、動きのある物体を実時間でモデリングする際、当然、計算に用いるプロセッサの処理能力に応じて１フレームのモデリングにかかる時間にばらつきが生じることになる。従来のモデリング手法ではそのアルゴリズム全体に沿った処理を実行することにより一つのモデルができあがることになる。このため、上記アルゴリズムが終了する時間が１フレーム時間より長くなると、モデリングが完了せずに無駄な処理となって終了することになるという問題があった。
【００１６】
従来、３Ｄメッシュ・データを伝送する場合、頂点座標の他に三角形の接続を表す情報を送る必要があった。このため３Ｄメッシュ・データの伝送には長時間を要するという問題があった。
【００１７】
そこで、本発明の目的は、上記問題を解決するためになされたものであり、静的物体だけではなく動的物体に対しても正則性を持つ３Ｄメッシュを実時間でモデリングすることができ、プログレッシブ性を併せ持つ３次元モデリング・プログラム等を提供することにある。
【００１８】
本発明の第２の目的は、アルゴリズムを途中で打ち切ることができ、打ち切った時点で最良の解像度を有する３Ｄメッシュをモデリングすることができる３次元モデリング・プログラム等を提供することにある。
【００１９】
本発明の第３の目的は、３Ｄメッシュ・データの伝送に長時間を要することのない３次元モデリング・データ伝送プログラムを提供することにある。
【００２０】
【課題を解決するための手段】
この発明の３次元モデリング・プログラムは、物体の周囲に配置された複数の撮像装置を用いて該物体の３次元モデルを作成する３次元モデリング・プログラムであって、該複数の撮像装置により各々取得させた、該物体までの距離を画素値として有する距離画像を各撮像装置に応じて記録した距離画像記録部と、所定の解像度を記録した初期解像度記録部とを用いるものであり、コンピュータに、前記距離画像記録部に記録させた各距離画像の対応点を決定させる対応点決定ステップ、
前記物体の距離画像を前記初期解像度記録部に記録された所定の解像度および前記対応点決定ステップで決定された対応点の下で前記複数の撮像装置により各々取得させると共に、取得させた時刻を時刻記録部に記録させ、該所定の解像度を解像度記録部に記録させる距離画像取得ステップ、
前記距離画像取得ステップで取得させた各距離画像を前記距離画像記録部に記録させる距離画像記録ステップ、
前記解像度記録部に記録された解像度の下で前記距離画像記録部に記録された各距離画像に基づき、前記物体の３次元メッシュ・モデルを作成させる３次元メッシュ・モデリング・ステップ、
前記時刻記録部に記録された時刻から所定の時間が経過していない場合、前記解像度記録部に記録された解像度を所定の割合で増加させて前記３次元メッシュ・モデリング・ステップから繰返して実行させ、前記時刻記録部に記録された時刻から所定の時間が経過し且つ所定の終了時刻を越えていない場合、前記距離画像取得ステップから繰返して実行させるステップ
を実行させるための３次元モデリング・プログラムである。
【００２１】
ここで、この発明の３次元モデリング・プログラムにおいて、前記３次元メッシュ・モデリング・ステップは、
前記距離画像記録部に記録された各距離画像の各画素値を仮想３次元空間内で逆射影することにより得られる各表面点の３次元座標値を求める表面点作成ステップと、
前記表面点作成ステップで求められた各表面点の３次元座標値を表面点記録部に記録する表面点記録ステップと、
前記表面点記録部に記録された３次元座標値を有する各表面点から仮想３次元空間に形成された仮想立体上へ射影することにより得られる各３次元射影点への３次元射影ベクトルを求める３次元射影点作成ステップと、
前記３次元射影点作成ステップで求められた各３次元射影点への３次元射影ベクトルを、各３次元射影点毎に、該３次元射影点と対応する表面点との間の３次元距離値を示す属性値と共に３次元射影点記録部に記録する３次元射影点記録ステップと、
前記３次元射影点記録部に記録された各３次元射影点毎の３次元射影ベクトルを前記仮想立体を切断して展開された仮想平面上へ変換して、２次元変換点への２次元変換ベクトルとして求める２次元変換点作成ステップと、
前記２次元変換点作成ステップで求められた各２次元変換点への２次元変換ベクトルを、各２次元変換点毎に前記３次元射影点記録部に記録された前記属性値と共に２次元変換点記録部に記録する２次元変換点記録ステップと、
前記仮想平面上に所定のサイズの画素を設定し、各画素の中心点の２次元座標値を求める画素中心点作成ステップと、
前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値と前記２次元変換点記録部に記録された２次元変換点の２次元変換ベクトルとに基づいて、所定の２次元距離値の範囲内における所定の数の２次元変換点を求め、該所定の数の２次元変換点について前記２次元変換点記録部に記録された前記属性値に基づき、各画素の中心点の補間距離値を求める補間距離値作成ステップと、
前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値を、前記補間距離値作成ステップで求められた各画素の中心点の補間距離値と共に画素中心点記録部に記録する画素中心点記録ステップと、
前記画素中心記録部に記録された各画素の中心点の２次元座標値を、前記仮想立体上の３次元座標値へ逆変換して各画素の中心点の３次元逆変換点を求める画素中心３次元逆変換点作成ステップと、
前記画素中心３次元逆変換点作成ステップで求められた各画素の中心点の３次元逆変換点の３次元座標値を、前記画素中心点記録部に記録された各画素の中心点の補間距離値と共に画素中心３次元逆変換点記録部に記録する画素中心３次元逆変換点記録ステップと、
前記画素中心３次元逆変換点記録部に記録された各画素の中心点の補間距離値を仮想３次元空間内で逆射影することにより得られる各画素の中心点に対応する各表面点の３次元座標値を求める画素中心対応表面点作成ステップと、
前記画素中心対応表面点作成ステップで求められた各画素の中心点に対応する各表面点の３次元座標値を画素中心対応表面点記録部に記録する画素中心対応表面点記録ステップとを備えることができる。
【００２２】
ここで、この発明の３次元モデリング・プログラムにおいて、前記３次元射影点記録ステップは、前記解像度記録部に記録された解像度が前記初期解像度記録部に記録された所定の解像度と等しい場合、前記３次元射影点作成ステップで求められた各３次元射影点への３次元射影ベクトルを、各３次元射影点毎に、該３次元射影点に対応する表面点の３次元座標値を示す属性値と共に３次元射影点記録部に記録し、
前記３次元射影点作成ステップは、前記解像度記録部に記録された解像度が前記初期解像度記録部に記録された所定の解像度より大きい場合、
前記表面点記録部に既に記録された３次元座標値を有する従前の表面点に関しては、該従前の表面点に対応する従前の３次元射影点毎に既に前記３次元射影点記録部に記録された従前の３次元射影ベクトルを用い、解像度の増加により新たに前記表面点記録部に記録された３次元座標値を有する新表面点に関しては、新表面点の近傍における従前の表面点に対応する該従前の３次元射影ベクトルの前記属性値に基づいて新たな３次元射影ベクトルを求めることができる。
【００２３】
ここで、この発明の３次元モデリング・プログラムにおいて、前記３次元射影点作成ステップは、前記仮想立体が仮想円筒または仮想円錐の場合、表面点に対応する画素を含む距離画像を取得した撮像装置と物体との間の空間位置情報が記録された空間位置情報部をさらに備え、
仮想円筒または仮想円錐の中心線を仮想空間上で作成する仮想立体作成ステップと、
前記仮想立体作成ステップで作成された中心線に垂直で且つ前記表面点記録部に記録された表面点を通る仮想円筒または仮想円錐を切断する平面であって周囲が該仮想円筒または仮想円錐の周囲と一致する切断平面を作成する切断平面作成ステップと、
前記切断平面作成ステップで作成された切断平面の中心点の座標値を求める中心点座標値作成ステップと、
前記中心点座標値作成ステップで求められた中心点の座標値と前記表面点記録部に記録された前記表面点の座標値とを結ぶ線分が前記切断平面の周囲と交わる２点を求める２点作成ステップと、
前記空間位置情報部に記録された空間位置情報に基づき、前記表面点記録部に記録された前記表面点に対応する画素を含む距離画像に対応する仮想平面を仮想空間上で作成する仮想平面作成ステップと、
前記仮想平面作成ステップで作成された仮想平面における前記表面点に対応する画素の座標値を求める画素座標値作成ステップと、
前記画素座標値作成ステップで求められた画素の座標値から前記中心点座標値作成ステップで求められた中心点の座標値へ向かう画素ベクトルと、前記表面点記録部に記録された前記表面点の座標値から前記２点作成ステップで求められた２点への２つの表面点ベクトルを求めるベクトル作成ステップと、
前記ベクトル作成ステップで求められた画素ベクトルと前記２つの表面点ベクトルの各々との間の角度を比較し、小さい角度に対応する表面点ベクトルを３次元射影ベクトルとするステップとを備えることができる。
【００２４】
ここで、この発明の３次元モデリング・プログラムにおいて、前記３次元射影点作成ステップは、前記仮想立体が仮想球の場合、
仮想球の中心点の座標値を求める中心点座標値作成ステップと、
前記中心点座標値作成ステップで求められた中心点の座標値と前記表面点記録部に記録された前記表面点の座標値とを結ぶ線分上において、該中心点の座標値から該表面点へ向かう方向のベクトルを３次元射影ベクトルとするステップとを備えることができる。
【００２５】
この発明の３次元モデリング制御プログラムは、物体の周囲に配置された複数の撮像装置を用いて該物体の３次元モデルを作成する処理を制御する３次元モデリング制御プログラムであって、該複数の撮像装置により各々取得させた、該物体までの距離を画素値として有する距離画像を各撮像装置に応じて記録した距離画像記録部と、所定の解像度を記録した初期解像度記録部とを用いるものであり、各撮像装置を用いて該物体の３次元モデルを作成する各距離画像記録部に対応した複数のコンピュータと、該複数のコンピュータの処理結果を統合する統合コンピュータとを有し、
３次元モデリング制御プログラムを実行するコンピュータに、
前記距離画像記録部に記録させた各距離画像の対応点を決定させる対応点決定ステップを実行させ、
前記複数のコンピュータの各々に、
前記物体の距離画像を前記初期解像度記録部に記録された所定の解像度および前記対応点決定ステップで決定された対応点の下で前記撮像装置により取得させると共に、取得させた時刻を時刻記録部に記録させ、該所定の解像度を解像度記録部に記録させる距離画像取得ステップ、
前記距離画像取得ステップで取得させた距離画像を前記距離画像記録部に記録させる距離画像記録ステップ、
前記距離画像記録部に記録された距離画像の各画素値を仮想３次元空間内で逆射影することにより得られる各表面点の３次元座標値を求める表面点作成ステップ、
前記表面点作成ステップで求められた各表面点の３次元座標値を表面点記録部に記録する表面点記録ステップ、
前記表面点記録部に記録された３次元座標値を有する各表面点から仮想３次元空間に形成された仮想立体上へ射影することにより得られる各３次元射影点への３次元射影ベクトルを求める３次元射影点作成ステップ、
前記３次元射影点作成ステップで求められた各３次元射影点への３次元射影ベクトルを、各３次元射影点毎に、該３次元射影点と対応する表面点との間の３次元距離値を示す属性値と共に３次元射影点記録部に記録する３次元射影点記録ステップを実行させ、
前記統合コンピュータに、
前記３次元射影点記録部に記録された各３次元射影点毎の３次元射影ベクトルを前記仮想立体を切断して展開された仮想平面上へ変換して、２次元変換点への２次元変換ベクトルとして求める２次元変換点作成ステップ、
前記２次元変換点作成ステップで求められた各２次元変換点への２次元変換ベクトルを、各２次元変換点毎に前記３次元射影点記録部に記録された前記属性値と共に２次元変換点記録部に記録する２次元変換点記録ステップ、
前記仮想平面上に所定のサイズの画素を設定し、各画素の中心点の２次元座標値を求める画素中心点作成ステップ、
前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値と前記２次元変換点記録部に記録された２次元変換点の２次元変換ベクトルとに基づいて、所定の２次元距離値の範囲内における所定の数の２次元変換点を求め、該所定の数の２次元変換点について前記２次元変換点記録部に記録された前記属性値に基づき、各画素の中心点の補間距離値を求める補間距離値作成ステップ、前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値を、前記補間距離値作成ステップで求められた各画素の中心点の補間距離値と共に画素中心点記録部に記録する画素中心点記録ステップ、
前記画素中心記録部に記録された各画素の中心点の２次元座標値を、前記仮想立体上の３次元座標値へ逆変換して各画素の中心点の３次元逆変換点を求める画素中心３次元逆変換点作成ステップ、
前記画素中心３次元逆変換点作成ステップで求められた各画素の中心点の３次元逆変換点の３次元座標値を、前記画素中心点記録部に記録された各画素の中心点の補間距離値と共に画素中心３次元逆変換点記録部に記録する画素中心３次元逆変換点記録ステップ、
前記画素中心３次元逆変換点記録部に記録された各画素の中心点の補間距離値を仮想３次元空間内で逆射影することにより得られる各画素の中心点に対応する各表面点の３次元座標値を求める画素中心対応表面点作成ステップ、
前記画素中心対応表面点作成ステップで求められた各画素の中心点に対応する各表面点の３次元座標値を画素中心対応表面点記録部に記録する画素中心対応表面点記録ステップを実行させ、
前記時刻記録部に記録された時刻から所定の時間が経過していない場合、前記解像度記録部に記録された解像度を所定の割合で増加させて、前記複数のコンピュータの各々に前記表面点作成ステップから繰返して実行させ、前記時刻記録部に記録された時刻から所定の時間が経過し且つ所定の終了時刻を越えていない場合、前記複数のコンピュータの各々に前記距離画像取得ステップから繰返して実行させるステップ
を実行させるための３次元モデリング制御プログラムである。
【００２６】
この発明の３次元モデリング・データ伝送プログラムは、物体の周囲に配置された複数の撮像装置を用いて作成された該物体の３次元モデリング・データを伝送する３次元モデリング・データ伝送プログラムであって、該複数の撮像装置により各々取得させた、該物体までの距離を画素値として有する距離画像を各撮像装置に応じて記録した距離画像記録部と、所定の解像度を記録した初期解像度記録部とを用いるものであり、コンピュータに、
前記距離画像記録部に記録させた各距離画像の対応点を決定させる対応点決定ステップ、
前記物体の距離画像を前記初期解像度記録部に記録された所定の解像度および前記対応点決定ステップで決定された対応点の下で前記複数の撮像装置により各々取得させると共に、取得させた時刻を時刻記録部に記録させ、該所定の解像度を解像度記録部に記録させる距離画像取得ステップ、
前記距離画像取得ステップで取得させた各距離画像を前記距離画像記録部に記録させる距離画像記録ステップ、
前記距離画像記録部に記録された各距離画像の各画素値を仮想３次元空間内で逆射影することにより得られる各表面点の３次元座標値を求める表面点作成ステップ、
前記表面点作成ステップで求められた各表面点の３次元座標値を表面点記録部に記録する表面点記録ステップ、
前記表面点記録部に記録された３次元座標値を有する各表面点から仮想３次元空間に形成された仮想立体上へ射影することにより得られる各３次元射影点への３次元射影ベクトルを求める３次元射影点作成ステップ、
前記３次元射影点作成ステップで求められた各３次元射影点への３次元射影ベクトルを、各３次元射影点毎に、該３次元射影点と対応する表面点との間の３次元距離値を示す属性値と共に３次元射影点記録部に記録する３次元射影点記録ステップ、
前記３次元射影点記録部に記録された各３次元射影点毎の３次元射影ベクトルを前記仮想立体を切断して展開された仮想平面上へ変換して、２次元変換点への２次元変換ベクトルとして求める２次元変換点作成ステップ、
前記２次元変換点作成ステップで求められた各２次元変換点への２次元変換ベクトルを、各２次元変換点毎に前記３次元射影点記録部に記録された前記属性値と共に２次元変換点記録部に記録する２次元変換点記録ステップ、
前記仮想平面上に所定のサイズの画素を設定し、各画素の中心点の２次元座標値を求める画素中心点作成ステップ、
前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値と前記２次元変換点記録部に記録された２次元変換点の２次元変換ベクトルとに基づいて、所定の２次元距離値の範囲内における所定の数の２次元変換点を求め、該所定の数の２次元変換点について前記２次元変換点記録部に記録された前記属性値に基づき、各画素の中心点の補間距離値を求める補間距離値作成ステップ、
前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値を、前記補間距離値作成ステップで求められた各画素の中心点の補間距離値と共に画素中心点記録部に記録する画素中心点記録ステップ、
前記時刻記録部に記録された時刻から所定の時間が経過していない場合、前記解像度記録部に記録された解像度を所定の割合で増加させて前記表面点作成ステップから繰返して実行させ、前記時刻記録部に記録された時刻から所定の時間が経過し且つ所定の終了時刻を越えていない場合、前記距離画像取得ステップから繰返して実行させ、前記時刻記録部に記録された時刻から所定の時間が経過し且つ所定の終了時刻を越えた場合、前記画素中心点記録部に記録された各画素の中心点の２次元座標値と該各画素の中心点の補間距離値とを含む３次元モデリング・データを伝送するステップ
を実行させるための３次元モデリング・データ伝送プログラムである。
【００２７】
この発明の記録媒体は本発明のいずれかのプログラムを記録したコンピュータ読取り可能な記録媒体である。
【００２８】
この発明の３次元モデリング方法は、物体の周囲に配置された複数の撮像装置を用いて該物体の３次元モデルを作成する３次元モデリング方法であって、該複数の撮像装置により各々取得させた、該物体までの距離を画素値として有する距離画像を各撮像装置に応じて記録した距離画像記録部と、所定の解像度を記録した初期解像度記録部とを用いるものであり、コンピュータに、
前記距離画像記録部に記録させた各距離画像の対応点を決定させる対応点決定ステップと、
前記物体の距離画像を前記初期解像度記録部に記録された所定の解像度および前記対応点決定ステップで決定された対応点の下で前記複数の撮像装置により各々取得させると共に、取得させた時刻を時刻記録部に記録させ、該所定の解像度を解像度記録部に記録させる距離画像取得ステップと、
前記距離画像取得ステップで取得させた各距離画像を前記距離画像記録部に記録させる距離画像記録ステップと、
前記解像度記録部に記録された解像度の下で前記距離画像記録部に記録された各距離画像に基づき、前記物体の３次元メッシュ・モデルを作成させる３次元メッシュ・モデリング・ステップと、
前記時刻記録部に記録された時刻から所定の時間が経過していない場合、前記解像度記録部に記録された解像度を所定の割合で増加させて前記３次元メッシュ・モデリング・ステップから繰返して実行させ、前記時刻記録部に記録された時刻から所定の時間が経過し且つ所定の終了時刻を越えていない場合、前記距離画像取得ステップから繰返して実行させるステップと
を実行させることを特徴とする。
【００２９】
ここで、この発明の３次元モデリング方法において、前記３次元メッシュ・モデリング・ステップは、
前記距離画像記録部に記録された各距離画像の各画素値を仮想３次元空間内で逆射影することにより得られる各表面点の３次元座標値を求める表面点作成ステップと、
前記表面点作成ステップで求められた各表面点の３次元座標値を表面点記録部に記録する表面点記録ステップと、
前記表面点記録部に記録された３次元座標値を有する各表面点から仮想３次元空間に形成された仮想立体上へ射影することにより得られる各３次元射影点への３次元射影ベクトルを求める３次元射影点作成ステップと、
前記３次元射影点作成ステップで求められた各３次元射影点への３次元射影ベクトルを、各３次元射影点毎に、該３次元射影点と対応する表面点との間の３次元距離値を示す属性値と共に３次元射影点記録部に記録する３次元射影点記録ステップと、
前記３次元射影点記録部に記録された各３次元射影点毎の３次元射影ベクトルを前記仮想立体を切断して展開された仮想平面上へ変換して、２次元変換点への２次元変換ベクトルとして求める２次元変換点作成ステップと、
前記２次元変換点作成ステップで求められた各２次元変換点への２次元変換ベクトルを、各２次元変換点毎に前記３次元射影点記録部に記録された前記属性値と共に２次元変換点記録部に記録する２次元変換点記録ステップと、
前記仮想平面上に所定のサイズの画素を設定し、各画素の中心点の２次元座標値を求める画素中心点作成ステップと、
前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値と前記２次元変換点記録部に記録された２次元変換点の２次元変換ベクトルとに基づいて、所定の２次元距離値の範囲内における所定の数の２次元変換点を求め、該所定の数の２次元変換点について前記２次元変換点記録部に記録された前記属性値に基づき、各画素の中心点の補間距離値を求める補間距離値作成ステップと、前記画素中心点作成ステップで求められた各画素の中心点の２次元座標値を、前記補間距離値作成ステップで求められた各画素の中心点の補間距離値と共に画素中心点記録部に記録する画素中心点記録ステップと、
前記画素中心記録部に記録された各画素の中心点の２次元座標値を、前記仮想立体上の３次元座標値へ逆変換して各画素の中心点の３次元逆変換点を求める画素中心３次元逆変換点作成ステップと、
前記画素中心３次元逆変換点作成ステップで求められた各画素の中心点の３次元逆変換点の３次元座標値を、前記画素中心点記録部に記録された各画素の中心点の補間距離値と共に画素中心３次元逆変換点記録部に記録する画素中心３次元逆変換点記録ステップと、
前記画素中心３次元逆変換点記録部に記録された各画素の中心点の補間距離値を仮想３次元空間内で逆射影することにより得られる各画素の中心点に対応する各表面点の３次元座標値を求める画素中心対応表面点作成ステップと、
前記画素中心対応表面点作成ステップで求められた各画素の中心点に対応する各表面点の３次元座標値を画素中心対応表面点記録部に記録する画素中心対応表面点記録ステップとを備えることができる。
【００３０】
ここで、この発明の３次元モデリング方法において、前記３次元射影点記録ステップは、前記解像度記録部に記録された解像度が前記初期解像度記録部に記録された所定の解像度と等しい場合、前記３次元射影点作成ステップで求められた各３次元射影点への３次元射影ベクトルを、各３次元射影点毎に、該３次元射影点に対応する表面点の３次元座標値を示す属性値と共に３次元射影点記録部に記録し、
前記３次元射影点作成ステップは、前記解像度記録部に記録された解像度が前記初期解像度記録部に記録された所定の解像度より大きい場合、
前記表面点記録部に既に記録された３次元座標値を有する従前の表面点に関しては、該従前の表面点に対応する従前の３次元射影点毎に既に前記３次元射影点記録部に記録された従前の３次元射影ベクトルを用い、解像度の増加により新たに前記表面点記録部に記録された３次元座標値を有する新表面点に関しては、新表面点の近傍における従前の表面点に対応する該従前の３次元射影ベクトルの前記属性値に基づいて新たな３次元射影ベクトルを求めることができる。
【００３１】
ここで、この発明の３次元モデリング方法において、前記３次元射影点作成ステップは、前記仮想立体が仮想円筒または仮想円錐の場合、表面点に対応する画素を含む距離画像を取得した撮像装置と物体との間の空間位置情報が記録された空間位置情報部をさらに備え、
仮想円筒または仮想円錐の中心線を仮想空間上で作成する仮想立体作成ステップと、
前記仮想立体作成ステップで作成された中心線に垂直で且つ前記表面点記録部に記録された表面点を通る仮想円筒または仮想円錐を切断する平面であって周囲が該仮想円筒または仮想円錐の周囲と一致する切断平面を作成する切断平面作成ステップと、
前記切断平面作成ステップで作成された切断平面の中心点の座標値を求める中心点座標値作成ステップと、
前記中心点座標値作成ステップで求められた中心点の座標値と前記表面点記録部に記録された前記表面点の座標値とを結ぶ線分が前記切断平面の周囲と交わる２点を求める２点作成ステップと、
前記空間位置情報部に記録された空間位置情報に基づき、前記表面点記録部に記録された前記表面点に対応する画素を含む距離画像に対応する仮想平面を仮想空間上で作成する仮想平面作成ステップと、
前記仮想平面作成ステップで作成された仮想平面における前記表面点に対応する画素の座標値を求める画素座標値作成ステップと、
前記画素座標値作成ステップで求められた画素の座標値から前記中心点座標値作成ステップで求められた中心点の座標値へ向かう画素ベクトルと、前記表面点記録部に記録された前記表面点の座標値から前記２点作成ステップで求められた２点への２つの表面点ベクトルを求めるベクトル作成ステップと、
前記ベクトル作成ステップで求められた画素ベクトルと前記２つの表面点ベクトルの各々との間の角度を比較し、小さい角度に対応する表面点ベクトルを３次元射影ベクトルとするステップとを備えることができる。
【００３２】
ここで、この発明の３次元モデリング方法において、前記３次元射影点作成ステップは、前記仮想立体が仮想球の場合、
仮想球の中心点の座標値を求める中心点座標値作成ステップと、
前記中心点座標値作成ステップで求められた中心点の座標値と前記表面点記録部に記録された前記表面点の座標値とを結ぶ線分上において、該中心点の座標値から該表面点へ向かう方向のベクトルを３次元射影ベクトルとするステップとを備えることができる。
【００３３】
【発明の実施の形態】
以下、本発明の実施の形態につき図面を参照して詳細に説明する。本明細書では最も一般的なすべての多角形が三角形となる三角メッシュのみを取り扱うこととする。以下の説明に用いる各図において、同じ符号を付した箇所は同じ要素を示すため、当該符号が付された要素に関する説明は原則として最初に現れた図に関してのみ行い、再度現れた図に関しては省略する。
【００３４】
実施の形態１．
図１は、本発明の実施の形態１における３次元モデリング・プログラム等が動作する環境を示す。図１において、符号１は本発明の３次元モデリング・プログラム等が実行されるコンピュータ、１０はコンピュータ１に接続されたディスク等の記録装置、２０は３次元モデルを作成する対象となる物体、２５ａ、２５ｂ、２５ｃ、２５ｄ、２５ｅおよび２５ｆは、コンピュータ１に接続され物体２０の周囲に配置された複数のカメラ（撮像装置）である。図１ではカメラ２５ａ等は６台示されているが、これは図面の都合上のものであり、本発明の３次元モデリング・プログラム等は２台以上のカメラについて適用することが可能である。
【００３５】
図１に示されるように、記録装置１０は、複数のカメラ２５ａ等により各々取得させた距離画像を各カメラ２５ａ等に応じて記録した距離画像記録部３を有している。ここで距離画像とは、３Ｄ空間内にある平面（画像）を仮定し、その平面から物体２０までの距離を画素値として有する所定の解像度の画像である。記録装置１０はさらに、所定の解像度（Ｎ×Ｍの場合はＮおよびＭ）を記録した初期解像度記録部４、カメラ２５ａ等により距離画像を取得させた時刻を記録する時刻記録部５、後述するように更新される解像度を記録した初期解像度記録部６を有している。記録装置１０は所定のデータを記録する他の表面点記録部１１、３次元射影点記録部１２、２次元変換点記録部１３、画素中心点記録部１４、画素中心３次元逆変換点記録部１５、画素中心対応表面点記録部１６および空間位置情報部１７を有しているが、これらの記録部については後述する。
【００３６】
図２は、距離画像について説明する図である。図２（Ａ）において、符号３０は例えばカメラ２５ａにより取得された物体２０の距離画像である。図２（Ａ）に示されるように、距離画像３０上の画素Ａは画素Ａと物体２０上の点Ｂとの間の距離ｒａを画素値として有している。図２（Ｂ）において、符号３０は例えばカメラ２５ａにより取得された物体２０の距離画像、３１は例えばカメラ２５ｂにより取得された物体２０の距離画像、３２は例えばカメラ２５ｃにより取得された物体２０の距離画像、３３は例えばカメラ２５ｄにより取得された物体２０の距離画像、３４は例えばカメラ２５ｅにより取得された物体２０の距離画像、３５は例えばカメラ２５ｆにより取得された物体２０の距離画像である。図２（Ｂ）に示されるように、距離画像３５上の画素Cは画素Cと物体２０上の点Ｂとの間の距離ｒｂを画素値として有している。これらの距離画像３０等が各カメラ２５ａ等に応じて距離画像記録部３に各々記録されている。
【００３７】
本発明の３次元モデリング・プログラムはカメラ２５ａ等の数（Ｉ）に対応する枚数分作成された距離画像を用いる。複数のカメラ２５ａ等を用いて距離画像３０等を得る手法は多数存在するが、本明細書では距離画像３０等を得る手法は問わず、あるアルゴリズムを用いて距離画像３０等が得られ、距離画像記録部３に各々記録されていることを前提とする。距離画像３０等を取得する手法としては、一般的なステレオ画像法を用いることができる。光レーダ型レンジファインダ、位相差検出型レンジファインダ等を用いた光飛行時間測定法により距離を測定する方法を用いてもよい。あるいはスポット光投影法、スリット光投影法、ステップ光投影法またはパターン光投影法等を用いた三角測量に基づくレンジファインダを用いることもできる。
【００３８】
次に、本発明の３次元モデリング・プログラムの機能について説明する。図３は本発明の３次元モデリング・プログラムの機能をフローチャートで示す。図３に示されるように、まず距離画像記録部３に記録させた各距離画像３０等の対応点を決定させる（対応点決定ステップ。ステップＳ１０）。複数の距離画像３０等を用いる場合、ある距離画像３０における画素Ａが他の距離画像３５におけるどの画素に対応するのかという対応点（対応位置）の決定を予め行っておく必要がある。この対応点の決定には例えばＩＣＰ(Iterative Closest Point Method)を用いることができる。各距離画像３０等の対応点を調べることをレジストレーション(Registration)と呼ぶ。以上により各距離画像３０等の合成ができる。対応点決定ステップ（ステップＳ１０）の処理はオフラインで行う。
【００３９】
続いて、物体２０の距離画像３０等を初期解像度記録部４に記録された所定の解像度（Ｎ×Ｍ）および対応点決定ステップ（ステップＳ１０）で決定された対応点の下で複数のカメラ２５ａ等により各々取得させる。この結果、カメラ２５ａ等の数（Ｉ）分の距離画像３０等を取得することになる。併せて、距離画像３０等を取得させた時刻（ｔ）を時刻記録部５に記録させ、所定の解像度（Ｎ×Ｍ）を解像度記録部６に記録させる（距離画像取得ステップ。ステップＳ１２）。
【００４０】
次に、距離画像取得ステップ（ステップＳ１２）で取得させた各距離画像３０等を距離画像記録部３に各々記録させる（距離画像記録ステップ。ステップＳ１４）。
【００４１】
解像度記録部６に記録された解像度（最初はＮ×Ｍ）の下で、距離画像記録部３に記録された各距離画像３０等に基づき、物体２０の３次元メッシュ・モデルを作成させる（３次元メッシュ・モデリング・ステップ。ステップＳ１６）。当該ステップＳ１６については後で詳述する。
【００４２】
時刻記録部５に記録された時刻（ｔ）から所定の時間（フレームレート：δＴ）が経過したか否かを判定し（ステップＳ１８）、経過していない場合、解像度記録部６に記録された解像度（最初はＮ×Ｍ）を所定の割合、例えば２倍ずつの割合（２Ｎ×２Ｍ）で増加させて、３次元メッシュ・モデリング・ステップ（ステップＳ１６）から繰返して実行させる（ステップＳ２０）。一方、時刻記録部５に記録された時刻（ｔ）から所定の時間（δＴ）が経過して時刻がｔ＋δＴとなっており、且つ所定の終了時刻を越えていない場合、距離画像取得ステップ（ステップＳ１２）から繰返して実行させる（ステップＳ２２）。距離画像取得ステップ（ステップＳ１２）からステップＳ２０またはＳ２２までの処理は実時間で実行する。
【００４３】
次に、３次元メッシュ・モデリング・ステップ（ステップＳ１６）について詳述する。図４は本発明の３次元モデリング・プログラムにおける３次元メッシュ・モデリング・ステップの機能をフローチャートで示す。以下、図４のフローチャートについて図５ないし図１１を併せ用いて説明する。
【００４４】
すべての距離画像３０等の画素値（すなわち画素Ａの中心から物体２０までの距離）を３Ｄ平面上に逆射影することにより、物体２０の表面上に点を定義することができる。この点を以下では「表面点」と呼ぶことにする。図４に示されるように、まず距離画像記録部３に記録された各距離画像３０等の各画素値を仮想３次元空間内で逆射影することにより得られる各表面点の３次元座標値を求める（表面点作成ステップ。ステップＳ３０）。図５は表面点を説明するための図である。図５において、符号２８は物体２０の表面上に上述のように定義された表面点の１つを示す。
【００４５】
表面点作成ステップ（ステップＳ３０）で求められた各表面点２８等の３次元座標値を表面点記録部１１に記録する（表面点記録ステップ。ステップＳ３２）。
【００４６】
事前に決めておいた基準点を中心とする仮想的に３Ｄ空間内に定義された仮想立体、例えば仮想円筒(Virtual Cylinder)を定義する。図６は仮想円筒を説明するための図である。図６（Ａ）および（Ｂ）において、符号４５は事前に決めておいた基準点、４０は基準点４５を中心として定義された仮想円筒である。図６（Ａ）は仮想円筒４０を上から見た平面図を示し、図６（Ｂ）は斜視図を示す。図７（Ａ）、（Ｂ）および（Ｃ）は、各々図６に示される仮想円筒４０と位相同型の多様体４０ａ、４０ｂおよび４０ｃを示す。以下では仮想円筒の例を用いて説明していくが、図７に示されるような多様体４０ａ、４０ｂおよび４０ｃであれば３次元メッシュ・モデリング・ステップ（ステップＳ１６）の処理を同様に適用することができる。
【００４７】
次に、物体２０の表面上のすべての表面点２８等を仮想円筒４０上に射影（マッピング）する。図４に示されるように、表面点記録部１１に記録された３次元座標値を有する各表面点２８等から仮想３次元空間に形成された仮想立体、たとえば仮想円筒４０上へ射影することにより得られる各３次元射影点への３次元射影ベクトルを求める（３次元射影点作成ステップ。ステップＳ３４）。ここで射影は、例えば透視射影法または直交射影法等を用いることができる。
【００４８】
図８および図９は、表面点２８等の仮想円筒４０への射影を説明する図である。図８（Ａ）に示されるように、まず仮想円筒４０の中心線４１に垂直で且つ物体２０上の一つの表面点２８を通る切断平面（または円）４２を想定する。次に図８（Ｂ）に示されるように切断平面（または円）４２により仮想円筒４０を輪切りにする。図９（Ａ）に示されるように、輪切りにされた円４２の中心（基準点）４５と表面点２８とを結ぶ線Ｌ１を定義する。射影点（３次元射影点）は円４２の円周上に点４４ａと点４４ｂとの２点が得られる。そこで図９（Ｂ）に示されるように、表面点２８に相当する画素Ａから表面点２８へ向かうベクトル（画素ベクトル）Ｇと、表面点２８から円４２上のある射影点４４ａへ向かうベクトル（表面点ベクトル）Ｍａおよび表面点２８から円４２上の別の射影点４４ｂへ向かうベクトル（表面点ベクトル）Ｍｂとを想定する。ベクトルＧおよびベクトルＭａがなす角θａとベクトルＧおよびベクトルＭｂがなす角θｂとを比較して角度が近い方に射影する。例えば図９（Ｂ）の場合、θａ＜θｂであるため、上部、つまり射影点４４ａへ射影される。この射影に用いたベクトルＭａを射影ベクトル（３次元射影ベクトル）と呼ぶ。以上の操作をすべての表面点２８等に対して行うことにより距離画像３０等のパノラマを作成することができる。
【００４９】
仮想円筒４０上の射影点４４ａ等はその点から物体２０までの距離を値として持つ。そのような点の集合をVとする。図４に示されるように、３次元射影点作成ステップ（ステップＳ３４）で求められた各３次元射影点４４ａ等への３次元射影ベクトルＭａ等を、各３次元射影点４４ａ等毎に、当該３次元射影点４４ａ等と対応する表面点２８等との間の３次元距離値を示す属性値と共に３次元射影点記録部１２に記録する（３次元射影点記録ステップ。ステップＳ３６）。すなわち３次元射影点記録部１２に上述の集合Ｖが記録される。
【００５０】
次に、仮想円筒４０をある線で切断することにより一枚の展開された平面を得る。図４に示されるように、３次元射影点記録部１２に記録された各３次元射影点４４ａ等毎の３次元射影ベクトルＭａ等を仮想立体４０を切断して展開された仮想平面上へ変換して、２次元変換点への２次元変換ベクトルとして求める（２次元変換点作成ステップ。ステップＳ３８）。図１０は仮想円筒４０の展開を説明するための図である。図１０（Ａ）に示されるように、仮想円筒４０をある線Ｌ２で切断すると、図１０（Ｂ）に示されるような仮想平面Ｐへ展開することができる。図１０（Ｂ）に示されるように、仮想平面Ｐ上の点４４ａ（２）は３次元射影点４４ａが２次元空間へ変換された２次元変換点である。
【００５１】
図４に示されるように、２次元変換点作成ステップ（ステップＳ３８）で求められた各２次元変換点４４ａ（２）等への２次元変換ベクトルを、各２次元変換点４４ａ(２)等毎に３次元射影点記録部１２に記録された属性値と共に２次元変換点記録部１３に記録する（２次元変換点記録ステップ。ステップＳ４０）。
【００５２】
次に図４に示されるように、仮想平面Ｐ上に所定のサイズの矩形の画素を定義し、さらにそれぞれの画素の中心に中心点を新たに定義しその座標値を求める（画素中心点作成ステップ。ステップＳ４２）。中心点の集合をＶｃとする。図１１は仮想平面Ｐを示す。図１１（Ａ）に示されるように、仮想平面Ｐは矩形の画素Ｐ_１１、Ｐ_１５等が定義されている。２次元変換点４４ａ(２)は画素Ｐ_１５内に含まれており、他の２次元変換点４６（２）、４７（２）および４８（２）等が周囲の画素内に含まれている。図１１（Ｂ）に示されるように、仮想平面Ｐに上述の中心点を定義する。例えば画素Ｐ_１１の中心点はＣ_１１、画素Ｐ_１５の中心点はＣ_１５等のように定義される。集合Ｖｃの要素はＣ_１１、Ｃ_１５等となる。
【００５３】
ここで集合Ｖｃの各点が持つ物体２０までの距離値を以下の手順で計算する。まず集合Ｖｃのそれぞれの点Ｃ_１５等に仮想平面Ｐ内で定義されるユークリッド距離が最も近いＶの点を数点見つける。例えば２次元変換点４４ａ(２)、４６（２）、４７（２）および４８（２）等を見つけることができる。それらの数点の距離値（属性値）を用いて単純な線形補間によりＶｃの距離値を求める。詳しくは、図４に示されるように、画素中心点作成ステップ（ステップＳ４２）で求められた各画素Ｐ_１５等の中心点Ｃ_１５等の２次元座標値と２次元変換点記録部１３に記録された２次元変換点４４ａ(２)等の２次元変換ベクトルとに基づいて、所定の２次元距離（ユークリッド距離）値の範囲内における所定の数の２次元変換点４４ａ(２)等を求める。当該所定の数の２次元変換点４４ａ(２)等について２次元変換点記録部１３に記録された属性値（表面点２８等と３次元射影点４４ａ等との間の３次元距離値）に基づき、各画素Ｐ_１５等の中心点Ｃ_１５等の補間された補間距離値を求める（補間距離値作成ステップ。ステップＳ４４）。ここで補間距離値を求める場合、例えば線形補間またはスプライン補間等を用いることができる。
【００５４】
図４に示されるように、画素中心点作成ステップ（ステップＳ４２）で求められた各画素Ｐ_１５等の中心点Ｃ_１５等の２次元座標値を、補間距離値作成ステップ（ステップＳ４４）で求められた各画素Ｐ_１５等の中心点Ｃ_１５等の補間距離値と共に画素中心点記録部１４に記録する（画素中心点記録ステップ。ステップＳ４６）。
【００５５】
次に図１１（Ｃ）に示されるように、各中心点Ｃ_１５等を三角形でつなぐことにより実際に３Ｄモデルを三角形化する。上述のＶｃは規則性を持っているため、三角形化は図１１（Ｃ）に示す通り、単純に左上、上、左、右、下、右下の画素Ｐ_１５等の中心点Ｃ_１５等と接続させることにより達成することができる。仮想平面Ｐの右端の点と左端の点とも同様に接続する。距離画像３０等の解像度に合わせて仮想平面Ｐ上のグリッドの数も増加させる。
【００５６】
次に、仮想平面Ｐを仮想円筒４０へ戻す。新たに作られた距離値Ｖｃを物体２０上に逆射影することにより正則メッシュを持つ３Ｄモデルを作成することができる。詳しくは、図４に示されるように、画素中心記録部１４に記録された各画素Ｐ_１５等の中心点Ｃ_１５等の２次元座標値を、仮想円筒４０上の３次元座標値へ逆変換して各画素Ｐ_１５等の中心点Ｃ_１５等の３次元逆変換点を求める（画素中心３次元逆変換点作成ステップ。ステップＳ４８）。画素中心３次元逆変換点作成ステップ（ステップＳ４８）で求められた各画素Ｐ_１５等の中心点Ｃ_１５等の３次元逆変換点の３次元座標値を、画素中心点記録部１４に記録された各画素Ｐ_１５等の中心点Ｃ_１５等の補間距離値と共に画素中心３次元逆変換点記録部１５に記録する（画素中心３次元逆変換点記録ステップ。ステップＳ５０）。画素中心３次元逆変換点記録部１５に記録された各画素Ｐ_１５等の中心点Ｃ_１５等の補間距離値を仮想３次元空間内で逆射影することにより得られる各画素Ｐ_１５等の中心点Ｃ_１５等に対応する各表面点２８等の３次元座標値を求める（画素中心対応表面点作成ステップ。ステップＳ５２）。画素中心対応表面点作成ステップ（ステップ５２）で求められた各画素Ｐ_１５等の中心点Ｃ_１５等に対応する各表面点２８の３次元座標値を画素中心対応表面点記録部１６に記録する（画素中心対応表面点記録ステップ。ステップＳ５４）。
【００５７】
本発明の３次元モデリング・プログラムは、上述したように「実時間性」と「正則性」とに関わるところに特徴があるだけではなく、その他にプログレッシブなモデリングが可能となるところにも大きな特徴を持っている。本発明の３次元モデリング・プログラムは計算時間にスケーラビリティを持っている。ここでスケーラビリティとは、アルゴリズムの一部またはすべてを選択し実行することで、モデリングにおけるある変数（ここでは計算の複雑さ）を制御できる特性のことを言う。したがって本発明の３次元モデリング・プログラムは１フレームに対するアルゴリズム全体を実行せずに途中でうち切ったとしても、その時点での最適な解像度を持つモデルを取得することができるという特徴を持つ。プロセッサの処理能力が高い場合は高解像度の、低い場合は低解像度の３Ｄメッシュがモデリング可能である。これは実時間モデリングに大きな利点をもたらす。アルゴリズムを繰り返し実行すればするほど解像度が上がるという意味で、本発明の３次元モデリング・プログラムをプログレッシブモデリングと呼ぶことができる。
【００５８】
以上説明したように、本発明の実施の形態１によれば、まず予めオフライン処理で距離画像記録部３に記録させた各距離画像３０等の対応点を決定させておく。次に、オンライン処理で物体２０の距離画像３０等を初期解像度記録部６に記録された所定の解像度（Ｎ×Ｍ）および決定された対応点の下で複数のカメラ２５ａ等により各々取得させると共に、取得させた時刻（ｔ）を時刻記録部５に記録させ、所定の解像度（Ｎ×Ｍ）を解像度記録部６に記録させておく。取得させた各距離画像３０等は距離画像記録部に記録させる。解像度記録部６に記録された解像度の下で距離画像記録部３に記録された各距離画像３０等に基づき、物体２０の３次元メッシュ・モデルを作成させる。
【００５９】
３次元メッシュ・モデルの作成は、まず複数の距離画像３０等の画素値を３Ｄ空間上に逆射影して物体２０上に表面点を定義する。３Ｄ空間に仮想円筒４０を定義し、物体２０上のすべての表面点を仮想円筒４０上に射影して３次元射影点を求める。３次元射影点には表面点２８と射影点との間の距離値を持たせる。仮想円筒を２次元空間の仮想平面Ｐへ展開し、３次元射影点を２次元空間へ変換した２次元変換点を求める。この２次元変換点は３次元射影点の距離値を有している。仮想平面Ｐ上に所定のサイズの矩形の画素を定義し、さらにそれぞれの画素の中心に中心点を新たに定義する。中心点の距離値は周囲の２次元変換点の距離値を補間することにより求める。各中心点を三角形でつなぐことにより３Ｄモデルを三角形化する。仮想平面Ｐを仮想円筒４０へ戻し、中心点の距離値を物体２０上へ逆射影することにより、正則メッシュを持つ３Ｄメッシュ・モデルを作成することができる。
【００６０】
続いて、時刻記録部５に記録された時刻（ｔ）から所定の時間（δＴ）が経過していない場合、解像度記録部６に記録された解像度を所定の割合、例えば２倍で増加させて３次元メッシュ・モデリング・ステップから繰返して実行させる。時刻記録部に記録された時刻（ｔ）から所定の時間が経過して時刻がｔ＋δＴとなり、且つ所定の終了時刻を越えていない場合、距離画像取得ステップから繰返して実行させる。以上により、静的物体だけではなく動的物体に対しても正則性を持つ３Ｄメッシュを実時間でモデリングすることができ、プログレッシブ性を併せ持つ３次元モデリング・プログラム等を提供することができる。このプログレッシブ性を併せ持つことにより、アルゴリズムを途中で打ち切ることができ、打ち切った時点で最良の解像度を有する３Ｄメッシュをモデリングすることができる３次元モデリング・プログラム等を提供することができる。
【００６１】
実施の形態２．
実施の形態１では３次元射影点作成ステップ(ステップＳ３４)で、表面点２８等から仮想円筒４０へのマッピングの方法を示した。本実施の形態２では、別のマッピングの方法を説明する。
【００６２】
各時間tにおける１回目の繰り返し、つまり距離画像３０等の解像度がＮ×Ｍの時は３次元射影点作成ステップ（ステップＳ３４）と同様の処理を行う。ただし３次元射影点４４ａ等は物体２０までの距離値ではなく、物体２０上の表面点２８等の３次元座標(x, y, z)を値として持つ。詳しくは、３次元射影点記録ステップ（ステップＳ３６）において、解像度記録部６に記録された解像度が初期解像度記録部４に記録された所定の解像度と等しい場合、３次元射影点作成ステップ（ステップＳ３４）で求められた各３次元射影点４４ａ等への３次元射影ベクトルＭａ等を、各３次元射影点４４ａ等毎に、表面点２８等の３次元座標値を示す属性値と共に３次元射影点記録部１２に記録しておく。
【００６３】
２回目の繰り返し、つまり解像度が２Ｎ×２Ｍ以上の時は以下の手順で射影を行う。Ｎ×Ｍから２Ｎ×２Ｍに解像度を上げることによりできる新たな画素の３次元射影ベクトルをまず求める。図１２は、本発明の実施の形態２におけるマッピングの方法を説明するための図である。図１２（Ａ）は解像度がＮ×Ｍの場合の仮想平面Ｐ上の画素Ｐ_１１、Ｐ_１２等（黒丸）を示し、図１２（Ｂ）は解像度が２Ｎ×２Ｍの場合の仮想平面Ｐ上の画素Ｐ_１１、Ｐ_１２、Ｐ_１１−２、Ｐ_１１−３、Ｐ_１１−４等を示す。図１２（Ｂ）で白丸で示される画素Ｐ_１１−２、Ｐ_１１−３、Ｐ_１１−４等が新たな画素である。Ｎ×Ｍの画素Ｐ_１１、Ｐ_１２等（黒丸）が持つ３次元射影ベクトルＭａ等は既に求まっており、その３次元射影ベクトルを内挿または外挿することにより新たに加えられた画素Ｐ_１１−２、Ｐ_１１−３、Ｐ_１１−４等の３次元射影ベクトルを求める。詳しくは、３次元射影点作成ステップ（ステップＳ３４）において、解像度記録部６に記録された解像度が初期解像度記録部４に記録された所定の解像度（Ｎ×Ｍ）より大きい場合、表面点記録部１１に既に記録された３次元座標値を有する従前の表面点２８等に関しては、従前の表面点２８等に対応する従前の３次元射影点４４ａ等毎に既に３次元射影点記録部１２に記録された従前の３次元射影ベクトルＭａ等を用いる。一方、解像度の増加により新たに表面点記録部１１に記録された３次元座標値を有する新表面点に関しては、新表面点の近傍における従前の表面点２８等に対応する該従前の３次元射影ベクトルＭａ等の属性値（３次元距離値）に基づいて新たな３次元射影ベクトルを求めることができる。後は、新たな３次元射影ベクトルと仮想円筒４０との交点をもとめマッピングする。３次元射影点は相当する物体２０上の点の３次元座標(x, y, z)を値として持つ。
【００６４】
以上説明したように、本発明の実施の形態２によれば、各時間tにおける１回目の繰り返しの時は、３次元射影点４４ａ等は物体２０までの距離値ではなく、物体２０上の表面点２８等の３次元座標(x, y, z)を値として持つ点を除き、３次元射影点作成ステップ（ステップＳ３４）と同様の処理を行う。２回目以降の繰り返しの時は、新たな画素の３次元射影ベクトルは既に前回の繰返しの時に求めた３次元射影ベクトルを内挿または外挿することにより求めることができる。この結果、繰返しにより解像度が増加した場合に新たな３次元ベクトルを効率よく求めることができる。
【００６５】
実施の形態３．
実施の形態１および２では射影をする対象として仮想円筒を用いた。本実施の形態３では仮想円錐の場合を説明する。本発明の３次元モデリング・プログラムのアルゴリズムでは仮想円筒４０と同様に仮想円錐（または円錐と位相同型の多様体）の場合にも、そのまま適用できる。以下では、図４に示されるフローチャート中の３次元射影点作成ステップ（ステップＳ３４）について仮想円筒４０を仮想円錐へ置き換えて説明する。他のステップは同様であるため説明は省略する。
【００６６】
図１３は、本発明の実施の形態３における仮想円錐を用いたマッピングを説明する図である。図１３（Ａ）に示されるように、仮想円錐５０をある線Ｌ２で切断して展開すると、図１３（Ｂ）に示される仮想平面Ｐ２を得ることができる。まず、表面点２８等に対応する画素を含む距離画像３０等を取得したカメラ２５ａ等と物体２０との間の空間位置情報が記録された空間位置情報部１７をさらに設けておく。図１４は、本発明の実施の形態３における３次元射影点作成ステップのフローチャートを示す。図１４に示されるように、仮想円錐５０の中心線（仮想円筒４０の場合の中心線４１に相当）を仮想空間上で作成する（仮想立体作成ステップ。ステップＳ６０）。作成された中心線に垂直で且つ表面点記録部１１に記録された表面点２８等を通る仮想円錐５０を切断する平面であって、周囲が仮想円錐５０の周囲と一致する切断平面（仮想円筒４０の場合の切断平面４２に相当）を作成する（切断平面作成ステップ。ステップＳ６２）。作成された切断平面の中心点の座標値を求め（中心点座標値作成ステップ。ステップＳ６４）、求められた中心点の座標値と表面点記録部１１に記録された表面点２８等の座標値とを結ぶ線分が切断平面の周囲と交わる２点（仮想円筒４０の場合の４４ａおよび４４ｂに相当）を求める（２点作成ステップ。ステップＳ６６）。空間位置情報部１７に記録された空間位置情報に基づき、表面点記録部１１に記録された表面点２８等に対応する画素を含む距離画像３０等に対応する仮想平面を仮想空間上で作成する（仮想平面作成ステップ。ステップＳ６８）。作成された仮想平面における表面点２８等に対応する画素の座標値を求める（画素座標値作成ステップ。ステップＳ７０）。求められた画素の座標値から上記中心点の座標値へ向かう画素ベクトル（仮想円筒４０の場合のベクトルＧに相当）と、表面点記録部１１に記録された表面点２８等の座標値から上記２点への２つの表面点ベクトル（仮想円筒４０の場合のベクトルＭａおよびＭｂに相当）を求める（ベクトル作成ステップ。ステップＳ７２）。画素ベクトルと上記２つの表面点ベクトルの各々との間の角度（仮想円筒４０の場合の角度θａおよびθｂに相当）を比較し（ステップＳ７４）、小さい角度に対応する表面点ベクトルを３次元射影ベクトルとする（ステップＳ７６、Ｓ７８）。
【００６７】
以上説明したように、本発明の実施の形態３によれば、本発明の３次元モデリング・プログラムのアルゴリズムは仮想円筒４０と同様に仮想円錐（または円錐と位相同型の多様体）の場合にも、そのまま適用することができる。
【００６８】
実施の形態４．
上述の実施の形態では射影をする対象として仮想円筒または仮想円錐を用いた場合を説明した。本発明の３次元モデリング・プログラムのアルゴリズムは仮想円筒４０または仮想円錐と同様に仮想球（または球と位相同型の多様体）の場合にも、一部を変更することにより同様に適用できる。以下では、図４に示されるフローチャート中の３次元射影点作成ステップ（ステップＳ３４）と２次元変換点作成ステップ（ステップＳ３８）とについて仮想円筒４０を仮想球へ置き換えて説明する。他のステップは同様であるため説明は省略する。
【００６９】
図１５は、本発明の実施の形態４における仮想球を用いたマッピングを説明する図である。図１５（Ａ）に示されるように、３次元射影点作成ステップ（ステップＳ３４）では、まず仮想球６０の中心点６５の座標値を求め（中心点座標値作成ステップ）、次に中心点６５の座標値と表面点記録部１１に記録された表面点２８等の座標値とを結ぶ線分上において、中心点６５の座標値から表面点２８等へ向かう方向のベクトルを３次元射影ベクトルとする。図１５（Ｂ）に示されるように、２次元変換点作成ステップ（ステップＳ３８）では、地球儀を世界地図へマッピングするのと同様の方法で行うことにより仮想平面Ｐ３を得ることができる。
【００７０】
以上説明したように、本発明の実施の形態４によれば、本発明の３次元モデリング・プログラムのアルゴリズムは仮想円筒４０と同様に仮想球（または球と位相同型の多様体）の場合にも、仮想球６０の中心点６５から表面点２８等へ向かう方向のベクトルを３次元射影ベクトルとすることにより、適用することができる。
【００７１】
実施の形態５．
上述した本発明の各実施の形態における３次元モデリング・プログラムのアルゴリズムは、１台のコンピュータではなく、複数台のコンピュータ（パーソナル・コンピュータ：ＰＣ）をツリー状に配置したシステムにより実行することができる。
【００７２】
図１６は、本発明の実施の形態５における物体の３次元モデルを作成する処理を制御する３次元モデリング制御プログラムが実行される環境（システム）を示す。図１６において、符号７０ａないし７０ｍはｍ台のカメラ、８０ａないし８０ｍはカメラ７０ａないし７０ｍに対応する距離画像記録部、５ａないし５ｎは各カメラ７０ａ等を用いて物体２０の３次元モデルを作成する各距離画像記録部８０ａないし８０ｍに対応した複数のコンピュータ、５ｐは複数のコンピュータ５ａないし５ｎの処理結果を統合する統合コンピュータ、５ｑは３次元モデリング制御プログラム自体が実行されるコンピュータである。カメラ７０ａないし７０ｍ、距離画像記録部８０ａないし８０ｍ、コンピュータ５ａないし５ｎ、統合コンピュータ５ｐおよびコンピュータ５ｑはバス７５により接続されており、レイヤ３を構成する。レイヤ２では距離画像記録部８０ａないし８０ｍを２つずつまとめた結果を記録する統合記録部９０ａないし９０_ｍ／２により構成され、レイヤ１では最終結果を記録する最終結果記録部１００を有している。
【００７３】
以下では、図３および図４に示されるフローチャート中の各ステップをどのコンピュータに実行させるかについて説明する。まず、コンピュータ５ｑに距離画像記録部８０ａないし８０ｍに記録させた各距離画像の対応点を決定させる（対応点決定ステップ。ステップＳ１０）を実行させる。レイヤ３では、距離画像取得ステップ（ステップＳ１２）から３次元射影点記録ステップ（ステップＳ３６）までを複数のコンピュータ５ａないし５ｎの各々に担当させる。各ＰＣは一枚の距離画像のみを取り扱う。レイヤ２では、統合コンピュータ５ｐに２次元変換点作成ステップ（ステップＳ３８）から画素中心対応表面点記録ステップ（ステップＳ５４）までを担当させる。つまり、１台の統合コンピュータ５ｐが仮想平面の一部を取り扱う。次にコンピュータ５ｑに、ステップＳ１８、Ｓ２０およびＳ２２の繰返し判定の処理を行わせる。
【００７４】
以上説明したように、本発明の実施の形態５によれば、本発明の３次元モデリング・プログラムのアルゴリズムは、複数台のＰＣをツリー状に配置し、各処理を複数のＰＣに担当させることにより、短時間で効率よく実行することができる。
【００７５】
実施の形態６．
本実施の形態６では、３次元モデリング・データを伝送する３次元モデリング・データ伝送プログラムについて説明する。
【００７６】
図１６に示されるように、最終結果記録部１００には上述の実施の形態で説明された図１１（Ｂ）に示されるような矩形画素の画素値が記録されている（画素中心点記録ステップ。ステップＳ５４）。そこで３次元モデリング・データをネットワーク１０２を介してクライアントの記録部１０４、１０６、１０８および１１０等へ伝送する際、３Ｄメッシュ・データを直接伝送するのではなく、最終結果記録部１００に記録された矩形画素の画素値を伝送する。クライアント側では受け取ったデータに対して上述の三角形化を施し、それを３Ｄ空間内に逆射影することで復号、レンダリングが可能になる。以上により、３Ｄメッシュ・データを伝送する際に、頂点座標の他に三角形の接続を表す情報を送る必要がなくなる。
【００７７】
以上説明したように、本発明の実施の形態６によれば、３次元モデリング・データをネットワーク１０２を介してクライアント側へ伝送する場合、最終結果記録部１００に記録された矩形画素の画素値を伝送することにより、頂点座標の他に三角形の接続を表す情報を送る必要をなくすことができる。この結果、３Ｄメッシュ・データの伝送に長時間を要することのない３次元モデリング・データ伝送プログラムを提供することができる。
【００７８】
実施の形態７．
図１７は、上述した各実施の形態を実現するための本発明のコンピュータ・プログラムを実行するコンピュータ１等の内部回路１２０を示すブロック図である。図１７において、上述の本発明のコンピュータ・プログラムは、ＲＯＭ１２２、ディスク１２５ａまたはＣＤ−ＲＯＭ１２５ｎ等の記録媒体に記録されている。このコンピュータ・プログラムは、ＲＯＭ１２２、ディスク１２５ａまたはＣＤ−ＲＯＭ１２５ｎ等の記録媒体からコンピュータローラ１２４を介しバス１２６を通ってＲＡＭ１２３へロードされる。ディスク１２５ａ等の記録媒体は記録装置１０等を対応させておくことができる。入力操作部１２８はコンピュータ１等に入力を行うためのマウスまたはテンキー等の入力装置であり、入力制御部１２７は入力操作部１２８と接続され入力制御等を行う。外部インタフェース（Ｉ／Ｆ）部１２９は、コンピュータ５ｑ等がネットワーク１０２を介して通信を行う際のインタフェース機能を有する。
【００７９】
ＣＰＵ１２１がＲＡＭ１２３内の上述の本発明のコンピュータ・プログラムを実行することにより、本発明の目的を達成することができる。当該コンピュータ・プログラムは上述のようにＣＤ−ＲＯＭ１２５ｎ等の脱着可能な記録媒体の形態でコンピュータＣＰＵ１２１に供給することができ、当該コンピュータ・プログラムを記録したＣＤ−ＲＯＭ１２５ｎ等の記録媒体も同様に本発明を構成することになる。当該コンピュータ・プログラムを記録した記録媒体としては上述された記録媒体の他に、例えばＤＶＤ、光ディスク、メモリ・カード、メモリスティック、ＭＯ、ＦＤ等を用いることができる。
【００８０】
【発明の効果】
以上説明したように、本発明の３次元モデリング・プログラムによれば、距離画像３０等の解像度を段階的に増大させながら入力し、その入力距離画像３０等をもとに３Ｄメッシュを構成している。これによりアルゴリズムを長時間実行すればするほど３Ｄメッシュの解像度が上がる「プログレッシブ性」を実現することができる。本発明の３次元モデリング・プログラム等では、いったんモデリングした非正則３Ｄメッシュを正則に変換するという二重の手順を踏むことなく、直接正則な３Ｄメッシュを生成することができる。以上により、静的物体だけではなく動的物体に対しても正則性を持つ３Ｄメッシュを実時間でモデリングすることができ、プログレッシブ性を併せ持つ３次元モデリング・プログラム等を提供することができる。
【００８１】
本発明の３次元モデリング・プログラム等ではアルゴリズムを途中で打ち切ることが可能である。このため、打ち切った時点での最良の解像度を持つ３Ｄメッシュを出力することができる３次元モデリング・プログラム等を提供することができる。
【００８２】
本発明の３次元モデリング伝送プログラムでは実時間でモデリングし伝送することにより、遠隔地にいるクライアントがその３Ｄメッシュをあらゆる角度から、しかも実時間で見ることができるようにすることができる。クライアント側へ送るデータは３Ｄメッシュそのものではなく、矩形画素の画素値という画像に似たデータであるため、静止画像または動画像の圧縮アルゴリズムを直接適用することができる。これは高圧縮につながるだけでなく、ＳＮＲスケーラビリティ、解像度スケーラビリティ等の各種スケーラビリティを容易に実現することができるという利点を有している。以上により、３Ｄメッシュ・データの伝送に長時間を要することのない３次元モデリング・データ伝送プログラムを提供することができる。
【図面の簡単な説明】
【図１】 本発明の実施の形態１における３次元モデリング・プログラム等が動作する環境を示す図である。
【図２】 距離画像について説明する図である。
【図３】 本発明の３次元モデリング・プログラムの機能を示すフローチャートである。
【図４】 本発明の３次元モデリング・プログラムにおける３次元メッシュ・モデリング・ステップの機能を示すフローチャートである。
【図５】 表面点を説明するための図である。
【図６】 仮想円筒を説明するための図である。
【図７】 仮想円筒４０と位相同型の多様体４０ａ、４０ｂおよび４０ｃを示す図である。
【図８】 表面点２８等の仮想円筒４０への射影を説明する図である。
【図９】 表面点２８等の仮想円筒４０への射影を説明する図である。
【図１０】 仮想円筒４０の展開を説明するための図である。
【図１１】 仮想平面Ｐを示す図である。
【図１２】 本発明の実施の形態２におけるマッピングの方法を説明するための図である。
【図１３】 本発明の実施の形態３における仮想円錐を用いたマッピングを説明する図である。
【図１４】 本発明の実施の形態３における３次元射影点作成ステップを示すフローチャートである。
【図１５】 本発明の実施の形態４における仮想球を用いたマッピングを説明する図である。
【図１６】 本発明の実施の形態５における物体の３次元モデルを作成する処理を制御する３次元モデリング制御プログラムが実行される環境（システム）を示す図である。
【図１７】 本発明のコンピュータ・プログラムを実行するコンピュータ１等の内部回路１２０を示すブロック図である。
【図１８】 頂点と三角形とで物体（オブジェクト）の表面を近似した３Ｄメッシュを例示する図である。
【図１９】 正則メッシュと非正則メッシュとを例示する図である。
【符号の説明】
１ コンピュータ、 ３,８０ａ，８０ｂ，８０ｃ，８０ｄ，８０ｍ−１，８０ｍ 距離画像記録部、 ４ 初期解像度記録部、 ５ 時刻記録部、 ６ 初期解像度記録部、 １０ 記録装置、 １１ 表面点記録部、 １２ ３次元射影点記録部、１３ ２次元変換点記録部、 １４ 画素中心点記録部、 １５画素中心３次元逆変換点記録部、 １６ 画素中心対応表面点記録部、 ２０物体、 ２５ａ，２５ｂ，２５ｃ，２５ｄ，２５ｅ，２５ｆ，７０ａ、７０ｂ，７０ｃ，７０ｄ，７０ｅ、７０ｆ カメラ（撮像装置）、 ２８ 表面点、 ３０，３１，３２，３３，３４，３５ 距離画像、 ４０ 仮想円筒、 ４０ａ，４０ｂ，４０ｃ 仮想円筒と位相同型の多様体、 ４１ 中心線、４２ 切断平面、 ４４ａ、４４ｂ ３次元射影点、 ４４ａ（２）、４６（２），４７（２），４８（２） ２次元変換点、 ４５ 基準点、 ５０ 仮想円錐、 ６０仮想球、 ６５ 中心点、 ７５，１２６ バス、 ９０ａ，９０ｂ，９０ｍ／２ 統合記録部、 １００ 最終結果記録部、 １０２ ネットワーク、 １０４，１０６，１０８，１１０ 記録部、 １２０ 内部回路、 １２１ ＣＰＵ、 １２２ ＲＯＭ、 １２３ ＲＡＭ、 １２４ コンピュータローラ、 １２５ａ ディスク、 １２５ｎ ＣＤ−ＲＯＭ、 １２７ 入力制御部、 １２８ 入力操作部、 １２９ 外部Ｉ／Ｆ部。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional modeling program and the like, and in particular, a three-dimensional modeling program for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, and a three-dimensional model of the object. The present invention relates to a three-dimensional modeling control program for controlling processing, a three-dimensional modeling / data transmission program for transmitting three-dimensional modeling / data of an object, and the like.
[0002]
[Prior art]
[Non-Patent Document 1]
Proceedings of IEEE Visualization '97, Phoenix, Arizona, October 1997, pp. 277-283, 552
[Non-Patent Document 2]
Nobuhara Shohei, Wada Toshikazu, Matsuyama Takashi, “Highly accurate 3D shape restoration from multi-viewpoint images using elastic mesh models”, Information Processing Society of Japan, CVIM Technical Journal, Vol. 43, SIG11 (CVIM5), pp.53 -63, 2002.12
[Non-Patent Document 3]
Takeshi Aoi, Masayuki Nakajima, "Computer Graphics", pp.107-109, Shosodo
[Non-Patent Document 4]
Lee, AWF, Sweldens, W., Schroder, P., Cowsar, L., and Dobkin, D., “MAPS: Multiresolution Adaptive Parameterization of Surfaces.”, Proceedings of SIGGRAPH 98, pp.95-104
[Patent Document 1]
JP 2002-324246 A
[Patent Document 2]
JP 2001-67495 A
[0003]
Conventionally, techniques for editing, acquiring, and displaying a three-dimensional (hereinafter abbreviated as “3D”) model of an object have been mainly developed in the field of entertainment or medical care. With recent advances in semiconductor technology and higher transmission speeds in networks, the demand for 3D models as digital content following text, audio, and images has further increased. Along with this increase in demand, research on various related technologies centering on modeling and rendering has been actively promoted. In general, there are various representation methods for 3D models. For example, a volume model that is a 3D discrete model reconstructed as a collection of voxels, a solid model that represents a 3D shape as a solid in a geometric model, a spline patch that approximates a free-form surface with a polyhedron of curved surface patches using spline curves, and the like. It is done.
[0004]
FIG. 18 illustrates a 3D mesh (mesh) model (hereinafter abbreviated as “3D mesh”) that approximates the surface of an object with vertices and triangles. As shown in FIG. 18, the 3D mesh approximates the surface of an object with a polygonal polygonal surface such as a triangle, and is widely used because a high-definition model and animation can be expressed relatively easily.
[0005]
Here, regularity and irregularity of the 3D mesh will be described. In general, the geometric information is composed of the three-dimensional coordinates Vi of the vertices expressed by Equation 1 and the vertex index set (p, q, r) of the triangle. X in Equation 1 _i , Y _i , Z _i Indicates a coordinate value in xyz coordinates, and R ³ Indicates a 3D space, and N indicates the number of vertices. A set of vertex indices can be considered, but no further entry will be made in this specification.
[0006]
[Expression 1]

[0007]
All vertices with k edges (the vertex order is k) are called k-regular meshes or simply regular meshes, and vertices with any number of orders are non- This is called a regular mesh. In addition, a case where almost all vertices are regular may be called semi-regular meshes in order to distinguish them from regular meshes. In this specification, all vertices other than the boundary part of the 3D mesh are regular. In this case, it is simply called a regular mesh. FIG. 19 illustrates a regular mesh and a non-regular mesh. As shown in FIG. 19A, in a 3D mesh composed of vertices and triangles, if all (or almost all) vertices have k triangles (or sides), the 3D mesh has regularity. That's it. On the other hand, as shown in FIG. 19B, when all (or almost all) vertices do not have k triangles (or sides), the 3D mesh is said to be irregular. A 3D mesh is a 3D model that approximates the surface of an object by connecting vertices arranged in a three-dimensional space with polygons.
[0008]
Next, a typical conventional modeling method will be described. As described in Non-Patent Document 1, there is a 3D video generation system developed at Carnegie Mellon University in the United States called “Virtualized Reality”. This 3D moving image generation system uses a technique for obtaining a 3D mesh using a plurality of range images. This distance image is an image in which a plane (image) in the 3D space is assumed and the distance from the plane to the object is given as a pixel value. However, the above-described 3D moving image generation system has a problem that regularity cannot be realized. Furthermore, there is a problem that the real-time property of the processing cannot be realized, and the progressiveness (gradual property), which is a characteristic that the image quality (resolution) is improved as the processing time is increased.
[0009]
As another conventional modeling method, there is a 3D video generation system having real-time characteristics as a modeling system developed by Professor Matsuyama of Kyoto University as described in Non-Patent Document 2. This 3D moving image generation system uses a visual volume intersection method. The visual volume intersection method is one of methods for reconstructing a 3D space from a two-dimensional image, and is a method for obtaining the position and surrounding shape of an object using a plurality of images taken by a camera at the same time. Although this 3D moving image generation system has real-time characteristics, it has a problem that it does not have regularity and progressiveness.
[0010]
As another modeling method, there is a method described in Patent Document 1. However, although this method has real-time properties, there is a problem in that it does not consider regularity and progressiveness.
[0011]
Next, techniques relating to meshing and triangulation will be described. As described in Non-Patent Document 3, there is a Delauney method as a well-known triangulation method. The Delauney method is a method in which if there are no other points in a circle passing through three points, a triangle is formed from those three points for all vertices. However, since the Delauney method has a large amount of processing calculation, there is a problem that there is no real time property and regularity is not promised.
[0012]
As another meshing and triangulation method, as described in Non-Patent Document 4, there is a well-known method called MAPS that converts a non-regular mesh into a regular mesh. MAPS targets non-regular meshes that have already been modeled. MAPS has a problem that it has a large amount of processing calculation and lacks real-time performance.
[0013]
As another method related to meshing and triangulation, there is a method described in Patent Document 1, but this method reproduces an acute angle feature and takes into account regularity, progressiveness, real-time characteristics, and the like. There was a problem that was not.
[0014]
[Problems to be solved by the invention]
There are various “3D mesh processing” techniques for 3D meshes acquired in some way, and many techniques such as smoothing, feature extraction, simplification / detailing, and encoding have been proposed. The 3D mesh constructs polygons by connecting randomly sampled points in 3D space by various methods. Since sampling is performed randomly as described above, there is no concept such as grids and pixels sampled at regular intervals in a normal digital signal. For this reason, it becomes difficult to process as time-sequential data at equal intervals, and the 3D mesh processing is complicated. For this reason, conventionally, attempts have been actively made to facilitate the handling of 3D meshes by parameterizing meshes with irregularly connected vertices (non-regular meshes) and converting them into regular meshes. However, there is a problem that none of them has regularity and progressiveness. In particular, since the two steps of modeling a non-regular 3D mesh and then converting it to a regular mesh are taken, the amount of processing calculation becomes very large. As a result, there is a problem that it is inconvenient when it is desired to obtain a regular 3D mesh in a short time, or when it is desired to generate a dynamic regular 3D mesh for a moving object (dynamic object) in real time.
[0015]
Many conventional 3D modeling techniques aim to obtain the shape of a stationary object (static object) or obtain 3D animation off-line using a video signal sequence stored on disk. It is an object. Several methods for processing and modeling a video signal sequence sent from a camera in real time have been proposed for acquiring a 3D animation of a dynamic object. However, when modeling a moving object in real time, naturally, the time required for modeling one frame varies depending on the processing capability of the processor used for calculation. In the conventional modeling method, one model is completed by executing processing along the entire algorithm. For this reason, there is a problem that if the time for completing the above algorithm is longer than one frame time, the modeling is not completed and the processing is wasted.
[0016]
Conventionally, when transmitting 3D mesh data, it is necessary to send information representing the connection of triangles in addition to the vertex coordinates. Therefore, there is a problem that it takes a long time to transmit 3D mesh data.
[0017]
Therefore, an object of the present invention is to solve the above-described problem, and a 3D mesh having regularity not only for a static object but also for a dynamic object can be modeled in real time. The purpose is to provide a 3D modeling program and the like having both progressive characteristics.
[0018]
A second object of the present invention is to provide a three-dimensional modeling program or the like that can abort an algorithm halfway and can model a 3D mesh having the best resolution at the time of termination.
[0019]
A third object of the present invention is to provide a three-dimensional modeling data transmission program that does not require a long time for transmission of 3D mesh data.
[0020]
[Means for Solving the Problems]
The three-dimensional modeling program according to the present invention is a three-dimensional modeling program for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, each acquired by the plurality of imaging devices. A distance image recording unit that records a distance image having a distance to the object as a pixel value in accordance with each imaging device, and an initial resolution recording unit that records a predetermined resolution. A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
The distance image of the object is acquired by each of the plurality of imaging devices under a predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determination step, and the acquired time is a time A distance image acquisition step of recording in the recording unit and recording the predetermined resolution in the resolution recording unit;
A distance image recording step of causing the distance image recording unit to record each distance image acquired in the distance image acquisition step;
A three-dimensional mesh modeling step of creating a three-dimensional mesh model of the object based on each distance image recorded in the distance image recording unit under the resolution recorded in the resolution recording unit;
If a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate and repeatedly executed from the three-dimensional mesh modeling step. A step of repeatedly executing from the distance image acquiring step when a predetermined time has elapsed from a time recorded in the time recording unit and a predetermined end time has not been exceeded.
Is a three-dimensional modeling program for executing
[0021]
Here, in the three-dimensional modeling program of the present invention, the three-dimensional mesh modeling step includes:
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of each distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step for recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. A three-dimensional projection point creation step;
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value;
The two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is recorded in the pixel center point recording unit together with the interpolation distance value of the center point of each pixel obtained in the interpolation distance value creation step. A pixel center point recording step;
Pixel center for obtaining a three-dimensional inverse transformation point of the center point of each pixel by inversely transforming the two-dimensional coordinate value of the center point of each pixel recorded in the pixel center recording unit into a three-dimensional coordinate value on the virtual solid A three-dimensional inverse transformation point creation step;
The three-dimensional coordinate value of the three-dimensional inverse transform point of the center point of each pixel obtained in the pixel center three-dimensional inverse transform point creation step is used as the interpolation distance of the center point of each pixel recorded in the pixel center point recording unit. A pixel center 3D inverse transform point recording step for recording in the pixel center 3D inverse transform point recording unit together with the value;
3 of each surface point corresponding to the center point of each pixel obtained by back-projecting the interpolation distance value of the center point of each pixel recorded in the pixel center three-dimensional inverse transformation point recording unit in the virtual three-dimensional space. A step of creating a surface point corresponding to a pixel center for obtaining a dimensional coordinate value;
A pixel center corresponding surface point recording step for recording a three-dimensional coordinate value of each surface point corresponding to the center point of each pixel obtained in the pixel center corresponding surface point creating step in a pixel center corresponding surface point recording unit. Can do.
[0022]
Here, in the three-dimensional modeling program of the present invention, the three-dimensional projection point recording step includes the step of recording the three-dimensional projection point recording step when the resolution recorded in the resolution recording unit is equal to a predetermined resolution recorded in the initial resolution recording unit The three-dimensional projection vector to each three-dimensional projection point obtained in the three-dimensional projection point creation step is combined with an attribute value indicating the three-dimensional coordinate value of the surface point corresponding to the three-dimensional projection point for each three-dimensional projection point. Record in the 3D projection point recording unit,
In the three-dimensional projection point creation step, when the resolution recorded in the resolution recording unit is larger than a predetermined resolution recorded in the initial resolution recording unit,
With respect to the previous surface point having the three-dimensional coordinate value already recorded in the surface point recording unit, the previous three-dimensional projection point corresponding to the previous surface point is already recorded in the three-dimensional projection point recording unit. A new surface point having a three-dimensional coordinate value newly recorded in the surface point recording unit by increasing the resolution using the previous three-dimensional projection vector corresponds to the previous surface point in the vicinity of the new surface point. A new three-dimensional projection vector can be obtained based on the attribute value of the conventional three-dimensional projection vector.
[0023]
Here, in the three-dimensional modeling program of the present invention, when the virtual solid is a virtual cylinder or a virtual cone, the three-dimensional projection point creating step includes an imaging device that has acquired a distance image including pixels corresponding to surface points. A spatial position information section in which spatial position information between the object and the object is recorded;
A virtual solid creation step of creating a center line of a virtual cylinder or virtual cone in a virtual space;
A plane that cuts the virtual cylinder or virtual cone that is perpendicular to the center line created in the virtual solid creation step and passes through the surface point recorded in the surface point recording unit, and the periphery is the circumference of the virtual cylinder or virtual cone A cutting plane creation step to create a cutting plane that matches
A center point coordinate value creating step for obtaining a coordinate value of the center point of the cutting plane created in the cutting plane creating step;
Find two points where a line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit intersects the periphery of the cutting plane 2 A point creation step;
Virtual plane creation that creates a virtual plane corresponding to a distance image including a pixel corresponding to the surface point recorded in the surface point recording unit in a virtual space based on the spatial position information recorded in the spatial position information unit Steps,
A pixel coordinate value creating step for obtaining a coordinate value of a pixel corresponding to the surface point in the virtual plane created in the virtual plane creating step;
A pixel vector that goes from the coordinate value of the pixel obtained in the pixel coordinate value creation step to the coordinate value of the center point obtained in the center point coordinate value creation step, and the surface point recorded in the surface point recording unit A vector creation step for obtaining two surface point vectors from the coordinate values to the two points obtained in the two-point creation step;
Comparing the angle between the pixel vector obtained in the vector creation step and each of the two surface point vectors, and setting the surface point vector corresponding to the small angle as a three-dimensional projection vector. .
[0024]
Here, in the three-dimensional modeling program of the present invention, the three-dimensional projection point creation step includes the step of:
A center point coordinate value creation step for obtaining a coordinate value of the center point of the virtual sphere;
On the line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit, the surface point is calculated from the coordinate value of the center point. And a step of setting a vector in the direction toward the three-dimensional projection vector.
[0025]
A three-dimensional modeling control program according to the present invention is a three-dimensional modeling control program for controlling processing for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, the plurality of imaging A distance image recording unit that records a distance image having a pixel value as a distance to the object acquired by each device, and an initial resolution recording unit that records a predetermined resolution are used. A plurality of computers corresponding to each distance image recording unit that creates a three-dimensional model of the object using each imaging device, and an integrated computer that integrates the processing results of the plurality of computers,
To a computer that executes a 3D modeling control program,
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
In each of the plurality of computers,
The distance image of the object is acquired by the imaging device under the predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determining step, and the acquired time is stored in the time recording unit. A distance image acquisition step of recording and recording the predetermined resolution in a resolution recording unit;
A distance image recording step of recording the distance image acquired in the distance image acquisition step in the distance image recording unit;
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of the distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step of recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. 3D projection point creation step,
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
In the integrated computer,
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value, and a two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is interpolated with the center point of each pixel obtained in the interpolation distance value creation step. A pixel center point recording step for recording in the pixel center point recording unit together with the distance value;
Pixel center for obtaining a three-dimensional inverse transformation point of the center point of each pixel by inversely transforming the two-dimensional coordinate value of the center point of each pixel recorded in the pixel center recording unit into a three-dimensional coordinate value on the virtual solid 3D inverse transformation point creation step,
The three-dimensional coordinate value of the three-dimensional inverse transform point of the center point of each pixel obtained in the pixel center three-dimensional inverse transform point creation step is used as the interpolation distance of the center point of each pixel recorded in the pixel center point recording unit. A pixel center three-dimensional inverse transform point recording step for recording in the pixel center three-dimensional inverse transform point recording unit together with the value;
3 of each surface point corresponding to the center point of each pixel obtained by back-projecting the interpolation distance value of the center point of each pixel recorded in the pixel center three-dimensional inverse transformation point recording unit in the virtual three-dimensional space. A step for creating a surface point corresponding to a pixel center for obtaining a dimensional coordinate value;
Performing a pixel center corresponding surface point recording step of recording a three-dimensional coordinate value of each surface point corresponding to the center point of each pixel obtained in the pixel center corresponding surface point creating step in a pixel center corresponding surface point recording unit;
When a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate, and the surface point creating step is performed in each of the plurality of computers. When the predetermined time has elapsed from the time recorded in the time recording unit and the predetermined end time has not been exceeded, each of the plurality of computers is repeatedly executed from the distance image acquisition step. Step
Is a three-dimensional modeling control program for executing
[0026]
A three-dimensional modeling data transmission program of the present invention is a three-dimensional modeling data transmission program for transmitting three-dimensional modeling data of an object created using a plurality of imaging devices arranged around the object. A distance image recording unit that records a distance image having a distance to the object as a pixel value acquired by each of the plurality of imaging devices according to each imaging device, and an initial resolution recording unit that records a predetermined resolution. To the computer,
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
The distance image of the object is acquired by each of the plurality of imaging devices under a predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determination step, and the acquired time is a time A distance image acquisition step of recording in the recording unit and recording the predetermined resolution in the resolution recording unit;
A distance image recording step of causing the distance image recording unit to record each distance image acquired in the distance image acquisition step;
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of each distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step of recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. 3D projection point creation step,
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value;
The two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is recorded in the pixel center point recording unit together with the interpolation distance value of the center point of each pixel obtained in the interpolation distance value creation step. Pixel center point recording step,
When a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate and repeatedly executed from the surface point creating step, and the time When a predetermined time has elapsed since the time recorded in the recording unit and the predetermined end time has not been exceeded, the distance image obtaining step is repeatedly executed, and the predetermined time from the time recorded in the time recording unit is set. 3D modeling including a two-dimensional coordinate value of the center point of each pixel recorded in the pixel center point recording unit and an interpolation distance value of the center point of each pixel when a predetermined end time has passed. Steps for transmitting data
Is a three-dimensional modeling data transmission program.
[0027]
The recording medium of the present invention is a computer-readable recording medium on which any program of the present invention is recorded.
[0028]
The three-dimensional modeling method of the present invention is a three-dimensional modeling method for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, each of which is acquired by the plurality of imaging devices. A distance image recording unit that records a distance image having a distance to the object as a pixel value according to each imaging device, and an initial resolution recording unit that records a predetermined resolution.
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
The distance image of the object is acquired by each of the plurality of imaging devices under a predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determination step, and the acquired time is a time A distance image acquisition step of recording in the recording unit and recording the predetermined resolution in the resolution recording unit;
A distance image recording step of recording each distance image acquired in the distance image acquisition step in the distance image recording unit;
A three-dimensional mesh modeling step of creating a three-dimensional mesh model of the object based on each distance image recorded in the distance image recording unit under the resolution recorded in the resolution recording unit;
If a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate and repeatedly executed from the three-dimensional mesh modeling step. A step of repeatedly executing from the distance image acquiring step when a predetermined time has elapsed from a time recorded in the time recording unit and a predetermined end time has not been exceeded;
Is executed.
[0029]
Here, in the three-dimensional modeling method of the present invention, the three-dimensional mesh modeling step includes:
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of each distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step for recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. A three-dimensional projection point creation step;
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value, and a two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is used as the center point of each pixel obtained in the interpolation distance value creation step. A pixel center point recording step for recording in the pixel center point recording unit together with the interpolation distance value;
Pixel center for obtaining a three-dimensional inverse transformation point of the center point of each pixel by inversely transforming the two-dimensional coordinate value of the center point of each pixel recorded in the pixel center recording unit into a three-dimensional coordinate value on the virtual solid A three-dimensional inverse transformation point creation step;
The three-dimensional coordinate value of the three-dimensional inverse transform point of the center point of each pixel obtained in the pixel center three-dimensional inverse transform point creation step is used as the interpolation distance of the center point of each pixel recorded in the pixel center point recording unit. A pixel center 3D inverse transform point recording step for recording in the pixel center 3D inverse transform point recording unit together with the value;
3 of each surface point corresponding to the center point of each pixel obtained by back-projecting the interpolation distance value of the center point of each pixel recorded in the pixel center three-dimensional inverse transformation point recording unit in the virtual three-dimensional space. A step of creating a surface point corresponding to a pixel center for obtaining a dimensional coordinate value;
A pixel center corresponding surface point recording step for recording a three-dimensional coordinate value of each surface point corresponding to the center point of each pixel obtained in the pixel center corresponding surface point creating step in a pixel center corresponding surface point recording unit. Can do.
[0030]
Here, in the three-dimensional modeling method of the present invention, when the resolution recorded in the resolution recording unit is equal to a predetermined resolution recorded in the initial resolution recording unit, the three-dimensional projection point recording step The three-dimensional projection vector to each three-dimensional projection point obtained in the projection point creation step is set together with an attribute value indicating the three-dimensional coordinate value of the surface point corresponding to the three-dimensional projection point for each three-dimensional projection point. Recorded in the dimensional projection point recording section,
In the three-dimensional projection point creation step, when the resolution recorded in the resolution recording unit is larger than a predetermined resolution recorded in the initial resolution recording unit,
With respect to the previous surface point having the three-dimensional coordinate value already recorded in the surface point recording unit, the previous three-dimensional projection point corresponding to the previous surface point is already recorded in the three-dimensional projection point recording unit. A new surface point having a three-dimensional coordinate value newly recorded in the surface point recording unit by increasing the resolution using the previous three-dimensional projection vector corresponds to the previous surface point in the vicinity of the new surface point. A new three-dimensional projection vector can be obtained based on the attribute value of the conventional three-dimensional projection vector.
[0031]
Here, in the three-dimensional modeling method according to the present invention, the three-dimensional projection point creation step includes: an imaging device and an object that acquire a distance image including pixels corresponding to surface points when the virtual solid is a virtual cylinder or a virtual cone; Further comprising a spatial position information section in which spatial position information is recorded between
A virtual solid creation step of creating a center line of a virtual cylinder or virtual cone in a virtual space;
A plane that cuts the virtual cylinder or virtual cone that is perpendicular to the center line created in the virtual solid creation step and passes through the surface point recorded in the surface point recording unit, and the periphery is the circumference of the virtual cylinder or virtual cone A cutting plane creation step to create a cutting plane that matches
A center point coordinate value creating step for obtaining a coordinate value of the center point of the cutting plane created in the cutting plane creating step;
Find two points where a line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit intersects the periphery of the cutting plane 2 A point creation step;
Virtual plane creation that creates a virtual plane corresponding to a distance image including a pixel corresponding to the surface point recorded in the surface point recording unit in a virtual space based on the spatial position information recorded in the spatial position information unit Steps,
A pixel coordinate value creating step for obtaining a coordinate value of a pixel corresponding to the surface point in the virtual plane created in the virtual plane creating step;
A pixel vector that goes from the coordinate value of the pixel obtained in the pixel coordinate value creation step to the coordinate value of the center point obtained in the center point coordinate value creation step, and the surface point recorded in the surface point recording unit A vector creation step for obtaining two surface point vectors from the coordinate values to the two points obtained in the two-point creation step;
Comparing the angle between the pixel vector obtained in the vector creation step and each of the two surface point vectors, and setting the surface point vector corresponding to the small angle as a three-dimensional projection vector. .
[0032]
Here, in the three-dimensional modeling method of the present invention, the three-dimensional projection point creation step includes the step of:
A center point coordinate value creation step for obtaining a coordinate value of the center point of the virtual sphere;
On the line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit, the surface point is calculated from the coordinate value of the center point. And a step of setting a vector in the direction toward the three-dimensional projection vector.
[0033]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this specification, only the triangular mesh in which all the most general polygons are triangles will be handled. In the drawings used in the following description, the parts with the same reference numerals indicate the same elements. Therefore, the description of the elements with the reference numerals is performed only for the first figure that appears in principle, and is omitted for the figure that appears again. To do.
[0034]
Embodiment 1 FIG.
FIG. 1 shows an environment in which a three-dimensional modeling program or the like according to Embodiment 1 of the present invention operates. In FIG. 1, reference numeral 1 is a computer on which the three-dimensional modeling program of the present invention is executed, 10 is a recording device such as a disk connected to the computer 1, 20 is an object for creating a three-dimensional model, 25a , 25b, 25c, 25d, 25e, and 25f are a plurality of cameras (imaging devices) connected to the computer 1 and arranged around the object 20. In FIG. 1, six cameras 25a and the like are shown, but this is for the convenience of the drawing, and the three-dimensional modeling program of the present invention can be applied to two or more cameras.
[0035]
As shown in FIG. 1, the recording apparatus 10 includes a distance image recording unit 3 that records distance images acquired by a plurality of cameras 25a and the like according to the cameras 25a and the like. Here, the distance image is an image with a predetermined resolution having a pixel value as a distance from the plane to the object 20 assuming a plane (image) in the 3D space. The recording apparatus 10 further includes an initial resolution recording unit 4 that records a predetermined resolution (N and M in the case of N × M), a time recording unit 5 that records the time when the distance image is acquired by the camera 25a, and the like, which will be described later. Thus, the initial resolution recording unit 6 that records the updated resolution is provided. The recording apparatus 10 includes another surface point recording unit 11 that records predetermined data, a three-dimensional projection point recording unit 12, a two-dimensional conversion point recording unit 13, a pixel center point recording unit 14, and a pixel center three-dimensional inverse conversion point recording unit. 15. A pixel center-corresponding surface point recording unit 16 and a spatial position information unit 17 are provided. These recording units will be described later.
[0036]
FIG. 2 is a diagram illustrating the distance image. In FIG. 2A, reference numeral 30 denotes a distance image of the object 20 acquired by the camera 25a, for example. As shown in FIG. 2A, the pixel A on the distance image 30 has a distance ra between the pixel A and a point B on the object 20 as a pixel value. In FIG. 2B, reference numeral 30 is a distance image of the object 20 acquired by the camera 25a, for example, 31 is a distance image of the object 20 acquired by the camera 25b, for example, and 32 is an object 20 acquired by the camera 25c, for example. A distance image 33 is a distance image of the object 20 acquired by the camera 25d, for example, 34 is a distance image of the object 20 acquired by the camera 25e, for example, and 35 is a distance image of the object 20 acquired by the camera 25f, for example. As shown in FIG. 2B, the pixel C on the distance image 35 has a distance rb between the pixel C and the point B on the object 20 as a pixel value. These distance images 30 and the like are recorded in the distance image recording unit 3 according to each camera 25a and the like.
[0037]
The three-dimensional modeling program of the present invention uses distance images created for the number of cameras 25a and the like corresponding to the number (I). There are many methods for obtaining the distance image 30 and the like using a plurality of cameras 25a and the like. However, regardless of the method for obtaining the distance image 30 or the like in this specification, the distance image 30 or the like is obtained using a certain algorithm. It is assumed that each image is recorded in the image recording unit 3. As a method for acquiring the distance image 30 or the like, a general stereo image method can be used. A method of measuring distance by an optical time-of-flight measurement method using an optical radar range finder, a phase difference detection range finder, or the like may be used. Alternatively, a range finder based on triangulation using a spot light projection method, a slit light projection method, a step light projection method, a pattern light projection method, or the like can be used.
[0038]
Next, the function of the three-dimensional modeling program of the present invention will be described. FIG. 3 is a flowchart showing the function of the three-dimensional modeling program of the present invention. As shown in FIG. 3, first, corresponding points of each distance image 30 and the like recorded in the distance image recording unit 3 are determined (corresponding point determination step, step S10). When using a plurality of distance images 30 and the like, it is necessary to determine in advance the corresponding point (corresponding position) to which pixel in the other distance image 35 the pixel A in a certain distance image 30 corresponds. For example, ICP (Iterative Closest Point Method) can be used to determine the corresponding points. Examining the corresponding point of each distance image 30 or the like is called registration. Thus, the distance images 30 and the like can be combined. The corresponding point determination step (step S10) is performed offline.
[0039]
Subsequently, a plurality of cameras 25a under the predetermined resolution (N × M) recorded in the initial resolution recording unit 4 such as the distance image 30 of the object 20 and the corresponding points determined in the corresponding point determination step (step S10). Each is acquired by etc. As a result, the number (I) of distance images 30 and the like of the cameras 25a and the like are acquired. At the same time, the time (t) at which the distance image 30 or the like is acquired is recorded in the time recording unit 5, and a predetermined resolution (N × M) is recorded in the resolution recording unit 6 (distance image acquisition step, step S12).
[0040]
Next, each distance image 30 acquired in the distance image acquisition step (step S12) is recorded in the distance image recording unit 3 (distance image recording step, step S14).
[0041]
Under the resolution (initially N × M) recorded in the resolution recording unit 6, a three-dimensional mesh model of the object 20 is created based on each distance image 30 and the like recorded in the distance image recording unit 3 (3 Dimensional mesh modeling step, step S16). Step S16 will be described in detail later.
[0042]
It is determined whether or not a predetermined time (frame rate: δT) has elapsed from the time (t) recorded in the time recording unit 5 (step S18). If not, it is recorded in the resolution recording unit 6. The resolution (initially N × M) is increased at a predetermined rate, for example, a rate of 2 times (2N × 2M), and is repeatedly executed from the three-dimensional mesh modeling step (step S16) (step S20). On the other hand, when the predetermined time (δT) has elapsed from the time (t) recorded in the time recording unit 5 and the time is t + δT and the predetermined end time has not been exceeded, the distance image acquisition step (step The process is repeated from S12) (step S22). The processing from the distance image acquisition step (step S12) to step S20 or S22 is executed in real time.
[0043]
Next, the three-dimensional mesh modeling step (step S16) will be described in detail. FIG. 4 is a flowchart showing the function of the three-dimensional mesh modeling step in the three-dimensional modeling program of the present invention. Hereinafter, the flowchart of FIG. 4 will be described with reference to FIGS.
[0044]
A point can be defined on the surface of the object 20 by back-projecting all pixel values of the distance image 30 and the like (that is, the distance from the center of the pixel A to the object 20) on the 3D plane. This point is hereinafter referred to as “surface point”. As shown in FIG. 4, first, the three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value such as each distance image 30 recorded in the distance image recording unit 3 in the virtual three-dimensional space is obtained. Obtain (surface point creation step. Step S30). FIG. 5 is a diagram for explaining the surface points. In FIG. 5, reference numeral 28 denotes one of the surface points defined as described above on the surface of the object 20.
[0045]
The three-dimensional coordinate values such as each surface point 28 obtained in the surface point creation step (step S30) are recorded in the surface point recording unit 11 (surface point recording step, step S32).
[0046]
A virtual solid, for example, a virtual cylinder (Virtual Cylinder), which is virtually defined in the 3D space around a reference point determined in advance, is defined. FIG. 6 is a diagram for explaining a virtual cylinder. In FIGS. 6A and 6B, reference numeral 45 is a reference point determined in advance, and 40 is a virtual cylinder defined around the reference point 45. 6A shows a plan view of the virtual cylinder 40 as viewed from above, and FIG. 6B shows a perspective view. 7A, 7B, and 7C show manifolds 40a, 40b, and 40c that are in phase with the virtual cylinder 40 shown in FIG. 6, respectively. In the following, description will be made using an example of a virtual cylinder. However, in the case of manifolds 40a, 40b and 40c as shown in FIG. 7, the processing of the three-dimensional mesh modeling step (step S16) is similarly applied. be able to.
[0047]
Next, all surface points 28 on the surface of the object 20 are projected (mapped) onto the virtual cylinder 40. As shown in FIG. 4, by projecting from each surface point 28 having the three-dimensional coordinate value recorded in the surface point recording unit 11 onto a virtual solid formed in a virtual three-dimensional space, for example, a virtual cylinder 40. A three-dimensional projection vector for each obtained three-dimensional projection point is obtained (three-dimensional projection point creation step, step S34). Here, for the projection, for example, a perspective projection method or an orthogonal projection method can be used.
[0048]
8 and 9 are diagrams for explaining the projection of the surface point 28 and the like onto the virtual cylinder 40. FIG. As shown in FIG. 8A, first, a cutting plane (or circle) 42 that is perpendicular to the center line 41 of the virtual cylinder 40 and passes through one surface point 28 on the object 20 is assumed. Next, as shown in FIG. 8B, the virtual cylinder 40 is cut into circles by a cutting plane (or circle) 42. As shown in FIG. 9A, a line L1 connecting the center (reference point) 45 and the surface point 28 of the circle 42 cut into a circle is defined. As projection points (three-dimensional projection points), two points of points 44 a and 44 b are obtained on the circumference of the circle 42. Therefore, as shown in FIG. 9B, a vector (pixel vector) G from the pixel A corresponding to the surface point 28 toward the surface point 28 and a vector (from the surface point 28 toward a projection point 44a on the circle 42) ( A surface point vector Ma and a vector (surface point vector) Mb from the surface point 28 to another projection point 44b on the circle 42 are assumed. The angle θa formed by the vector G and the vector Ma and the angle θb formed by the vector G and the vector Mb are compared and projected to the closer angle. For example, in the case of FIG. 9B, since θa <θb, the projection is performed on the upper portion, that is, the projection point 44a. The vector Ma used for this projection is called a projection vector (three-dimensional projection vector). A panorama such as the distance image 30 can be created by performing the above operation on all the surface points 28 and the like.
[0049]
The projected point 44a etc. on the virtual cylinder 40 has the value from the point to the object 20 as a value. Let V be the set of such points. As shown in FIG. 4, for each 3D projection point 44a etc., the 3D projection vector Ma etc. to each 3D projection point 44a etc. obtained in the 3D projection point creation step (step S34) The attribute value indicating the three-dimensional distance value between the three-dimensional projected point 44a and the like and the corresponding surface point 28 is recorded in the three-dimensional projected point recording unit 12 (three-dimensional projected point recording step, step S36). That is, the set V is recorded in the three-dimensional projection point recording unit 12.
[0050]
Next, one developed plane is obtained by cutting the virtual cylinder 40 with a certain line. As shown in FIG. 4, the three-dimensional projection vector Ma and the like recorded for each three-dimensional projection point 44a and the like recorded in the three-dimensional projection point recording unit 12 are converted into a virtual plane developed by cutting the virtual solid 40. As a two-dimensional conversion vector to a two-dimensional conversion point (two-dimensional conversion point creation step, step S38). FIG. 10 is a view for explaining the development of the virtual cylinder 40. As shown in FIG. 10A, when the virtual cylinder 40 is cut along a certain line L2, it can be developed into a virtual plane P as shown in FIG. 10B. As shown in FIG. 10B, the point 44a (2) on the virtual plane P is a two-dimensional conversion point obtained by converting the three-dimensional projection point 44a into a two-dimensional space.
[0051]
As shown in FIG. 4, the two-dimensional conversion vector to each two-dimensional conversion point 44a (2) obtained in the two-dimensional conversion point creation step (step S38) is converted into each two-dimensional conversion point 44a (2) etc. Each attribute value recorded in the three-dimensional projection point recording unit 12 is recorded in the two-dimensional conversion point recording unit 13 (two-dimensional conversion point recording step, step S40).
[0052]
Next, as shown in FIG. 4, a rectangular pixel of a predetermined size is defined on the virtual plane P, and a new center point is defined at the center of each pixel, and its coordinate value is obtained (pixel center point creation) Step, step S42). Let Vc be the set of center points. FIG. 11 shows the virtual plane P. As shown in FIG. 11A, the virtual plane P is a rectangular pixel P. ₁₁ , P ₁₅ Etc. are defined. The two-dimensional conversion point 44a (2) is a pixel P ₁₅ The other two-dimensional conversion points 46 (2), 47 (2), 48 (2) and the like are included in the surrounding pixels. As shown in FIG. 11B, the above-mentioned center point is defined on the virtual plane P. For example, pixel P ₁₁ The center point of C is C ₁₁ , Pixel P ₁₅ The center point of C is C ₁₅ And so on. The element of the set Vc is C ₁₁ , C ₁₅ Etc.
[0053]
Here, the distance value to the object 20 which each point of the set Vc has is calculated in the following procedure. First, each point C of the set Vc ₁₅ For example, several points of V having the closest Euclidean distance defined in the virtual plane P are found. For example, two-dimensional conversion points 44a (2), 46 (2), 47 (2), 48 (2), etc. can be found. Using the distance values (attribute values) of these several points, the distance value of Vc is obtained by simple linear interpolation. Specifically, as shown in FIG. 4, each pixel P obtained in the pixel center point creation step (step S42). ₁₅ Center point C ₁₅ Within a range of a predetermined two-dimensional distance (Euclidean distance) value based on a two-dimensional coordinate value such as a two-dimensional conversion vector such as a two-dimensional conversion point 44a (2) recorded in the two-dimensional conversion point recording unit 13. A predetermined number of two-dimensional conversion points 44a (2) and the like are obtained. For the predetermined number of two-dimensional conversion points 44a (2), etc., the attribute values (three-dimensional distance values between the surface points 28, etc. and the three-dimensional projection points 44a, etc.) recorded in the two-dimensional conversion point recording unit 13 are used. Based on each pixel P ₁₅ Center point C ₁₅ The interpolated interpolation distance value is obtained (interpolation distance value creating step, step S44). Here, when obtaining the interpolation distance value, for example, linear interpolation or spline interpolation can be used.
[0054]
As shown in FIG. 4, each pixel P obtained in the pixel center point creation step (step S42). ₁₅ Center point C ₁₅ The two-dimensional coordinate values such as the pixel P obtained in the interpolation distance value creating step (step S44). ₁₅ Center point C ₁₅ Are recorded in the pixel center point recording unit 14 together with the interpolation distance value such as (pixel center point recording step, step S46).
[0055]
Next, as shown in FIG. ₁₅ The 3D model is actually triangulated by connecting etc. with triangles. Since the above-described Vc has regularity, the triangulation is simply performed at the upper left, upper, left, right, lower, lower right pixels P as shown in FIG. ₁₅ Center point C ₁₅ It can be achieved by connecting with the above. The right end point and the left end point of the virtual plane P are similarly connected. The number of grids on the virtual plane P is also increased in accordance with the resolution of the distance image 30 or the like.
[0056]
Next, the virtual plane P is returned to the virtual cylinder 40. A 3D model having a regular mesh can be created by back-projecting the newly created distance value Vc onto the object 20. Specifically, as shown in FIG. 4, each pixel P recorded in the pixel center recording unit 14. ₁₅ Center point C ₁₅ Are converted back to the three-dimensional coordinate values on the virtual cylinder 40 to convert each pixel P ₁₅ Center point C ₁₅ Are obtained (pixel center three-dimensional inverse transformation point creation step, step S48). Each pixel P obtained in the pixel center three-dimensional inverse transformation point creation step (step S48) ₁₅ Center point C ₁₅ For each pixel P recorded in the pixel center point recording unit 14. ₁₅ Center point C ₁₅ Are recorded in the pixel center three-dimensional inverse transform point recording unit 15 together with the interpolation distance value such as (pixel center three-dimensional inverse transform point recording step, step S50). Each pixel P recorded in the pixel center three-dimensional inverse transformation point recording unit 15 ₁₅ Center point C ₁₅ Each pixel P obtained by back-projecting the interpolation distance value such as in the virtual three-dimensional space ₁₅ Center point C ₁₅ The three-dimensional coordinate value of each surface point 28 etc. corresponding to etc. is obtained (pixel center corresponding surface point creation step, step S52). Each pixel P obtained in the pixel center corresponding surface point creation step (step 52) ₁₅ Center point C ₁₅ Are recorded in the pixel center corresponding surface point recording unit 16 (pixel center corresponding surface point recording step, step S54).
[0057]
As described above, the three-dimensional modeling program of the present invention has not only a feature related to “real time property” and “regularity” but also a feature that enables progressive modeling. have. The three-dimensional modeling program of the present invention has scalability in calculation time. Here, scalability refers to a characteristic that can control a certain variable (here, computational complexity) in modeling by selecting and executing part or all of the algorithm. Therefore, the three-dimensional modeling program of the present invention has a feature that even if the entire algorithm for one frame is cut off without being executed, a model having the optimum resolution at that time can be obtained. A high-resolution 3D mesh can be modeled when the processing capacity of the processor is high, and a low-resolution 3D mesh when it is low. This provides significant advantages for real-time modeling. The three-dimensional modeling program of the present invention can be called progressive modeling in the sense that the resolution increases as the algorithm is repeatedly executed.
[0058]
As described above, according to the first embodiment of the present invention, first, corresponding points such as each distance image 30 recorded in the distance image recording unit 3 in advance by offline processing are determined. Next, the distance image 30 or the like of the object 20 is acquired by the plurality of cameras 25a or the like under the predetermined resolution (N × M) recorded in the initial resolution recording unit 6 and the determined corresponding points by online processing. The acquired time (t) is recorded in the time recording unit 5 and a predetermined resolution (N × M) is recorded in the resolution recording unit 6. The acquired distance images 30 and the like are recorded in the distance image recording unit. Based on the distance images 30 and the like recorded in the distance image recording unit 3 under the resolution recorded in the resolution recording unit 6, a three-dimensional mesh model of the object 20 is created.
[0059]
In creating a three-dimensional mesh model, first, pixel values of a plurality of distance images 30 and the like are projected back onto the 3D space to define surface points on the object 20. A virtual cylinder 40 is defined in the 3D space, and all surface points on the object 20 are projected onto the virtual cylinder 40 to obtain a three-dimensional projection point. The three-dimensional projection point has a distance value between the surface point 28 and the projection point. A virtual cylinder is developed on a virtual plane P in a two-dimensional space, and a two-dimensional conversion point obtained by converting a three-dimensional projection point into a two-dimensional space is obtained. This two-dimensional conversion point has a distance value of a three-dimensional projection point. A rectangular pixel of a predetermined size is defined on the virtual plane P, and a center point is newly defined at the center of each pixel. The distance value of the center point is obtained by interpolating the distance values of the surrounding two-dimensional conversion points. Triangulate the 3D model by connecting each center point with a triangle. A 3D mesh model having a regular mesh can be created by returning the virtual plane P to the virtual cylinder 40 and back-projecting the distance value of the center point onto the object 20.
[0060]
Subsequently, when the predetermined time (δT) has not elapsed since the time (t) recorded in the time recording unit 5, the resolution recorded in the resolution recording unit 6 is increased by a predetermined rate, for example, twice. Repeat from the 3D mesh modeling step. When the predetermined time has elapsed from the time (t) recorded in the time recording unit and the time becomes t + δT and the predetermined end time has not been exceeded, the distance image acquisition step is repeatedly executed. As described above, a 3D mesh having regularity not only for a static object but also for a dynamic object can be modeled in real time, and a three-dimensional modeling program having progressive characteristics can be provided. By combining this progressive property, it is possible to provide a three-dimensional modeling program or the like that can abort the algorithm halfway and model a 3D mesh having the best resolution at the time of termination.
[0061]
Embodiment 2. FIG.
In the first embodiment, the mapping method from the surface point 28 or the like to the virtual cylinder 40 is shown in the three-dimensional projection point creation step (step S34). In the second embodiment, another mapping method will be described.
[0062]
When the first repetition at each time t, that is, when the resolution of the distance image 30 or the like is N × M, the same processing as the three-dimensional projection point creation step (step S34) is performed. However, the three-dimensional projection point 44a or the like has not a distance value to the object 20 but a three-dimensional coordinate (x, y, z) of the surface point 28 or the like on the object 20 as a value. Specifically, in the three-dimensional projection point recording step (step S36), when the resolution recorded in the resolution recording unit 6 is equal to the predetermined resolution recorded in the initial resolution recording unit 4, a three-dimensional projection point creating step (step S34). The three-dimensional projection vector Ma to the respective three-dimensional projection points 44a and the like obtained in (3) is obtained for each three-dimensional projection point 44a and the like together with the attribute value indicating the three-dimensional coordinate value of the surface point 28 and the like. Recorded in the recording unit 12.
[0063]
When the second repetition, that is, when the resolution is 2N × 2M or more, the projection is performed according to the following procedure. First, a three-dimensional projection vector of a new pixel that can be obtained by increasing the resolution from N × M to 2N × 2M is obtained. FIG. 12 is a diagram for explaining a mapping method according to Embodiment 2 of the present invention. FIG. 12A shows a pixel P on the virtual plane P when the resolution is N × M. ₁₁ , P ₁₂ Etc. (black circles), and FIG. 12B shows a pixel P on the virtual plane P when the resolution is 2N × 2M. ₁₁ , P ₁₂ , P _11-2 , P _11-3 , P _11-4 Etc. Pixel P indicated by a white circle in FIG. _11-2 , P _11-3 , P _11-4 Etc. are new pixels. N × M pixel P ₁₁ , P ₁₂ Etc. (black circle) has already been obtained, such as the three-dimensional projection vector Ma, and the pixel P newly added by interpolating or extrapolating the three-dimensional projection vector _11-2 , P _11-3 , P _11-4 A three-dimensional projection vector such as is obtained. Specifically, when the resolution recorded in the resolution recording unit 6 is larger than the predetermined resolution (N × M) recorded in the initial resolution recording unit 4 in the three-dimensional projection point creation step (step S34), the surface point recording unit For the previous surface point 28 having the three-dimensional coordinate value already recorded in 11, the previous three-dimensional projection point 44a corresponding to the previous surface point 28, etc. is already recorded in the three-dimensional projection point recording unit 12. A conventional three-dimensional projection vector Ma or the like is used. On the other hand, for a new surface point having a three-dimensional coordinate value newly recorded in the surface point recording unit 11 due to an increase in resolution, the previous three-dimensional projection corresponding to the previous surface point 28 in the vicinity of the new surface point, etc. A new three-dimensional projection vector can be obtained based on an attribute value (three-dimensional distance value) such as the vector Ma. After that, the intersection between the new three-dimensional projection vector and the virtual cylinder 40 is determined and mapped. The three-dimensional projection point has a three-dimensional coordinate (x, y, z) of a point on the corresponding object 20 as a value.
[0064]
As described above, according to the second embodiment of the present invention, at the first repetition at each time t, the three-dimensional projection point 44a and the like are not distance values to the object 20, but the surface on the object 20. Except for the point having the three-dimensional coordinates (x, y, z) such as the point 28 as a value, the same processing as the three-dimensional projected point creation step (step S34) is performed. In the second and subsequent iterations, the three-dimensional projection vector of the new pixel can be obtained by interpolating or extrapolating the three-dimensional projection vector already obtained in the previous iteration. As a result, a new three-dimensional vector can be efficiently obtained when the resolution increases due to repetition.
[0065]
Embodiment 3 FIG.
In the first and second embodiments, a virtual cylinder is used as an object to be projected. In the third embodiment, the case of a virtual cone will be described. The algorithm of the three-dimensional modeling program of the present invention can be applied as it is to the case of a virtual cone (or a manifold having the same shape as the cone) as well as the virtual cylinder 40. In the following, the three-dimensional projection point creation step (step S34) in the flowchart shown in FIG. 4 will be described by replacing the virtual cylinder 40 with a virtual cone. Since other steps are the same, description thereof is omitted.
[0066]
FIG. 13 is a diagram for explaining mapping using a virtual cone in Embodiment 3 of the present invention. As shown in FIG. 13A, when the virtual cone 50 is cut along a certain line L2 and developed, a virtual plane P2 shown in FIG. 13B can be obtained. First, a spatial position information unit 17 in which the spatial position information between the camera 25a and the object 20 that acquired the distance image 30 including the pixels corresponding to the surface point 28 and the like and the object 20 is recorded is further provided. FIG. 14 shows a flowchart of the three-dimensional projection point creation step in the third embodiment of the present invention. As shown in FIG. 14, the center line of the virtual cone 50 (corresponding to the center line 41 in the case of the virtual cylinder 40) is created in the virtual space (virtual solid creation step, step S60). A cutting plane (virtual cylinder) that cuts the virtual cone 50 that is perpendicular to the created center line and passes through the surface point 28 and the like recorded in the surface point recording unit 11 and whose circumference coincides with the circumference of the virtual cone 50 (Corresponding to the cutting plane 42 in the case of 40) is created (cutting plane creation step, step S62). The coordinate value of the center point of the created cutting plane is obtained (center point coordinate value creating step, step S64), and the obtained coordinate value of the center point and the coordinate value of the surface point 28 etc. recorded in the surface point recording unit 11 are obtained. 2 points (corresponding to 44a and 44b in the case of the virtual cylinder 40) where the line segment connecting the two intersects the periphery of the cutting plane is obtained (two-point creation step, step S66). Based on the spatial position information recorded in the spatial position information unit 17, a virtual plane corresponding to the distance image 30 including pixels corresponding to the surface points 28 recorded in the surface point recording unit 11 is created in the virtual space. (Virtual plane creation step. Step S68). The coordinate value of the pixel corresponding to the surface point 28 etc. in the created virtual plane is obtained (pixel coordinate value creation step, step S70). The pixel vector (corresponding to the vector G in the case of the virtual cylinder 40) heading from the obtained pixel coordinate value to the coordinate value of the center point, and the coordinate value of the surface point 28 etc. recorded in the surface point recording unit 11 Two surface point vectors to two points (corresponding to vectors Ma and Mb in the case of the virtual cylinder 40) are obtained (vector creation step, step S72). The angle between the pixel vector and each of the two surface point vectors (corresponding to the angles θa and θb in the case of the virtual cylinder 40) is compared (step S74), and the surface point vector corresponding to the small angle is three-dimensionally projected. A vector is set (steps S76 and S78).
[0067]
As described above, according to the third embodiment of the present invention, the algorithm of the three-dimensional modeling program of the present invention is similar to the virtual cylinder 40 in the case of a virtual cone (or a manifold that is in phase with the cone). Can be applied as is.
[0068]
Embodiment 4 FIG.
In the above-described embodiment, the case where a virtual cylinder or a virtual cone is used as a target to be projected has been described. The algorithm of the three-dimensional modeling program of the present invention can be similarly applied to a virtual sphere (or a manifold that is in phase with the sphere) as well as the virtual cylinder 40 or the virtual cone by changing a part thereof. In the following, the three-dimensional projection point creation step (step S34) and the two-dimensional conversion point creation step (step S38) in the flowchart shown in FIG. 4 will be described by replacing the virtual cylinder 40 with a virtual sphere. Since other steps are the same, description thereof is omitted.
[0069]
FIG. 15 is a diagram for explaining mapping using a virtual sphere according to Embodiment 4 of the present invention. As shown in FIG. 15A, in the three-dimensional projection point creation step (step S34), first, the coordinate value of the center point 65 of the phantom sphere 60 is obtained (center point coordinate value creation step), and then the center point 65. A vector in a direction from the coordinate value of the center point 65 to the surface point 28 etc. on a line segment connecting the coordinate value of the surface point 28 and the coordinate value of the surface point 28 recorded in the surface point recording unit 11 is a three-dimensional projection vector. To do. As shown in FIG. 15B, in the two-dimensional transformation point creation step (step S38), the virtual plane P3 can be obtained by performing the same method as mapping the globe to the world map.
[0070]
As described above, according to the fourth embodiment of the present invention, the algorithm of the three-dimensional modeling program of the present invention is used in the case of a virtual sphere (or a manifold that is in phase with the sphere) as with the virtual cylinder 40. This can also be applied by making a vector in the direction from the center point 65 of the phantom sphere 60 to the surface point 28 etc. a three-dimensional projection vector.
[0071]
Embodiment 5 FIG.
The algorithm of the three-dimensional modeling program in each embodiment of the present invention described above can be executed by a system in which a plurality of computers (personal computers: PCs) are arranged in a tree shape instead of a single computer. .
[0072]
FIG. 16 shows an environment (system) in which a three-dimensional modeling control program for controlling processing for creating a three-dimensional model of an object according to Embodiment 5 of the present invention is executed. In FIG. 16, reference numerals 70a to 70m are m cameras, 80a to 80m are distance image recording units corresponding to the cameras 70a to 70m, and 5a to 5n are each a camera 70a. A plurality of computers corresponding to the distance image recording units 80a to 80m, 5p is an integrated computer that integrates the processing results of the plurality of computers 5a to 5n, and 5q is a computer on which the three-dimensional modeling control program itself is executed. The cameras 70a to 70m, the distance image recording units 80a to 80m, the computers 5a to 5n, the integrated computer 5p, and the computer 5q are connected by a bus 75 and constitute layer 3. In layer 2, integrated recording units 90a to 90 that record the results of collecting the distance image recording units 80a to 80m two by two. _{m / 2} The layer 1 has a final result recording unit 100 that records a final result.
[0073]
Below, which computer is made to perform each step in the flowchart shown by FIG. 3 and FIG. 4 is demonstrated. First, the computer 5q is caused to determine the corresponding point of each distance image recorded in the distance image recording units 80a to 80m (corresponding point determination step, step S10). In layer 3, each of the plurality of computers 5a to 5n is in charge of the distance image acquisition step (step S12) to the three-dimensional projection point recording step (step S36). Each PC handles only one distance image. In layer 2, the integrated computer 5p is in charge of the process from the two-dimensional conversion point creation step (step S38) to the pixel center corresponding surface point recording step (step S54). That is, one integrated computer 5p handles a part of the virtual plane. Next, the computer 5q is caused to perform the repeated determination process of steps S18, S20 and S22.
[0074]
As described above, according to the fifth embodiment of the present invention, the algorithm of the three-dimensional modeling program of the present invention arranges a plurality of PCs in a tree shape and assigns each process to a plurality of PCs. Thus, it can be executed efficiently in a short time.
[0075]
Embodiment 6 FIG.
In the sixth embodiment, a three-dimensional modeling data transmission program that transmits three-dimensional modeling data will be described.
[0076]
As shown in FIG. 16, the final result recording unit 100 records pixel values of rectangular pixels as shown in FIG. 11B described in the above embodiment (pixel center point recording step). Step S54). Therefore, when the 3D modeling data is transmitted to the client recording units 104, 106, 108, 110, etc. via the network 102, the 3D mesh data is not directly transmitted but is recorded in the final result recording unit 100. Transmit the pixel value of a rectangular pixel. On the client side, the above-described triangulation is performed on the received data, and decoding and rendering are possible by back-projecting it into the 3D space. As described above, when 3D mesh data is transmitted, it is not necessary to send information representing the connection of triangles in addition to the vertex coordinates.
[0077]
As described above, according to the sixth embodiment of the present invention, when three-dimensional modeling data is transmitted to the client side via the network 102, the pixel values of the rectangular pixels recorded in the final result recording unit 100 are determined. By transmitting, it is possible to eliminate the need to send information representing the connection of the triangle in addition to the vertex coordinates. As a result, it is possible to provide a 3D modeling data transmission program that does not require a long time for transmission of 3D mesh data.
[0078]
Embodiment 7 FIG.
FIG. 17 is a block diagram showing an internal circuit 120 of the computer 1 or the like that executes the computer program of the present invention for realizing the above-described embodiments. In FIG. 17, the above-described computer program of the present invention is recorded on a recording medium such as a ROM 122, a disk 125a, or a CD-ROM 125n. This computer program is loaded from the recording medium such as the ROM 122, the disk 125a, or the CD-ROM 125n to the RAM 123 via the computer roller 124 and the bus 126. The recording medium such as the disk 125a can correspond to the recording device 10 or the like. The input operation unit 128 is an input device such as a mouse or a numeric keypad for inputting to the computer 1 or the like. The input control unit 127 is connected to the input operation unit 128 and performs input control or the like. The external interface (I / F) unit 129 has an interface function when the computer 5q or the like communicates via the network 102.
[0079]
The object of the present invention can be achieved by the CPU 121 executing the above-described computer program of the present invention in the RAM 123. The computer program can be supplied to the computer CPU 121 in the form of a removable recording medium such as a CD-ROM 125n as described above, and a recording medium such as a CD-ROM 125n on which the computer program is recorded is also the present invention. Will be configured. As a recording medium on which the computer program is recorded, for example, a DVD, an optical disk, a memory card, a memory stick, an MO, an FD, or the like can be used in addition to the recording medium described above.
[0080]
【The invention's effect】
As described above, according to the three-dimensional modeling program of the present invention, input is performed while increasing the resolution of the distance image 30 or the like in stages, and a 3D mesh is configured based on the input distance image 30 or the like. Yes. This makes it possible to realize “progressiveness” in which the resolution of the 3D mesh increases as the algorithm is executed for a longer time. In the three-dimensional modeling program of the present invention, a regular 3D mesh can be directly generated without going through the double procedure of converting a modeled irregular regular 3D mesh into regular. As described above, a 3D mesh having regularity not only for a static object but also for a dynamic object can be modeled in real time, and a three-dimensional modeling program having progressive characteristics can be provided.
[0081]
In the three-dimensional modeling program or the like of the present invention, the algorithm can be aborted halfway. Therefore, it is possible to provide a 3D modeling program or the like that can output a 3D mesh having the best resolution at the time of termination.
[0082]
In the 3D modeling transmission program of the present invention, modeling and transmission in real time enables a remote client to view the 3D mesh from any angle and in real time. Since the data sent to the client side is not a 3D mesh itself but data similar to an image called a pixel value of a rectangular pixel, a compression algorithm for still images or moving images can be directly applied. This not only leads to high compression, but also has an advantage that various scalability such as SNR scalability and resolution scalability can be easily realized. As described above, it is possible to provide a 3D modeling data transmission program that does not require a long time for transmission of 3D mesh data.
[Brief description of the drawings]
FIG. 1 is a diagram showing an environment in which a three-dimensional modeling program or the like according to Embodiment 1 of the present invention operates.
FIG. 2 is a diagram illustrating a distance image.
FIG. 3 is a flowchart showing functions of the three-dimensional modeling program of the present invention.
FIG. 4 is a flowchart showing functions of a three-dimensional mesh modeling step in the three-dimensional modeling program of the present invention.
FIG. 5 is a diagram for explaining surface points;
FIG. 6 is a diagram for explaining a virtual cylinder.
7 is a diagram showing manifolds 40a, 40b, and 40c that are in phase with the virtual cylinder 40. FIG.
FIG. 8 is a diagram for explaining projection of a surface point 28 or the like onto a virtual cylinder 40;
FIG. 9 is a diagram for explaining the projection of a surface point 28 or the like onto a virtual cylinder 40;
10 is a view for explaining the development of the virtual cylinder 40. FIG.
11 is a diagram showing a virtual plane P. FIG.
FIG. 12 is a diagram for explaining a mapping method according to Embodiment 2 of the present invention;
FIG. 13 is a diagram illustrating mapping using a virtual cone in Embodiment 3 of the present invention.
FIG. 14 is a flowchart showing a three-dimensional projection point creation step in the third embodiment of the present invention.
FIG. 15 is a diagram illustrating mapping using a virtual sphere according to the fourth embodiment of the present invention.
FIG. 16 is a diagram showing an environment (system) in which a three-dimensional modeling control program for controlling processing for creating a three-dimensional model of an object according to Embodiment 5 of the present invention is executed.
FIG. 17 is a block diagram showing an internal circuit 120 of the computer 1 or the like that executes the computer program of the present invention.
FIG. 18 is a diagram illustrating a 3D mesh that approximates the surface of an object with vertices and triangles.
FIG. 19 is a diagram illustrating a regular mesh and a non-regular mesh.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Computer, 3,80a, 80b, 80c, 80d, 80m-1,80m Distance image recording part, 4 Initial resolution recording part, 5 Time recording part, 6 Initial resolution recording part, 10 Recording apparatus, 11 Surface point recording part, 12 three-dimensional projection point recording unit, 13 two-dimensional transformation point recording unit, 14 pixel center point recording unit, 15 pixel center three-dimensional inverse transformation point recording unit, 16 pixel center corresponding surface point recording unit, 20 objects, 25a, 25b, 25c, 25d, 25e, 25f, 70a, 70b, 70c, 70d, 70e, 70f Camera (imaging device), 28 Surface points, 30, 31, 32, 33, 34, 35 Distance image, 40 Virtual cylinder, 40a, 40b , 40c A manifold that is in phase with the virtual cylinder, 41 center line, 42 cutting plane, 44a, 44b three-dimensional projection point, 44a (2), 46 (2), 47 (2), 8 (2) 2D conversion point, 45 reference point, 50 virtual cone, 60 virtual sphere, 65 center point, 75, 126 bus, 90a, 90b, 90m / 2 integrated recording unit, 100 final result recording unit, 102 network, 104, 106, 108, 110 Recording unit, 120 Internal circuit, 121 CPU, 122 ROM, 123 RAM, 124 Computer roller, 125a disk, 125n CD-ROM, 127 Input control unit, 128 Input operation unit, 129 External I / F Department.

Claims (13)

A three-dimensional modeling program for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, the distance to the object obtained by each of the plurality of imaging devices being a pixel A distance image recording unit that records a distance image as a value according to each imaging device and an initial resolution recording unit that records a predetermined resolution are used.
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
The distance image of the object is acquired by each of the plurality of imaging devices under a predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determination step, and the acquired time is a time A distance image acquisition step of recording in the recording unit and recording the predetermined resolution in the resolution recording unit;
A distance image recording step of causing the distance image recording unit to record each distance image acquired in the distance image acquisition step;
A three-dimensional mesh modeling step of creating a three-dimensional mesh model of the object based on each distance image recorded in the distance image recording unit under the resolution recorded in the resolution recording unit;
If a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate and repeatedly executed from the three-dimensional mesh modeling step. A three-dimensional modeling program for executing a step of repeatedly executing from the distance image acquiring step when a predetermined time has elapsed from a time recorded in the time recording unit and a predetermined end time has not been exceeded. The three-dimensional modeling program according to claim 1, wherein the three-dimensional mesh modeling step includes:
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of each distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step for recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. A three-dimensional projection point creation step;
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value, and a two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is used as the center point of each pixel obtained in the interpolation distance value creation step. A pixel center point recording step for recording in the pixel center point recording unit together with the interpolation distance value;
Pixel center for obtaining a three-dimensional inverse transformation point of the center point of each pixel by inversely transforming the two-dimensional coordinate value of the center point of each pixel recorded in the pixel center recording unit into a three-dimensional coordinate value on the virtual solid A three-dimensional inverse transformation point creation step;
The three-dimensional coordinate value of the three-dimensional inverse transform point of the center point of each pixel obtained in the pixel center three-dimensional inverse transform point creation step is used as the interpolation distance of the center point of each pixel recorded in the pixel center point recording unit. A pixel center 3D inverse transform point recording step for recording in the pixel center 3D inverse transform point recording unit together with the value;
3 of each surface point corresponding to the center point of each pixel obtained by back-projecting the interpolation distance value of the center point of each pixel recorded in the pixel center three-dimensional inverse transformation point recording unit in the virtual three-dimensional space. A step of creating a surface point corresponding to a pixel center for obtaining a dimensional coordinate value;
A pixel center corresponding surface point recording step for recording a three-dimensional coordinate value of each surface point corresponding to the center point of each pixel obtained in the pixel center corresponding surface point creating step in a pixel center corresponding surface point recording unit; A three-dimensional modeling program characterized by this. 3. The three-dimensional modeling program according to claim 2, wherein the three-dimensional projection point recording step is performed when the resolution recorded in the resolution recording unit is equal to a predetermined resolution recorded in the initial resolution recording unit. The 3D projection vector to each 3D projection point obtained in the projection point creation step is set to 3 for each 3D projection point together with an attribute value indicating the 3D coordinate value of the surface point corresponding to the 3D projection point. Recorded in the dimensional projection point recording section,
In the three-dimensional projection point creation step, when the resolution recorded in the resolution recording unit is larger than a predetermined resolution recorded in the initial resolution recording unit,
With respect to the previous surface point having the three-dimensional coordinate value already recorded in the surface point recording unit, the previous three-dimensional projection point corresponding to the previous surface point is already recorded in the three-dimensional projection point recording unit. A new surface point having a three-dimensional coordinate value newly recorded in the surface point recording unit by increasing the resolution using the previous three-dimensional projection vector corresponds to the previous surface point in the vicinity of the new surface point. A three-dimensional modeling program characterized in that a new three-dimensional projection vector is obtained based on the attribute value of the conventional three-dimensional projection vector. 4. The three-dimensional modeling program according to claim 2, wherein the three-dimensional projection point creation step acquires a distance image including pixels corresponding to surface points when the virtual solid is a virtual cylinder or a virtual cone. A spatial position information section in which the spatial position information between the object and the object is recorded;
A virtual solid creation step of creating a center line of a virtual cylinder or virtual cone in a virtual space;
A plane that cuts the virtual cylinder or virtual cone that is perpendicular to the center line created in the virtual solid creation step and passes through the surface point recorded in the surface point recording unit, and the periphery is the circumference of the virtual cylinder or virtual cone A cutting plane creation step to create a cutting plane that matches
A center point coordinate value creating step for obtaining a coordinate value of the center point of the cutting plane created in the cutting plane creating step;
Find two points where a line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit intersects the periphery of the cutting plane 2 A point creation step;
Virtual plane creation that creates a virtual plane corresponding to a distance image including a pixel corresponding to the surface point recorded in the surface point recording unit in a virtual space based on the spatial position information recorded in the spatial position information unit Steps,
A pixel coordinate value creating step for obtaining a coordinate value of a pixel corresponding to the surface point in the virtual plane created in the virtual plane creating step;
A pixel vector that goes from the coordinate value of the pixel obtained in the pixel coordinate value creation step to the coordinate value of the center point obtained in the center point coordinate value creation step, and the surface point recorded in the surface point recording unit A vector creation step for obtaining two surface point vectors from the coordinate values to the two points obtained in the two-point creation step;
Comparing the angle between the pixel vector obtained in the vector creation step and each of the two surface point vectors, and setting the surface point vector corresponding to the small angle as a three-dimensional projection vector. Characteristic 3D modeling program. The three-dimensional modeling program according to claim 2 or 3, wherein the three-dimensional projection point creating step includes the step of:
A center point coordinate value creation step for obtaining a coordinate value of the center point of the virtual sphere;
On the line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit, from the coordinate value of the center point to the surface point A three-dimensional modeling program comprising a step of setting a vector in a direction toward the three-dimensional projection vector. A three-dimensional modeling control program for controlling processing for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, up to the object acquired by each of the plurality of imaging devices A distance image recording unit that records a distance image having a distance of 2 as a pixel value in accordance with each imaging device, and an initial resolution recording unit that records a predetermined resolution. A plurality of computers corresponding to each distance image recording unit for creating a three-dimensional model, and an integrated computer for integrating the processing results of the plurality of computers,
To a computer that executes a 3D modeling control program,
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
In each of the plurality of computers,
The distance image of the object is acquired by the imaging device under the predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determining step, and the acquired time is stored in the time recording unit. A distance image acquisition step of recording and recording the predetermined resolution in a resolution recording unit;
A distance image recording step of recording the distance image acquired in the distance image acquisition step in the distance image recording unit;
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of the distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step of recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. 3D projection point creation step,
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
In the integrated computer,
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value, and a two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is interpolated with the center point of each pixel obtained in the interpolation distance value creation step. A pixel center point recording step for recording in the pixel center point recording unit together with the distance value;
Pixel center for obtaining a three-dimensional inverse transformation point of the center point of each pixel by inversely transforming the two-dimensional coordinate value of the center point of each pixel recorded in the pixel center recording unit into a three-dimensional coordinate value on the virtual solid 3D inverse transformation point creation step,
The three-dimensional coordinate value of the three-dimensional inverse transform point of the center point of each pixel obtained in the pixel center three-dimensional inverse transform point creation step is used as the interpolation distance of the center point of each pixel recorded in the pixel center point recording unit. A pixel center three-dimensional inverse transform point recording step for recording in the pixel center three-dimensional inverse transform point recording unit together with the value;
3 of each surface point corresponding to the center point of each pixel obtained by back-projecting the interpolation distance value of the center point of each pixel recorded in the pixel center three-dimensional inverse transformation point recording unit in the virtual three-dimensional space. A step for creating a surface point corresponding to a pixel center for obtaining a dimensional coordinate value;
Performing a pixel center corresponding surface point recording step of recording a three-dimensional coordinate value of each surface point corresponding to the center point of each pixel obtained in the pixel center corresponding surface point creating step in a pixel center corresponding surface point recording unit;
When a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate, and the surface point creating step is performed in each of the plurality of computers. When the predetermined time has elapsed from the time recorded in the time recording unit and the predetermined end time has not been exceeded, each of the plurality of computers is repeatedly executed from the distance image acquisition step. A three-dimensional modeling control program for executing steps. A three-dimensional modeling data transmission program for transmitting three-dimensional modeling data of the object created using a plurality of imaging devices arranged around the object, each acquired by the plurality of imaging devices, A distance image recording unit that records a distance image having a distance to the object as a pixel value according to each imaging device and an initial resolution recording unit that records a predetermined resolution are used.
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
The distance image of the object is acquired by each of the plurality of imaging devices under a predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determination step, and the acquired time is a time A distance image acquisition step of recording in the recording unit and recording the predetermined resolution in the resolution recording unit;
A distance image recording step of causing the distance image recording unit to record each distance image acquired in the distance image acquisition step;
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of each distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step of recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. 3D projection point creation step,
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
Two-dimensional conversion to a two-dimensional conversion point by converting a three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value;
The two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is recorded in the pixel center point recording unit together with the interpolation distance value of the center point of each pixel obtained in the interpolation distance value creation step. Pixel center point recording step,
When a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate and repeatedly executed from the surface point creating step, and the time When a predetermined time has elapsed since the time recorded in the recording unit and the predetermined end time has not been exceeded, the distance image obtaining step is repeatedly executed, and the predetermined time from the time recorded in the time recording unit is set. 3D modeling including a two-dimensional coordinate value of the center point of each pixel recorded in the pixel center point recording unit and an interpolation distance value of the center point of each pixel when a predetermined end time has passed. A three-dimensional modeling data transmission program for executing a step of transmitting data. A computer-readable recording medium on which the program according to claim 1 is recorded. A three-dimensional modeling method for creating a three-dimensional model of an object using a plurality of imaging devices arranged around the object, the distance to the object obtained by each of the plurality of imaging devices being a pixel value Using a distance image recording unit that records a distance image according to each imaging device, and an initial resolution recording unit that records a predetermined resolution.
A corresponding point determination step for determining a corresponding point of each distance image recorded in the distance image recording unit;
The distance image of the object is acquired by each of the plurality of imaging devices under a predetermined resolution recorded in the initial resolution recording unit and the corresponding point determined in the corresponding point determination step, and the acquired time is a time A distance image acquisition step of recording in the recording unit and recording the predetermined resolution in the resolution recording unit;
A distance image recording step of recording each distance image acquired in the distance image acquisition step in the distance image recording unit;
A three-dimensional mesh modeling step of creating a three-dimensional mesh model of the object based on each distance image recorded in the distance image recording unit under the resolution recorded in the resolution recording unit;
If a predetermined time has not elapsed since the time recorded in the time recording unit, the resolution recorded in the resolution recording unit is increased at a predetermined rate and repeatedly executed from the three-dimensional mesh modeling step. A step of repeatedly executing from the distance image acquiring step when a predetermined time has elapsed from a time recorded in the time recording unit and a predetermined end time has not been exceeded. Modeling method. The three-dimensional modeling method according to claim 9, wherein the three-dimensional mesh modeling step includes:
A surface point creating step for obtaining a three-dimensional coordinate value of each surface point obtained by back-projecting each pixel value of each distance image recorded in the distance image recording unit in a virtual three-dimensional space;
A surface point recording step for recording a three-dimensional coordinate value of each surface point obtained in the surface point creation step in a surface point recording unit;
A three-dimensional projection vector to each three-dimensional projection point obtained by projecting from each surface point having a three-dimensional coordinate value recorded in the surface point recording unit onto a virtual solid formed in a virtual three-dimensional space is obtained. A three-dimensional projection point creation step;
The three-dimensional projection vector for each three-dimensional projection point obtained in the three-dimensional projection point creation step is calculated for each three-dimensional projection point by a three-dimensional distance value between the three-dimensional projection point and the corresponding surface point. A three-dimensional projection point recording step for recording in the three-dimensional projection point recording unit together with an attribute value indicating
Two-dimensional conversion to a two-dimensional conversion point by converting the three-dimensional projection vector for each three-dimensional projection point recorded in the three-dimensional projection point recording unit onto a virtual plane developed by cutting the virtual solid A step of creating a two-dimensional transformation point obtained as a vector;
The two-dimensional conversion point to each two-dimensional conversion point obtained in the two-dimensional conversion point creation step is converted into a two-dimensional conversion point together with the attribute value recorded in the three-dimensional projection point recording unit for each two-dimensional conversion point. A two-dimensional conversion point recording step for recording in the recording unit;
A pixel center point creating step of setting a pixel of a predetermined size on the virtual plane and obtaining a two-dimensional coordinate value of the center point of each pixel;
Based on the two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step and the two-dimensional conversion vector of the two-dimensional conversion point recorded in the two-dimensional conversion point recording unit, a predetermined two-dimensional A predetermined number of two-dimensional conversion points within a range of distance values are obtained, and the center point of each pixel is determined based on the attribute value recorded in the two-dimensional conversion point recording unit for the predetermined number of two-dimensional conversion points. An interpolation distance value creation step for obtaining an interpolation distance value;
The two-dimensional coordinate value of the center point of each pixel obtained in the pixel center point creation step is recorded in the pixel center point recording unit together with the interpolation distance value of the center point of each pixel obtained in the interpolation distance value creation step. A pixel center point recording step;
Pixel center for obtaining a three-dimensional inverse transformation point of the center point of each pixel by inversely transforming the two-dimensional coordinate value of the center point of each pixel recorded in the pixel center recording unit into a three-dimensional coordinate value on the virtual solid A three-dimensional inverse transformation point creation step;
The three-dimensional coordinate value of the three-dimensional inverse transform point of the center point of each pixel obtained in the pixel center three-dimensional inverse transform point creation step is used as the interpolation distance of the center point of each pixel recorded in the pixel center point recording unit. A pixel center 3D inverse transform point recording step for recording in the pixel center 3D inverse transform point recording unit together with the value;
3 of each surface point corresponding to the center point of each pixel obtained by back-projecting the interpolation distance value of the center point of each pixel recorded in the pixel center three-dimensional inverse transformation point recording unit in the virtual three-dimensional space. A step of creating a surface point corresponding to a pixel center for obtaining a dimensional coordinate value;
A pixel center corresponding surface point recording step for recording a three-dimensional coordinate value of each surface point corresponding to the center point of each pixel obtained in the pixel center corresponding surface point creating step in a pixel center corresponding surface point recording unit; A three-dimensional modeling method characterized by this. 11. The three-dimensional modeling method according to claim 10, wherein the three-dimensional projection point recording step includes the step of recording the three-dimensional projection when the resolution recorded in the resolution recording unit is equal to a predetermined resolution recorded in the initial resolution recording unit. The three-dimensional projection vector for each three-dimensional projection point obtained in the point creation step is displayed for each three-dimensional projection point together with an attribute value indicating the three-dimensional coordinate value of the surface point corresponding to the three-dimensional projection point. Record in the projection point recording section,
In the three-dimensional projection point creation step, when the resolution recorded in the resolution recording unit is larger than a predetermined resolution recorded in the initial resolution recording unit,
With respect to the previous surface point having the three-dimensional coordinate value already recorded in the surface point recording unit, the previous three-dimensional projection point corresponding to the previous surface point is already recorded in the three-dimensional projection point recording unit. A new surface point having a three-dimensional coordinate value newly recorded in the surface point recording unit by increasing the resolution using the previous three-dimensional projection vector corresponds to the previous surface point in the vicinity of the new surface point. A three-dimensional modeling method, wherein a new three-dimensional projection vector is obtained based on the attribute value of the conventional three-dimensional projection vector. 12. The three-dimensional modeling method according to claim 10 or 11, wherein the three-dimensional projection point creation step includes an imaging device that acquires a distance image including pixels corresponding to surface points when the virtual solid is a virtual cylinder or a virtual cone. A spatial position information section in which spatial position information between the object and the object is recorded;
A virtual solid creation step of creating a center line of a virtual cylinder or virtual cone in a virtual space;
A plane that cuts the virtual cylinder or virtual cone that is perpendicular to the center line created in the virtual solid creation step and passes through the surface point recorded in the surface point recording unit, and the periphery is the circumference of the virtual cylinder or virtual cone A cutting plane creation step to create a cutting plane that matches
A center point coordinate value creating step for obtaining a coordinate value of the center point of the cutting plane created in the cutting plane creating step;
Find two points where a line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit intersects the periphery of the cutting plane 2 A point creation step;
Virtual plane creation that creates a virtual plane corresponding to a distance image including a pixel corresponding to the surface point recorded in the surface point recording unit in a virtual space based on the spatial position information recorded in the spatial position information unit Steps,
A pixel coordinate value creating step for obtaining a coordinate value of a pixel corresponding to the surface point in the virtual plane created in the virtual plane creating step;
A pixel vector that goes from the coordinate value of the pixel obtained in the pixel coordinate value creation step to the coordinate value of the center point obtained in the center point coordinate value creation step, and the surface point recorded in the surface point recording unit A vector creation step for obtaining two surface point vectors from the coordinate values to the two points obtained in the two-point creation step;
Comparing the angle between the pixel vector obtained in the vector creation step and each of the two surface point vectors, and setting the surface point vector corresponding to the small angle as a three-dimensional projection vector. Characteristic 3D modeling method. The three-dimensional modeling method according to claim 10 or 11, wherein the three-dimensional projection point creation step includes the step of:
A center point coordinate value creation step for obtaining a coordinate value of the center point of the virtual sphere;
On the line segment connecting the coordinate value of the center point obtained in the center point coordinate value creating step and the coordinate value of the surface point recorded in the surface point recording unit, the surface point is calculated from the coordinate value of the center point. A three-dimensional modeling method comprising: a step of setting a vector in a direction toward the three-dimensional projection vector.

Images