JP3960536B2 - Computer-implemented method and computer-executable program for automatically adapting a parametric dynamic model to human actor size for motion capture - Google Patents

Description

このような関節に対する、角度θ_iの動きによって引き起こされる身体部品の反映は、以下のエクスポーネンシャルマップ［７］を用いて計算される。
【００４５】
【数２】

ここでＲ（θ_i, n _i ）は単位軸n_iの周りのθ_iだけの回転を記述する直交行列である。
【００４６】
人間のモーションキャプチャデータからヒューマノイドロボットの動きを生成するために、人間の関節の動きを３−Ｄのマーカの動きと関連させる必要がある。マーカが付される剛体である身体部品のローカルな座標系におけるマーカの座標がy _i で与えられるとすれば、身体構成（r, d, θi１…, θin）でのその３−Ｄ位置は以下のように計算される。
【００４７】
【数３】

ここでベクトルξ _{i １}…，ξ _inはマーカの動きy_ｊを生じさせる動力学的連鎖を記述するひねりであり、Ｇ_ｘはマーカが付されるローカル身体部品座標系の位置および向きを、基本構成でのグローバル身体座標系に対して関連付ける同次行列である。（図１に示したヒューマノイドロボットの場合、ｘ＝上体、骨盤、頭、左上腕、左下腕、左手、右上腕、右下腕、右手、左上脚、左下脚、左足、右上脚、右下脚、右足である。）ｒおよびｄは世界座標に対するグローバル身体座標系の回転ベクトルおよび位置によって表される向き、ｇ（ｒ，ｄ）はｒおよびｄに対応する同次行列である。なお、マーカの動きに影響を与えるひねりの組は、マーカが付される身体部品が何であるかによって変わる。
【００４８】
このような設定では、動力学的モデリングは関節軸の位置と、ローカル身体部品座標系の方向と位置とを特定することを含み、これらパラメータはすべて、身体が基本構成にある状態でのグローバル身体座標系で特定される。ローカル身体部品座標系とは別に、空間における身体の位置および向きを特定するグローバル身体座標系も固定する必要がある。
【００４９】
［自動モデル生成］
この実施例のアプローチの基本は、人間の身体の動力学とヒューマノイドロボットまたはＣＧキャラクタの動力学との間の関係を確立することである。これは人間の行為者の動力学を、ヒューマノイドエージェント用のモデル標準でありかつ人間の行為者の物理的サイズに拡大または縮小したものでモデル化することによって達成される。ヒューマノイドエージェントの動力学は人間の身体の動力学と類似するので、広範囲の人間の動きにこの方法で対処することができる。
【００５０】
この目的で、ヒューマノイドロボットの典型的な動力学的構造と人間の身体の動力学とを考察してみる。バイオメカニクスおよびコンピュータグラフィックスでは、首の関節、肩関節、股関節および踝の関節を通常は球面関節としてモデル化する。本件のヒューマノイドロボットではこれは当てはまらない。なぜなら、ヒューマノイドロボットのこれらの場所には、直交するが必ずしも交差しないいくつかの（典型的には３個の）連続した回転関節を有するためである。加えて、対応する関節軸の方向は基本構成においては、通常、身体の三つの主軸（上下、左右および前後）と一致する。被験者またはヒューマノイドエージェントが手足を伸ばして直立姿勢にあるとき身体は基本構成にあると考える。３個の連続した回転関節は球面関節と同様に任意の回転を生じさせることができ、これにより、二つの動力学的構造がある程度まで等価となる。従って、３自由度（ＤＯＦ）の球面関節を、身体の主軸に沿った方向を向いた軸を持つ連続した３個の回転関節によってモデル化することには意味がある。軸の順序と符号とは、考察されるヒューマノイドエージェントの関節軸の順序および方向と同じになるよう選択しなければならない。
【００５１】
どのような動力学的パラメータシステムを採用するかにかかわらず、パラメータ（関節角度）の実際の値はどのようなローカル座標系を選択するかに依存する。なぜなら、これらの値の間の変換を特定するのがこれらローカル座標系だからである。身体の姿勢がどのような場合でも、関節角度の値が、その姿勢に対応する姿勢をとっているヒューマノイドエージェントの関節角度の値と同じになるようローカル座標系を選択することが必須である。さもなければ測定された人間の関節角度をヒューマノイドエージェントの関節にマッピングすることはできない。従って、ここでは人間の行為者に対するグローバルおよびローカル身体部品座標系を、行為者が手足を伸ばして直立した姿勢にあるときそれらがすべて整列するような方向とした。この構成では、それらの軸は身体の主軸と平行になるように選択された。こうすれば、基本構成におけるグローバル座標系の関節軸方向は座標軸と一致する。
【００５２】
［座標系と身体上のマーカ位置の実際の決定］
マーカの動きを、観察対象の行為者の関節の動きと関連付けられるようにするには、ローカル身体部品座標系での身体のマーカ位置を知らなければならない。このデータは、行為者がその関節角度が既知である直立姿勢をとっている間にマーカの位置を測定することで得られる。この目的には基本構成が好適である。実験では、行為者にこのような姿勢をとってもらい、マーカを記録した。他のロボットでは異なる基本構成が必要となる場合もありうるだろう。各身体部品につき、そこに付されるマーカすべての重心にローカル座標系の原点がくるようにした。
【００５３】
身体の主軸の方向を決定するために、光学トラッキングシステムを較正してどこが上、下であるかがわかるようにした。従って、基本構成における身体軸の一つを（この実施例ではＺ軸）世界座標系のＺ軸と平行に設定することができる。左肩から右肩までの軸は、左肩および右肩の対向する位置に２個のマーカを付すことにより、またはデータから肩関節の場所を推定することにより、推定できる。これは［５］、［６］に記載の方法の一つを用いて行なうことができる。いずれの場合も二つの対向する位置間の線が第二の身体軸（このシステムではｙ軸）を規定する。測定および/またはマーカ設置の不正確さのため、導出された軸は正確には直交しない。このため、新たなＺ軸方向を再計算することにより、強制的にこれらを直交させるようにする。
【００５４】
【数４】

ここでｙ_lおよびｙ_rはそれぞれ左肩および右肩に（または左肩関節および右肩関節の場所に）付された特別のマーカの座標である。この新たな方向はｙ軸と直交し、ｙ軸に直交するすべての方向のうち元のｚ軸からもっとも偏位が少ないものである。最後に、ｙ軸およびｚ軸方向のベクトルのクロス乗積を用いて、ｘ軸方向が計算される。
【００５５】
グローバル身体座標系の原点は、二つの肩関節（または二つの特別に位置づけられた肩マーカ）を結ぶ線の中間にあると仮定される。これは上体に固定される。基本構成では、グローバル身体座標系とすべてのローカル身体部品座標系とが整列している。この仮定を用いて、グローバル身体座標系（式（３）のＧ_x）における基本構成での各身体部品の位置と向きを計算し、マーカの世界座標系をローカル座標系（式（３）の〜y _ｊ および_yｊ ）に変換することができる。
【００５６】
関節角度はゼロに等しいので式（３）を以下のように簡略化することができる。
【００５７】
【数５】

グローバル座標系の位置は以下で与えられる。
【００５８】
【数６】

一方、向きはその列に３個の座標軸を有する回転行列Ｒによって与えられる。
【００５９】
【数７】

基本構成での各ローカル身体部品座標系の向きはグローバル身体座標系の向きと同じである。従って、相対的な向きは恒等行列で与えられる。相対的な位置はグローバル身体座標系の位置を減算することによって計算できる。従って、身体部品ｘについて、以下が成立する。
【００６０】
【数８】

ここで、ｎ_ｘは身体部品に付けられたマーカの数である。最後に、マーカｙ _j のローカル座標が数式（５）の逆をとることにより得られる。
【００６１】
［関節位置の推定］
この時点で、依然として推定が必要なパラメータは関節軸の位置のみである。我々の目標は、身体上の関節軸の位置を、身体を中心とした座標系で推定することである。これらのパラメータを推定するために、被験者にある一組の動きをしてもらい、これをモーションキャプチャシステムで測定する。
【００６２】
我々は、一つの大きな最適化プロセスですべての関節場所を推定しようとするのではなく、推定を１０個のより小さい最適化問題に分割することとした。すなわち、首、腰、左右の肩＋肘、左右の手首、左右の臀部＋膝、および左右の踵である。
【００６３】
３−Ｄ空間における関節軸の位置の自由度は２つのみである、なぜなら、軸に沿った位置は任意だからである。基本構成におけるすべての関節軸の方向は、グローバル身体座標系の座標軸と平行であり、従ってすべてベクトル［１,０,０］^T、［０,１,０］^T、［０,０,１］^T、［-１,０,０］^T、［０,-１,０］^T、および［０,０,-１］^Tの中の一つにより与えられる。注意すべきことは、関節軸の順序を、検討中のヒューマノイドロボットのそれと同じに保つことである。関節軸方向のベクトルの非―ゼロ座標に対応する座標が数式（１）で与えられる結果として生じるひねりに影響を与えないことが容易に理解できる。たとえば、［a,b,c］^Tにあり［０,１,０］^Tと平行な関節軸について、対応するひねりは以下に等しい。
【００６４】
【数９】

従ってこれは第二の座標、すなわちｂから独立している。このことから、各関節軸の場所を関節軸の方向ベクトルの非―ゼロ座標とは異なる二つの座標でパラメータ化することが可能である。
【００６５】
関節軸の場所について独立した座標を得たので、好適な最適化指標を最小にすることでこれらを推定することができる。関節軸の場所とは別に、モデルマーカが測定されたマーカの位置と一致するように、空間内での身体の位置と向きおよび関節角度も推定する必要がある。最適化のプロセスをより小さくするために、検討すべき関節に先立つすべての自由度を、別個の最適化プロセスで推定する。この場合最適化プロセスに含まれるのは推定する必要のある関節角度の位置と、対応する関節角度のみである。それでも、結果として非常に大きな最適化問題が残る。たとえば、首関節の場所を推定するためには、以下の最適化基準を最小化する必要がある。
【００６６】
【数１０】

パラメータの数は測定時間の数とともに増加することに注意する必要がある。実験では、典型的には１０個の最適化問題の各々について３００回の測定を行なった。各関節軸の位置は２個のパラメータでパラメータ化できる。各関節軸について２個のパラメータがあり、従って３本の関節軸の場合には６個のパラメータ（首、腰、手および踵の場合）となり、４本の関節軸の場合には８個のパラメータ（肩＋肘、臀部＋膝の場合）となる。これらパラメータは基本構成で与えられ、変化しない。
【００６７】
さらに、測定されたマーカの位置とこれらのパラメータとを関連付けるために、これらの軸に対する関節角度を推定する必要がある。関節軸の位置と異なり、関節角度は測定の組ごとに変化する。従って、３００回の測定をする場合、最適化問題ごとに３×（３００+２）＝９０６個のパラメータ、または４×（３００＋２）＝１２０８個のパラメータを推定する必要がある。
【００６８】
このような最適化問題を解くために、サブ空間信頼領域法(subspace trust region approach)および疎行列代数［８］を利用した。信頼領域法は指標関数ｈの第一導関数の計算を必要とする。これらの導関数の計算は関節角度θ１(t_k)、θ２(t_k)、θ３(t_k)、ｋ＝１、…Ｎおよび関節位置パラメータａ、ｂ、ｃ、ｄ、ｅ、ｆに関するフォワード動力学関数（３）のヤコビアンの計算に基づいている。関節角度に関するヤコビアンの計算は周知の問題であり、これを［７］に記載の方法を用いて実現した。最適化基準（９）のヤコビアンは疎であり、計算時間を減じるためここでは最適化方法に疎行列代数を用いた。この実施例で用いた関数およびツールはＭＡＴＬＡＢ（ＴＭ）等の最適化ツールキットで利用可能である。
【００６９】
［好ましい実施例］
図３はこの実施例の全体図を示す。図３を参照して、システムは人間の行為者１８のさまざまな身体部品に付されたマーカの３−Ｄ位置を測定するための光学トラッカ３０と、測定データの演算を行なってモーションキャプチャのためにパラメトリックな動力学モデルを人間の行為者のサイズに適応させるためにパラメータを最適化するコンピュータシステム２０とを含む。コンピュータシステム２０によってこうして計算された適応パラメータ８０を、光学トラッカ３０がキャプチャした同じ行為者１８の動きデータに基づきロボットを制御するためのデータを生成する動き生成システム８２で用いることができる。
【００７０】
図４はコンピュータシステム２０を示し、図５はシステム２０をブロック図で示す。図４を参照して、コンピュータシステム２０は、ＦＤ（フレキシブルディスク）ドライブ５２およびＣＤ−ＲＯＭ（コンパクトディスク読出専用メモリ）ドライブ５０を有するコンピュータ４０と、キーボード４６と、マウス４８と、モニタ４２とを含む。光学トラッカ３０はコンピュータ４０に接続される。
【００７１】
図５を参照して、コンピュータ４０は、ＦＤドライブ５２およびＣＤ−ＲＯＭドライブ５０に加えて、ＣＰＵ（中央処理装置）５６と、ＣＰＵ５６、ＦＤドライブ５２およびＣＤ−ＲＯＭドライブ５０に接続されたバス６６と、ブートアッププログラム等を記憶する読出専用メモリ（ＲＯＭ）５８と、ＣＰＵ５６に接続され、プログラム命令、システムプログラム、およびデータを記憶するランダムアクセスメモリ（ＲＡＭ）６０とを含む。
【００７２】
ここでは示さないが、コンピュータ４０はさらにローカルエリアネットワーク（ＬＡＮ）への接続を提供するネットワークアダプタボードを含んでもよい。ヒューマノイドロボットの場合、システムはさらに、カメラとロボットの種々の部品をコンピュータシステム２０の制御に従って動かすための多数のアクチュエータを含む。
【００７３】
コンピュータシステム２０にパラメトリックな動力学モデルを人間の行為者のサイズに自動的に適応させる方法を行なわせるプログラムは、ＣＤ−ＲＯＭドライブ５０またはＦＤドライブ５２に挿入されるＣＤ−ＲＯＭ６２またはＦＤ６４に記憶されさらにハードディスク５４に転送される。これに代えて、プログラムは図示しないネットワークを通じてコンピュータ４０に送信されハードディスク５４に記憶されてもよい。プログラムは実行の際にＲＡＭ６０にロードされる。プログラムはＣＤ−ＲＯＭ６２、ＦＤ６４、またはネットワークを介してＲＡＭ６０に直接ロードされてもよい。
【００７４】
以下に説明するプログラムは、コンピュータ４０にこの実施例の方法を行なわせるいくつかの命令を含む。この方法を行なわせるのに必要な基本的機能のいくつかはコンピュータ４０のオペレーティングシステム（ＯＳ）またはサードパーティのプログラム、もしくはコンピュータ４０にインストールされる最適化ツールボックス等のモジュールにより提供されるので、このプログラムはこの実施例の方法を実現するのに必要な機能全てを必ずしも含まなくてよい。プログラムが含む必要があるのは、命令のうち、所望の結果が得られるように制御された態様で適切な機能または「ツール」を呼出すことによって最適化プロセスを行なう部分のみである。コンピュータシステム２０の動作は周知であるので、ここでは繰り返さない。
【００７５】
図６はこの実施例の方法を実現するプログラムのための全体的な制御構造を示す。図６を参照して、人間の行為者の身体にマーカが付けられた後（９０）、プログラムが開始する。肩の離れた位置に２個のマーカが位置付けられるので、基本構成においてこれら２個のマーカを結ぶ線によってグローバル身体座標系の水平軸が規定できる。まず始めに、コンピュータ４０は行為者が基本構成の姿勢で立っている間にモーションキャプチャデータ９２からマーカの３−Ｄデータをキャプチャする（９４）。
【００７６】
その後、ローカル身体部品座標系におけるマーカの位置と、グローバルフレームに対するローカルフレームの位置とが計算される（９６）。なおこれに先立って、上述のとおりグローバル身体座標系が計算される。
【００７７】
次に、行為者は一組の動きをするよう求められ、これが光学トラッカ３０によって測定されて、キャプチャされたデータ１１０がコンピュータシステムに送られる（９８）。この方法があいまいさのない回答を返す必要があるのであれば、行為者は適切な自由度全てについて動きを行なわなければならない。
【００７８】
ステップ１００で、コンピュータは、キャプチャされたデータ１１０とパラメトリックな動力学モデル１１２とに基づき大規模な最適化を用いて身体上の関節軸の位置を推定する。このステップの詳細は後に図７を参照して説明する。ステップ１００の出力は、モーションキャプチャのためにパラメトリックな動力学モデルを人間の行為者のサイズに適応させる一組のパラメータである（１０２）。このステップでは、上述のとおり、推定プロセスが１０個の別個のより小さい最適化問題に分割されている。
【００７９】
図７はステップ１００で行なわれるパラメトリックモデルを拡大または縮小するのに必要な最適化プロセスを示す。図７を参照して、モデル内のすべての動力学的連鎖について、以下のステップ１２２から１２６を繰り返す。なお、各鎖は、先行する関節複合体の動きが後続の関節複合体の位置に影響を及ぼすような形で動力学的構造内に連続して配置された一連の関節複合体を含んでいる。通常、動力学的連鎖は、上体、頭、左腕、右腕、左脚、右脚の自由度（ＤＯＦ）またはそのサブセットからなる。関節複合体は首のＤＯＦ、左右の肩＋肘のＤＯＦ、左右の手首のＤＯＦ、腰のＤＯＦ、左右の臀部＋膝のＤＯＦ、左右の踵のＤＯＦ、またはそれらのサブセットからなる。
【００８０】
ステップ１２２で、現在の連鎖内のすべての関節複合体がシーケンシャルな順序で分析され、各関節複合体についてステップ１２４および１２６が繰り返される。
【００８１】
ステップ１２４で、モーションキャプチャデータ１１０のすべての測定時について、身体の姿勢（位置および向き）と、現在の関節複合体に先行する自由度とが、非線形最適化を用いて推定される。身体の姿勢と現在の関節複合体に先行する自由度との推定のためのパラメータは各測定時に互いに独立して推定できるので、ここでの他の最適化プロセスは大規模なものではない。
【００８２】
式（１）から（３）で理解されるように、マーカの３−Ｄ位置は、現在の関節複合体に先行する関節複合体の関節角度、現在の関節複合体の関節角度、基本構成でのマーカの位置、および基本構成における現在と先行する関節軸のｑ _ｉ （軸上のいずれかの点）の関数である。これらのうち、基本構成におけるマーカの位置は既知であり、先行する関節軸のｑ _ｉ も、また、先の繰返しにより既知である。これらのパラメータは一定であるとみなすことができる。各測定データはマーカの３−Ｄ位置を含み、その各々は従って、身体の姿勢および先行する関節軸の関節角度の関数となるはずである。各測定時で最適化することにより、その測定に関し先行する関節軸に対する身体の姿勢および関節感度がステップ１２４で決定できる。
【００８３】
次に、ステップ１２６で、身体上の現在の関節軸の位置と関連の角度との、大規模な非線形最適化を用いた推定が行なわれる。ここで、ステップ１２４と同様に、マーカの３−Ｄ位置は、身体の姿勢、現在の関節複合体に先行する関節複合体の関節角度、現在の関節複合体の関節角度、基本構成でのマーカの位置、および基本構成における現在と先行する関節軸のｑ _ｉ （軸上の任意の点）の関数である。これらのうち、基本構成におけるマーカの位置は既知であり、先行する関節軸のｑ _ｉ も既知である。さらに、身体の姿勢および先行する関節軸に関する関節角度はステップ１２４で決定されている。これらのパラメータは一定であるとみなすことができる。そこで、すべてのモーションキャプチャデータ１１０に対し大規模な最適化が行なわれる。この最適化により、身体上の関節軸の位置と、現在の関節複合体についての関連する角度が推定される。この最適化は依然として大規模なものであるが、これは一時にすべての測定データについてなされる最適化に比べればはるかに小さく、必要とされる時間も短い。
【００８４】
上述のプロセスを、基本構成における右腕の連鎖を示す図８を参照してさらに説明する。２個の特別のマーカ１４２および１４４が、左右の肩の特定の場所にそれぞれ注意深く付けられる。始めに、光学トラッキングシステムを較正して、ｚ軸が上下方向にくるようにする。ｙ軸の方向１５０はマーカ１４２と１４４とを結ぶことで規定される。ｚ軸の方向は、ｚ軸が線１５０の中心で線１５０と交わり、かつ上下方向からの離間量が最小となるように再計算される。従って、グローバル身体座標系の原点は、ｙ軸とｚ軸との交点で規定される。ｘ軸方向はｙ軸およびｚ軸における単位ベクトルのクロス乗積によって計算される。
【００８５】
基本構成における関節軸の方向は、グローバル身体座標系の方向の一つと平行になるように選択される。
【００８６】
図８に示す関節４、６、８、１０および１２の位置は以下のように推定される。まず始めに、行為者は、関節４、６および８のすべてのＤＯＦを発揮するような一組の動きをするよう求められる。関節４、６および８の位置は、モーションキャプチャデータの最適化により、図７のステップ１２４および１２６のプロセスを行なうことで推定される。ここで関節角度も推定されるが、これらは破棄される。
【００８７】
次に、２度めの繰返しで、ステップ１２４において空間における身体の位置および向きと、関節４、６、８のＤＯＦの関節角度が、非線形の小規模最適化を用いて推定される。その後、ステップ１２６で、関節軸１０および１２の位置とその関連の角度が、大規模な非線形最適化を用いて推定される。
【００８８】
パラメトリック動力学的連鎖のすべての鎖について、同様のプロセスが行なわれる。
【００８９】
［技術的問題］
指標関数（９）では、複数自由度の関節では関節軸が互いに交差することは仮定していない。この最適化指標関数にそのような仮定を組入れることは容易である。この場合、式（９）の６個のパラメータはただ３個に減る。というのも、それらの各々が３個のひねりのパラメータ化において２度ずつ現れるからである。関節軸の場所のうちいくつかを、たとえば左右の肩の軸の場所を同時に推定して、最適化指標に対称性という制約を課してもよい。
【００９０】
モーションキャプチャのセッションの最終目的がコンピュータグラフィックのアニメーションを生成することであれば、このセクションで説明されるステップのいくつかは不要である。また、ヒューマノイドロボットの動力学に課されるいくつかの制約を緩和することもできる。たとえば、ロボットの関節軸が身体の主軸と平行であること、またはそれらが直交するということさえ、必要ではない。しかし、人間の動きをヒューマノイドロボットの動きにマッピングするためには、身体の主軸またはグローバル身体座標系として選択されたどのような他の座標系に対しても、これらの軸の向きを知ることは不可欠である。そうすれば、動力学的モデルをこれらの既知の関節軸方向を用いてパラメータ化することができる。
【００９１】
この方策で誤差の源となるのは、一つは、人間の行為者が正確に基本構成の姿勢に動けないことである。このため、人の動力学的パラメータをロボットの関節角度にマッピングする場合、そのマッピングが幾分不正確になる。しかし、関節角度が既知であるようなヒューマノイドロボットの姿勢に関連付けることができる少なくとも一つの身体の姿勢におけるマーカの位置を知ることなしには、人間の関節角度をヒューマノイドロボットの関節角度に関連付けることは不可能である。
【００９２】
２個の肩マーカを特別に配置することは、たとえばオブライエンら［５］またはシラギら［６］の方法を用いて左右の肩関節位置を推定することで避けられる。その後これらの関節位置を用いて、身体の第二の主軸方向を推定し、グローバル身体座標系の原点を決定することもできる。これらの方法はともに、関節の場所を、それによって結ばれる身体の二つの部分のローカル座標系を用いて推定している。この明細書に開示した方法と同様に、行為者は検討される関節のすべての自由度を発揮する一組の動きを行なうよう求められる。どちらのアプローチも、動きの間のすべての身体部品の位置および向きの推定に基づいている。これらの向きは、測定のたびごとに、身体部品のすべてについてそれに付された少なくとも３個のマーカが可視であれば（たとえばエイランら［９］の方法を用いて）、マーカデータから推定可能である。残念ながら、関節場所の推定はノイズの多いプロセスであり、推定は実際に行なわれた動きに依存して変化する。なぜなら、人間の関節は完璧な球状関節ではなく、動きの間固定されているわけではないからである。従って、このようにして生成されたモデルパラメータが、２個のマーカの特別な配置に基づくパラメータより正確であるという保証はない。加えて、多くの光学トラッキングシステムは疎なマーカデータを提供するのみであり（典型的には較正を行なう場合身体部品の各々につきわずか３個または４個のマーカがあるだけである）、このため、身体部品の位置および向きの推定がオクルージョンのため信頼性の低いものとなってしまう。これらの理由で、実現がより容易な、２個の肩マーカの特別な配置を利用する方が好ましい。
【００９３】
人間の行為者のモデルを、モーションキャプチャのセッションを開始するたびに一から推定する必要はない。次のモーションキャプチャセッションでは、基本構成でのマーカの位置を測定するだけでよい。「自動モデル生成」の項で説明したとおり、こうして新たなローカルマーカ位置およびローカル座標フレームの位置と向きとを、グローバル身体座標系で推定することができる。間接軸の位置は古い動力学的モデルと同じままであり、それらを新しく推定するために一連の動きを行なうことなく、これらを再び用いることができる。
【００９４】
［実験］
動力学的モデルの自動生成のためのこの方法を試験するため、複数の実験を行い、人間の行為者の全身運動を含むいくつかの動きの軌跡をキャプチャした。図９はデータ収集プロセスの例を示す。マーカベースの光学トラッカ３０がこの目的で用いられた。光学トラッカ３０は特定可能なアクティブマーカを用いる。全身運動の場合にはマーカのうちいくつかがオクルージョンをこうむることがしばしばあるので、アクティブマーカを用いると有利である。このような場合、パッシブなマーカを用いるシステムはよりマッチング誤差に弱い。
【００９５】
実験により、この発明のモデル構築システムがコンピュータアニメーションの生成にも、ヒューマノイドロボットの動きにも、十分正確であることが示された。図９に示すとおり、被験者１５０は、すべての自由度を用いようと、関節複合体の各々（首、左右の肩、肘、手首、腰、左右の臀部、膝、および踵）を動かした。回復された動力学的パラメータは驚くほど安定していた。自動的に回復された関節位置の方に、手動で測定したものより自信を持てるほどであった。なぜなら、多くの関節については、身体での関節の実際の位置を判断するのは困難だからである。上体の自由度を４回にわたって行使した被験者一名の実際の値を表ＩからＶＩに示す。
【００９６】
【表１】

【００９７】
【表２】

【００９８】
【表３】

【００９９】
【表４】

【０１００】
【表５】

【０１０１】
【表１】

【０１０２】
この発明の方法で得られたモデルは、データ収集プロセスの始めに被験者がとる基本構成と、２個の特別なマーカの配置とに依存する。明らかに、同じ被験者であっても、データ収集プロセスごとにデータがわずかに相違する。しかしながら、異なるセッションで得られたパラメータは同等であり、毎回すべての自由度を発現させる必要なしに、再びモデルを用いることができた。
【０１０３】
この発明を特定の実施例に則して説明したが、この発明はこれに限られるものではない。発明の範囲は請求項によってのみ制限される。
【図面の簡単な説明】
【図１】 ヒューマノイドロボットの例を示す写真による図である。
【図２】 パラメトリックな動力学モデルの概略図である。
【図３】 この発明の一実施例を実現するシステムの概略図である。
【図４】 この発明の実施例が実現されるコンピュータシステムの斜視図である。
【図５】 図４に示すコンピュータシステムの構造図である。
【図６】 この発明の一実施例に従ったシステムの動作の全体的制御を示すフローチャートである。
【図７】 パラメトリックモデルの拡大または縮小に必要な最適化プロセスのフローチャートである。
【図８】 この発明の実施例で用いられるパラメトリックな動力学モデルの関節連鎖の概略図である。
【図９】 モーションキャプチャプロセスを示す写真による図である。
【符号の説明】
２ ロボット、４、６、８、１０、１２ 関節、２０ コンピュータシステム、３０ 光学トラッカ、４０ コンピュータ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method and program for automatically adapting a parametric dynamic model for motion capture to the size of a human actor, and in particular, the invention relates to the captured movement of a human actor. The present invention relates to a method and a program for automatically determining parameters required for motion generation.
[0002]
[Prior art]
There is growing interest in the production of humanoid robots. Generating movement for such robots is a large number of joints, relevance between joints, redundancy, and humans robots that interact with such robots behave like humans It is difficult for reasons such as thinking that it is a trap. The same problem is encountered when generating animated character motion in computer graphics. Such a humanoid robot or CG animation character is referred to herein as an “agent”.
[0003]
The applicant has developed a humanoid robot as shown in FIG. As can be seen from FIG. 1, as the name suggests, humanoid robots have a dynamic structure similar to that of humans, taking advantage of the similarity between humanoid robots and humans, An appropriate robot trajectory can be generated based on the above. Furthermore, since humanoid robots are usually required to work with human-like movements, the nature of such work can also be used. Therefore, considering motion capture technology from the entertainment industry and computer graphics [1], robot remote control field, robot programming by direct teaching or viewing [2] and virtual reality field It would make sense.
[0004]
Motion capture is one of the most important technologies for generating humanoid movements of humanoid agents. This includes measuring a human actor's movement using a suitable measuring device and converting the measured movement into a humanoid agent's movement.
[0005]
[References]
[1] B. B. Bodenheimer, C.I. C. Rose, “The process of motion capture: Dealing with the data”, Computer Animation and Simulation '97, Euro Graphics・ Proc. Eurographics Workshop. Pp. 3-18. Published by Springer Fairlark, 1997
[2] T.M. "Robot programming" by T. Lozano-Perez, IEEE Proceedings (Proc. IEEE) vol. 71, no. 7, pp. 821-841, July 1983
[3] B. B Karan and M.C. M. Vukobratovic, “Calibration and accuracy of manipulation robot models—an overview”, Mechanism and machine Theory, vol. 29, no. 3, pp. 479-500, 1994
[4] M.M. A. Lafortune, P.A. R. P.R. Cavanaugh, H.C. J. et al. H.J. Sommer and A.S. A. Kalenka, “Three-dimensional kinematics of the human knee during walking”, Journal of Biomechanics, vol. 25, no 4, pp. 347-357, April 1992.
[5] J. et al. F. O'Brien, R.C. E. R.E. Bodenheimer, G. J. et al. Brostow and J.H. K. JK Hodgins, "Automatic joint parameter estimation from magnetic motion capture data", Proc. Graphics Interface 2000 Conference, Canada , Montreal, May 2000
[6] M.M. -C. M.-C. Silaghi, R.A. R. Plaenkers, R.P. R. Boulic, Pl Fua and D.B. "Local and global skeleton fitting techniques for optical motion capture" by D. Thalmann, modeling and motion capture techniques for virtual environments (Modeling and Motion Capture Techniques for Virtual Environments), N.M. N. Magnerat Thalmann and D.M. Edited by D. Thalmann, p. 26-40, Lecture Notes in Computer Science 1537, published by Springer Fairlark, Berlin, New York, 1998
[7] R.M. M.M. Murray, Z. Z. Li and S. S. By S.S. Sastry, "A Mathematical Introduction to Robotic Manipulation", C.R.C. Press, Boca Raton, New York, 1994.
[8] T.M. C. Coleman, M.C. A. Branch and A.B. A. Grace, “Optimization Toolbox User ’s Guide”, published by The MathWorks, Natick MA. 1999
[9] K.K. S. K.S. Arun, T.W. s. T.S. Huange and S.H. D. By SD Blostein, “Least-squared fitting of two 3-D point sets, IEEE Transactions of Pattern Analysis and Machine Intelligence (IEEE Trans Pattern Anal. Machine Intell.), Vol. 9, no. 5, pp. 698-700, September 1987
[Problems to be solved by the invention]
Due to the requirement to actually control the physical device, it has been found that the technology from virtual reality and entertainment applications needs modification. The main problems are that it is difficult to process teaching data and that motion capture is a complicated task.
[0006]
In order to generate movement for the agent based on the captured movement, it would be necessary to capture the whole body movement of a human actor using a suitable measurement device. One such device is a system using a goniometer. When the actor wears a goniometer, the goniometer directly measures the actor's movement as the position and orientation of the joint. The main drawback of this goniometer system is that the goniometer is of a specific size designed for a specific robot and can only be worn by a person of the correct size. If a human actor is too short, you will not be able to wear a goniometer system. Therefore, there is a need for a method and program that easily captures human movement and automatically adapts the parametric dynamic model to the size of the human actor.
[0007]
The technology for automatically building dynamic models for robot manipulators is well established in the field of robotics. Numerous studies have been done to identify the most suitable kinetic parameter system according to criteria such as completeness, ratio, and equivalence [3]. Most commonly, the unknown dynamic parameters are identified from the posture measurement of the robot hand (the part of the robot hand that operates the object) and the joint position reading of the robot.
[0008]
Unfortunately, joint position data is not available when building human dynamics models. In addition, the technology that has been developed in robotics often requires the robot to move only on a single joint without movement in other degrees of freedom, which is difficult for humans to do this It is. Finally, apart from identifying the dynamic parameters of human movement, the position of the body marker must also be determined by the motion estimation process in order to be able to use the model in the future.
[0009]
The identification of dynamic models for humans has been studied in the fields of biomechanics and computer graphics. It is well known in the biomechanical world that human joints are not ideal rotational joints and therefore do not have a fixed center of rotation [4]. Instead of fixing joint positions, biomechanics researchers often estimate the instantaneous center of rotation.
[0010]
However, if the goal is to generate humanoid robot motion, it is not necessary to model human motion as accurately as possible. If the position of the joint axis of the humanoid robot is fixed, the rotation axis of the human actor's dynamic model should also be fixed, which is true with some accuracy.
[0011]
Accordingly, an object of the present invention is to provide a method and program for motion capture that can automatically adapt a parametric dynamic model to the size of a human actor.
[0012]
Another object of the present invention is to provide a method and program capable of automatically adapting a parametric dynamic model for motion capture to the size of a human actor without any manual measurement. It is.
[0013]
Yet another object of the present invention is to create a parametric dynamic model for motion capture, with a small amount of computation, without the need to estimate all joint axis positions in a single long optimization problem. It is to provide a method and program that can be automatically adapted to the size of a computer.
[0014]
[Means for Solving the Problems]
  According to one aspect of the present invention, a computer-implemented method for automatically adapting a parametric dynamic model for motion capture to the size of a human actor is applied to the body part of the human actor. A set of computer-readable position data for the marked markers in each local body part coordinate system and the local body relative to the global body coordinate systempartsPrepare the position and orientation of the coordinate systemPosition data preparationAnd preparing a first set of computer readable motion capture data representing the location of markers attached to the body part of the human actor, the motion capture data being human actors Captured during a set of movements, plus position dataSet ofAnd a first set of computer readable motion capture data, causing the computer to estimate the position of the joint axis of the body part of the human actor.
[0015]
  The step of causing the computer to estimate is computer-readable marker position data.Set ofAnd a first set of computer-readable motion capture data and a chain of parametric dynamic modelsInsidePrecedeIn placeUsing a position of the joint axis set to cause the computer to automatically calculate the position of the current joint axis set within the chain of parametric dynamic models,In the chainPrecedeIn placeThe set of joint axes isIn the chainPrecedeIn placeThey are placed in the dynamic chain in such a way that movement around the joint axis set affects the position of the current joint axis set.
[0017]
  The step of calculating the position of the current set of joint axes is preferably (a) for all measurement times of the first set of motion capture data.,The position and orientation of the human actor's body at each measurement time, and the chain at that time, if anyInsidePrecedeIn placeComputer-readable marker position data for the joint angle of a set of axesSet ofCalculated in the first set of motion capture data and the preceding steps, if anyIn the chainPrecedeIn placeUsing the position of the set of joint axes, and (b) computer-readable marker position dataSet ofCalculated in the first set of motion capture data and the preceding steps, if anyIn the chainPrecedeIn placeThe position of the joint axis set, the position and orientation of the body, and, if any, calculated in step (a)In the chainPrecedeIn placeEstimating the position of the current set of joint axes in the chain and the associated joint angle using the joint angles associated with the set of joint axes.
[0018]
  Since the computer automatically calculates the position of the preceding set of joint axes and the position of the second set of joint axes in the chain, thus calculating all subsequent joint axes in the chain, it is parametric. The dynamic model can be automatically adapted to the size of the human actor. Since the position of the marker is used for the calculation, it is not necessary to manually measure the position of the joint axis of the human actor. further,Since the calculation is performed in two steps and the result of the first step is used in the second step, the amount of calculation is reduced.
[0019]
  Step (a) includes a first set of motion capture data and,Position dataSet ofAnd the position and orientation of the body,In the chainPrecedeIn placeCalculated at the joint angle of the set of joint axes and the preceding stepIn the chainPrecedeIn placeBy optimizing the preferred index function defined by the position of the joint axis set,In the chainPrecedeIn placeEstimating the joint angle of the set of joint axes may be included.
[0020]
Since the variables to be optimized only include joint angles and body position and orientation, the amount of computation is reduced and results can be obtained in a short time.
[0021]
  Step (b) includes a first set of motion capture data and,Position dataSet ofAnd the position and orientation of the body,In the chainPrecedeIn placeCalculated in the preceding step with the joint angles of the joint axis set and the current joint axis setIn the chainPrecedeIn placeEstimating the position of the current set of joint axes by optimizing a suitable index function defined by the position of the set of joint axes and the position of the current set of joint axes may be included.
[0022]
Since the variables to be optimized only include the joint angle and the current position of the set of joint axes, the amount of calculation is reduced and a result can be obtained in a short time.
[0023]
  PlacePlacePreparationTep prepares a second set of computer-readable motion capture data that represents the position of markers on the human actor's body parts while the human actor is in a predetermined reference posture. Steps may be included.
[0024]
Since human actors are required to stand in a predetermined reference posture, it is easy to prepare position data.
[0025]
At a predetermined reference position, the human actor may stand in an upright posture with limbs extended.
[0026]
  According to another aspect of the invention, for motion captureIn addition,For automatically adapting parametric dynamic models to the size of human actors,The computer-executable program consists of a set of computer readable position data in the local body part coordinate system of markers attached to the body part of the human actor and the local body relative to the global body coordinate system.partsA first program portion for preparing a coordinate system position and orientation and a first set of computer readable motion capture data representing the position of a marker attached to a body part of a human actor A second program portion for capturing motion capture data, wherein a human actor is captured during a set of movements, and further includes position dataSet ofAnd a first set of computer readable motion capture data including a third program portion for estimating the position of the joint axis of the body part of the human actor, the third program portion comprising: Possible marker position dataSet ofAnd a first set of computer-readable motion capture data and a chain of parametric dynamic modelsInsidePrecedeIn placeA fourth program part for calculating the position of the current joint axis set in the chain of dynamic models using the position of the joint axis set;In the chainPrecedeIn placeThe set of joint axes isIn the chainPrecedeIn placeThey are placed in the dynamic chain in such a way that movement around the joint axis set affects the position of the current joint axis set.
[0027]
Since the computer automatically calculates the position of the preceding set of joint axes and the position of the second set of joint axes in the chain, thus calculating all subsequent joint axes in the chain, it is parametric power. The academic model can be automatically adapted to the size of the human actor. Since the position of the marker is used in the calculation, it is not necessary to manually measure the position of the joint axis of the human actor.
[0028]
  The fourth program part is for all measurement times of the first set of motion capture data.,The position and orientation of the human actor's body at each measurement time, and the chain at that time, if anyInsidePrecedeIn placeComputer-readable marker position data for the joint angle of a set of axesSet ofCalculated in the first set of motion capture data and the preceding steps, if anyIn the chainPrecedeIn placeProgram portion (a) to be estimated using the position of the joint axis set, and computer-readable marker position dataSet ofCalculated in the first set of motion capture data and the preceding steps, if anyIn the chainPrecedeIn placeCalculated in the program part (a), the position of the joint axis set, the body position and orientation, if anyIn the chainPrecedeIn placeUsing the joint angle associated with the set of joint axes,During ~A program part (b) for estimating the current position of the set of joint axes and the associated joint angle.
[0029]
  Since the computer automatically calculates the position of the preceding set of joint axes and the position of the second set of joint axes in the chain, thus calculating all subsequent joint axes in the chain, it is parametric power. The academic model can be automatically adapted to the size of the human actor. Since the position of the marker is used for the calculation, it is not necessary to manually measure the position of the joint axis of the human actor.Since the calculation is performed in two program parts and the result of the first program part is used in the second program part, the amount of calculation is reduced.
[0030]
  Program part (a) is the first set of motion capture data and,Position dataSet ofAnd the position and orientation of the body,In the chainPrecedeIn placeCalculated by the joint angle of the joint axis set and the fourth program partIn the chainPrecedeIn placeBy optimizing a suitable index function defined by the position of the set of joint axes, and the body position and orientation;In the chainPrecedeIn placeA program portion for estimating the joint angle of the set of joint axes may be included.
[0031]
Since the variables to be optimized only include joint angles and body position and orientation, the amount of computation is reduced and results can be obtained in a short time.
[0032]
  Program part (b) is the first set of motion capture data and,Position dataSet ofAnd the position and orientation of the body,In the chainPrecedeIn placeThe joint angles of the joint axis set and the current joint axis set and calculated in the fourth program partIn the chainPrecedeIn placeA program portion may be included that estimates the position of the current set of joint axes by optimizing a suitable index function defined by the position of the set of joint axes and the position of the current set of joint axes.
[0033]
Since the variables to be optimized only include the joint angle and the current position of the set of joint axes, the amount of calculation is reduced and a result can be obtained in a short time.
[0034]
The first program portion prepares a second set of motion capture data representing the positions of markers attached to the human actor's body parts while the human actor is in a predetermined reference posture. A program part may be included.
[0035]
Since the human actor is in a predetermined reference posture, it is easy to prepare position data.
[0036]
In a predetermined reference posture, the human actor may stand in an upright posture with limbs extended.
[0037]
DETAILED DESCRIPTION OF THE INVENTION
[Dynamic model]
Human motion can be modeled as articulated motion of a rigid body part with some degree of accuracy. Therefore, the position of the humanoid agent in the Cartesian space is usually given by the position and orientation of the global coordinate system fixedly attached to one of the body parts and the value of the joint angle with respect to the body axis. Thus, the humanoid agent's body motion can be described using a parametric dynamic model that identifies the position and orientation of the body joint axis relative to the selected body coordinate system.
[0038]
  Any movement of the humanoid agent is then given by a trajectory identifying the position and orientation of this selected body coordinate system at a certain time and a trajectory of the joint angle. In this specification, the body position and orientation are collectively referred to as “body posture”. Some accuracyTheThus, human movement can be modeled in this way. The method according to an embodiment of the present invention provides an automated technique for synchronization of parametric dynamic models that can describe both human movements and humanoid agent movements.
[0039]
Such a model makes it possible to measure human movement in the joint space described by the underlying parametric model using motion capture. Since the movement of the humanoid agent also occurs in the same dynamic space, the movement of the humanoid agent can be driven directly using the trajectory of the measured joint angle.
[0040]
The method begins by identifying a parametric dynamic model that can describe the whole body movement of a humanoid agent, or a subspace of such movement. In order to measure the movement of a human actor in the space defined in this mode, the parameters defining the model need to be scaled up or down to the physical size of the human actor. In this method, parameters of the kinetic model are based on 3D (three-dimensional) marker position measurements acquired while the human actor is performing various movements that exhibit all relevant degrees of freedom. Can be estimated. There is no need to manually measure the length of the actor's limbs.
[0041]
  FIG. 2 shows the dynamic structure of the humanoid robot shown in FIG. 1. Here, our goal is to model the human movement by the specific dynamic structure of the humanoid robot as shown in FIG. It is to become. As shown in FIG. 2, this structure has a number of joints positioned on the right arm of the robot 2, such as joints 4, 6, 8, 10 and 12.16including. The structure further includes a left arm, right foot, left foot, neck and hip joints.
[0042]
It is very natural to specify such a parametric model using a twist expression [7]. This is because it allows working directly in the global body coordinate system instead of specifying the transformation between successive local coordinate frames in the dynamic chain. Here, “dynamic linkage” means a series of joints arranged in a dynamic structure such that the movement of the preceding joint affects the position of the subsequent joint. The only drawback of the twist representation is that it is not minimal (this requires 6 parameters instead of 4 for each transformation), but as described below, a set for calibration based on twist. Independent parameters can be derived.
[0043]
  The goal here is to determine the location and direction of the joint axis of the human body. The twist is defined by these parameters. By using joint axis parameterization by twisting, two independent parameters are derived for each of the joint axis positions. Regarding rotary jointsvectorn_iIs the unit vector in the joint axis direction,vectorq_iLet be any point on the axis. Both are given in the basic configuration, ie the global body coordinate system with all joint angles equal to zero. in this case,Twist about the i-th joint axisVector ξ _iTakes the following form:
[0044]
[Expression 1]

The angle θ for such a joint_iThe reflection of the body part caused by the movement of is calculated using the following exponential map [7].
[0045]
[Expression 2]

Where R (θ_i,n _i ) Is the unit axis n_iΘ around_iIs an orthogonal matrix that describes only the rotation.
[0046]
In order to generate humanoid robot motion from human motion capture data, it is necessary to correlate human joint motion with 3-D marker motion. The coordinate of the marker in the local coordinate system of the body part that is the rigid body to which the marker is attached isy _i If given by the body composition (r,d, θi1..., θin), the 3-D position is calculated as follows:
[0047]
[Equation 3]

hereVector ξ _{i 1}..., Ξ _inIs the marker movement y_jIs a twist that describes the kinetic chain that gives rise to G_xIs a homogeneous matrix that associates the position and orientation of the local body part coordinate system to which the marker is attached to the global body coordinate system in the basic configuration. (In the case of the humanoid robot shown in FIG. 1, x = upper body, pelvis, head, upper left arm, lower left arm, left hand, upper right arm, lower right arm, right hand, upper left leg, lower left leg, left leg, upper right leg, lower right leg R and d are orientations represented by the rotation vector and position of the global body coordinate system with respect to world coordinates, and g (r, d) is a homogeneous matrix corresponding to r and d. Note that the twist set that affects the movement of the marker varies depending on what body part the marker is attached to.
[0048]
In such a setting, dynamic modeling includes identifying the position of the joint axis and the direction and position of the local body part coordinate system, all of which are global body conditions with the body in its basic configuration. Identified in the coordinate system. Apart from the local body part coordinate system, a global body coordinate system that identifies the position and orientation of the body in space also needs to be fixed.
[0049]
[Automatic model generation]
The basis of this example approach is to establish a relationship between the dynamics of the human body and the dynamics of a humanoid robot or CG character. This is accomplished by modeling the dynamics of human actors as a model standard for humanoid agents and scaled up or down to the physical size of human actors. Since the dynamics of humanoid agents are similar to the dynamics of the human body, a wide range of human movements can be addressed in this way.
[0050]
For this purpose, let us consider the typical dynamic structure of a humanoid robot and the dynamics of the human body. In biomechanics and computer graphics, the neck joint, shoulder joint, hip joint, and hip joint are typically modeled as spherical joints. This is not the case for the humanoid robot in this case. This is because these locations of the humanoid robot have several (typically three) consecutive rotating joints that are orthogonal but not necessarily intersecting. In addition, the direction of the corresponding joint axis usually coincides with the three main axes of the body (vertical, horizontal, and longitudinal) in the basic configuration. When the subject or humanoid agent is in an upright position with their limbs extended, the body is considered to be in the basic configuration. Three consecutive rotating joints can produce any rotation, similar to a spherical joint, so that the two dynamic structures are equivalent to some extent. Therefore, it makes sense to model a three degree of freedom (DOF) spherical joint with three consecutive rotating joints with axes oriented in the direction along the main axis of the body. The order and sign of the axes must be chosen to be the same as the order and direction of the joint axes of the humanoid agent being considered.
[0051]
Regardless of what dynamic parameter system is employed, the actual value of the parameter (joint angle) depends on what local coordinate system is selected. This is because it is these local coordinate systems that specify the transformation between these values. Whatever the posture of the body, it is essential to select the local coordinate system so that the joint angle value is the same as the joint angle value of the humanoid agent taking the posture corresponding to the posture. Otherwise, the measured human joint angle cannot be mapped to the humanoid agent joint. Therefore, here the global and local body part coordinate systems for human actors are oriented so that they are all aligned when the actor is in an upright position with their limbs extended. In this configuration, those axes were chosen to be parallel to the main body axis. In this way, the joint axis direction of the global coordinate system in the basic configuration coincides with the coordinate axis.
[0052]
[The actual determination of the coordinate system and the marker position on the body]
In order for the movement of the marker to be correlated with the movement of the joint of the actor being observed, the position of the marker in the body in the local body part coordinate system must be known. This data is obtained by measuring the position of the marker while the actor is taking an upright posture whose joint angle is known. A basic configuration is suitable for this purpose. In the experiment, the actors took such a posture and recorded the markers. Other robots may require different basic configurations. For each body part, the origin of the local coordinate system was placed at the center of gravity of all markers attached to it.
[0053]
To determine the direction of the body's principal axis, the optical tracking system was calibrated to see where it was up and down. Therefore, one of the body axes in the basic configuration (in this embodiment, the Z axis) can be set in parallel with the Z axis of the world coordinate system. The axis from the left shoulder to the right shoulder can be estimated by attaching two markers at opposite positions of the left shoulder and the right shoulder, or by estimating the location of the shoulder joint from the data. This can be performed using one of the methods described in [5] and [6]. In either case, the line between the two opposing positions defines the second body axis (y-axis in this system). Due to inaccuracies in measurement and / or marker placement, the derived axes are not exactly orthogonal. For this reason, by recalculating new Z-axis directions, they are forced to be orthogonal.
[0054]
[Expression 4]

Where y_lAnd y_rAre the coordinates of special markers affixed to the left shoulder and right shoulder, respectively (or at the location of the left shoulder joint and right shoulder joint). This new direction is perpendicular to the y-axis and yaxisOf all directions orthogonal to the original z-axis. Finally,Using the cross product of vectors in the y-axis and z-axis directions,xThe axial direction is calculated.
[0055]
  The origin of the global body coordinate system is assumed to be in the middle of a line connecting two shoulder joints (or two specially positioned shoulder markers). This is fixed to the upper body. In the basic configuration, the global body coordinate system and all local body part coordinate systems are aligned. Using this assumption, the global body coordinate system (G in equation (3)_x) Calculates the position and orientation of each body part in the basic configuration, and sets the global coordinate system of the marker to the local coordinate system (y _j and_{y j} ).
[0056]
Since the joint angle is equal to zero, equation (3) can be simplified as follows.
[0057]
[Equation 5]

The position of the global coordinate system is given below.
[0058]
[Formula 6]

On the other hand, the orientation is given by a rotation matrix R having three coordinate axes in its column.
[0059]
[Expression 7]

The orientation of each local body part coordinate system in the basic configuration is the same as the orientation of the global body coordinate system. Thus, the relative orientation is given by the identity matrix. The relative position can be calculated by subtracting the position of the global body coordinate system. Therefore, the following holds for the body part x.
[0060]
[Equation 8]

Where n_xIs the number of markers attached to the body part. Finally, the markery _j Is obtained by taking the inverse of equation (5).
[0061]
[Estimation of joint position]
At this point, the only parameter that still needs to be estimated is the position of the joint axis. Our goal is to estimate the position of the joint axis on the body in a coordinate system centered on the body. To estimate these parameters, have the subject make a set of movements and measure them with a motion capture system.
[0062]
We decided not to try to estimate all joint locations in one big optimization process, but to divide the estimation into 10 smaller optimization problems. That is, the neck, waist, left and right shoulders + elbows, left and right wrists, left and right buttocks + knees, and left and right heels.
[0063]
There are only two degrees of freedom for the position of the joint axis in the 3-D space because the position along the axis is arbitrary. The directions of all joint axes in the basic configuration are parallel to the coordinate axes of the global body coordinate system, and therefore all vectors [1, 0, 0]^T, [0,1,0]^T, [0,0,1]^T, [-1, 0, 0]^T, [0, -1,0]^TAnd [0,0, -1]^TGiven by one of the It should be noted that the order of the joint axes is kept the same as that of the humanoid robot under consideration. It can be easily understood that the coordinates corresponding to the non-zero coordinates of the vector in the joint axis direction do not affect the resulting twist given by equation (1). For example, [a, b, c]^T[0,1,0]^TFor joint axes parallel to, the corresponding twist is equal to
[0064]
[Equation 9]

It is therefore independent of the second coordinate, b. From this, it is possible to parameterize the location of each joint axis with two coordinates different from the non-zero coordinate of the direction vector of the joint axis.
[0065]
Having obtained independent coordinates for the joint axis location, these can be estimated by minimizing the preferred optimization index. Apart from the location of the joint axis, it is also necessary to estimate the body position and orientation in space and the joint angle so that the model marker matches the measured marker position. In order to make the optimization process smaller, all degrees of freedom prior to the joint to be considered are estimated in a separate optimization process. In this case, the optimization process includes only the position of the joint angle that needs to be estimated and the corresponding joint angle. Nevertheless, the result is a very large optimization problem. For example, in order to estimate the location of the neck joint, the following optimization criteria need to be minimized.
[0066]
[Expression 10]

Note that the number of parameters increases with the number of measurement times. In the experiment, typically 300 measurements were made for each of the 10 optimization problems. The position of each joint axis can be parameterized with two parameters. There are two parameters for each joint axis, so for three joint axes there are six parameters (for the neck, waist, hand and heel) and for four joint axes, eight parameters. Parameters (shoulder + elbow, buttocks + knee). These parameters are given in the basic configuration and do not change.
[0067]
Furthermore, in order to correlate the measured marker positions with these parameters, it is necessary to estimate the joint angles for these axes. Unlike the position of the joint axis, the joint angle changes with each set of measurements. Therefore, when performing 300 measurements, it is necessary to estimate 3 × (300 + 2) = 906 parameters or 4 × (300 + 2) = 1208 parameters for each optimization problem.
[0068]
In order to solve such optimization problems, the subspace trust region approach and sparse matrix algebra [8] were used. The trust region method requires calculation of the first derivative of the index function h. The calculation of these derivatives is based on the joint angle θ1 (t_k), Θ2 (t_k), Θ3 (t_k), K = 1,... N and joint position parameters a, b, c, d, e, f based on the Jacobian calculation of the forward dynamic function (3). The calculation of the Jacobian regarding the joint angle is a well-known problem, and was realized using the method described in [7]. The Jacobian of the optimization criterion (9) is sparse, and sparse matrix algebra is used as the optimization method here in order to reduce the calculation time. The functions and tools used in this example can be used in an optimization tool kit such as MATLAB (TM).
[0069]
  [Preferred embodiment]
  FIG. 3 shows an overall view of this embodiment. Referring to FIG. 3, the system is a human actor.18An optical tracker 30 for measuring the 3-D position of markers attached to various body parts, and a parametric dynamic model for motion capture by performing measurement data computations to the size of a human actor And a computer system 20 that optimizes the parameters for adaptation. The same actor that the optical tracker 30 captured the adaptation parameter 80 thus calculated by the computer system 2018It can be used in a motion generation system 82 that generates data for controlling the robot based on the motion data.
[0070]
  FIG. 4 shows the computer system 20 and FIG. 5 shows the system 20 in a block diagram. Referring to FIG. 4, the computer system 20 is an FD (flexible disk) drive.The52 and a computer 40 having a CD-ROM (Compact Disc Read Only Memory) drive 50, a keyboard 46, a mouse 48, and a monitor 42. The optical tracker 30 is connected to the computer 40.
[0071]
  Figure5With reference to FIG.TheIn addition to the CPU 52 and the CD-ROM drive 50, a CPU (Central Processing Unit) 56, a CPU 56, and an FD driverThe52 and a bus 66 connected to the CD-ROM drive 50, a read only memory (ROM) 58 for storing a boot-up program and the like, and a random access memory connected to the CPU 56 for storing program instructions, system programs and data (RAM) 60.
[0072]
Although not shown here, the computer 40 may further include a network adapter board that provides a connection to a local area network (LAN). In the case of a humanoid robot, the system further includes a number of actuators for moving the various parts of the camera and robot under the control of the computer system 20.
[0073]
A program that causes the computer system 20 to automatically adapt the parametric dynamic model to the size of the human actor is stored in a CD-ROM 62 or FD 64 that is inserted into the CD-ROM drive 50 or FD drive 52. Further, it is transferred to the hard disk 54. Alternatively, the program may be transmitted to the computer 40 through a network (not shown) and stored in the hard disk 54. The program is loaded into the RAM 60 at the time of execution. The program may be directly loaded into the RAM 60 via the CD-ROM 62, the FD 64, or the network.
[0074]
The program described below includes several instructions that cause the computer 40 to perform the method of this embodiment. Since some of the basic functions necessary to carry out this method are provided by modules such as the operating system (OS) of the computer 40 or a third party program, or an optimization toolbox installed on the computer 40, This program does not necessarily include all functions necessary to realize the method of this embodiment. The program only needs to include the part of the instruction that performs the optimization process by calling the appropriate function or “tool” in a controlled manner to achieve the desired result. The operation of computer system 20 is well known and will not be repeated here.
[0075]
  FIG. 6 shows the overall control structure for a program that implements the method of this embodiment. Referring to FIG. 6, human actionsPerson ofAfter the body has been marked(90)The program starts. Since two markers are positioned at positions apart from the shoulder, the horizontal axis of the global body coordinate system can be defined by a line connecting these two markers in the basic configuration. First, the computer 40 captures 3-D data of the marker from the motion capture data 92 while the actor is standing in the basic configuration posture (94).
[0076]
  Then local bodypartsThe position of the marker in the coordinate system and the position of the local frame relative to the global frame are calculated (96). Prior to this,A global body coordinate system is calculated as described above.
[0077]
  Next, the actor is asked to make a set of moves,OpticalThe captured data 110 measured by the tracker 30 is sent to the computer system (98). If this method needs to return an unambiguous answer, the actor must act on all appropriate degrees of freedom.
[0078]
In step 100, the computer estimates the position of the joint axis on the body using extensive optimization based on the captured data 110 and the parametric dynamic model 112. Details of this step will be described later with reference to FIG. The output of step 100 is a set of parameters that adapt the parametric dynamic model to the size of the human actor for motion capture (102). In this step, the estimation process is divided into ten separate smaller optimization problems as described above.
[0079]
FIG. 7 shows the optimization process required to scale the parametric model performed in step 100. Referring to FIG. 7, the following steps 122 to 126 are repeated for all dynamic chains in the model. Note that each chain includes a series of joint complexes arranged sequentially in the dynamic structure in such a way that the movement of the preceding joint complex affects the position of the subsequent joint complex. . Typically, the dynamic chain consists of upper body, head, left arm, right arm, left leg, right leg degrees of freedom (DOF) or a subset thereof. The joint complex consists of a neck DOF, left and right shoulder + elbow DOF, left and right wrist DOF, waist DOF, left and right buttocks + knee DOF, left and right heel DOF, or a subset thereof.
[0080]
At step 122, all joint complexes in the current chain are analyzed in sequential order, and steps 124 and 126 are repeated for each joint complex.
[0081]
At step 124, for all measurements of motion capture data 110, the body posture (position and orientation) and the degrees of freedom preceding the current joint complex are estimated using non-linear optimization. The other optimization processes here are not extensive, since the parameters for the estimation of the body posture and the degrees of freedom preceding the current joint complex can be estimated independently of each other at each measurement.
[0082]
As can be seen in equations (1) to (3), the 3-D position of the marker depends on the joint angle of the joint complex preceding the current joint complex, the joint angle of the current joint complex, and the basic configuration. Of the marker and the current and preceding joint axes in the basic configurationq _i (Any point on the axis). Among these, the position of the marker in the basic configuration is known and the preceding joint axisq _i Is also known from the previous iteration. These parameters can be considered constant. Each measurement data includes the 3-D position of the marker, each of which should therefore be a function of body posture and joint angle of the preceding joint axis. By optimizing at each measurement, the body posture and joint sensitivity relative to the preceding joint axis for that measurement can be determined in step 124.
[0083]
Next, at step 126, an estimate of the current joint axis position on the body and the associated angle is made using large scale nonlinear optimization. Here, as in step 124, the 3-D position of the marker indicates the body posture, the joint angle of the joint complex preceding the current joint complex, the joint angle of the current joint complex, and the marker in the basic configuration. Position of the current and preceding joint axes in the basic configurationq _i (Any point on the axis). Among these, the position of the marker in the basic configuration is known and the preceding joint axisq _i Is also known. Further, the joint angle with respect to the body posture and the preceding joint axis is determined in step 124. These parameters can be considered constant. Therefore, large-scale optimization is performed on all the motion capture data 110. This optimization estimates the position of the joint axis on the body and the associated angle for the current joint complex. This optimization is still extensive, but it is much smaller and requires less time than the optimization done for all measurement data at once.
[0084]
The above process is further described with reference to FIG. 8, which shows the right arm chain in the basic configuration. Two special markers 142 and 144 are carefully attached to specific locations on the left and right shoulders, respectively. First, the optical tracking system is calibrated so that the z-axis is in the vertical direction. The y-axis direction 150 is defined by connecting the markers 142 and 144. The z-axis direction is recalculated so that the z-axis intersects the line 150 at the center of the line 150 and the amount of separation from the vertical direction is minimized. Therefore, the origin of the global body coordinate system is defined by the intersection of the y axis and the z axis. The x-axis direction is calculated by the cross product of unit vectors in the y-axis and z-axis.
[0085]
The direction of the joint axis in the basic configuration is selected to be parallel to one of the directions of the global body coordinate system.
[0086]
The positions of the joints 4, 6, 8, 10 and 12 shown in FIG. 8 are estimated as follows. First of all, the actor is required to make a set of movements that exerts all DOFs of joints 4, 6 and 8. The positions of joints 4, 6 and 8 are estimated by performing the processes of steps 124 and 126 in FIG. 7 by optimizing the motion capture data. The joint angles are also estimated here, but these are discarded.
[0087]
Next, in the second iteration, the position and orientation of the body in space and the DOF joint angles of joints 4, 6, 8 are estimated in step 124 using non-linear small scale optimization. Thereafter, at step 126, the positions of the joint axes 10 and 12 and their associated angles are estimated using extensive non-linear optimization.
[0088]
A similar process is performed for all chains of the parametric kinetic chain.
[0089]
[Technical issues]
In the index function (9), it is not assumed that the joint axes intersect each other in a joint having a plurality of degrees of freedom. It is easy to incorporate such assumptions into this optimization index function. In this case, the six parameters in equation (9) are reduced to just three. This is because each of them appears twice in the parameterization of the three twists. Some of the locations of the joint axes, for example, the locations of the left and right shoulder axes, may be estimated simultaneously to impose a symmetry constraint on the optimization index.
[0090]
If the ultimate goal of a motion capture session is to generate computer graphic animation, some of the steps described in this section are unnecessary. It can also alleviate some of the constraints imposed on the dynamics of humanoid robots. For example, it is not necessary that the robot's joint axes are parallel to the body's principal axes, or even that they are orthogonal. However, in order to map human movement to humanoid robot movement, it is not possible to know the orientation of these axes with respect to any other coordinate system selected as the principal axis of the body or the global body coordinate system. It is essential. The dynamic model can then be parameterized using these known joint axis directions.
[0091]
One of the sources of error in this measure is that human actors cannot accurately move to the basic configuration posture. This makes mapping somewhat inaccurate when mapping human dynamic parameters to robot joint angles. However, without knowing the position of the marker in at least one body posture that can be related to the posture of a humanoid robot where the joint angle is known, it is not possible to associate a human joint angle with the joint angle of a humanoid robot. Impossible.
[0092]
The special arrangement of the two shoulder markers can be avoided by estimating the left and right shoulder joint positions using, for example, the method of O'Brien et al. [5] or Shiragi et al. [6]. These joint positions can then be used to estimate the second principal axis direction of the body and determine the origin of the global body coordinate system. Both of these methods estimate the joint location using the local coordinate system of the two parts of the body connected by it. Similar to the method disclosed in this specification, the actor is required to perform a set of movements that exhibit all the degrees of freedom of the joints being considered. Both approaches are based on estimates of the position and orientation of all body parts during movement. These orientations can be estimated from the marker data for each measurement if at least three markers attached to all of the body parts are visible (eg, using the method of Eilan et al. [9]). is there. Unfortunately, joint location estimation is a noisy process and the estimation varies depending on the actual movements made. This is because human joints are not perfect spherical joints and are not fixed during movement. Therefore, there is no guarantee that the model parameters generated in this way are more accurate than the parameters based on the special arrangement of the two markers. In addition, many optical tracking systems only provide sparse marker data (typically there are only 3 or 4 markers for each body part when calibrating), and thus The estimation of the position and orientation of the body part is unreliable due to occlusion. For these reasons, it is preferable to use a special arrangement of two shoulder markers that is easier to implement.
[0093]
The human actor model does not have to be estimated from scratch each time a motion capture session is started. In the next motion capture session, it is only necessary to measure the position of the marker in the basic configuration. As described in the section “Automatic Model Generation”, the new local marker position and the position and orientation of the local coordinate frame can thus be estimated in the global body coordinate system. The position of the indirect axes remains the same as the old dynamic model, and they can be used again without a series of movements to estimate them newly.
[0094]
  [Experiment]
  To test this method for automatic generation of dynamic models, several experiments were performed to capture several trajectories, including whole body movements of human actors. FIG. 9 shows an example of a data collection process. Marker-based opticsTracker30 was used for this purpose. OpticalTracker30 uses an identifiable active marker. In the case of whole body movement, it is advantageous to use an active marker because some of the markers often suffer from occlusion. In such cases, systems using passive markers are more vulnerable to matching errors.
[0095]
Experiments have shown that the model building system of the present invention is sufficiently accurate for both computer animation generation and humanoid robot movement. As shown in FIG. 9, subject 150 moved each of the joint complexes (neck, left and right shoulders, elbows, wrists, hips, left and right hips, knees, and heels) to use all degrees of freedom. The recovered kinetic parameters were surprisingly stable. The joint positions that were automatically recovered were more confident than those measured manually. This is because for many joints, it is difficult to determine the actual position of the joint in the body. From Table I, the actual values of one subject who exercised the upper body freedom four timesVIShown in
[0096]
[Table 1]

[0097]
[Table 2]

[0098]
[Table 3]

[0099]
[Table 4]

[0100]
[Table 5]

[0101]
[Table 1]

[0102]
The model obtained by the method of the present invention depends on the basic configuration taken by the subject at the beginning of the data collection process and the arrangement of two special markers. Clearly, even for the same subject, the data is slightly different for each data collection process. However, the parameters obtained in different sessions were comparable and the model could be used again without having to express all the degrees of freedom each time.
[0103]
Although the invention has been described with reference to specific embodiments, the invention is not limited thereto. The scope of the invention is limited only by the claims.
[Brief description of the drawings]
FIG. 1 is a photograph showing an example of a humanoid robot.
FIG. 2 is a schematic diagram of a parametric dynamic model.
FIG. 3 is a schematic diagram of a system for realizing an embodiment of the present invention.
FIG. 4 is a perspective view of a computer system in which an embodiment of the present invention is implemented.
FIG. 5 is a structural diagram of the computer system shown in FIG. 4;
FIG. 6 is a flowchart showing overall control of the operation of the system according to one embodiment of the present invention.
FIG. 7 is a flowchart of an optimization process necessary for expansion or reduction of a parametric model.
FIG. 8 is a schematic diagram of a joint chain of a parametric dynamic model used in an embodiment of the present invention.
FIG. 9 is a photographic diagram illustrating a motion capture process.
[Explanation of symbols]
  2robot4, 6, 8, 10, 12 joints, 20 computer systems, 30 optical trackers, 40 computers

Claims (10)

A computer-implemented method for automatically adapting a parametric dynamic model for motion capture to the size of a human actor,
Prepares a set of computer-readable position data for each human body coordinate system and the position and orientation of the local body part coordinate system relative to the global body coordinate system for markers attached to the body parts of human actors Position data preparation steps to be performed;
Providing a first set of computer readable motion capture data representing the position of a marker attached to a body part of a human actor, wherein the motion capture data is a set of human actors captured while performing motion, further by using the first set of pairs and before liver Activation capture data of the position data, to estimate the position of the joint axis of a human actor's body parts in the computer Including steps,
Causing the computer to estimate:
Using a set of the position data, a first set of motion capture data, and a set of positions of the joint axes at the location preceding in said chain of parametric dynamic model, the parametric dynamic model comprises the step of automatically performing the current step of calculating a set of positions of the joint axes to the computer in the chain, the set of the joint axis at a location preceding by the the chain is preceded by the the chain Placed in the dynamic chain in such a way that the movement around the joint shaft set in place affects the position of the current joint shaft set ,
Said step of calculating a set of positions of the current joint axis for (a) all said measurement time of the first set of motion capture data, the position and orientation of the body human actors at each measurement time, and if any, a set of joint angle of the axis at the location preceding by a chain of at that time, a set of the position data, a first set of motion capture data, calculated in the preceding step, if any Using the position of the set of joint axes at a preceding location in the chain, and
(B) a set of said position data, said a first set of motion capture data, calculated in the preceding step, if any, and the set position of the joint axis at a location preceding by said in the chain, the body And the joint angle associated with the set of joint axes at the preceding location in the chain calculated in step (a), if any, and the current set of joint axes in the chain. the position of, and estimating the angle of the associated joint axis, methods. Wherein step (a) includes a first set of motion capture data, a set of the position data, the position and orientation of the body, a set of joint angle of the joint axis at a location preceding by said in the chain, by optimizing the preferred indicator function defined preceding by a set position of the joint axis at a location preceding by said in the chain, which is calculated in step, the joint axis at a location preceding by said in the chain The method of claim 1 , comprising estimating a set of joint angles. Wherein step (b) includes a first set of motion capture data, a set of pre-Symbol position location data, the location and orientation of the body, the joint axis at a location preceding by the the chain set and the current A suitable index function defined by the joint angle of the joint axis set and the position of the joint axis set at the preceding location in the chain calculated in the preceding step and the position of the current joint axis set. 3. A method according to claim 1 or claim 2 , comprising estimating the position of the current set of joint axes by optimizing. It said position data preparation step, during which the human actor is at the reference posture predetermined motion capture data computer readable represents the position of a marker attached to a human actor's body parts first comprising the step of preparing the second set, the method according to any one of claims 1 to 3. The method according to claim 4 , wherein, in the predetermined reference posture, the human actor stands in an upright posture with limbs extended. A computer-executable program for automatically adapting a parametric dynamic model to the size of a human actor for motion capture,
Prepares a set of computer-readable position data for each human body coordinate system and the position and orientation of the local body part coordinate system relative to the global body coordinate system for markers attached to the body parts of human actors The first program part to do,
A second program portion for preparing a first set of computer readable motion capture data representing the position of a marker attached to a body part of a human actor, the motion capture data comprising: Captured during a set of movements, and
Comprises a third program portion for estimating the position of the joint axis of a human actor's body part by using the first set of the set with the motion capture data of the position data, the third subprogram
A set of said position data, said a first set of motion capture data, with a set of positions of the joint axes at the location preceding in the chain of parametric dynamic model, parametric dynamic model chain It includes a fourth program portion for calculating a set of positions of the current joint axis in the inner set of joint axis at the location preceding by the in chain, the set of joint axes at the location preceding by said in the chain A computer-executable program arranged in a dynamic chain in such a way that the surrounding movement affects the position of the current set of joint axes ,
The fourth program part is
For all measurement time of the first set the motion capture data, the position and orientation of the body human actors at each measurement time, and if any, at the location preceding by said in the chain at that time a set of joint angle of the axis, a set of the position data, a first set of motion capture data, place some of the joint axis set of the preceding in the in the chain, which is calculated in the preceding step, if any A program part (a) to be estimated using the position;
A set of said position data, a first set of motion capture data, a set of positions of the joint axes at the location preceding by said in the chain, which is calculated in the preceding step, if any, the position and orientation of the body When, if any, by using the joint angles associated with the set of joint shaft at a location preceding by said in the chain, which is calculated by the program part (a), the current in the the chain joint axis pairs position and includes a program portion for estimating an associated joint angle (b), a program executable by a computer. Said program portion (a) includes a first set of motion capture data, a set of the position data, the position and orientation of the body, a set of joint angle of the joint axis at a location preceding by said in the chain , by optimizing a suitable indicator function defined by a set of positions of the joint axes at the location preceding by said in the chain, which is calculated in the fourth program portion, the position and orientation of the body, the linkage 7. The computer-executable program according to claim 6 , comprising a program part for estimating a joint angle of a set of joint axes at a preceding location therein . The program part (b) comprises a first set of motion capture data, a set of the position data, the position and orientation of the body, the set and the current joint of the joint axis at a location preceding by said in the chain a set of joint angle of the shaft, suitable indicator function defined by a set of positions of a set of locations and the current joint axis of the joint axis at a location that precedes in the fourth program part-calculated above in the chain The computer-executable program according to claim 6 or 7 , comprising a program part for estimating the position of the current set of joint axes by optimizing. The first program portion prepares a second set of motion capture data representing the positions of markers attached to the body parts of the human actor while the human actor is in a predetermined reference posture. A computer-executable program according to any one of claims 6 to 8 , comprising a program part to be executed. The computer-executable program according to claim 9 , wherein in the reference posture, the human actor stands in an upright posture with limbs extended.

Images