JP2004130426A - Robot device and its operation control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a moving body in a picked-up image by simple technique to allow tracking. <P>SOLUTION: A difference image forming module 110 generates difference image data by taking difference between adjacent frames on a time axis on image data picked up by a CCD camera 22 and provided by a virtual robot 43. A center-of-gravity computing module 111 computes the center-of-gravity position of the difference image data and sends the information to an action model library 90. The action model library 90 determines action facing or approaching the center-of-gravity position detected immediately before when the moving body disappears from the difference image data, for instance. A tracking signal processing module 73 sends a servo command value for manifesting action, to corresponding actuators 28<SB>1</SB>-28<SB>n</SB>through the virtual robot 43 or the like based on an action command from an action changeover module 91. <P>COPYRIGHT: (C)2004,JPO

Description

【００６６】
なお、各認識結果や出力セマンティクスコンバータモジュール７９からの通知が各情動のパラメータ値の変動量△Ｅ［ｔ］にどの程度の影響を与えるかは予め決められており、例えば「叩かれた」といった認識結果は「怒り」の情動のパラメータ値の変動量△Ｅ［ｔ］に大きな影響を与え、「撫でられた」といった認識結果は「喜び」の情動のパラメータ値の変動量△Ｅ［ｔ］に大きな影響を与えるようになっている。
【００６７】
ここで、出力セマンティクスコンバータモジュール７９からの通知とは、いわゆる行動のフィードバック情報（行動完了情報）であり、行動の出現結果の情報であり、感情モデル９３は、このような情報によっても感情を変化させる。これは、例えば、「吠える」といった行動により怒りの感情レベルが下がるといったようなことである。なお、出力セマンティクスコンバータモジュール７９からの通知は、上述した学習モジュール９２にも入力されており、学習モジュール９２は、その通知に基づいて行動モデル９０_１〜９０_ｎの対応する遷移確率を変更する。
【００６８】
なお、行動結果のフィードバックは、行動切換モジュール９１の出力（感情が付加された行動）によりなされるものであってもよい。
【００６９】
一方、本能モデル９４は、「運動欲（ｅｘｅｒｃｉｓｅ）」、「愛情欲（ａｆｆｅｃｔｉｏｎ）」、「食欲（ａｐｐｅｔｉｔｅ）」及び「好奇心（ｃｕｒｉｏｓｉｔｙ）」の互いに独立した４つの欲求について、これら欲求ごとにその欲求の強さを表すパラメータを保持している。そして、本能モデル９４は、これらの欲求のパラメータ値を、それぞれ入力セマンティクスコンバータモジュール７１から与えられる認識結果や、経過時間及び行動切換モジュール９１からの通知などに基づいて周期的に更新する。
【００７０】
具体的には、本能モデル９４は、「運動欲」、「愛情欲」及び「好奇心」については、認識結果、経過時間及び出力セマンティクスコンバータモジュール６８からの通知などに基づいて所定の演算式により算出されるそのときのその欲求の変動量をΔＩ［ｋ］、現在のその欲求のパラメータ値をＩ［ｋ］、その欲求の感度を表す係数ｋ_ｉとして、所定周期で（２）式を用いて次の周期におけるその欲求のパラメータ値Ｉ［ｋ＋１］を算出し、この演算結果を現在のその欲求のパラメータ値Ｉ［ｋ］と置き換えるようにしてその欲求のパラメータ値を更新する。また、本能モデル９４は、これと同様にして「食欲」を除く各欲求のパラメータ値を更新する。
【００７１】
【数２】

【００７２】
なお、認識結果及び出力セマンティクスコンバータモジュール７９からの通知などが各欲求のパラメータ値の変動量△Ｉ［ｋ］にどの程度の影響を与えるかは予め決められており、例えば出力セマンティクスコンバータモジュール７９からの通知は、「疲れ」のパラメータ値の変動量△Ｉ［ｋ］に大きな影響を与えるようになっている。
【００７３】
なお、本実施の形態においては、各情動及び各欲求（本能）のパラメータ値がそれぞれ０から１００までの範囲で変動するように規制されており、また係数ｋ_ｅ、ｋ_ｉの値も各情動及び各欲求ごとに個別に設定されている。
【００７４】
一方、ミドル・ウェア・レイヤ５０の出力セマンティクスコンバータモジュール７９は、図４に示すように、上述のようにしてアプリケーション・レイヤ５１の行動切換モジュール９１から与えられる「前進」、「喜ぶ」、「鳴く」又は「トラッキング（ボールを追いかける）」といった抽象的な行動コマンドを出力系８０の対応する信号処理モジュール７２〜７８に与える。
【００７５】
そしてこれら信号処理モジュール７２〜７８は、行動コマンドが与えられると当該行動コマンドに基づいて、その行動を行うために対応するアクチュエータ２８_１〜２８_ｎ（図２）に与えるべきサーボ指令値や、スピーカ２０（図２）から出力する音の音声データ及び／又は発光部２５（図２）のＬＥＤに与える駆動データを生成し、これらのデータをロボティック・サーバ・オブジェクト４２のバーチャル・ロボット４３及び信号処理回路１４（図２）を順次介して対応するアクチュエータ２８_１〜２８_ｎ、スピーカ２０又は発光部２５に順次送出する。
【００７６】
このようにしてロボット装置１においては、制御プログラムに基づいて、自己（内部）及び周囲（外部）の状況や、使用者からの指示及び働きかけに応じた自律的な行動を行うことができるようになされている。
【００７７】
（３）ロボット装置への本発明の適用
以下、本発明を上述したような構造を有するロボット装置１に適用した例について説明する。上述したように、本実施の形態におけるロボット装置１は、ＣＣＤカメラ２２（図２）によって撮像した画像データ内の動体を検出し、検出した動体の方向を向く、或いはトラッキングするなどといった行動を行う。
【００７８】
ここで、フレーム間の差分画像を生成した場合、動体の動きが停止した時点で差分値が０となる。例えば、図９に示すように、それぞれ時刻ｔ_１〜ｔ_４における人間を撮像した画像データＰ_１〜Ｐ_４について差分画像データＤ_１〜Ｄ_３を生成した場合、時刻ｔ_３及びｔ_４間で顔が静止していると、差分画像データＤ_３から顔の差分データが消失してしまう。つまり、差分画像データから動体が消失したということは、動体がその場から消失したのではなく、消えた場所に動体が存在するということを意味している。
【００７９】
そこで、本実施の形態におけるロボット装置１は、この差分が０となる時点を検出し、その直前の差分画像における重心位置の方向にＣＣＤカメラ２２を向けることで重心位置の方向を向き、又は重心位置の方向に近付く。すなわち、図１０のフローチャートに示すように、先ずステップＳ１において、差分画像データの重心位置を計算することで動体を検出し、ステップＳ２において、検出した動体が差分画像データから消失したか否かが判別される。ステップＳ２において動体が消えていない場合（Ｎｏ）にはステップＳ１に戻る。一方、ステップＳ２において動体が消えた場合（Ｙｅｓ）にはステップＳ３に進み、消失した方向、すなわち直前の差分画像における重心位置の方向を向く、或いはその重心位置の方向に近付く。
【００８０】
なお、検出した動体がロボット装置１の視覚範囲から外れた場合にも差分画像から動体が消失するが、この場合にも上述のステップＳ３において最後に検出された重心位置の方向を向くことで、ほぼ動体の方向を向くことができる。
【００８１】
このように、本実施の形態におけるロボット装置１は、視覚範囲内で動体が静止したことにより差分画像データから消失するのを検出し、その重心位置の方向を向くようにすることで、例えば人間等の動体の気配を感じてその方向を向くという自律的なインタラクションを実現できる。また、動体が視覚範囲から外れたことにより差分画像データから消失するのを検出し、最後に検出された重心位置の方向を向くようにすることで、ほぼ動体の方向を向くことができる。
【００８２】
また、ロボット装置１は、差分画像データから動体が消失した場合のみならず、所定の時間間隔毎、或いは動体の重心位置が視覚範囲から外れそうになる毎に検出された重心方向を向き、動体をトラッキングするようにしても構わない。すなわち、図１１のフローチャートに示すように、先ずステップＳ１０において、差分画像データの重心位置を計算することで動体を検出し、ステップＳ１１において、所定の時間間隔毎、或いは動体が視覚範囲から外れそうになる毎に検出された重心位置の方向を向く。
【００８３】
ここで、ロボット装置１は、前述のように差分画像データから動体が消失した場合の他、ステップＳ１１におけるロボット装置１の動きが大きい場合には、動き補償によって自己の動きと動体の動きとを区別することができなくなり、動体を見失ってしまう。そこでステップＳ１２において、動体を見失ったか否かが判別される。ステップＳ１２において動体を見失っていない場合（Ｎｏ）にはステップＳ１０に戻る。一方、ステップＳ１２において動体を見失った場合（Ｙｅｓ）にはステップＳ１３に進み、最後に検出された重心位置の方向を向く。
【００８４】
このように、本実施の形態におけるロボット装置１は、所定の時間間隔毎、或いは動体が視覚範囲から外れそうになる毎に検出された重心方向を向き、動体を見失った場合に最後に検出された重心位置の方向を向くようにすることで、頭部ユニット４に設けられたＣＣＤカメラ２２によって撮像した画像内の動体を簡易な手法により検出し、トラッキングすることが可能となる。
【００８５】
以上の処理は、図４に示したミドル・ウェア・レイヤ５０の認識系７１における動き検出用信号処理モジュール６８を、差分画像生成モジュール１１０と重心計算モジュール１１１とによって構成することで実現される。
【００８６】
すなわち、図１２に示すように、ロボティック・サーバ・オブジェクト４２のバーチャル・ロボット４３は、ＣＣＤカメラ２２によって撮像されたフレーム単位の画像データをＤＲＡＭ１１から読み出し、この画像データをミドル・ウェア・レイヤ５０の認識系７１に含まれる差分画像生成モジュール１１０に送出する。
【００８７】
差分画像生成モジュール１１０は、画像データを入力する毎に時間軸上で隣接する前フレームの画像データとの差分をとって差分画像データを生成し、この差分画像データを重心計算モジュール１１１に与える。例えば、上述した画像データＰ_２と画像データＰ_３との差分画像データＤ_２を生成する場合、位置（ｉ，ｊ）における差分画像データＤ_２の輝度値Ｄ_２（ｉ，ｊ）は、位置（ｉ，ｊ）における画像データＰ_３の輝度値Ｐ_３（ｉ，ｊ）から同位置における画像データＰ_２の輝度値Ｐ_２（ｉ，ｊ）を減算することで得られる。差分画像生成モジュール１１０は、全画素について同様の計算を行って差分画像データＤ_２を生成し、この差分画像データＤ_２を重心計算モジュール１１１に与える。
【００８８】
そして、重心計算モジュール１１１は、差分画像データのうち、輝度値が閾値Ｔｈよりも大きい部分についての重心位置Ｇ（ｘ，ｙ）を計算する。ここで、ｘ、ｙは、それぞれ以下の（３）式、（４）式を用いて計算される。
【００８９】
【数３】

【００９０】
これにより、図１３に示すように、例えば上述した画像データＰ_２と画像データＰ_３との差分画像データＤ_２から、重心位置Ｇ_２が求められる。
【００９１】
重心計算モジュール１１１は、求めた重心位置のデータをアプリケーション・レイヤ５１の行動モデルライブラリ９０に送出する。
【００９２】
行動モデルライブラリ９０は、必要に応じて情動のパラメータ値や欲求のパラメータ値を参照しながら続く行動を決定し、決定結果を行動切換モジュール９１に与える。例えば、差分画像データから動体が消失した場合には、直前に検出された重心位置を向く、或いは近付く行動を決定し、決定結果を行動切換モジュール９１に与える。また、所定の時間間隔毎に動体をトラッキングする場合には、その時間間隔毎に検出された重心位置を向く、或いは近付く行動を決定し、決定結果を行動切換モジュール９１に与える。そして、行動切換モジュール９１は、当該決定結果に基づく行動コマンドをミドル・ウェア・レイヤ５０の出力系８０におけるトラッキング用信号処理モジュール７３に送出する。
【００９３】
トラッキング用信号処理モジュール７３は、行動コマンドが与えられると当該行動コマンドに基づいて、その行動を行うために対応するアクチュエータ２８_１〜２８_ｎに与えるべきサーボ指令値を生成し、このデータをロボティック・サーバ・オブジェクト４２のバーチャル・ロボット４３及び信号処理回路１４（図２）を順次介して対応するアクチュエータ２８_１〜２８_ｎに順次送出する。
【００９４】
この結果、例えば、差分画像データから動体が消失した場合には、行動モデルライブラリ９０によって、直前に検出された重心位置を向く、或いは近付く行動が決定され、行動切換モジュール９１によって、その行動を行わせるための行動コマンドが生成される。また、所定の時間間隔毎に動体をトラッキングする場合には、行動モデルライブラリ９０によって、その時間間隔毎に検出された重心位置を向く、或いは近付く行動が決定され、行動切換モジュール９１によって、その行動を行わせるための行動コマンドが生成される。
【００９５】
そして、この行動コマンドがトラッキング用信号処理モジュール７３に与えられると、当該トラッキング用信号処理モジュール７３は、その行動コマンドに基づくサーボ指令値を対応するアクチュエータ２８_１〜２８_ｎに送出し、これによりロボット装置１が動体に興味を示して頭部をその方向に向けたり、動体の方向に近付いたりする行動が発現される。
【００９６】
なお、本発明は上述した実施の形態のみに限定されるものではなく、本発明の要旨を逸脱しない範囲において種々の変更が可能であることは勿論である。
【００９７】
例えば、上述の実施の形態では、ＣＣＤカメラ２２によって撮像した画像データ中の動体を検出する処理をソフトウェアによって実現するものとして説明したが、これに限定されるものではなく、少なくとも一部をハードウェアによって実現するようにしてもよい。
【００９８】
【発明の効果】
以上詳細に説明したように本発明に係るロボット装置は、外部刺激及び／又は内部状態に応じて自律的に行動する自律型のロボット装置において、少なくとも外部の動体を撮像する撮像手段と、上記撮像手段によって撮像された時間軸上で隣接する画像間の差分画像を生成する差分画像生成手段と、上記差分画像の重心位置を計算する重心計算手段と、少なくとも上記重心位置に基づいて行動を生成する行動生成手段とを備え、上記行動生成手段は、上記差分画像中の動体が消失した場合に、上記差分画像の最後に検出された重心位置の方向に上記撮像手段を向ける行動を生成する。
【００９９】
また、本発明に係るロボット装置は、外部刺激及び／又は内部状態に応じて自律的に行動する自律型のロボット装置において、少なくとも外部の動体を撮像する撮像手段と、上記撮像手段によって撮像された時間軸上で隣接する画像間の差分画像を生成する差分画像生成手段と、上記差分画像の重心位置を計算する重心計算手段と、少なくとも上記重心位置に基づいて行動を生成する行動生成手段とを備え、上記行動生成手段は、所定の時間間隔毎又は上記差分画像の重心位置が当該差分画像から外れそうになる毎に上記差分画像の重心位置の方向に上記撮像手段を向ける行動を生成すると共に、上記差分画像中の動体を見失った場合には、上記差分画像の最後に検出された重心位置の方向に上記撮像手段を向ける行動を生成する。
【０１００】
このようなロボット装置は、撮像手段によって撮像された時間軸上で隣接する画像間の差分画像の重心位置を計算し、差分画像中の動体が消失した場合に、最後に検出された重心位置の方向に上記撮像手段を向ける。
【０１０１】
また、ロボット装置は、撮像手段によって撮像された時間軸上で隣接する画像間の差分画像の重心位置を計算し、所定の時間間隔毎又は差分画像の重心位置が当該差分画像から外れそうになる毎に上記差分画像の重心位置の方向に撮像手段を向けると共に、上記差分画像中の動体を見失った場合には、最後に検出された重心位置の方向に上記撮像手段を向ける。
【０１０２】
これにより、撮像手段により撮像した画像内の動体を簡易な手法により検出し、トラッキングすることが可能となる。
【０１０３】
また、本発明に係るロボット装置の動作制御方法は、外部刺激及び／又は内部状態に応じて自律的に行動する自律型のロボット装置の動作制御方法において、撮像手段により少なくとも外部の動体を撮像する撮像工程と、上記撮像工程にて撮像された時間軸上で隣接する画像間の差分画像を生成する差分画像工程と、上記差分画像の重心位置を計算する重心計算工程と、少なくとも上記重心位置に基づいて行動を生成する行動生成工程とを有し、上記行動生成工程では、上記差分画像中の動体が消失した場合に、上記差分画像の最後に検出された重心位置の方向に上記撮像手段を向ける行動が生成される。
【０１０４】
また、本発明に係るロボット装置の動作制御方法は、外部刺激及び／又は内部状態に応じて自律的に行動する自律型のロボット装置の動作制御方法において、撮像手段により少なくとも外部の動体を撮像する撮像工程と、上記撮像工程にて撮像された時間軸上で隣接する画像間の差分画像を生成する差分画像工程と、上記差分画像の重心位置を計算する重心計算工程と、少なくとも上記重心位置に基づいて行動を生成する行動生成工程とを有し、上記行動生成工程では、所定の時間間隔毎又は上記差分画像の重心位置が当該差分画像から外れそうになる毎に上記差分画像の重心位置の方向に上記撮像手段を向ける行動が生成されると共に、上記差分画像中の動体が消失した場合には、上記差分画像の最後に検出された重心位置の方向に上記撮像手段を向ける行動が生成される。
【０１０５】
このようなロボット装置の動作制御方法では、撮像手段によって撮像された時間軸上で隣接する画像間の差分画像の重心位置が計算され、差分画像中の動体が消失した場合に、最後に検出された重心位置の方向に上記撮像手段が向けられる。
【０１０６】
また、ロボット装置の動作制御方法では、撮像手段によって撮像された時間軸上で隣接する画像間の差分画像の重心位置が計算され、所定の時間間隔毎又は差分画像の重心位置が当該差分画像から外れそうになる毎に上記差分画像の重心位置の方向に撮像手段が向けられると共に、上記差分画像中の動体を見失った場合には、最後に検出された重心位置の方向に上記撮像手段が向けられる。
【０１０７】
これにより、撮像手段により撮像した画像内の動体を簡易な手法により検出し、トラッキングすることが可能となる。
【図面の簡単な説明】
【図１】本発明の実施の形態におけるロボット装置の外観構成を示す斜視図である。
【図２】同ロボット装置の回路構成を示すブロック図である。
【図３】同ロボット装置のソフトウェア構成を示すブロック図である。
【図４】同ロボット装置のソフトウェア構成におけるミドル・ウェア・レイヤの構成を示すブロック図である。
【図５】同ロボット装置のソフトウェア構成におけるアプリケーション・レイヤの構成を示すブロック図である。
【図６】同アプリケーション・レイヤの行動モデルライブラリの構成を示すブロック図である。
【図７】同ロボット装置の行動決定のための情報となる有限確率オートマトンを説明する図である。
【図８】有限確率オートマトンの各ノードに用意された状態遷移表を示す図である。
【図９】差分画像データから動体が消える例を説明する図である。
【図１０】差分画像データから動体が消失した方向を向く場合の手順を説明するフローチャートである。
【図１１】動体をトラッキングする場合の手順を説明するフローチャートである。
【図１２】同ロボット装置の動体検出に関連する部分のソフトウェア構成を示すブロック図である。
【図１３】差分画像データの重心位置を求める例を説明する図である。
【符号の説明】
１　ロボット装置、１０　ＣＰＵ、１１　ＤＲＡＭ、１４　信号処理回路、２２　ＣＣＤカメラ、２８_１〜２８_ｎ　アクチュエータ、４２　ロボティック・サーバ・オブジェクト、４３　バーチャル・ロボット、５０　ミドル・ウェア・レイヤ、５１　アプリケーション・レイヤ、６８　動き検出用信号処理モジュール、７０　入力セマンティクスコンバータモジュール、７１　認識系、７３　トラッキング用信号処理モジュール、７９　出力セマンティクスコンバータモジュール、８０　出力系、９０　行動モデルライブラリ、９１　行動切換モジュール、１１０　差分画像生成モジュール、１１１　重心計算モジュール[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an autonomous robot apparatus that directs or tracks a moving object in an external environment and an operation control method thereof.
[0002]
[Prior art]
A mechanical device that performs a motion similar to the motion of a human (living organism) using an electric or magnetic action is called a “robot”. Robots have begun to spread in Japan since the late 1960s, and most of them have been industrial robots (Industrial Robots) such as manipulators and transfer robots for the purpose of automation and unmanned production work in factories. Met.
[0003]
Recently, practical robots have been developed which support life as a human partner, that is, support human activities in various situations in a living environment and other daily lives. Unlike an industrial robot, such a practical robot has the ability to learn a human being having different personalities individually or a method of adapting to various environments in various aspects of a human living environment. For example, a "pet-type" robot that simulates the body mechanism and operation of a four-legged animal such as a dog or a cat, or a model designed based on the body mechanism or motion of an animal that walks upright on two legs. Legged mobile robots, such as "humanoid" or "humanoid" robots, are already being put into practical use.
[0004]
Since these legged mobile robots can perform various operations that emphasize entertainment properties as compared with industrial robots, they may be referred to as entertainment robots.
[0005]
The legged mobile robot has an appearance shape as close as possible to the appearance of animals and humans, and is designed to perform an operation as close as possible to the motion of animals and humans. For example, in the case of the above-mentioned four-legged “pet-type” robot, it has an appearance similar to that of a dog or a cat bred in a general household, and the user (owner) makes an action such as “hit” or “stroke”. And act autonomously according to the surrounding environment. For example, as an autonomous action, an action such as “barking” or “sleeping” is performed in the same manner as an actual animal.
[0006]
By the way, some legged mobile robots determine the surrounding situation based on an image captured by a CCD (Charge Coupled Device) camera, and act autonomously based on the determination result. According to such a legged mobile robot, for example, a moving object existing in the surroundings can be detected, and the moving object can be tracked.
[0007]
Here, as a method of detecting a moving object in an image by image processing, various methods have been conventionally proposed and put into practical use. These methods are roughly classified into a first method of obtaining a pixel difference between frame images, a second method of obtaining an optical flow at an arbitrary point on an image using a spatio-temporal filter, and a small region between frames. And a third method of obtaining the motion of the area. These methods are based on the premise that a camera for capturing an image is fixed and the detected movement is an external movement.
[0008]
However, when the robot apparatus itself operates like the above-mentioned legged mobile robot, the mounted CCD camera moves large and complicated with its own operation. For this reason, when trying to detect a moving object in an image by the above-described first to third methods, the entire image greatly moves between frames. Therefore, even if the moving object is actually imaged, the movement is detected. Was difficult.
[0009]
In view of this, the applicant of the present application has proposed, in Patent Literature 1 below, a technique for accurately detecting a moving object existing in the surroundings regardless of the presence or absence of its own operation. According to the technology described in Patent Literature 1, an image region having a high correlation with the center portion of the previous frame image is extracted from the current frame image and the movement of the entire image is detected. It is possible to accurately detect a moving object in an image regardless of the presence or absence of movement.
[0010]
[Patent Document 1]
JP 2002-251615 A
[0011]
[Problems to be solved by the invention]
However, in the technology described in Patent Document 1, in order to extract an image region having a high correlation with the central portion of the previous frame image, each template divided region obtained by dividing the central portion of the previous frame image and the current frame image are divided into predetermined pixels. It is necessary to sequentially calculate the matching score while shifting by one pixel in the horizontal direction and the vertical direction between each search image divided area divided while overlapping by an amount, and there is a problem that the calculation load increases. .
[0012]
Therefore, in a robot device having a limited computing capability, such as the above-mentioned pet-type robot, a technique for more easily detecting a moving object in an imaged image is desired.
[0013]
The present invention has been proposed in view of such a conventional situation, and it is an object of the present invention to provide a robot apparatus capable of detecting a moving object in a captured image by a simple method and enabling tracking, and a moving object detecting method thereof. Aim.
[0014]
[Means for Solving the Problems]
In order to achieve the above-described object, a robot device according to the present invention is an autonomous robot device that autonomously acts in response to an external stimulus and / or an internal state, and includes an imaging unit that captures at least an external moving object; Difference image generation means for generating a difference image between images adjacent on the time axis captured by the imaging means, gravity center calculation means for calculating a gravity center position of the difference image, and an action based on at least the gravity center position An action generating means for generating, when the moving object in the difference image has disappeared, the action generating means generates an action of directing the imaging means in the direction of the last detected center of gravity of the difference image. .
[0015]
Further, in order to achieve the above-described object, a robot apparatus according to the present invention is an autonomous robot apparatus that autonomously behaves in response to an external stimulus and / or an internal state. A difference image generation unit that generates a difference image between images adjacent on the time axis captured by the imaging unit, a center of gravity calculation unit that calculates a center of gravity of the difference image, and at least based on the position of the center of gravity. Action generating means for generating an action, wherein the action generating means is arranged in the direction of the center of gravity of the difference image at predetermined time intervals or each time the center of gravity of the difference image is likely to deviate from the difference image. In addition to generating an action of directing the imaging unit, if the moving object in the difference image is lost, the imaging unit is moved in the direction of the last detected center of gravity of the difference image. To generate a kick action.
[0016]
Here, the action generation means can also generate an action approaching the direction of the position of the center of gravity.
[0017]
The robot device has an external shape imitating an animal, for example, and the imaging means is provided on a head movably connected to the body.
[0018]
Such a robot device calculates the center of gravity of the difference image between the images adjacent on the time axis captured by the imaging unit, and when the moving object in the difference image disappears, calculates the center of gravity of the last detected center of gravity. Orient the imaging means in the direction.
[0019]
Further, the robot apparatus calculates the center of gravity of the difference image between the images adjacent on the time axis captured by the imaging unit, and the center of gravity of the difference image is likely to deviate from the difference image at every predetermined time interval or Each time the imaging means is directed in the direction of the center of gravity of the difference image, and when the moving object in the difference image is lost, the imaging means is directed in the direction of the last detected center of gravity.
[0020]
According to another aspect of the present invention, there is provided an operation control method for a robot apparatus according to the present invention, wherein the operation control method for an autonomous robot apparatus that autonomously acts in response to an external stimulus and / or an internal state includes an An imaging step of capturing at least an external moving body by means, a difference image step of generating a difference image between images adjacent on the time axis taken in the imaging step, and a center of gravity for calculating a center of gravity of the difference image A calculating step, and an action generating step of generating an action based on at least the position of the center of gravity. In the action generating step, when a moving object in the difference image has disappeared, the motion is detected at the end of the difference image. An action of pointing the imaging unit in the direction of the center of gravity is generated.
[0021]
According to another aspect of the present invention, there is provided an operation control method for a robot apparatus, the method comprising: An imaging step of capturing at least an external moving body by means, a difference image step of generating a difference image between images adjacent on the time axis taken in the imaging step, and a center of gravity for calculating a center of gravity of the difference image A calculating step and an action generating step of generating an action based on at least the position of the center of gravity, wherein in the action generating step, the center of gravity of the difference image is likely to deviate from the difference image at predetermined time intervals or Each time an action of pointing the imaging means in the direction of the center of gravity of the difference image is generated, and when the moving object in the difference image has disappeared, the detection is performed at the end of the difference image. Behavioral directing the imaging means in the direction of the gravity center position is generated.
[0022]
Here, in the action generation step, an action approaching the direction of the position of the center of gravity can be generated.
[0023]
In addition, the robot device has an external shape imitating an animal, for example, and the imaging unit is provided on a head movably connected to a body.
[0024]
In such an operation control method for a robot device, the position of the center of gravity of the difference image between the images adjacent on the time axis captured by the imaging unit is calculated, and when the moving object in the difference image disappears, it is detected last. The imaging means is directed in the direction of the position of the center of gravity.
[0025]
Further, in the operation control method of the robot apparatus, the position of the center of gravity of the difference image between adjacent images on the time axis captured by the imaging unit is calculated, and the center of gravity of the difference image is determined at predetermined time intervals or from the difference image. Each time it is likely to deviate, the imaging means is directed in the direction of the center of gravity of the difference image, and if the moving object in the difference image is lost, the imaging means is directed in the direction of the last detected center of gravity. Can be
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. This embodiment is an autonomous robot device that performs an autonomous action according to an external environment (or an external stimulus) or an internal state.
[0027]
In the embodiment, first, the configuration of the robot device will be described, and thereafter, a portion to which the present invention is applied in the robot device will be described in detail.
[0028]
(1) Configuration of robot device according to the present embodiment
As shown in FIG. 1, a robot device 1 according to the present embodiment is a so-called pet-type robot having a shape imitating an animal such as a “dog”, and has leg units 3A and 3B on the front, rear, left and right sides of a body unit 2. , 3C, and 3D, and a head unit 4 is connected to the front end of the body unit 2.
[0029]
As shown in FIG. 2, the body unit 2 includes a CPU (Central Processing Unit) 10, a DRAM (Dynamic Random Access Memory) 11, a flash ROM (Read Only Memory) 12, a PC (Personal Computer) card interface circuit 13, and the like. A control unit 16 formed by connecting the signal processing circuits 14 via an internal bus 15 and a battery 17 as a power source of the robot device 1 are housed therein. Further, the body unit 2 houses an angular velocity sensor 18 and an acceleration sensor 19 for detecting the acceleration of the direction and the movement of the robot apparatus 1. Further, a speaker 20 for outputting a sound such as a cry or a melody is arranged at a predetermined position in the body unit 2 as shown in FIG. The tail 5 of the body unit 2 is provided with an operation switch 21 as a detection mechanism for detecting an operation input from a user. The operation switch 21 is a switch that can detect the type of operation performed by the user, and the robot device 1 is, for example, “praised” or “reprimanded” according to the type of operation detected by the operation switch 21. Recognize.
[0030]
The head unit 4 includes a CCD (Charge Coupled Device) camera 22 for capturing an external situation and the color, shape, movement, and the like of the object, and a distance for measuring a distance to the object located in front. A sensor 23, a microphone 24 for collecting external sound, and a light emitting unit 25 provided with, for example, an LED (Light Emitting Diode) are arranged at predetermined positions as shown in FIG. However, the light emitting unit 25 is referred to as an LED 25 as necessary in the description of the configuration and the like. Although not shown in FIG. 1, a head switch 26 is provided inside the head unit 4 as a detection mechanism for indirectly detecting contact of the user with the head unit 4. The head switch 26 is, for example, a switch that can detect the tilt direction when the head is moved by a user's contact, and the robot apparatus 1 detects the tilt direction of the head detected by the head switch 26. Depending on, it recognizes whether it was "praised" or "reproached."
[0031]
Actuators 28 for the number of degrees of freedom are provided at joints of the leg units 3A to 3D, connecting portions of the leg units 3A to 3D and the body unit 2, and connecting portions of the head unit 4 and the body unit 2. ₁ ~ 28 _n And potentiometer 29 ₁ ~ 29 _n Are arranged respectively. Actuator 28 ₁ ~ 28 _n Has, for example, a servomotor. By driving the servomotor, the leg units 3A to 3D are controlled to transition to a target posture or operation. At positions corresponding to the "paws" at the tips of the leg units 3A to 3D, paws switches 27A to 27D are provided as detection mechanisms mainly for detecting contact from the user, and can detect contact and the like by the user. It has become.
[0032]
Although not shown here, the robot apparatus 1 also includes a light emitting unit for indicating an operation state (operation mode) different from the internal state of the robot apparatus 1, a charging operation, a starting operation, a start / stop operation, and the like. And a status lamp indicating the state of the internal power supply may be appropriately provided at an appropriate place.
[0033]
Then, in the robot apparatus 1, various switches such as the operation switch 21, the head switch 26, and the pad switch 27, various sensors such as the angular velocity sensor 18, the acceleration sensor 19, and the distance sensor 23, the speaker 20, the microphone 24, and the light emission Part 25, each actuator 28 ₁ ~ 28 _n , Each potentiometer 29 ₁ ~ 29 _n Are the corresponding hubs 30 ₁ ~ 30 _n Is connected to the signal processing circuit 14 of the control unit 16 via the. On the other hand, the CCD camera 22 and the battery 17 are directly connected to the signal processing circuit 14, respectively.
[0034]
The signal processing circuit 14 sequentially receives the switch data supplied from the various switches described above, the sensor data supplied from the various sensors, the image data, and the audio data, and stores them at predetermined positions in the DRAM 11 via the internal bus 15. Store sequentially. Further, the signal processing circuit 14 sequentially takes in the remaining battery level data indicating the remaining battery level supplied from the battery 17 and stores the data in a predetermined position in the DRAM 11.
[0035]
The switch data, sensor data, image data, audio data, and remaining battery data stored in the DRAM 11 in this manner are used when the CPU 10 controls the operation of the robot device 1.
[0036]
The CPU 10 reads the control program stored in the flash ROM 12 and stores the control program in the DRAM 11 at the initial stage when the power of the robot apparatus 1 is turned on. Alternatively, the CPU 10 reads out a control program stored in a semiconductor memory device, for example, a memory card 31 mounted on a PC card slot of the body unit 2 not shown in FIG. Store.
[0037]
The CPU 10, based on the sensor data, image data, audio data, and remaining battery data sequentially stored in the DRAM 11 from the signal processing circuit 14 as described above, the situation of itself and the surroundings, and instructions and actions from the user. Is determined.
[0038]
Further, the CPU 10 determines the subsequent action based on the result of the determination and the control program stored in the DRAM 11, and determines the necessary actuator 28 based on the determined result. ₁ ~ 28 _n , The head unit 4 is swung up and down, left and right, and the leg units 3A to 3D are driven to walk. In particular, in the present embodiment, as described later, a moving object in image data is detected, and a necessary actuator 28 is detected based on the detection result. ₁ ~ 28 _n Is driven to perform an action such as pointing in the direction of the moving object or performing tracking.
[0039]
Further, at this time, the CPU 10 generates audio data as necessary, and supplies the generated audio data to the speaker 20 as an audio signal via the signal processing circuit 14 so that the audio based on the audio signal is output to the outside. A signal for instructing lighting and extinguishing of the LED in the light emitting unit 25 is generated, and the light emitting unit 25 is turned on and off.
[0040]
In this way, the robot apparatus 1 can autonomously act according to the situation of itself and the surroundings, and instructions and actions from the user.
[0041]
(2) Software configuration of control program
Here, the software configuration of the above-described control program in the robot device 1 is as shown in FIG. In FIG. 3, the device driver layer 40 is located at the lowest layer of the control program, and includes a device driver set 41 including a plurality of device drivers. In this case, each device driver is an object permitted to directly access hardware used in a normal computer, such as a CCD camera 22 (FIG. 2) and a timer, and receives an interrupt from the corresponding hardware. Perform processing.
[0042]
The robotic server object 42 is located at the lowest layer of the device driver layer 40, and includes, for example, the various sensors and actuators 28 described above. ₁ ~ 28 _n A virtual robot 43, which is a software group that provides an interface for accessing hardware such as a virtual robot, a power manager 44, which is a software group that manages switching of power supplies, and software that manages various other device drivers It comprises a group of device driver managers 45 and a designed robot 46 which is a group of software for managing the mechanism of the robot apparatus 1.
[0043]
The manager object 47 includes an object manager 48 and a service manager 49. The object manager 48 is a software group that manages activation and termination of each software group included in the robotic server object 42, the middleware layer 50, and the application layer 51, and the service manager 49 This is a group of software for managing the connection of each object based on the connection information between each object described in the connection file stored in the memory card 31 (FIG. 2).
[0044]
The middleware layer 50 is located on the upper layer of the robotic server object 42 and is composed of a software group that provides basic functions of the robot device 1 such as image processing and sound processing. Further, the application layer 51 is located on the upper layer of the middleware layer 50, and determines the action of the robot device 1 based on the processing result processed by each software group constituting the middleware layer 50. It consists of a group of software for performing
[0045]
FIG. 4 shows specific software configurations of the middleware layer 50 and the application layer 51.
[0046]
As shown in FIG. 4, the middle wear layer 50 includes noise detection, temperature detection, brightness detection, scale recognition, distance detection, posture detection, contact detection, operation input detection, and motion detection. A recognition system 71 having signal processing modules 60 to 69 for detection and color recognition, an input semantics converter module 70, and the like, an output semantics converter module 79, and posture management, tracking, motion reproduction, walking, and fallback. And an output system 80 having respective signal processing modules 72 to 78 for lighting, LED lighting and sound reproduction.
[0047]
Each of the signal processing modules 60 to 69 of the recognition system 71 transmits corresponding sensor data, image data, and sound data read from the DRAM 11 (FIG. 2) by the virtual robot 43 of the robotic server object 42. The input semantics converter module 70 takes in the data, performs predetermined processing based on the data, and provides the processing result to the input semantics converter module 70. Here, for example, the virtual robot 43 is configured as a part that exchanges or converts signals according to a predetermined communication protocol.
[0048]
The input semantics converter module 70 detects “noisy”, “hot”, “bright”, “sounds of domeso”, and “obstacle” based on the processing results given from each of the signal processing modules 60 to 69. The situation of self and surroundings, such as "has detected", "detected a fall", "reprimanded", "praised", "detected a moving object", or "detected a ball", or a command from the user And recognizes the action and outputs the recognition result to the application layer 51 (FIG. 2).
[0049]
As shown in FIG. 5, the application layer 51 includes five modules: a behavior model library 90, a behavior switching module 91, a learning module 92, an emotion model 93, and an instinct model 94.
[0050]
As shown in FIG. 6, the behavior model library 90 includes “when the remaining battery power is low”, “returns to fall”, “when avoids obstacles”, “when expressing emotions”, Are respectively associated with several pre-selected condition items such as "when a ₁ ~ 90 _n Is provided.
[0051]
And these behavior models 90 ₁ ~ 90 _n Are stored in the emotion model 93 as described later, as necessary, when a recognition result is provided from the input semantics converter module 71 or when a certain time has elapsed since the last recognition result was provided. The subsequent action is determined with reference to the corresponding emotion parameter value and the corresponding desire parameter value held in the instinct model 94, and the determination result is output to the action switching module 91.
[0052]
In this embodiment, each behavior model 90 ₁ ~ 90 _n Is a one node (state) NODE as shown in FIG. ₀ ~ NODE _n From any other node NODE ₀ ~ NODE _n To each node NODE ₀ ~ NODE _n Arc ARC connecting between ₁ ~ ARC _n Transition probability P set for ₁ ~ P _n An algorithm called finite stochastic automaton, which determines stochastically based on, is used.
[0053]
Specifically, each behavior model 90 ₁ ~ 90 _n Are their own behavior models 90 ₁ ~ 90 _n NODE that forms ₀ ~ NODE _n Corresponding to each of these nodes NODE ₀ ~ NODE _n Each has a state transition table 100 as shown in FIG.
[0054]
In this state transition table 100, the node NODE ₀ ~ NODE _n , Input events (recognition results) as transition conditions are listed in order of priority in the column of “input event name”, and further conditions for the transition condition are described in corresponding rows in the columns of “data name” and “data range”. Have been.
[0055]
Therefore, the node NODE represented by the state transition table 100 of FIG. ₁₀₀ In the above, when the recognition result of “detection of ball (BALL)” is given, the “size” of the ball given together with the recognition result is in the range of “0 to 1000”, When a recognition result of “obstacle detected (OBSTACLE)” is given, the other node that the “distance” to the obstacle given together with the recognition result is in the range of “0 to 100”. This is the condition for transitioning to.
[0056]
Also, this node NODE ₁₀₀ Then, even if there is no input of the recognition result, the behavior model 90 ₁ ~ 90 _n Of the parameter values of each emotion and each desire held in the emotion model 93 and the instinct model 94 that are periodically referred to by the user, “joy”, “surprise” or “surprise” held in the emotion model 93 When any of the parameter values of “Sadness” is in the range of “50 to 100”, it is possible to transition to another node.
[0057]
In the state transition table 100, the row of “transition destination node” in the column of “transition probability to another node” indicates that node NODE. ₀ ~ NODE _n The node names that can be transitioned from are listed, and other nodes NODE that can transition when all the conditions described in the columns of “input event name”, “data value”, and “data range” are met ₀ ~ NODE _n To the corresponding node in the column “Transition probability to another node”, and the node NODE ₀ ~ NODE _n The action to be output when transitioning to is described in the row of “output action” in the column of “transition probability to another node”. Note that the sum of the probabilities of each row in the column of “transition probability to another node” is 100 [%].
[0058]
Therefore, the node NODE represented by the state transition table 100 of FIG. ₁₀₀ Then, for example, when "the ball is detected (BALL)" and a recognition result indicating that the "SIZE (size)" of the ball is in the range of "0 to 1000" is given, "30 [%]" With the probability of "node NODE ₁₂₀ (Node 120) ", and the action of" ACTION1 "is output at that time.
[0059]
Each behavior model 90 ₁ ~ 90 _n Is a node NODE described as such a state transition table 100. ₀ ~ NODE _n Are connected to each other, and when a recognition result is given from the input semantics converter module 71 or the like, the corresponding node NODE ₀ ~ NODE _n The next action is determined stochastically using the state transition table, and the determination result is output to the action switching module 91.
[0060]
The behavior switching module 91 shown in FIG. ₁ ~ 90 _n Out of the actions respectively output from the action models 90 having a predetermined high priority. ₁ ~ 90 _n And outputs a command to execute the action (hereinafter referred to as an action command) to the output semantics converter module 79 of the middleware layer 50. In this embodiment, the behavior model 90 shown on the lower side in FIG. ₁ ~ 90 _n The higher the priority, the higher the priority.
[0061]
Further, the action switching module 91 notifies the learning module 92, the emotion model 93, and the instinct model 94 that the action has been completed, based on the action completion information provided from the output semantics converter module 79 after the action is completed.
[0062]
On the other hand, the learning module 92 inputs, from among the recognition results given from the input semantics converter module 71, the recognition result of the instruction received from the user, such as “reprimanded” or “praised”. Then, based on the recognition result and the notification from the action switching module 91, the learning module 92 lowers the probability of occurrence of the action when "scored" and increases the probability of occurrence of the action when "praised". Corresponding to the corresponding behavior model 90 in the behavior model library 90. ₁ ~ 90 _n Change the corresponding transition probability of.
[0063]
On the other hand, the emotion model 93 is a sum of “joy”, “sadness”, “anger”, “surprise”, “disgust”, and “fear” (FEAR). For the six emotions, a parameter indicating the intensity of the emotion is held for each emotion. Then, the emotion model 93 converts the parameter values of each of these emotions into a specific recognition result such as “scolded” and “praised” given from the input semantics converter module 71 and the elapsed time and action switching module 91. It is updated periodically based on the notification from.
[0064]
Specifically, the emotion model 93 is calculated by a predetermined arithmetic expression based on the recognition result given from the input semantics converter module 71, the behavior of the robot device 1 at that time, the elapsed time since the last update, and the like.変 動 E [t] is the variation amount of the emotion at that time, E [t] is the current parameter value of the emotion, and k is a coefficient representing the sensitivity of the emotion. _e Then, the parameter value E [t + 1] of the emotion in the next cycle is calculated by Expression (1), and the parameter value of the emotion is updated by replacing the parameter value E [t] with the current parameter value E [t] of the emotion. . The emotion model 73 updates the parameter values of all emotions in the same manner.
[0065]
(Equation 1)

[0066]
The degree to which each recognition result and the notification from the output semantics converter module 79 affect the variation ΔE [t] of the parameter value of each emotion is determined in advance, for example, “hit”. The recognition result has a great influence on the variation ΔE [t] of the parameter value of the emotion of “anger”, and the recognition result such as “stroke” indicates the variation ΔE [t] of the parameter value of the emotion of “joy”. Has become a major influence.
[0067]
Here, the notification from the output semantics converter module 79 is so-called action feedback information (action completion information), information on the appearance result of the action, and the emotion model 93 changes the emotion by such information. Let it. This is, for example, a behavior such as "barking" that lowers the emotional level of anger. Note that the notification from the output semantics converter module 79 is also input to the above-described learning module 92, and the learning module 92 generates an action model 90 based on the notification. ₁ ~ 90 _n Change the corresponding transition probability of.
[0068]
The feedback of the action result may be made by the output of the action switching module 91 (the action to which the emotion is added).
[0069]
On the other hand, the instinct model 94 provides four independent desires of “exercise”, “affection”, “appetite”, and “curiosity” for each of these desires. It holds a parameter indicating the strength of the desire. Then, the instinct model 94 periodically updates these desire parameter values based on the recognition result given from the input semantics converter module 71, the elapsed time, the notification from the action switching module 91, and the like.
[0070]
Specifically, the instinct model 94 uses a predetermined arithmetic expression based on the recognition result, the elapsed time, the notification from the output semantics converter module 68, and the like for “exercise desire”, “affection desire”, and “curiosity”. ΔI [k] is the variation of the desire at that time, I [k] is the current parameter value of the desire, and a coefficient k representing the sensitivity of the desire. _i The parameter value I [k + 1] of the desire in the next cycle is calculated using the equation (2) in a predetermined cycle, and the calculation result is replaced with the current parameter value I [k] of the desire. Update the parameter value of desire. Similarly, the instinct model 94 updates the parameter values of each desire except “appetite”.
[0071]
(Equation 2)

[0072]
Note that the degree to which the recognition result and the notification from the output semantics converter module 79 affect the amount of change ΔI [k] of the parameter value of each desire is determined in advance. Has a large effect on the variation ΔI [k] of the parameter value of “fatigue”.
[0073]
Note that, in the present embodiment, the parameter values of each emotion and each desire (instinct) are regulated so as to fluctuate in the range of 0 to 100, and the coefficient k _e , K _i Is also set individually for each emotion and each desire.
[0074]
On the other hand, as shown in FIG. 4, the output semantics converter module 79 of the middleware layer 50 outputs “forward”, “please”, and “squeals” given from the action switching module 91 of the application layer 51 as described above. ”Or“ tracking (follow the ball) ”to the corresponding signal processing modules 72 to 78 of the output system 80.
[0075]
When an action command is given, the signal processing modules 72 to 78 execute a corresponding actuator 28 for performing the action based on the action command. ₁ ~ 28 _n A servo command value to be given to (FIG. 2), audio data of a sound output from the speaker 20 (FIG. 2), and / or drive data to be given to the LED of the light emitting section 25 (FIG. 2) are generated. The corresponding actuator 28 via the virtual robot 43 of the tick server object 42 and the signal processing circuit 14 (FIG. 2) sequentially. ₁ ~ 28 _n , And sequentially to the speaker 20 or the light emitting unit 25.
[0076]
In this way, the robot apparatus 1 can perform autonomous actions according to its own (internal) and surrounding (external) conditions, instructions and actions from the user, based on the control program. Has been done.
[0077]
(3) Application of the present invention to a robot device
Hereinafter, an example in which the present invention is applied to a robot device 1 having the above-described structure will be described. As described above, the robot device 1 according to the present embodiment detects a moving object in image data captured by the CCD camera 22 (FIG. 2) and performs an action such as turning to the direction of the detected moving object or performing tracking. .
[0078]
Here, when a difference image between frames is generated, the difference value becomes 0 when the movement of the moving object stops. For example, as shown in FIG. ₁ ~ T ₄ Data P of a human being captured at ₁ ~ P ₄ For the difference image data D ₁ ~ D ₃ Is generated, the time t ₃ And t ₄ If the face is still between, the difference image data D ₃ , The difference data of the face is lost. That is, the fact that the moving object has disappeared from the difference image data means that the moving object does not disappear from the place but exists at the place where it disappeared.
[0079]
Therefore, the robot apparatus 1 according to the present embodiment detects the point in time when the difference becomes 0, and directs the CCD camera 22 toward the direction of the center of gravity in the immediately preceding difference image to direct the direction of the center of gravity, or Approach the direction of the position. That is, as shown in the flowchart of FIG. 10, first, in step S1, a moving object is detected by calculating the center of gravity of the difference image data, and in step S2, it is determined whether the detected moving object has disappeared from the difference image data. Is determined. If the moving object has not disappeared in step S2 (No), the process returns to step S1. On the other hand, if the moving object has disappeared in step S2 (Yes), the process proceeds to step S3, and the moving object faces the direction of the disappearance, that is, the direction of the center of gravity position in the immediately preceding difference image, or approaches the direction of the center of gravity.
[0080]
In addition, even when the detected moving object is out of the visual range of the robot apparatus 1, the moving object disappears from the difference image, but also in this case, by turning to the direction of the centroid position last detected in the above-described step S3, It can be almost directed to the moving body.
[0081]
As described above, the robot apparatus 1 according to the present embodiment detects that the moving object has disappeared from the difference image data due to the stillness of the moving object within the visual range, and directs the moving object to the direction of the position of the center of gravity. It is possible to realize an autonomous interaction of feeling the sign of a moving object and turning in that direction. In addition, by detecting that the moving object has disappeared from the difference image data due to departure from the visual range, and by pointing the moving object to the direction of the last detected center of gravity, the moving object can be almost turned.
[0082]
In addition, the robot apparatus 1 turns to the direction of the center of gravity detected not only when the moving object has disappeared from the difference image data but also at predetermined time intervals or whenever the position of the center of gravity of the moving object is likely to be out of the visual range. May be tracked. That is, as shown in the flowchart of FIG. 11, first, in step S10, the moving object is detected by calculating the position of the center of gravity of the difference image data, and in step S11, the moving object is likely to be out of the visual range at predetermined time intervals. Each time becomes, the direction of the detected center of gravity is turned.
[0083]
Here, in addition to the case where the moving object has disappeared from the difference image data as described above, when the movement of the robot device 1 is large in step S11, the robot device 1 determines its own movement and the movement of the moving object by motion compensation. You cannot distinguish them, and you lose sight of the moving object. Therefore, in step S12, it is determined whether the moving object has been lost. If the moving object has not been lost in step S12 (No), the process returns to step S10. On the other hand, if the moving object is lost in step S12 (Yes), the process proceeds to step S13, and the direction of the last detected center of gravity is turned.
[0084]
As described above, the robot apparatus 1 according to the present embodiment turns to the direction of the center of gravity detected at predetermined time intervals or each time the moving object is likely to be out of the visual range, and is finally detected when the moving object is lost. The moving body in the image picked up by the CCD camera 22 provided in the head unit 4 can be detected and tracked by a simple method by pointing the direction of the center of gravity position.
[0085]
The above processing is realized by configuring the motion detection signal processing module 68 in the recognition system 71 of the middleware layer 50 shown in FIG. 4 by the difference image generation module 110 and the center of gravity calculation module 111.
[0086]
That is, as shown in FIG. 12, the virtual robot 43 of the robotic server object 42 reads out frame-based image data captured by the CCD camera 22 from the DRAM 11, and reads this image data into the middleware layer 50. Is sent to the difference image generation module 110 included in the recognition system 71.
[0087]
Each time image data is input, the difference image generation module 110 generates a difference image data by calculating a difference from the image data of the previous frame adjacent on the time axis, and supplies the difference image data to the centroid calculation module 111. For example, the image data P ₂ And image data P ₃ Difference image data D ₂ Is generated, the difference image data D at the position (i, j) ₂ Brightness value D ₂ (I, j) is the image data P at the position (i, j). ₃ Brightness value P ₃ Image data P at the same position from (i, j) ₂ Brightness value P ₂ It is obtained by subtracting (i, j). The difference image generation module 110 performs the same calculation for all pixels to obtain difference image data D ₂ And the difference image data D ₂ To the center-of-gravity calculation module 111.
[0088]
Then, the center-of-gravity calculation module 111 calculates a center-of-gravity position G (x, y) of a portion of the difference image data having a luminance value larger than the threshold Th. Here, x and y are calculated using the following equations (3) and (4), respectively.
[0089]
[Equation 3]

[0090]
Thereby, for example, as shown in FIG. ₂ And image data P ₃ Difference image data D ₂ From the center of gravity G ₂ Is required.
[0091]
The center-of-gravity calculation module 111 sends the data of the obtained center-of-gravity position to the behavior model library 90 of the application layer 51.
[0092]
The behavior model library 90 determines a subsequent behavior by referring to the parameter value of the emotion or the parameter value of the desire as needed, and gives the determination result to the behavior switching module 91. For example, when the moving object has disappeared from the difference image data, an action to face or approach the center of gravity detected immediately before is determined, and the determination result is given to the action switching module 91. When the moving object is tracked at predetermined time intervals, an action toward or near the center of gravity detected at each time interval is determined, and the determination result is provided to the action switching module 91. Then, the action switching module 91 sends an action command based on the determination result to the tracking signal processing module 73 in the output system 80 of the middleware layer 50.
[0093]
When an action command is given, the tracking signal processing module 73 performs a corresponding actuator 28 to perform the action based on the action command. ₁ ~ 28 _n Is generated, and this data is sequentially transmitted to the corresponding actuator 28 via the virtual robot 43 of the robotic server object 42 and the signal processing circuit 14 (FIG. 2). ₁ ~ 28 _n Sequentially.
[0094]
As a result, for example, when the moving object disappears from the differential image data, the action model library 90 determines the action to face or approach the center of gravity detected immediately before, and the action switching module 91 performs the action. An action command for causing the action command is generated. When a moving object is tracked at predetermined time intervals, the action model library 90 determines an action toward or near the center of gravity detected at each time interval, and the action switching module 91 determines the action. Is generated.
[0095]
When this action command is given to the tracking signal processing module 73, the tracking signal processing module 73 sends a servo command value based on the action command to the corresponding actuator 28. ₁ ~ 28 _n This causes the robot apparatus 1 to take an interest in the moving body, turn the head in that direction, or approach the moving body.
[0096]
It should be noted that the present invention is not limited to only the above-described embodiment, and it is needless to say that various changes can be made without departing from the gist of the present invention.
[0097]
For example, in the above-described embodiment, the process of detecting a moving object in image data captured by the CCD camera 22 is described as being realized by software. However, the present invention is not limited to this. It may be realized by.
[0098]
【The invention's effect】
As described in detail above, the robot apparatus according to the present invention is an autonomous robot apparatus that autonomously acts in response to an external stimulus and / or an internal state. Means for generating a difference image between images adjacent on the time axis captured by the means, means for calculating the center of gravity of the difference image, and means for generating an action based at least on the position of the center of gravity An action generating means for generating, when the moving object in the difference image has disappeared, the action of pointing the imaging means in the direction of the last detected center of gravity of the difference image.
[0099]
In addition, the robot apparatus according to the present invention is an autonomous robot apparatus that autonomously acts in response to an external stimulus and / or an internal state. A difference image generating means for generating a difference image between images adjacent on a time axis, a center of gravity calculating means for calculating a center of gravity of the difference image, and an action generating means for generating an action based at least on the position of the center of gravity. Wherein the action generation means generates an action of directing the imaging means in the direction of the center of gravity of the difference image at predetermined time intervals or each time the center of gravity of the difference image is likely to deviate from the difference image. If the moving object in the difference image is lost, an action of pointing the imaging unit in the direction of the last detected center of gravity of the difference image is generated.
[0100]
Such a robot device calculates the center of gravity of the difference image between the images adjacent on the time axis captured by the imaging unit, and when the moving object in the difference image disappears, calculates the center of gravity of the last detected center of gravity. Orient the imaging means in the direction.
[0101]
Further, the robot apparatus calculates the center of gravity of the difference image between the images adjacent on the time axis captured by the imaging unit, and the center of gravity of the difference image is likely to deviate from the difference image at every predetermined time interval or Each time the imaging means is directed in the direction of the center of gravity of the difference image, and when the moving object in the difference image is lost, the imaging means is directed in the direction of the last detected center of gravity.
[0102]
This makes it possible to detect and track a moving object in an image captured by the imaging unit using a simple method.
[0103]
The operation control method for a robot device according to the present invention is the operation control method for an autonomous robot device that autonomously acts according to an external stimulus and / or an internal state, wherein at least an external moving body is imaged by the imaging unit. An imaging step, a difference image step of generating a difference image between images adjacent on the time axis taken in the imaging step, a gravity center calculation step of calculating a gravity center position of the difference image, and at least the gravity center position. And a behavior generating step of generating a behavior based on the motion generating step. An aiming action is generated.
[0104]
The operation control method for a robot device according to the present invention is the operation control method for an autonomous robot device that autonomously acts according to an external stimulus and / or an internal state, wherein at least an external moving body is imaged by the imaging unit. An imaging step, a difference image step of generating a difference image between images adjacent on the time axis taken in the imaging step, a gravity center calculation step of calculating a gravity center position of the difference image, and at least the gravity center position. An action generation step of generating an action based on the difference generation image, wherein in the action generation step, every predetermined time interval or each time the center of gravity of the difference image is likely to deviate from the difference image, When the action of pointing the imaging unit in the direction is generated and the moving object in the difference image disappears, the imaging is performed in the direction of the last detected center of gravity of the difference image. Action directing means is generated.
[0105]
In such an operation control method for a robot device, the position of the center of gravity of the difference image between the images adjacent on the time axis captured by the imaging unit is calculated, and when the moving object in the difference image disappears, it is detected last. The imaging means is directed in the direction of the position of the center of gravity.
[0106]
Further, in the operation control method of the robot apparatus, the position of the center of gravity of the difference image between adjacent images on the time axis captured by the imaging unit is calculated, and the center of gravity of the difference image is determined at predetermined time intervals or from the difference image. Each time it is likely to deviate, the imaging means is directed in the direction of the center of gravity of the difference image, and if the moving object in the difference image is lost, the imaging means is directed in the direction of the last detected center of gravity. Can be
[0107]
This makes it possible to detect and track a moving object in an image captured by the imaging unit using a simple method.
[Brief description of the drawings]
FIG. 1 is a perspective view illustrating an external configuration of a robot device according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a circuit configuration of the robot device.
FIG. 3 is a block diagram showing a software configuration of the robot device.
FIG. 4 is a block diagram showing a configuration of a middleware layer in a software configuration of the robot apparatus.
FIG. 5 is a block diagram showing a configuration of an application layer in a software configuration of the robot apparatus.
FIG. 6 is a block diagram showing a configuration of a behavior model library of the application layer.
FIG. 7 is a diagram illustrating a finite probability automaton serving as information for determining an action of the robot apparatus.
FIG. 8 is a diagram showing a state transition table prepared for each node of the finite probability automaton.
FIG. 9 is a diagram illustrating an example in which a moving object disappears from difference image data.
FIG. 10 is a flowchart illustrating a procedure in a case where a moving object is directed in a direction in which a moving object has disappeared from difference image data.
FIG. 11 is a flowchart illustrating a procedure for tracking a moving object.
FIG. 12 is a block diagram showing a software configuration of a portion related to moving object detection of the robot device.
FIG. 13 is a diagram illustrating an example of obtaining a center of gravity of difference image data.
[Explanation of symbols]
1 robot apparatus, 10 CPU, 11 DRAM, 14 signal processing circuit, 22 CCD camera, 28 ₁ ~ 28 _n Actuator, 42 robotic server object, 43 virtual robot, 50 middleware layer, 51 application layer, 68 signal processing module for motion detection, 70 input semantics converter module, 71 recognition system, 73 signal processing for tracking Module, 79 output semantics converter module, 80 output system, 90 behavior model library, 91 behavior switching module, 110 difference image generation module, 111 center of gravity calculation module

Claims (12)

In an autonomous robot device that autonomously acts according to an external stimulus and / or an internal state,
Imaging means for imaging at least an external moving body,
Difference image generation means for generating a difference image between images adjacent on the time axis captured by the imaging means,
Centroid calculating means for calculating the centroid position of the difference image;
Behavior generating means for generating a behavior based on at least the position of the center of gravity,
The robot apparatus according to claim 1, wherein, when the moving object in the difference image disappears, the action generation unit generates an action of pointing the imaging unit in the direction of the center of gravity detected last in the difference image. The robot apparatus according to claim 1, wherein the action generating means generates an action approaching the direction of the position of the center of gravity. The robot apparatus according to claim 1, wherein the robot apparatus has an external appearance imitating an animal, and the imaging means is provided on a head movably connected to a body part. In an autonomous robot device that autonomously acts according to an external stimulus and / or an internal state,
Imaging means for imaging at least an external moving body,
Difference image generation means for generating a difference image between images adjacent on the time axis captured by the imaging means,
Centroid calculating means for calculating the centroid position of the difference image;
Behavior generating means for generating a behavior based on at least the position of the center of gravity,
The action generating means generates an action of directing the imaging means in the direction of the center of gravity of the difference image at predetermined time intervals or each time the center of gravity of the difference image is likely to deviate from the difference image, A robot apparatus, wherein, when a moving object in a difference image is lost, an action of pointing the image pickup means in a direction of a center of gravity detected last in the difference image is generated. The robot apparatus according to claim 4, wherein the action generating means generates an action approaching the direction of the position of the center of gravity. The robot apparatus according to claim 4, wherein the robot has an external shape imitating an animal, and the imaging means is provided on a head movably connected to a body. In an operation control method of an autonomous robot device that autonomously acts according to an external stimulus and / or an internal state,
An imaging step of imaging at least an external moving body by imaging means;
A difference image step of generating a difference image between images adjacent on the time axis captured in the imaging step,
A center of gravity calculation step of calculating the center of gravity of the difference image;
An action generation step of generating an action based on at least the position of the center of gravity,
In the behavior generating step, when a moving object in the difference image disappears, an action of pointing the imaging unit in a direction of a center of gravity detected last in the difference image is generated. Operation control method. The method according to claim 7, wherein in the action generating step, an action approaching the direction of the center of gravity is generated. 8. The operation of the robot device according to claim 7, wherein the robot device has an appearance shape imitating an animal, and the imaging unit is provided on a head movably connected to a body part. Control method. In an operation control method of an autonomous robot device that autonomously acts according to an external stimulus and / or an internal state,
An imaging step of imaging at least an external moving body by imaging means;
A difference image step of generating a difference image between images adjacent on the time axis captured in the imaging step,
A center of gravity calculation step of calculating the center of gravity of the difference image;
An action generation step of generating an action based on at least the position of the center of gravity,
In the action generation step, an action of pointing the imaging unit in the direction of the center of gravity of the difference image is generated at predetermined time intervals or each time the center of gravity of the difference image is likely to deviate from the difference image, An operation control method for a robot apparatus, wherein when a moving object in the difference image disappears, an action of directing the imaging unit in a direction of a center of gravity detected last in the difference image is generated. 11. The operation control method for a robot device according to claim 10, wherein in the action generating step, an action approaching the direction of the position of the center of gravity is generated. 11. The operation of the robot device according to claim 10, wherein the robot device has an external shape imitating an animal, and the imaging unit is provided on a head movably connected to a body. Control method.

Images