JP3586880B2 - Image extraction apparatus and image extraction method - Google Patents

Description

以上の図１５の動作フローチャートで示される図１０のステップ１０１１の重心計算処理の後、図１０のステップ１０１２で、衝突レジスタの値が“１”になっているか否かが判定される。このレジスタの値は、前述したステップ１００６の衝突座標検出処理において、処理画面マップ上のバット１０１の存在を示す論理値が“１”であるブロックが１つでも、ボール１０５の存在を示すＹＳＯＢＪ信号と重なっている場合に、“１”にセットされる（図１４のステップ１４０８参照）。
【０１２２】
そして、ステップ１０１２の判定がＹＥＳの場合は、ステップ１０１３で、図５のＣＰＵ５０９への割込みフラグである衝突フラグが“１”にセットされる。この結果、ＣＰＵ５０９において後述する衝突フラグ割込み処理（図２０）が発生し、この割込み処理において、図１のバット１０１がボール１０５に当ったことに対応して表示装置１０３に表示されるＣＧ画像１０４を制御する処理が実行される。なお、画像認識回路５０８は、上述の衝突フラグのセット処理と共に、ステップ１０１１で計算した重心座標（ＸＧ、ＹＧ）を、図５のＤＲＡＭ５０７に転送する。また、画像認識回路５０８は、ステップ１００６の衝突座標検出処理によってＲＡＭ（図２６）に記憶した最初の衝突座標及び最後の衝突座標、並びに、ステップ１００５の処理画面マップ作成処理によって上述のＲＡＭに記憶した上端座標、下端座標、左端座標、及び右端座標を、必要に応じて、ＤＲＡＭ５０７に転送する。
【０１２３】
ステップ１０１３の処理の後は、ステップ１０１４で、衝突フラグが“０”に戻ったか否かが判定される。この衝突フラグの値は、ＣＰＵ５０９が実行する後述する衝突フラグ割込み処理が終了する時点で“０”に戻される。この結果、例えばホームラン画像が進行している複数フレーム周期の間は、図５の画像認識回路５０８は図１０の動作フローチャートの処理をステップ１０１４より先に進めないことになる。なお、衝突フラグ割込み処理が実行されている間にも、ステップ１００４〜ステップ１０１５の処理をフレーム周期に同期して進行させる構成にし衝突レジスタの値に応じてＣＰＵ５０９に更に別の割込みがかけられるようにしてもよい。
【０１２４】
衝突フラグの値が“０”に戻りステップ１０１４の判定がＹＥＳとなると、ステップ１０１５で衝突レジスタの値も“０”に戻される。
ステップ１０１５の処理の後又はステップ１０１２の判定がＮＯであった場合には、次のフレーム周期の開始時点に同期してステップ１００４の処理に戻る。この場合、前述したように、処理対象となるクロマキー信号用バッファ及びＹＳＯＢＪ信号用バッファが変更される。
【０１２５】
次に、図１６及び図１７は、図５のＣＰＵ５０９が実行するＣＰＵ処理の動作フローチャートである。
ＣＰＵ５０９は、基本的には、特には図示しないスタートスイッチ及びストップスイッチなどから入力されるキースキャン信号５２７を処理しながら、時間経過と共に、アプリケーションカートリッジ５１３内のアプリケーションＲＯＭ５１４からＣＧ画像１０４（図１）を構成するオブジェクト画面データ、オブジェクトテーブルデータ、及びバックグランド画面データを順次読み出し、同期信号５１９に同期した垂直ブランキング期間内に、それらのデータをバス５２６からＶＤＰ５０６に転送し、更に時間経過と共に、ＤＲＡＭ５０７に転送した各オブジェクト画面データのＣＧ画像１０４（図１）上での表示位置を、ＶＤＰ５０６に対して順次指定する。
【０１２６】
図１６において、まず、ステップ１６０１では、図５のＤＲＡＭ５０７の内容などがイニシャライズされる。
ステップ１６０２では、後述する各種割込み処理が許可される。
【０１２７】
ステップ１６０３では、特には図示しないスロットに挿入された図５のアプリケーションカートリッジ５１３内のアプリケーションＲＯＭ５１４（＃１又は＃２の何れか）から、図５のＤＲＡＭ５１０上のプログラム領域にアプリケーションプログラムがロードされる。以下、ＣＰＵ５０９は、このアプリケーションプログラムに従って、ステップ１６０４以降の動作を実行する。
【０１２８】
ステップ１６０４では、ＣＰＵ５０９内のＤＭＡＣ（ダイレクトメモリアクセスコントローラ）に、ＣＧ画像１０４（図１）を構成するバックグランド画面データのアプリケーションカートリッジ５１３内のアプリケーションＲＯＭ５１４上での読出し開始アドレスと読出しデータ数がセットされる。
【０１２９】
ステップ１６０５では、上記ＤＭＡＣに、ＣＧ画像１０４（図１）を構成する最初のオブジェクト画面データの上記アプリケーションＲＯＭ５１４上での読出し開始アドレスと読出しデータ数がセットされる。
【０１３０】
ステップ１６０６では、上記ＤＭＡＣに、オブジェクトテーブルデータの上記アプリケーションＲＯＭ５１４上での読出し開始アドレスと読出しデータ数がセットされる。このオブジェクトテーブルデータは、前述したように、ＤＲＡＭ５０７に記憶される各オブジェクト画面データとそれぞれに対応するＩＤ番号との対応テーブルを示すテーブルである。
【０１３１】
上記ステップ１６０４〜ステップ１６０６の処理の後、ステップ１６０７で、レジスタＡＣＫＦに“１”がセットされる。そして、ステップ１６０８で、レジスタＡＣＫＦの値が“０”に戻るまで待機状態となる。
【０１３２】
ここで、ＣＰＵ５０９は、同期信号５１９に基づいて決定される各フレーム周期内の垂直ブランキング期間が開始される時点毎に、ステップ１６０８の処理を中断して、図１８の動作フローチャートで示される垂直ブランキング割込み処理を実行する。
【０１３３】
そして、上述のＣＰＵ処理によってレジスタＡＣＫＦに“１”がセットされた場合は、この割込み処理が実行されるタイミングで、ステップ１８０１の判定がＹＥＳとなることにより、前述した図１６のステップ１６０４〜ステップ１６０６の処理によってＣＰＵ５０９内のＤＭＡＣに設定された各データに基づいて、アプリケーションカートリッジ５１３内のアプリケーションＲＯＭ５１４に記憶されている各データがＶＤＰ５０６に転送される。
【０１３４】
即ち、図１６のステップ１６０４〜ステップ１６０６の処理は電源オン直後に実行されるため、図１８のステップ１８０２の判定がＹＥＳとなる。
この結果、まず、ステップ１８０３で、ＤＭＡＣは、ステップ１６０４の処理により設定されたバックグランド画面データの読出し開始アドレスと読出しデータ数に基づいて、アプリケーションＲＯＭ５１４内のバックグランド画面データを、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。ＶＤＰ５０６は、このバックグランド画面データを、図８のＣＰＵインタフェース回路８０１からＤＲＡＭコントロール回路８０２を介して図５のＤＲＡＭ５０７に転送する。
【０１３５】
次に、ステップ１８０４で、ＤＭＡＣは、ステップ１６０５の処理により設定されたオブジェクト画面データの読出し開始アドレスと読出しデータ数に基づいて、アプリケーションＲＯＭ５１４内のオブジェクト画面データを、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。ＶＤＰ５０６は、このオブジェクト画面データも、図８のＣＰＵインタフェース回路８０１からＤＲＡＭコントロール回路８０２を介して図５のＤＲＡＭ５０７に転送する。
【０１３６】
更に、ステップ１８０５で、ＤＭＡＣは、ステップ１６０６の処理により設定されたオブジェクトテーブルデータの読出し開始アドレスと読出しデータ数に基づき、アプリケーションＲＯＭ５１４内のオブジェクトテーブルデータを、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。ＶＤＰ５０６は、このオブジェクトテーブルデータを、ＣＰＵインタフェース回路８０１からオブジェクトコントロール回路８０３を介してオブジェクトテーブルＲＡＭ８０４に転送する。
【０１３７】
その後、ステップ１８０６で、レジスタＡＣＫＦの値が“０”に戻され、図１８に示される垂直ブランキング割込み処理を終了する。
この結果、図１６のステップ１６０８の処理が再開され、この処理における判定がＹＥＳとなることにより、まず、ユーザが特には図示しないスタートスイッチを押しそれに対応するキースキャン信号５２７（図５）が入力されるまで、ステップ１６０９の判定がＮＯ→ステップ１６１２の判定がＮＯ→ステップ１６０９の判定がＮＯ・・・という処理ループが繰り返され、待機状態となる。なお、レジスタＳＴＦの内容は、図１６のステップ１６０１において“０”にイニシャライズされている。
【０１３８】
スタートスイッチが押されそれに対応するキースキャン信号５２７が入力されると、ステップ１６０９の判定がＹＥＳとなる。
この結果、まず、ステップ１６１０において、レジスタｔに時刻０がセットされる。このレジスタｔは、ユーザがスタートスイッチを押することによってゲームを開始させた時点からの経過時刻を示すレジスタであり、このレジスタｔの値は、バス５２６に接続されているクロック回路５１２（図５）から一定時間毎に入力される割込み要求に基づいて実行される、図１９のステップ１９０１として示されるタイマー割込み処理によって、＋１ずつインクリメントされる。
【０１３９】
次に、ステップ１６１１で、レジスタＳＴＦに“１”がセットされる。
この結果、ステップ１６１２の判定がＹＥＳとなり、これ以降、通常は、図１７のステップ１６１３〜ステップ１６１７→ステップ１６１９→ステップ１６０９（図１６）→ステップ１６１２→ステップ１６１３という処理ループが繰り返される。
【０１４０】
上述の処理ループにおいて、ステップ１６１４では、時間経過を示す現在のレジスタｔの値に基づいて、表示をすべきオブジェクト画面データに対応するオブジェクトテーブルデータに対して指定するＩＤ番号が演算され、ステップ１６１５では、現在のレジスタｔの値に基づいて、オブジェクト画面データの表示位置が演算される。これらのデータは、ＤＲＡＭ５０７上の所定のアドレスにセットされ、その読出し開始アドレスと読出しデータ数がＣＰＵ５０９内のＤＭＡＣにセットされる。なお、ステップ１６１４又はステップ１６１５では、後述する音源割込み処理（図２２参照）においてアプリケーションカートリッジ５１３内の音源回路５１５に転送される発音指示データも生成され、ＤＲＡＭ５０７などにセットされる。
【０１４１】
また、オブジェクト画面データを変更する必要がある場合には、ステップ１６１４で、ＣＰＵ５０９内のＤＭＡＣに、新たなオブジェクト画面データに対応するオブジェクトテーブルデータのアプリケーションＲＯＭ５１４上での読出し開始アドレスと読出しデータ数がセットされる。また、ステップ１６１５で、ＤＭＡＣに、新たなオブジェクト画面データのアプリケーションＲＯＭ５１４上での読出し開始アドレスと読出しデータ数がセットされる。
【０１４２】
ステップ１６１６では、レジスタＡＣＫＦに“１”がセットされる。
その後、前述の垂直ブランキング割込み処理が実行されてレジスタＡＣＫＦの値が“０”に戻されるまでは、通常、ステップ１６１７の判定がＮＯ→ステップ１６１９の判定がＮＯ→ステップ１６０９（図１６）の判定がＮＯ→ステップ１６１２の判定がＹＥＳ→ステップ１６１３（図１７）の判定がＮＯ→ステップ１６１７の判定がＮＯ・・・という処理ループが繰り返され、待機状態となる。
【０１４３】
ＣＰＵ５０９は、前述したように、垂直ブランキング期間が開始される時点毎に、上述の処理ループの処理を中断して、図１８の動作フローチャートで示される垂直ブランキング割込み処理を実行する。
【０１４４】
この結果、ステップ１８０１の判定がＹＥＳ→ステップ１８０２の判定がＮＯとなることにより、ステップ１８０４が実行される。
ステップ１８０４では、ＣＰＵ５０９内のＤＭＡＣは、ステップ１６１５の処理による設定に基づいて、オブジェクト画面データの表示位置を示すデータを、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。また、ステップ１８０５では、ＤＭＡＣは、ステップ１６１４の処理による設定に基づいて、表示を行うべきオブジェクト画面データのＩＤ番号を、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。この結果、ＶＤＰ５０６内のオブジェクトコントロール回路８０３は、ＣＰＵ５０９から指定されたＩＤ番号によってオブジェクトテーブルＲＡＭ８０４を参照することにより、そのＩＤ番号に対応するオブジェクト画面データをＤＲＡＭコントロール回路８０２を介してＤＲＡＭ５０７から読み出し、ＣＰＵ５０９から指定された表示位置の情報に基づいてラインバッファＲＡＭ８０８に展開した後、その内容をＲＧＢ色差データ処理回路８０６に出力する。
【０１４５】
新たなオブジェクト画面データ及びオブジェクトテーブルデータが転送される場合のステップ１８０４と１８０５の処理は、前述した図１６のステップ１６０４〜ステップ１６０６が実行された直後の場合と同様である。
【０１４６】
その後、ステップ１８０６で、レジスタＡＣＫＦの値が“０”に戻され、図１８に示される垂直ブランキング割込み処理を終了する。
この結果、ＣＰＵ処理における前述した処理ループが再開され、ステップ１６１３の判定がＹＥＳとなることにより、次の画面制御処理が実行される。
【０１４７】
上述の処理ループの実行過程で、ユーザが特には図示しないストップスイッチを押しそれに対応するキースキャン信号５２７（図５）が入力されると、ステップ１６１７の判定がＹＥＳとなる。この結果、ステップ１６１８で、レジスタＳＴＦに“０”がセットされる。従って、この時点以降は、ユーザが特には図示しないスタートスイッチを再び押しそれに対応するキースキャン信号５２７が入力されるまで、ステップ１６０９（図１６）の判定がＮＯ→ステップ１６１２の判定がＮＯ→ステップ１６０９の判定がＮＯ・・・という処理ループが繰り返されて、待機状態となる。なお、レジスタｔの値はクリアされないため、次にスタートスイッチが押されると、ストップスイッチが押された時点からゲームの進行が再開される。
【０１４８】
また、上述の処理ループの実行過程で、レジスタｔの値が予め設定された終了時刻Ｔになると、ステップ１６１９の判定がＹＥＳとなる。この結果、ステップ１６２０で、レジスタｔの値が０にリセットされる。従って、この時点以降は、ゲームが再び最初から繰り返されることになる。
【０１４９】
更に、上述の処理ループの実行過程で、ユーザが特には図示しないスタートスイッチを再度押しそれに対応するキースキャン信号５２７が入力されると、ステップ１６０９の判定がＹＥＳとなる。この結果、ステップ１６１０でレジスタｔの値が強制的に０にリセットされる。従って、この時点以降も、ゲームが再び最初から繰り返されることになる。
【０１５０】
一方、上述の処理ループの実行過程で、画像認識回路５０８によって、前述したように衝突フラグが１にセットされると（図１０のステップ１０１３参照）、ＣＰＵ５０９は、上述の処理ループの処理を中断して、図２０の動作フローチャートで示される衝突フラグ割込み処理を実行する。
【０１５１】
まず、ステップ２００１では、レジスタｔによって示されている現在の時刻ｔが、予め設定された所定の衝突判定期間Ｔ_１ 〜Ｔ_２ の範囲に入っているか否かが判定される。このような衝突判定期間を設けた理由は、本来、図１のバット１０１がボール１０５に当るべき時間以外の時間で、バット１０１に対応するクロマキーがボール１０５に対応するオブジェクト画面データと重なってしまった場合に、誤った処理が実行されてしまうことを防止するためである。
【０１５２】
ステップ２００２においては、画像認識回路５０８からＤＲＡＭ５１０に転送されているバット１０１を示すクロマキーの重心座標が、ＣＰＵ５０９内に取り込まれる。
【０１５３】
ステップ２００３においては、上述のバット１０１を示すクロマキーの重心座標と、ＶＤＰ５０６内のオブジェクトコントロール回路８０３から取得した現在表示されているボール１０５を示すオブジェクト画面データの重心座標との位置関係が判定される。
【０１５４】
この結果、図２に示されるように、２つの重心座標のずれが所定の許容範囲内にある場合には、図１の表示装置１０３上のＣＧ画像１０４を例えばホームランに対応する画像に変化させるべく、ステップ２００４で、第１画像処理が実行される。なお、処理画面マップ上におけるバット１０１を示すクロマキーの重心座標と、ボール１０５を示すオブジェクト画面データの重心座標は、ユーザがバット１０１をちょうどストライクゾーンに対応する位置に振り下ろしたときに一致するように、予め図１のカメラ１０２の方向が調整されているものとする。
【０１５５】
また、図３に示されるように、ボール１０５を示すオブジェクト画面データの重心座標がバット１０１を示すクロマキーの重心座標の下方にずれている場合には、図１の表示装置１０３上のＣＧ画像１０４を例えばゴロに対応する画像に変化させるべく、ステップ２００５で、第２画像処理が実行される。
【０１５６】
更に、図４に示されるように、ボール１０５を示すオブジェクト画面データの重心座標がバット１０１を示すクロマキーの重心座標の上方にずれている場合には、図１の表示装置１０３上のＣＧ画像１０４を例えばフライに対応する画像に変化させるべく、ステップ２００６で、第３画像処理が実行される。
【０１５７】
図２１は、ステップ２００４〜ステップ２００６に示される第１〜第３画像処理を一般的に表した第ｎ画像処理の動作フローチャートである。
ステップ２１０１では、現在の処理が第１、第２、又は第３の何れの画像処理かによって、また、時間経過を示す現在のレジスタｔの値に基づいて、オブジェクト画面データの表示位置が演算される。また、ステップ２１０２では、現在の処理が第１、第２、又は第３の何れの画像処理かによって、また、現在のレジスタｔの値に基づいて、表示をすべきオブジェクト画面データに対応するオブジェクトテーブルデータに対して指定するＩＤ番号が演算される。なお、ステップ２１０１又はステップ２１０２では、後述する音源割込み処理（図２２参照）においてアプリケーションカートリッジ５１３内の音源回路５１５に転送される発音指示データも生成され、ＤＲＡＭ５０７などにセットされる。
【０１５８】
ステップ２１０３では、レジスタＡＣＫＦに“１”がセットされる。
その後、前述した垂直ブランキング割込み処理が実行されてレジスタＡＣＫＦの値が“０”に戻されるまでは、通常は、ステップ２１０４の判定がＮＯ→ステップ２１０５の判定がＮＯ→ステップ２１０６の判定がＮＯ→ステップ２１０４の判定がＮＯ・・・という処理ループが繰り返され、待機状態となる。
【０１５９】
ＣＰＵ５０９は、前述したように、垂直ブランキング期間が開始される時点毎に、上述の処理ループの処理を中断して、図１８の動作フローチャートで示される垂直ブランキング割込み処理を実行する。
【０１６０】
この結果、ステップ１８０１の判定がＹＥＳ→ステップ１８０２の判定がＮＯとなることにより、ステップ１８０４が実行される。
ステップ１８０４では、ＣＰＵ５０９内のＤＭＡＣは、ステップ２１０１の処理による設定に基づいて、オブジェクト画面データの表示位置を示すデータを、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。また、ステップ１８０５では、ＤＭＡＣは、ステップ２１０２の処理による設定に基づいて、表示を行うべきオブジェクト画面データのＩＤ番号を、バス５２６を介してＶＤＰ５０６にＤＭＡ転送する。この結果、ＶＤＰ５０６内のオブジェクトコントロール回路８０３は、ＣＰＵ５０９から指定されたＩＤ番号によってオブジェクトテーブルＲＡＭ８０４を参照することにより、そのＩＤ番号に対応するオブジェクト画面データをＤＲＡＭコントロール回路８０２を介してＤＲＡＭ５０７から読み出し、ＣＰＵ５０９から指定された表示位置の情報に基づいてラインバッファＲＡＭ８０８に展開した後、その内容をＲＧＢ色差データ処理回路８０６に出力する。
【０１６１】
その後、ステップ１８０６で、レジスタＡＣＫＦの値が“０”に戻され、図１８に示される垂直ブランキング割込み処理を終了する。
この結果、第ｎ画像処理における前述した処理ループが再開され、ステップ２１０６の判定がＹＥＳとなることにより、第ｎ画像処理の次のレジスタｔの値に対応する画面制御処理が実行される。
【０１６２】
上述の処理ループの実行過程で、ユーザが特には図示しないストップスイッチを押しそれに対応するキースキャン信号５２７（図５）が入力されると、ステップ２１０５の判定がＹＥＳとなる。この結果、ステップ２１０８で、レジスタＳＴＦに“０”がセットされ、図２１の第ｎ画像処理を終了する。この結果、後述する図２０のステップ２００７を実行した後に衝突フラグ割込み処理も終了し、ＣＰＵ５０９処理のループに戻る。従って、この時点以降は、ユーザが特には図示しないスタートスイッチを再び押しそれに対応するキースキャン信号５２７が入力されるまで、ステップ１６０９（図１６）の判定がＮＯ→ステップ１６１２の判定がＮＯ→ステップ１６０９の判定がＮＯ・・・という処理ループが繰り返されて、待機状態となる。
【０１６３】
また、図２１の第ｎ画像処理の実行過程で、レジスタｔの値が予め設定された第ｎ画像処理の終了時刻ＥＮＤになると、ステップ２１０４の判定がＹＥＳとなる。この結果、ステップ２１０７で、レジスタｔの値が０にリセットされ、図２１の第ｎ画像処理を終了し、更に、後述する図２０のステップ２００７を実行した後に衝突フラグ割込み処理も終了し、ＣＰＵ５０９処理のループに戻る。従って、この時点以降は、ゲームが再び最初から繰り返されることになる。
【０１６４】
なお、ステップ２１０７で、レジスタｔの値を０にリセットせず、強制的に予め定められた所定の値にセットすることにより、表示装置１０３上のＣＧ画像１０４の表示を、次の場面に移行させることも可能である。
【０１６５】
図２１の第ｎ画像処理を終了した後に実行される図２０の衝突フラグ割込み処理におけるステップ２００７においては、衝突フラグが“０”に戻される。この処理は、前述したように、画像認識回路５０８が実行する図１０に示される動作フローチャートにおいて、ステップ１０１４より先に処理を進ませるための処理である。
【０１６６】
次に、図２２は、ＣＰＵ５０９が実行する音源割込み処理の動作フローチャートである。この割込み処理は、図５のアプリケーションカートリッジ５１３内の音源回路５１５が、後述する音源処理（図２５）を実行する過程において、ＣＰＵ５０９への割込みフラグとして、音源割込み要求フラグをセットした時点で実行される。
【０１６７】
まず、ステップ２２０１では、監視タイマーがリセットされる。監視タイマーは、模造されたアプリケーションカートリッジ５１３の使用を阻止するために使用される。
【０１６８】
上述の監視タイマーの値は、バス５２６に接続されているクロック回路５１２（図５）から一定時間毎に入力される割込み要求に基づいて実行される図２３の動作フローチャートで示される監視タイマーカウント処理のステップ２３０１において、順次インクリメントされる。そして、ステップ２３０２で、この監視タイマーの値が、オーバーフローしたか否かが判定されている。
【０１６９】
ここで、アプリケーションカートリッジ５１３が正規のメーカによって製造されたものである場合には、アプリケーションカートリッジ５１３は、後述する音源処理（図２５）を実行する過程において、ＣＰＵ５０９への割込みフラグである音源割込み要求フラグをセット及びリセットする動作を順次実行する。これに対して、ＣＰＵ５０９は、音源割込み要求フラグがセットされる毎に、図２２の音源割込み処理のステップ２２０１を実行することにより、監視タイマーは適当な時間間隔でリセットされることになる。
【０１７０】
従って、アプリケーションカートリッジ５１３が正規のメーカによって製造されたものである場合には、図２３の監視タイマーカウント処理のステップ２３０２の判定は常にＮＯとなる。
【０１７１】
一方、アプリケーションカートリッジ５１３が模造されたものである場合は、コスト的な理由などからＣＰＵ５０９への割込みフラグである音源割込み要求フラグをセット及びリセットする動作が装備されていない場合がほとんどとなるため、音源回路５１５がＣＰＵ５０９に対して適度に短い時間間隔で音源割込み要求を発生することがなくなり、この結果、監視タイマーがリセットされなくなるか或いは適度に短い時間間隔でリセットされることがなくなり、やがて、監視タイマーがオーバーフローすることにより図２３の監視タイマーカウント処理のステップ２３０２の判定がＹＥＳとなる。
【０１７２】
この場合には、ステップ２３０３で、ＣＰＵ５０９への再度の割込みフラグとして、オーバーフローフラグＯＶＦが１にセットされる。
この結果、ＣＰＵ５０９は、再度の割込み処理として、図２４の動作フローチャートで示されるオーバーフロー割込み処理を実行し、ステップ２４０１で、ＣＰＵ５０９自身の動作を停止（ホールト）させる。
【０１７３】
以上の監視タイマーを使って制御により、模造されたアプリケーションカートリッジ５１３の使用が阻止される。
次に、図２２の音源割込み処理において、ステップ２２０２では、予めＤＲＡＭ５０７上に作成されている発音指示データが、バス５２６を介して、アプリケーションカートリッジ５１３内の音源回路５１５に転送される。この発音指示データは、ＣＰＵ５０９が前述した図１７のステップ１６１４又はステップ１６１５、或いは、図２１のステップ２１０１又はステップ２１０２などのＣＧ画像１０４（図１）の制御処理を実行するのに合せて生成され、ＤＲＡＭ５０７に記憶されている。
【０１７４】
なお、アプリケーションカートリッジ５１３内の音源回路５１５に対してのみではなく、本体側に装備されている音源回路５１６に対しても同様の制御が実行されてもよい。
【０１７５】
以上説明した図１６〜図２４の動作フローチャートで示される各処理が、ＣＰＵ５０９が実行する処理である。
最後に、図２５は、アプリケーションカートリッジ５１３内の音源回路５１５が実行する音源処理の動作フローチャートである。
【０１７６】
アプリケーションカートリッジ５１３が特には図示しないホルダーに装着されると、図２５の動作フローチャートの実行が開始され、まず、ステップ２５０１において、音源回路５１５内部のレジスタやワークＲＡＭの内容などがイニシャライズされる。
【０１７７】
その後、ステップ２５０２〜ステップ２５０７の各処理が、繰り返し実行される。
まず、音源回路５１５は、１発音単位分の楽音信号を生成する毎（例えば数秒分の楽音波形データの読出しを完了する毎）に、ステップ２５０２で、図５のＣＰＵ５０９への割込みフラグである音源割込み要求フラグをセットする。これに対しＣＰＵ５０９は、前述した図２２の音源割込み処理を実行することにより、次の発音単位に対する発音指示データを音源回路５１５に出力する。
【０１７８】
音源回路５１５は、ステップ２５０３において、入力されるデータに変化が起こるまで待機状態とする。入力されるデータに変化が起こると、更にステップ２５０４で、入力されるデータが安定するまで待機状態となる。
【０１７９】
入力されるデータが安定すると、ステップ２５０５で音源割込み要求フラグを０にリセットした後、ステップ２５０６で、入力される発音指示データを解釈する。
【０１８０】
そして、ステップ２５０７で、そのデータに対応する楽音信号を生成する。
１発音単位分の楽音信号の生成を完了すると、音源回路５１５は、再びステップ２５０２を実行し、図５のＣＰＵ５０９への割込みフラグである音源割込み要求フラグをセットする。
【０１８１】
以上のようにして音源回路５１５で生成された楽音信号は、ＣＰＵ５０９及び音源回路５１６を介して、アンプ５１７に送られ、ここで増幅された後、特には図示しないスピーカに、音声出力信号５２８として出力される。
＜第２の実施例の具体的な動作＞
次に、本発明の第２の実施例の具体的な動作について説明する。
【０１８２】
第２の実施例では、ゲームをスタートさせる前に、ユーザが、処理画面マップ上のどのブロック位置がクロマキーに関する部分として認識されているかを確認することを可能にする。前述したように、バット１０１がそれと同じ色彩を有する背景対象物と重なった場合は、そこにはバット１０１は存在しないものとして処理される（図１２のステップ１２０３〜ステップ１２０５参照）。この結果、バット１０１の位置の検出精度が下がり、ゲームの進行に影響を与えるおそれがある。従って、第２の実施例の機能により、ユーザは、バット１０１と同じ色彩を有する背景対象物が存在する位置にバット１０１を持っていかないように注意することができる。
【０１８３】
第２の実施例が前述した第１の実施例と異なる部分は、図５のＣＰＵ５０９が実行するＣＰＵ処理の部分である。図２９〜図３２に、第２の実施例におけるＣＰＵ処理の動作フローチャートを示す。この動作フローチャートは、第１の実施例における図１６に示されるＣＰＵ処理に対応する。
【０１８４】
まず、図２９において、ステップ２９０１〜ステップ２９０３の機能は、図１６のステップ１６０１〜ステップ１６０３の機能と同じである。
次に、ステップ２９０４において、図５のＶＤＰ５０６に対して画面クリア命令が実行される。これによって、図１の表示装置１０３の表示画面がクリアされる。また、ステップ２９０４では、処理画面マップの水平方向の列位置を示すレジスタＸと垂直方向の行位置を示すレジスタＹの値が０にリセットされる。
【０１８５】
続いて、ステップ２９１１でレジスタＸの値がクリアされると共にレジスタＹの値が＋１ずつインクリメントされながら、ステップ２９１２でレジスタＹの値が９６に達した（９５を越えた）と判定されるまで、ステップ２９０５〜ステップ２９０７→ステップ２９１１→ステップ２９１２→ステップ２９０５の処理が繰り返し実行される。このようにして、処理画面マップにおける９６行分の処理が実行される。
【０１８６】
また、ステップ２９１０でレジスタＸの値が＋１ずつインクリメントされながら、ステップ２９０９でレジスタＸの値が９６に達した（９５を越えた）と判定されるまで、ステップ２９０５〜ステップ２９０９→ステップ２９１０の処理が繰り返し実行される。このようにして、処理画面マップにおける１行内の９６ブロック分の処理が実行される。
【０１８７】
そして、レジスタＸとＹの現在値毎に実行されるステップ２９０５〜ステップ２９０８が、処理画面マップにおける１ブロック分の処理ということになる。以下の説明では、レジスタＸとＹのそれぞれの現在値を単にＸ、Ｙと記載し、現在のＸとＹの各値によって特定されるブロックを処理対象ブロックと呼ぶ。
【０１８８】
ステップ２９０５では、図５の画像認識回路５０８内の図２６に示されるＲＡＭの処理画面マップ領域からバス５２６を介して、水平方向に列位置Ｘ、垂直方向に行位置Ｙの処理対象ブロックの位置の論理値が読み出されて、その論理値が“１”であるか否かが判定される。
【０１８９】
ここで、図５の画像認識回路５０８が実行する、前述した図１２の動作フローチャートで示される図１０のステップ１００５の処理画面マップ作成処理において、ユーザが特には図示しないスタートスイッチを押すまでは、特には図示しないリジェクトスイッチは無効にされている。この結果、スタートスイッチが押される前に画像認識回路５０８において実行されている処理画面マップ作成処理では、ステップ１２０３の判定は必ずＮＯとなるため、クロマキー信号５２３に基づいてクロマキーに関する部分として認識されるブロック位置の論理値は、必ず“１”となる。
【０１９０】
なお、リジェクトスイッチは有効にしておき、ステップ２９０５では、図５の画像認識回路５０８内の図２６に示されるＲＡＭの初期画面マップ領域及び処理画面マップ領域の両方の領域からバス５２６を介して、水平方向に列位置Ｘ、垂直方向に行位置Ｙの処理対象ブロックの位置の論理値が読み出されて、その何れかの論理値が“１”であるか否かが判定されるように構成されてもよい。
【０１９１】
図２９の説明に戻って、ステップ２９０５の判定がＮＯならば、ステップ２９０９に進み、次の処理対象ブロックに対する処理に移る。
これに対して、ステップ２９０５の判定がＹＥＳなら、現在の処理対象ブロックの位置には、バット１０１又はバット１０１と同じ色彩を有する背景対象物が存在することになる。
【０１９２】
この場合には、ステップ２９０６で、Ｘ及びＹの値がそれぞれ６倍及び２倍されることにより、処理対象ブロックの位置から元のフレーム上での座標ｘ、ｙが算出され、その算出結果が図５のＤＲＡＭ５０７に記憶される（前述した図２５に関する説明参照）。
【０１９３】
ステップ２９０７では、ＣＰＵ５０９内のＤＭＡＣに、ＤＲＡＭ５０７に記憶された上記表示位置（ｘ、ｙ）を示すデータの読出し開始アドレスと読出しデータ数を示すオブジェクトテーブルデータの転送指示がセットされる。
【０１９４】
また、ステップ２９０８では、クロマキー表示を行わせるための所定のオブジェクト画面データ（例えば矩形オブジェクトなど）の、アプリケーションカートリッジ５１３内のアプリケーションＲＯＭ５１４上での読出し開始アドレスと読出しデータ数がセットされる。
【０１９５】
その後、ステップ２９０９に進み次の処理対象ブロックの位置に対する処理に移る。
以上の動作が９６×９６ブロックのサイズを有する処理画面マップの１画面分に対して実行される結果、ステップ２９１２の判定がＹＥＳとなった時点で、ＣＰＵ５０９内のＤＭＡＣには、バット１０１又はバット１０１と同じ色彩を有する背景対象物が存在するフレーム内位置に所定のオブジェクト画面データを表示させるためのデータ群がセットされたことになる。
【０１９６】
その後、ステップ２９１３で、レジスタＡＣＫＦに“１”がセットされる。そして、ステップ２９１４で、レジスタＡＣＫＦの値が“０”に戻るまで待機状態となる。
【０１９７】
前述したように、ＣＰＵ５０９は、同期信号５１９に基づいて決定される各フレーム周期内の垂直ブランキング期間が開始される時点毎に、ステップ２９１４の処理を中断して、図１８の動作フローチャートで示される垂直ブランキング割込み処理を実行する。これによって、上述したようにしてＤＭＡＣにセットされた、バット１０１又はバット１０１と同じ色彩を有する背景対象物が存在するフレーム内位置に所定のオブジェクト画面データを表示させるためのデータ群が、ＶＤＰ５０６に転送される。そして、ＶＤＰ５０６は、指定されたフレーム内位置に指定されたオブジェクト画面データが配置されたＣＧ画像１０４（図１）を表示する。
【０１９８】
転送が終了すると、レジスタＡＣＫＦの値が“０”に戻され（図１８のステップ１８０６参照）、図１８に示される垂直ブランキング割込み処理を終了するため、再開された図２９のステップ２９１４の判定がＹＥＳとなる。
【０１９９】
従って、ユーザが特には図示しないスタートスイッチを押しそれに対応するキースキャン信号５２７（図５）が入力されるまでの間、ステップ２９１５（図３０）の判定がＮＯとなり、図２９のステップ２９０４〜ステップ２９１４→図３０のステップ２９１５・・・という処理ループが繰り返され、スタート待機状態となる。そして、この間は、上述したように、フレーム周期毎に順次変化する処理画面マップの内容に従って、バット１０１又はバット１０１と同じ色彩を有する背景対象物が存在するフレーム内位置に所定のオブジェクト画面データを表示させるためのデータ群が順次ＶＤＰ５０６に転送され、ＶＤＰ５０６において、指定されたフレーム内位置に指定されたオブジェクト画面データが配置されたＣＧ画像１０４が順次表示される。
【０２００】
以上の動作の結果、図１のカメラ１０２が、例えば図３２（ａ） に示されるようは情景を撮像している場合、ＣＰＵ５０９からＶＤＰ５０６に対し、図３２（ｂ） に示されるようなＣＧ画像１０４の表示指示が行われ、ユーザは、図３２（ｃ） に示されるような、ＣＧ画像１０４とカメラ１０２によって撮像された実画像とが合成された画像を、表示装置１０３上で確認することができる。
【０２０１】
なお、ＣＧ画像１０４とカメラ１０２により撮像された実画像の合成処理は、図５の画像認識回路５０８が特には図示しない制御処理に基づいてビデオ信号変換回路５０５内の選択回路７０７（図７）にスーパーインポーズ信号５２５を出力することにより実現される。
【０２０２】
この場合、バット１０１に対応する部分はユーザの動きに合せて常に動き続けているが、バット１０１と同じ色彩を有する背景対象物の部分は静止している。このようにして、ユーザは、バット１０１と、バット１０１と同じ色彩を有する背景対象物とを確認することができる。
【０２０３】
スタートスイッチが押されそれに対応するキースキャン信号５２７が入力されると、図３０のステップ２９１５の判定がＹＥＳとなる。
その後の図３０のステップ２９１６〜ステップ２９２０、ステップ２９２１〜ステップ２９２３、図３１のステップ２９２４〜ステップ２９３１の機能は、それぞれ、第１の実施例における図１６のステップ１６０４〜ステップ１６０８、ステップ１６１０〜ステップ１６１２、図１７のステップ１６１３〜ステップ１６２０の機能と同じである。
＜第３の実施例の具体的な動作＞
続いて、本発明の第３の実施例の具体的な動作について説明する。
【０２０４】
第３の実施例では、第１の実施例で説明した初期画面マップの作成時に、ユーザが持っているバット１０１が、バット１０１と同じ色彩の背景対象物として初期画面マップに記憶されてしまうことを防止することができる。
【０２０５】
第３の実施例が前述した第１の実施例と異なる部分は、図５の画像認識回路５０８が実行する図１０のステップ１００３の初期画面マップ作成処理である。図３３に、第３の実施例における図１０のステップ１００３の初期画面マップ作成処理の詳細な動作フローチャートを示す。この動作フローチャートは、第１の実施例における図１１に示される動作フローチャートに対応しており、図３３と図１１とで、同じステップ番号が付加されたステップは、同じ機能を有する。
【０２０６】
第３の実施例における図３３の動作フローチャートが第１の実施例における図１１の動作フローチャートと異なる点は、図１１のステップ１１０９が図３３のステップ１１０９′に変更されている点のみである。
【０２０７】
前述したように、第１の実施例に関する図１１の動作フローチャートで示される初期画面マップ作成処理では、初期画面マップを安定して取得するために、複数フレーム分（複数画面分）のクロマキー信号５２３（図５）について処理が繰り返し実行され、ステップ１１０９で、画面マップ上のブロック位置毎に各繰返し処理で得られる論理値の論理和（ＯＲ）が演算された。この結果、上記複数のフレームにおいて撮像された情景内に存在するクロマキーは、全てバット１０１と同じ色彩の背景対象物として初期画面マップに記憶される。
【０２０８】
これに対して、第３の実施例に関する図３３の動作フローチャートで示される初期画面マップ作成処理では、複数フレーム分（複数画面分）のクロマキー信号５２３（図５）について処理が繰り返し実行され、ステップ１１０９′で、画面マップ上のブロック位置毎に各繰返し処理で得られる論理値の論理積（ＡＮＤ）が演算される。
【０２０９】
この結果、上記複数のフレーム期間でユーザが持っているバット１０１が撮像されてしまっても、そのバット１０１に対応するクロマキーは、フレーム間で常にゆれ動いているため、ブロック位置毎にフレーム間で論理積を演算した結果は“０”となる。このようにして、ユーザが持っているバット１０１が、バット１０１と同じ色彩の背景対象物として初期画面マップに記憶されてしまうことを防止することができるのである。
＜第４の実施例の具体的な動作＞
最後に、本発明の第４の実施例の具体的な動作について説明する。
【０２１０】
第４の実施例では、ゲームをスタートさせる場合に、撮像される情景内から、バット１０１と同じ色彩を有する背景対象物が完全になくなったことが検知された場合に初めて、ゲームをスタートできるように動作する。前述したように、バット１０１がそれと同じ色彩を有する背景対象物と重なった場合は、それがバット１０１であるか背景対象物であるか区別がつかなくなる。この結果、バット１０１の位置の検出精度が下がり、ゲームの進行に影響を与えるおそれがある。従って、第４の実施例の機能により、ユーザは、バット１０１と同じ色彩を有する背景対象物が撮像される情景内から完全になくなったことを確認した後にゲームを開始することができる。
【０２１１】
第４の実施例が前述した第１の実施例と異なる部分は、図５の画像認識回路５０８が実行する全体動作フローチャートの処理、及びＣＰＵ５０９が実行するＣＰＵ処理の部分である。
【０２１２】
まず、図３４及び図３５に、第４の実施例において画像認識回路５０８が実行する全体動作フローチャートを示す。この動作フローチャートは、第１の実施例における図１０に示される動作フローチャートに対応する。
【０２１３】
図３４において、まず、ステップ３４０１とステップ３４０２の機能は、図１０のステップ１００１、ステップ１００２の機能と同じである。
次に、ステップ３４０３では、初期画面マップが作成される毎に、レジスタＩＦに“０”がセットされ、この値は図５のバス５２６を介してＣＰＵ５０９に常に通知されている。
【０２１４】
また、ステップ３４０５の初期画面マップ作成処理は、図１０のステップ１００３の処理と同じであり、前述した図１１の動作フローチャートによって示される。
【０２１５】
次に、ステップ３４０６において、ステップ３４０５で生成された初期画面マップの水平方向の列位置を示すレジスタＸと垂直方向の行位置を示すレジスタＹの値が０にリセットされた後、ステップ３４１０でレジスタＸの値がクリアされると共にレジスタＹの値が＋１ずつインクリメントされながら、ステップ３４１１でレジスタＹの値が９６に達した（９５を越えた）と判定されるまで、ステップ３４０７〜ステップ３４０９→ステップ３４１０→ステップ３４１１→ステップ３４０７の処理が繰り返し実行される。このようにして、初期画面マップにおける９６行分の処理が実行される。
【０２１６】
また、ステップ３４０８でレジスタＸの値が＋１ずつインクリメントされながら、ステップ３４０９でレジスタＸの値が９６に達した（９５を越えた）と判定されるまで、ステップ３４０７〜ステップ３４０９→ステップ３４０７、・・・の処理が繰り返し実行される。このようにして、初期画面マップにおける１行内の９６ブロック分の処理が実行される。
【０２１７】
そして、ステップ３４０７において、初期画面マップのレジスタＸとＹの現在値によって特定されるブロック位置毎に、そのブロック位置の論理値が図２６のＲＡＭ内の初期画面マップ領域から読み出され、その論理値が“１”であるか否かが判定される。
【０２１８】
ステップ３４０７の判定がＮＯならば、ステップ３４０８に進んで次のブロック位置に対する処理に移る。
そして、ステップ３４０７の判定がＹＥＳとなったら、再びステップ３４０５の初期画面マップ作成処理に戻る。
【０２１９】
以上のようにして、第４の実施例では、生成された初期画面マップ内に１ブロックでも論理値が“１”である、即ち、１ブロックでもバット１０１と同じ色彩を有する背景対象物が存在すると判定された場合には、ユーザが、バット１０１と同じ色彩を有する背景対象物を撮像される情景内から除去しない限り、又は、撮像範囲をバット１０１と同じ色彩を有する背景対象物が存在しない範囲に移動させない限り、初期画面マップ作成処理が延々と繰り返される。
【０２２０】
撮像される情景内にバット１０１と同じ色彩を有する背景対象物が存在しなくなると、初期画面マップの全ブロックの論理値が“０”となるため、全ブロックについてステップ３４０７の判定がＮＯとなり、やがてステップ３４１１の判定がＹＥＳとなる。
【０２２１】
この結果、図３５のステップ３４１２で、レジスタＩＦの値が“０”にリセットされ、この値が図５のバス５２６を介してＣＰＵ５０９に通知される。
その後のステップ３４１３〜ステップ３４２４の処理は、第１の実施例における図１０のステップ１００４〜ステップ１０１５の処理と同じである。
【０２２２】
なお、第４の実施例では、最終的に得られる初期画面マップの全ブロックの論理値は“０”である。従って、図３５のステップ３４１４の処理画面マップ作成処理に対応する前述した図１２の動作フローチャートにおいて、処理画面マップのブロック位置毎に、その論理値を初期画面マップの対応するブロック位置の論理値と比較する必要はない。従って、ステップ１２０２の判定がＹＥＳとなった場合には、ステップ１２０３〜ステップ１２０５の処理を実行する必要はなく、すぐにステップ１２０６が実行されるように動作する。
【０２２３】
図５の画像認識回路５０８の以上説明した動作に対して、図５のＣＰＵ５０９は、図３６〜図３８に示される動作フローチャートの処理を実行する。この動作フローチャートの機能は、基本的には前述した第２の実施例に関する図２９〜図３１の動作フローチャートの機能と同じである。
【０２２４】
即ち、図３６〜図３８と図２９〜図３１とで、同じステップ番号が付加されたステップは、同じ機能を有する。具体的には、第４の実施例では、第２の実施例の場合と同様に、ゲームをスタートさせる前に、ユーザが、処理画面マップ上のどのブロック位置がクロマキーに関する部分として認識されているかを確認することができる。但し、第４の実施例では、図３４及び図３５の動作フローチャートとして説明したように、図５の画像認識回路５０８において作成される初期画面マップ上の全ブロックの論理値が“０”になった後でないと、図３５のステップ３４１４の処理画面マップ作成処理は実行されない。このため、ゲームをスタートさせる前にユーザが処理画面マップ上で確認するのは、背景対象物ではなくて、ユーザが手に持っているバット１０１の存在するブロック位置ということになる。
【０２２５】
第４の実施例における図３６〜図３８の動作フローチャートが第２の実施例における図２９〜図３１の動作フローチャートと異なる点は、図３０のステップ２９１５が図３７のステップ２９１５′に変更されている点のみである。
【０２２６】
即ち、第２の実施例では、ユーザが特には図示しないスタートスイッチを押せば、ステップ２９１５の判定がＹＥＳとなって、ゲームがスタートした。これに対して、第４の実施例では、図５の画像認識回路５０８からバス５２６を介してＣＰＵ５０９に通知されるレジスタＩＦの値が“０”になるまで、ゲームはスタートしない。
【０２２７】
このレジスタＩＦの値は、撮像される情景からバット１０１と同じ色彩を有する背景対象物が除去されずに初期画面マップ作成処理が繰り返されている間は、図３４のステップ３４０３で“１”にセットされ続けている。そして、撮像される情景からバット１０１と同じ色彩を有する背景対象物が除去され、初期画面マップ上の全てのブロックの論理値が“０”となった時点において、図３５のステップ３４１２において“０”にリセットされる。
【０２２８】
従って、第４の実施例では、撮像される情景からバット１０１と同じ色彩を有する背景対象物が除去された時点で初めて、ゲームのスタートが可能になるのである。
＜他の実施例＞
以上説明した実施例では、ＣＰＵ５０９は、画像認識回路５０８からＤＲＡＭ５１０に転送されてきた処理画面マップ（図２６参照）上のバット１０１の重心位置と、ＶＤＰ５０６内のオブジェクトコントロール回路８０３から取得した現在表示されているボール１０５（図１）を示すオブジェクトの重心位置との位置関係のみに基づいて、図１の表示装置１０３に表示されるＣＧ画像１０４を制御している。しかし、本発明はこれに限られるものではなく、画像認識回路５０８が、図１０のステップ１００６の衝突座標検出処理によってＲＡＭ（図２６）に記憶した最初の衝突座標及び最後の衝突座標、並びに、ステップ１００５の処理画面マップ作成処理によって上述のＲＡＭに記憶したバット１０１の存在範囲を示す上端座標、下端座標、左端座標、及び右端座標を、ＤＲＡＭ５０７に転送するように構成し、これらの各種位置データに基づいて図１の表示装置１０３に表示されるＣＧ画像１０４を制御するように構成されてもよい。
【０２３１】
【発明の効果】
本発明によれば、第１の状態において所定色の画像領域と同じ所定色が検知されなくなってから、第１の状態を解除して第２の状態を開始し、撮像画面内から所定色の画像領域の抽出を行うようにしているので、撮像すべき対象が安定した状態に移行してから、すなわち、バットなどの目的物と同じ所定色を有する背景対象物が撮像範囲内からなくなってから、バットなどの目的物のみを示す所定色の画像領域を安定に抽出できる。
【０２３２】
また、バットを振るユーザなどの現実の情景を撮像して得られる画面から抽出された、バットなどの所定色の画像領域の位置と、オブジェクト画面データの表示画面上での位置との位置関係を判断し、その判断内容に基づき、次に表示されるべき画面に対応する信号が生成されるようにすることも可能であり、そのようにすれば、ユーザは、あたかも実際のプレーをするようにバットを振る動作などを行うことにより、ＣＧ画像などによって表示されるテレビゲームに参加することなどが可能となる。この場合特に、撮像された画面からバットなどの特定の画像領域を抽出する際、クロマキーなどによる所定色の画像領域として抽出することによって、ユーザの動作を正確に把握することが可能となる。
【０２３３】
なお、本発明によれば、ユーザの動作に反応して画像制御を行うテレビゲームだけでなく、現実の様々な情景を撮像した結果に基づいて様々な画像制御を行う装置を実現することが可能となる。
【図面の簡単な説明】
【図１】実施例の動作の外観図（第１〜第４の実施例）である。
【図２】実施例の動作原理図（その１）（第１〜第４の実施例）である。
【図３】実施例の動作原理図（その２）（第１〜第４の実施例）である。
【図４】実施例の動作原理図（その３）（第１〜第４の実施例）である。
【図５】本発明の実施例の全体構成図（第１〜第４の実施例）である。
【図６】ビデオ信号入出力回路の構成図（第１〜第４の実施例）である。
【図７】ビデオ信号変換回路の構成図（第１〜第４の実施例）である。
【図８】ＶＤＰの構成図（第１〜第４の実施例）である。
【図９】クロック回路の説明図（第１〜第４の実施例）である。
【図１０】画像認識回路の全体動作フローチャート（第１、第２、第３の実施例）である。
【図１１】初期画面マップ作成処理の動作フローチャート（第１、第２、第４の実施例）である。
【図１２】処理画面マップ作成処理の動作フローチャート（その１）（第１〜第４の実施例）である。
【図１３】処理画面マップ作成処理の動作フローチャート（その２）（第１〜第４の実施例）である。
【図１４】衝突検出処理の動作フローチャート（第１〜第４の実施例）である。
【図１５】重心計算処理の動作フローチャート（第１〜第４の実施例）である。
【図１６】ＣＰＵ処理の動作フローチャート（その１）（第１、第３の実施例）である。
【図１７】ＣＰＵ処理の動作フローチャート（その２）（第１、第３の実施例）である。
【図１８】垂直ブランキング割込み処理の動作フローチャート（第１〜第４の実施例）である。
【図１９】タイマー割込み処理の動作フローチャート（第１〜第４の実施例）である。
【図２０】衝突フラグ割込み処理の動作フローチャート（第１〜第４の実施例）である。
【図２１】第ｎ画像処理の動作フローチャート（第１〜第４の実施例）である。
【図２２】音源割込み処理の動作フローチャート（第１〜第４の実施例）である。
【図２３】監視タイマーカウント処理の動作フローチャート（第１〜第４の実施例）である。
【図２４】オーバーフロー割込み処理の動作フローチャート（第１〜第４の実施例）である。
【図２５】音源処理の動作フローチャート（第１〜第４の実施例）である。
【図２６】フレームの画素構成と画面マップとの関係図（第１〜第４の実施例）である。
【図２７】画像認識回路のＲＡＭマップを示した図（第１〜第４の実施例）である。
【図２８】上・下・左・右端座標の検出処理の説明図（第１〜第４の実施例）である。
【図２９】ＣＰＵ処理の動作フローチャート（その１）（第２の実施例）である。
【図３０】ＣＰＵ処理の動作フローチャート（その２）（第２の実施例）である。
【図３１】ＣＰＵ処理の動作フローチャート（その３）（第２の実施例）である。
【図３２】第２の実施例の説明図である。
【図３３】初期画面マップ作成処理の動作フローチャート（第３の実施例）である。
【図３４】画像認識回路の全体動作フローチャート（その１）（第４の実施例）である。
【図３５】画像認識回路の全体動作フローチャート（その２）（第４の実施例）である。
【図３６】ＣＰＵ処理の動作フローチャート（その１）（第４の実施例）である。
【図３７】ＣＰＵ処理の動作フローチャート（その２）（第４の実施例）である。
【図３８】ＣＰＵ処理の動作フローチャート（その３）（第４の実施例）である。
【符号の説明】
１０１ バット
１０２ カメラ
１０３ 表示装置
１０４ ＣＧ画像
１０５ ボール
５０１ ＣＣＤ
５０２ クロックドライバー
５０３ ビデオ信号入出力回路
５０４ 発振器（ＯＳＣ）
５０５ ビデオ信号変換回路
５０６ ＶＤＰ
５０７ ＤＲＡＭ
５０８ 画像認識回路
５０９ ＣＰＵ
５１０ ＤＲＡＭ
５１１ ＯＳ ＲＯＭ
５１２ クロック回路
５１３ アプリケーションカートリッジ
５１４ アプリケーションＲＯＭ
５１５、５１６ 音源回路
５１７ アンプ
５１８ ＣＣＤ出力信号
５１９ 同期信号
５２０ デジタルコンポジットビデオ入力信号
５２１ デジタルコンポジットビデオ出力信号
５２２ アナログコンポジットビデオ出力信号
５２３ クロマキー信号
５２４ デジタルＲＧＢ信号
５２５ スーパーインポーズ信号
５２６ バス
５２７ キースキャン信号
５２８ 音声出力信号５２８[0001]
[Industrial application fields]
The present inventionFor example, in a video game or the like, an image extraction apparatus and an image extraction method for extracting an image area of a specific condition from a captured screen based on a color signalAbout.
[0002]
Traditionally,As this kind of image extraction device,ImagedscreenA predetermined color, for example, a bright blue imageregionThere is known a chroma key detection device that detects a key as a chroma key.
[0003]
Such a chroma key detecting device is used for the purpose of synthesizing an image in television shooting or the like, and for example, sequentially detects the position of a bat of a predetermined color held by a user and displays a display image based on the detected position. The present invention can also be applied to an image control apparatus such as a video game that controls the image.
[0004]
[Problems to be solved by the invention]
However, the conventional chroma key detection device simply detects an image of a predetermined color from among the captured images. For example, in the above-described video game or the like, in the case where it is desired to detect a chroma key of a constantly moving object such as a bat, When there is an object having the same color as the bat, the object is detected as a bat or the like, and there is a possibility that the intended chroma key cannot be detected.
[0005]
The subject of the present invention isFrom the captured screen,AimImage areaExactlyAllow extractionThere is.
[0006]
[Means for Solving the Problems]
The image extraction apparatus of the present invention isSetting means for setting an image area of a predetermined color to be extracted in advance, an imaging means, and an imaging area by the imaging meansExtract an image area of a predetermined colorWith extraction meansIn the image extraction device,A predetermined color included in an image area set by the setting means from an imaging area imaged by the imaging means;The same predetermined color continues to be detectedDuring this time, the extraction operation by the extraction means is prohibited, and when the same predetermined color is no longer detected, the control means for starting the extraction operation by the extraction means (the image recognition circuit 508 for executing the operation flowcharts of FIGS. 34 and 35). And CPU 509 for executing the operation flowcharts of FIGS.. Here, the image area is obtained by sequentially capturing, for example, a user's action with a bat or the like.Further, the state in which the extraction operation is prohibited and the state after the extraction operation is started correspond to the state before the game start and the state after the game start in the example of the video game, for example.
[0011]
In addition, image storage means (application ROM 514, DRAM 507, etc. in FIG. 5) for storing the object screen data to be output, and a display for designating the position to be output on the display screen of the object screen data stored in the image storage means Position specifying means (such as CPU 509 in FIG. 5) and output control means (FIG. 8) for controlling to output an image corresponding to the object screen data stored in the image storage means at the position specified by the display position specifying means. The object control circuit 803 in FIG.controlInitiated by meansExtraction operation by the extraction meansDetermining means (image recognition circuit 508 for executing step 1006 in FIG. 10 and the like) for determining the positional relationship between the position of the image area of the predetermined color extracted in step S3 on the imaged screen and the position where the object screen data is displayed. Etc.) and signal generation means (CPU 509 in FIG. 5) for generating a signal corresponding to the screen to be displayed next based on the content of the determination by the determination means. Here, the image storage means can be configured to store a plurality of object screen data. In that case, the display position specifying means may be configured to specify at least one type of the plurality of object screen data stored in the image storage means. Further, the positional relationship is, for example, the overlapping of positions.
[0012]
Further, it is possible to further include signal output means (such as the chroma key circuit 706 in the video signal conversion circuit 505 in FIG. 7) for outputting a chroma key signal based on the color signal of the imaged screen.controlInitiated by meansExtraction operation by the extraction meansIs extraction of the image area of the predetermined color by recognizing the chroma key signal output by the signal output means.
[0013]
Next, the image extraction method of the present invention includes:An image area of a predetermined color to be extracted in advance is set and included in the area captured by the imaging unitAn image extraction method for extracting an image area of a predetermined color,the aboveCaptured by the imaging unitA predetermined color included in an image area set in advance from the imaging area;The same predetermined color continues to be detectedWhile the extraction operation is prohibited, the same as aboveWhen the predetermined color is not detected,The extraction operation is started (operation flowchart in FIGS. 34 to 38)..Here, the state in which the extraction operation is prohibited and the state after the extraction operation is started correspond to the state before the game start and the state after the game start in the example of the video game, for example.
[0014]
This image extraction method of the present invention can further include a signal output step (such as step 1003 in the overall operation flowchart of the screen recognition circuit in FIG. 10) for outputting a chroma key signal based on the color signal of the captured screen. Yes, in that caseExtraction operation started aboveIs the extraction of the image area of the predetermined color by recognizing the chroma key signal output in the signal output step.
[0020]
[Action]
In the present invention, for example, a first state which is a state before the start of the video game(The above extraction operation is prohibited)The first state is maintained as long as the same predetermined color as the predetermined color image region indicating the target object such as a bat is continuously detected. Thereafter, when the predetermined color is no longer detected, the maintenance of the first state is canceled.(The above extraction operation starts)Then, for example, a second state, which is a state after the start (in progress) of the video game, is started, and an image region of a predetermined color indicating an object such as a bat is sequentially obtained.
[0025]
By doing this,After the subject to be imaged has shifted to a stable state,That is, after the background object having the same predetermined color as the object such as the bat disappears from the imaging range,Show only objects such as batsPredetermined colorCan be extracted stably.
[0027]
Furthermore,In the second state,The bat etc. extracted from the screen obtained by imaging the real scene such as the user shaking the batPredetermined colorIf the positional relationship between the position of the image area and the position of the object screen data on the display screen is determined, and a signal corresponding to the screen to be displayed next is generated based on the determination content The user can participate in a video game displayed by a CG image or the like by performing an operation such as swinging a bat so as to actually play.
[0028]
Especially in this case,When extracting a specific image area such as a bat from the imaged screen,Predetermined color image with chroma key etc.regionAsExtractionBy doing so, it is possible to accurately grasp the user's operation.
[0029]
【Example】
Hereinafter, first to fourth embodiments of the present invention will be described in detail with reference to the drawings.
<Appearance of user's operation in embodiment and operation principle of embodiment>
Before describing the configurations and specific operations of the first to fourth embodiments, the appearance of the user's common operations and the common operation principles of these embodiments will be briefly described.
[0030]
In the first to fourth embodiments, the present invention is applied to a baseball video game.
As shown in FIG. 1, a user holds a bat 101 painted in a predetermined color, for example, blue, and is displayed on a display device 103 as a CG (computer graphics) image 104. Shake bat 101 toward 105.
[0031]
The camera 102 images a scene including a user with the bat 101. By extracting the chroma key signal related to the predetermined color from the captured image, a coordinate map 202 of the bat 101 as shown in FIG. 2 is created, and further, the bat 101 on the coordinate map 202 is created. The barycentric coordinate (x mark in the figure) is detected. Then, a positional relationship 203 between the coordinate map 202 and the barycentric coordinates of the bat 101 and the coordinates 201 of the ball 105 and the barycentric coordinates (marked in the drawing) by the CG as shown in FIG. Thus, the presence / absence of the collision of the ball 105 with the bat 101 and the CG and the positional relationship between the gravity center coordinate of the bat 101 and the gravity center coordinate of the ball 105 when the collision occurs are extracted.
[0032]
As shown in FIG. 2, when it is determined that the CG ball 105 hits the bat 101 and the deviation between the centroid coordinates of the bat 101 and the CG coordinates of the ball 105 is within a predetermined allowable range. For example, it is determined that the user has hit a home run, and the CG image 104 changes to an image corresponding to the home run.
[0033]
Alternatively, as shown in FIG. 3, when it is determined that the CG 101 hits the bat 101 and the center of gravity coordinates of the CG 101 are shifted below the center of gravity of the bat 101, for example, It is determined that the user has hit the ball, and the CG image 104 changes to an image corresponding to the ball.
[0034]
Further, as shown in FIG. 4, when it is determined that the CG ball 105 hits the bat 101 and the center of gravity coordinates of the ball 105 by CG are shifted above the center of gravity coordinates of the bat 101, for example, It is determined that the user hits the fly, and the CG image 104 changes to an image corresponding to the fly.
[0035]
As described above, in the first to fourth embodiments, the user can participate in the video game displayed by the CG image 104 by swinging the bat 101 as if playing the actual game.
<Configuration of Example>
A specific specific configuration common to the first to fourth embodiments for realizing the above-described operation will be described below.
[0036]
FIG. 5 is a common overall configuration diagram of the first to fourth embodiments of the present invention.
First, the clock driver 502 directly controls the operation of the CCD 501 provided in the camera 102 of FIG. 1 based on the clock input from the oscillator (OSC) 504 via the video signal input / output circuit 503 and at the same time HSYNC. A synchronization signal 519 such as a signal and a VSYNC signal is generated and supplied to a video signal input / output circuit 503, a VDP 506, an image recognition circuit 508, and a CPU 509. Therefore, these circuits can operate in synchronization with each other based on the synchronization signal 519.
[0037]
A CCD output signal 518 output from the CCD 501 is input to the video signal input / output circuit 503.
The video signal input / output circuit 503 performs an operation of converting the CCD output signal 518 into a digital composite video input signal 520 and an operation of converting the digital composite video output signal 521 from the video signal conversion circuit 505 into an analog composite video output signal 522. .
[0038]
The configuration of the video signal input / output circuit 503 is shown in FIG.
In FIG. 6, a CCD output signal 518 from a CCD 501 (FIG. 5) is sampled in an analog manner by a sampling circuit 601.
[0039]
The amplitude level of the output of the sampling circuit 601 is adjusted by an automatic gain adjustment circuit (AGC) 602.
The output of the AGC 602 is converted by the γ correction circuit 603 so that the signal level with respect to the optical power has a predetermined γ characteristic.
[0040]
The output of the γ correction circuit 603 is converted into an 8-bit digital composite video input signal 520 by an 8-bit A / D converter 604. The digital composite video input signal 520 is output to the video signal conversion circuit 505 in FIG.
[0041]
On the other hand, the 8-bit digital composite video output signal 521 output from the video signal conversion circuit 505 is converted into an analog composite video output signal 522 by the 8-bit D / A converter 605. The analog composite video output signal 522 is output to the display device 103 in FIG. As a result, the display device 103 displays the CG image 104 corresponding to the analog composite video output signal 522.
[0042]
Returning to FIG. 5, the video signal conversion circuit 505 generates the chroma key signal 523 from the digital composite video input signal 520 and outputs it to the image recognition circuit 508, based on the superimpose signal 525 from the image recognition circuit 508, A color signal corresponding to the digital composite video output signal 521 input from the video signal input / output circuit 503 or a digital RGB signal 524 output from the video display processor (VDP) 506 is selected, and the selected color signal is selected. An operation for outputting the corresponding digital composite video output signal 521 is performed.
[0043]
The configuration of the video signal conversion circuit 505 is shown in FIG.
The color separation filter 701 receives a digital composite video input signal 520 input from the video signal input / output circuit 503 (FIG. 5), and receives a G (green) signal, a Cy (cyan) signal, and Ye (corresponding to the signal). The three types of color signals are output.
[0044]
The contour correction circuit 702 receives the above-described three kinds of color signals and performs a process of correcting the contour of an image determined by these color signals.
Of the outputs from the contour correction circuit 702, signals other than the G (green) signal are input to the white balance circuit 703. The white balance circuit 703 performs white balance adjustment on the input signal.
[0045]
The output of the white balance circuit 703 is input to the selection filter 704. The selection filter 704 outputs an R (red) signal and a B (blue) signal corresponding to the input.
[0046]
The matrix circuit 705 inputs the R (red) signal, the B (blue) signal, and the G (green) signal output from the contour correction circuit 702, and corresponds to the conversion represented by the following equation. By executing the calculation, the luminance signal Y and the color difference signals RY and BY are output.
Y = (8/27) R + (16/27) G + (1/9) B
RY = (15/27) R- (16/27) G- (1/9) B
B−Y = − (8/27) R− (16/27) G + (8/9) B
In the chroma key circuit 706, as shown in FIG. 9, the color difference signal RY has a value in the range from the minimum value RYmin to the maximum value RYmax, and the color difference signal BY has a minimum value B-. When the value falls within the range of Ymin to the maximum value B−Ymax, a chroma key signal 523 having a logical value “1” is output. The chroma key signal 523 indicates a logical value “1” when, for example, a color signal indicating bright blue corresponding to the bat 101 in FIG. 1 is input. By using the chroma key signal 523, an image recognition circuit 508 (FIG. 5), which will be described later, can detect the position of the bat 101 shaken by the user, and determines whether or not the bat 101 and the ball 105 by the CG collide. Can be judged.
[0047]
Next, the matrix circuit 708 receives the digital RGB signal 524 from the VDP 506 and outputs a luminance signal Y and color difference signals RY and BY corresponding thereto.
[0048]
The selection circuit 707 selects one of the outputs of the matrix circuit 705 or 708 based on the superimpose signal 525 from the image recognition circuit 508 (FIG. 5) and outputs the selected output to the modulation circuit 709. The modulation circuit 709 generates and outputs a digital composite video output signal 521 corresponding to the input. In the first to fourth embodiments, the superimpose signal 525 from the image recognition circuit 508 (FIG. 5) has a specification that causes the selection circuit 707 to always select the output of the matrix circuit 708. As a result, the CG image 104 corresponding to the digital RGB signal 524 generated by the VDP 506 is always displayed on the display device 103 of FIG. However, by controlling the superimpose signal 525, the actual image corresponding to the digital composite video input signal 520 generated based on the imaging result of the camera 102 and the CG image generated by the VDP 506 are combined and combined. The displayed image can be displayed on the display device 103.
[0049]
Returning to FIG. 5, the video display processor (VDP) 506 stores the object screen data and background screen data constituting the CG image 104 in the DRAM 507 as needed based on the designation from the CPU 509, and based on the designation from the CPU 509. These data are arranged at designated coordinate positions, a digital RGB signal 524 corresponding to the arrangement result is generated, and output to the video signal conversion circuit 505. As a result, the video signal conversion circuit 505 generates the digital composite video output signal 521 corresponding to the digital RGB signal 524 as described above, and the video signal input / output circuit 503 corresponds to the digital composite video output signal 521. By generating the analog composite video output signal 522, the CG image 104 is displayed on the display device 103 (FIG. 1).
[0050]
The configuration of the VDP 506 is shown in FIG.
Object screen data and background screen data transferred from the CPU 509 on the bus 526 are stored in the DRAM 507 (FIG. 5) via the CPU interface circuit 801 and the DRAM control circuit 802.
[0051]
The object table data transferred from the CPU 509 on the bus 526 is stored in the object table RAM 804 via the CPU interface circuit 801 and the object control circuit 803. This object table data is a table showing a correspondence table between each object screen data stored in the DRAM 507 and the corresponding ID number. When the CPU 509 designates the display position of each object screen data on the CG image 104 (FIG. 1) as time passes, the CPU 509 designates an ID number corresponding to each object screen data and each display position. As a result, the object control circuit 803 reads the object screen data corresponding to the designated ID number from the DRAM 507 via the DRAM control circuit 802, develops it in the line buffer RAM 808, and then stores the contents in the RGB color difference data processing circuit 806. Output. Further, the object control circuit 803 notifies the CPU 509 of the center of gravity of the object indicating the ball 105 (FIG. 1) currently displayed based on the designation from the CPU 509 via the bus 526 from the CPU interface circuit 801.
[0052]
On the other hand, the background control circuit 805 reads background screen data designated by the CPU 509 from the DRAM 507 via the RAM control circuit 802, performs display position control or scroll control on the data, and then performs RGB control. The data is output to the color difference data processing circuit 806.
[0053]
The RGB color difference data processing circuit 806 refers to the color look-up table in the color look-up table RAM 807, and thereby the object screen data designated from the object control circuit 803 and the background screen data designated from the background control circuit 805. Are generated, and are output to the video signal conversion circuit 505 (FIG. 5). In this case, the RGB color difference data processing circuit 806 also performs control for determining whether to display object screen data or background screen data for each display position. The RGB color difference data processing circuit 806 sets the logical value of the object screen data output signal YSOBJ (hereinafter referred to as the YSOBJ signal) to “1” and outputs the background screen data when outputting the object screen data. In this case, the logic value of the background screen data output in progress signal YSBG (hereinafter referred to as YSBG signal) is set to “1”. The YSOBJ signal and the YSBG signal are output to the image recognition circuit 508.
[0054]
Returning to FIG. 5, the image recognition circuit 508 uses the chroma key signal 523 for one frame input from the video signal conversion circuit 505 for the two internal chroma key signals for each frame period determined in synchronization with the synchronization signal 519. The YSOBJ signals for one screen map (described later) output from the RGB color difference data processing circuit 806 in the VDP 506 are alternately held in two internal YSOBJ signal buffers. At the same time, the image recognition circuit 508 uses the chroma key signal 523 for one frame held in one of the two chroma key signal buffers in the previous frame period to detect the bat 101 (FIG. 1) that the user shakes. Create a screen map showing the location. When the image recognition circuit 508 determines that the bat 101 exists at a block position (described later) on the screen map in the course of this process, the image recognition circuit 508 uses two YSOBJ signals for the previous frame period. In the YSOBJ signal for one screen map held in one of the buffers, it is determined whether or not the logical value of the YSOBJ signal corresponding to the block position is “1”, that is, the bat 101 exists. It is determined whether or not object screen data indicating the ball 105 (FIG. 1) exists at the current block position. In this way, the image recognition circuit 508 detects the presence or absence of a collision between the bat 101 and the ball 105. Further, the image recognition circuit 508 calculates the position of the center of gravity of the bat 101 on the screen map every time the creation of the screen map for one screen is completed, stores the calculation result in the DRAM 510 via the bus 526, and the collision state Is detected, a collision flag is set as an interrupt flag to the CPU 509.
[0055]
The CPU 509 operates in accordance with the operating system stored in the OS ROM 511. In particular, an application program is loaded from the application ROM 514 (either # 1 or # 2) in the application cartridge 513 inserted in a slot (not shown) into the program area on the DRAM 510. To load a video game such as the baseball game described in the first to fourth embodiments. In this case, the CPU 509 processes the key scan signal 527 input from a start switch and a stop switch (not shown) in particular, and displays the CG image 104 (FIG. 1) from the application ROM 514 in the application cartridge 513 as time passes. The object screen data and background screen data to be configured are sequentially read out, and these data are transferred from the bus 526 to the DRAM 507 via the VDP 506 within the vertical blanking period synchronized with the synchronization signal 519.
[0056]
At the same time, the CPU 509 sequentially designates the display position on the CG image 104 (FIG. 1) of each object screen data transferred to the DRAM 507 as time elapses.
[0057]
In addition, the CPU 509 executes the collision flag interrupt process at the timing when the collision flag indicating that the image recognition circuit 508 has detected the collision state is set. In this interrupt processing, the CPU 509 determines the position of the center of gravity of the bat 101 on the screen map calculated by the image recognition circuit 508 stored in the DRAM 510 and the currently displayed ball 105 (from the object control circuit 803 in the VDP 506). The positional relationship with the gravity center position of the object shown in FIG. Then, based on the determination result, the CPU 509 executes necessary image processing.
[0058]
Further, the CPU 509 causes the tone generator circuit 515 to generate a musical sound signal by sequentially transferring necessary data to the tone generator circuit 515 in the application cartridge 513. Further, the CPU 509 causes the tone generator circuit 516 in the main body to generate a musical tone signal as necessary. The musical tone signal generated by the tone generator circuit 515 is sent to the amplifier 517 via the CPU 509 and the tone generator circuit 516, amplified there, and then output as an audio output signal 528 to a speaker (not shown).
[0059]
In addition, the CPU 509 also performs an inspection process as to whether or not the application cartridge 513 is imitated.
Next, specific operations of the embodiment having the above-described configuration will be sequentially described below.
<Specific operation of the first embodiment>
First, the specific operation of the first embodiment will be described.
[0060]
FIG. 10 is an overall operation flowchart realized as an operation in which the image recognition circuit 508 executes a program stored in a memory (not shown).
The image recognition circuit 508 creates a screen map indicating the position of the bat 101 (FIG. 1) shaken by the user based on the chroma key signal 523 from the video signal conversion circuit 505, and supplies it as a YSOBJ signal from the VDP 506 based on the screen map. 1 is detected whether or not the object screen data indicating the ball 105 in FIG.
[0061]
Here, as shown in FIG. 26, a frame that is an image pickup unit for one screen in the CCD 501 in FIG. 5 has a configuration of 576 columns in the horizontal direction × 192 rows in the vertical direction. Therefore, the chroma key signal 523 output from the video signal conversion circuit 505 is a signal indicating a logic value for 576 × 192 pixels. In order to reduce the processing load, as shown in FIG. 26, the image recognition circuit 508 converts the logical values of a total of 12 pixels including 6 pixels in the horizontal direction and 2 pixels in the vertical direction in the chroma key signal 523 into one logical value. Process to compress to value. A unit in which this logical value is stored is called a block. As a result, as shown in FIG. 26, the logical value of 576 × 192 pixels in the chroma key signal 523 is compressed to the logical value of a block of 96 columns in the horizontal direction × 96 rows in the vertical direction. A screen map related to the chroma key is defined by a set of logical values of 96 × 96 blocks.
[0062]
On the other hand, the object screen data and the background screen data specified by the YSOBJ signal and the YSBG signal are expressed with a resolution of 96 columns in the horizontal direction and 96 rows in the vertical direction. Therefore, the VDP 506 generates digital data based on the screen data. When the RGB signal 524 is generated, these screen data are expanded in accordance with the display resolution of the display device 103 (FIG. 1).
[0063]
In FIG. 10, first, in step 1001, the contents of a RAM (not shown) in the image recognition circuit 508 (see FIG. 27 described later) are initialized.
Next, after the power is turned on, if it is determined in step 1002 that an initial screen map has not been created, an initial screen map creation process is executed in step 1003.
[0064]
The initial screen map is a screen map related to a chroma key for a scene in which a user with the bat 101 does not exist, which is captured by the camera 102 of FIG. The purpose of creating the initial screen map is that when a chroma key corresponding to the bat 101 is detected as a processing screen map, which will be described later, if an object having the same color as the bat 101 is present in the background, the background is an error of the bat 101. In order to prevent the detected position from being detected, the position of the background object having the same color as the bat 101 is detected in advance. In the processing screen map creation processing in step 1005 described later, a processing screen map is created for each frame period based on the result of the camera 102 in FIG. 1 capturing a scene where a user with the bat 101 is present. In this embodiment, when the position recognized as the portion related to the chroma key on the processing screen map is simultaneously recognized as the position where the background object having the same color as the bat 101 exists on the initial screen map. In this position, the bat 101 is processed as not existing.
[0065]
As a result of the initial screen map creation processing in step 1003, an initial screen map having a size of 96 × 96 blocks is created in the data format shown in FIG. 27 on a RAM (not shown) in the image recognition circuit 508. Is done. Since this RAM stores data with 16 bits as one word, the logical value for one row is stored as data of 6 words (= 96/16). Therefore, on the RAM, the initial screen map occupies a data area of 6 × 96 words.
[0066]
FIG. 11 shows a detailed operation flowchart of the initial screen map creation processing in step 1003.
In the following processing, in order to stably acquire the initial screen map, the processing is repeatedly executed for the chroma key signal 523 for a plurality of frames (for a plurality of screens), and is obtained in each repetition processing for each block position on the screen map. The final logical value for each block position is determined by calculating the logical sum (OR) of the logical values. For this reason, the number of times of sampling, which is the number of repetitions of the process, is set in step 1101, and the value of the register n reaches the set number of times each time processing of one frame is completed because the determination in step 1114 described later becomes YES. In step 1116, the value of the register n is incremented by +1, and the processing for each frame shown in steps 1102 to 1114 is executed.
[0067]
Here, although not particularly illustrated, as described above, the image recognition circuit 508 has two frames (two screens) of chroma key signal buffers each having the size shown in FIG. For each frame period determined in synchronization with the synchronization signal 519 (FIG. 5), the chroma key signal 523 for one frame shown in FIG. 26 (a) input from the video signal conversion circuit 505 is converted into the two chroma keys described above. Alternately held in the signal buffer. Then, the processing for each frame shown in steps 1102 to 1114 in FIG. 11 is performed on the chroma key signal 523 for one frame held in one of the two chroma key signal buffers in the previous frame period. Is done. At the same time, the chroma key signal 523 corresponding to the current frame period is sequentially input to the other chroma key signal buffer. Each time the determination in step 1114 is NO, the chroma key signal buffer to be processed is switched.
[0068]
Note that one chroma key signal buffer having the size shown in FIG. 26A stores 1-bit data for storing a logical value of “0” or “1” of the chroma key signal 523 corresponding to each pixel. As long as it is possible, the storage capacity is 1 bit × 576 × 192 = 110592 bits = 6912 words.
[0069]
Next, details of the processing for each frame shown in steps 1102 to 1114 will be described.
First, in step 1102, the values of the register X indicating the horizontal column position and the register Y indicating the vertical row position of the generated initial screen map are reset to 0, and then the value of the register X in step 1113. Is cleared and the value of the register Y is incremented by +1. Until it is determined in step 1114 that the value of the register Y has reached 96 (exceeded 95), step 1103 to step 1112 → step 1113 → step 1114 → Step 1103 is repeatedly executed. In this way, the process for 96 lines in the initial screen map is executed.
[0070]
Further, while the value of the register X is incremented by +1 at step 1111, the processing from step 1103 to step 1112 → step 1103 until it is determined at step 1112 that the value of the register X has reached 96 (exceeded 95). Is repeatedly executed. In this way, processing for 96 blocks in one line in the initial screen map is executed.
[0071]
Steps 1103 to 1110 executed for each current value of the registers X and Y are processing for one block in the initial screen map. In the following description, the current values of the registers X and Y are simply referred to as X and Y, and a block specified by the current values of X and Y is referred to as a processing target block.
[0072]
In step 1103, each logical value of the chroma key signal 523 at the column positions (6X + 1) to (6X + 6) in the horizontal direction and the row positions (2Y + 1) and (2Y + 2) in the vertical direction from the chroma key signal buffer to be processed. Is read out. For example, if X = 0 and Y = 0, the data at the column positions 1 to 6 in the horizontal direction and the row positions 1 and 2 in the vertical direction (block 1 in FIG. 26A) are read. If X = 1 and Y = 0, the data at the column positions 7 to 12 in the horizontal direction and the row positions 1 and 2 in the vertical direction (block 2 in FIG. 26A) are read. Further, if X = 0 and Y = 1, the data at the column positions 1 to 6 in the horizontal direction and the row positions 3 and 4 in the vertical direction (block 97 in FIG. 26A) are read.
[0073]
In step 1104, it is determined whether or not the logical value “1” of the chroma key signal 523 for 12 pixels in the processing target block read in step 1103 is 6 pixels or more.
[0074]
If the determination in step 1104 is YES, “1” is set as a logical value corresponding to the processing target block in a predetermined register in step 1105. If the determination in step 1104 is NO, “0” is set in the register as a logical value in step 1106. "Is set.
[0075]
If it is determined in step 1107 that the current process is the process in the first frame among the processes for a plurality of frames described above for creating the initial screen map, in step 1110, The logical value of the processing target block set in the predetermined register in step 1106 corresponds to the processing target block of the Xth column in the horizontal direction and the Yth row in the vertical direction in the initial screen map area in the RAM (FIG. 27). The address is written as it is.
[0076]
On the other hand, if it is determined in step 1107 that the current process is a process in the second and subsequent frames among the processes for the plurality of frames, the initial screen map area in the RAM (FIG. 27) is used. The logical value of the address corresponding to the processing target block of the Xth column in the horizontal direction and the Yth row in the vertical direction is read, and the processing target block set in the predetermined register in step 1105 or step 1106 A logical sum (OR) is calculated with the logical value of, and the contents of the predetermined register are rewritten with the calculation result. Thereafter, in step 1110, the contents of the address corresponding to the processing target block in the X-th column in the horizontal direction and the Y-th row in the vertical direction in the initial screen map area in the RAM (FIG. 27) are set in the logic set in the register. Rewritten to a value.
[0077]
The processes in steps 1103 to 1110 are repeated for 96 × 96 blocks of the initial screen map as described above to create an initial screen map for one frame, and there are a plurality of such initial screen maps. An initial screen map for one screen is finally obtained in the RAM (FIG. 27) by calculating the logical sum between the frames for each block position, and processing in step 1003 in FIG. Exit.
[0078]
The initial screen map creation process in step 1003 is configured to be automatically executed when it is determined in step 1002 that the initial screen map has not been created. For example, the user can use the camera 102 in FIG. The initial screen map creation process of step 1003 may be executed at a timing when a predetermined set button (not shown) is pressed at a position where the image is not picked up.
[0079]
Next, the processing from step 1004 to step 1015 in FIG. 10 is repeatedly executed in synchronization with the frame period determined by the synchronization signal 519 (FIG. 5). Here, mainly, a processing screen map showing the position of the bat 101 (FIG. 1) shaken by the user and a processing screen map creation processing for sequentially creating data necessary for the center-of-gravity calculation processing executed in step 1011 described later (step 1005). The collision coordinate detection process (step 1006) for determining whether or not the object screen data indicating the ball 105 (FIG. 1) exists at the block position where the presence of the bat 101 is detected by this process, the bat on the process screen map The center of gravity calculation process (step 1011) for calculating the center of gravity position of 101 and the process of setting a collision flag (step 1013) as an interrupt flag to the CPU 509 when it is determined that the bat 101 and the ball 105 collide with each other. Executed.
[0080]
In this case, although not particularly illustrated, as described above, the image recognition circuit 508 has two frames (two screens) of chroma key signal buffers each having the size shown in FIG. For each frame period determined in synchronization with the synchronization signal 519 (FIG. 5), the chroma key signal 523 for one frame shown in FIG. 26 (a) input from the video signal conversion circuit 505 is used for two chroma key signals. Hold alternately in the buffer. Although not shown in particular, as described above, the image recognition circuit 508 has YSOBJ signal buffers each having a size shown in FIG. 26B for two screen maps, and a synchronization signal 519 (FIG. For each frame period determined in synchronization with 5), the YSOBJ signals for one screen map output from the RGB color difference data processing circuit 806 in the VDP 506 are alternately held in the two YSOBJ signal buffers. Then, the processing for each frame shown in Step 1004 to Step 1015 in FIG. 10 includes the chroma key signal 523 for one frame held in one of the two chroma key signal buffers in the previous frame period, and the previous time. This is executed for the YSOBJ signals for one screen map held in one of the two YSOBJ signal buffers in the frame period. At the same time, the chroma key signal 523 and the YSOBJ signal corresponding to the current frame period are sequentially input to the other chroma key signal buffer and YSOBJ signal buffer, respectively. Each time the processing in step 1015 is completed, the chroma key signal buffer and the YSOBJ signal buffer to be processed are switched.
[0081]
In FIG. 10, the process screen map creation process in step 1005 and the collision coordinate detection process in step 1006 are executed for each block position (see FIG. 26).
Therefore, as in the case of the initial screen map creation process described with reference to FIG. 11, first, in step 1004, the register X indicating the horizontal column position and the register Y indicating the vertical row position of the generated processing screen map. After the value of the register Y is reset to 0, the value of the register X is cleared in step 1009 and the value of the register Y is incremented by +1, while the value of the register Y reaches 96 in step 1010 (over 95) Steps 1005 to 1008 → Step 1009 → Step 1010 → Step 1005 are repeatedly executed until it is determined. In this way, processing for 96 lines in the processing screen map is executed.
[0082]
In addition, while the value of the register X is incremented by +1 at step 1007, the process from step 1005 to step 1008 → step 1005 until it is determined at step 1008 that the value of the register X has reached 96 (exceeded 95). Is repeatedly executed. In this way, processing for 96 blocks in one line in the processing screen map is executed.
[0083]
Then, the processing screen map creation processing in step 1005 and the collision coordinate detection processing in step 1006 executed for each current value of the registers X and Y are processing for one block in the processing screen map.
[0084]
In step 1004, the values of the upper end coordinates and the left end coordinates stored in the RAM of FIG. 27 used in the processing screen map creation process in step 1005 described later are initialized to 97, and the lower end coordinates and the right end coordinates are similarly set. The values are initialized to 0, and the values of the centroid calculation processing data S_M, S_MxX, and S_MyY that are also used in the processing screen map creation processing are initialized to 0. These data are secured in a work area (not shown) in the same RAM as the RAM of FIG.
[0085]
In the following description, the current values of the registers X and Y are simply referred to as X and Y, and a block specified by the current values of X and Y is referred to as a processing target block.
Detailed operation flowcharts of the processing screen map creation processing in step 1005 are shown in FIGS. Here, the processing screen map stored in the RAM shown in FIG. 27, the upper end coordinates, the lower end coordinates, the left end coordinates indicating the existence range of the bat 101 stored in the RAM shown in FIG. A process for detecting the right end coordinate and a process for calculating data for obtaining the barycentric coordinate of the bat 101 are executed.
[0086]
First, in step 1201, each of the chroma key signals 523 at the column positions (6X + 1) to (6X + 6) in the horizontal direction and the row positions (2Y + 1) and (2Y + 2) in the vertical direction from the chroma key signal buffer to be processed. A logical value is read. As in the case of the initial screen map creation processing, for example, if X = 0 and Y = 1, the data at the column positions 1 to 6 in the horizontal direction and the row positions 3 and 4 in the vertical direction (block 97 in FIG. 26A). ) Is read out.
[0087]
In step 1202, it is determined whether or not the logical value “1” of the chroma key signal 523 for 12 pixels in the processing target block read in step 1201 is 6 pixels or more.
[0088]
If the determination in step 1202 is YES and a reject switch (not shown) is turned off in step 1203, “1” is set as a logical value corresponding to the processing target block in step 1206, and the determination in step 1202 is performed. If NO, “0” is set as a logical value in the register in step 1207.
[0089]
Here, the reject switch is operated by the user. If the position recognized as the portion related to the chroma key on the processing screen map is simultaneously recognized as the position where the background object having the same color as the bat 101 exists on the initial screen map, the position of the bat 101 If the user wishes to treat as nonexistent, the user turns on this switch.
[0090]
If the determination in step 1202 is YES, that is, if the processing target block is recognized as a portion related to the chroma key, and it is determined in step 1203 that a reject switch (not shown) is turned on, in step 1204, From the initial screen map area in the RAM (FIG. 27), the logical value at the same block position as the processing target block in the Xth column in the horizontal direction and the Yth row in the vertical direction is read.
[0091]
Subsequently, in step 1205, it is determined whether or not the logical value is “1”.
If the determination in step 1205 is YES, that is, the position of the processing target block recognized as the portion related to the chroma key on the processing screen map is simultaneously the position where the background object having the same color as the bat 101 exists on the initial screen map. If it is recognized, “0” is set as a logical value in the above-mentioned predetermined register in step 1207, so that the bat 101 does not exist at the block position.
[0092]
On the other hand, the determination in step 1205 is NO, that is, the processing target block position recognized as a portion related to the chroma key on the processing screen map is recognized as the position where the background target object having the same color as the bat 101 exists on the initial screen map. If not, in step 1206, “1” is set as the logical value in the predetermined register described above, so that the bat 101 is processed at the block position.
[0093]
Step 1219 of FIG. 13 described later is executed after the processing of step 1207, that is, for the processing target block recognized that the bat 101 does not exist on the processing screen map.
[0094]
On the other hand, after the processing of step 1206, that is, for the processing target block recognized as having the bat 101 on the processing screen map, steps 1208 to 1218 of FIG. 13 are further executed. Here, the process of updating the upper end coordinates, the lower end coordinates, the left end coordinates, and the right end coordinates stored in the RAM of FIG. 27 indicating the existence range of the bat 101 on the processing screen map, and the barycentric coordinates of the bat 101 are obtained. A process for calculating data for this is executed.
[0095]
That is, first, in step 1208, it is determined whether or not the value (Y + 1) is smaller than the value of the upper end coordinate stored in the RAM (FIG. 27). If this determination is YES, the upper end is determined in step 1209. The coordinate value is rewritten to the value (Y + 1).
[0096]
After step 1209 or when the determination in step 1208 is NO, it is determined in step 1210 whether the value (Y + 1) is larger than the value of the lower end coordinates stored in the RAM (FIG. 27). If this determination is YES, in step 1211, the value of the lower end coordinate is rewritten to a value (Y + 1).
[0097]
After step 1211 or when the determination in step 1210 is NO, it is determined in step 1212 whether the value (X + 1) is smaller than the value of the left end coordinate stored in the RAM (FIG. 27). If this determination is YES, in step 1213, the value of the left end coordinate is rewritten to the value (X + 1).
[0098]
After step 1213 or when the determination in step 1212 is NO, it is determined in step 1214 whether the value (X + 1) is larger than the value of the right end coordinate stored in the RAM (FIG. 27). If this determination is YES, in step 1215, the value of the right end coordinate is rewritten to the value (X + 1).
[0099]
As a result of the processing in steps 1208 to 1215 described above, in the block range having the logical value “1” on the processing screen map, for example, as shown in FIG. 28, the upper end coordinate, the lower end coordinate, the left end coordinate, The right end coordinates are obtained in the RAM of FIG. This value is simultaneously transferred to the DRAM 510 of FIG.
[0100]
After step 1215 or when the determination in step 1214 is NO, the value of data S_M is incremented by +1 in step 1216, and the value of data S_MxX is incremented by X in step 1217. Thus, the value of the data S_MyY is incremented by Y. The principle for obtaining the barycentric coordinates from these data will be described later.
[0101]
After the processing in step 1218 or after the processing in step 1207 (FIG. 12) described above, in step 1219, the Xth column in the horizontal direction and the Yth row in the vertical direction in the processing screen map area in the RAM shown in FIG. The content of the address corresponding to the target block to be processed is rewritten to the logical value set in the predetermined register in step 1206 or 1207 described above.
[0102]
The processing screen map creation process of step 1005 of FIG. 10 shown in the operation flowchart of FIG. 12 and FIG. 13 is executed as a part of the repetition process of step 1005 to step 1010, so that FIG. A processing screen map for one screen is created in the RAM shown in FIG. 27, and upper end coordinates, lower end coordinates, left end coordinates, and right end coordinates indicating the existence range of the bat 101 are obtained in the RAM shown in FIG. Data S_M, S_MxX, and S_MyY are obtained in an area (not shown) on the RAM shown in FIG.
[0103]
Next, FIG. 14 shows a detailed operation flowchart of the collision detection process in step 1006 of FIG.
First, in step 1401, the logical value of the processing target block in the Xth column in the horizontal direction and the Yth row in the vertical direction is read from the processing screen map area in the RAM (FIG. 27).
[0104]
In step 1402, it is determined whether or not the logical value is “1”, that is, whether or not the bat 101 exists at the position of the processing target block.
If the determination in step 1402 is NO, the collision detection process in FIG. 14 is terminated as it is, and the process returns to the operation flowchart in FIG. 10 to continue the process for the next block position.
[0105]
If the determination in step 1402 is YES, in step 1403, the YSOBJ signal corresponding to the processing target block in the Xth column in the horizontal direction and the Yth row in the vertical direction is extracted from the YSOBJ signal buffer to be processed. The logical value of (CG data) is read out.
[0106]
In step 1404, it is determined whether or not the logical value of the read YSOBJ signal is “1”, that is, whether or not the object screen data indicating the ball 105 (FIG. 1) exists in the processing target block. .
[0107]
If the determination in step 1404 is NO, the collision detection process of FIG. 14 is terminated as it is, and the process for the next block position is continued by returning to the operation flowchart of FIG.
[0108]
If the determination in step 1404 is YES, it is determined in step 1405 whether or not the value of the predetermined collision register is “0”, that is, whether or not a collision has not been detected so far and this time is the first collision detection. Is determined.
[0109]
If the determination in step 1405 is YES, in steps 1406 and 1407, the X and Y coordinates of the first collision coordinate held in the RAM of FIG. 27 are updated to the values in the registers X and Y, respectively. Subsequently, in step 1408, the value of the collision register is set to a value “1” indicating that a collision has occurred. Thereafter, the collision detection process of FIG. 14 is terminated, and the process returns to the operation flowchart of FIG. 10 to continue the process for the next block position.
[0110]
On the other hand, if the determination in step 1405 is NO, in steps 1409 and 1410, the X and Y coordinates of the last collision coordinate held in the RAM of FIG. 27 are updated to the values in the registers X and Y, respectively. The Thereafter, the collision detection process of FIG. 14 is terminated, and the process returns to the operation flowchart of FIG. 10 to continue the process for the next block position.
[0111]
The collision detection process of Step 1006 of FIG. 10 shown in the operation flowchart of FIG. 14 is executed over the entire range of the processing screen map based on the repetition of the processes of Step 1005 to Step 1010, so that the collision is detected. The block range on the processing screen map where the presence / absence and the collision occurred is detected.
[0112]
Next, the center-of-gravity calculation process in step 1011 in FIG. 10 is executed once per frame period, that is, every time the processing screen map for one screen is created and the determination in step 1010 is YES. In this process, the position of the center of gravity of the bat 101 on the process screen map stored in the RAM (FIG. 27) is calculated.
[0113]
In general, the x-coordinate XG and the y-coordinate YG of the center of gravity of a region existing on a two-dimensional coordinate can be calculated by the following equations (1) and (2).
[0114]
[Expression 1]
XG = Σ (m_xi・ X_i) / Σm_xi= Σ (m_xi・ X_i) / M
[0115]
[Expression 2]
YG = Σ (m_yj・ Y_j) / Σm_yj= Σ (m_yj・ Y_j) / M
Here, the suffixes i and j are positions in the centroid calculation area in the column direction (x direction) and the row direction (y direction), respectively._iAnd y_jAre the x-coordinate value and the y-coordinate value at the column direction position i and the row direction position j, respectively._xiAnd m_yjAre sums of masses at the column direction position i and the row direction position j, respectively. M is the sum of the masses of the regions existing on the two-dimensional coordinates.
[0116]
Here, when the x-coordinate value and the y-coordinate value are values having the above-described block as a unit, m_xiAnd m_yjAre equivalent to the total number of blocks in column direction block position i and row direction block position j, respectively. Then, in the processing screen map creation processing of FIG. 12 and FIG. 13 corresponding to step 1005 of FIG. 10 described above, the mass M, which is the denominator term in Equation 1 and Equation 2, is set as the area S_M in Step 1216 of FIG. And the term Σ (m_xi・ X_i) Is calculated as data S_MxX by repeating step 1217 of FIG. 13, and the term Σ (m_yj・ Y_j) Is calculated as data S_MyY by repeating step 1218 of FIG.
[0117]
FIG. 15 shows a more detailed operation flowchart of the center-of-gravity calculation process in step 1011 of FIG. 10 based on the above-described data S_M, S_MxX, S_MyY, and the calculation formulas of Formula 1 and Formula 2.
[0118]
That is, in step 1501, the center-of-gravity coordinates (XG, YG) of the bat 101 existing on the processing screen map are obtained by the following formulas 3 and 4 corresponding to the formulas 1 and 2.
[0119]
[Equation 3]
XG = S_MxX / S_M
[0120]
[Expression 4]
YG = S_MyY / S_M
For example, in the example shown in FIG. 28, the center-of-gravity coordinates (XG, YG) = (32.6, 16.1) are obtained by executing the calculation of the following equation.
[0121]

After the gravity center calculation process in step 1011 of FIG. 10 shown in the operation flowchart of FIG. 15 described above, it is determined in step 1012 of FIG. 10 whether or not the value of the collision register is “1”. The value of this register is the YSOBJ signal indicating the presence of the ball 105 even if there is one block whose logical value indicating the presence of the bat 101 on the processing screen map is “1” in the collision coordinate detection process of step 1006 described above. Is set to “1” (see step 1408 in FIG. 14).
[0122]
If the determination in step 1012 is YES, a collision flag that is an interrupt flag to the CPU 509 in FIG. 5 is set to “1” in step 1013. As a result, a collision flag interruption process (FIG. 20), which will be described later, occurs in the CPU 509. In this interruption process, the CG image 104 displayed on the display device 103 in response to the bat 101 of FIG. The process which controls is performed. The image recognition circuit 508 transfers the center-of-gravity coordinates (XG, YG) calculated in step 1011 to the DRAM 507 in FIG. 5 together with the above-described collision flag setting process. Further, the image recognition circuit 508 stores the first and last collision coordinates stored in the RAM (FIG. 26) in the collision coordinate detection process in step 1006 and the above-described RAM in the process screen map creation process in step 1005. The upper end coordinates, lower end coordinates, left end coordinates, and right end coordinates are transferred to the DRAM 507 as necessary.
[0123]
After the processing in step 1013, it is determined in step 1014 whether or not the collision flag has returned to “0”. The value of this collision flag is returned to “0” when a later-described collision flag interrupt process executed by the CPU 509 ends. As a result, for example, the image recognition circuit 508 in FIG. 5 cannot advance the processing of the operation flowchart in FIG. It should be noted that while the collision flag interrupt processing is being executed, the processing of step 1004 to step 1015 is configured to proceed in synchronization with the frame period so that another interrupt can be applied to the CPU 509 according to the value of the collision register. It may be.
[0124]
If the value of the collision flag returns to “0” and the determination in step 1014 becomes YES, the value of the collision register is also returned to “0” in step 1015.
After the process of step 1015 or when the determination of step 1012 is NO, the process returns to the process of step 1004 in synchronization with the start time of the next frame period. In this case, as described above, the chroma key signal buffer and the YSOBJ signal buffer to be processed are changed.
[0125]
Next, FIGS. 16 and 17 are operation flowcharts of CPU processing executed by the CPU 509 of FIG.
The CPU 509 basically processes the key scan signal 527 input from a start switch and a stop switch (not shown), and the CG image 104 (FIG. 1) from the application ROM 514 in the application cartridge 513 as time passes. The object screen data, the object table data, and the background screen data that constitute the data are sequentially read out, and these data are transferred from the bus 526 to the VDP 506 within the vertical blanking period synchronized with the synchronization signal 519. The display position on the CG image 104 (FIG. 1) of each object screen data transferred to the DRAM 507 is sequentially designated to the VDP 506.
[0126]
In FIG. 16, first, in step 1601, the contents of the DRAM 507 in FIG. 5 are initialized.
In step 1602, various interrupt processes to be described later are permitted.
[0127]
In Step 1603, an application program is loaded into the program area on the DRAM 510 in FIG. 5 from the application ROM 514 (either # 1 or # 2) in the application cartridge 513 in FIG. . Thereafter, the CPU 509 executes the operations after step 1604 in accordance with this application program.
[0128]
In step 1604, the read start address and the number of read data on the application ROM 514 in the application cartridge 513 of the background screen data constituting the CG image 104 (FIG. 1) are set in the DMAC (direct memory access controller) in the CPU 509. Is done.
[0129]
In step 1605, the read start address and the number of read data of the first object screen data constituting the CG image 104 (FIG. 1) on the application ROM 514 are set in the DMAC.
[0130]
In step 1606, the read start address and the number of read data of the object table data on the application ROM 514 are set in the DMAC. As described above, this object table data is a table indicating a correspondence table between each object screen data stored in the DRAM 507 and an ID number corresponding to each object screen data.
[0131]
After the processing of steps 1604 to 1606, “1” is set in the register ACCF in step 1607. In step 1608, the process waits until the value of the register ACCF returns to “0”.
[0132]
Here, the CPU 509 interrupts the processing of step 1608 every time the vertical blanking period within each frame period determined based on the synchronization signal 519 is started, and the vertical flow shown in the operation flowchart of FIG. Perform blanking interrupt processing.
[0133]
If “1” is set in the register ACCF by the above-described CPU processing, the determination in step 1801 becomes YES at the timing when this interrupt processing is executed, so that the above-described step 1604 to step in FIG. Each data stored in the application ROM 514 in the application cartridge 513 is transferred to the VDP 506 based on each data set in the DMAC in the CPU 509 by the processing of 1606.
[0134]
That is, since the processing from step 1604 to step 1606 in FIG. 16 is executed immediately after the power is turned on, the determination in step 1802 in FIG. 18 is YES.
As a result, first, in step 1803, the DMAC transfers the background screen data in the application ROM 514 via the bus 526 based on the read start address and the number of read data of the background screen data set by the processing in step 1604. DMA transfer to the VDP 506. The VDP 506 transfers the background screen data from the CPU interface circuit 801 in FIG. 8 to the DRAM 507 in FIG. 5 via the DRAM control circuit 802.
[0135]
Next, in step 1804, the DMAC transfers the object screen data in the application ROM 514 to the VDP 506 via the bus 526 based on the read start address and the number of read data of the object screen data set by the processing in step 1605. Forward. The VDP 506 also transfers this object screen data from the CPU interface circuit 801 in FIG. 8 to the DRAM 507 in FIG. 5 via the DRAM control circuit 802.
[0136]
In step 1805, the DMAC DMA-transfers the object table data in the application ROM 514 to the VDP 506 via the bus 526 based on the read start address and the number of read data of the object table data set by the processing in step 1606. . The VDP 506 transfers this object table data from the CPU interface circuit 801 to the object table RAM 804 via the object control circuit 803.
[0137]
Thereafter, in step 1806, the value of the register ACCF is returned to “0”, and the vertical blanking interrupt process shown in FIG. 18 is terminated.
As a result, the process of step 1608 in FIG. 16 is resumed, and when the determination in this process is YES, the user first presses a start switch (not shown) and the corresponding key scan signal 527 (FIG. 5) is input. Until the determination is made, the processing loop in which the determination in step 1609 is NO → the determination in step 1612 is NO → the determination in step 1609 is NO is repeated to enter a standby state. Note that the contents of the register STF are initialized to “0” in step 1601 of FIG.
[0138]
When the start switch is pressed and the corresponding key scan signal 527 is input, the determination in step 1609 becomes YES.
As a result, first, in step 1610, time 0 is set in the register t. The register t is a register indicating the elapsed time from the time when the user starts the game by pressing the start switch. The value of the register t is a clock circuit 512 connected to the bus 526 (FIG. 5). ) Is incremented by +1 by the timer interrupt process shown as step 1901 in FIG. 19 which is executed based on the interrupt request input at regular intervals.
[0139]
In step 1611, “1” is set in the register STF.
As a result, the determination in step 1612 is YES, and thereafter, normally, the processing loop of step 1613 to step 1617 → step 1619 → step 1609 (FIG. 16) → step 1612 → step 1613 in FIG. 17 is repeated.
[0140]
In the processing loop described above, in step 1614, an ID number designated for the object table data corresponding to the object screen data to be displayed is calculated based on the value of the current register t indicating the passage of time. Then, the display position of the object screen data is calculated based on the current value of the register t. These data are set at predetermined addresses on the DRAM 507, and the read start address and the number of read data are set in the DMAC in the CPU 509. In step 1614 or step 1615, sound generation instruction data to be transferred to the sound source circuit 515 in the application cartridge 513 is also generated in a sound source interrupt process (see FIG. 22) described later, and is set in the DRAM 507 or the like.
[0141]
If the object screen data needs to be changed, in step 1614, the DMAC in the CPU 509 stores the read start address and the number of read data on the application ROM 514 of the object table data corresponding to the new object screen data. Set. In step 1615, the reading start address and the number of read data of the new object screen data on the application ROM 514 are set in the DMAC.
[0142]
In step 1616, “1” is set in the register ACCF.
Thereafter, until the above-described vertical blanking interrupt process is executed and the value of the register ACCF is returned to “0”, the determination of step 1617 is normally NO → the determination of step 1619 is NO → step 1609 (FIG. 16). The process loop is repeated in which the determination is NO, the determination in step 1612 is YES, the determination in step 1613 (FIG. 17) is NO, the determination in step 1617 is NO, and so on.
[0143]
As described above, the CPU 509 interrupts the processing of the processing loop described above every time the vertical blanking period is started, and executes the vertical blanking interrupt processing shown in the operation flowchart of FIG.
[0144]
As a result, when the determination at step 1801 is YES → the determination at step 1802 is NO, step 1804 is executed.
In step 1804, the DMAC in the CPU 509 DMA-transfers data indicating the display position of the object screen data to the VDP 506 via the bus 526 based on the setting by the processing in step 1615. In step 1805, the DMAC DMA-transfers the ID number of the object screen data to be displayed to the VDP 506 via the bus 526 based on the setting by the processing in step 1614. As a result, the object control circuit 803 in the VDP 506 reads the object screen data corresponding to the ID number from the DRAM 507 via the DRAM control circuit 802 by referring to the object table RAM 804 by the ID number specified by the CPU 509. After being expanded in the line buffer RAM 808 based on the display position information designated by the CPU 509, the contents are output to the RGB color difference data processing circuit 806.
[0145]
The processing in steps 1804 and 1805 when new object screen data and object table data are transferred is the same as that immediately after the execution of steps 1604 to 1606 in FIG.
[0146]
Thereafter, in step 1806, the value of the register ACCF is returned to “0”, and the vertical blanking interrupt process shown in FIG. 18 is terminated.
As a result, the processing loop described above in the CPU process is resumed, and the determination in step 1613 is YES, whereby the next screen control process is executed.
[0147]
When the user presses a stop switch (not shown) and a corresponding key scan signal 527 (FIG. 5) is input in the process of executing the processing loop described above, the determination in step 1617 is YES. As a result, in step 1618, “0” is set in the register STF. Accordingly, from this point onward, the determination in step 1609 (FIG. 16) is NO → the determination in step 1612 is NO → step until the user presses the start switch (not shown) again and the corresponding key scan signal 527 is input. A processing loop in which the determination of 1609 is NO is repeated, and a standby state is entered. Since the value of the register t is not cleared, the next time the start switch is pressed, the progress of the game is resumed from the point when the stop switch is pressed.
[0148]
In addition, when the value of the register t reaches a preset end time T in the process of executing the processing loop described above, the determination in step 1619 is YES. As a result, in step 1620, the value of the register t is reset to 0. Accordingly, after this point, the game is repeated from the beginning.
[0149]
Further, when the user presses a start switch (not shown) again and inputs a corresponding key scan signal 527 in the course of executing the above processing loop, the determination in step 1609 becomes YES. As a result, the value of the register t is forcibly reset to 0 in step 1610. Therefore, the game is repeated again from the beginning after this point.
[0150]
On the other hand, when the collision flag is set to 1 as described above by the image recognition circuit 508 in the process of executing the above processing loop (see step 1013 in FIG. 10), the CPU 509 interrupts the processing of the above processing loop. Then, the collision flag interrupt process shown in the operation flowchart of FIG. 20 is executed.
[0151]
First, in step 2001, the current time t indicated by the register t is set to a predetermined collision determination period T set in advance.₁~ T₂It is determined whether or not it is within the range. The reason for providing such a collision determination period is that the chroma key corresponding to the bat 101 overlaps the object screen data corresponding to the ball 105 at a time other than the time when the bat 101 of FIG. This is to prevent erroneous processing from being executed in the event of failure.
[0152]
In step 2002, the coordinates of the center of gravity of the chroma key indicating the bat 101 transferred from the image recognition circuit 508 to the DRAM 510 are taken into the CPU 509.
[0153]
In step 2003, the positional relationship between the above-described center coordinates of the chroma key indicating the bat 101 and the center coordinates of the object screen data indicating the currently displayed ball 105 acquired from the object control circuit 803 in the VDP 506 is determined. .
[0154]
As a result, as shown in FIG. 2, when the shift between the two barycentric coordinates is within a predetermined allowable range, the CG image 104 on the display device 103 in FIG. 1 is changed to an image corresponding to a home run, for example. Accordingly, in step 2004, the first image processing is executed. It should be noted that the center-of-gravity coordinates of the chroma key indicating the bat 101 and the center-of-gravity coordinates of the object screen data indicating the ball 105 on the processing screen map match when the user swings the bat 101 down to a position corresponding to the strike zone. Further, it is assumed that the direction of the camera 102 in FIG. 1 has been adjusted in advance.
[0155]
Further, as shown in FIG. 3, when the center of gravity coordinates of the object screen data indicating the ball 105 are shifted below the center coordinates of the chroma key indicating the bat 101, the CG image 104 on the display device 103 of FIG. In step 2005, the second image processing is executed so as to change the image to an image corresponding to, for example, Goro.
[0156]
Further, as shown in FIG. 4, when the center of gravity coordinates of the object screen data indicating the ball 105 are shifted above the center coordinates of the chroma key indicating the bat 101, the CG image 104 on the display device 103 in FIG. In step 2006, the third image processing is executed so as to change, for example, to an image corresponding to a fly.
[0157]
FIG. 21 is an operation flowchart of the n-th image processing that generally represents the first to third image processing shown in steps 2004 to 2006.
In step 2101, the display position of the object screen data is calculated based on whether the current process is the first, second, or third image process and based on the value of the current register t indicating the passage of time. The In step 2102, the object corresponding to the object screen data to be displayed depends on whether the current processing is the first, second, or third image processing, and based on the value of the current register t. An ID number designated for the table data is calculated. In step 2101 or step 2102, sound generation instruction data transferred to the sound source circuit 515 in the application cartridge 513 is also generated in a sound source interrupt process (see FIG. 22) described later, and set in the DRAM 507 or the like.
[0158]
In step 2103, “1” is set in the register ACCF.
Thereafter, until the above-described vertical blanking interrupt processing is executed and the value of the register ACCF is returned to “0”, the determination of step 2104 is normally NO → the determination of step 2105 is NO → the determination of step 2106 is NO → The processing loop in which the determination in step 2104 is NO is repeated to enter a standby state.
[0159]
As described above, the CPU 509 interrupts the processing of the processing loop described above every time the vertical blanking period is started, and executes the vertical blanking interrupt processing shown in the operation flowchart of FIG.
[0160]
As a result, when the determination at step 1801 is YES → the determination at step 1802 is NO, step 1804 is executed.
In step 1804, the DMAC in the CPU 509 DMA-transfers data indicating the display position of the object screen data to the VDP 506 via the bus 526 based on the setting by the processing in step 2101. In step 1805, the DMAC DMA-transfers the ID number of the object screen data to be displayed to the VDP 506 via the bus 526 based on the setting by the processing in step 2102. As a result, the object control circuit 803 in the VDP 506 reads the object screen data corresponding to the ID number from the DRAM 507 via the DRAM control circuit 802 by referring to the object table RAM 804 by the ID number specified by the CPU 509. After being expanded in the line buffer RAM 808 based on the display position information designated by the CPU 509, the contents are output to the RGB color difference data processing circuit 806.
[0161]
Thereafter, in step 1806, the value of the register ACCF is returned to “0”, and the vertical blanking interrupt process shown in FIG. 18 is terminated.
As a result, the processing loop described above in the n-th image processing is resumed, and the determination in step 2106 is YES, whereby the screen control processing corresponding to the value of the register t next to the n-th image processing is executed.
[0162]
If the user presses a stop switch (not shown) and the corresponding key scan signal 527 (FIG. 5) is input during the execution of the processing loop described above, the determination in step 2105 is YES. As a result, in step 2108, “0” is set in the register STF, and the n-th image processing in FIG. 21 ends. As a result, after executing step 2007 in FIG. 20 to be described later, the collision flag interrupt processing is also terminated, and the processing returns to the CPU 509 processing loop. Accordingly, from this point onward, the determination in step 1609 (FIG. 16) is NO → the determination in step 1612 is NO → step until the user presses the start switch (not shown) again and the corresponding key scan signal 527 is input. A processing loop in which the determination of 1609 is NO is repeated, and a standby state is entered.
[0163]
Further, when the value of the register t reaches the preset end time END of the n-th image processing in the execution process of the n-th image processing in FIG. As a result, the value of the register t is reset to 0 in step 2107, the n-th image processing in FIG. 21 is terminated, and further, after executing step 2007 in FIG. 20 described later, the collision flag interrupt processing is also terminated. Return to the processing loop. Accordingly, after this point, the game is repeated from the beginning.
[0164]
In step 2107, the display of the CG image 104 on the display device 103 is shifted to the next scene by forcibly setting the value of the register t to a predetermined value without resetting it to 0. It is also possible to make it.
[0165]
In step 2007 in the collision flag interrupt process of FIG. 20 executed after the completion of the n-th image process of FIG. 21, the collision flag is returned to “0”. As described above, this process is a process for advancing the process before step 1014 in the operation flowchart shown in FIG. 10 executed by the image recognition circuit 508.
[0166]
Next, FIG. 22 is an operation flowchart of a sound source interrupt process executed by the CPU 509. This interrupt processing is executed when the tone generator circuit 515 in the application cartridge 513 in FIG. 5 sets a tone generator interrupt request flag as an interrupt flag to the CPU 509 in the process of executing tone generator processing (FIG. 25) described later. The
[0167]
First, in step 2201, the monitoring timer is reset. The monitoring timer is used to prevent the use of the imitation application cartridge 513.
[0168]
The value of the monitoring timer described above is the monitoring timer count process shown in the operation flowchart of FIG. 23 that is executed based on an interrupt request that is input at regular intervals from the clock circuit 512 (FIG. 5) connected to the bus 526. In step 2301 of FIG. In step 2302, it is determined whether or not the value of the monitoring timer has overflowed.
[0169]
Here, when the application cartridge 513 is manufactured by an authorized manufacturer, the application cartridge 513 performs a sound source interrupt request which is an interrupt flag to the CPU 509 in the process of executing sound source processing (FIG. 25) described later. The operations for setting and resetting the flag are sequentially executed. On the other hand, the CPU 509 executes step 2201 of the sound source interrupt process in FIG. 22 every time the sound source interrupt request flag is set, so that the monitoring timer is reset at an appropriate time interval.
[0170]
Therefore, when the application cartridge 513 is manufactured by a regular manufacturer, the determination in step 2302 of the monitoring timer count process in FIG. 23 is always NO.
[0171]
On the other hand, when the application cartridge 513 is imitated, there is almost no case where an operation for setting and resetting a sound source interrupt request flag that is an interrupt flag to the CPU 509 is not provided for cost reasons. The sound source circuit 515 does not generate a sound source interrupt request to the CPU 509 at a reasonably short time interval, and as a result, the monitoring timer is not reset or reset at a reasonably short time interval. When the monitoring timer overflows, the determination in step 2302 of the monitoring timer count process in FIG. 23 becomes YES.
[0172]
In this case, in step 2303, the overflow flag OVF is set to 1 as the interrupt flag to the CPU 509 again.
As a result, the CPU 509 executes the overflow interrupt process shown in the operation flowchart of FIG. 24 as the interrupt process again, and stops (halts) the operation of the CPU 509 itself in step 2401.
[0173]
The use of the imitation application cartridge 513 is blocked by the control using the above monitoring timer.
Next, in the sound source interrupt process of FIG. 22, in step 2202, the sound generation instruction data created in advance on the DRAM 507 is transferred to the sound source circuit 515 in the application cartridge 513 via the bus 526. The sound generation instruction data is generated when the CPU 509 executes the control processing of the CG image 104 (FIG. 1) such as step 1614 or step 1615 of FIG. 17 or step 2101 or step 2102 of FIG. Is stored in the DRAM 507.
[0174]
Similar control may be executed not only on the sound source circuit 515 in the application cartridge 513 but also on the sound source circuit 516 provided on the main body side.
[0175]
Each process shown in the operation flowchart of FIGS. 16 to 24 described above is a process executed by the CPU 509.
Finally, FIG. 25 is an operation flowchart of the sound source processing executed by the sound source circuit 515 in the application cartridge 513.
[0176]
When the application cartridge 513 is mounted on a holder (not shown) in particular, the execution of the operation flowchart of FIG. 25 is started. First, in step 2501, the contents of the internal register of the tone generator circuit 515 and the work RAM are initialized.
[0177]
Thereafter, the processes in steps 2502 to 2507 are repeatedly executed.
First, the tone generator circuit 515 generates a tone signal that is an interrupt flag to the CPU 509 of FIG. 5 in step 2502 every time a tone signal for one tone generation unit is generated (for example, every time reading of tone waveform data for several seconds is completed). Set the interrupt request flag. On the other hand, the CPU 509 outputs the sound generation instruction data for the next sound generation unit to the sound source circuit 515 by executing the sound source interrupt process of FIG.
[0178]
In step 2503, the tone generator circuit 515 is in a standby state until a change occurs in the input data. When a change occurs in the input data, in step 2504, a standby state is entered until the input data is stabilized.
[0179]
When the input data is stable, after the sound source interrupt request flag is reset to 0 in step 2505, the input sound generation instruction data is interpreted in step 2506.
[0180]
In step 2507, a musical tone signal corresponding to the data is generated.
When the generation of the tone signal for one sounding unit is completed, the tone generator circuit 515 executes step 2502 again, and sets a tone generator interrupt request flag that is an interrupt flag to the CPU 509 in FIG.
[0181]
The musical tone signal generated by the tone generator circuit 515 as described above is sent to the amplifier 517 via the CPU 509 and the tone generator circuit 516, and after being amplified there, it is output as an audio output signal 528 to a speaker (not shown) in particular. Is output.
<Specific operation of the second embodiment>
Next, the specific operation of the second embodiment of the present invention will be described.
[0182]
In the second embodiment, before the game is started, the user can check which block position on the processing screen map is recognized as the portion related to the chroma key. As described above, when the bat 101 overlaps with a background object having the same color as the bat 101, it is processed that the bat 101 does not exist (see step 1203 to step 1205 in FIG. 12). As a result, the detection accuracy of the position of the bat 101 is lowered, which may affect the progress of the game. Therefore, the function of the second embodiment allows the user to be careful not to have the bat 101 at a position where a background object having the same color as the bat 101 exists.
[0183]
The second embodiment is different from the first embodiment described above in the CPU processing executed by the CPU 509 in FIG. FIG. 29 to FIG. 32 show operation flowcharts of CPU processing in the second embodiment. This operation flowchart corresponds to the CPU processing shown in FIG. 16 in the first embodiment.
[0184]
First, in FIG. 29, the functions of steps 2901 to 2903 are the same as the functions of steps 1601 to 1603 in FIG.
Next, in step 2904, a screen clear command is executed for the VDP 506 in FIG. As a result, the display screen of the display device 103 in FIG. 1 is cleared. In step 2904, the values of the register X indicating the horizontal column position and the register Y indicating the vertical row position of the processing screen map are reset to zero.
[0185]
Subsequently, while the value of the register X is cleared in step 2911 and the value of the register Y is incremented by +1, until it is determined in step 2912 that the value of the register Y has reached 96 (exceeded 95), Steps 2905 to 2907 → step 2911 → step 2912 → step 2905 are repeatedly executed. In this way, processing for 96 lines in the processing screen map is executed.
[0186]
In addition, while the value of the register X is incremented by +1 in step 2910, the process from step 2905 to step 2909 → step 2910 until it is determined in step 2909 that the value of the register X has reached 96 (exceeded 95). Is repeatedly executed. In this way, processing for 96 blocks in one line in the processing screen map is executed.
[0187]
Steps 2905 to 2908 executed for each current value of the registers X and Y are processing for one block in the processing screen map. In the following description, the current values of the registers X and Y are simply referred to as X and Y, and a block specified by the current values of X and Y is referred to as a processing target block.
[0188]
In step 2905, the position of the processing target block at the column position X in the horizontal direction and the row position Y in the vertical direction from the processing screen map area of the RAM shown in FIG. 26 in the image recognition circuit 508 of FIG. 5 via the bus 526. Is read out and it is determined whether or not the logical value is “1”.
[0189]
Here, in the processing screen map creation process of step 1005 of FIG. 10 shown in the operation flowchart of FIG. 12 executed by the image recognition circuit 508 of FIG. 5, until the user presses a start switch not specifically shown, In particular, a reject switch (not shown) is disabled. As a result, in the processing screen map creation process executed in the image recognition circuit 508 before the start switch is pressed, the determination in step 1203 is always NO, so that it is recognized as a portion related to the chroma key based on the chroma key signal 523. The logical value of the block position is always “1”.
[0190]
Note that the reject switch is enabled, and in step 2905, both the initial screen map area and the processing screen map area of the RAM shown in FIG. 26 in the image recognition circuit 508 of FIG. The logical value of the position of the processing target block of the column position X in the horizontal direction and the row position Y in the vertical direction is read, and it is determined whether any one of the logical values is “1”. May be.
[0191]
Returning to the description of FIG. 29, if the determination in step 2905 is NO, the process proceeds to step 2909, and the process for the next processing target block is started.
On the other hand, if the determination in step 2905 is YES, the bat 101 or the background object having the same color as the bat 101 exists at the current position of the processing target block.
[0192]
In this case, in step 2906, the values of X and Y are respectively multiplied by 6 and 2, whereby the coordinates x and y on the original frame are calculated from the position of the processing target block, and the calculation result is The data is stored in the DRAM 507 in FIG. 5 (see the description regarding FIG. 25 described above).
[0193]
In step 2907, the DMAC in the CPU 509 is set with a read start address of data indicating the display position (x, y) stored in the DRAM 507 and an object table data transfer instruction indicating the number of read data.
[0194]
In step 2908, the read start address and the number of read data on the application ROM 514 in the application cartridge 513 for predetermined object screen data (for example, a rectangular object) for performing chroma key display are set.
[0195]
Thereafter, the process proceeds to step 2909 and the process for the position of the next processing target block is started.
As a result of the above operation being performed for one screen of the processing screen map having a size of 96 × 96 blocks, when the determination in step 2912 is YES, the DMAC in the CPU 509 includes the bat 101 or bat A data group for displaying predetermined object screen data is set at a position in the frame where a background object having the same color as 101 exists.
[0196]
Thereafter, in step 2913, “1” is set in the register ACCF. In step 2914, the process waits until the value of the register ACCF returns to “0”.
[0197]
As described above, the CPU 509 interrupts the processing of step 2914 at each time when the vertical blanking period within each frame period determined based on the synchronization signal 519 is started, and is shown in the operation flowchart of FIG. Execute vertical blanking interrupt processing. As a result, the data group for displaying predetermined object screen data at the position in the frame where the background object having the same color as the bat 101 or the bat 101 set in the DMAC as described above is displayed in the VDP 506. Transferred. The VDP 506 displays the CG image 104 (FIG. 1) in which the designated object screen data is arranged at the designated position in the frame.
[0198]
When the transfer is completed, the value of the register ACCF is returned to “0” (see step 1806 in FIG. 18), and the determination of step 2914 in FIG. 29 resumed to end the vertical blanking interrupt process shown in FIG. Becomes YES.
[0199]
Therefore, the determination in step 2915 (FIG. 30) is NO until the user presses a start switch (not shown) and the corresponding key scan signal 527 (FIG. 5) is input, and step 2904 to step in FIG. The processing loop of 2914 → step 2915... In FIG. 30 is repeated to enter the start standby state. During this time, as described above, predetermined object screen data is placed at a position in the frame where the bat 101 or a background object having the same color as the bat 101 exists according to the contents of the processing screen map that sequentially changes every frame period. Data groups to be displayed are sequentially transferred to the VDP 506, and the CG image 104 in which the specified object screen data is arranged at the specified position in the frame is sequentially displayed in the VDP 506.
[0200]
As a result of the above operation, when the camera 102 in FIG. 1 is capturing a scene as shown in FIG. 32A, for example, the CG image as shown in FIG. 104 is displayed, and the user confirms on the display device 103 an image obtained by combining the CG image 104 and the actual image captured by the camera 102 as shown in FIG. Can do.
[0201]
The CG image 104 and the actual image captured by the camera 102 are combined by the selection circuit 707 (FIG. 7) in the video signal conversion circuit 505 by the image recognition circuit 508 in FIG. This is realized by outputting a superimpose signal 525.
[0202]
In this case, the portion corresponding to the bat 101 is constantly moving in accordance with the movement of the user, but the portion of the background object having the same color as the bat 101 is stationary. In this way, the user can confirm the bat 101 and the background object having the same color as the bat 101.
[0203]
When the start switch is pressed and the corresponding key scan signal 527 is input, the determination in step 2915 of FIG. 30 is YES.
The subsequent functions of step 2916 to step 2920, step 2921 to step 2923 in FIG. 30 and step 2924 to step 2931 in FIG. 31 are the same as those in step 1604 to step 1608 and step 1610 to step 1610 in FIG. 1612 and the function of Step 1613 to Step 1620 of FIG.
<Specific operation of the third embodiment>
Next, the specific operation of the third embodiment of the present invention will be described.
[0204]
In the third embodiment, when the initial screen map described in the first embodiment is created, the bat 101 held by the user is stored in the initial screen map as a background object having the same color as the bat 101. Can be prevented.
[0205]
The third embodiment differs from the first embodiment described above in the initial screen map creation process in step 1003 in FIG. 10 executed by the image recognition circuit 508 in FIG. FIG. 33 shows a detailed operation flowchart of the initial screen map creation process in step 1003 of FIG. 10 in the third embodiment. This operation flowchart corresponds to the operation flowchart shown in FIG. 11 in the first embodiment. In FIG. 33 and FIG. 11, steps to which the same step numbers are added have the same functions.
[0206]
The operation flowchart of FIG. 33 in the third embodiment is different from the operation flowchart of FIG. 11 in the first embodiment only in that step 1109 in FIG. 11 is changed to step 1109 ′ in FIG.
[0207]
As described above, in the initial screen map creation process shown in the operation flowchart of FIG. 11 relating to the first embodiment, the chroma key signal 523 for a plurality of frames (for a plurality of screens) is obtained in order to stably acquire the initial screen map. The processing was repeatedly executed for (FIG. 5), and in step 1109, the logical sum (OR) of the logical values obtained by each repetition processing was calculated for each block position on the screen map. As a result, all the chroma keys present in the scene imaged in the plurality of frames are stored in the initial screen map as background objects having the same color as the bat 101.
[0208]
On the other hand, in the initial screen map creation process shown in the operation flowchart of FIG. 33 relating to the third embodiment, the process is repeatedly executed for chroma key signals 523 (FIG. 5) for a plurality of frames (for a plurality of screens). In 1109 ′, a logical product (AND) of logical values obtained by each iteration is calculated for each block position on the screen map.
[0209]
As a result, even if the bat 101 possessed by the user is imaged in the plurality of frame periods, the chroma key corresponding to the bat 101 always moves between the frames, and therefore, between the frames for each block position. The result of calculating the logical product is “0”. In this way, it is possible to prevent the bat 101 held by the user from being stored in the initial screen map as a background object having the same color as the bat 101.
<Specific operation of the fourth embodiment>
Finally, the specific operation of the fourth embodiment of the present invention will be described.
[0210]
In the fourth embodiment, when the game is started, the game can be started only when it is detected that the background object having the same color as the bat 101 is completely removed from the captured scene. To work. As described above, when the bat 101 overlaps with a background object having the same color as the bat 101, it cannot be distinguished whether it is the bat 101 or the background object. As a result, the detection accuracy of the position of the bat 101 is lowered, which may affect the progress of the game. Therefore, the function of the fourth embodiment allows the user to start the game after confirming that the background object having the same color as the bat 101 has completely disappeared from the scene being imaged.
[0211]
The fourth embodiment is different from the first embodiment described above in the processing of the overall operation flowchart executed by the image recognition circuit 508 in FIG. 5 and the CPU processing executed by the CPU 509.
[0212]
First, FIGS. 34 and 35 show flowcharts of the entire operation executed by the image recognition circuit 508 in the fourth embodiment. This operation flowchart corresponds to the operation flowchart shown in FIG. 10 in the first embodiment.
[0213]
34, first, the functions of Step 3401 and Step 3402 are the same as the functions of Step 1001 and Step 1002 of FIG.
Next, in step 3403, every time an initial screen map is created, “0” is set in the register IF, and this value is always notified to the CPU 509 via the bus 526 in FIG.
[0214]
Further, the initial screen map creation process in step 3405 is the same as the process in step 1003 in FIG. 10, and is shown by the operation flowchart in FIG.
[0215]
Next, in step 3406, the values of the register X indicating the horizontal column position and the register Y indicating the vertical row position of the initial screen map generated in step 3405 are reset to 0, and then the register is registered in step 3410. While the value of X is cleared and the value of register Y is incremented by +1, step 3411 to step 3409 are performed until it is determined in step 3411 that the value of register Y has reached 96 (exceeded 95). The processing of 3410 → step 3411 → step 3407 is repeatedly executed. In this way, the process for 96 lines in the initial screen map is executed.
[0216]
Further, while the value of the register X is incremented by +1 at step 3408, until it is determined at step 3409 that the value of the register X has reached 96 (exceeded 95), step 3407 to step 3409 → step 3407,. The process of .. is repeatedly executed. In this way, processing for 96 blocks in one line in the initial screen map is executed.
[0217]
In step 3407, for each block position specified by the current values of the registers X and Y in the initial screen map, the logical value of the block position is read from the initial screen map area in the RAM of FIG. It is determined whether or not the value is “1”.
[0218]
If the determination in step 3407 is NO, the process proceeds to step 3408 to move to the process for the next block position.
If the determination in step 3407 is YES, the process returns to the initial screen map creation process in step 3405 again.
[0219]
As described above, in the fourth embodiment, even if there is one block in the generated initial screen map, the logical value is “1”, that is, there is a background object having the same color as the bat 101 even in one block. If it is determined that the background object having the same color as the bat 101 is not removed from the scene being imaged, or the background object having the same color as the bat 101 does not exist. Unless the screen is moved to the range, the initial screen map creation process is repeated endlessly.
[0220]
When the background object having the same color as the bat 101 does not exist in the imaged scene, the logical value of all blocks in the initial screen map becomes “0”, so the determination in step 3407 is NO for all blocks. Eventually, the determination at step 3411 becomes YES.
[0221]
As a result, in step 3412 of FIG. 35, the value of the register IF is reset to “0”, and this value is notified to the CPU 509 via the bus 526 of FIG.
The subsequent processing in steps 3413 to 3424 is the same as the processing in steps 1004 to 1015 in FIG. 10 in the first embodiment.
[0222]
In the fourth embodiment, the logical values of all blocks in the initial screen map finally obtained are “0”. Therefore, in the above-described operation flowchart of FIG. 12 corresponding to the processing screen map creation processing in step 3414 of FIG. 35, the logical value of each block position of the processing screen map is set to the logical value of the corresponding block position of the initial screen map. There is no need to compare. Therefore, when the determination in step 1202 is YES, it is not necessary to execute the processing in steps 1203 to 1205, and the operation is performed so that step 1206 is immediately executed.
[0223]
In contrast to the above-described operation of the image recognition circuit 508 in FIG. 5, the CPU 509 in FIG. 5 executes the processing of the operation flowcharts shown in FIGS. 36 to 38. The function of this operation flowchart is basically the same as the function of the operation flowchart of FIGS. 29 to 31 relating to the second embodiment described above.
[0224]
That is, in FIG. 36 to FIG. 38 and FIG. 29 to FIG. 31, steps to which the same step number is added have the same function. Specifically, in the fourth embodiment, as in the case of the second embodiment, before the game is started, which block position on the processing screen map is recognized as a portion related to the chroma key by the user. Can be confirmed. However, in the fourth embodiment, as described in the operation flowcharts of FIGS. 34 and 35, the logical values of all blocks on the initial screen map created by the image recognition circuit 508 of FIG. 5 are “0”. The process screen map creation process in step 3414 in FIG. 35 is not executed until later. For this reason, what the user confirms on the processing screen map before starting the game is not the background object but the block position where the bat 101 held by the user is present.
[0225]
The operation flowchart of FIGS. 36 to 38 in the fourth embodiment is different from the operation flowchart of FIGS. 29 to 31 in the second embodiment in that step 2915 in FIG. 30 is changed to step 2915 ′ in FIG. It is only a point.
[0226]
That is, in the second embodiment, if the user presses a start switch (not shown), the determination in step 2915 is YES, and the game is started. On the other hand, in the fourth embodiment, the game does not start until the value of the register IF notified to the CPU 509 from the image recognition circuit 508 of FIG. 5 via the bus 526 becomes “0”.
[0227]
The value of the register IF is set to “1” in step 3403 in FIG. 34 while the initial screen map creation process is repeated without removing the background object having the same color as the bat 101 from the imaged scene. It continues to be set. Then, when the background object having the same color as the bat 101 is removed from the imaged scene and the logical values of all the blocks on the initial screen map become “0”, “0” in step 3412 of FIG. To "".
[0228]
Therefore, in the fourth embodiment, the game can be started only when the background object having the same color as the bat 101 is removed from the imaged scene.
<Other embodiments>
In the embodiment described above, the CPU 509 displays the position of the center of gravity of the bat 101 on the processing screen map (see FIG. 26) transferred from the image recognition circuit 508 to the DRAM 510 and the current display acquired from the object control circuit 803 in the VDP 506. The CG image 104 displayed on the display device 103 of FIG. 1 is controlled based only on the positional relationship with the position of the center of gravity of the object indicating the ball 105 (FIG. 1). However, the present invention is not limited to this, and the image recognition circuit 508 stores the first and last collision coordinates stored in the RAM (FIG. 26) by the collision coordinate detection processing in step 1006 of FIG. The upper end coordinate, the lower end coordinate, the left end coordinate, and the right end coordinate indicating the existence range of the bat 101 stored in the above-described RAM by the processing screen map creation processing in step 1005 are configured to be transferred to the DRAM 507, and these various position data 1 may be configured to control the CG image 104 displayed on the display device 103 of FIG.
[0231]
【The invention's effect】
According to the present invention, after the predetermined color that is the same as the image area of the predetermined color is no longer detected in the first state, the first state is canceled and the second state is started, Since the image area is extracted,After the subject to be imaged has shifted to a stable state,That is, after the background object having the same predetermined color as the object such as the bat disappears from the imaging range,Show only objects such as batsPredetermined colorCan be extracted stably.
[0232]
Also,The bat etc. extracted from the screen obtained by imaging the real scene such as the user shaking the batPredetermined colorThe positional relationship between the position of the image area and the position of the object screen data on the display screen is determined, and a signal corresponding to the screen to be displayed next is generated based on the determination content. In such a case, the user can participate in a video game displayed by a CG image or the like by performing an operation of swinging the bat so as to actually play. . In this case, in particular, when a specific image area such as a bat is extracted from the imaged screen, it is possible to accurately grasp the user's action by extracting the image area as a predetermined color using a chroma key or the like.
[0233]
According to the present invention, it is possible to realize not only a video game that performs image control in response to a user's action, but also a device that performs various image controls based on the results of imaging various real scenes. It becomes.
[Brief description of the drawings]
FIG. 1 is an external view (first to fourth embodiments) of an operation of an embodiment.
FIG. 2 is an operation principle diagram of an embodiment (part 1) (first to fourth embodiments);
FIG. 3 is an operation principle diagram of the embodiment (part 2) (first to fourth embodiments);
FIG. 4 is an operation principle diagram of the embodiment (part 3) (first to fourth embodiments);
FIG. 5 is an overall configuration diagram (first to fourth embodiments) of an embodiment of the present invention.
FIG. 6 is a configuration diagram (first to fourth embodiments) of a video signal input / output circuit;
FIG. 7 is a configuration diagram (first to fourth embodiments) of a video signal conversion circuit;
FIG. 8 is a configuration diagram of VDP (first to fourth embodiments).
FIG. 9 is an explanatory diagram (first to fourth embodiments) of a clock circuit;
FIG. 10 is an overall operation flowchart of the image recognition circuit (first, second, and third embodiments).
FIG. 11 is an operation flowchart (first, second, and fourth embodiments) of initial screen map creation processing;
FIG. 12 is an operation flowchart (part 1) of processing screen map creation processing (first to fourth embodiments);
FIG. 13 is an operation flowchart (part 2) of the processing screen map creation process (first to fourth embodiments);
FIG. 14 is an operation flowchart (first to fourth embodiments) of collision detection processing;
FIG. 15 is an operation flowchart (first to fourth embodiments) of a center of gravity calculation process;
FIG. 16 is an operation flowchart (No. 1) of CPU processing (first and third embodiments);
FIG. 17 is an operation flowchart (No. 2) of CPU processing (first and third embodiments);
FIG. 18 is an operation flowchart (first to fourth embodiments) of vertical blanking interrupt processing;
FIG. 19 is an operation flowchart (first to fourth embodiments) of timer interrupt processing;
FIG. 20 is an operational flowchart (first to fourth embodiments) of a collision flag interrupt process.
FIG. 21 is an operation flowchart (first to fourth embodiments) of n-th image processing.
FIG. 22 is an operation flowchart (first to fourth embodiments) of a sound source interrupt process;
FIG. 23 is an operation flowchart (first to fourth embodiments) of a monitoring timer count process.
FIG. 24 is an operation flowchart (first to fourth embodiments) of overflow interrupt processing;
FIG. 25 is an operation flowchart of sound source processing (first to fourth embodiments);
FIG. 26 is a relationship diagram between a pixel configuration of a frame and a screen map (first to fourth embodiments).
FIG. 27 is a diagram (first to fourth embodiments) showing a RAM map of an image recognition circuit;
FIG. 28 is an explanatory diagram (first to fourth embodiments) of detection processing of upper / lower / left / right end coordinates;
FIG. 29 is an operation flowchart (No. 1) of CPU processing (second embodiment);
FIG. 30 is an operation flowchart of the CPU processing (part 2) (second embodiment);
FIG. 31 is an operation flowchart (No. 3) of CPU processing (second embodiment);
FIG. 32 is an explanatory diagram of the second embodiment.
FIG. 33 is an operational flowchart (third embodiment) of the initial screen map creation process;
FIG. 34 is an overall operation flowchart (No. 1) of the image recognition circuit (fourth embodiment);
FIG. 35 is an overall operation flowchart (part 2) of the image recognition circuit (fourth embodiment);
FIG. 36 is an operation flowchart (No. 1) of CPU processing (fourth embodiment);
FIG. 37 is an operational flowchart (part 2) of the CPU processing (fourth embodiment).
FIG. 38 is an operation flowchart (part 3) of the CPU processing (fourth embodiment);
[Explanation of symbols]
101 Bat
102 camera
103 Display device
104 CG image
105 balls
501 CCD
502 Clock driver
503 Video signal input / output circuit
504 Oscillator (OSC)
505 Video signal conversion circuit
506 VDP
507 DRAM
508 Image recognition circuit
509 CPU
510 DRAM
511 OS ROM
512 clock circuit
513 Application cartridge
514 Application ROM
515, 516 tone generator circuit
517 amplifier
518 CCD output signal
519 Sync signal
520 Digital composite video input signal
521 Digital composite video output signal
522 Analog composite video output signal
523 Chroma key signal
524 Digital RGB signal
525 Superimpose signal
526 bus
527 Key scan signal
528 Audio output signal 528

Claims (5)

In an image extraction apparatus comprising: setting means for setting an image area of a predetermined color to be extracted in advance; imaging means; and extraction means for extracting an image area of a predetermined color included in the imaging area by the imaging means .
While the predetermined color that is the same as the predetermined color of the image area set by the setting unit is continuously detected from the imaging area captured by the imaging unit, the extraction operation by the extracting unit is prohibited and the same predetermined color is detected. An image extraction apparatus comprising control means for starting the extraction operation by the extraction means when it is not performed. Image storage means for storing object screen data to be output;
Display position designating means for designating a position to be output on the display screen of the object screen data stored in the image storage means;
Output control means for controlling to output an image corresponding to the object screen data stored in the image storage means at a position designated by the display position designation means;
Determining means for determining a positional relationship between a position on the imaged screen of an image area of a predetermined color extracted by the extracting operation by the extracting means started by the control means and a position at which the object screen data is displayed; ,
Signal generating means for generating a signal corresponding to a screen to be displayed next based on the content of the determination by the determining means;
The image extracting apparatus according to claim 1, further comprising: The apparatus further comprises signal output means for outputting a chroma key signal based on the color signal of the imaged screen, and the extraction operation by the extraction means started by the control means recognizes the chroma key signal output by the signal output means. The image extracting apparatus according to claim 1, wherein the image area is an image area of the predetermined color. An image extraction method for setting an image area of a predetermined color to be extracted in advance and extracting an image area of a predetermined color included in an area captured by an imaging unit ,
During the same predetermined color and a predetermined color having a preset image area from the image area to be imaged by the imaging unit is sensed continuously, while prohibiting the extraction operation, when the same predetermined color is no longer detected, extracting operation The image extraction method characterized by starting . The method further includes a signal output step of outputting a chroma key signal based on the color signal of the imaged screen, and the extraction operation to be started is performed by recognizing the chroma key signal output in the signal output step. 5. The image extracting method according to claim 4, wherein the image extracting method is a color image region extraction.

Images