JP4020982B2 - Moving image processing device - Google Patents

Description

ここで、この距離算出の際には整合窓の制約条件として下記式（２）を課すものとする。
【００２７】
｜ｉ−ｊ｜≦ｒ（ｒは予め設定されていた正の整数） ・・・（２）
距離Ｄが予め設定されていた閾値以下となったとき、これらの動領域を同一の移動物体のものとする。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００２８】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として輝度の度数分布を算出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００２９】
（実施例６）
以下、本発明の第６の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００３０】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの輝度情報を処理し、動領域内の輝度をＭ個（Ｍは正の整数）に分割する。Ｍ＝２の場合、閾値として平均値を用いることができる。その他にクラス間分散が最大となる閾値を用いる方法が考えられ、この場合は、Ｍが２以上であっても分割数が決定できれば用いることが可能である。さらに、度数分布を用いて分割数を決定し、分割する方法も考えられる。以下、この処理を図４を用いて説明する。
【００３１】
図４は度数分布である。度数分布は、２５６階調のデータの場合、８階調、１６階調もしくは３２階調毎に度数を計算し分布を求める。このような度数分布となった場合、極大極小点検出により度数分布の分割を行う。ここでは度数分布に対して微分処理を行う。ここで微係数が正から負に変化した数を領域分割数と考えることができる。図４では点Ａ、点Ｂ、点Ｃの３極大点が存在するため、分割数は３である。更に微係数が負から正に変化した時の階調値が領域分割閾値と考えることができる。図４では点Ｄ及び点Ｅが相当する。但し輝度分布の主成分を検出するために、微分処理の際には必要であれば平滑化処理を行う。こうして領域分割数Ｍ及び分割閾値が決定したならばこれら閾値を用いて領域分割を行い、Ｍ個の分割領域におけるそれぞれの輝度の代表値を求める。代表値としては、各分割領域毎の輝度の平均値でも良いし、また各分割領域における最頻度となる輝度値でも良い。これらを特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームと前フレームの領域分割数Ｍの比較、各代表値の差の評価、時系列処理部７において算出された分割数及び代表値との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００３２】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として動領域を輝度分布を基にＭ個に分割し、各分割領域の代表値となる輝度を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００３３】
（実施例７）
以下、本発明の第７の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００３４】
図１において、入力部１から動領域整形部４までの処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は実施例６と同様に、輝度の度数分布を算出して輝度分割閾値を検出し、この閾値を基に度数分布をＭ個（Ｍは正の整数）に分割する。そして各分割領域の大きさの比率を検出する。領域の大きさとしては、各分割領域に属する全画素数でも良いし、各分割領域における最頻度の度数を用いても良い。これらを特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームと前フレームにおけるこの特徴量の差の評価、時系列処理部７において算出された評価値との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００３５】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として動領域を輝度分布を基にＭ個に分割し、その分割領域の大きさの比率を検出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００３６】
（実施例８）
以下、本発明の第８の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００３７】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例６で示したように動領域内画像データを輝度情報を基にＭ個（Ｍは正の整数）に分割する。更に分割された各分割領域の位置情報を検出する。以下、この処理を図５を用いて説明する。
【００３８】
図５において、５１〜５３は分割領域であり、この中から領域５１の位置情報を検出する。まず動領域の中から分割領域５１に属する画像データを検出する。このとき各画素の輝度情報は検出する必要なく、各画素の位置情報のみ検出すれば良い。次にこの検出した画像データを走査線に対して垂直及び水平方向に加算処理して分割領域の上限座標ｙ_U 、下限座標ｙ_L 、左限座標ｘ_L 、右限座標ｘ_R を検出する。これらの座標決定の際には、ノイズを除去するために閾値以上の加算値が得られた点を求める座標として判定する。これらを位置情報としても良いが、上限と下限の中点ｙ_C 及び左限と右限の中点ｘ_C の座標Ｐを計算し、それをこの分割領域の位置情報とする。図５においては１個の領域として検出されたが、２個以上の領域に分割された場合はその全体の中心の座標を位置情報としても良いし、予め設定してある閾値以上の大きさとなる領域についてそれぞれの中心座標を位置情報として検出しても良い。この処理を全ての分割領域について行う。これらを特徴量として時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は，これら送られてきた特徴量の評価を行う。特徴量評価は、実施例６の評価に加えて、各分割領域の位置情報から対応する現フレームと前フレームの分割領域間の距離を算出し、その総和値を全分割領域の差として評価する。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００３９】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として動領域を輝度分布を基にＭ個に分割し、各分割領域の位置を検出してそれら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００４０】
（実施例９）
以下、本発明の第９の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００４１】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの色情報として彩度を処理し、動領域内の彩度の平均値を算出する。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレーム平均値と前フレーム平均値の差の評価、時系列処理部７において算出された平均値との差、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００４２】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として彩度の平均値を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００４３】
（実施例１０）
以下、本発明の第１０の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００４４】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの色情報として彩度を処理し、動領域内の彩度の度数分布を算出する。度数分布は、彩度の値を基に８分割、１６分割もしくは３２分割し、その分割毎に度数を計算して分布を求める。求められた度数分布には、動領域全画素数もしくは最頻度度数を基に正規化処理を行う。これを移動物体の特徴量として移動物体識別部８に送出する。移動物体識別部８は、前フレームの彩度度数分布を保持しており、これと現フレームの度数分布との距離を算出する。距離算出方法としては、市街地距離やユークリッド距離を用いても良いが、ここではＤＰマッチング法を用いる。現フレームの度数分布列をｃ_i 、前フレームの度数分布列をｐ_j （ｉ，ｊ＝１〜Ｎ、Ｎは正の整数）としたとき，距離Ｄは前記式（１）により算出される。ここでこの距離算出の際には、整合窓の制約条件として前記式（２）を課すものとする。距離Ｄが予め設定されていた閾値以下となったとき、これらの動領域を同一の移動物体のものとする。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００４５】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として彩度の度数分布を算出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００４６】
（実施例１１）
以下、本発明の第１１の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００４７】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの色情報として彩度を処理し、彩度が予め与えられた閾値未満になる画素数を計測し、その動領域全体に占める割合を低彩度率として算出する。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレーム低彩度率と前フレーム低彩度率の差の評価、時系列処理部７において算出された低彩度率との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００４８】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として低彩度の画素の全体に占める割合を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００４９】
（実施例１２）
以下、本発明の第１２の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００５０】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの色情報として彩度と色相を処理し、彩度が予め与えられた閾値以上になる画素の色相値の度数分布を算出する。度数分布を求める色相幅は、１０度、２０度もしくは３０度毎とする。求められた度数分布には、動領域全画素数もしくは最頻度度数を基に正規化処理を行う。これを移動物体の特徴量として移動物体識別部８に送出する。移動物体識別部８は、前フレームの色相度数分布を保持しており、これと現フレームの度数分布との距離を算出する。距離算出方法としては、市街地距離やユークリッド距離を用いても良いが、ここではＤＰマッチング法を用いる。現フレームの度数分布列をｃ_i 、前フレームの度数分布列をｐ_j （ｉ，ｊ＝１〜Ｎ、Ｎは正の整数）としたとき、距離Ｄは前記式（１）により算出される。ここでこの距離算出の際には、整合窓の制約条件として前記式（２）を課すものとする。距離Ｄが予め設定されていた閾値以下となったとき、これらの動領域を同一の移動物体のものとする。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００５１】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として色相の度数分布を算出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００５２】
（実施例１３）
以下、本発明の第１３の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００５３】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの色情報として彩度と色相を処理し、彩度が予め与えられた閾値以上になる画素の色相値の度数分布を算出する。度数分布を求める色相幅は、１０度、２０度もしくは３０度毎とする。そしてその度数分布を、実施例６と同様に、Ｍ個（Ｍは正の整数）に領域分割し、各分割領域毎の色相値の代表値を求める。代表値としては各分割領域毎の平均値でも良いし、また各分割領域における最頻度となる色相値でも良い。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームと前フレームの領域分割数Ｍの比較、各代表値の差の評価、時系列処理部７において算出された分割数及び代表値との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００５４】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として動領域を色相値の度数分布を基にＭ個に分割し、各分割領域の代表値となる色相値を検出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００５５】
（実施例１４）
以下、本発明の第１４の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００５６】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例１３と同様に、色相値の度数分布を算出して色相分割閾値を検出し、この閾値を基に度数分布をＭ個（Ｍは正の整数）に分割する。そして各分割領域の大きさの比率を検出する。領域の大きさとしては、各分割領域に属する全画素数でも良いし、各分割領域における最頻度の度数を用いても良い。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームと前フレームにおけるこの特徴量の差の評価、時系列処理部７において算出された評価値との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００５７】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域を色相値の度数分布を基にＭ個に分割し、その分割領域の大きさの比率を検出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００５８】
（実施例１５）
以下、本発明の第１５の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００５９】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例１３と同様に、色相値の度数分布を算出して色相分割閾値を検出し、この閾値を基に度数分布をＭ個（Ｍは正の整数）に分割する。そして各分割領域に属する画像データを検出する。検出した画像データから位置情報を検出する方法は、実施例８と同様に行う。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、実施例１３の評価に加えて、各分割領域の位置情報より対応する現フレームと前フレームの分割領域間の距離を算出し、その総和値を全分割領域の差として評価する。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００６０】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域を色相値の度数分布を基にＭ個に分割し、各分割領域の位置を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００６１】
（実施例１６）
以下、本発明の第１６の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００６２】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データから輝度及び色情報を検出する。情報はＲＧＢ座標系或いはＹＵＶ座標系を用いて検出する方法が考えられるが、ここでは輝度、彩度及び色相という３つの物理量によって検出する。そしてこれら３つの物理量を成分とする３次元のベクトル空間を定義し、この空間における動領域内画像データの度数分布を求める。度数分布を求める際には、各成分を８分割、１６分割もしくは３２分割し、その分割されたベクトル空間毎に度数を計算する。こうして求められた３次元ベクトル空間における度数分布は、動領域全画素数もしくは最頻度度数を基に正規化処理を行う。これを移動物体の特徴量として、移動物体識別部８に送出する。移動物体識別部８は、前フレームの３次元ベクトル空間における度数分布を保持しており、これと現フレームの度数分布との距離を算出する。距離算出方法としては、ＤＰマッチング法を用いる方法も考えられるが、データ量が多いことを考慮し、簡略化した手法として市街地距離を用いる。もちろんこれ以外にユークリッド距離を用いて距離を算出する方法も考えられる。現フレームの度数分布をｃ_i 、前フレームの度数分布をｐ_i （ｉ＝１〜Ｎ、Ｎは正の整数）としたとき、距離Ｄは下記式（３）により算出される。
【００６３】
【数２】

距離Ｄが予め設定されていた閾値以下となったとき、これらの動領域を同一の移動物体のものする。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００６４】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、輝度、彩度及び色相の３つの物理量を成分とする３次元ベクトル空間の度数分布を算出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００６５】
（実施例１７）
以下、本発明の第１７の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００６６】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの形状情報として動領域を包含する外接長方形を作成し、この縦横の辺の長さを検出する。更に縦横比率を算出し、これらを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームの辺の長さと前フレームの辺の長さの差及び縦横比率の差の評価、時系列処理部７において算出された辺の長さ及び縦横比率との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００６７】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域外接長方形の縦横辺の長さ及びその比率を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００６８】
（実施例１８）
以下、本発明の第１８の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００６９】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、その動領域を形成する画素数を計測する。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームの動領域画素数と前フレームの動領域画素数の差の評価、時系列処理部７において算出された動領域画素数との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００７０】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域を形成する画素数を検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００７１】
（実施例１９）
以下、本発明の第１９の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００７２】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、その動領域を形成する画素数を計測し、その動領域の外接長方形を作成する。そして動領域の画素数の外接長方形の画素数に占める割合を算出する。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームの特徴量と前フレームの特徴量の差の評価、時系列処理部７において算出された時系列特徴量との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００７３】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域を形成する画素数の外接長方形の画素数に占める割合を検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００７４】
（実施例２０）
以下、本発明の第２０の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００７５】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、その動領域内の画像データの輝度に対する空間微分処理を行い、輝度変化値を検出する。そして全輝度変化値の平均値を求め、画像データのテクスチャ情報とする。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームのテクスチャ情報と前フレームのテクスチャ情報の差の評価、時系列処理部７において算出されたテクスチャ情報との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００７６】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域内輝度変化値の平均値をテクスチャ情報として検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００７７】
（実施例２１）
以下、本発明の第２１の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００７８】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例２０と同様に、その動領域内の輝度変化値をテクスチャ情報として検出する。そしてその度数分布をテクスチャ度数分布として算出する。求められた度数分布には、動領域全画素数もしくは最頻度度数を基に正規化処理を行う。これを移動物体の特徴量として、移動物体識別部８に送出する。移動物体識別部８は、前フレームのテクスチャ度数分布を保持しており、これと現フレームの度数分布との距離を算出する。距離算出方法としては、市街地距離やユークリッド距離を用いても良いが、ここではＤＰマッチング法を用いる。現フレームの度数分布列をｃ_i 、前フレームの度数分布列をｐ_j （ｉ，ｊ＝１〜Ｎ、Ｎは正の整数）としたとき、距離Ｄは前記式（１）により算出される。ここでこの距離算出の際には、整合窓の制約条件として前記式（２）を課すものとする。距離Ｄが予め設定されていた閾値以下となったとき、これらの動領域を同一の移動物体のものとする。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００７９】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域内輝度変化値の度数分布をテクスチャ度数分布として検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００８０】
（実施例２２）
以下、本発明の第２２の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００８１】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例２０と同様に、その動領域内の輝度変化値をテクスチャ情報として検出し、その度数分布をテクスチャ度数分布として算出する。そしてその度数分布を、実施例６と同様に、Ｍ個（Ｍは正の整数）に領域分割し、各分割領域毎の輝度変化値の代表値を求める。代表値としては、各分割領域毎の輝度変化値の平均値でも良いし、また各分割領域における最頻度となる輝度変化値でも良い。これらを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームと前フレームの領域分割数Ｍの比較、各代表値の差の評価、時系列処理部７において算出された分割数及び代表値との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００８２】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域をテクスチャ度数分布を基にＭ個に分割し各分割領域のテクスチャの代表値を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００８３】
（実施例２３）
以下、本発明の第２３の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００８４】
図１において、入力部１から動領域整形部４までの処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は実施例２２と同様に、輝度変化値をテクスチャ情報としてその度数分布を算出して分割閾値を検出し、この閾値を基に度数分布をＭ個（Ｍは正の整数）に分割する。そして各分割領域の大きさの比率を検出する。領域の大きさとしては、各分割領域に属する全画素数でも良いし、各分割領域における最頻度の度数を用いても良い。これらを特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームと前フレームにおけるこの特徴量の差の評価、時系列処理部７において算出された評価値との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００８５】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域をテクスチャ度数分布を基にＭ個に分割しその分割領域の大きさの比率を検出し、これを特徴量として用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００８６】
（実施例２４）
以下、本発明の第２４の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００８７】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例２２と同様に、輝度変化値をテクスチャ情報としてその度数分布を算出して分割閾値を検出し、この閾値を基に度数分布を分割し、Ｍ個（Ｍは正の整数）の分割領域を検出する。そして各分割領域に属する画像データを検出する。検出した画像データから位置情報を検出する方法は、実施例８と同様に行う。これを移動物体の特徴量として時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、実施例２２の評価に加えて、各分割領域の位置情報より対応する現フレームと前フレームの分割領域間の距離を算出し、その総和値を全分割領域の差として評価する。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００８８】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として、動領域をテクスチャ度数分布を基にＭ個に分割し、各分割領域の位置を検出し、これら特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００８９】
（実施例２５）
以下、本発明の第２５の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは特徴量検出部５以降における動作である。以下、本実施例の動作について説明する。
【００９０】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、動領域内画像データの輪郭線を検出する。輪郭線検出の際にはノイズの影響を除去するためにノイズ除去処理や平滑化処理を行っても良い。以下、輪郭線検出処理を図６を用いて説明する。
【００９１】
図６（ａ）において、輪郭線検出処理は、Ａ点から開始して動領域を一回りしてＡ点に戻ってくる。Ａ点の決定方法は、一番最初に動領域を検出した時点において、その動領域の最左上点をＡ点とする。次の処理からは、直前に検出されたＡ点の近傍領域Ｅ内に存在する動領域を検出し、その最左上点を処理開始点Ａとする。輪郭線が検出されたならば、その輪郭線から輪郭線ベクトルを検出する。いま、輪郭線ベクトルを検出したいｉ個目の輪郭線座標をＰ_i （ｘ_i ，ｙ_i ）、そのｋ個後（ｉ，ｋは１以上の整数）の輪郭線座標をＰ_i+k （ｘ_i+k ，ｙ_i+k ）とすると、その点の輪郭線ベクトルは、下記式（４）として算出される。
【００９２】
Ｖ_i （ｖ_xi，ｖ_yi）＝Ｖ_i （ｘ_i+k −ｘ_i ，ｙ_i+k −ｙ_i ） ・・・（４）
次に、この輪郭線ベクトルの特徴点を検出するため、下記式（５）を用いて評価値Ｒとして算出する。
Ｒ＝（ｖ_xi−ｖ_xi-1）² ＋（ｖ_yi−ｖ_yi-1）² ・・・（５）
但し、Ｖ_i-i （ｖ_xi-1，ｖ_yi-1）はｉ−１番目の輪郭線ベクトル
【００９３】
図６（ａ）におけるＲの度数分布を図６（ｂ）に示す。直線部分の輪郭ベクトルの評価値Ｒは、原点付近に集まるため、それ以外の度数分布が集積した部分を輪郭線における特徴点とし、その座標を輪郭線の特徴点座標とする。図６（ｃ）のような端点は唯一の特徴点として検出される。また、図６（ｄ）のように複数の特徴点が連続して検出された場合は評価値Ｒ最大の一点のみを特徴点とする。図６（ａ）においては、点Ａ、点Ｂ、点Ｃ、点Ｄが特徴点として検出される。こうして検出された特徴点は、Ｎフレーム前（Ｎは正の整数）の特徴点と対応付け処理を行い、移動ベクトルとして検出する。対応付け処理では、特徴点座標及び輪郭線ベクトルの類似度より対応点を検索し、該当する特徴点が存在しない場合は、その特徴点における輪郭線ベクトルは検出しないものとする。移動ベクトルのｘ及びｙ方向成分分布を図６（ｅ）及び図６（ｆ）に示す。このそれぞれの最頻度値ｘ_v 及びｙ_v を成分とするベクトルＶ（ｘ_v ，ｙ_v ）を現在着目している移動物体の移動ベクトルとして算出する。但し、移動物体の形状よっては特徴点が余り検出されない場合があり、十分な移動ベクトルの成分分布が得られない。このときは、各移動ベクトル成分の平均値やメジアンを成分とする移動ベクトルを算出し、移動物体の移動ベクトルとする。これを移動物体の特徴量として、時系列処理部７及び移動物体識別部８に送出する。時系列処理部７は、履歴情報に基づいて時系列特徴量を算出する。移動物体識別部８は、これら送られてきた特徴量の評価を行う。特徴量評価は、現フレームの特徴量と前フレームの特徴量の差の評価、時系列処理部７において算出された時系列特徴量との差の評価、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００９４】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として移動物体の移動ベクトルを検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【００９５】
（実施例２６）
（位置検出部）
以下、本発明の第２６の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは位置検出部６以降における動作である。以下、本実施例の動作について説明する。
【００９６】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって検出された動領域を包含する外接長方形を図７に示す。
【００９７】
図７において、Ｗは外接長方形の幅である。位置検出部６は移動物体の運動平面との接点の最上点の画像上の座標Ｑを算出し、これを位置情報とする。点Ｐの画像上位置座標をＰ（ｘ，ｙ）とすると、点Ｑの位置座標は下記式（６）として算出される。
Ｑ＝（ｘ＋Ｗ／２，ｙ） ・・・（６）
【００９８】
こうして算出された移動物体位置情報は、時系列処理部７及び移動物体識別部８へと送られる。時系列処理部７は、履歴情報に基づいて時系列処理された位置情報を算出する。移動物体識別部８は、これら送られてきた位置情報の評価を行う。評価は、現フレームと前フレームの位置情報の差の比較、時系列処理部７において算出された位置情報との差の比較により行う。またこの位置情報と動領域の大きさから、複数の移動物体が画面上で交差したというような特殊事象認識を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【００９９】
以上のように、本実施例によれば、位置検出部６によって移動物体の位置の特徴量として移動物体の運動平面との接点の最上点の座標を検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体を識別し追跡することができるという点で優れた効果が得られる。
【０１００】
（実施例２７）
以下、本発明の第２７の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは位置検出部６以降における動作である。以下、本実施例の動作について説明する。
【０１０１】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって検出された動領域を包含する外接長方形を図７に示す。
【０１０２】
図７において、Ｈは外接長方形の高さである。位置検出部６は、移動物体の運動平面との接点の画像上の座標Ｒを算出し、これを位置情報とする。点Ｐの画像上位置座標をＰ（ｘ，ｙ）とすると、点Ｒの位置座標は下記式（７）として算出される。
Ｒ＝（ｘ＋Ｗ／２，ｙ＋Ｈ） ・・・（７）
【０１０３】
こうして算出された移動物体位置情報は、時系列処理部７及び移動物体識別部８へと送られる。時系列処理部７は、履歴情報に基づいて時系列処理された位置情報を算出する。移動物体識別部８は、これら送られてきた位置情報の評価を行う。評価は、現フレームと前フレームの位置情報の差の比較、時系列処理部７において算出された位置情報との差の比較により行う。またこの位置情報と動領域の大きさから、複数の移動物体が画面上で交差したというような特殊事象認識を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【０１０４】
以上のように、本実施例によれば、位置検出部６によって移動物体の位置の特徴量として移動物体の運動平面との接点座標を検出し、この特徴量及びその時系列特徴量を用いることにより、移動物体を識別し追跡することができるという点で優れた効果が得られる。
【０１０５】
（実施例２８）
以下、本発明の第２８の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは位置検出部６以降における動作である。以下、本実施例の動作について説明する。
【０１０６】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって検出された動領域を包含する外接長方形を図７に示す。
【０１０７】
位置検出部６は、移動物体の運動平面との接点の最上点の画像上の座標Ｑ及び運動平面との接点の画像上の座標Ｒを算出し、これらを位置情報とする。点Ｐの画像上位置座標をＰ（ｘ，ｙ）とすると、点Ｑの位置座標は上記式（６）、点Ｒの位置座標は上記式（７）として算出される。こうして算出された移動物体位置情報は時系列処理部７及び移動物体識別部８へと送られる。時系列処理部７は、履歴情報に基づいて時系列処理された位置情報を算出する。移動物体の運動平面との接点が障害物によって遮蔽された場合、移動物体識別部８は、これら位置情報において座標Ｑとその時系列処理によって算出された座標Ｑ’との差、座標Ｒとその時系列処理によって算出された座標Ｒ’との差、外接長方形等の特徴量及び移動物体の画面上での交差等特殊事象認識による評価結果を基に総合的に判断し、移動物体の運動平面との接点が障害物によって遮蔽されているものと判断する。この判断がなされたならば、移動物体識別部８は、座標Ｑ及び移動物体の推定高Ｈ_e を用いて推定座標Ｒ_e を下記式（８）として算出し、移動物体の運動平面との接点座標とする。
Ｒ_e （ｘ＋Ｗ／２，ｙ＋Ｈ_e ） ・・・（８）
【０１０８】
移動物体推定高Ｈ_e は、動領域外接長方形の前フレーム特徴量及びその時系列特徴量を用いて算出する。移動物体識別部８は、こうして求められた運動平面との接点座標を特徴量として、移動物体の識別追跡のために用いる。
【０１０９】
以上のように、本実施例によれば、位置検出部６によって検出された移動物体の特徴量として、検出された移動物体の運動平面との接点及びその最上点の座標を用いて、移動物体の運動平面との接点が障害物によって遮蔽されていることを検出し、移動物体の運動平面との接点座標を特徴量として推定算出することにより、移動物体を識別し追跡することができるという点で優れた効果が得られる。
【０１１０】
（実施例２９）
（時系列処理部）
以下、本発明の第２９の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは時系列処理部７以降における動作である。以下、本実施例の動作について説明する。
【０１１１】
図１において、入力部１から特徴量検出部５及び位置検出部６迄の処理の流れは、実施例１における説明と同様である。特徴量検出部５及び位置検出部６によって検出された情報は、時系列処理部７に送られる。時系列処理部７は、現フレーム処理において検出された特徴量Ｔ_c とそれ以前のフレーム処理において算出されていた時系列特徴量Ｔ_p に対し、下記式（９）を用いて現フレームにおける時系列特徴量として算出する。

【０１１２】
処理開始の一番最初は、現フレーム処理において検出された特徴量を時系列特徴量として登録し、時間が経過するにつれて更新して行く。算出された時系列特徴量は、移動物体識別部８へと送出され、実施例１と同様に、特徴量と統合されて識別処理が行われる。識別処理は、現フレーム特徴量と前フレーム特徴量との差、時系列特徴量との差、移動物体の画面上での交差等特殊事象認識による評価を行う。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【０１１３】
以上のように、本実施例によれば、時系列処理部７によって移動物体の特徴量の時系列特徴量を算出し、これを特徴量として用いることにより、ノイズ等の影響による特徴量の急峻な変化を吸収し、時間的に変化している特徴量については追従して行く特徴量を算出することにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【０１１４】
（実施例３０）
以下、本発明の第３０の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは時系列処理部７以降における動作である。以下、本実施例の動作について説明する。
【０１１５】
図１において、入力部１から特徴量検出部５及び位置検出部６迄の処理の流れは、実施例１における説明と同様である。特徴量検出部５及び位置検出部６によって検出された情報は、時系列処理部７に送られる。時系列処理部７は、現フレーム処理において検出された特徴量とＫフレーム前（Ｋは正の整数）の処理に検出された特徴量との差の絶対値ｄＴ_c と、それ以前のフレーム処理において算出されていたＫフレーム間の差の絶対値の時系列特徴量ｄＴ_p に対し、下記式（１０）を用いて現フレームにおける時系列特徴量として算出する。

【０１１６】
この処理は、Ｋフレーム経過する迄実行されない。処理開始の一番最初は現フレーム処理において検出された特徴量の差を時系列特徴量として登録し、時間が経過するにつれて更新して行く。算出された時系列特徴量は、移動物体識別部８へと送出され、実施例１と同様に、特徴量と統合されて識別処理が行われる。識別処理は、現フレーム特徴量とＫフレーム前の特徴量の差と、この時系列特徴量との差を評価して行う。この処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【０１１７】
以上のように、本実施例によれば、時系列処理部７によって移動物体の特徴量のＫフレーム間の差の絶対値及びその時系列特徴量を算出し、これを特徴量として用いることにより、時間的に一定の変化をしている特徴量の特性を検出することができると共に、ノイズ等の影響による特徴量の急峻な変化を吸収しながら、時間的に変化している特徴量については追従して行く特徴量を算出することにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【０１１８】
（実施例３１）
（移動物体識別部）
以下、本発明の第３１の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは主に移動物体識別部８における動作である。以下、本実施例の動作について説明する。
【０１１９】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例２５と同様に、移動物体の移動ベクトルを検出する。そして最頻度値ｘ_v 及びｙ_v を成分とするベクトルＶ（ｘ_v ，ｙ_v ）を現在着目している移動物体の移動ベクトルとして算出する。ここでベクトル群Ｖｇ（ｘ_v ±ｄｘ，ｙ_v ±ｄｙ）に属する移動ベクトルを生成する特徴点の数の、全特徴点数に占める割合を移動ベクトルが検出できなかった特徴点も含めて算出する。ｄｘ及びｄｙは予め設定しておく。そしてこの割合が予め与えられた閾値以上の場合に剛体、未満の場合に非剛体として判定する。この判定結果を移動物体識別部８に送出し、移動物体識別部８は、その判定結果を移動物体の属性として、移動物体追跡処理に用いる。
【０１２０】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として移動物体の移動ベクトルを検出し、この特徴量を用いて移動物体が剛体であるか非剛体であるかを判定することにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【０１２１】
（実施例３２）
以下、本発明の第３２の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは主に移動物体識別部８における動作である。以下、本実施例の動作について説明する。
【０１２２】
図１において、入力部１から動領域整形部４迄の処理の流れは、実施例１における説明と同様である。動領域整形部４によって整形された動領域に対し、特徴量検出部５は、実施例２５と同様に、移動物体の移動ベクトルを検出する。そして実施例３１と同様の方法により、その移動物体が剛体であるか非剛体であるかを判定し、その判定結果を時系列処理部７及び移動物体識別部８に送出する。非剛体と判定された場合、時系列処理部７は、履歴情報に基づいて移動物体の形状特徴量の時間変化の固有周期を求める。２×Ｎフレーム（Ｎは正の整数）の特徴量の数列を格納しておき、そのうち手前のＮフレームの特徴量数列を０からＮ−１フレーム迄シフトさせて各特徴量毎の差の自乗を計算し、その総和の最小値を相関値Ｒとして下記式（１１）を用いて算出する。
【０１２３】
【数３】

【０１２４】
処理の簡単化を図るために、特徴量の差の絶対値の総和を相関値Ｒとしても良い。導かれた相関値Ｒが予め設定されていた閾値に比べて小さいときは、特徴量に周期性があったと判定し、そのときのシフト数Ｋを特徴量の固有周期とする。この判定結果を移動物体識別部８に送り、移動物体識別部８は、この判定結果を移動物体の属性として、移動物体追跡処理に用いる。
【０１２５】
以上のように、本実施例によれば、特徴量検出部５によって移動物体の特徴量として移動物体の移動ベクトルを検出し、この特徴量を用いて移動物体が剛体であるか非剛体であるかを判定し、非剛体であると判定された場合に、その移動物体の形状特徴量の時間的変化の固有周期を検出することにより、移動物体の識別を実現することができるという点で優れた効果が得られる。
【０１２６】
（実施例３３）
以下、本発明の第３３の実施例について説明する。本実施例における動画像処理装置は、図１に示した第１の実施例と同じ構成を備え、異なるのは移動物体識別部８以降における動作である。以下、本実施例の動作について説明する。
【０１２７】
図１において、入力部１から特徴量検出部５、位置検出部６及び時系列処理部７迄の処理の流れは、実施例１における説明と同様である。これらの処理結果は移動物体識別部８へと送られ、移動物体識別部８は、これら送られてきた情報を統合処理し、移動物体を識別する。識別処理は、特徴量の評価と判定処理により行われる。特徴量評価は、検出対象に基づく特徴量それ自体の評価、現フレーム特徴量と前フレーム特徴量との距離による評価、時系列特徴量の評価、そして位置情報を用いた複数の移動物体が画面上で交差したというような特殊事象認識による評価を行う。例えば現フレーム処理において検出された特徴量をＴ_c 、時系列特徴量をＴ_p 、判定閾値をαとした時、下記式（１２）が満足されない場合、移動物体識別部８は特徴量の信頼度が低いと判断する。
｜Ｔ_c −Ｔ_p ｜≦α ・・・（１２）
【０１２８】
次に、これら評価結果を用いた判定処理を行う。この判定処理では、複数の移動物体交差という特殊事象の発生により、特徴量評価結果が判定条件を満たさないという結果を許容する。特殊事象の発生が検出されず、特徴量評価結果が不十分である場合は、その特徴量検出に相当する特徴量検出部５或いは位置検出部６にパラメータ変更指令を与える。更に移動物体識別部８は、ある程度十分な結果が得られた特徴量については、その特徴量を基にその移動物体を検出する最適パラメータを検出し、それを前記画像間演算処理部３、動領域整形部４、特徴量検出部５、位置検出部６に伝達することもできる。例えば、特徴量を統合することによって移動物体の移動速度及び方向が算出されたならば、その条件を画像間演算処理部３及び動領域整形部４に送り、これらの処理部は、この条件に最も適した時間間隔で画像間演算処理や動領域整形処理を行うことが可能である。また移動物体の大きさからノイズ除去処理の最適パラメータを動的に決定し、動領域整形部４に送出することも可能である。或いは高信頼度の時系列特徴量Ｔ_p を特徴量検出部５及び位置検出部６に送出し、各検出部はこの情報を自分自身の処理に反映させることも可能である。これらの処理を動画像に対して連続的に行い、前処理画面と同一移動物体と判断された動領域を同一動領域と決定することにより、同一移動物体を追跡する。
【０１２９】
以上のように、本実施例によれば、移動物体識別部８が、特徴量検出部５、位置検出部６及び時系列処理部７から送られてきた情報を統合して移動物体を識別し、評価結果が不十分である場合は、それを画像間演算処理部３、動領域整形部４、特徴量検出部５、位置検出部６に伝達することによってパラメータの最適化を図り、正確な移動物体の識別を実現することができるという点で優れた効果が得られる。
【０１３０】
【発明の効果】
以上のように、本発明は、移動物体の特徴量を検出する特徴量検出部を設けることにより、移動物体を特徴量により評価判定することで移動物体を識別し追跡することができる。また、フレーム間差分処理を用いた場合に、移動物体が静止して動領域が検出不可能な場合においても、再度動作を開始して動領域が検出された時に、移動物体を特徴量によって同定することにより、同一物体と識別して追跡することができる。さらに、移動物体識別部は、特徴量それ自体と時系列特徴量を用いて移動物体検出の評価を行うため、特徴量のノイズ等による急峻な変化を吸収して正確な移動物体の追跡を実現することができ、また移動物体の識別結果が不十分であった場合は、その特徴量検出に相当する処理部に対して処理条件の変更を指令することができ、ある程度十分な結果が得られた特徴量については、その特徴量を基にその移動物体を検出する最適条件を検出し、それを画像間演算処理部、動領域整形部、特徴量検出部、位置検出部に伝達することにより、優れた動画像処理装置を実現できるものである。
【図面の簡単な説明】
【図１】本発明の第１から第３３の実施例における動画像処理装置のブロック結線図
【図２】本発明の第２の実施例における動領域内からの移動物体検出を示す概念図
【図３】本発明の第３の実施例における動領域整形法を示す概念図
【図４】本発明の第６の実施例における度数分布図
【図５】本発明の第８の実施例における分割領域の位置座標決定法を示す概念図
【図６】（ａ）本発明の第２５の実施例における輪郭線検出法を示す概念図
（ｂ）本発明の第２５の実施例における輪郭ベクトル評価値の度数分布
（ｃ）本発明の第２５の実施例における輪郭線特徴点
（ｄ）本発明の第２５の実施例における輪郭線特徴点
（ｅ）本発明の第２５の実施例における移動ベクトルｘ成分分布
（ｆ）本発明の第２５の実施例における移動ベクトルｙ成分分布
【図７】本発明の第２６〜第２８の実施例における動領域位置情報を示す概念図
【図８】従来の実施例における動画像処理装置のブロック結線図
【符号の説明】
１ 画像入力部
２ 画像格納部
３ 画像間演算処理部
４ 動領域整形部
５ 特徴量検出部
６ 位置検出部
７ 時系列処理部
８ 移動物体識別部
２１ 画像間演算処理部によって検出された動領域
２２ 検出対象移動物体領域
３１ Ｎフレーム（Ｎは正の整数）前の動領域及び外接長方形
３２ 現フレームの動領域
３３ 現フレームの動領域
３４ ３１〜３３の外接長方形を統合した状態
３５ ３４を包含する外接長方形に現フレーム動領域をプロットした状態
３６ 整形された外接長方形
５１ 動領域の分割領域
５２ 動領域の分割領域
５３ 動領域の分割領域
８１ 画像入力部
８２ 画像格納部
８３ 画像前処理部
８４ 動領域整形部
８５ 移動物体追跡部[0001]
[Industrial application fields]
The present invention continuously processes the image data input from the moving image input device, detects the moving object from the information obtained from it, and tracks the moving path, thereby monitoring the operation of the moving object such as a person or a car. The present invention relates to a moving image processing apparatus.
[0002]
[Prior art]
Research and development has already been conducted on a system for detecting moving objects by continuously processing moving images, and several systems have been put into practical use. The moving object detection process in these moving image processing apparatuses will be described with reference to FIG. FIG. 8 is a block connection diagram of a conventional moving image processing apparatus. In FIG. 8, 81 is an image input unit, 82 is an image storage unit, 83 is an image preprocessing unit, 84 is a moving area shaping unit, and 85 is a moving object tracking unit.
[0003]
The operation of the conventional moving image processing apparatus will be described below. The moving image input from the image input unit 81 is stored in the image storage unit 82 and preprocessed by the image preprocessing unit 83. Pre-processing includes inter-frame difference processing for performing difference processing between an image K frames before (K is a positive integer) and the current image, background difference processing for creating a background image and performing difference processing with the current image, or A single image spatial differentiation process is used. The moving area detected by the preprocessing is subjected to area shaping processing such as noise removal in the moving area shaping unit 84 to determine the moving area. When the moving area is determined, the moving object tracking unit 85 identifies the moving area previously detected using the moving area information and its history information, and tracks the same moving object. By continuously performing these processes on a moving image, a moving object can be detected and tracked.
[0004]
In the preprocessing of the image preprocessing unit 83, threshold processing is performed. The determination parameter is also used in processing such as noise removal and region integration by the moving region shaping unit 84. Conventionally, for these parameters, optimum values matched with imaging conditions such as a visual field range and a depression angle have been provided to an image processing apparatus in advance. For dynamic parameters such as difference thresholds that need to be adapted to changing environmental conditions, such as outdoor brightness, a method of detecting the brightness change in the processed image by image processing or other sensors can be used. A detection method is performed, and the image processing apparatus selects an optimum parameter based on the result.
[0005]
[Problems to be solved by the invention]
However, in such a conventional moving image processing system, the following problems have been pointed out. First, in the inter-frame difference processing, when the moving object is stationary, the difference is not detected, and tracking becomes difficult. In addition, in background difference processing, it is necessary to always create an optimal background image, and the creation and updating of a background image is a big problem in the outdoors where the environment changes rapidly. Since the spatial differentiation process is not a process in which only a moving region is detected, it is necessary to set an object existing region in advance when used for detecting a moving object. Alternatively, a method of detecting a moving area by differential processing of a spatial differential image or recognizing a moving object including a stationary object can be considered, but a simple differential processing of a spatial differential image can accurately It is difficult to detect, and there are the same problems as the inter-frame difference process and the background difference process. On the other hand, static object recognition is difficult to implement unless it is an environment in which a recognition target object can be specified, and the applicable range must be limited. Therefore, in order to detect and track an accurate moving object using these preprocessing methods, a solution to the above is required.
[0006]
In the image pre-processing unit 83 and the moving area shaping unit 84, parameters are used. However, since the parameters given in advance are not sufficient, if a moving object cannot be detected well or noise and shadow removal processing is performed successfully. In some cases, environmental parameters could not be detected well, and appropriate parameters could not be selected. Alternatively, even if the environmental conditions can be detected well, there are cases where the optimum parameters cannot be determined only by the environmental conditions.
[0007]
The present invention solves the above-described conventional problems, and realizes tracking including the stationary state of the moving object even if inter-frame difference processing is used by detecting and evaluating the feature amount of the moving object. An object of the present invention is to provide a moving image processing apparatus capable of realizing accurate moving object detection by immediately changing to an optimum parameter when parameters determined by environmental conditions are insufficient. .
[0008]
[Means for Solving the Problems]
To achieve the above object, a moving image processing apparatus according to the present invention includes an image input unit that inputs a moving image, an image storage unit that stores image data from the image input unit in units of frames, and the image storage unit. An inter-image calculation processing unit that performs inter-frame difference calculation processing on the frame image data stored in the image, and detects only a region where a moving object exists in the moving region detected by the inter-image calculation processing unit. A moving region shaping unit, a feature amount detecting unit that detects the feature amount of a moving object by processing image data in the moving region detected by the moving region shaping unit, and a moving region detected by the moving region shaping unit. A new time series in the current frame from the position detection unit that detects the feature quantity of the position of the moving object based on the region information, and the feature quantity of the moving object in the current frame and the time series feature quantity calculated in the previous frame The time series processing unit that calculates and updates the collected amount, the feature amount detected by the feature amount detection unit and the position detection unit, and the result of the time series feature amount calculated by the time series processing unit are integrated and moved Equipped with a moving object identification unit that identifies objects The moving object identification unit integrates the information sent from the feature amount detection unit, the position detection unit, and the time series processing unit to evaluate the feature amount of the moving object, and the evaluation result of the feature amount is insufficient If it is, a parameter change command is instructed to the inter-image calculation processing unit, the moving region shaping unit, the feature amount detection unit, and the position detection unit. is doing.
[0009]
[Action]
With the above-described configuration, the moving image processing apparatus of the present invention can identify and track a moving object by detecting a feature amount of the moving object and performing an evaluation determination. In addition, when using the inter-frame difference processing, even if the moving object is stationary and the moving area cannot be detected, the moving object is identified by the feature amount when the moving area is detected after the operation is started again. By doing so, it is possible to identify and track the same object. In addition, the moving object identification unit evaluates the moving object detection using the feature quantity itself and the time series feature quantity, and if the result is insufficient, the moving object identification section performs the processing corresponding to the feature quantity detection. For feature quantities that can be commanded to change the processing conditions, and that have obtained a sufficiently satisfactory result, an optimum condition for detecting the moving object is detected based on the feature quantities, and the inter-image calculation processing unit is detected. It can also be transmitted to the moving region shaping unit, the feature amount detection unit, and the position detection unit.
[0010]
【Example】
Example 1
(Overall configuration)
A first embodiment of the present invention will be described below with reference to FIG. FIG. 1 is a block connection diagram of the moving image processing apparatus. In FIG. 1, 1 is an image input unit, 2 is an image storage unit, 3 is an inter-image calculation processing unit, 4 is a moving region shaping unit, 5 is a feature amount detection unit, 6 is a position detection unit, and 7 is a time-series processing unit. , 8 are moving object identification units.
[0011]
Next, the operation of the above embodiment will be described. The moving image input from the image input unit 1 is stored in the image storage unit 2. The image storage unit 2 can store images for N frames (N is a positive integer). The inter-image calculation processing unit 3 performs an inter-frame difference calculation at a set time interval using these N images, and extracts pixels in which a difference equal to or greater than a certain threshold is detected. This is N frames The Cumulative addition did background image And the current frame It may be a difference calculation process. Based on the difference result, the moving area shaping unit 4 performs noise and shadow removal processing and area integration processing to determine the moving area. The feature amount detection unit 5 processes the image data in the moving region determined by the moving region shaping unit 4 and detects the feature amount. The position detection unit 6 calculates the feature amount of the position coordinates on the screen from the moving area detection information as moving object position information. Information detected by the feature amount detection unit 5 and the position detection unit 6 is sent to the time series processing unit 7, and a time series feature amount is calculated based on the history information. (See Example 29 described later) . The output information of the feature quantity detection unit 5, the position detection unit 6, and the time series processing unit 7 is sent to the moving object identification unit 8. The moving object identification unit 8 performs an integration process on the received information and moves the information. Identify the object. The identification processing is performed by feature amount evaluation and determination processing. The feature amount evaluation is based on the detection target based on the feature amount itself, the evaluation based on the distance between the current frame feature amount and the previous frame feature amount, the time series feature amount evaluation, and multiple moving objects using position information on the screen. We evaluate by special event recognition such as crossing above. Next, determination processing using these evaluation results is performed. In this determination process, the result that the feature amount evaluation result does not satisfy the determination condition due to the occurrence of a special event such as a plurality of moving object intersections is allowed. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0012]
As described above, according to the present embodiment, the feature of the moving object is detected by the feature amount detection unit 5 and the position detection unit 6 from the moving region detected by the inter-image arithmetic processing unit 3 and shaped by the moving region shaping unit 4. The moving object can be identified and tracked by detecting the quantity and integrating and processing the feature quantity and the time series feature quantity created by the time series processing section 7 by the moving object identification section 8. Excellent effect is obtained.
[0013]
(Example 2)
(Moving area shaping part)
Next, a second embodiment of the present invention will be described. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the difference is the operation after the moving area shaping unit 4. Hereinafter, the operation of this embodiment will be described.
[0014]
In FIG. 1, the flow of processing from the input unit 1 to the inter-image arithmetic processing unit 3 is the same as that described in the first embodiment. For the moving area detected by the inter-image arithmetic processing unit 3, the moving area shaping unit 4 performs a spatial differentiation process on the luminance of the image data in the moving area to detect edge feature points. Spatial differentiation processing can be performed by using a gradient operator, or by a method of detecting a zero crossing point using a second-order difference method. The moving region is corrected by regarding the edge feature point as a contour. Hereinafter, this process will be described with reference to FIG.
[0015]
In FIG. 2, 21 is a moving area detected by the inter-image arithmetic processing unit 3, and 22 is a detection target moving object area. Edge feature point data in the moving region 21 detected by the inter-image arithmetic processing unit 3 is added in the scanning line direction to detect the contour upper limit A and lower limit B. A region between A and B is a processing region. First, an edge feature point is searched in the right direction from the moving region left outline L1. If a feature point is detected, it is registered as a contour candidate point. Similarly, the right contour line L2 is searched in the left direction. Next, edge feature point data in the moving region 21 is added in the direction perpendicular to the scanning line, and the contour left limit C and right limit D are detected. The upper and lower contour lines are detected using the region between C and D as a processing region. Edge feature points are searched in the downward direction on L3 to detect the upper contour line, and edge feature points are searched in the upward direction on L4 to detect the lower contour line. In these contour extraction processes, a correct contour point is not detected when noise or edge loss occurs, and therefore noise removal processing and smoothing processing may be performed before the contour extraction processing. Using the contour line thus detected as a moving area, the same moving object is tracked in the same manner as in the first embodiment.
[0016]
As described above, according to the present embodiment, the moving region shaping unit 4 realizes accurate contour extraction of a moving object, thereby determining a moving region in which an accurate feature amount of the moving object can be detected. An excellent effect is obtained in that it can be performed.
[0017]
(Example 3)
Next, a third embodiment of the present invention will be described. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the difference is the operation after the moving area shaping unit 4. Hereinafter, the operation of this embodiment will be described.
[0018]
In FIG. 1, the flow of processing from the input unit 1 to the inter-image arithmetic processing unit 3 is the same as that described in the first embodiment. The moving area shaping unit 4 performs moving area shaping processing on the moving area detected by the inter-image arithmetic processing unit 3. Hereinafter, this process will be described with reference to FIG.
[0019]
In FIG. 3, 31 is a moving area and circumscribed rectangle of N frames (N is a positive integer), 32 and 33 are moving areas and circumscribed rectangles of the current frame, 34 is a combination of these circumscribed rectangles, and 35 is all A plot of the current frame moving area within the integrated circumscribed rectangle including the circumscribed rectangle 34, and 36 indicate a circumscribed rectangle newly shaped so as to circumscribe the integrated circumscribed rectangle 35 to the current frame moving area. The moving area shaping unit 4 integrates the circumscribed rectangles 32 and 33 for the moving area in the current frame and the circumscribed rectangle 31 N frames before, as shown by 34, and creates a circumscribed rectangle 35 including all of them. Then, the only circumscribed rectangle 36 circumscribing the current frame moving area is detected from the circumscribed rectangle 35. This is determined as the circumscribed rectangle in the current frame, and the moving area in the circumscribed rectangle is set as the moving area of the same moving object.
[0020]
As described above, according to the present embodiment, it is possible to accurately identify and track the moving object by realizing the accurate detection of the circumscribed rectangle including the moving object by the moving area shaping unit 4. An excellent effect is obtained in terms of points.
[0021]
(Example 4)
(Feature amount detector)
The fourth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0022]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature quantity detection unit 5 processes the luminance information of the image data in the moving area, and calculates the luminance average value and the variance value in the moving area. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature amount of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the current frame average value and the variance value and the previous frame average value and the variance value, evaluating the difference between the average value and the variance value calculated by the time series processing unit 7, and on the moving object screen. Evaluation by recognition of special events such as intersections at These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0023]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the luminance average value and the variance value as the feature amount of the moving object, and uses these feature amount and its time series feature amount, thereby An excellent effect is obtained in that identification can be realized.
[0024]
(Example 5)
The fifth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0025]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature quantity detection unit 5 processes the luminance information of the image data in the moving area, and calculates the frequency distribution of the luminance in the moving area. In the case of data of 256 gradations, the frequency distribution is obtained by calculating the frequency for every 8 gradations, 16 gradations, or 32 gradations. The obtained frequency distribution is subjected to normalization processing based on the total number of pixels in the moving region or the most frequent frequency. This is sent to the moving object identification unit 8 as a feature amount of the moving object. The moving object identification unit 8 holds the luminance frequency distribution of the previous frame, and calculates the distance between this and the frequency distribution of the current frame. As a distance calculation method, a city block distance (city block distance, the sum of absolute values of differences between frequencies, see Example 16) or Euclidean distance (the sum of squares of differences between frequencies) may be used. Here, the DP matching method is used. The frequency distribution sequence of the current frame is c _i P for the frequency distribution sequence of the previous frame _j When (i, j = 1 to N, N is a positive integer), the distance D is calculated by the following equation (1).
[0026]
[Expression 1]

Here, when calculating the distance, the following equation (2) is imposed as a constraint condition of the matching window.
[0027]
| I−j | ≦ r (r is a positive integer set in advance) (2)
When the distance D is equal to or less than a preset threshold value, these moving areas are those of the same moving object. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0028]
As described above, according to the present embodiment, the feature amount detector 5 calculates the luminance frequency distribution as the feature amount of the moving object, and uses this as the feature amount, thereby realizing the identification of the moving object. An excellent effect is obtained in that it can be performed.
[0029]
(Example 6)
The sixth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0030]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 processes the luminance information of the image data in the moving area and divides the luminance in the moving area into M pieces (M is a positive integer). To do. In the case of M = 2, an average value can be used as the threshold value. Another possible method is to use a threshold value that maximizes the variance between classes. In this case, even if M is 2 or more, it can be used if the number of divisions can be determined. Furthermore, a method of determining the number of divisions using a frequency distribution and dividing the data is also conceivable. Hereinafter, this process will be described with reference to FIG.
[0031]
FIG. 4 shows a frequency distribution. In the case of data of 256 gradations, the frequency distribution is obtained by calculating the frequency for every 8 gradations, 16 gradations, or 32 gradations. When such a frequency distribution is obtained, the frequency distribution is divided by maximum / minimum point detection. Here, differential processing is performed on the frequency distribution. Here, the number in which the derivative has changed from positive to negative can be considered as the number of area divisions. In FIG. 4, since there are three local maximum points, point A, point B, and point C, the number of divisions is three. Furthermore, the gradation value when the differential coefficient changes from negative to positive can be considered as the region division threshold. In FIG. 4, point D and point E correspond. However, in order to detect the main component of the luminance distribution, a smoothing process is performed if necessary in the differentiation process. When the area division number M and the division threshold are determined in this manner, area division is performed using these thresholds, and a representative value of each luminance in the M division areas is obtained. The representative value may be an average value of luminance for each divided region, or a luminance value that is the most frequent in each divided region. These are sent as feature quantities to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by comparing the area division number M between the current frame and the previous frame, evaluating the difference between the representative values, evaluating the difference between the division number and the representative value calculated by the time-series processing unit 7, and the moving object screen. Perform evaluation by recognition of special events such as intersections above. These processes are continuously performed on the moving image, and the moving area determined to be the same moving object as the pre-processing screen is determined as the same moving area, thereby tracking the same moving object.
[0032]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the luminance distribution as the feature amount of the moving object, and detects the luminance that is a representative value of each divided region. By using these feature quantities and their time series feature quantities, it is possible to obtain an excellent effect in that it is possible to realize identification of a moving object.
[0033]
(Example 7)
The seventh embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0034]
In FIG. 1, the flow of processing from the input unit 1 to the moving region shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 calculates the luminance frequency distribution and detects the luminance division threshold value as in the sixth embodiment, and based on this threshold value, calculates the frequency distribution. Divide into M pieces (M is a positive integer). Then, the ratio of the size of each divided area is detected. The size of the area may be the total number of pixels belonging to each divided area, or the most frequently used frequency in each divided area. These are sent as feature quantities to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the feature amount in the current frame and the previous frame, evaluating the difference from the evaluation value calculated in the time series processing unit 7, and evaluating by recognition of special events such as crossing of moving objects on the screen. I do. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0035]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the luminance distribution as the feature amount of the moving object, and detects the ratio of the size of the divided regions. By using this as a feature amount, an excellent effect can be obtained in that a moving object can be identified.
[0036]
(Example 8)
The eighth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0037]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 converts the image data in the moving area to M pieces (M is a positive integer) based on the luminance information as shown in the sixth embodiment. To divide. Further, position information of each divided area is detected. Hereinafter, this process will be described with reference to FIG.
[0038]
In FIG. 5, reference numerals 51 to 53 denote divided areas, from which position information of the area 51 is detected. First, image data belonging to the divided area 51 is detected from the moving area. At this time, it is not necessary to detect the luminance information of each pixel, and only the position information of each pixel may be detected. Next, the detected image data is added in the vertical and horizontal directions with respect to the scanning line, and the upper limit coordinate y of the divided area _U , Lower limit coordinate y _L , Left limit x _L , Right limit coordinate x _R Is detected. In determining these coordinates, it is determined as a coordinate for obtaining a point at which an added value equal to or greater than a threshold value is obtained in order to remove noise. These may be used as position information, but the upper and lower midpoint y _C And the midpoint x of the left and right limits _C Coordinate P is calculated and used as position information of this divided area. Although detected as one area in FIG. 5, when the area is divided into two or more areas, the coordinates of the center of the entire area may be used as position information, and the size is larger than a preset threshold value. For each region, the center coordinates may be detected as position information. This process is performed for all the divided areas. These are sent as feature quantities to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates these sent feature quantities. In the feature amount evaluation, in addition to the evaluation in the sixth embodiment, the distance between the corresponding current frame and the previous frame is calculated from the position information of each divided area, and the total value is evaluated as the difference between all divided areas. . These processes are continuously performed on the moving image, and the moving area determined to be the same moving object as the pre-processing screen is determined as the same moving area, thereby tracking the same moving object.
[0039]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the luminance distribution as the feature amount of the moving object, detects the position of each divided region, and detects these feature amounts. In addition, by using the time-series feature amount, an excellent effect can be obtained in that the identification of the moving object can be realized.
[0040]
Example 9
The ninth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0041]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 processes the saturation as the color information of the image data in the moving area, and calculates the average value of the saturation in the moving area. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating a difference between the current frame average value and the previous frame average value, a difference between the average value calculated by the time-series processing unit 7, and a special event recognition such as a crossing of a moving object on the screen. . These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0042]
As described above, according to the present embodiment, the feature value detection unit 5 detects the average value of the saturation as the feature value of the moving object, and uses these feature values and their time series feature values to identify the moving object. An excellent effect is obtained in that it can be realized.
[0043]
(Example 10)
The tenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0044]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 processes the saturation as the color information of the image data in the moving area, and calculates the frequency distribution of the saturation in the moving area. The frequency distribution is divided into 8 divisions, 16 divisions or 32 divisions based on the saturation value, and the distribution is obtained by calculating the frequency for each division. The obtained frequency distribution is subjected to normalization processing based on the total number of pixels in the moving region or the most frequent frequency. This is sent to the moving object identification unit 8 as a feature amount of the moving object. The moving object identification unit 8 holds the saturation frequency distribution of the previous frame, and calculates the distance between this and the frequency distribution of the current frame. As the distance calculation method, the city distance or the Euclidean distance may be used, but here the DP matching method is used. The frequency distribution sequence of the current frame is c _i P for the frequency distribution sequence of the previous frame _j When (i, j = 1 to N, N is a positive integer), the distance D is calculated by the above equation (1). Here, when calculating the distance, the expression (2) is imposed as a constraint condition of the matching window. When the distance D is equal to or less than a preset threshold value, these moving areas are those of the same moving object. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0045]
As described above, according to the present embodiment, the feature amount detection unit 5 calculates the saturation frequency distribution as the feature amount of the moving object, and uses this as the feature amount, thereby realizing identification of the moving object. An excellent effect can be obtained in that
[0046]
(Example 11)
The eleventh embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0047]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detecting unit 5 processes the saturation as the color information of the image data in the moving area, and calculates the number of pixels whose saturation is less than a predetermined threshold value. Measure and calculate the ratio of the entire moving area as a low saturation rate. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the current frame low saturation rate and the previous frame low saturation rate, evaluating the difference from the low saturation rate calculated by the time series processing unit 7, and intersecting moving objects on the screen. Evaluate by special event recognition. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0048]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the ratio of the low saturation pixels as the feature amount of the moving object, and uses these feature amounts and the time-series feature amounts. An excellent effect is obtained in that identification of a moving object can be realized.
[0049]
(Example 12)
The twelfth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0050]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 processes the saturation and hue as the color information of the image data in the moving region, and the saturation is equal to or higher than a predetermined threshold value. The frequency distribution of the hue values of is calculated. The hue width for obtaining the frequency distribution is every 10 degrees, 20 degrees, or 30 degrees. The obtained frequency distribution is subjected to normalization processing based on the total number of pixels in the moving region or the most frequent frequency. This is sent to the moving object identification unit 8 as a feature amount of the moving object. The moving object identification unit 8 holds the hue frequency distribution of the previous frame, and calculates the distance between this and the frequency distribution of the current frame. As the distance calculation method, the city distance or the Euclidean distance may be used, but here the DP matching method is used. The frequency distribution sequence of the current frame is c _i P for the frequency distribution sequence of the previous frame _j When (i, j = 1 to N, N is a positive integer), the distance D is calculated by the above equation (1). Here, when calculating the distance, the expression (2) is imposed as a constraint condition of the matching window. When the distance D is equal to or less than a preset threshold value, these moving areas are those of the same moving object. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0051]
As described above, according to the present embodiment, the feature amount detection unit 5 calculates the frequency distribution of the hue as the feature amount of the moving object, and uses this as the feature amount, thereby realizing the identification of the moving object. An excellent effect is obtained in that it can be performed.
[0052]
(Example 13)
The thirteenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0053]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 processes the saturation and hue as the color information of the image data in the moving region, and the saturation is equal to or higher than a predetermined threshold value. The frequency distribution of the hue values of is calculated. The hue width for obtaining the frequency distribution is every 10 degrees, 20 degrees, or 30 degrees. Then, the frequency distribution is divided into M areas (M is a positive integer) as in the sixth embodiment, and a representative value of the hue value for each divided area is obtained. The representative value may be an average value for each divided region, or a hue value that is the most frequent in each divided region. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by comparing the area division number M between the current frame and the previous frame, evaluating the difference between the representative values, evaluating the difference between the division number and the representative value calculated by the time-series processing unit 7, and the moving object screen. Perform evaluation by recognition of special events such as intersections above. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0054]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces as the feature amount of the moving object based on the frequency distribution of the hue value, and the hue that becomes the representative value of each divided region. By detecting a value and using this as a feature amount, an excellent effect is obtained in that it is possible to identify a moving object.
[0055]
(Example 14)
The fourteenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0056]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 calculates a hue value frequency distribution and detects a hue division threshold value as in the thirteenth embodiment, and detects the frequency division based on this threshold value. The distribution is divided into M pieces (M is a positive integer). Then, the ratio of the size of each divided area is detected. The size of the area may be the total number of pixels belonging to each divided area, or the most frequently used frequency in each divided area. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the feature amount in the current frame and the previous frame, evaluating the difference from the evaluation value calculated in the time series processing unit 7, and evaluating by recognition of special events such as crossing of moving objects on the screen. I do. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0057]
As described above, according to the present embodiment, the moving region is divided into M pieces based on the frequency distribution of the hue value as the feature amount of the moving object by the feature amount detection unit 5, and the ratio of the size of the divided region By using this as a feature amount, an excellent effect can be obtained in that a moving object can be identified.
[0058]
(Example 15)
The fifteenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0059]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 calculates a hue value frequency distribution and detects a hue division threshold value as in the thirteenth embodiment, and detects the frequency division based on this threshold value. The distribution is divided into M pieces (M is a positive integer). Then, image data belonging to each divided area is detected. The method for detecting the position information from the detected image data is performed in the same manner as in the eighth embodiment. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. In the feature amount evaluation, in addition to the evaluation in the thirteenth embodiment, the distance between the corresponding current frame and the previous frame is calculated from the position information of each divided area, and the total value is evaluated as the difference between all the divided areas. . These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0060]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the frequency distribution of the hue values as the feature amount of the moving object, and detects the position of each divided region. By using these feature quantities and their time series feature quantities, it is possible to obtain an excellent effect in that it is possible to realize identification of a moving object.
[0061]
(Example 16)
The sixteenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0062]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 detects luminance and color information from the image data in the moving area. Information can be detected by using an RGB coordinate system or a YUV coordinate system. Here, the information is detected by three physical quantities such as luminance, saturation, and hue. Then, a three-dimensional vector space having these three physical quantities as components is defined, and the frequency distribution of the image data in the moving area in this space is obtained. When obtaining the frequency distribution, each component is divided into 8, 16, or 32, and the frequency is calculated for each of the divided vector spaces. The frequency distribution in the three-dimensional vector space thus obtained is subjected to normalization processing based on the total number of moving area pixels or the most frequent frequency. This is sent to the moving object identification unit 8 as the feature quantity of the moving object. The moving object identification unit 8 holds the frequency distribution in the three-dimensional vector space of the previous frame, and calculates the distance between this and the frequency distribution of the current frame. As a method for calculating the distance, a method using the DP matching method is also conceivable. In consideration of the large amount of data, the city distance is used as a simplified method. Of course, other than this, a method of calculating the distance using the Euclidean distance is also conceivable. The frequency distribution of the current frame is c _i , P the frequency distribution of the previous frame _i When i is 1 to N and N is a positive integer, the distance D is calculated by the following equation (3).
[0063]
[Expression 2]

When the distance D is less than or equal to a preset threshold value, these moving areas are those of the same moving object. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0064]
As described above, according to the present embodiment, the feature quantity detection unit 5 calculates the frequency distribution of the three-dimensional vector space having the three physical quantities of luminance, saturation, and hue as components as the feature quantity of the moving object, By using this as a feature amount, an excellent effect can be obtained in that a moving object can be identified.
[0065]
(Example 17)
The seventeenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0066]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 creates a circumscribed rectangle that includes the moving area as the shape information of the image data in the moving area, and detects the lengths of the vertical and horizontal sides. To do. Further, the aspect ratio is calculated, and these are sent to the time-series processing unit 7 and the moving object identification unit 8 as feature quantities of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the length of the side of the current frame and the length of the side of the previous frame and the difference of the aspect ratio, and evaluating the difference between the length of the side and the aspect ratio calculated by the time series processing unit 7. , Evaluation by recognition of special events such as crossing on the screen of moving objects. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0067]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the length and the ratio of the vertical and horizontal sides of the moving region circumscribed rectangle as the feature amount of the moving object, and the feature amount and the time series feature amount thereof. By using, an excellent effect can be obtained in that identification of a moving object can be realized.
[0068]
(Example 18)
The eighteenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0069]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 measures the number of pixels forming the moving area. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the number of moving area pixels in the current frame and the number of moving area pixels in the previous frame, evaluating the difference between the number of moving area pixels calculated in the time-series processing unit 7, and on the moving object screen. Evaluation by recognition of special events such as intersections. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0070]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the number of pixels forming a moving area as the feature amount of the moving object, and uses this feature amount and its time-series feature amount to move. An excellent effect is obtained in that identification of an object can be realized.
[0071]
Example 19
The nineteenth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0072]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature quantity detection unit 5 measures the number of pixels forming the moving area and creates a circumscribed rectangle of the moving area. Then, the ratio of the number of pixels in the moving region to the number of pixels in the circumscribed rectangle is calculated. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the feature amount of the current frame and the feature amount of the previous frame, evaluating the difference between the time series feature amount calculated by the time series processing unit 7, and crossing the moving object on the screen. Evaluate by event recognition. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0073]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the ratio of the number of pixels forming the moving region to the number of pixels of the circumscribed rectangle as the feature amount of the moving object. By using the sequence feature amount, an excellent effect can be obtained in that the moving object can be identified.
[0074]
(Example 20)
The twentieth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0075]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 performs a spatial differentiation process on the luminance of the image data in the moving area to detect a luminance change value. Then, an average value of all luminance change values is obtained and used as texture information of the image data. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. Feature amount evaluation is performed by evaluating the difference between the texture information of the current frame and the texture information of the previous frame, evaluating the difference between the texture information calculated by the time-series processing unit 7, and recognizing special events such as crossing of moving objects on the screen. Perform an evaluation. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0076]
As described above, according to the present embodiment, the feature value detection unit 5 detects the average value of the luminance change value in the moving region as the texture information as the feature value of the moving object, and the feature value and its time-series feature value are detected. By using it, an excellent effect can be obtained in that identification of a moving object can be realized.
[0077]
(Example 21)
The twenty-first embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0078]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 detects the luminance change value in the moving area as texture information, as in the 20th embodiment. Then, the frequency distribution is calculated as a texture frequency distribution. The obtained frequency distribution is subjected to normalization processing based on the total number of pixels in the moving region or the most frequent frequency. This is sent to the moving object identification unit 8 as the feature quantity of the moving object. The moving object identification unit 8 holds the texture frequency distribution of the previous frame, and calculates the distance between this and the frequency distribution of the current frame. As the distance calculation method, the city distance or the Euclidean distance may be used, but here the DP matching method is used. The frequency distribution sequence of the current frame is c _i P for the frequency distribution sequence of the previous frame _j When (i, j = 1 to N, N is a positive integer), the distance D is calculated by the above equation (1). Here, when calculating the distance, the expression (2) is imposed as a constraint condition of the matching window. When the distance D is equal to or less than a preset threshold value, these moving areas are those of the same moving object. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0079]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the frequency distribution of the luminance change value in the moving region as the feature amount of the moving object as the texture frequency distribution, and this feature amount and its time series feature amount. By using, an excellent effect can be obtained in that identification of a moving object can be realized.
[0080]
(Example 22)
The twenty-second embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0081]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detection unit 5 detects the luminance change value in the moving area as texture information, and the frequency distribution is the texture frequency distribution, as in the 20th embodiment. Calculate as Then, the frequency distribution is divided into M pieces (M is a positive integer) as in the sixth embodiment, and a representative value of the luminance change value for each divided area is obtained. The representative value may be an average value of luminance change values for each divided region, or may be a luminance change value that is the most frequent in each divided region. These are sent to the time-series processing unit 7 and the moving object identification unit 8 as feature quantities of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by comparing the area division number M between the current frame and the previous frame, evaluating the difference between the representative values, evaluating the difference between the division number and the representative value calculated by the time-series processing unit 7, and the moving object screen. Perform evaluation by recognition of special events such as intersections above. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0082]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the texture frequency distribution as the feature amount of the moving object, and detects the representative value of the texture of each divided region. By using these feature quantities and their time series feature quantities, it is possible to obtain an excellent effect in that it is possible to realize identification of a moving object.
[0083]
(Example 23)
The twenty-third embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0084]
In FIG. 1, the flow of processing from the input unit 1 to the moving region shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 detects the division threshold by calculating the frequency distribution using the luminance change value as texture information in the same manner as in the twenty-second embodiment. The frequency distribution is divided into M pieces (M is a positive integer). Then, the ratio of the size of each divided area is detected. The size of the area may be the total number of pixels belonging to each divided area, or the most frequently used frequency in each divided area. These are sent as feature quantities to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the feature amount in the current frame and the previous frame, evaluating the difference from the evaluation value calculated in the time series processing unit 7, and evaluating by recognition of special events such as crossing of moving objects on the screen. I do. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0085]
As described above, according to this embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the texture frequency distribution as the feature amount of the moving object, and detects the size ratio of the divided regions. By using this as a feature amount, an excellent effect can be obtained in that a moving object can be identified.
[0086]
(Example 24)
The twenty-fourth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0087]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 calculates the frequency distribution using the luminance change value as texture information and detects the division threshold as in the twenty-second embodiment. And the frequency distribution is divided to detect M (M is a positive integer) divided regions. Then, image data belonging to each divided area is detected. The method for detecting the position information from the detected image data is performed in the same manner as in the eighth embodiment. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature amount of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. In the feature amount evaluation, in addition to the evaluation in the twenty-second embodiment, the distance between the corresponding current frame and the previous frame is calculated from the position information of each divided area, and the total value is evaluated as the difference between all divided areas. . These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0088]
As described above, according to the present embodiment, the feature amount detection unit 5 divides the moving region into M pieces based on the texture frequency distribution as the feature amount of the moving object, detects the position of each divided region, and By using the feature amount and its time-series feature amount, an excellent effect can be obtained in that a moving object can be identified.
[0089]
(Example 25)
The twenty-fifth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0090]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving area shaped by the moving area shaping unit 4, the feature amount detecting unit 5 detects the contour line of the image data in the moving area. When detecting the contour line, noise removal processing or smoothing processing may be performed to remove the influence of noise. Hereinafter, the outline detection process will be described with reference to FIG.
[0091]
In FIG. 6A, the contour line detection process starts from point A and goes around the moving region and returns to point A. In the determination method of the point A, when the moving area is first detected, the upper left point of the moving area is set as the A point. From the next processing, a moving region existing in the vicinity region E of the point A detected immediately before is detected, and the upper left point is set as the processing start point A. If a contour line is detected, a contour line vector is detected from the contour line. Now, the i-th contour coordinate whose contour vector is to be detected is P _i (X _i , Y _i ), The contour coordinates after k (i and k are integers of 1 or more) _{i + k} (X _{i + k} , Y _{i + k} ), The contour vector of the point is calculated as the following equation (4).
[0092]
V _i (V _xi , V _yi ) = V _i (X _{i + k} -X _i , Y _{i + k} -Y _i (4)
Next, in order to detect the feature point of the contour line vector, the evaluation value R is calculated using the following equation (5).
R = (v _xi -V _xi-1 ) ² + (V _yi -V _yi-1 ) ² ... (5)
However, V _ii (V _xi-1 , V _yi-1 ) Is the i-1th contour vector
[0093]
The frequency distribution of R in FIG. 6A is shown in FIG. Since the evaluation value R of the contour vector of the straight line portion gathers in the vicinity of the origin, other portions where the frequency distribution is accumulated are used as feature points in the contour line, and the coordinates are used as the feature point coordinates of the contour line. The end points as shown in FIG. 6C are detected as the only feature points. In addition, when a plurality of feature points are continuously detected as shown in FIG. 6D, only one point having the maximum evaluation value R is set as a feature point. In FIG. 6A, points A, B, C, and D are detected as feature points. The feature points thus detected are associated with the feature points N frames before (N is a positive integer), and detected as a movement vector. In the associating process, a corresponding point is searched based on the similarity between the feature point coordinates and the contour line vector, and when there is no corresponding feature point, the contour line vector at the feature point is not detected. The x and y direction component distribution of the movement vector is shown in FIG. 6 (e) and FIG. 6 (f). Each of the most frequent values x _v And y _v V (x _v , Y _v ) As a movement vector of the moving object currently focused on. However, depending on the shape of the moving object, feature points may not be detected so much, and a sufficient movement vector component distribution cannot be obtained. At this time, a movement vector having an average value or median of each movement vector component as a component is calculated and set as a movement vector of the moving object. This is sent to the time-series processing unit 7 and the moving object identification unit 8 as the feature quantity of the moving object. The time series processing unit 7 calculates a time series feature amount based on the history information. The moving object identification unit 8 evaluates the sent feature quantities. The feature amount evaluation is performed by evaluating the difference between the feature amount of the current frame and the feature amount of the previous frame, evaluating the difference between the time series feature amount calculated by the time series processing unit 7, and crossing the moving object on the screen. Evaluate by event recognition. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0094]
As described above, according to the present embodiment, the feature amount detection unit 5 detects the moving vector of the moving object as the feature amount of the moving object, and uses this feature amount and its time series feature amount to identify the moving object. An excellent effect is obtained in that it can be realized.
[0095]
(Example 26)
(Position detector)
The twenty-sixth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0096]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. A circumscribed rectangle including the moving area detected by the moving area shaping unit 4 is shown in FIG.
[0097]
In FIG. 7, W is the width of the circumscribed rectangle. The position detection unit 6 calculates a coordinate Q on the image of the uppermost point of contact with the moving plane of the moving object, and uses this as position information. Assuming that the position coordinates of the point P on the image are P (x, y), the position coordinates of the point Q are calculated as the following equation (6).
Q = (x + W / 2, y) (6)
[0098]
The moving object position information thus calculated is sent to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates position information that has been time series processed based on the history information. The moving object identification unit 8 evaluates the sent position information. The evaluation is performed by comparing the difference between the position information of the current frame and the previous frame, and comparing the difference between the position information calculated by the time series processing unit 7. In addition, special event recognition is performed such that a plurality of moving objects intersect on the screen from the position information and the size of the moving area. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0099]
As described above, according to the present embodiment, the position detection unit 6 detects the coordinates of the top point of the contact point with the moving plane of the moving object as the feature amount of the position of the moving object, and this feature amount and its time-series feature amount. By using, an excellent effect can be obtained in that a moving object can be identified and tracked.
[0100]
(Example 27)
The twenty-seventh embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0101]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. A circumscribed rectangle including the moving area detected by the moving area shaping unit 4 is shown in FIG.
[0102]
In FIG. 7, H is the height of the circumscribed rectangle. The position detection unit 6 calculates a coordinate R on the image of the contact point of the moving object with the motion plane, and uses this as position information. Assuming that the position coordinates of the point P on the image are P (x, y), the position coordinates of the point R are calculated as the following equation (7).
R = (x + W / 2, y + H) (7)
[0103]
The moving object position information thus calculated is sent to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates position information that has been time series processed based on the history information. The moving object identification unit 8 evaluates the sent position information. The evaluation is performed by comparing the difference between the position information of the current frame and the previous frame, and comparing the difference between the position information calculated by the time series processing unit 7. In addition, special event recognition is performed such that a plurality of moving objects intersect on the screen from the position information and the size of the moving area. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0104]
As described above, according to the present embodiment, the position detection unit 6 detects the contact coordinates with the moving plane of the moving object as the feature amount of the position of the moving object, and uses this feature amount and its time series feature amount. An excellent effect is obtained in that a moving object can be identified and tracked.
[0105]
(Example 28)
The twenty-eighth embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. Hereinafter, the operation of this embodiment will be described.
[0106]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. A circumscribed rectangle including the moving area detected by the moving area shaping unit 4 is shown in FIG.
[0107]
The position detection unit 6 calculates the coordinates Q on the image of the uppermost point of contact with the motion plane of the moving object and the coordinates R on the image of the contact with the motion plane, and uses these as position information. Assuming that the position coordinate of the point P on the image is P (x, y), the position coordinate of the point Q is calculated as the above equation (6), and the position coordinate of the point R is calculated as the above equation (7). The moving object position information thus calculated is sent to the time-series processing unit 7 and the moving object identification unit 8. The time series processing unit 7 calculates position information that has been time series processed based on the history information. When the contact point of the moving object with the motion plane is shielded by an obstacle, the moving object identifying unit 8 determines the difference between the coordinate Q and the coordinate Q ′ calculated by the time series processing in the position information, the coordinate R and the time series. Comprehensive judgment based on the difference from the coordinate R ′ calculated by the process, the feature amount such as the circumscribed rectangle and the evaluation result by the special event recognition such as the intersection of the moving object on the screen, and the movement plane of the moving object It is determined that the contact is shielded by an obstacle. If this determination is made, the moving object identification unit 8 determines the coordinate Q and the estimated height H of the moving object. _e Estimated coordinates R using _e Is calculated as the following equation (8), and is defined as the contact coordinates with the moving plane of the moving object.
R _e (X + W / 2, y + H _e (8)
[0108]
Estimated moving object height H _e Is calculated using the previous frame feature amount and its time series feature amount of the moving region circumscribed rectangle. The moving object identification unit 8 uses the contact coordinates with the motion plane thus obtained as a feature amount for identifying and tracking the moving object.
[0109]
As described above, according to the present embodiment, the moving object is detected by using the contact point of the detected moving object with the motion plane and the coordinates of the uppermost point as the feature amount of the moving object detected by the position detection unit 6. It is possible to identify and track a moving object by detecting that the contact point with the motion plane is blocked by an obstacle and estimating the contact coordinates of the moving object with the motion plane as a feature quantity An excellent effect can be obtained.
[0110]
(Example 29)
(Time series processing section)
The 29th embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the difference is the operation after the time series processing unit 7. Hereinafter, the operation of this embodiment will be described.
[0111]
In FIG. 1, the flow of processing from the input unit 1 to the feature amount detection unit 5 and the position detection unit 6 is the same as that described in the first embodiment. Information detected by the feature quantity detection unit 5 and the position detection unit 6 is sent to the time-series processing unit 7. The time series processing unit 7 uses the feature amount T detected in the current frame processing. _c And the time-series feature amount T calculated in the previous frame processing _p On the other hand, the time series feature amount in the current frame is calculated using the following equation (9).

[0112]
At the very beginning of the process, the feature quantity detected in the current frame process is registered as a time-series feature quantity and updated as time passes. The calculated time-series feature amount is sent to the moving object identification unit 8 and is integrated with the feature amount and identification processing is performed as in the first embodiment. In the identification process, evaluation is performed by special event recognition such as a difference between the current frame feature value and the previous frame feature value, a difference between time series feature values, and a crossing of a moving object on the screen. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0113]
As described above, according to the present embodiment, the time-series processing unit 7 calculates the time-series feature amount of the moving object feature amount, and uses this as the feature amount, thereby sharpening the feature amount due to the influence of noise or the like. An excellent effect is obtained in that it is possible to realize identification of a moving object by calculating a feature value that absorbs such changes and follows a feature value that changes with time.
[0114]
(Example 30)
The 30th embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the difference is the operation after the time series processing unit 7. Hereinafter, the operation of this embodiment will be described.
[0115]
In FIG. 1, the flow of processing from the input unit 1 to the feature amount detection unit 5 and the position detection unit 6 is the same as that described in the first embodiment. Information detected by the feature quantity detection unit 5 and the position detection unit 6 is sent to the time-series processing unit 7. The time series processing unit 7 calculates the absolute value dT of the difference between the feature quantity detected in the current frame process and the feature quantity detected in the process before K frames (K is a positive integer). _c And the time-series feature amount dT of the absolute value of the difference between the K frames calculated in the previous frame processing _p On the other hand, it is calculated as a time series feature amount in the current frame using the following equation (10).

[0116]
This process is not executed until K frames have elapsed. At the very beginning of processing, the difference between the feature amounts detected in the current frame processing is registered as a time-series feature amount, and is updated as time passes. The calculated time-series feature amount is sent to the moving object identification unit 8 and is integrated with the feature amount and identification processing is performed as in the first embodiment. The identification process is performed by evaluating the difference between the current frame feature quantity and the feature quantity before K frames and the time series feature quantity. This process is continuously performed on the moving image, and the moving area determined to be the same moving object as the pre-processing screen is determined as the same moving area, thereby tracking the same moving object.
[0117]
As described above, according to the present embodiment, the time series processing unit 7 calculates the absolute value of the difference between the K frames of the feature amount of the moving object and the time series feature amount, and uses this as the feature amount. It is possible to detect the characteristics of features that change over time, and to follow the features that change over time while absorbing sudden changes in features due to noise and other effects. By calculating the feature amount, the moving object can be identified, and an excellent effect can be obtained.
[0118]
(Example 31)
(Moving object identification part)
The 31st embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the main difference is the operation in the moving object identification unit 8. Hereinafter, the operation of this embodiment will be described.
[0119]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 detects the movement vector of the moving object, as in the twenty-fifth embodiment. And the most frequent value x _v And y _v V (x _v , Y _v ) As a movement vector of the moving object currently focused on. Where vector group Vg (x _v ± dx, y _v The ratio of the number of feature points that generate movement vectors belonging to ± dy) to the total number of feature points is calculated including the feature points for which the movement vectors could not be detected. dx and dy are set in advance. When this ratio is equal to or greater than a predetermined threshold value, it is determined as a rigid body. The determination result is sent to the moving object identification unit 8, and the moving object identification unit 8 uses the determination result as an attribute of the moving object for the moving object tracking process.
[0120]
As described above, according to the present embodiment, the feature quantity detection unit 5 detects the movement vector of the moving object as the feature quantity of the moving object, and using this feature quantity, the moving object is a rigid body or a non-rigid body. By determining whether or not the moving object can be identified, an excellent effect can be obtained.
[0121]
(Example 32)
The thirty-second embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the main difference is the operation in the moving object identification unit 8. Hereinafter, the operation of this embodiment will be described.
[0122]
In FIG. 1, the flow of processing from the input unit 1 to the moving area shaping unit 4 is the same as that described in the first embodiment. For the moving region shaped by the moving region shaping unit 4, the feature amount detection unit 5 detects the movement vector of the moving object, as in the twenty-fifth embodiment. Then, it is determined whether the moving object is a rigid body or a non-rigid body by the same method as in the embodiment 31, and the determination result is sent to the time-series processing unit 7 and the moving object identification unit 8. When it is determined as a non-rigid body, the time-series processing unit 7 obtains the natural period of the temporal change in the shape feature amount of the moving object based on the history information. A sequence of feature quantities of 2 × N frames (N is a positive integer) is stored, and the feature quantity sequence of the previous N frame is shifted from 0 to N−1 frames, and the square of the difference for each feature quantity is stored. And the minimum value of the sum is calculated as the correlation value R using the following equation (11).
[0123]
[Equation 3]

[0124]
In order to simplify the processing, the correlation value R may be the sum of the absolute values of the feature value differences. When the derived correlation value R is smaller than a preset threshold value, it is determined that the feature quantity has periodicity, and the shift number K at that time is set as the characteristic period of the feature quantity. The determination result is sent to the moving object identification unit 8, and the moving object identification unit 8 uses the determination result as an attribute of the moving object for the moving object tracking process.
[0125]
As described above, according to the present embodiment, the feature quantity detection unit 5 detects the movement vector of the moving object as the feature quantity of the moving object, and using this feature quantity, the moving object is a rigid body or a non-rigid body. If it is determined that the object is non-rigid, it is possible to identify the moving object by detecting the natural period of the temporal change in the shape feature value of the moving object. Effect.
[0126]
(Example 33)
The thirty-third embodiment of the present invention will be described below. The moving image processing apparatus according to the present embodiment has the same configuration as that of the first embodiment shown in FIG. 1, and the difference is the operation after the moving object identification unit 8. Hereinafter, the operation of this embodiment will be described.
[0127]
In FIG. 1, the processing flow from the input unit 1 to the feature amount detection unit 5, the position detection unit 6, and the time series processing unit 7 is the same as that described in the first embodiment. These processing results are sent to the moving object identification unit 8. The moving object identification unit 8 integrates the sent information to identify the moving object. The identification processing is performed by feature amount evaluation and determination processing. The feature amount evaluation is based on the detection target based on the feature amount itself, the evaluation based on the distance between the current frame feature amount and the previous frame feature amount, the time series feature amount evaluation, and multiple moving objects using position information on the screen. We evaluate by special event recognition such as crossing above. For example, the feature amount detected in the current frame processing is expressed as T _c , T _p When the determination threshold value is α and the following equation (12) is not satisfied, the moving object identification unit 8 determines that the reliability of the feature amount is low.
｜ T _c -T _p | ≦ α (12)
[0128]
Next, determination processing using these evaluation results is performed. In this determination processing, a result that the characteristic amount evaluation result does not satisfy the determination condition due to occurrence of a special event such as a plurality of moving object intersections is allowed. When the occurrence of a special event is not detected and the feature quantity evaluation result is insufficient, a parameter change command is given to the feature quantity detection unit 5 or the position detection unit 6 corresponding to the feature quantity detection. Further, the moving object identification unit 8 detects an optimum parameter for detecting the moving object based on the feature amount for which a sufficiently satisfactory result is obtained, It can also be transmitted to the region shaping unit 4, the feature amount detection unit 5, and the position detection unit 6. For example, if the moving speed and direction of the moving object are calculated by integrating the feature amounts, the conditions are sent to the inter-image arithmetic processing unit 3 and the moving area shaping unit 4, and these processing units It is possible to perform inter-image calculation processing and dynamic region shaping processing at the most suitable time interval. It is also possible to dynamically determine the optimum parameter for noise removal processing from the size of the moving object and send it to the moving area shaping unit 4. Or a highly reliable time series feature T _p Can be sent to the feature amount detection unit 5 and the position detection unit 6, and each detection unit can reflect this information in its own processing. These processes are continuously performed on the moving image, and the same moving object is traced by determining the moving area determined to be the same moving object as the pre-processing screen as the same moving area.
[0129]
As described above, according to the present embodiment, the moving object identification unit 8 integrates the information sent from the feature amount detection unit 5, the position detection unit 6, and the time series processing unit 7 to identify the moving object. If the evaluation result is insufficient, the parameter is optimized by transmitting it to the inter-image calculation processing unit 3, the moving region shaping unit 4, the feature amount detection unit 5, and the position detection unit 6, and accurate An excellent effect is obtained in that identification of a moving object can be realized.
[0130]
【The invention's effect】
As described above, according to the present invention, by providing a feature amount detection unit that detects a feature amount of a moving object, the moving object can be identified and tracked by evaluating and determining the moving object based on the feature amount. In addition, when using the inter-frame difference processing, even when the moving object is stationary and the moving area cannot be detected, the moving object is identified by the feature amount when the moving area is detected after the operation is started again. By doing so, it is possible to identify and track the same object. Furthermore, since the moving object identification unit evaluates moving object detection using the feature value itself and the time-series feature value, it realizes accurate tracking of the moving object by absorbing steep changes due to noise in the feature value. If the moving object identification result is insufficient, the processing unit corresponding to the feature amount detection can be instructed to change the processing condition, and a sufficiently satisfactory result can be obtained. By detecting the optimum condition for detecting the moving object based on the feature amount, and transmitting it to the inter-image calculation processing unit, moving region shaping unit, feature amount detection unit, and position detection unit, An excellent moving image processing apparatus can be realized.
[Brief description of the drawings]
FIG. 1 is a block connection diagram of a moving image processing apparatus according to first to thirty-third embodiments of the present invention.
FIG. 2 is a conceptual diagram showing detection of a moving object from within a moving area in a second embodiment of the present invention.
FIG. 3 is a conceptual diagram showing a moving area shaping method according to a third embodiment of the present invention.
FIG. 4 is a frequency distribution diagram in the sixth embodiment of the present invention.
FIG. 5 is a conceptual diagram showing a method for determining position coordinates of a divided region in an eighth embodiment of the present invention.
FIG. 6A is a conceptual diagram showing a contour detection method in a twenty-fifth embodiment of the present invention.
(B) Frequency distribution of contour vector evaluation values in the 25th embodiment of the present invention
(C) Outline feature points in the twenty-fifth embodiment of the present invention
(D) Outline feature points in the twenty-fifth embodiment of the present invention
(E) Movement vector x component distribution in the twenty-fifth embodiment of the present invention
(F) Movement vector y component distribution in the twenty-fifth embodiment of the present invention
FIG. 7 is a conceptual diagram showing moving region position information in the twenty-sixth to twenty-eighth embodiments of the present invention.
FIG. 8 is a block connection diagram of a moving image processing apparatus according to a conventional embodiment.
[Explanation of symbols]
1 Image input section
2 Image storage
3 Image processing unit
4 dynamic area shaping part
5 Feature detection unit
6 Position detector
7 Time series processing section
8 Moving object identification unit
21 Moving region detected by the inter-image arithmetic processing unit
22 Detection target moving object area
31 Moving area and circumscribed rectangle before N frames (N is a positive integer)
32 Moving area of the current frame
33 Current frame motion area
34 31-33 circumscribed rectangles integrated
A state in which the current frame moving area is plotted in a circumscribed rectangle including 35 34
36 Shaped circumscribed rectangle
51 Dividing area of moving area
52 Dividing area of moving area
53 Dividing area of moving area
81 Image input unit
82 Image storage
83 Image pre-processing unit
84 Moving area shaping part
85 Moving object tracking unit

Claims (27)

An image input unit for inputting a moving image, an image storage unit for storing image data from the image input unit in units of frames, and inter-frame difference calculation processing for frame image data stored in the image storage unit An inter-image arithmetic processing unit to be performed; a moving region shaping unit that detects only a region where a moving object is present with respect to the moving region detected by the inter-image arithmetic processing unit; and a motion detected by the moving region shaping unit A feature amount detection unit that processes image data in a region to detect a feature amount of the moving object, and a position detection that detects a feature amount of the position of the moving object based on the moving region information detected by the moving region shaping unit A time series processing unit that calculates and updates a new time series feature quantity in the current frame from the feature quantity of the moving object in the current frame and the time series feature quantity calculated in the previous frame, and the feature quantity detection Parts and integrates the results of the time series feature amount calculated in the time-series processing section and the detected feature amounts by the position detecting unit and a moving object identification unit that performs identification of moving objects,
The moving object identification unit evaluates the feature amount of the moving object by integrating the information sent from the feature amount detection unit, the position detection unit, and the time series processing unit, and the evaluation result of the feature amount sets the determination threshold value. If not satisfied, a parameter change command is instructed to the feature value detection unit or the position detection unit corresponding to the feature value detection, and a feature value for which a result satisfying the determination threshold is obtained is the feature value. A moving image processing apparatus that detects an optimal parameter for detecting the moving object based on the image and transmits the detected parameter to the inter-image arithmetic processing unit, the moving region shaping unit, the feature amount detecting unit, and the position detecting unit .   The moving image processing apparatus according to claim 1, wherein the moving region shaping unit extracts a contour of a moving object by detecting an edge feature point of image data in the moving region, and performs moving region shaping from the contour extraction result.   The moving image processing apparatus according to claim 1, wherein the moving region shaping unit shapes the moving region detected by the inter-image arithmetic processing unit and the moving region N frames before to correct the moving region.   The moving image processing apparatus according to claim 1, wherein the feature amount detection unit detects luminance information of the image data in the moving region as a feature amount of the moving object.   The feature amount detection unit divides the moving region into M pieces (M is a positive integer) based on the luminance distribution of the image data in the moving region as the luminance information of the image data in the moving region, and the divided divided moving region The moving image processing apparatus according to claim 4, wherein luminance representing a representative value is detected as a feature amount of the moving object. The moving image processing apparatus according to claim 5 , wherein the feature amount detection unit detects a ratio of the sizes of the divided moving areas divided by the luminance as the feature amount of the moving object. The moving image processing apparatus according to claim 5, wherein the feature amount detection unit obtains each position information of the divided divided moving area and detects it as a feature amount of the moving object.   The moving image processing apparatus according to claim 1, wherein the feature amount detection unit detects color information of the image data in the moving region as a feature amount of the moving object. Feature amount detection unit, as the color information of the image data of the moving area, claim to detect a percentage of the total of the number of pixels less than a certain threshold saturation of moving area image data as a feature quantity of the moving object 8 The moving image processing apparatus described. The feature amount detection unit detects M moving areas based on the frequency distribution of hue values of pixels whose saturation of the image data in the moving area is equal to or greater than a threshold value as color information of the image data in the moving area (M is a positive value). The moving image processing apparatus according to claim 8 , wherein a hue value that is a representative value of the divided divided moving area is detected as a feature amount of the moving object. The moving image processing apparatus according to claim 10 , wherein the feature amount detection unit detects a ratio of the size of the divided moving area divided by the hue value as a feature amount of the moving object. The moving image processing apparatus according to claim 10, wherein the feature amount detection unit obtains each position information of the divided divided moving area and detects it as a feature amount of the moving object.   4. The moving image processing apparatus according to claim 1, wherein the feature amount detection unit obtains shape information of the image data in the moving region and detects it as a feature amount of the moving object. The moving image processing apparatus according to claim 13 , wherein the feature amount detection unit detects the number of pixels of the moving region as shape information of the moving object corresponding to the moving region, and uses the detected information as the feature amount of the moving object. The feature amount detection unit creates a circumscribed rectangle of the moving region, detects a ratio of the number of pixels of the moving region to the number of pixels of the circumscribed rectangle as shape information of the moving region image data, and sets it as a feature amount of the moving object. Item 14. A moving image processing apparatus according to Item 13 .   The moving image processing apparatus according to claim 1, wherein the feature amount detection unit obtains texture information of the image data in the moving region and detects it as a feature amount of the moving object. The feature amount detection unit obtains a luminance differential value in the moving region as texture information of the image data in the moving region, divides the moving region into M pieces (M is a positive integer) based on the distribution, and is divided The moving image processing apparatus according to claim 16 , wherein a luminance differential value that is a representative value of the divided moving area is detected as a feature amount of the moving object. The moving image processing apparatus according to claim 17 , wherein the feature amount detection unit detects a ratio of the sizes of the divided moving areas divided by the texture information as the feature amount of the moving object. The moving image processing apparatus according to claim 17, wherein the feature amount detection unit obtains each position information of the divided divided moving area and detects it as a feature amount of the moving object.   The moving image processing apparatus according to claim 1, wherein the feature amount detection unit includes a movement vector detection unit at a feature point of the moving object. Position detecting unit, the moving image processing device according to claim 1, the contact is detected and location information uppermost point of the moving object and the motion plane 20 either. Position detecting unit, the moving image processing apparatus according to claim 1, the position information by detecting a contact between the moving object and the motion plane 20. The position detection unit detects the contact point between the moving object and the motion plane and uses it as position information. 23. The moving image processing apparatus according to claim 22 , wherein the position of the contact point with the motion plane is obtained from the estimated height and used as position information. The time series processing unit is detected as the time series feature quantity of the feature quantity detected by each feature quantity detection unit in the feature quantity detected in the current frame process and the process before K frames (K is a positive integer). A new time-series feature value is calculated based on the time-series feature value of the absolute value of the difference from the feature value and the absolute value of the difference between frames calculated in the previous frame processing. moving image processing apparatus according to claim 1 which is calculated as series feature amounts 23. 21. The moving image processing apparatus according to claim 20 , wherein the moving object identification unit identifies whether the moving object is a rigid body or a non-rigid body based on a plurality of movement vectors in the moving area detected at the feature point of the moving object. 26. The moving image processing apparatus according to claim 25 , wherein the moving object identification unit extracts and identifies the feature of the motion of the non-rigid body based on the natural period of the temporal change in the shape feature amount of the non-rigid body. Moving object identification unit, in the case of a difference between the detected feature quantity and the time series feature amount determination threshold value or less in the current frame processing, to any one of claims 1 to feature amount is evaluated to be insufficient 26 The moving image processing apparatus described.

Images