JP2004030659A - Computer readable information recording medium with object detection program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect an object without being affected by illumination fluctuation by setting a reference point at an arbitrary position. <P>SOLUTION: A selection means for a plurality of reference points means selects a plurality of reference pixels for each concerned pixel of an image (steps A1-A3), and a background model creating means creates an illumination fluctuation background model when luminance of a background is changed due to the effect of the illumination fluctuation with respect to the concerned pixels and the reference pixels. In a luminance space, a distance calculating means calculates a distance between the illumination fluctuation background model and a point represented by couples of the concerned pixels and luminance values of background pixels corresponding to the reference pixels (step A4). Based upon the distance, an object discriminating means discriminates whether or not the concerned pixels represent the object (steps A5-A9). <P>COPYRIGHT: (C)2004,JPO

Description

【００４６】
【数３】

次に、物体判別手段１４は、ステップＡ４において求められた距離Ｄ_１、Ｄ_２と、あらかじめ設定されているしきい値Ｔ_１、Ｔ_２とを比べ（ステップＡ５）、これらの距離およびしきい値が［数４］を満たす場合には、注目画素ｐ_ａは物体を表す画素であり、対応箇所に物体が存在すると判断し、一方、［数４］を満たさない場合には、注目画素ｐ_ａは背景を表す画素であると判断する（ステップＡ６、Ｓ７、Ｓ８、Ｓ９）。
【００４７】
【数４】

より詳しくは、物体判別手段１４は、複数距離計算手段が計算した距離Ｄ_１、Ｄ_２がそれぞれ上記しきい値Ｔ_１、Ｔ_２より大きいか否かを調べる（ステップＡ５）。たとえば距離Ｄ_１がしきい値Ｔ_１より大きい場合には、物体判別手段１４は注目画素ｐ_ａおよび参照画素ｐ_ｂは物体を表す画素である可能性があるとし、検出結果として両画素に論理”１”を対応づける。また、距離Ｄ_２も、しきい値Ｔ_２より大きい場合には、物体判別手段１４は、注目画素ｐ_ａおよび参照画素ｐ_ｃは物体を表す画素である可能性があるとし、検出結果として両画素に論理”１”を対応づける。
一方、距離Ｄ_１、Ｄ_２がＴ_１、Ｔ_２より小さい場合には、各画素には論理”０”を対応づける。
【００４８】
その上で、物体判別手段１４は、これらの検出結果の論理積をとり（ステップＡ６）、そして値を判定して（ステップＡ７）、結果が論理”１”の場合は注目画素ｐ_ａは物体を表す画素であるとし（ステップＡ８）、論理”０”の場合は注目画素ｐ_ａは物体を表さず、背景を表す画素であるとする（ステップＡ９）。
たとえば、距離Ｄ_１がしきい値Ｔ_１より大きい場合、物体判別手段１４が各画素ｐ_ａ、ｐ_ｂ、ｐ_ｃに対応づける値は”１”、”１”、”０”となり、また、距離Ｄ_２がしきい値Ｔ_２より大きい場合、物体判別手段１４が各画素ｐ_ａ、ｐ_ｂ、ｐ_ｃに対応づける値は”１”、”０”、”１”となる。したがって、各画素ごとに、これらの値の論理積をとると、それぞれ”１”、”０”、”０”となり、画素ｐ_ａに対する論理値のみが”１”となる。したがって、物体判別手段１４は、注目画素ｐ_ａが物体を表す画素であると判断する。
【００４９】
一般に注目画素以外にＮ個の参照点をとった場合、各参照点ごとに距離Ｄ_ｉを計算し、各参照点ごとのしきい値Ｔ_ｉを用いて、［数５］が成立するか否かにより注目画素が物体を表す画素であるか否かを判別することができる。
【００５０】
【数５】

このようなステップＡ２からＳ９までの処理を、画像入力手段４が保持している全画素を次々に注目画素に設定して行うことで（ステップＡ１０）、撮影画像上の物体を検出することができる。
検出された物体は、画像出力手段１５により出力され、表示装置に表示される。その際、ステップＡ７、Ａ８において、物体を表すと判別された注目画素の表示色を、背景とは異なる色で表示すれば、画面上で検出物体の形状などをより明瞭に把握することができる。
【００５１】
なお、ステップＡ１０において、すべての画素について処理が終わったと判断された場合に、物体であると判別された画素の数をカウントして、このカウント値が予め設定されたしきい値より大きい場合にのみ物体が存在すると判断するようにしてもよい。これによって、ある大きさ以上の物体のみを検出することが可能となる。
【００５２】
一方、ステップＡ７、Ａ９で注目画素が物体を表す画素ではないと判別された場合には、背景作成手段１６は、同注目画素および対応する複数の参照画素の輝度値にもとづいて、対応する背景画像の画素の輝度値を更新し、背景画像保持手段８に保持させる（ステップＡ３）。
なお、最初の入力画像に移動物体が含まれていなければ、基本的に背景画像の各画素の輝度値は更新する必要はない。しかし、ノイズ等の影響で、背景領域の輝度値が照明変動以外の要因でばらつきを起こしうるため、物体判別手段１４により背景とみなされた画素において随時輝度値を更新していくことで誤差を抑えることができる。
【００５３】
また、このような背景画像の更新において、新旧の輝度値に重みを乗じて徐々に更新するという方法をとることができる。すなわち、物体判別手段１４で注目画素ｐ_ａが背景画素とみなされた場合、［数６］のように新しく加わるデータに重みをかけ徐々に更新することができる。
【００５４】
【数６】

ここで、Ｐ_Ａ＿ｎｅｗは注目画素ｐ_ａと参照画素ｐ_ｂ、ｐ_ｃに対応する背景画素の輝度値による３次元輝度値行列、Ｐ_Ａ＿ｏｌｄは今までの注目画素ｐ_ａと参照画素ｐ_ｂ、ｐ_ｃに対応する背景画素の輝度値による３次元背景輝度行列である。また、Ｉ_Ａは、注目画素ｐ_ａと参照画素ｐ_ｂ、ｐ_ｃの輝度値行列、αは上記重み定数である。
【００５５】
次に、ステップＡ１０において、すべての画素に対して前述した判別処理が終了したと判断された場合、画像入力手段４は、次の画像があるか否かを判断し（ステップＡ１１）、次の画像がある場合はステップＡ１に戻って、画像を取得し、以降、上述した各処理が繰り返される。そして、すべての画像に対して処理が終了した時点で、物体検出処理を終了する。
以上の処理を監視カメラから画像が取り込まれるごとに行うことで、物体検出を逐次行うことができる。
【００５６】
図４の（Ａ）は、道路を黒い車が走っている原画像を示す模式図、（Ｂ）は従来技術による車の検出結果を示す説明図、（Ｃ）は本実施の形態例による車の検出結果を示す説明図である。
図４の（Ｂ）は、図４の（Ａ）に示した原画像および背景画像により図８などを用いて説明した従来技術にもとづき車の検出を行った結果を示し、注目点および参照点を隣接させて物体検出を行ったため、車の輪郭部のみが検出されている。
【００５７】
これに対して、本実施の形態例による検出結果である図４の（Ｃ）では、車の輪郭部だけでなく、輪郭の内側も含め、車全体が検出されており、より明瞭に、したがって確実に車が検出されている。
本実施の形態例では、上述したように、各注目画素ごとに選択する参照画素を、注目画素に対し、物体の大きさ以上に離して設定する。したがって、注目画素と参照画素とを接近させて設定した場合のように物体の輪郭部のみが検出されるといったことがない。
そして、１つの注目画素に対して複数の参照画素を設定し、注目画素と各参照画素との組ごとに物体検出を行って、最終的に論理演算により注目画素が物体の画素か否かを判別するので、注目画素と参照画素とを離して設定しても、物体の位置を確実に特定できる。
【００５８】
また、上記従来技術と同様、輝度空間における照明変動背景モデルと輝度点との距離にもとづいて注目画素が画像を表すか否かを判定しているので、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体を検出することができる。
【００５９】
次に、本発明の第２の実施形態について説明する。
図５は第２の実施の形態例の物体検出装置２を示すブロック図、図６は図５の物体検出装置の動作を示すフローチャートである。以下では、これらの図面を参照して本発明の第２の実施の形態例の物体検出装置について説明すると同時に本発明の第２の実施の形態例の物体検出方法および物体検出プログラムを記録したコンピュータ読み取り可能な情報記録媒体について説明する。なお、図５において図１と同一の要素には同一の符号が付されており、それらに関する説明はここでは省略する。
【００６０】
図５に示した物体検出装置１８が上記物体検出装置２と異なるのは、物体自体を検出するのではなく物体の外接矩形を求める点であり、その結果、少ない計算量で物体を検出することが可能となる。
物体検出装置１８は、画像入力手段４、本発明に係わる水平参照点選択手段および垂直参照点選択手段として機能する複数参照点選択手段２０、背景画像保持手段８、本発明に係わる水平背景モデル作成手段および垂直背景モデル作成手段として機能する背景モデル作成手段２２、本発明に係わる水平距離計算手段および垂直距離計算手段として機能する距離計算手段２４、物体判別手段２６、背景作成手段２８、ならびに画像出力手段１５を含み、たとえば監視カメラにより撮影した画像から物体を検出する構成となっている。
【００６１】
物体検出装置１８は上記物体検出装置２と同様、具体的には、一例として監視カメラやビデオテープレコーダとのインターフェースなども備えたコンピュータにより構成されており、コンピュータを構成する入力装置に、本発明の物体検出プログラムを記録したコンピュータ読み取り可能な情報記録媒体としてのたとえばＣＤ−ＲＯＭを装着してＣＤ−ＲＯＭからプログラムデータを読み込み、コンピュータのメモリにロードして、コンピュータのＣＰＵを同プログラムデータにもとづいて動作させることで実現されている。
【００６２】
物体検出装置１８の複数参照点選択手段２０は、水平参照点選択手段として機能して、撮影画像において水平方向に配列された複数の画素から成る注目水平画素列ごとに、注目水平画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段４から取得する。
また、複数参照点選択手段２０は、垂直参照点選択手段として機能して、撮影画像において垂直方向に配列された複数の画素から成る注目垂直画素列ごとに、注目垂直画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段４から取得する。
【００６３】
そして、背景モデル作成手段２２は、水平背景モデル作成手段として機能して、水平参照点選択手段が選択した複数の参照画素にそれぞれ対応する、背景画像における複数の画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
また、背景モデル作成手段２２は、垂直背景モデル作成手段として機能して、垂直参照点選択手段が選択した複数の参照画素にそれぞれ対応する、背景画像における複数の画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
【００６４】
複数距離計算手段２は、水平距離計算手段として機能して、多次元輝度空間において、水平背景モデル作成手段としての背景モデル作成手段２２が作成した前記照明変動背景モデルと、水平参照点選択手段としての複数参照点選択手段２０が取得した複数の輝度値が表す点との間の距離を計算する。
また、複数距離計算手段２は、垂直距離計算手段として機能して、多次元輝度空間において、垂直背景モデル作成手段としての背景モデル作成手段２２が作成した前記照明変動背景モデルと、垂直参照点選択手段としての複数参照点選択手段２０が取得した前記複数の輝度値が表す点との間の距離を計算する。
【００６５】
物体判別手段２６は、上記水平距離計算手段および垂直距離計算手段としての複数距離計算手段２が計算した前記距離にもとづいて、注目水平画素列および注目垂直画素列に物体を表す画素が含まれているか否かを判別し、両画素列に係わる判別結果に対して論理演算を行って物体を含む矩形領域を特定する。
背景作成手段２８は、物体判別手段２６が物体を表す画素を含まないと判別した前記注目水平画素列および注目垂直画素列を構成する各画素の輝度値にもとづいて、対応する前記背景画像の画素の輝度値を更新して背景画像保持手段８に保持させる。
【００６６】
次に、このように構成された物体検出装置１８の動作について説明する。
図７は第２の実施の形態例の動作を説明するための図であって、（Ａ）は垂直方向における物体の存在可能領域の検出を表す説明図、（Ｂ）は水平方向における物体の存在可能領域の検出を表す説明図、（Ｃ）は検出結果を示す説明図である。以下では、図５とともに図６、図７を参照して説明する。
【００６７】
図６に示したように、まず、画像入力手段４は、監視カメラなどによって撮像された画像を読み込み、画像を構成する各画素ごとの輝度値を画像入力手段４内に設けられた不図示のメモリに保持する（ステップＢ１）。
次に、複数参照点選択手段２０は、撮影画像において垂直方向のアドレスｙにおける１行分の画素を注目水平画素列とし、またこの１行分の画素を参照画素（参照画素群）に選択して、各参照画素の輝度値を画像入力手段４から取得する（ステップＢ４）。したがって、撮影画像の幅がＷｉｄｔｈ画素（Ｗｉｄｔｈは正の整数）であったとすると、上記注目水平画素列にはＷｉｄｔｈ個の画素が含まれ、複数参照点選択手段２０は、本実施の形態例ではこれらすべて画素を参照画素とする。
【００６８】
そして、背景モデル作成手段２２は、複数参照点選択手段２０が選択した上記参照画素にそれぞれ対応する、背景画像における各画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
その後、複数距離計算手段２４は、参照画素の輝度値によって形成される多次元輝度空間において、背景モデル作成手段２２が作成した前記照明変動背景モデルと、複数参照点選択手段２０が取得した上記複数の輝度値が表す点との間の距離を計算する（ステップＢ６）。
【００６９】
つづいて、物体判別手段２６は、複数距離計算手段２４がこのように計算した距離をしきい値と比較して、距離がしきい値より大きい場合には、上記注目水平画素列に対応づけて論理”１”を保持し、距離がしきい値より小さい場合には論理”０”を保持する（ステップＢ８）。
複数参照点選択手段２０、背景モデル作成手段２２、ならびに複数距離計算手段２は、このような処理を、アドレスｙの値を１ずつ大きくしながら、画像の垂直方向の全範囲において行う（ステップＢ１０）。
その結果、図７の（Ａ）に示したように、物体１０８が図のように存在する場合、Ｈの範囲内の各注目水平画素列に対して、物体判別手段２６は論理”１”を保持することになる。
【００７０】
また、複数参照点選択手段２０、背景モデル作成手段２２、ならびに複数距離計算手段２は、同様の処理を、画像の水平方向においても行う。
すなわち、複数参照点選択手段２０は、撮影画像において水平方向のアドレスｘにおける１列分の画素を注目垂直画素列とし、またこの１列分の画素を参照画素（参照画素群）に選択して、各参照画素の輝度値を画像入力手段４から取得する（ステップＢ２）。したがって、撮影画像の高さがＨｅｉｇｈｔ画素（Ｈｅｉｇｈｔは正の整数）であったとすると、上記注目垂直画素列にはＨｅｉｇｈｔ個の画素が含まれ、複数参照点選択手段２０は、本実施の形態例ではこれらすべて画素を参照画素とする。
【００７１】
そして、背景モデル作成手段２２は、複数参照点選択手段２０が選択した上記参照画素にそれぞれ対応する、背景画像における各画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
その後、複数距離計算手段２４は、参照画素の輝度値によって形成される多次元輝度空間において、背景モデル作成手段２２が作成した前記照明変動背景モデルと、複数参照点選択手段２０が取得した上記複数の輝度値が表す点との間の距離を計算する（ステップＢ５）。
【００７２】
つづいて、物体判別手段２６は、複数距離計算手段２がこのように計算した距離をしきい値と比較して、距離がしきい値より大きい場合には、上記注目垂直画素列に対応づけて論理”１”を保持し、距離がしきい値より小さい場合には論理”０”を保持する（ステップＢ７）。
複数参照点選択手段２０、背景モデル作成手段２２、ならびに複数距離計算手段２４は、このような処理を、アドレスｙの値を１ずつ大きくしながら、画像の水平方向の全範囲において行う（ステップＢ９）。
その結果、図７の（Ｂ）に示したように、Ｗの範囲内の各注目垂直画素列に対して、物体判別手段２６は論理”１”を保持することになる。
【００７３】
その後、物体判別手段２６は、各注目水平画素列および各注目垂直画素列ごとに保持している論理値の論理積をとることで（ステップＢ１１）、検出結果としての矩形領域の画像を保持するためのメモリ（図示せず）において、各画素ごとに、その値が論理”１”か論理”０”かを決定する。
詳しくは、値を決定すべき画素に対応する位置の撮影画像の画素を含む注目水平画素列および注目垂直画素列に対して、共に論理”１”を保持している場合には、上記画素の値は論理”１”に設定する。一方、対応する撮影画像の画素を含む注目水平画素列および注目垂直画素列の少なくとも一方に対して論理”０”を保持している場合には、上記画素の値は論理”０”に設定する（ステップＢ１２、Ｂ１３、Ｂ１４）。その結果、検出結果を保持する上記メモリにおいて、図７の（Ｃ）に示したように、物体を含む可能性のある矩形領域Ｒ内の画素の値は論理”１”となり、それ以外の画素の値は論理”０”となる。
【００７４】
したがって、このメモリの内容を画像出力手段１５により表示装置に表示すれば、物体を含む矩形領域が画面に表示されることになる。また、矩形領域Ｒの４辺は物体１０８の外接矩形となっている。
そして、撮影画像の幅がＷｉｄｔｈ画素、高さがＨｅｉｇｈｔ画素であった場合、上記第１の実施の形態例では、各画素ごとに背景モデルの作成や距離の計算を行うので、このような演算処理を（Ｗｉｄｔｈ×Ｈｅｉｇｈ）回行う必要がある。
しかし、第２の実施の形態例では、各水平画素列ごとに上記演算処理を行い、また各垂直画素列ごとに上記演算処理を行えばよいので、演算処理の回数は全体で（Ｗｉｄｔｈ＋Ｈｅｉｇｈｔ）回となる。したがって、演算量は第１の実施の形態例の場合にくらべ大幅に低減し、高速に物体検出を行ったり、装置の低コスト化を図る場合に非常に有利である。
【００７５】
なお、物体検出装置２の場合と同様に、背景作成手段２８は、物体判別手段２６が物体を表す画素を含まないと判別した注目水平画素列および注目垂直画素列を構成する各画素の輝度値にもとづいて、対応する背景画像の画素の輝度値を更新して背景画像保持手段８に保持させる。これにより最新の背景画像を用いて精度の高い物体検出を行うことができる。
また、処理すべき画像がさらに存在する場合には、物体検出装置２の場合と同様、以上の手順が次の画像に対して繰り返される（ステップＢ１５）。
【００７６】
なお、この第２の実施の形態例では、注目水平画素列および注目垂直画素列において、各画素列のすべての画素を参照画素にするとしたが、数画素ごとに選択するなどの間引きを行って、さらに演算量の低減を図ることも可能である。
さらに、たとえば水平方向および垂直方向のいずれか一方でまず物体の存在可能領域を確定し、その上で、その領域内でのみ他方の方向における処理を行えば、物体が存在する可能性のない領域では演算処理を行わなくても済むため、さらに演算量を削減することが可能となる。
【００７７】
なお、ここでは物体検出装置２および物体検出装置１８はコンピュータにより構成されているとしたが、物体検出装置２および物体検出装置１８を専用のハードウェアにより構成することも無論可能である。また、本発明に係わる機能のうち、一部の機能をコンピュータに所定のプログラムをロードして実現し、他の機能は専用のハードウェアによって実現したり、さらには、コンピュータにすでに組み込まれているプログラムを組み合わせて用いることで物体検出装置２および物体検出装置１８を構成とすることも可能である。
【００７８】
また、コンピュータ読み取り可能な情報記録媒体とは、上述したＣＤ−ＲＯＭに限らず、フロッピー（登録商標）ディスク、光磁気ディスク、ＲＯＭなどの可搬媒体や、コンピュータに内蔵されるハードディスクなどの記憶装置を含んでいる。
さらに、コンピュータ読み取り可能な情報記録媒体は、インターネットなどの通信ネットワークや電話回線などの通信回線を介してプログラムを送信する場合の通信線のように、短時間の間、動的にプログラムを保持するものも含み、その場合のサーバやクライアントとなるコンピュータシステム内部の揮発性メモリのように、一定時間プログラムを保持するものも含んでいる。
【００７９】
【発明の効果】
以上説明したように本発明の物体検出装置では、画像入力手段は、撮影画像の各画素の輝度値を取り込んで保持し、複数参照点選択手段は、前記画像の注目画素ごとに複数の参照画素を選択して、選択した複数の参照画素および前記注目画素の輝度値を画像入力手段から注目画素ごとに取得する。そして、背景モデル作成手段は、複数参照点選択手段が選択した注目画素および同画素に対応する前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を背景画像保持手段から取得して、背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを注目画素と、同画素に対応する各参照画素との組ごとに作成する。
【００８０】
その上で、距離計算手段は、輝度空間において、背景モデル作成手段が注目画素と参照画素との前記組ごとに作成した前記照明変動背景モデルと、複数参照点選択手段が注目画素ごとに取得した前記複数の参照画素の輝度値のそれぞれと注目画素の輝度値との組が表す点との間の距離をそれぞれ計算する。
そして、物体判別手段は、注目画素と各参照画素との組ごとに、距離計算手段が計算した前記距離にもとづいて前記注目画素が物体を表す画素か否かを判別し、前記組ごとの判別結果に対し論理演算を行って注目画素が物体を表す画素か否かを最終的に判別する。
【００８１】
このように、本発明の物体検出装置では、１つの注目画素に対して複数の参照画素を設定し、注目画素と各参照画素との組ごとに物体検出を行って、最終的に論理演算により注目画素が物体の画素か否かを判別するので、注目画素と参照画素とを離して設定しても、物体の位置を確実に特定できる。
したがって、各注目画素ごとに選択する参照画素を、注目画素に対し、物体の大きさ以上に離して設定することができ、その結果、注目画素と参照画素とを接近させて設定した場合のように物体の輪郭部のみが検出されるといったことがない。
また、上述した従来技術と同様、輝度空間における照明変動背景モデルと輝度点との距離にもとづいて注目画素が画像を表すか否かを判定するので、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体を検出することができる。
【００８２】
また、本発明の物体検出装置では、画像入力手段は撮影画像の各画素の輝度値を取り込んで保持し、水平参照点選択手段は、撮影画像において水平方向に配列された複数の画素から成る注目水平画素列ごとに、注目水平画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段から取得する。また、垂直参照点選択手段は、撮影画像において垂直方向に配列された複数の画素から成る注目垂直画素列ごとに、注目垂直画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段から取得する。
そして、水平背景モデル作成手段は、水平参照点選択手段が選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を背景画像保持手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成し、垂直背景モデル作成手段は、垂直参照点選択手段が選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を背景画像保持手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
【００８３】
その後、水平距離計算手段は、多次元輝度空間において、水平背景モデル作成手段が作成した前記照明変動背景モデルと、水平参照点選択手段が取得した前記複数の輝度値が表す点との間の距離を計算し、垂直距離計算手段は、多次元輝度空間において、垂直背景モデル作成手段が作成した前記照明変動背景モデルと、垂直参照点選択手段が取得した前記複数の輝度値が表す点との間の距離を計算する。
そして、物体判別手段は、水平距離計算手段および垂直距離計算手段が計算した前記距離にもとづいて、注目水平画素列および注目垂直画素列に物体を表す画素が含まれているか否かを判別し、両画素列に係わる判別結果に対して論理演算を行って物体を含む矩形領域を特定する。
【００８４】
本発明の物体検出装置では、各水平画素列ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行い、また各垂直画素列ごとに同様の演算処理を行えばよいので、演算処理の回数は全体で画像の水平方向の画素数に、画像の垂直方向の画素数を加えた値となる。
一方、各画素ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行った場合には、全体の演算処理の回数は、画像の水平方向の画素数に、画像の垂直方向の画素数を乗じたものとなる。
したがって、本発明では、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体の存在領域を検出できる上に、演算量が大幅に低減し、その結果、高速に物体検出を行ったり、装置の低コスト化を図る上で非常に有利となる。
【００８５】
本発明の物体検出方法および本発明のコンピュータ読み取り可能な情報記録媒体に記録された物体検出プログラムでは、画像入力ステップにおいて、撮影画像の各画素の輝度値を取り込んで第１の記憶手段に保持させ、複数参照点選択ステップでは、前記画像の注目画素ごとに複数の参照画素を選択して、選択した複数の参照画素および前記注目画素の輝度値を第１の記憶手段から注目画素ごとに取得する。そして、背景モデル作成ステップでは、複数参照点選択ステップで選択した注目画素および同画素に対応する前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を第２の記憶手段から取得して、背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを注目画素と、同画素に対応する各参照画素との組ごとに作成する。
【００８６】
その上で、距離計算ステップでは、輝度空間において、背景モデル作成ステップで注目画素と参照画素との前記組ごとに作成した前記照明変動背景モデルと、複数参照点選択ステップで注目画素ごとに取得した前記複数の参照画素の輝度値のそれぞれと注目画素の輝度値との組が表す点との間の距離をそれぞれ計算する。
そして、物体判別ステップでは、注目画素と各参照画素との組ごとに、距離計算ステップで計算した前記距離にもとづいて前記注目画素が物体を表す画素か否かを判別し、前記組ごとの判別結果に対し論理演算を行って注目画素が物体を表す画素か否かを最終的に判別する。
【００８７】
このように本発明では、１つの注目画素に対して複数の参照画素を設定し、注目画素と各参照画素との組ごとに物体検出を行って、最終的に論理演算により注目画素が物体の画素か否かを判別するので、注目画素と参照画素とを離して設定しても、物体の位置を確実に特定できる。
したがって、各注目画素ごとに選択する参照画素を、注目画素に対し、物体の大きさ以上に離して設定することができ、その結果、注目画素と参照画素とを接近させて設定した場合のように物体の輪郭部のみが検出されるといったことがない。
また、上述した従来技術と同様、輝度空間における照明変動背景モデルと輝度点との距離にもとづいて注目画素が画像を表すか否かを判定するので、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体を検出することができる。
【００８８】
また、本発明の物体検出方法および本発明のコンピュータ読み取り可能な情報記録媒体に記録された物体検出プログラムでは、画像入力ステップにおいて撮影画像の各画素の輝度値を取り込んで第１の記憶手段に保持させ、水平参照点選択ステップでは、撮影画像において水平方向に配列された複数の画素から成る注目水平画素列ごとに、注目水平画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を第１の記憶手段から取得する。また、垂直参照点選択ステップでは、撮影画像において垂直方向に配列された複数の画素から成る注目垂直画素列ごとに、注目垂直画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を第１の記憶手段から取得する。
そして、水平背景モデル作成ステップでは、水平参照点選択ステップで選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を第１の記憶手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成し、垂直背景モデル作成ステップでは、垂直参照点選択ステップで選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を第２の記憶手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
【００８９】
その後、水平距離計算ステップでは、多次元輝度空間において、水平背景モデル作成ステップで作成した前記照明変動背景モデルと、水平参照点選択ステップで取得した前記複数の輝度値が表す点との間の距離を計算し、垂直距離計算ステップでは、多次元輝度空間において、垂直背景モデル作成ステップで作成した前記照明変動背景モデルと、垂直参照点選択ステップで取得した前記複数の輝度値が表す点との間の距離を計算する。
そして、物体判別ステップでは、水平距離計算ステップおよび垂直距離計算ステップで計算した前記距離にもとづいて、注目水平画素列および注目垂直画素列に物体を表す画素が含まれているか否かを判別し、両画素列に係わる判別結果に対して論理演算を行って物体を含む矩形領域を特定する。
【００９０】
このように、本発明では、各水平画素列ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行い、また各垂直画素列ごとに同様の演算処理を行えばよいので、演算処理の回数は全体で画像の水平方向の画素数に、画像の垂直方向の画素数を加えた値となる。
一方、各画素ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行った場合には、全体の演算処理の回数は、画像の水平方向の画素数に、画像の垂直方向の画素数を乗じたものとなる。
したがて、本発明では、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体の存在領域を検出できる上に、演算量が大幅に低減し、その結果、高速に物体検出を行ったり、装置の低コスト化を図る上で非常に有利となる。
【図面の簡単な説明】
【図１】第１の実施の形態例の物体検出装置を示すブロック図である。
【図２】図１の物体検出装置の動作を示すフローチャートである。
【図３】（Ａ）は画像面における注目点と参照点との位置関係を示す説明図、（Ｂ）は物体が撮影領域に侵入した場合の模式図、（Ｃ）は検出結果を示す模式図、（Ｄ）は画像面における注目点と参照点との位置関係を示す他の説明図、（Ｅ）は検出結果を示す他の模式図、（Ｆ）は最終的な検出結果を示す模式図である。
【図４】（Ａ）は、道路を黒い車が走っている原画像を示す模式図、（Ｂ）は従来技術による車の検出結果を示す説明図、（Ｃ）は本実施の形態例による車の検出結果を示す説明図である。
【図５】第２の実施の形態例の物体検出装置を示すブロック図である。
【図６】図５の物体検出装置の動作を示すフローチャートである。
【図７】第２の実施の形態例の動作を説明するための図であって、（Ａ）は垂直方向における物体の存在可能領域の検出を表す説明図、（Ｂ）は水平方向における物体の存在可能領域の検出を表す説明図、（Ｃ）は検出結果を示す説明図である。
【図８】背景画像を表す説明図である。
【図９】注目点および参照点の輝度値により形成される２次元の輝度空間を示す説明図である。
【図１０】監視カメラの撮影領域に物体が侵入した場合の撮影画像を示す説明図である。
【図１１】（Ａ）は画像面を示す説明図、（Ｂ）は物体が撮影領域に侵入した場合の模式図、（Ｃ）は検出結果を示す模式図である。
【符号の説明】
２……物体検出装置、４……画像入力手段、６……複数参照点選択手段、８……背景画像保持手段、１０……背景モデル作成手段、１２……距離計算手段、１４……物体判別手段、１６……背景作成手段、１８……物体検出装置、２０……複数参照点選択手段、２２……背景モデル作成手段、２４……距離計算手段、２６……物体判別手段、２８……背景作成手段、１０２……白ドット、１０４……直線、１０６……白丸ドット、１０８……物体、１１０……黒ドット。[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a computer-readable information recording medium that records an object detection program for detecting an object included in an image captured by a monitoring camera or the like.
[0002]
[Prior art]
When detecting an object from a moving image captured by a surveillance camera or the like, it is important that the object can be reliably detected even if the brightness of the background fluctuates due to sunshine conditions or the like. Therefore, the inventors of the present invention have devised an object detection technique which is hardly affected by a large and frequently occurring illumination fluctuation, and has already filed a patent application (Japanese Patent Application No. 11-078641). ).
[0003]
In this object detection technique, a reference point is provided at a position different from a point of interest in a background image created in advance and a current image, and an object is determined from correlation information of a change in luminance value of the reference point. . Hereinafter, the principle will be described in detail with reference to the drawings.
FIG. 8 is an explanatory diagram showing a background image, in which the X axis is set in the horizontal direction and the Y axis is set in the vertical direction. However, in order to avoid being unnecessarily complicated, it is assumed that the background image has no texture and has constant brightness regardless of the location. In FIG. 8, P _A Is the point of interest, P _B Is a reference point provided in the vicinity. And the attention point P _A The luminance value at _A , Reference point P _B The luminance value at _B And
[0004]
FIG. 9 is an explanatory diagram showing a two-dimensional luminance space formed by the luminance values of the target point and the reference point. In this luminance space, a two-dimensional coordinate system is set, and the horizontal axis is the point of interest P _A Brightness value I _A And the vertical axis represents the reference point P _B Brightness value I _B Is represented.
Here, attention point P _A Brightness value of I _A1 , Reference point P _B Brightness value of I _B1 , The point in the luminance space corresponding to the set of luminance values, that is, the luminance point (I _A1 , I _B1 ) Is, for example, the position of the white dot 102 in FIG.
Then, the brightness of the background changes, and the attention point P _A If the luminance value of the reference point P decreases, for example, _B Are also reduced at the same rate. In addition, attention point P _A Is 0, the reference point P _B Is also 0. Therefore, the point of interest P _A And reference point P _B Brightness I _A , I _B The points in the luminance space corresponding to the set move along the straight line 104 shown in FIG. 9 when the brightness of the background changes. Here, such a straight line is called an illumination variation background model.
[0005]
On the other hand, for example, attention point P _A Of the target point P _A Brightness value of I _A2 , The luminance point (I _A2 , I _B1 ) Is the position of the white circle dot 106, for example. Therefore, the distance 107 between such a luminance point and the straight line 104 is checked, and when the luminance point is apart from the straight line 104, it can be determined that an object has entered the position of the point of interest.
Then, illumination fluctuation occurs, for example, the reference point P _B Of the target point P on the object, _A Of the luminance point (I _A2 , I _B1 ) Moves in the upper right direction (arrow A) substantially parallel to the straight line 104, and the distance between the straight line 104 and the straight line 104 does not change much. Therefore, the object can be reliably detected even if the illumination fluctuation occurs.
[0006]
[Problems to be solved by the invention]
However, in the above conventional technology, when the reference point is arranged close to the point of interest, only the outline of the object is extracted, and the extracted object image is in a hollow state, and there is room for improvement in this point. Was.
Hereinafter, a specific description will be given with reference to FIG. FIG. 10 is an explanatory diagram illustrating a captured image when an object enters a capturing area of a monitoring camera, for example. The background is assumed to have a constant brightness regardless of the location, as in the case of FIG.
For this image, as shown in FIG. _A And set the reference point P _B Is set in the state of FIG. 10, all of these points are on the object 108, and the brightness of both points is reduced to the same extent. _A And reference point P _B Corresponds to the luminance point (I _A , I _B ) Is the position of the black dot 110 on the straight line 104 shown in FIG. Since the distance between the black dot 110 and the straight line 104 is zero, in this case, the attention point P _A Is determined not to be on the object, and the attention point P _A Cannot be detected even though is on the object 108.
[0007]
On the other hand, attention point P _A And reference point P _B Are set at the same interval, for example, at the point of interest P _A Is on the object 108 and the reference point P _B When each point is set at a position where is on the background, there is a luminance difference between the two points, and the corresponding luminance point in FIG. .
Therefore, in the technique of Japanese Patent Application No. 11-078641, when the point of interest and the reference line are arranged close to each other, only the edge of the object, that is, the outline is detected.
[0008]
Therefore, in order to detect not only the outline of the object but also the entire object, it is necessary to arrange the reference point sufficiently away from the point of interest.
In the technique disclosed in Japanese Patent Application No. 11-078641, the reference point must be set outside the area where an object may enter. This will be described with reference to FIG. FIG. 11A is an explanatory diagram showing an image plane, FIG. 11B is a schematic diagram when an object has entered a shooting area, and FIG. 11C is a schematic diagram showing a detection result.
[0009]
As shown in FIG. 11A, the point of interest P _A And attention point P _B Are set apart from each other, as shown in FIG. _A At the point of interest, the point of interest P _A And reference point P _B And the object 108 is detected. However, if object 108 is at reference point P _B Since the same luminance difference occurs when the camera enters the position, the object 108 is detected also in this case.
Therefore, the point of interest P _A When the object 108 enters and is detected, the object 108 _A Or the reference point P _B Cannot be specified at the position of the target point P. _A 11C, the object image 109 is generated such that the object 108 exists at both positions, as shown in FIG.
[0010]
Therefore, in order to avoid such a phenomenon, the reference point must always be set outside a region where an object may enter.
The present invention has been made in order to eliminate such a drawback, and has as its object to be able to detect an object with high accuracy, to be able to detect an object with high accuracy, to be stable even when it frequently occurs, and to set a reference point. It is an object of the present invention to provide an object detection device, an object detection method, and a computer-readable information recording medium on which an object detection program is recorded, which can be set at an arbitrary position to reliably detect an object.
[0011]
[Means for Solving the Problems]
The present invention, in order to achieve the above object, is an object detection device that detects an object using a captured image, an image input unit that captures and holds the luminance value of each pixel of the captured image, Selecting a plurality of reference pixels for each pixel of interest, a plurality of reference point selection means for obtaining the luminance values of the selected plurality of reference pixels and the pixel of interest from the image input means for each pixel of interest, Background image holding means for holding a luminance value of each pixel of the background image forming the background; and the background corresponding to the target pixel selected by the multiple reference point selecting means and the plurality of reference pixels corresponding to the same pixel, respectively. The luminance value of a plurality of pixels in an image is obtained from the background image holding unit, and the illumination variation background model when the luminance of the background changes under the influence of the illumination variation is defined as the pixel of interest. A background model creating means created for each set of reference pixels corresponding to the illumination variation background model created for each set of the target pixel and the reference pixel by the background model creating means in a luminance space; Distance calculating means for calculating a distance between each of the plurality of reference pixel luminance values obtained for each of the pixels of interest and a point represented by a set of the luminance value of the pixel of interest; Determining, for each set of the target pixel and each reference pixel, whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculating means; An object discriminating means for performing an operation to finally determine whether or not the pixel of interest is a pixel representing an object.
[0012]
In the object detection device of the present invention, the image input unit captures and holds the luminance value of each pixel of the captured image, and the multiple reference point selection unit selects a plurality of reference pixels for each pixel of interest in the image, The luminance values of the selected plurality of reference pixels and the target pixel are acquired from the image input means for each target pixel. Then, the background model creating means, from the background image holding means, stores the luminance values of the plurality of pixels in the background image corresponding to the pixel of interest selected by the plurality of reference point selecting means and the plurality of reference pixels corresponding to the same pixel. Then, an illumination variation background model when the luminance of the background changes under the influence of the illumination variation is created for each pair of the target pixel and each reference pixel corresponding to the pixel.
[0013]
Then, the distance calculation unit acquires, in the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel by the background model creation unit, and the multiple reference point selection unit acquires for each attention pixel. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
Then, the object determination unit determines, for each pair of the target pixel and each reference pixel, whether or not the pixel of interest is a pixel representing an object based on the distance calculated by the distance calculation unit. A logical operation is performed on the result to finally determine whether or not the pixel of interest is a pixel representing the object.
[0014]
Further, the present invention is an object detection device that detects an object using a captured image, wherein the image input means captures and holds the luminance value of each pixel of the captured image, and is arranged in the horizontal direction in the captured image. Horizontal reference point selecting means for selecting a plurality of reference pixels included in the noted horizontal pixel row and acquiring a luminance value of each reference pixel from the image input means for each noted horizontal pixel row including the plurality of pixels, For each noted vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the noted vertical pixel column are selected, and the luminance value of each reference pixel is determined from the image input unit. Vertical reference point selecting means for acquiring, background image holding means for holding the luminance value of each pixel of the image forming the background of the photographed image, and the plurality of reference pixels selected by the horizontal reference point selecting means. Horizontal background model creation for acquiring the corresponding brightness values of a plurality of pixels in the background image from the background image holding means and creating a lighting variation background model when the background brightness changes under the influence of lighting variation. Means for obtaining, from the background image holding means, luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the vertical reference point selecting means, wherein the luminance of the background is affected by illumination fluctuation. A vertical background model creating means for creating a lighting variation background model when the illumination variation background model changes in response to the received light, the lighting variation background model created by the horizontal background model creating means in a multidimensional luminance space, and the horizontal reference point selecting means. Horizontal distance calculating means for calculating a distance between the acquired points represented by the plurality of luminance values; and, in a multidimensional luminance space, the vertical background model creating means. Vertical distance calculation means for calculating a distance between the created illumination variation background model and a point represented by the plurality of luminance values obtained by the vertical reference point selection means; the horizontal distance calculation means and the vertical distance calculation Based on the distance calculated by the means, it is determined whether the horizontal pixel column of interest and the vertical pixel column of interest include a pixel representing an object, and a logical operation is performed on a determination result relating to both pixel columns. And an object discriminating means for specifying a rectangular area including the object.
[0015]
In the object detection device of the present invention, the image input unit captures and holds the luminance value of each pixel of the captured image, and the horizontal reference point selection unit determines the horizontal pixel of interest including a plurality of pixels arranged in the horizontal direction in the captured image. For each column, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is obtained from the image input unit. Further, the vertical reference point selecting means selects, for each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the target vertical pixel column, and sets the luminance of each reference pixel. The value is obtained from the image input means.
Then, the horizontal background model creating unit acquires the luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selecting unit from the background image holding unit, and adjusts the luminance of the background. Create an illumination variation background model when changed under the influence of illumination variation, the vertical background model creation means, respectively corresponding to the plurality of reference pixels selected by the vertical reference point selection means, a plurality of in the background image The luminance value of the pixel is acquired from the background image holding means, and an illumination variation background model is created when the luminance of the background changes under the influence of illumination variation.
[0016]
Thereafter, the horizontal distance calculation unit calculates a distance between the illumination variation background model created by the horizontal background model creation unit and a point represented by the plurality of brightness values acquired by the horizontal reference point selection unit in a multidimensional brightness space. Vertical distance calculation means, in a multidimensional brightness space, between the illumination variation background model created by the vertical background model creation means and the point represented by the plurality of brightness values acquired by the vertical reference point selection means Calculate the distance of
Then, the object determination unit determines whether or not the target horizontal pixel column and the target vertical pixel column include a pixel representing an object, based on the distance calculated by the horizontal distance calculation unit and the vertical distance calculation unit, A rectangular operation including the object is specified by performing a logical operation on the determination result relating to both pixel columns.
[0017]
The present invention also relates to an object detection method for detecting an object using a captured image, comprising: an image inputting step of capturing a luminance value of each pixel of the captured image and storing the luminance value in a first storage unit; Selecting a plurality of reference pixels for each pixel of interest, and obtaining a plurality of reference pixels and a luminance value of the pixel of interest for each pixel of interest from the first storage means; A background image holding step of holding the luminance value of each pixel of the background image forming the background of the captured image in the second storage unit; and the plurality of pixels corresponding to the target pixel and the same pixel selected in the multiple reference point selecting step The luminance values of a plurality of pixels in the background image, each corresponding to a reference pixel, are obtained from the second storage means, and the illumination change when the luminance of the background changes under the influence of the illumination fluctuation. A background model creating step of creating a background model for each set of the pixel of interest and each reference pixel corresponding to the pixel; and in the luminance space, for each set of the pixel of interest and the reference pixel in the background model creating step. Between the illumination variation background model created in the step (a) and a point represented by a set of each of the luminance values of the plurality of reference pixels acquired for each of the target pixels in the plural reference point selecting step and the luminance value of the target pixel. A distance calculation step of calculating the distance of each of the pixels, and for each set of the pixel of interest and each reference pixel, determine whether the pixel of interest is a pixel representing an object based on the distance calculated in the distance calculation step. And an object discriminating step of finally performing a logical operation on the discrimination result of each set to determine whether the pixel of interest is a pixel representing an object.
[0018]
Further, the present invention is a computer-readable information recording medium recording an object detection program for detecting an object using a photographed image, wherein the luminance value of each pixel of the photographed image is fetched and stored in a first storage means. An image input step to be held, and a plurality of reference pixels are selected for each pixel of interest of the photographed image, and the luminance values of the selected plurality of reference pixels and the pixel of interest are stored in the first storage means for each pixel of interest. A step of selecting a plurality of reference points to be acquired; a step of holding a background image for holding a luminance value of each pixel of a background image forming a background of the captured image in a second storage unit; Pixels corresponding to the plurality of reference pixels corresponding to the same pixel, respectively, obtain luminance values of a plurality of pixels in the background image from the second storage unit, A background model creating step of creating an illumination variation background model when the luminance of the scene changes under the influence of illumination variation for each pair of the pixel of interest and each reference pixel corresponding to the pixel; The illumination variation background model created for each pair of the pixel of interest and the reference pixel in the background model creation step, and the luminance value of the plurality of reference pixels acquired for each pixel of interest in the multiple reference point selection step A distance calculating step of calculating a distance between each of the pixels and a point represented by a set of the luminance value of the target pixel; and the distance calculated in the distance calculating step for each set of the target pixel and each reference pixel. It is determined whether or not the target pixel is a pixel representing an object, and a logical operation is performed on the determination result for each set to determine whether or not the target pixel is a pixel representing the object. Characterized in that it comprises a object determination step of determining.
[0019]
According to the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the captured image is taken and stored in the first storage means. In the step of selecting a plurality of reference points, a plurality of reference pixels are selected for each pixel of interest of the image, and the luminance values of the selected plurality of reference pixels and the pixel of interest are obtained from the first storage means for each pixel of interest. . Then, in the background model creation step, the luminance values of the plurality of pixels in the background image corresponding to the pixel of interest selected in the plurality of reference point selection step and the plurality of reference pixels corresponding to the same pixel are stored in a second storage unit. , An illumination variation background model when the luminance of the background changes under the influence of illumination variation is created for each pair of the target pixel and each reference pixel corresponding to the pixel.
[0020]
In addition, in the distance calculation step, in the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel in the background model creation step and the multiple reference point selection step are acquired for each target pixel. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
Then, in the object determination step, for each pair of the target pixel and each reference pixel, it is determined whether or not the pixel of interest is a pixel representing an object based on the distance calculated in the distance calculation step. A logical operation is performed on the result to finally determine whether or not the pixel of interest is a pixel representing the object.
[0021]
Further, the present invention is a computer-readable information recording medium recording an object detection program for detecting an object using a photographed image, wherein the luminance value of each pixel of the photographed image is fetched and stored in a first storage means. The image input step to be held, and for each target horizontal pixel row composed of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the target horizontal pixel row are selected and the luminance of each reference pixel is selected. A horizontal reference point selecting step of acquiring a value from the first storage means; and a plurality of pixels included in the noted vertical pixel column for each noted vertical pixel column including a plurality of pixels arranged in a vertical direction in the captured image. A vertical reference point selecting step of selecting a reference pixel and acquiring a luminance value of each reference pixel from the first storage unit; and a luminance of each pixel of an image forming a background of the captured image. In the second storage unit, and the second storage unit stores luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the horizontal reference point selection step, respectively. A horizontal background model creating step of creating an illumination variation background model when the luminance of the background obtained from the means changes under the influence of illumination variation; and the plurality of reference pixels selected in the vertical reference point selection step. Vertical background model creation for acquiring the corresponding brightness values of a plurality of pixels in the background image from the second storage means and creating an illumination variation background model when the background brightness changes under the influence of illumination variation. In the multi-dimensional luminance space, the illumination variation background model created in the horizontal background model creation step, and the horizontal reference point selection step. A horizontal distance calculation step of calculating a distance between the points represented by the plurality of luminance values obtained in the step, and in a multidimensional luminance space, the illumination variation background model created in the vertical background model creation step; A vertical distance calculation step of calculating a distance between a point represented by the plurality of luminance values acquired in the reference point selection step, and the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, An object that determines whether or not a pixel representing an object is included in the horizontal pixel column of interest and the vertical pixel column of interest, and performs a logical operation on the determination result relating to both pixel columns to specify a rectangular region including the object And a determining step.
[0022]
The present invention also relates to an object detection method for detecting an object using a captured image, comprising: an image inputting step of capturing a luminance value of each pixel of the captured image and storing the luminance value in a first storage unit; In the above, for each target horizontal pixel column including a plurality of pixels arranged in the horizontal direction, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is obtained from the first storage unit. Horizontal reference point selecting step, and for each noted vertical pixel column composed of a plurality of pixels arranged in the captured image in the vertical direction, selecting a plurality of reference pixels included in the noted vertical pixel column, A vertical reference point selecting step of obtaining a luminance value from the first storage means, a background image holding step of storing, in the second storage means, a luminance value of each pixel of an image forming a background of the captured image; The luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the horizontal reference point selection step are acquired from the second storage unit, and the luminance of the background is affected by the illumination fluctuation. A horizontal background model creating step of creating an illumination variation background model in the case of having changed, and respectively corresponding to the plurality of reference pixels selected in the vertical reference point selecting step, the luminance values of a plurality of pixels in the background image are A vertical background model creating step for creating an illumination variation background model obtained when the background brightness changes under the influence of illumination variation, obtained from the second storage means; and in the multidimensional brightness space, the horizontal background model creation step The distance between the illumination variation background model created in step 2 and the point represented by the plurality of luminance values acquired in the horizontal reference point selection step is calculated. Horizontal distance calculation step, and in the multidimensional luminance space, between the illumination variation background model created in the vertical background model creation step and the point represented by the plurality of brightness values acquired in the vertical reference point selection step. Based on the vertical distance calculation step of calculating the distance, and the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, the target horizontal pixel column and the target vertical pixel column include pixels representing an object. An object discriminating step of discriminating whether or not the object exists and performing a logical operation on the discrimination result relating to both pixel columns to specify a rectangular area including the object.
[0023]
According to the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the captured image is captured and stored in the first storage means, In the horizontal reference point selecting step, for each target horizontal pixel column including a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is set. Obtained from the first storage means. In the vertical reference point selecting step, for each target vertical pixel column including a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the target vertical pixel column are selected, and the luminance of each reference pixel is determined. The value is obtained from the first storage unit.
Then, in the horizontal background model creating step, the luminance values of the plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are obtained from the first storage unit, and the luminance of the background is obtained. Creates an illumination variation background model in the case where it changes under the influence of illumination variation, and in the vertical background model creating step, the plurality of pixels in the background image respectively correspond to the plurality of reference pixels selected in the vertical reference point selection step. The luminance value of the pixel is obtained from the second storage means, and an illumination fluctuation background model is created when the luminance of the background changes under the influence of the illumination fluctuation.
[0024]
Thereafter, in a horizontal distance calculation step, in a multidimensional luminance space, a distance between the illumination variation background model created in the horizontal background model creation step and a point represented by the plurality of brightness values acquired in the horizontal reference point selection step In the vertical distance calculation step, in the multidimensional brightness space, between the illumination variation background model created in the vertical background model creation step and the point represented by the plurality of brightness values acquired in the vertical reference point selection step Calculate the distance of
Then, in the object determination step, based on the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, determine whether or not the target horizontal pixel column and the target vertical pixel column include a pixel representing an object, A rectangular operation including the object is specified by performing a logical operation on the determination result relating to both pixel columns.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
First, the principle of object detection in the first embodiment will be described. FIG. 3A is an explanatory diagram showing a positional relationship between a point of interest and a reference point on an image plane, FIG. 3B is a schematic diagram when an object enters an imaging region, and FIG. 3C is a schematic diagram showing a detection result. , (D) is another explanatory diagram showing a positional relationship between a point of interest and a reference point on an image plane, (E) is another schematic diagram showing a detection result, and (F) is a schematic diagram showing a final detection result. It is.
[0026]
As shown in FIG. 3A, the point of interest P _A And attention point P _B Are set apart from each other, as shown in FIG. _A At the point of interest, the point of interest P _A And reference point P _B Therefore, the object 108 is detected by performing detection using the same method as the conventional method using the illumination variation background model. Then, as shown in FIG. 3 (C), the detection result indicates that the point of interest P _A And reference point P _B Indicates that there is a possibility that the object 108 may be present in both of them.
[0027]
On the other hand, in the present embodiment, as shown in FIG. _B Reference point P different from _C Set further. And the attention point P _A And reference point P _C The object is also detected by the same method as in the related art. As a result, as described above, the object 108 _A In the detection result, as shown in FIG. _A And reference point P _C Indicates that the object 108 may be present at both positions.
[0028]
In the present embodiment, a logical product of these two detection results, that is, (C) and (E) in FIG. 3 is further obtained. Therefore, as shown in FIG. 3 (F), the attention point P common to both (C) and (E) of FIG. _A , Only the object image 111 at the position _A The object 108 that has invaded the is specified and its position is reliably detected. In the present embodiment, the point of interest and the reference point can be arranged at a certain distance from each other. Therefore, the image of the detected object is an image in which not only the outline but also the inner area is drawn.
[0029]
In general, when N (N is an integer of 3 or more) different reference points including a target point are set for one target point, the presence area of the object is determined by satisfying a condition represented by the following equation. Can be identified.
[0030]
(Equation 1)
dist (Pi, Pj)> ObjectSize
However, 1 ≦ i, j ≦ N (i ≠ j), and dist (A, B) represents the distance between points A and B. ObjectSize represents the size of the object on the image.
[0031]
Next, a first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing an object detection device according to the first embodiment, and FIG. 2 is a flowchart showing the operation of the object detection device shown in FIG. Hereinafter, an example of the object detection device of the present invention will be described with reference to these drawings, and at the same time, an embodiment of a computer-readable information recording medium that records the object detection method and the object detection program of the present invention will be described. .
[0032]
The object detection device 2 shown in FIG. 1 includes an image input unit 4, a plurality of reference point selection units 6, a background image holding unit 8, a background model creation unit 10, a distance calculation unit 12, an object determination unit 14, and a background creation unit 16. And detects an object from an image captured by a monitoring camera, for example.
Specifically, the object detection device 2 includes, for example, an interface with a surveillance camera and a video tape recorder, and is configured by a computer in which a predetermined operating system is incorporated. A computer-readable information recording medium, such as a CD-ROM, on which the object detection program of the present invention is recorded is mounted, program data is read from the CD-ROM, loaded into a computer memory, and the computer CPU is converted to the program data. It is realized by operating based on.
[0033]
The image input unit 4 captures an image captured by the surveillance camera or an image recorded on the video tape recorder after the photographing from the surveillance camera or the VTR and holds the captured image. More specifically, the image input unit 4 captures and holds the luminance value of each pixel of the captured image. The image captured by the image input unit 4 may be a moving image or a still image. Further, the image input unit 4 may have a configuration including a camera that captures an image by itself.
[0034]
The plurality of reference point selection means 6 selects a plurality of predetermined reference pixels (pixels at the position of the reference point) for each pixel of interest (pixel at the position of the point of interest) of the image, and The luminance values of the pixel and the target pixel are acquired from the image input means 4 for each target pixel. The plurality of reference pixels corresponding to one target pixel are also referred to as a reference pixel group.
The background image holding unit 8 holds the luminance value of each pixel of the background image in the shooting area. In the present embodiment, the luminance value held by the background image holding unit 8 is appropriately updated by the background creating unit 16 as described later.
The background model creating means 10 stores the luminance values of a plurality of pixels in the background image corresponding to the pixel of interest selected by the plurality of reference point selecting means 6 and the plurality of reference pixels corresponding to the same pixel. , An illumination variation background model when the luminance of the background changes under the influence of illumination variation is created for each pair of the target pixel and each reference pixel corresponding to the pixel.
[0035]
Then, the distance calculation unit 12 obtains the illumination variation background model created for each pair of the target pixel and the reference pixel by the background model creation unit 10 and the multiple reference point selection unit 6 for each attention pixel in the luminance space. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
[0036]
The object discriminating unit 14 discriminates, for each pair of the target pixel and each reference pixel, whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculating unit 12, and determines the target pixel and each reference pixel. A logical operation is performed on the determination result for each set of pixels to finally determine whether or not the target pixel is a pixel representing an object.
Then, the image output unit 15 outputs the image data of the determination result output by the object determination unit 14 to a display device (not shown) or the like.
Further, the background creating unit 16 calculates a luminance value of a pixel of a background image corresponding to each of the pixel of interest determined by the object determining unit 14 not to be a pixel representing the object and the plurality of reference pixels corresponding to the pixel. It is updated based on the luminance values of the target pixel and the reference pixel, and is stored in the background image storage unit 8.
[0037]
Next, the operation of the first embodiment configured as described above will be described.
As shown in FIG. 2, first, the image input unit 4 of the object detection device 2 reads an image captured by a surveillance camera or the like, and sets a luminance value for each pixel constituting the image in the image input unit 4. (Step A1).
The image is composed of a large number of pixels arranged in a matrix, and the image input means 4 holds the luminance values of such pixels as two-dimensional array data. Each pixel can be specified by designating an x address and a y address of the memory. The x address corresponds to the column number of the matrix, and the y address corresponds to the row number of the matrix.
[0038]
Then, the plurality of reference point selecting means 6 selects a plurality of predetermined reference pixels for each pixel of interest of the image, and outputs the luminance values of the selected plurality of reference pixels and the pixel of interest to the image input means 4. It is obtained for each pixel of interest from the memory (step A2).
Each of the reference pixels is set at a position apart from the target pixel by a distance larger than the size of the intruding object to be detected.
For example, it is assumed that the size of the intruding object on the screen is less than 100 pixels, and the current target pixel point P _A And another pixel point P different from this pixel _B Is used as a reference point, as shown in FIG. _B Is the point of interest P _A Is set at a position 100 pixels away from in the x-axis direction. At this time, the luminance value at the target point PA can be expressed as I (x, y), and the luminance value at the reference point PB can be expressed as I (x + 100, y).
Similarly, another reference point P _C Is set, the reference point P is set as shown in FIG. _C Is set at a position 100 pixels away from the point of interest PA in the direction opposite to the x-axis. Reference point P _C Can be represented as I (x-100, y).
[0039]
In the present embodiment, the multiple reference point selecting means 6 sets the point of interest P _A Two reference points P besides the point of interest _B , P _C Therefore, one attention pixel and two reference pixels respectively corresponding to the attention point and each reference point are set. Next, the multiple reference point selecting unit 6 supplies the respective luminance values of the target pixel and the reference pixel obtained in step A2 to the background creating unit 16 (step A3). At this stage, since the luminance value of each pixel constituting the background image is not yet stored in the memory constituting the background image holding means 8, the background creating means 16 is supplied from the plural reference point selecting means 6. Each luminance value is stored in the background image storage unit 8 as a luminance value of a corresponding pixel of the background image.
[0040]
Next, the background model creation unit 10 creates an illumination variation background model using the luminance value of each pixel of the background image held by the background image holding unit 8.
For example, the target pixel is the target point P in FIG. _A P corresponding to _a And the reference pixel is the reference point P _B , P _C P corresponding to _b , P _c In this case, the background model creating means 10 first _a And the reference pixel p _b Are obtained from the background image holding means 8.
[0041]
Then, using these two luminance values, the pixel of interest p _a And the reference pixel p _b A lighting variation background model in the case where the luminance of the background changes under the influence of lighting variation is created. Specifically, as shown in FIG. _a The horizontal axis represents the luminance value of the background pixel corresponding to _b In the luminance space having the luminance value of the background pixel corresponding to _a And the reference pixel p _b And a straight line passing through the origin, which is represented by a set of the luminance values of the background pixels corresponding to the above, and the straight line is used as the illumination variation background model.
The background model creating means 10 calculates the target pixel p _a And the reference pixel p _c Similarly, an illumination variation background model is created for the set of.
[0042]
If the number of pixels to be referred to for each target pixel is N including the target pixel, the luminance values of the N background image pixels are used for each target pixel. A set of background luminance values can be represented as an N-dimensional vector.
In this embodiment, since two sets of the target pixel and one reference point are used, the luminance value of the background is two sets of two-dimensional vectors.
[0043]
Thereafter, the distance calculation means 12 calculates the target pixel p _a And the reference pixel p _b In the luminance space, the illumination variation background model and the pixel of interest p _a And the reference pixel p _b Of the pixel represented by the set of luminance values of _a And the reference pixel p _c A similar distance is calculated for the set (step A4).
[0044]
Here, the target pixel p _a , Reference pixel p _b , P _c The luminance values of the background pixels corresponding to _b0 , I _c0 And the target pixel p _a , Reference pixel p _b , P _c The luminance value of i _a1 , I _b1 , I _c1 Then, the target pixel p _a And the reference pixel p _b , The distance D1 between the illumination variation background model corresponding to the set of _a And the reference pixel p _c The distance D2 between the illumination variation background model corresponding to the set and the points corresponding to the luminance values ia1 and ic1 can be calculated by the equations shown in [Equation 2] and [Equation 4], respectively.
[0045]
(Equation 2)

[0046]
[Equation 3]

Next, the object discriminating means 14 calculates the distance D obtained in step A4. ₁ , D ₂ And a preset threshold T ₁ , T ₂ (Step A5), and when these distances and threshold values satisfy [Equation 4], the target pixel p _a Is a pixel representing the object, and it is determined that the object exists at the corresponding position. _a Are determined to be pixels representing the background (steps A6, S7, S8, S9).
[0047]
(Equation 4)

More specifically, the object determining means 14 calculates the distance D calculated by the plurality of distance calculating means. ₁ , D ₂ Is the threshold T ₁ , T ₂ It is checked whether it is greater than (step A5). For example, distance D ₁ Is the threshold T ₁ If it is larger than the target pixel p, _a And the reference pixel p _b Is determined to be a pixel representing an object, and a logical "1" is associated with both pixels as a detection result. Also, the distance D ₂ Also the threshold T ₂ If the target pixel p is larger than _a And the reference pixel p _c Is determined to be a pixel representing an object, and a logical "1" is associated with both pixels as a detection result.
On the other hand, the distance D ₁ , D ₂ Is T ₁ , T ₂ If smaller, a logical "0" is associated with each pixel.
[0048]
Then, the object discriminating means 14 calculates the logical product of these detection results (step A6) and determines the value (step A7). If the result is logic "1", the object pixel p _a Is a pixel representing an object (step A8). _a Is a pixel that does not represent an object but represents a background (step A9).
For example, distance D ₁ Is the threshold T ₁ If the pixel p is larger than _a , P _b , P _c Are "1", "1", "0", and the distance D ₂ Is the threshold T ₂ If the pixel p is larger than _a , P _b , P _c Are "1", "0", and "1". Therefore, when the logical product of these values is calculated for each pixel, they become "1", "0", and "0", respectively, and the pixel p _a Is only "1". Therefore, the object discriminating means 14 determines that the target pixel p _a Is a pixel representing an object.
[0049]
Generally, when N reference points are taken in addition to the target pixel, the distance D _i And calculate the threshold value T for each reference point. _i Can be used to determine whether or not the target pixel is a pixel representing an object, based on whether or not [Equation 5] holds.
[0050]
(Equation 5)

By performing such processing from steps A2 to S9 by sequentially setting all the pixels held by the image input unit 4 as target pixels (step A10), it is possible to detect an object on the captured image. it can.
The detected object is output by the image output means 15 and displayed on the display device. At this time, if the display color of the pixel of interest determined to represent the object in steps A7 and A8 is displayed in a color different from the background, the shape of the detected object can be more clearly grasped on the screen. .
[0051]
In step A10, when it is determined that the processing has been completed for all the pixels, the number of pixels determined to be an object is counted, and when this count value is larger than a preset threshold value, It may be determined that only an object exists. This makes it possible to detect only objects of a certain size or more.
[0052]
On the other hand, when it is determined in steps A7 and A9 that the target pixel is not a pixel representing the object, the background creating unit 16 determines the corresponding background based on the luminance values of the target pixel and a plurality of corresponding reference pixels. The luminance values of the pixels of the image are updated and stored in the background image storing means 8 (step A3).
If the moving object is not included in the first input image, there is basically no need to update the luminance value of each pixel of the background image. However, since the luminance value of the background area may vary due to factors other than the illumination fluctuation due to the influence of noise or the like, the error is updated by updating the luminance value at any time in the pixel regarded as the background by the object determination unit 14. Can be suppressed.
[0053]
In such updating of the background image, a method of gradually updating the old and new luminance values by multiplying them by weights can be adopted. That is, the target pixel p _a Is regarded as a background pixel, the newly added data can be weighted and updated gradually as in [Equation 6].
[0054]
(Equation 6)

Where P _A _New is the pixel of interest p _a And the reference pixel p _b , P _c , A three-dimensional brightness value matrix based on the brightness values of the background pixels corresponding to _A _Old is the pixel of interest p _a And the reference pixel p _b , P _c Is a three-dimensional background luminance matrix based on the luminance values of the background pixels corresponding to. Also, I _A Is the pixel of interest p _a And the reference pixel p _b , P _c And α is the above weight constant.
[0055]
Next, when it is determined in step A10 that the above-described determination processing has been completed for all pixels, the image input unit 4 determines whether or not there is a next image (step A11). If there is an image, the process returns to step A1 to acquire the image, and thereafter, the above-described processes are repeated. Then, when the processing is completed for all the images, the object detection processing ends.
By performing the above processing each time an image is captured from the monitoring camera, object detection can be performed sequentially.
[0056]
FIG. 4A is a schematic diagram showing an original image of a black car running on a road, FIG. 4B is an explanatory diagram showing a detection result of a car according to the related art, and FIG. 4C is a car according to this embodiment. FIG. 9 is an explanatory diagram showing a detection result of FIG.
FIG. 4B shows a result of detecting a car based on the conventional technique described with reference to FIG. 8 and the like using the original image and the background image shown in FIG. Are detected adjacent to each other, so that only the outline of the vehicle is detected.
[0057]
On the other hand, in FIG. 4C, which is the detection result according to the present embodiment, not only the outline of the vehicle but also the entire vehicle including the inside of the outline is detected, so that the detection is more clear, and accordingly. The car has been detected reliably.
In the present embodiment, as described above, the reference pixel to be selected for each target pixel is set apart from the target pixel by at least the size of the object. Therefore, unlike the case where the target pixel and the reference pixel are set close to each other, only the outline of the object is not detected.
Then, a plurality of reference pixels are set for one target pixel, an object is detected for each set of the target pixel and each reference pixel, and finally, a logical operation is performed to determine whether the target pixel is a pixel of the object. Since the determination is made, the position of the object can be reliably specified even if the target pixel and the reference pixel are set apart.
[0058]
In addition, as in the above-described related art, since it is determined whether or not the pixel of interest represents an image based on the distance between the illumination variation background model and the luminance point in the luminance space, illumination variation is large and frequently occurs. In this case, the object can be detected stably and with high accuracy.
[0059]
Next, a second embodiment of the present invention will be described.
FIG. 5 is a block diagram showing the object detection device 2 of the second embodiment, and FIG. 6 is a flowchart showing the operation of the object detection device of FIG. Hereinafter, an object detection device according to a second embodiment of the present invention will be described with reference to these drawings, and at the same time, a computer that records an object detection method and an object detection program according to the second embodiment of the present invention. A readable information recording medium will be described. In FIG. 5, the same elements as those in FIG. 1 are denoted by the same reference numerals, and a description thereof will be omitted.
[0060]
The object detecting device 18 shown in FIG. 5 is different from the object detecting device 2 in that the object itself is not detected but a circumscribed rectangle of the object is obtained. As a result, the object is detected with a small amount of calculation. Becomes possible.
The object detecting device 18 includes an image input unit 4, a plurality of reference point selecting units 20 functioning as a horizontal reference point selecting unit and a vertical reference point selecting unit according to the present invention, a background image holding unit 8, and a horizontal background model generation according to the present invention. Means and a background model creating means 22 functioning as a vertical background model creating means, distance calculating means 24 functioning as horizontal distance calculating means and vertical distance calculating means according to the present invention, object discriminating means 26, background creating means 28, and image output Including the means 15, for example, an object is detected from an image captured by a surveillance camera.
[0061]
The object detection device 18 is, similarly to the object detection device 2, specifically, configured by a computer having an interface with a surveillance camera or a video tape recorder, for example. For example, a CD-ROM as a computer-readable information recording medium on which the object detection program is recorded is loaded, program data is read from the CD-ROM, loaded into a memory of a computer, and a CPU of the computer is configured based on the program data. It is realized by operating it.
[0062]
The multiple reference point selecting means 20 of the object detecting device 18 functions as a horizontal reference point selecting means, and includes, for each noted horizontal pixel row composed of a plurality of pixels arranged in the horizontal direction in the captured image, the focused horizontal pixel row. A plurality of reference pixels to be selected are selected, and the luminance value of each reference pixel is obtained from the image input means 4.
Further, the plurality of reference point selection means 20 functions as a vertical reference point selection means, and for each attention vertical pixel row composed of a plurality of pixels arranged in the captured image in the vertical direction, a plurality of reference A reference pixel is selected, and the luminance value of each reference pixel is obtained from the image input unit 4.
[0063]
The background model creating unit 22 functions as a horizontal background model creating unit, and stores the luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selecting unit, as the background image holding unit. 8, an illumination variation background model is created when the background luminance changes under the influence of illumination variation.
The background model creating unit 22 functions as a vertical background model creating unit, and stores the luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the vertical reference point selecting unit, into the background image holding unit. 8, an illumination variation background model is created when the background luminance changes under the influence of illumination variation.
[0064]
The multiple distance calculation means 2 functions as a horizontal distance calculation means, and in a multidimensional luminance space, the illumination variation background model created by the background model creation means 22 as a horizontal background model creation means, and a horizontal reference point selection means. Then, the distance between the multiple reference point selecting means 20 and the point represented by the plurality of luminance values is calculated.
The multiple distance calculation means 2 functions as a vertical distance calculation means, and in a multidimensional luminance space, the illumination variation background model created by the background model creation means 22 as the vertical background model creation means, and a vertical reference point selection. The distance from the point represented by the plurality of luminance values acquired by the plurality of reference point selecting means 20 is calculated.
[0065]
The object discriminating means 26 includes, based on the distances calculated by the plurality of distance calculating means 2 as the horizontal distance calculating means and the vertical distance calculating means, a pixel representing the object in the horizontal pixel column of interest and the vertical pixel column of interest. Then, a logical operation is performed on the determination result regarding both pixel columns to specify a rectangular area including the object.
The background creating means 28, based on the luminance values of the pixels constituting the noted horizontal pixel row and the noted vertical pixel row determined not to include the pixel representing the object by the object determining means 26, Is updated and stored in the background image storage unit 8.
[0066]
Next, the operation of the object detection device 18 thus configured will be described.
FIGS. 7A and 7B are diagrams for explaining the operation of the second embodiment, wherein FIG. 7A is an explanatory diagram showing detection of a possible area of an object in a vertical direction, and FIG. FIG. 3C is an explanatory diagram illustrating detection of a possible area, and FIG. 4C is an explanatory diagram illustrating a detection result. Hereinafter, a description will be given with reference to FIGS.
[0067]
As shown in FIG. 6, first, the image input unit 4 reads an image captured by a surveillance camera or the like, and outputs a luminance value for each pixel constituting the image (not shown) provided in the image input unit 4. The data is stored in the memory (step B1).
Next, the multiple reference point selecting means 20 sets one row of pixels at the vertical address y in the captured image as a horizontal pixel column of interest, and selects one row of pixels as a reference pixel (reference pixel group). Then, the luminance value of each reference pixel is obtained from the image input means 4 (step B4). Therefore, assuming that the width of the captured image is Width pixels (Width is a positive integer), the horizontal pixel row of interest includes Width pixels, and the plurality of reference point selection means 20 in this embodiment is All these pixels are set as reference pixels.
[0068]
Then, the background model creating unit 22 acquires from the background image holding unit 8 the luminance value of each pixel in the background image corresponding to each of the reference pixels selected by the multiple reference point selecting unit 20, and sets the luminance of the background to the illumination fluctuation. Create a lighting variation background model when it changes under the influence of.
Thereafter, a plurality of distance calculation means 24 calculates the illumination variation background model created by the background model creation means 22 and the plurality of plurality of reference points acquired by the plurality of reference point selection means 20 in the multidimensional brightness space formed by the brightness values of the reference pixels. Then, the distance to the point represented by the luminance value is calculated (step B6).
[0069]
Subsequently, the object discriminating means 26 compares the distance calculated by the plurality of distance calculating means 24 with a threshold, and if the distance is larger than the threshold, associates the distance with the horizontal pixel row of interest. The logic "1" is held, and if the distance is smaller than the threshold, the logic "0" is held (step B8).
The multiple reference point selecting unit 20, the background model creating unit 22, and the multiple distance calculating unit 2 perform such processing over the entire vertical range of the image while increasing the value of the address y by one (step B10). ).
As a result, as shown in FIG. 7A, when the object 108 exists as shown in the figure, the object discriminating unit 26 sets the logic “1” for each target horizontal pixel row in the range of H. Will be retained.
[0070]
The multiple reference point selecting unit 20, the background model creating unit 22, and the multiple distance calculating unit 2 perform the same processing in the horizontal direction of the image.
That is, the multiple reference point selecting means 20 sets one column of pixels at a horizontal address x in the captured image as a target vertical pixel column, and selects one column of pixels as a reference pixel (reference pixel group). Then, the luminance value of each reference pixel is obtained from the image input means 4 (step B2). Therefore, assuming that the height of the captured image is a Height pixel (Height is a positive integer), the noted vertical pixel column includes Height pixels, and the multiple reference point selecting unit 20 is configured to execute the processing according to the present embodiment. Then, all of these pixels are set as reference pixels.
[0071]
Then, the background model creating unit 22 acquires from the background image holding unit 8 the luminance value of each pixel in the background image corresponding to each of the reference pixels selected by the multiple reference point selecting unit 20, and sets the luminance of the background to the illumination fluctuation. Create a lighting variation background model when it changes under the influence of.
Thereafter, a plurality of distance calculation means 24 calculates the illumination variation background model created by the background model creation means 22 and the plurality of plurality of reference points acquired by the plurality of reference point selection means 20 in the multidimensional brightness space formed by the brightness values of the reference pixels. Then, the distance to the point represented by the luminance value is calculated (step B5).
[0072]
Subsequently, the object discriminating means 26 compares the distance calculated by the plural distance calculating means 2 with a threshold value, and if the distance is larger than the threshold value, associates the distance with the noted vertical pixel column. The logic "1" is held, and if the distance is smaller than the threshold, the logic "0" is held (step B7).
The multiple reference point selecting unit 20, the background model creating unit 22, and the multiple distance calculating unit 24 perform such processing over the entire horizontal range of the image while increasing the value of the address y by one (step B9). ).
As a result, as shown in FIG. 7B, the object discriminating means 26 holds the logic “1” for each vertical pixel row of interest within the range of W.
[0073]
Thereafter, the object discriminating unit 26 holds the image of the rectangular area as a detection result by taking the logical product of the logical values held for each of the horizontal pixel columns of interest and each vertical pixel column of interest (step B11). (Not shown) for each pixel determines whether the value is logic "1" or logic "0".
More specifically, when the logical “1” is held for both the horizontal pixel column of interest and the vertical pixel column of interest including the pixel of the captured image at the position corresponding to the pixel whose value is to be determined, The value is set to logic "1". On the other hand, when at least one of the horizontal pixel column of interest and the vertical pixel column of interest including the pixel of the corresponding captured image holds logic "0", the value of the pixel is set to logic "0". (Steps B12, B13, B14). As a result, in the memory holding the detection result, as shown in FIG. 7C, the value of the pixel in the rectangular area R that may include the object becomes logic “1”, and the other pixels Becomes logic "0".
[0074]
Therefore, if the contents of this memory are displayed on the display device by the image output means 15, a rectangular area including the object is displayed on the screen. The four sides of the rectangular area R are circumscribed rectangles of the object 108.
If the width of the captured image is Width pixels and the height is Height pixels, the first embodiment creates a background model and calculates a distance for each pixel. It is necessary to perform the processing (Width × High) times.
However, in the second embodiment, the above-described arithmetic processing may be performed for each horizontal pixel column, and the arithmetic processing may be performed for each vertical pixel column. Therefore, the total number of arithmetic processes is (Width + Height) times. It becomes. Therefore, the amount of calculation is greatly reduced as compared with the first embodiment, which is very advantageous in performing high-speed object detection and reducing the cost of the apparatus.
[0075]
Note that, as in the case of the object detection device 2, the background creating unit 28 determines the luminance value of each pixel constituting the target horizontal pixel column and the target vertical pixel column determined by the object determination unit 26 to not include a pixel representing the object. Based on this, the luminance value of the pixel of the corresponding background image is updated and held by the background image holding means 8. Thereby, highly accurate object detection can be performed using the latest background image.
If there are more images to be processed, the above procedure is repeated for the next image as in the case of the object detection device 2 (step B15).
[0076]
In the second embodiment, in the horizontal pixel column of interest and the vertical pixel column of interest, all the pixels in each pixel column are set as reference pixels, but thinning is performed by selecting every several pixels. It is also possible to further reduce the amount of calculation.
Further, for example, first, in any one of the horizontal direction and the vertical direction, the possible area of the object is determined, and then the processing in the other direction is performed only within that area, so that there is no possibility that the object exists. Since it is not necessary to perform the arithmetic processing, the amount of operation can be further reduced.
[0077]
Here, the object detection device 2 and the object detection device 18 are configured by a computer, but it is a matter of course that the object detection device 2 and the object detection device 18 can be configured by dedicated hardware. Also, among the functions according to the present invention, some functions are realized by loading a predetermined program into a computer, and other functions are realized by dedicated hardware, or further, are already incorporated in the computer. By using the programs in combination, the object detection device 2 and the object detection device 18 can be configured.
[0078]
The computer-readable information recording medium is not limited to the above-described CD-ROM, but may be a portable medium such as a floppy (registered trademark) disk, a magneto-optical disk, or a ROM, or a storage device such as a hard disk built in the computer. Contains.
Further, the computer-readable information recording medium dynamically holds the program for a short time, such as a communication line for transmitting the program via a communication network such as the Internet or a communication line such as a telephone line. In such a case, such as a volatile memory in a computer system serving as a server or a client that holds a program for a certain period of time is also included.
[0079]
【The invention's effect】
As described above, in the object detection device of the present invention, the image input unit captures and holds the luminance value of each pixel of the captured image, and the multiple reference point selection unit includes a plurality of reference pixels for each pixel of interest in the image. Is selected, and the luminance values of the selected reference pixels and the target pixel are obtained from the image input means for each target pixel. Then, the background model creating means, from the background image holding means, stores the luminance values of the plurality of pixels in the background image corresponding to the pixel of interest selected by the plurality of reference point selecting means and the plurality of reference pixels corresponding to the same pixel. Then, an illumination variation background model when the luminance of the background changes under the influence of the illumination variation is created for each pair of the target pixel and each reference pixel corresponding to the pixel.
[0080]
Then, the distance calculation unit acquires, in the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel by the background model creation unit, and the multiple reference point selection unit acquires for each attention pixel. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
Then, the object determination unit determines, for each pair of the target pixel and each reference pixel, whether or not the pixel of interest is a pixel representing an object based on the distance calculated by the distance calculation unit. A logical operation is performed on the result to finally determine whether or not the pixel of interest is a pixel representing the object.
[0081]
As described above, in the object detection device of the present invention, a plurality of reference pixels are set for one target pixel, an object is detected for each set of the target pixel and each reference pixel, and finally, a logical operation is performed. Since it is determined whether or not the target pixel is a pixel of the object, the position of the object can be reliably specified even if the target pixel and the reference pixel are set apart.
Therefore, the reference pixel to be selected for each target pixel can be set apart from the target pixel by at least the size of the object, and as a result, the target pixel and the reference pixel can be set close to each other. Does not detect only the contour of the object.
Further, similarly to the above-described related art, since it is determined whether or not a pixel of interest represents an image based on a distance between an illumination variation background model and a luminance point in a luminance space, the illumination variation is large and frequently occurs. Object can be detected with high stability and high accuracy.
[0082]
Further, in the object detection device of the present invention, the image input means captures and holds the luminance value of each pixel of the captured image, and the horizontal reference point selecting means includes a plurality of pixels arranged in the captured image in a horizontal direction. For each horizontal pixel column, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is obtained from the image input means. Further, the vertical reference point selecting means selects, for each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the target vertical pixel column, and sets the luminance of each reference pixel. The value is obtained from the image input means.
Then, the horizontal background model creating unit acquires the luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selecting unit from the background image holding unit, and adjusts the luminance of the background. Create an illumination variation background model when changed under the influence of illumination variation, the vertical background model creation means, respectively corresponding to the plurality of reference pixels selected by the vertical reference point selection means, a plurality of in the background image The luminance value of the pixel is acquired from the background image holding means, and an illumination variation background model is created when the luminance of the background changes under the influence of illumination variation.
[0083]
Thereafter, the horizontal distance calculation unit calculates a distance between the illumination variation background model created by the horizontal background model creation unit and a point represented by the plurality of brightness values acquired by the horizontal reference point selection unit in a multidimensional brightness space. Vertical distance calculation means, in a multidimensional brightness space, between the illumination variation background model created by the vertical background model creation means and the point represented by the plurality of brightness values acquired by the vertical reference point selection means Calculate the distance of
Then, the object determination unit determines whether or not the target horizontal pixel column and the target vertical pixel column include a pixel representing an object, based on the distance calculated by the horizontal distance calculation unit and the vertical distance calculation unit, A rectangular operation including the object is specified by performing a logical operation on the determination result relating to both pixel columns.
[0084]
In the object detection device of the present invention, calculation processing such as creation of a background model for each horizontal pixel row, calculation of the distance between the background model and a luminance point is performed, and similar calculation processing is performed for each vertical pixel row. Therefore, the number of times of the arithmetic processing is a value obtained by adding the number of pixels in the vertical direction of the image to the number of pixels in the horizontal direction of the image as a whole.
On the other hand, when an arithmetic process such as creation of a background model for each pixel or calculation of a distance between the background model and a luminance point is performed, the total number of arithmetic processes is reduced by the number of pixels in the horizontal direction of the image. , Multiplied by the number of pixels in the vertical direction of the image.
Therefore, according to the present invention, even when illumination fluctuations are large and occur frequently, the present invention can detect the area where the object exists with high accuracy, and can greatly reduce the amount of calculation. This is very advantageous in performing detection and reducing the cost of the apparatus.
[0085]
According to the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the captured image is taken and stored in the first storage means. In the step of selecting a plurality of reference points, a plurality of reference pixels are selected for each pixel of interest of the image, and the luminance values of the selected plurality of reference pixels and the pixel of interest are obtained from the first storage means for each pixel of interest. . Then, in the background model creation step, the luminance values of the plurality of pixels in the background image corresponding to the pixel of interest selected in the plurality of reference point selection step and the plurality of reference pixels corresponding to the same pixel are stored in a second storage unit. , An illumination variation background model when the luminance of the background changes under the influence of illumination variation is created for each pair of the target pixel and each reference pixel corresponding to the pixel.
[0086]
In addition, in the distance calculation step, in the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel in the background model creation step and the multiple reference point selection step are acquired for each target pixel. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
Then, in the object determination step, for each pair of the target pixel and each reference pixel, it is determined whether or not the pixel of interest is a pixel representing an object based on the distance calculated in the distance calculation step. A logical operation is performed on the result to finally determine whether or not the pixel of interest is a pixel representing the object.
[0087]
As described above, in the present invention, a plurality of reference pixels are set for one target pixel, object detection is performed for each pair of the target pixel and each reference pixel, and finally the target pixel is logically operated to determine the target pixel. Since it is determined whether the pixel is a pixel or not, the position of the object can be reliably specified even if the target pixel and the reference pixel are set apart.
Therefore, the reference pixel to be selected for each target pixel can be set apart from the target pixel by at least the size of the object, and as a result, the target pixel and the reference pixel can be set close to each other. Does not detect only the contour of the object.
Further, similarly to the above-described related art, since it is determined whether or not a pixel of interest represents an image based on a distance between an illumination variation background model and a luminance point in a luminance space, the illumination variation is large and frequently occurs. Object can be detected with high stability and high accuracy.
[0088]
According to the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the photographed image is taken and stored in the first storage means. In the horizontal reference point selection step, a plurality of reference pixels included in the target horizontal pixel row are selected for each target horizontal pixel row including a plurality of pixels arranged in the horizontal direction in the captured image, and the luminance of each reference pixel is determined. The value is obtained from the first storage unit. In the vertical reference point selecting step, for each target vertical pixel column including a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the target vertical pixel column are selected, and the luminance of each reference pixel is determined. The value is obtained from the first storage unit.
Then, in the horizontal background model creating step, the luminance values of the plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are obtained from the first storage unit, and the luminance of the background is obtained. Creates an illumination variation background model in the case where it changes under the influence of illumination variation, and in the vertical background model creating step, the plurality of pixels in the background image respectively correspond to the plurality of reference pixels selected in the vertical reference point selection step. The luminance value of the pixel is obtained from the second storage means, and an illumination fluctuation background model is created when the luminance of the background changes under the influence of the illumination fluctuation.
[0089]
Thereafter, in a horizontal distance calculation step, in a multidimensional luminance space, a distance between the illumination variation background model created in the horizontal background model creation step and a point represented by the plurality of brightness values acquired in the horizontal reference point selection step In the vertical distance calculation step, in the multidimensional brightness space, between the illumination variation background model created in the vertical background model creation step and the point represented by the plurality of brightness values acquired in the vertical reference point selection step Calculate the distance of
Then, in the object determination step, based on the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, determine whether or not the target horizontal pixel column and the target vertical pixel column include a pixel representing an object, A rectangular operation including the object is specified by performing a logical operation on the determination result relating to both pixel columns.
[0090]
As described above, in the present invention, calculation processing such as creation of a background model for each horizontal pixel row and calculation of the distance between the background model and a luminance point is performed, and similar calculation processing is performed for each vertical pixel row. Therefore, the number of times of the arithmetic processing is a value obtained by adding the number of pixels in the vertical direction of the image to the number of pixels in the horizontal direction of the image as a whole.
On the other hand, when an arithmetic process such as creation of a background model for each pixel or calculation of a distance between the background model and a luminance point is performed, the total number of arithmetic processes is reduced by the number of pixels in the horizontal direction of the image. , Multiplied by the number of pixels in the vertical direction of the image.
Therefore, according to the present invention, the illumination fluctuation is large and stable even when it frequently occurs. In addition to being able to detect the existence area of the object with high accuracy, the amount of calculation is greatly reduced. This is very advantageous in performing object detection and reducing the cost of the apparatus.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an object detection device according to a first embodiment.
FIG. 2 is a flowchart illustrating an operation of the object detection device of FIG. 1;
3A is an explanatory diagram illustrating a positional relationship between a point of interest and a reference point on an image plane, FIG. 3B is a schematic diagram illustrating a case where an object enters a shooting area, and FIG. 3C is a schematic diagram illustrating a detection result; FIG. 3D is another explanatory diagram showing a positional relationship between a point of interest and a reference point on an image plane, FIG. 4E is another schematic diagram showing a detection result, and FIG. 4F is a schematic diagram showing a final detection result. FIG.
4A is a schematic diagram showing an original image of a black car running on a road, FIG. 4B is an explanatory diagram showing a detection result of a car according to the related art, and FIG. 4C is a diagram according to the embodiment; It is an explanatory view showing a detection result of a car.
FIG. 5 is a block diagram illustrating an object detection device according to a second embodiment.
FIG. 6 is a flowchart showing the operation of the object detection device of FIG.
FIGS. 7A and 7B are diagrams for explaining the operation of the second embodiment, wherein FIG. 7A is an explanatory diagram showing detection of a possible area of an object in a vertical direction, and FIG. FIG. 7C is an explanatory diagram showing detection of a possible area, and FIG. 7C is an explanatory diagram showing a detection result.
FIG. 8 is an explanatory diagram illustrating a background image.
FIG. 9 is an explanatory diagram showing a two-dimensional luminance space formed by luminance values of a target point and a reference point.
FIG. 10 is an explanatory diagram illustrating a captured image when an object enters a capturing area of the monitoring camera.
11A is an explanatory diagram illustrating an image plane, FIG. 11B is a schematic diagram when an object enters a shooting area, and FIG. 11C is a schematic diagram illustrating a detection result.
[Explanation of symbols]
2 ... object detection device, 4 ... image input means, 6 ... multiple reference point selection means, 8 ... background image holding means, 10 ... background model creation means, 12 ... distance calculation means, 14 ... object Discriminating means 16 Background creating means 18 Object detecting device 20 Multiple reference point selecting means 22 Background model creating means 24 Distance calculating means 26 Object discriminating means 28 ... Background creating means, 102... White dots, 104... Straight lines, 106... White circle dots, 108.

Claims (4)

A computer-readable information recording medium recorded with an object detection program for detecting an object using a captured image,
An image inputting step of taking in a luminance value of each pixel of the photographed image and holding the luminance value in the first storage means;
A plurality of reference points selecting step of selecting a plurality of reference pixels for each pixel of interest of the photographed image and acquiring the luminance values of the selected plurality of reference pixels and the pixel of interest from the first storage means for each pixel of interest When,
A background image holding step of holding the luminance value of each pixel of the background image forming the background of the captured image in a second storage unit;
Acquiring, from the second storage unit, luminance values of a plurality of pixels in the background image corresponding to the target pixel selected in the multiple reference point selecting step and the plurality of reference pixels corresponding to the same pixel, A background model creating step of creating an illumination variation background model when the luminance of the background changes under the influence of illumination variation, for each set of the target pixel and each reference pixel corresponding to the pixel.
In the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel in the background model creation step, and the plurality of reference pixels acquired for each target pixel in the multiple reference point selection step A distance calculation step of calculating a distance between each of the brightness values of the pair and the point represented by the set of the brightness value of the pixel of interest,
For each set of the target pixel and each reference pixel, it is determined whether the target pixel is a pixel representing an object based on the distance calculated in the distance calculation step, and a logical operation is performed on the determination result for each set. And an object discriminating step of finally discriminating whether or not the pixel of interest is a pixel representing an object. The target pixel determined not to be a pixel representing the object in the object determination step, and a luminance value of a pixel of the background image corresponding to each of the plurality of reference pixels corresponding to the pixel, the target pixel, 2. A computer readable recording apparatus according to claim 1, further comprising a background creation step of updating the plurality of reference pixels corresponding to the pixels based on luminance values of the plurality of reference pixels and storing the updated data in the first storage unit. Information recording medium. A computer-readable information recording medium recorded with an object detection program for detecting an object using a captured image,
An image inputting step of taking in a luminance value of each pixel of the photographed image and holding the luminance value in the first storage means;
For each noted horizontal pixel row composed of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the noted horizontal pixel row are selected, and the luminance value of each reference pixel is stored in the first storage. Selecting a horizontal reference point obtained from the means;
For each noted vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the noted vertical pixel column are selected, and the luminance value of each reference pixel is stored in the first storage. Selecting a vertical reference point obtained from the means;
A background image holding step of holding the brightness value of each pixel of the image forming the background of the captured image in the second storage means;
The luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the horizontal reference point selection step are obtained from the second storage unit, and the luminance of the background is affected by illumination fluctuation. A horizontal background model creating step of creating a lighting variation background model when the
The luminance values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the vertical reference point selecting step are obtained from the second storage unit, and the luminance of the background is affected by the illumination fluctuation. Vertical background model creation step of creating a lighting variation background model in the event of a change,
Horizontal distance calculation for calculating a distance between the illumination variation background model created in the horizontal background model creation step and the point represented by the plurality of brightness values acquired in the horizontal reference point selection step in a multidimensional brightness space. Steps and
Vertical distance calculation for calculating a distance between the illumination variation background model created in the vertical background model creation step and a point represented by the plurality of brightness values acquired in the vertical reference point selection step in a multidimensional brightness space. Steps and
Based on the distances calculated in the horizontal distance calculation step and the vertical distance calculation step, it is determined whether or not the target horizontal pixel column and the target vertical pixel column include a pixel representing an object. An object determination step of performing a logical operation on the determination result according to the above to specify a rectangular area including the object. Based on the luminance values of the pixels constituting the horizontal pixel column of interest and the vertical pixel column of interest determined not to include the pixel representing the object in the object determination step, update the brightness value of the corresponding pixel of the background image. 4. A computer-readable information recording medium recording an object detection program according to claim 3, further comprising a background creation step of causing the second storage means to hold the background.

Images