JP4306333B2 - Computer-readable information recording medium recording object detection program - Google Patents

Description

【００４６】
【数３】

次に、物体判別手段１４は、ステップＡ４において求められた距離Ｄ₁、Ｄ₂と、あらかじめ設定されているしきい値Ｔ₁、Ｔ₂とを比べ（ステップＡ５）、これらの距離およびしきい値が［数４］を満たす場合には、注目画素ｐ_aは物体を表す画素であり、対応箇所に物体が存在すると判断し、一方、［数４］を満たさない場合には、注目画素ｐ_aは背景を表す画素であると判断する（ステップＡ６、Ｓ７、Ｓ８、Ｓ９）。
【００４７】
【数４】

より詳しくは、物体判別手段１４は、複数距離計算手段が計算した距離Ｄ₁、Ｄ₂がそれぞれ上記しきい値Ｔ₁、Ｔ₂より大きいか否かを調べる（ステップＡ５）。たとえば距離Ｄ₁がしきい値Ｔ₁より大きい場合には、物体判別手段１４は注目画素ｐ_aおよび参照画素ｐ_bは物体を表す画素である可能性があるとし、検出結果として両画素に論理”１”を対応づける。また、距離Ｄ₂も、しきい値Ｔ₂より大きい場合には、物体判別手段１４は、注目画素ｐ_aおよび参照画素ｐ_cは物体を表す画素である可能性があるとし、検出結果として両画素に論理”１”を対応づける。
一方、距離Ｄ₁、Ｄ₂がＴ₁、Ｔ₂より小さい場合には、各画素には論理”０”を対応づける。
【００４８】
その上で、物体判別手段１４は、これらの検出結果の論理積をとり（ステップＡ６）、そして値を判定して（ステップＡ７）、結果が論理”１”の場合は注目画素ｐ_aは物体を表す画素であるとし（ステップＡ８）、論理”０”の場合は注目画素ｐ_aは物体を表さず、背景を表す画素であるとする（ステップＡ９）。
たとえば、距離Ｄ₁がしきい値Ｔ₁より大きい場合、物体判別手段１４が各画素ｐ_a、ｐ_b、ｐ_cに対応づける値は”１”、”１”、”０”となり、また、距離Ｄ₂がしきい値Ｔ₂より大きい場合、物体判別手段１４が各画素ｐ_a、ｐ_b、ｐ_cに対応づける値は”１”、”０”、”１”となる。したがって、各画素ごとに、これらの値の論理積をとると、それぞれ”１”、”０”、”０”となり、画素ｐ_aに対する論理値のみが”１”となる。したがって、物体判別手段１４は、注目画素ｐ_aが物体を表す画素であると判断する。
【００４９】
一般に注目画素以外にＮ個の参照点をとった場合、各参照点ごとに距離Ｄ_iを計算し、各参照点ごとのしきい値Ｔ_iを用いて、［数５］が成立するか否かにより注目画素が物体を表す画素であるか否かを判別することができる。
【００５０】
【数５】

このようなステップＡ２からＳ９までの処理を、画像入力手段４が保持している全画素を次々に注目画素に設定して行うことで（ステップＡ１０）、撮影画像上の物体を検出することができる。
検出された物体は、画像出力手段１５により出力され、表示装置に表示される。その際、ステップＡ７、Ａ８において、物体を表すと判別された注目画素の表示色を、背景とは異なる色で表示すれば、画面上で検出物体の形状などをより明瞭に把握することができる。
【００５１】
なお、ステップＡ１０において、すべての画素について処理が終わったと判断された場合に、物体であると判別された画素の数をカウントして、このカウント値が予め設定されたしきい値より大きい場合にのみ物体が存在すると判断するようにしてもよい。これによって、ある大きさ以上の物体のみを検出することが可能となる。
【００５２】
一方、ステップＡ７、Ａ９で注目画素が物体を表す画素ではないと判別された場合には、背景作成手段１６は、同注目画素および対応する複数の参照画素の輝度値にもとづいて、対応する背景画像の画素の輝度値を更新し、背景画像保持手段８に保持させる（ステップＡ３）。
なお、最初の入力画像に移動物体が含まれていなければ、基本的に背景画像の各画素の輝度値は更新する必要はない。しかし、ノイズ等の影響で、背景領域の輝度値が照明変動以外の要因でばらつきを起こしうるため、物体判別手段１４により背景とみなされた画素において随時輝度値を更新していくことで誤差を抑えることができる。
【００５３】
また、このような背景画像の更新において、新旧の輝度値に重みを乗じて徐々に更新するという方法をとることができる。すなわち、物体判別手段１４で注目画素ｐ_aが背景画素とみなされた場合、［数６］のように新しく加わるデータに重みをかけ徐々に更新することができる。
【００５４】
【数６】

ここで、Ｐ_A＿ｎｅｗは注目画素ｐ_aと参照画素ｐ_b、ｐ_cに対応する背景画素の輝度値による３次元輝度値行列、Ｐ_A＿ｏｌｄは今までの注目画素ｐ_aと参照画素ｐ_b、ｐ_cに対応する背景画素の輝度値による３次元背景輝度行列である。また、Ｉ_Aは、注目画素ｐ_aと参照画素ｐ_b、ｐ_cの輝度値行列、αは上記重み定数である。
【００５５】
次に、ステップＡ１０において、すべての画素に対して前述した判別処理が終了したと判断された場合、画像入力手段４は、次の画像があるか否かを判断し（ステップＡ１１）、次の画像がある場合はステップＡ１に戻って、画像を取得し、以降、上述した各処理が繰り返される。そして、すべての画像に対して処理が終了した時点で、物体検出処理を終了する。
以上の処理を監視カメラから画像が取り込まれるごとに行うことで、物体検出を逐次行うことができる。
【００５６】
図４の（Ａ）は、道路を黒い車が走っている原画像を示す模式図、（Ｂ）は従来技術による車の検出結果を示す説明図、（Ｃ）は本実施の形態例による車の検出結果を示す説明図である。
図４の（Ｂ）は、図４の（Ａ）に示した原画像および背景画像により図８などを用いて説明した従来技術にもとづき車の検出を行った結果を示し、注目点および参照点を隣接させて物体検出を行ったため、車の輪郭部のみが検出されている。
【００５７】
これに対して、本実施の形態例による検出結果である図４の（Ｃ）では、車の輪郭部だけでなく、輪郭の内側も含め、車全体が検出されており、より明瞭に、したがって確実に車が検出されている。
本実施の形態例では、上述したように、各注目画素ごとに選択する参照画素を、注目画素に対し、物体の大きさ以上に離して設定する。したがって、注目画素と参照画素とを接近させて設定した場合のように物体の輪郭部のみが検出されるといったことがない。
そして、１つの注目画素に対して複数の参照画素を設定し、注目画素と各参照画素との組ごとに物体検出を行って、最終的に論理演算により注目画素が物体の画素か否かを判別するので、注目画素と参照画素とを離して設定しても、物体の位置を確実に特定できる。
【００５８】
また、上記従来技術と同様、輝度空間における照明変動背景モデルと輝度点との距離にもとづいて注目画素が画像を表すか否かを判定しているので、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体を検出することができる。
【００５９】
次に、本発明の第２の実施形態について説明する。
図５は第２の実施の形態例の物体検出装置２を示すブロック図、図６は図５の物体検出装置の動作を示すフローチャートである。以下では、これらの図面を参照して本発明の第２の実施の形態例の物体検出装置について説明すると同時に本発明の第２の実施の形態例の物体検出方法および物体検出プログラムを記録したコンピュータ読み取り可能な情報記録媒体について説明する。なお、図５において図１と同一の要素には同一の符号が付されており、それらに関する説明はここでは省略する。
【００６０】
図５に示した物体検出装置１８が上記物体検出装置２と異なるのは、物体自体を検出するのではなく物体の外接矩形を求める点であり、その結果、少ない計算量で物体を検出することが可能となる。
物体検出装置１８は、画像入力手段４、本発明に係わる水平参照点選択手段および垂直参照点選択手段として機能する複数参照点選択手段２０、背景画像保持手段８、本発明に係わる水平背景モデル作成手段および垂直背景モデル作成手段として機能する背景モデル作成手段２２、本発明に係わる水平距離計算手段および垂直距離計算手段として機能する距離計算手段２４、物体判別手段２６、背景作成手段２８、ならびに画像出力手段１５を含み、たとえば監視カメラにより撮影した画像から物体を検出する構成となっている。
【００６１】
物体検出装置１８は上記物体検出装置２と同様、具体的には、一例として監視カメラやビデオテープレコーダとのインターフェースなども備えたコンピュータにより構成されており、コンピュータを構成する入力装置に、本発明の物体検出プログラムを記録したコンピュータ読み取り可能な情報記録媒体としてのたとえばＣＤ−ＲＯＭを装着してＣＤ−ＲＯＭからプログラムデータを読み込み、コンピュータのメモリにロードして、コンピュータのＣＰＵを同プログラムデータにもとづいて動作させることで実現されている。
【００６２】
物体検出装置１８の複数参照点選択手段２０は、水平参照点選択手段として機能して、撮影画像において水平方向に配列された複数の画素から成る注目水平画素列ごとに、注目水平画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段４から取得する。
また、複数参照点選択手段２０は、垂直参照点選択手段として機能して、撮影画像において垂直方向に配列された複数の画素から成る注目垂直画素列ごとに、注目垂直画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段４から取得する。
【００６３】
そして、背景モデル作成手段２２は、水平背景モデル作成手段として機能して、水平参照点選択手段が選択した複数の参照画素にそれぞれ対応する、背景画像における複数の画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
また、背景モデル作成手段２２は、垂直背景モデル作成手段として機能して、垂直参照点選択手段が選択した複数の参照画素にそれぞれ対応する、背景画像における複数の画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
【００６４】
複数距離計算手段２は、水平距離計算手段として機能して、多次元輝度空間において、水平背景モデル作成手段としての背景モデル作成手段２２が作成した前記照明変動背景モデルと、水平参照点選択手段としての複数参照点選択手段２０が取得した複数の輝度値が表す点との間の距離を計算する。
また、複数距離計算手段２は、垂直距離計算手段として機能して、多次元輝度空間において、垂直背景モデル作成手段としての背景モデル作成手段２２が作成した前記照明変動背景モデルと、垂直参照点選択手段としての複数参照点選択手段２０が取得した前記複数の輝度値が表す点との間の距離を計算する。
【００６５】
物体判別手段２６は、上記水平距離計算手段および垂直距離計算手段としての複数距離計算手段２が計算した前記距離にもとづいて、注目水平画素列および注目垂直画素列に物体を表す画素が含まれているか否かを判別し、両画素列に係わる判別結果に対して論理演算を行って物体を含む矩形領域を特定する。
背景作成手段２８は、物体判別手段２６が物体を表す画素を含まないと判別した前記注目水平画素列および注目垂直画素列を構成する各画素の輝度値にもとづいて、対応する前記背景画像の画素の輝度値を更新して背景画像保持手段８に保持させる。
【００６６】
次に、このように構成された物体検出装置１８の動作について説明する。
図７は第２の実施の形態例の動作を説明するための図であって、（Ａ）は垂直方向における物体の存在可能領域の検出を表す説明図、（Ｂ）は水平方向における物体の存在可能領域の検出を表す説明図、（Ｃ）は検出結果を示す説明図である。以下では、図５とともに図６、図７を参照して説明する。
【００６７】
図６に示したように、まず、画像入力手段４は、監視カメラなどによって撮像された画像を読み込み、画像を構成する各画素ごとの輝度値を画像入力手段４内に設けられた不図示のメモリに保持する（ステップＢ１）。
次に、複数参照点選択手段２０は、撮影画像において垂直方向のアドレスｙにおける１行分の画素を注目水平画素列とし、またこの１行分の画素を参照画素（参照画素群）に選択して、各参照画素の輝度値を画像入力手段４から取得する（ステップＢ４）。したがって、撮影画像の幅がＷｉｄｔｈ画素（Ｗｉｄｔｈは正の整数）であったとすると、上記注目水平画素列にはＷｉｄｔｈ個の画素が含まれ、複数参照点選択手段２０は、本実施の形態例ではこれらすべて画素を参照画素とする。
【００６８】
そして、背景モデル作成手段２２は、複数参照点選択手段２０が選択した上記参照画素にそれぞれ対応する、背景画像における各画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
その後、複数距離計算手段２４は、参照画素の輝度値によって形成される多次元輝度空間において、背景モデル作成手段２２が作成した前記照明変動背景モデルと、複数参照点選択手段２０が取得した上記複数の輝度値が表す点との間の距離を計算する（ステップＢ６）。
【００６９】
つづいて、物体判別手段２６は、複数距離計算手段２４がこのように計算した距離をしきい値と比較して、距離がしきい値より大きい場合には、上記注目水平画素列に対応づけて論理”１”を保持し、距離がしきい値より小さい場合には論理”０”を保持する（ステップＢ８）。
複数参照点選択手段２０、背景モデル作成手段２２、ならびに複数距離計算手段２は、このような処理を、アドレスｙの値を１ずつ大きくしながら、画像の垂直方向の全範囲において行う（ステップＢ１０）。
その結果、図７の（Ａ）に示したように、物体１０８が図のように存在する場合、Ｈの範囲内の各注目水平画素列に対して、物体判別手段２６は論理”１”を保持することになる。
【００７０】
また、複数参照点選択手段２０、背景モデル作成手段２２、ならびに複数距離計算手段２は、同様の処理を、画像の水平方向においても行う。
すなわち、複数参照点選択手段２０は、撮影画像において水平方向のアドレスｘにおける１列分の画素を注目垂直画素列とし、またこの１列分の画素を参照画素（参照画素群）に選択して、各参照画素の輝度値を画像入力手段４から取得する（ステップＢ２）。したがって、撮影画像の高さがＨｅｉｇｈｔ画素（Ｈｅｉｇｈｔは正の整数）であったとすると、上記注目垂直画素列にはＨｅｉｇｈｔ個の画素が含まれ、複数参照点選択手段２０は、本実施の形態例ではこれらすべて画素を参照画素とする。
【００７１】
そして、背景モデル作成手段２２は、複数参照点選択手段２０が選択した上記参照画素にそれぞれ対応する、背景画像における各画素の輝度値を背景画像保持手段８から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
その後、複数距離計算手段２４は、参照画素の輝度値によって形成される多次元輝度空間において、背景モデル作成手段２２が作成した前記照明変動背景モデルと、複数参照点選択手段２０が取得した上記複数の輝度値が表す点との間の距離を計算する（ステップＢ５）。
【００７２】
つづいて、物体判別手段２６は、複数距離計算手段２がこのように計算した距離をしきい値と比較して、距離がしきい値より大きい場合には、上記注目垂直画素列に対応づけて論理”１”を保持し、距離がしきい値より小さい場合には論理”０”を保持する（ステップＢ７）。
複数参照点選択手段２０、背景モデル作成手段２２、ならびに複数距離計算手段２４は、このような処理を、アドレスｙの値を１ずつ大きくしながら、画像の水平方向の全範囲において行う（ステップＢ９）。
その結果、図７の（Ｂ）に示したように、Ｗの範囲内の各注目垂直画素列に対して、物体判別手段２６は論理”１”を保持することになる。
【００７３】
その後、物体判別手段２６は、各注目水平画素列および各注目垂直画素列ごとに保持している論理値の論理積をとることで（ステップＢ１１）、検出結果としての矩形領域の画像を保持するためのメモリ（図示せず）において、各画素ごとに、その値が論理”１”か論理”０”かを決定する。
詳しくは、値を決定すべき画素に対応する位置の撮影画像の画素を含む注目水平画素列および注目垂直画素列に対して、共に論理”１”を保持している場合には、上記画素の値は論理”１”に設定する。一方、対応する撮影画像の画素を含む注目水平画素列および注目垂直画素列の少なくとも一方に対して論理”０”を保持している場合には、上記画素の値は論理”０”に設定する（ステップＢ１２、Ｂ１３、Ｂ１４）。その結果、検出結果を保持する上記メモリにおいて、図７の（Ｃ）に示したように、物体を含む可能性のある矩形領域Ｒ内の画素の値は論理”１”となり、それ以外の画素の値は論理”０”となる。
【００７４】
したがって、このメモリの内容を画像出力手段１５により表示装置に表示すれば、物体を含む矩形領域が画面に表示されることになる。また、矩形領域Ｒの４辺は物体１０８の外接矩形となっている。
そして、撮影画像の幅がＷｉｄｔｈ画素、高さがＨｅｉｇｈｔ画素であった場合、上記第１の実施の形態例では、各画素ごとに背景モデルの作成や距離の計算を行うので、このような演算処理を（Ｗｉｄｔｈ×Ｈｅｉｇｈ）回行う必要がある。
しかし、第２の実施の形態例では、各水平画素列ごとに上記演算処理を行い、また各垂直画素列ごとに上記演算処理を行えばよいので、演算処理の回数は全体で（Ｗｉｄｔｈ＋Ｈｅｉｇｈｔ）回となる。したがって、演算量は第１の実施の形態例の場合にくらべ大幅に低減し、高速に物体検出を行ったり、装置の低コスト化を図る場合に非常に有利である。
【００７５】
なお、物体検出装置２の場合と同様に、背景作成手段２８は、物体判別手段２６が物体を表す画素を含まないと判別した注目水平画素列および注目垂直画素列を構成する各画素の輝度値にもとづいて、対応する背景画像の画素の輝度値を更新して背景画像保持手段８に保持させる。これにより最新の背景画像を用いて精度の高い物体検出を行うことができる。
また、処理すべき画像がさらに存在する場合には、物体検出装置２の場合と同様、以上の手順が次の画像に対して繰り返される（ステップＢ１５）。
【００７６】
なお、この第２の実施の形態例では、注目水平画素列および注目垂直画素列において、各画素列のすべての画素を参照画素にするとしたが、数画素ごとに選択するなどの間引きを行って、さらに演算量の低減を図ることも可能である。
さらに、たとえば水平方向および垂直方向のいずれか一方でまず物体の存在可能領域を確定し、その上で、その領域内でのみ他方の方向における処理を行えば、物体が存在する可能性のない領域では演算処理を行わなくても済むため、さらに演算量を削減することが可能となる。
【００７７】
なお、ここでは物体検出装置２および物体検出装置１８はコンピュータにより構成されているとしたが、物体検出装置２および物体検出装置１８を専用のハードウェアにより構成することも無論可能である。また、本発明に係わる機能のうち、一部の機能をコンピュータに所定のプログラムをロードして実現し、他の機能は専用のハードウェアによって実現したり、さらには、コンピュータにすでに組み込まれているプログラムを組み合わせて用いることで物体検出装置２および物体検出装置１８を構成とすることも可能である。
【００７８】
また、コンピュータ読み取り可能な情報記録媒体とは、上述したＣＤ−ＲＯＭに限らず、フロッピー（登録商標）ディスク、光磁気ディスク、ＲＯＭなどの可搬媒体や、コンピュータに内蔵されるハードディスクなどの記憶装置を含んでいる。
さらに、コンピュータ読み取り可能な情報記録媒体は、インターネットなどの通信ネットワークや電話回線などの通信回線を介してプログラムを送信する場合の通信線のように、短時間の間、動的にプログラムを保持するものも含み、その場合のサーバやクライアントとなるコンピュータシステム内部の揮発性メモリのように、一定時間プログラムを保持するものも含んでいる。
【００７９】
【発明の効果】
以上説明したように本発明の物体検出装置では、画像入力手段は、撮影画像の各画素の輝度値を取り込んで保持し、複数参照点選択手段は、前記画像の注目画素ごとに複数の参照画素を選択して、選択した複数の参照画素および前記注目画素の輝度値を画像入力手段から注目画素ごとに取得する。そして、背景モデル作成手段は、複数参照点選択手段が選択した注目画素および同画素に対応する前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を背景画像保持手段から取得して、背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを注目画素と、同画素に対応する各参照画素との組ごとに作成する。
【００８０】
その上で、距離計算手段は、輝度空間において、背景モデル作成手段が注目画素と参照画素との前記組ごとに作成した前記照明変動背景モデルと、複数参照点選択手段が注目画素ごとに取得した前記複数の参照画素の輝度値のそれぞれと注目画素の輝度値との組が表す点との間の距離をそれぞれ計算する。
そして、物体判別手段は、注目画素と各参照画素との組ごとに、距離計算手段が計算した前記距離にもとづいて前記注目画素が物体を表す画素か否かを判別し、前記組ごとの判別結果に対し論理演算を行って注目画素が物体を表す画素か否かを最終的に判別する。
【００８１】
このように、本発明の物体検出装置では、１つの注目画素に対して複数の参照画素を設定し、注目画素と各参照画素との組ごとに物体検出を行って、最終的に論理演算により注目画素が物体の画素か否かを判別するので、注目画素と参照画素とを離して設定しても、物体の位置を確実に特定できる。
したがって、各注目画素ごとに選択する参照画素を、注目画素に対し、物体の大きさ以上に離して設定することができ、その結果、注目画素と参照画素とを接近させて設定した場合のように物体の輪郭部のみが検出されるといったことがない。
また、上述した従来技術と同様、輝度空間における照明変動背景モデルと輝度点との距離にもとづいて注目画素が画像を表すか否かを判定するので、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体を検出することができる。
【００８２】
また、本発明の物体検出装置では、画像入力手段は撮影画像の各画素の輝度値を取り込んで保持し、水平参照点選択手段は、撮影画像において水平方向に配列された複数の画素から成る注目水平画素列ごとに、注目水平画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段から取得する。また、垂直参照点選択手段は、撮影画像において垂直方向に配列された複数の画素から成る注目垂直画素列ごとに、注目垂直画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を画像入力手段から取得する。
そして、水平背景モデル作成手段は、水平参照点選択手段が選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を背景画像保持手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成し、垂直背景モデル作成手段は、垂直参照点選択手段が選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を背景画像保持手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
【００８３】
その後、水平距離計算手段は、多次元輝度空間において、水平背景モデル作成手段が作成した前記照明変動背景モデルと、水平参照点選択手段が取得した前記複数の輝度値が表す点との間の距離を計算し、垂直距離計算手段は、多次元輝度空間において、垂直背景モデル作成手段が作成した前記照明変動背景モデルと、垂直参照点選択手段が取得した前記複数の輝度値が表す点との間の距離を計算する。
そして、物体判別手段は、水平距離計算手段および垂直距離計算手段が計算した前記距離にもとづいて、注目水平画素列および注目垂直画素列に物体を表す画素が含まれているか否かを判別し、両画素列に係わる判別結果に対して論理演算を行って物体を含む矩形領域を特定する。
【００８４】
本発明の物体検出装置では、各水平画素列ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行い、また各垂直画素列ごとに同様の演算処理を行えばよいので、演算処理の回数は全体で画像の水平方向の画素数に、画像の垂直方向の画素数を加えた値となる。
一方、各画素ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行った場合には、全体の演算処理の回数は、画像の水平方向の画素数に、画像の垂直方向の画素数を乗じたものとなる。
したがって、本発明では、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体の存在領域を検出できる上に、演算量が大幅に低減し、その結果、高速に物体検出を行ったり、装置の低コスト化を図る上で非常に有利となる。
【００８５】
本発明の物体検出方法および本発明のコンピュータ読み取り可能な情報記録媒体に記録された物体検出プログラムでは、画像入力ステップにおいて、撮影画像の各画素の輝度値を取り込んで第１の記憶手段に保持させ、複数参照点選択ステップでは、前記画像の注目画素ごとに複数の参照画素を選択して、選択した複数の参照画素および前記注目画素の輝度値を第１の記憶手段から注目画素ごとに取得する。そして、背景モデル作成ステップでは、複数参照点選択ステップで選択した注目画素および同画素に対応する前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を第２の記憶手段から取得して、背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを注目画素と、同画素に対応する各参照画素との組ごとに作成する。
【００８６】
その上で、距離計算ステップでは、輝度空間において、背景モデル作成ステップで注目画素と参照画素との前記組ごとに作成した前記照明変動背景モデルと、複数参照点選択ステップで注目画素ごとに取得した前記複数の参照画素の輝度値のそれぞれと注目画素の輝度値との組が表す点との間の距離をそれぞれ計算する。
そして、物体判別ステップでは、注目画素と各参照画素との組ごとに、距離計算ステップで計算した前記距離にもとづいて前記注目画素が物体を表す画素か否かを判別し、前記組ごとの判別結果に対し論理演算を行って注目画素が物体を表す画素か否かを最終的に判別する。
【００８７】
このように本発明では、１つの注目画素に対して複数の参照画素を設定し、注目画素と各参照画素との組ごとに物体検出を行って、最終的に論理演算により注目画素が物体の画素か否かを判別するので、注目画素と参照画素とを離して設定しても、物体の位置を確実に特定できる。
したがって、各注目画素ごとに選択する参照画素を、注目画素に対し、物体の大きさ以上に離して設定することができ、その結果、注目画素と参照画素とを接近させて設定した場合のように物体の輪郭部のみが検出されるといったことがない。
また、上述した従来技術と同様、輝度空間における照明変動背景モデルと輝度点との距離にもとづいて注目画素が画像を表すか否かを判定するので、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体を検出することができる。
【００８８】
また、本発明の物体検出方法および本発明のコンピュータ読み取り可能な情報記録媒体に記録された物体検出プログラムでは、画像入力ステップにおいて撮影画像の各画素の輝度値を取り込んで第１の記憶手段に保持させ、水平参照点選択ステップでは、撮影画像において水平方向に配列された複数の画素から成る注目水平画素列ごとに、注目水平画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を第１の記憶手段から取得する。また、垂直参照点選択ステップでは、撮影画像において垂直方向に配列された複数の画素から成る注目垂直画素列ごとに、注目垂直画素列に含まれる複数の参照画素を選択して各参照画素の輝度値を第１の記憶手段から取得する。
そして、水平背景モデル作成ステップでは、水平参照点選択ステップで選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を第１の記憶手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成し、垂直背景モデル作成ステップでは、垂直参照点選択ステップで選択した前記複数の参照画素にそれぞれ対応する、前記背景画像における複数の画素の輝度値を第２の記憶手段から取得して背景の輝度が照明変動の影響を受けて変化した場合の照明変動背景モデルを作成する。
【００８９】
その後、水平距離計算ステップでは、多次元輝度空間において、水平背景モデル作成ステップで作成した前記照明変動背景モデルと、水平参照点選択ステップで取得した前記複数の輝度値が表す点との間の距離を計算し、垂直距離計算ステップでは、多次元輝度空間において、垂直背景モデル作成ステップで作成した前記照明変動背景モデルと、垂直参照点選択ステップで取得した前記複数の輝度値が表す点との間の距離を計算する。
そして、物体判別ステップでは、水平距離計算ステップおよび垂直距離計算ステップで計算した前記距離にもとづいて、注目水平画素列および注目垂直画素列に物体を表す画素が含まれているか否かを判別し、両画素列に係わる判別結果に対して論理演算を行って物体を含む矩形領域を特定する。
【００９０】
このように、本発明では、各水平画素列ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行い、また各垂直画素列ごとに同様の演算処理を行えばよいので、演算処理の回数は全体で画像の水平方向の画素数に、画像の垂直方向の画素数を加えた値となる。
一方、各画素ごとに背景モデルの作成や、背景モデルと輝度点との間の距離の計算といった演算処理を行った場合には、全体の演算処理の回数は、画像の水平方向の画素数に、画像の垂直方向の画素数を乗じたものとなる。
したがて、本発明では、照明変動が大きく、かつ頻繁に発生する場合にも安定で、しかも高精度に物体の存在領域を検出できる上に、演算量が大幅に低減し、その結果、高速に物体検出を行ったり、装置の低コスト化を図る上で非常に有利となる。
【図面の簡単な説明】
【図１】第１の実施の形態例の物体検出装置を示すブロック図である。
【図２】図１の物体検出装置の動作を示すフローチャートである。
【図３】（Ａ）は画像面における注目点と参照点との位置関係を示す説明図、（Ｂ）は物体が撮影領域に侵入した場合の模式図、（Ｃ）は検出結果を示す模式図、（Ｄ）は画像面における注目点と参照点との位置関係を示す他の説明図、（Ｅ）は検出結果を示す他の模式図、（Ｆ）は最終的な検出結果を示す模式図である。
【図４】（Ａ）は、道路を黒い車が走っている原画像を示す模式図、（Ｂ）は従来技術による車の検出結果を示す説明図、（Ｃ）は本実施の形態例による車の検出結果を示す説明図である。
【図５】第２の実施の形態例の物体検出装置を示すブロック図である。
【図６】図５の物体検出装置の動作を示すフローチャートである。
【図７】第２の実施の形態例の動作を説明するための図であって、（Ａ）は垂直方向における物体の存在可能領域の検出を表す説明図、（Ｂ）は水平方向における物体の存在可能領域の検出を表す説明図、（Ｃ）は検出結果を示す説明図である。
【図８】背景画像を表す説明図である。
【図９】注目点および参照点の輝度値により形成される２次元の輝度空間を示す説明図である。
【図１０】監視カメラの撮影領域に物体が侵入した場合の撮影画像を示す説明図である。
【図１１】（Ａ）は画像面を示す説明図、（Ｂ）は物体が撮影領域に侵入した場合の模式図、（Ｃ）は検出結果を示す模式図である。
【符号の説明】
２……物体検出装置、４……画像入力手段、６……複数参照点選択手段、８……背景画像保持手段、１０……背景モデル作成手段、１２……距離計算手段、１４……物体判別手段、１６……背景作成手段、１８……物体検出装置、２０……複数参照点選択手段、２２……背景モデル作成手段、２４……距離計算手段、２６……物体判別手段、２８……背景作成手段、１０２……白ドット、１０４……直線、１０６……白丸ドット、１０８……物体、１１０……黒ドット。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a computer-readable information recording medium on which an object detection program for detecting an object included in an image taken by a surveillance camera or the like is recorded.
[0002]
[Prior art]
When detecting an object from a moving image taken by a surveillance camera or the like, it is important that the object can be reliably detected even if the brightness of the background fluctuates due to sunshine conditions or the like. Therefore, the inventors of the present invention have devised an object detection technique that is not easily affected even when the illumination fluctuation is large and occurs frequently, and has already filed a patent application (Japanese Patent Application No. 11-078641). ).
[0003]
This object detection technique is to provide a reference point at a position different from the point of interest in the background image created in advance and the current image, and discriminate the object from the correlation information of the luminance value change of the reference point. . Hereinafter, the principle will be described in detail with reference to the drawings.
FIG. 8 is an explanatory diagram showing a background image, in which an X axis is set in the horizontal direction and a Y axis is set in the vertical direction. However, in order to avoid unnecessarily complicated, it is assumed that the background image has no texture and has a constant brightness regardless of the location. In FIG. 8, P _A Is the point of interest, P _B Is a reference point provided in the vicinity thereof. And attention point P _A The luminance value at _A , Reference point P _B The luminance value at _B And
[0004]
FIG. 9 is an explanatory diagram showing a two-dimensional luminance space formed by the luminance values of the attention point and the reference point. In this luminance space, a two-dimensional coordinate system is set, and the horizontal axis indicates the attention point P. _A Luminance value I _A The vertical axis represents the reference point P _B Luminance value I _B Represents.
Where attention point P _A The brightness value of I is I _A1 , Reference point P _B The brightness value of I is I _B1 , The point on the luminance space corresponding to the set of luminance values, that is, the luminance point (I _A1 , I _B1 ) Is, for example, the position of the white dot 102 in FIG.
Then, the brightness of the background changes, and the attention point P _A For example, if the luminance value of the reference point P becomes small, the reference point P _B The luminance value of becomes smaller at the same rate. In addition, attention point P _A If the luminance value of is 0, the reference point P _B The luminance value is also 0. Therefore, attention point P _A And reference point P _B Brightness I _A , I _B When the brightness of the background changes, the point on the luminance space corresponding to the set of moves along the straight line 104 shown in FIG. Here, such a straight line is called an illumination variation background model.
[0005]
On the other hand, for example, the attention point P _A If an object has entered the position of _A The brightness value of I is I _A2 In this case, the luminance point (I _A2 , I _B1 ) Is, for example, the position of the white circle dot 106. Therefore, the distance 107 between such a luminance point and the straight line 104 is examined, and when the luminance point is away from the straight line 104, it can be determined that the object has entered the position of the target point.
And illumination fluctuation occurs, for example, the reference point P _B If the luminance value of the object increases, the point of interest P on the object is usually _A Similarly, the luminance value of the luminance point (I _A2 , I _B1 ) Generally moves in the upper right direction (arrow A) parallel to the straight line 104, and the distance to the straight line 104 does not change much. Therefore, an object can be reliably detected even if illumination fluctuation occurs.
[0006]
[Problems to be solved by the invention]
However, in the above prior art, when the reference point is arranged close to the target point, only the contour portion of the object is extracted, and the extracted object image is in a hollow state, and there is room for improvement in this respect. It was.
Hereinafter, a specific description will be given with reference to FIG. FIG. 10 is an explanatory diagram illustrating a captured image when an object enters the imaging region of the surveillance camera, for example. As in the case of FIG. 8, the background is assumed to have a constant brightness regardless of the location.
For this image, as shown in FIG. _A And set the reference point P close to _B 10, these points are all on the object 108 in the state of FIG. 10, and the luminance of both points decreases to the same extent. _A And reference point P _B Corresponds to the luminance point (I _A , I _B ) Is the position of the black dot 110 on the straight line 104 shown in FIG. Since the distance between the black dot 110 and the straight line 104 is zero, in this case, the attention point P _A Is determined not to be on the object, and the point of interest P _A Although the object is on the object 108, the object cannot be detected.
[0007]
On the other hand, attention point P _A And reference point P _B Are set at similar intervals, for example, at the edge of the object 108, for example, the attention point P _A Is on the object 108 and the reference point P _B When each point is set at a position on the background, a luminance difference occurs between the two points, and the corresponding luminance point in FIG. 9 is located away from the straight line 104, so that the object 108 can be detected. .
Therefore, in the technique of Japanese Patent Application No. 11-078641, when the attention point and the reference line are arranged close to each other, only the edge portion of the object, that is, the contour portion is detected.
[0008]
Therefore, in order to detect not only the outline of the object but also the whole, it is necessary to dispose the reference point sufficiently away from the attention point.
In the technique of Japanese Patent Application No. 11-078641, the reference point must be set outside the area where the object may enter. This will be described with reference to FIG. 11A is an explanatory diagram showing an image plane, FIG. 11B is a schematic diagram when an object has entered the imaging region, and FIG. 11C is a schematic diagram showing a detection result.
[0009]
As shown in FIG. 11A, the attention point P is displayed on the image. _A And attention point P _B Are set apart from each other, as shown in FIG. _A The point of interest P _A And reference point P _B A difference in brightness occurs between the two and the object 108 is detected. However, the object 108 is at the reference point P _B Since the same luminance difference also occurs when entering the position, the object 108 is detected also in this case.
Therefore, attention point P _A When the object 108 enters and is detected, the object 108 _A Or the reference point P _B Cannot be specified as a result, and as a result, the point of interest P _A Although the object 108 exists only at the position of, the object image 109 is generated so that the object 108 exists at both positions as shown in FIG.
[0010]
Therefore, in order to avoid such a phenomenon, the reference point must be set outside the area where the object may enter.
The present invention has been made to eliminate such drawbacks. The purpose of the present invention is to stably detect an object with high accuracy and to detect an object with high accuracy and to detect a reference point. It is an object to provide an object detection apparatus, an object detection method, and a computer-readable information recording medium on which an object detection program is recorded, which can be set at an arbitrary position and reliably detect an object.
[0011]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides an object detection apparatus that detects an object using a captured image, an image input unit that captures and holds a luminance value of each pixel of the captured image, A plurality of reference pixels for each pixel of interest, a plurality of selected reference pixels and a plurality of reference point selection means for acquiring the luminance value of the pixel of interest for each pixel of interest from the image input means; A background image holding unit that holds a luminance value of each pixel of a background image that forms a background; and the background corresponding to the target pixel selected by the plurality of reference point selection units and the plurality of reference pixels corresponding to the pixel. The luminance value of a plurality of pixels in the image is acquired from the background image holding means, and an illumination variation background model when the background luminance is changed by the influence of illumination variation A background model creating means for creating a set of each reference pixel corresponding to each of the illumination variation background models created by the background model creating means for each set of the target pixel and the reference pixel in a luminance space; Distance calculating means for calculating the distance between each of the plurality of reference pixel brightness values acquired for each target pixel by the plurality of reference point selecting means and a point represented by the set of the brightness value of the target pixel; For each set of the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculation means, and a logical result is obtained for the determination result for each set. And an object discriminating means for finally discriminating whether or not the pixel of interest is a pixel representing an object by performing an operation.
[0012]
In the object detection device of the present invention, the image input means captures and holds the luminance value of each pixel of the captured image, and the plurality of reference point selection means selects a plurality of reference pixels for each target pixel of the image, A plurality of selected reference pixels and luminance values of the target pixel are acquired for each target pixel from the image input means. Then, the background model creating means outputs the luminance values of the plurality of pixels in the background image respectively corresponding to the target pixel selected by the plurality of reference point selecting means and the plurality of reference pixels corresponding to the same pixel from the background image holding means. The illumination variation background model is acquired for each set of the pixel of interest and each reference pixel corresponding to the pixel when the background luminance changes due to the illumination variation.
[0013]
In addition, in the luminance space, the distance calculation unit acquires the illumination variation background model created by the background model creation unit for each pair of the target pixel and the reference pixel, and the plurality of reference point selection units acquires the target pixel for each target pixel. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
The object determining unit determines, for each pair of the target pixel and each reference pixel, whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculating unit, and determines each group A logical operation is performed on the result to finally determine whether the target pixel is a pixel representing an object.
[0014]
In addition, the present invention is an object detection device that detects an object using a captured image, and includes an image input unit that captures and holds a luminance value of each pixel of the captured image, and is arranged in the horizontal direction in the captured image. A horizontal reference point selection unit that selects a plurality of reference pixels included in the target horizontal pixel column and obtains a luminance value of each reference pixel from the image input unit for each horizontal pixel column of interest composed of a plurality of pixels; For each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the photographed image, a plurality of reference pixels included in the target vertical pixel column are selected, and the luminance value of each reference pixel is obtained from the image input unit. A vertical reference point selection means to obtain, a background image holding means for holding the luminance value of each pixel of the image constituting the background of the photographed image, and the plurality of reference pixels selected by the horizontal reference point selection means; A horizontal background model creation that obtains brightness values of a plurality of pixels in the background image corresponding to the background image from the background image holding unit and creates a lighting fluctuation background model when the brightness of the background changes under the influence of lighting fluctuations And luminance values of a plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected by the vertical reference point selection unit are obtained from the background image holding unit, and the luminance of the background is affected by illumination variation A vertical background model creating means for creating a lighting fluctuation background model when changed by receiving, the lighting fluctuation background model created by the horizontal background model creating means in a multidimensional luminance space, and the horizontal reference point selecting means. Horizontal distance calculation means for calculating a distance between the points represented by the acquired plurality of luminance values, and the vertical background model creation means in a multidimensional luminance space Vertical distance calculation means for calculating the distance between the created illumination variation background model and the points represented by the plurality of luminance values acquired by the vertical reference point selection means, the horizontal distance calculation means, and the vertical distance calculation Based on the distance calculated by the means, it is determined whether or not a pixel representing an object is included in the target horizontal pixel column and the target vertical pixel column, and a logical operation is performed on a determination result related to both pixel columns. And an object discriminating means for specifying a rectangular area including the object.
[0015]
In the object detection device of the present invention, the image input unit captures and holds the luminance value of each pixel of the captured image, and the horizontal reference point selection unit includes the horizontal pixel of interest composed of a plurality of pixels arranged in the horizontal direction in the captured image. For each column, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is acquired from the image input unit. Further, the vertical reference point selection unit selects a plurality of reference pixels included in the target vertical pixel column for each target vertical pixel column including a plurality of pixels arranged in the vertical direction in the captured image, and brightness of each reference pixel. The value is acquired from the image input means.
Then, the horizontal background model creation means acquires the brightness values of the plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selection means from the background image holding means, and the background brightness is An illumination variation background model in the case of changing under the influence of illumination variation is created, and the vertical background model creation means is a plurality of reference pixels selected by the vertical reference point selection means, each of which corresponds to the plurality of reference pixels in the background image. A luminance value background model is created when the luminance value of the pixel is obtained from the background image holding means and the luminance of the background changes due to the influence of the illumination variation.
[0016]
Thereafter, in a multi-dimensional luminance space, the horizontal distance calculation unit is a distance between the illumination variation background model created by the horizontal background model creation unit and the point represented by the plurality of luminance values acquired by the horizontal reference point selection unit. The vertical distance calculation means is a multi-dimensional luminance space between the illumination variation background model created by the vertical background model creation means and the points represented by the plurality of brightness values obtained by the vertical reference point selection means. Calculate the distance.
Then, the object determining means determines whether or not a pixel representing the object is included in the target horizontal pixel column and the target vertical pixel column based on the distance calculated by the horizontal distance calculating unit and the vertical distance calculating unit, A logical operation is performed on the discrimination results relating to both pixel columns to identify a rectangular area including the object.
[0017]
The present invention is also an object detection method for detecting an object using a captured image, an image input step of capturing a luminance value of each pixel of the captured image and holding it in a first storage means, and the captured image Selecting a plurality of reference pixels for each target pixel, and acquiring a plurality of selected reference pixels and luminance values of the target pixel for each target pixel from the first storage unit; and The background image holding step of holding the luminance value of each pixel of the background image forming the background of the photographed image in the second storage means, and the plurality of pixels corresponding to the pixel of interest selected in the plurality of reference point selecting step and the same pixel The luminance value of the plurality of pixels in the background image respectively corresponding to the reference pixel is acquired from the second storage means, and the illumination change when the background luminance changes due to the influence of the illumination change. A background model creating step for creating a background model for each set of the target pixel and each reference pixel corresponding to the pixel, and for each set of the target pixel and the reference pixel in the background model creating step in a luminance space Between the illumination variation background model created in the above and a point represented by a set of each of the luminance values of the plurality of reference pixels and the luminance value of the pixel of interest acquired for each pixel of interest in the plurality of reference point selection step A distance calculation step for calculating the distance of each, and for each set of the target pixel and each reference pixel, it is determined whether the target pixel is a pixel representing an object based on the distance calculated in the distance calculation step. And an object determination step of finally determining whether or not the target pixel is a pixel representing an object by performing a logical operation on the determination result for each group.
[0018]
The present invention is also a computer-readable information recording medium in which an object detection program for detecting an object using a photographed image is recorded. The brightness value of each pixel of the photographed image is taken into the first storage means. An image input step to hold, a plurality of reference pixels for each target pixel of the captured image, and a plurality of selected reference pixels and luminance values of the target pixel for each target pixel from the first storage unit A plurality of reference point selection steps to be acquired; a background image holding step for holding a luminance value of each pixel of the background image forming the background of the photographed image in a second storage unit; and the attention selected in the plurality of reference point selection steps Obtaining luminance values of a plurality of pixels in the background image respectively corresponding to the pixel and the plurality of reference pixels corresponding to the pixel from the second storage unit; In a luminance space, a background model creating step for creating a lighting variation background model for each set of the target pixel and each reference pixel corresponding to the pixel when the luminance of the scene changes due to the influence of illumination variation, The illumination variation background model created for each pair of the target pixel and reference pixel in the background model creation step, and the luminance values of the plurality of reference pixels acquired for each target pixel in the multiple reference point selection step A distance calculating step for calculating a distance between each and a point represented by a set of luminance values of the target pixel, and the distance calculated in the distance calculating step for each set of the target pixel and each reference pixel Based on this, it is determined whether or not the target pixel is a pixel representing an object, and a logical operation is performed on the determination result for each set to finally determine whether or not the target pixel is a pixel representing an object Characterized in that it comprises a object determination step of determining.
[0019]
In the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the photographed image is captured and held in the first storage means. In the multiple reference point selection step, a plurality of reference pixels are selected for each target pixel of the image, and the selected plurality of reference pixels and the luminance value of the target pixel are acquired from the first storage unit for each target pixel. . In the background model creation step, the second storage means stores the luminance values of the plurality of pixels in the background image respectively corresponding to the target pixel selected in the plurality of reference point selection step and the plurality of reference pixels corresponding to the pixel. Thus, an illumination variation background model when the background luminance changes under the influence of illumination variation is created for each set of the target pixel and each reference pixel corresponding to the pixel.
[0020]
In addition, in the distance calculation step, in the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel in the background model creation step and the target reference pixel acquired in the multiple reference point selection step The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
In the object determining step, for each set of the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated in the distance calculating step. A logical operation is performed on the result to finally determine whether the target pixel is a pixel representing an object.
[0021]
The present invention is also a computer-readable information recording medium in which an object detection program for detecting an object using a photographed image is recorded. The brightness value of each pixel of the photographed image is taken into the first storage means. An image input step to be held, and for each horizontal pixel column of interest consisting of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the horizontal pixel column of interest are selected and the luminance of each reference pixel A horizontal reference point selecting step for acquiring a value from the first storage means, and a plurality of pixels included in the target vertical pixel column for each target vertical pixel column including a plurality of pixels arranged in the vertical direction in the captured image. A vertical reference point selection step of selecting a reference pixel and acquiring the luminance value of each reference pixel from the first storage means; and the luminance of each pixel of the image constituting the background of the photographed image Are stored in the second storage means, and brightness values of a plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are stored in the second storage. A horizontal background model creating step for creating an illumination variation background model obtained when the background luminance is changed by the influence of illumination variation obtained from the means, and each of the plurality of reference pixels selected in the vertical reference point selection step. A vertical background model creation that obtains corresponding brightness values of a plurality of pixels in the background image from the second storage means and creates a lighting fluctuation background model when the brightness of the background changes under the influence of lighting fluctuations And in the multi-dimensional luminance space, the illumination variation background model created in the horizontal background model creation step, and the horizontal reference point selection step. A horizontal distance calculating step for calculating a distance between the points represented by the plurality of luminance values acquired in step (a), a lighting variation background model created in the vertical background model creating step in a multidimensional luminance space, and the vertical Based on the vertical distance calculation step for calculating the distance between the points represented by the plurality of luminance values acquired in the reference point selection step, and the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, An object that determines whether or not a pixel representing an object is included in the horizontal pixel column of interest and the vertical pixel column of interest and performs a logical operation on the determination results relating to both pixel columns to identify a rectangular region including the object A discriminating step.
[0022]
The present invention is also an object detection method for detecting an object using a captured image, an image input step of capturing a luminance value of each pixel of the captured image and holding it in a first storage means, and the captured image For each horizontal pixel column of interest composed of a plurality of pixels arranged in the horizontal direction in FIG. 1, a plurality of reference pixels included in the horizontal pixel column of interest are selected and the luminance value of each reference pixel is obtained from the first storage means And selecting a plurality of reference pixels included in the target vertical pixel column for each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, and selecting each reference pixel. A vertical reference point selection step of acquiring a luminance value from the first storage unit, a background image holding step of holding a luminance value of each pixel of an image forming a background of the captured image in a second storage unit, The luminance values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are obtained from the second storage means, and the background luminance is affected by illumination fluctuations. A horizontal background model creating step for creating a lighting variation background model when the light source changes, and luminance values of a plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the vertical reference point selecting step. A vertical background model creating step for creating a lighting fluctuation background model obtained when the background luminance is changed by the influence of the lighting fluctuation obtained from the second storage means, and the horizontal background model creating step in a multi-dimensional luminance space; The distance between the illumination variation background model created in step 1 and the point represented by the plurality of luminance values acquired in the horizontal reference point selection step is measured. Horizontal distance calculating step, in a multidimensional luminance space, between the illumination variation background model created in the vertical background model creation step, and the points represented by the plurality of luminance values obtained in the vertical reference point selection step A vertical distance calculating step for calculating a distance; and a pixel representing an object in the horizontal pixel column of interest and the vertical pixel column of interest based on the distance calculated in the horizontal distance calculating step and the vertical distance calculating step. And an object determination step of performing a logical operation on the determination results relating to both pixel columns to identify a rectangular area including the object.
[0023]
In the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the captured image is captured and held in the first storage means, In the horizontal reference point selection step, for each target horizontal pixel column composed of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is determined. Obtained from the first storage means. Further, in the vertical reference point selection step, a plurality of reference pixels included in the target vertical pixel column are selected for each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, and the luminance of each reference pixel is selected. A value is obtained from the first storage means.
In the horizontal background model creation step, the luminance values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selection step are obtained from the first storage means, and the background luminance is obtained. A lighting fluctuation background model is created when the lighting changes due to the influence of lighting fluctuation, and in the vertical background model creation step, a plurality of background pixels in the background image respectively corresponding to the plurality of reference pixels selected in the vertical reference point selection step The luminance value of the pixel is obtained from the second storage means, and an illumination variation background model is created when the background luminance changes under the influence of illumination variation.
[0024]
Thereafter, in the horizontal distance calculation step, in a multidimensional luminance space, the distance between the illumination variation background model created in the horizontal background model creation step and the point represented by the plurality of luminance values acquired in the horizontal reference point selection step In the vertical distance calculation step, in the multidimensional luminance space, between the illumination variation background model created in the vertical background model creation step and the points represented by the plurality of luminance values obtained in the vertical reference point selection step Calculate the distance.
Then, in the object determination step, based on the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, it is determined whether or not the pixel representing the object is included in the target horizontal pixel column and the target vertical pixel column, A logical operation is performed on the discrimination results relating to both pixel columns to identify a rectangular area including the object.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
First, the principle of object detection in the first embodiment will be described. 3A is an explanatory diagram showing the positional relationship between the attention point and the reference point on the image plane, FIG. 3B is a schematic diagram when an object has entered the imaging region, and FIG. 3C is a schematic diagram showing a detection result. , (D) is another explanatory diagram showing the positional relationship between the attention point and the reference point on the image plane, (E) is another schematic diagram showing the detection result, and (F) is a schematic diagram showing the final detection result. It is.
[0026]
As shown in FIG. 3A, the attention point P is displayed on the image. _A And attention point P _B Are set apart from each other, as shown in FIG. _A The point of interest P _A And reference point P _B Therefore, the object 108 is detected by performing detection by the same method as the conventional one using the illumination variation background model. Then, as shown in FIG. 3C, the detection result is the attention point P as in the conventional case. _A And reference point P _B This indicates that there is a possibility that the object 108 exists in both.
[0027]
On the other hand, in this embodiment, as shown in FIG. _B Reference point P different from _C Set further. And attention point P _A And reference point P _C The object is detected by the same method as the conventional method. As a result, as described above, the object 108 becomes the attention point P. _A In the detection result, as shown in (E) of FIG. _A And reference point P _C This indicates that there is a possibility that the object 108 exists in both of the positions.
[0028]
In the present embodiment, the two detection results, that is, the logical product of (C) and (E) in FIG. Therefore, as shown in (F) of FIG. 3, the attention point P that exists in common in both (C) and (E) of FIG. _A Only the object image 111 at the position of _A The object 108 that has entered the area is reliably detected by specifying its position. In the present embodiment, the point of interest and the reference point can be arranged at a certain distance, so that the image of the detected object is an image in which not only the contour portion but also the inner region is drawn.
[0029]
In general, when N different reference points including an attention point (N is an integer of 3 or more) are set for one attention point, an object existence region is determined by satisfying the condition expressed by the following equation. Can be identified.
[0030]
[Expression 1]
dist (Pi, Pj)> ObjectSize
However, 1 ≦ i, j ≦ N (i ≠ j), and dist (A, B) represents the distance between points A and B. ObjectSize represents the size of the object on the image.
[0031]
Next, a first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing an object detection apparatus according to the first embodiment, and FIG. 2 is a flowchart showing an operation of the object detection apparatus of FIG. Hereinafter, an example of the object detection apparatus of the present invention will be described with reference to these drawings, and at the same time, an embodiment of a computer-readable information recording medium recording the object detection method and the object detection program of the present invention will be described. .
[0032]
1 includes an image input unit 4, a plurality of reference point selection unit 6, a background image holding unit 8, a background model creation unit 10, a distance calculation unit 12, an object discrimination unit 14, and a background creation unit 16. For example, an object is detected from an image taken by a surveillance camera.
Specifically, the object detection device 2 includes, for example, an interface with a surveillance camera and a video tape recorder, and is configured by a computer in which a predetermined operating system is incorporated. For example, a CD-ROM as a computer-readable information recording medium in which the object detection program of the invention is recorded is loaded, the program data is read from the CD-ROM, loaded into the memory of the computer, and the CPU of the computer is converted into the program data. It is realized by operating based on it.
[0033]
The image input means 4 captures and holds an image captured by the surveillance camera or an image recorded on the video tape recorder after the capture from the surveillance camera or VTR. More specifically, the image input unit 4 captures and holds the luminance value of each pixel of the captured image. The image captured by the image input unit 4 may be a moving image or a still image. Further, the image input means 4 may be configured to include a camera that captures an image.
[0034]
The plurality of reference point selection means 6 selects a plurality of predetermined reference pixels (pixels at the position of the reference point) for each target pixel (pixel at the position of the attention point) of the image, and selects the plurality of selected references. Each luminance value of the pixel and the target pixel is acquired from the image input unit 4 for each target pixel. The plurality of reference pixels corresponding to one target pixel is also referred to as a reference pixel group.
The background image holding means 8 holds the luminance value of each pixel of the background image in the shooting area. In this embodiment, the luminance value held by the background image holding unit 8 is appropriately updated by the background creating unit 16 as will be described later.
The background model creation means 10 is a background image holding means 8 for the luminance values of a plurality of pixels in the background image respectively corresponding to the target pixel selected by the plurality of reference point selection means 6 and the plurality of reference pixels corresponding to the same pixel. Thus, an illumination variation background model when the background luminance changes under the influence of illumination variation is created for each set of the target pixel and each reference pixel corresponding to the pixel.
[0035]
Then, the distance calculation unit 12 acquires, in the luminance space, the illumination variation background model created by the background model creation unit 10 for each pair of the target pixel and the reference pixel, and the multiple reference point selection unit 6 for each target pixel. The distance between each of the brightness values of the plurality of reference pixels and the point represented by the set of the brightness value of the target pixel is calculated.
[0036]
The object discriminating unit 14 discriminates whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculation unit 12 for each set of the target pixel and each reference pixel, and the target pixel and each reference pixel. A logical operation is performed on the discrimination result for each pair with the pixel to finally discriminate whether or not the target pixel is a pixel representing an object.
Then, the image output means 15 outputs the image data of the discrimination result output from the object discrimination means 14 to a display device (not shown).
Further, the background creating unit 16 determines the luminance value of the pixel of the background image corresponding to each of the target pixel determined by the object determining unit 14 not to be a pixel representing the object and each of the plurality of reference pixels corresponding to the pixel. Based on the luminance value of the target pixel and the reference pixel, it is updated and held in the background image holding means 8.
[0037]
Next, the operation of the first embodiment configured as described above will be described.
As shown in FIG. 2, first, the image input unit 4 of the object detection device 2 reads an image captured by a surveillance camera or the like, and provides a luminance value for each pixel constituting the image in the image input unit 4. The data is held in a memory (not shown) (step A1).
The image is composed of a large number of pixels arranged in a matrix, and the image input means 4 holds the luminance values of such pixels as two-dimensional array data. Each pixel can be specified by designating the x address and y address of the memory. The x address corresponds to the column number of the matrix, and the y address corresponds to the row number of the matrix.
[0038]
Then, the plurality of reference point selection means 6 selects a plurality of reference pixels determined in advance for each target pixel of the image, and sets the luminance values of the selected reference pixels and target pixel of the image input means 4. Each pixel of interest is acquired from the memory (step A2).
Each of the reference pixels is set at a position away from the target pixel by a distance larger than the size of the intruding object to be detected.
For example, if the size of the intruding object on the screen is less than 100 pixels, the current pixel point of interest P _A And another pixel point P different from this pixel _B Is used as a reference point, as shown in FIG. _B Is the point of interest P _A Is set at a position 100 pixels away in the x-axis direction. At this time, the luminance value at the attention point PA can be expressed as I (x, y), and the luminance value at the reference point PB can be expressed as I (x + 100, y).
Similarly, another reference point P _C , The reference point P is set as shown in FIG. _C Is set at a position 100 pixels away from the point of interest PA in the direction opposite to the x-axis. Reference point P _C The luminance value at can be expressed as I (x-100, y).
[0039]
In the present embodiment, the multiple reference point selection means 6 _A In addition to the attention point, two reference points P _B , P _C Therefore, one attention pixel and two reference pixels respectively corresponding to the attention point and each reference point are set. Next, the multiple reference point selection unit 6 supplies the luminance values of the target pixel and the reference pixel obtained in step A2 to the background creation unit 16 (step A3). At this stage, since the luminance value of each pixel constituting the background image is not yet stored in the memory constituting the background image holding means 8, the background creating means 16 is supplied from the plurality of reference point selecting means 6. Each luminance value is held in the background image holding means 8 as the luminance value of the pixel of the corresponding background image.
[0040]
Next, the background model creating unit 10 creates an illumination variation background model using the luminance value of each pixel of the background image held by the background image holding unit 8.
For example, the target pixel is the target point P in FIG. _A P corresponding to _a And the reference pixel is the reference point P _B , P _C P corresponding to _b , P _c If so, the background model creation means 10 first starts with the pixel of interest p _a And reference pixel p _b The luminance values of the two pixels in the background image corresponding to are acquired from the background image holding means 8.
[0041]
Then, using these two luminance values, the target pixel p _a And reference pixel p _b As a pair corresponding to the above, a lighting fluctuation background model is created in the case where the luminance of the background changes under the influence of lighting fluctuation. Specifically, the target pixel p as shown in FIG. _a The luminance value of the background pixel corresponding to the horizontal axis, the reference pixel p _b In the luminance space with the vertical axis representing the luminance value of the background pixel corresponding to _a And reference pixel p _b A point represented by a set of luminance values of background pixels corresponding to and a straight line passing through the origin are obtained, and the straight line is used as the illumination variation background model.
The background model creation means 10 uses the pixel of interest p _a And reference pixel p _c Similarly, a lighting variation background model is created for the pair.
[0042]
If the number of pixels to be referred to for each target pixel is N including the target pixel, the luminance values of the N background image pixels are used for each target pixel, and therefore, for each target pixel. A set of background luminance values can be represented as an N-dimensional vector.
In this embodiment, since two sets of the target pixel and one reference point are used, the background luminance values are two sets of two-dimensional vectors.
[0043]
Thereafter, the distance calculation means 12 uses the pixel of interest p _a And reference pixel p _b In the luminance space, the illumination variation background model and the pixel of interest p _a And reference pixel p _b The distance between the point represented by the set of luminance values of the pixel of interest and the pixel of interest p _a And reference pixel p _c The same distance is calculated for the set of (step A4).
[0044]
Here, the target pixel p _a , Reference pixel p _b , P _c The luminance values of the background pixels corresponding to ia0, i _b0 , I _c0 And the pixel of interest p _a , Reference pixel p _b , P _c The luminance value of i _a1 , I _b1 , I _c1 Then, the target pixel p _a And reference pixel p _b The distance D1 between the illumination variation background model corresponding to the pair and the points corresponding to the luminance values ia1 and ib1, and the target pixel p _a And reference pixel p _c The distance D2 between the illumination variation background model corresponding to the pair and the points corresponding to the luminance values ia1 and ic1 can be calculated by the equations shown in [Expression 2] and [Expression 4], respectively.
[0045]
[Expression 2]

[0046]
[Equation 3]

Next, the object discriminating unit 14 determines the distance D obtained in step A4. ₁ , D ₂ And a preset threshold T ₁ , T ₂ (Step A5), and when the distance and threshold value satisfy [Equation 4], the target pixel p _a Is a pixel representing an object, and it is determined that an object is present at the corresponding location. On the other hand, if [Expression 4] is not satisfied, the pixel of interest p _a Is determined to be a pixel representing the background (steps A6, S7, S8, S9).
[0047]
[Expression 4]

More specifically, the object determination unit 14 calculates the distance D calculated by the multiple distance calculation unit. ₁ , D ₂ Are the threshold values T ₁ , T ₂ It is checked whether or not it is larger (step A5). For example, distance D ₁ Is the threshold T ₁ If larger, the object discriminating means 14 determines the target pixel p _a And reference pixel p _b May be a pixel representing an object, and a logical “1” is associated with both pixels as a detection result. Also, distance D ₂ Threshold value T ₂ In the case of being larger, the object discrimination means 14 determines the target pixel p _a And reference pixel p _c May be a pixel representing an object, and a logical “1” is associated with both pixels as a detection result.
On the other hand, distance D ₁ , D ₂ Is T ₁ , T ₂ If it is smaller, each pixel is associated with logic “0”.
[0048]
After that, the object discriminating means 14 calculates the logical product of these detection results (step A6), determines the value (step A7), and if the result is logic “1”, the object pixel p _a Is a pixel representing an object (step A8), and in the case of logic "0", the target pixel p _a Are pixels that do not represent an object but represent the background (step A9).
For example, distance D ₁ Is the threshold T ₁ If larger, the object discriminating means 14 determines each pixel p _a , P _b , P _c The values associated with are “1”, “1”, “0”, and the distance D ₂ Is the threshold T ₂ If larger, the object discriminating means 14 determines each pixel p _a , P _b , P _c The values associated with are “1”, “0”, and “1”. Therefore, for each pixel, the logical product of these values is “1”, “0”, “0”, and the pixel p _a Only the logical value for is “1”. Therefore, the object discrimination means 14 _a Are pixels representing an object.
[0049]
In general, when N reference points are taken in addition to the target pixel, the distance D for each reference point _i And the threshold T for each reference point _i Can be used to determine whether the pixel of interest is a pixel representing an object based on whether [Equation 5] holds.
[0050]
[Equation 5]

By performing such processing from step A2 to S9 by sequentially setting all the pixels held by the image input means 4 as the target pixel (step A10), an object on the captured image can be detected. it can.
The detected object is output by the image output means 15 and displayed on the display device. At this time, if the display color of the pixel of interest determined to represent the object in steps A7 and A8 is displayed in a color different from the background, the shape of the detected object can be grasped more clearly on the screen. .
[0051]
When it is determined in step A10 that the processing has been completed for all the pixels, the number of pixels determined to be an object is counted, and this count value is greater than a preset threshold value. It may be determined that only the object exists. As a result, it is possible to detect only objects having a certain size or more.
[0052]
On the other hand, if it is determined in steps A7 and A9 that the pixel of interest is not a pixel representing an object, the background creating means 16 determines the corresponding background based on the luminance values of the pixel of interest and the corresponding reference pixels. The luminance value of the pixel of the image is updated and held in the background image holding means 8 (step A3).
If no moving object is included in the first input image, basically, it is not necessary to update the luminance value of each pixel of the background image. However, since the luminance value of the background region can vary due to factors other than illumination fluctuations due to the influence of noise or the like, the error can be corrected by updating the luminance value at any time in the pixels regarded as the background by the object discrimination means 14. Can be suppressed.
[0053]
In addition, in such update of the background image, it is possible to take a method of gradually updating the old and new luminance values by multiplying them with weights. That is, the target pixel p is detected by the object discrimination means 14. _a Is regarded as a background pixel, the newly added data can be weighted and gradually updated as shown in [Formula 6].
[0054]
[Formula 6]

Where P _A _New is the pixel of interest p _a And reference pixel p _b , P _c A three-dimensional luminance value matrix by luminance values of background pixels corresponding to P, _A _Old is the current pixel of interest p _a And reference pixel p _b , P _c Is a three-dimensional background luminance matrix based on luminance values of background pixels corresponding to. I _A Is the pixel of interest p _a And reference pixel p _b , P _c Is a weighting constant.
[0055]
Next, when it is determined in step A10 that the above-described determination processing has been completed for all pixels, the image input unit 4 determines whether there is a next image (step A11), and the next If there is an image, the process returns to step A1 to acquire the image. Thereafter, the above-described processes are repeated. Then, when the processing is completed for all the images, the object detection processing is ended.
By performing the above processing every time an image is captured from the monitoring camera, object detection can be performed sequentially.
[0056]
4A is a schematic diagram showing an original image of a black car running on a road, FIG. 4B is an explanatory diagram showing a detection result of a car according to the prior art, and FIG. 4C is a car according to the present embodiment. It is explanatory drawing which shows the detection result of.
FIG. 4B shows the result of detection of a vehicle based on the prior art described with reference to FIG. 8 using the original image and background image shown in FIG. Since the object detection is performed by adjoining each other, only the contour portion of the car is detected.
[0057]
On the other hand, in FIG. 4C, which is a detection result according to the present embodiment, not only the contour portion of the vehicle but also the inside of the contour is detected, so that the whole vehicle is detected more clearly. The car is reliably detected.
In the present embodiment, as described above, the reference pixel to be selected for each target pixel is set apart from the target pixel by more than the size of the object. Therefore, only the contour part of the object is not detected unlike the case where the target pixel and the reference pixel are set close to each other.
Then, a plurality of reference pixels are set for one target pixel, object detection is performed for each pair of the target pixel and each reference pixel, and finally whether or not the target pixel is a pixel of the object is determined by a logical operation. Thus, even if the target pixel and the reference pixel are set apart from each other, the position of the object can be reliably specified.
[0058]
In addition, as in the above-described prior art, since it is determined whether or not the pixel of interest represents an image based on the distance between the illumination variation background model and the luminance point in the luminance space, the illumination variation is large and frequently occurs. Even in this case, an object can be detected with high accuracy.
[0059]
Next, a second embodiment of the present invention will be described.
FIG. 5 is a block diagram showing the object detection device 2 of the second embodiment, and FIG. 6 is a flowchart showing the operation of the object detection device of FIG. Hereinafter, an object detection apparatus according to a second embodiment of the present invention will be described with reference to these drawings, and at the same time, an object detection method and an object detection program according to the second embodiment of the present invention will be recorded. A readable information recording medium will be described. In FIG. 5, the same elements as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted here.
[0060]
The object detection apparatus 18 shown in FIG. 5 is different from the object detection apparatus 2 in that the circumscribing rectangle of the object is obtained instead of detecting the object itself, and as a result, the object is detected with a small amount of calculation. Is possible.
The object detection device 18 includes an image input means 4, a plurality of reference point selection means 20 functioning as a horizontal reference point selection means and a vertical reference point selection means according to the present invention, a background image holding means 8, and a horizontal background model creation according to the present invention. Background model creating means 22 functioning as means and vertical background model creating means, distance calculating means 24 functioning as horizontal distance calculating means and vertical distance calculating means according to the present invention, object discriminating means 26, background creating means 28, and image output For example, an object is detected from an image photographed by a surveillance camera.
[0061]
Similar to the object detection device 2, the object detection device 18 is specifically configured by a computer having an interface with a surveillance camera or a video tape recorder as an example. The input device constituting the computer is the present invention. For example, a CD-ROM as a computer-readable information recording medium on which the object detection program is recorded is loaded, the program data is read from the CD-ROM, loaded into the memory of the computer, and the computer CPU is based on the program data. It is realized by operating.
[0062]
The multiple reference point selection unit 20 of the object detection device 18 functions as a horizontal reference point selection unit, and is included in the horizontal pixel column of interest for each horizontal pixel column of interest consisting of a plurality of pixels arranged in the horizontal direction in the captured image. A plurality of reference pixels are selected and the luminance value of each reference pixel is acquired from the image input means 4.
Further, the plurality of reference point selection means 20 functions as a vertical reference point selection means, and each of the plurality of reference vertical pixel columns made up of a plurality of pixels arranged in the vertical direction in the photographed image includes a plurality of the plurality of reference point selection units 20 included in the target vertical pixel column. A reference pixel is selected and the luminance value of each reference pixel is acquired from the image input means 4.
[0063]
The background model creation means 22 functions as horizontal background model creation means, and the brightness values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected by the horizontal reference point selection means are background image holding means. 8 is used to create an illumination fluctuation background model when the brightness of the background changes under the influence of illumination fluctuation.
The background model creation means 22 functions as a vertical background model creation means, and the brightness values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected by the vertical reference point selection means are background image holding means. 8 is used to create an illumination fluctuation background model when the brightness of the background changes under the influence of illumination fluctuation.
[0064]
The multiple distance calculation means 2 functions as a horizontal distance calculation means, and in the multi-dimensional luminance space, the illumination variation background model created by the background model creation means 22 as the horizontal background model creation means and the horizontal reference point selection means. The distance between the points represented by the plurality of luminance values acquired by the plurality of reference point selection means 20 is calculated.
The multiple distance calculation means 2 functions as a vertical distance calculation means, and the illumination variation background model created by the background model creation means 22 as the vertical background model creation means in the multidimensional luminance space, and the vertical reference point selection A plurality of reference point selection means 20 as a means calculates a distance between the points represented by the plurality of luminance values acquired.
[0065]
The object discriminating unit 26 includes pixels representing an object in the target horizontal pixel column and the target vertical pixel column based on the distances calculated by the plural distance calculating unit 2 as the horizontal distance calculating unit and the vertical distance calculating unit. And determining a rectangular area including the object by performing a logical operation on the determination results relating to both pixel columns.
The background creating unit 28 corresponds to the pixel of the corresponding background image based on the luminance value of each pixel constituting the target horizontal pixel column and the target vertical pixel column determined that the object determining unit 26 does not include the pixel representing the object. Are updated and held in the background image holding means 8.
[0066]
Next, the operation of the object detection apparatus 18 configured as described above will be described.
7A and 7B are diagrams for explaining the operation of the second embodiment, in which FIG. 7A is an explanatory diagram showing detection of a possible area of an object in the vertical direction, and FIG. Explanatory drawing showing detection of a possible area | region, (C) is explanatory drawing which shows a detection result. Hereinafter, description will be made with reference to FIGS. 6 and 7 together with FIG.
[0067]
As shown in FIG. 6, first, the image input unit 4 reads an image captured by a surveillance camera or the like, and a luminance value for each pixel constituting the image is provided in the image input unit 4 (not shown). It is held in the memory (step B1).
Next, the plurality of reference point selection means 20 selects the pixels for one row at the address y in the vertical direction in the captured image as a target horizontal pixel column, and selects the pixels for this one row as reference pixels (reference pixel group). Thus, the luminance value of each reference pixel is acquired from the image input means 4 (step B4). Therefore, if the width of the captured image is Width pixels (Width is a positive integer), the noted horizontal pixel row includes Width pixels, and the plurality of reference point selection means 20 is the same in the present embodiment. All these pixels are used as reference pixels.
[0068]
Then, the background model creation means 22 acquires the brightness value of each pixel in the background image corresponding to each of the reference pixels selected by the plurality of reference point selection means 20 from the background image holding means 8, and the background brightness changes with illumination fluctuation. Create a lighting fluctuation background model when it changes under the influence of the.
Thereafter, the multi-distance calculation means 24 is a multi-dimensional luminance space formed by the luminance values of the reference pixels, and the illumination variation background model created by the background model creating means 22 and the plurality of reference points selecting means 20 obtained by the multiple reference point selection means 20. The distance to the point represented by the luminance value is calculated (step B6).
[0069]
Subsequently, the object discriminating unit 26 compares the distance calculated in this way by the multiple distance calculating unit 24 with a threshold value, and if the distance is larger than the threshold value, the object discriminating unit 26 associates the distance with the target horizontal pixel column. The logic “1” is held, and if the distance is smaller than the threshold value, the logic “0” is held (step B8).
The multiple reference point selection means 20, the background model creation means 22, and the multiple distance calculation means 2 perform such processing over the entire range in the vertical direction of the image while increasing the value of the address y by 1 (step B10). ).
As a result, as shown in FIG. 7A, when the object 108 exists as shown in the figure, the object discriminating means 26 sets logic “1” for each target horizontal pixel column in the range of H. Will hold.
[0070]
The multiple reference point selection means 20, the background model creation means 22, and the multiple distance calculation means 2 perform the same processing in the horizontal direction of the image.
That is, the plurality of reference point selection means 20 selects one column of pixels at the horizontal address x in the captured image as a target vertical pixel column, and selects one column of pixels as a reference pixel (reference pixel group). The brightness value of each reference pixel is acquired from the image input means 4 (step B2). Therefore, assuming that the height of the captured image is Height pixels (Height is a positive integer), the target vertical pixel column includes Height pixels, and the plurality of reference point selection means 20 is used in the present embodiment. Then, all these pixels are used as reference pixels.
[0071]
Then, the background model creation means 22 acquires the brightness value of each pixel in the background image corresponding to each of the reference pixels selected by the plurality of reference point selection means 20 from the background image holding means 8, and the background brightness changes with illumination fluctuation. Create a lighting fluctuation background model when it changes under the influence of the.
Thereafter, the multi-distance calculation means 24 is a multi-dimensional luminance space formed by the luminance values of the reference pixels, and the illumination variation background model created by the background model creating means 22 and the plurality of reference points selecting means 20 obtained by the multiple reference point selection means 20. The distance to the point represented by the luminance value is calculated (step B5).
[0072]
Subsequently, the object discriminating means 26 compares the distance calculated in this way by the multiple distance calculating means 2 with a threshold value. If the distance is larger than the threshold value, the object discriminating means 26 associates the distance with the target vertical pixel column. The logic “1” is held, and if the distance is smaller than the threshold value, the logic “0” is held (step B7).
The multiple reference point selection means 20, the background model creation means 22, and the multiple distance calculation means 24 perform such processing in the entire horizontal range of the image while increasing the value of the address y by 1 (step B9). ).
As a result, as shown in FIG. 7B, the object discriminating means 26 holds the logic “1” for each target vertical pixel column within the range of W.
[0073]
Thereafter, the object discrimination means 26 takes the logical product of the logical values held for each horizontal pixel column of interest and each vertical pixel column of interest (step B11), and holds the image of the rectangular area as the detection result. In the memory (not shown), whether the value is logic “1” or logic “0” is determined for each pixel.
Specifically, when both of the attention horizontal pixel column and the attention vertical pixel column including the pixel of the photographed image at the position corresponding to the pixel whose value is to be determined hold logic “1”, The value is set to logic “1”. On the other hand, when logic “0” is held for at least one of the target horizontal pixel column and the target vertical pixel column including the corresponding captured image pixel, the value of the pixel is set to logic “0”. (Steps B12, B13, B14). As a result, in the memory holding the detection result, as shown in FIG. 7C, the value of the pixel in the rectangular region R that may contain the object is logical “1”, and the other pixels The value of is logical “0”.
[0074]
Therefore, when the contents of this memory are displayed on the display device by the image output means 15, a rectangular area including the object is displayed on the screen. Further, the four sides of the rectangular area R are circumscribed rectangles of the object 108.
When the width of the captured image is Width pixels and the height is Height pixels, in the first embodiment, the background model is created and the distance is calculated for each pixel. The process needs to be performed (Width × Height) times.
However, in the second embodiment, the calculation process is performed for each horizontal pixel column and the calculation process is performed for each vertical pixel column, so the total number of calculation processes is (Width + Height) times. It becomes. Therefore, the amount of calculation is greatly reduced compared to the case of the first embodiment, which is very advantageous when performing object detection at a high speed or reducing the cost of the apparatus.
[0075]
As in the case of the object detection device 2, the background creation unit 28 determines the luminance value of each pixel constituting the target horizontal pixel column and the target vertical pixel column that the object determination unit 26 has determined not to include the pixel representing the object. Based on this, the luminance value of the pixel of the corresponding background image is updated and held in the background image holding means 8. This makes it possible to detect an object with high accuracy using the latest background image.
If there are more images to be processed, the above procedure is repeated for the next image as in the case of the object detection device 2 (step B15).
[0076]
In the second embodiment, in the target horizontal pixel column and the target vertical pixel column, all the pixels in each pixel column are used as reference pixels. However, thinning out such as selecting every several pixels is performed. Further, it is possible to further reduce the amount of calculation.
Further, for example, if an area where an object can exist first is determined in one of the horizontal direction and the vertical direction, and then processing in the other direction is performed only within that area, an area where there is no possibility that an object exists Then, since it is not necessary to perform arithmetic processing, the amount of calculation can be further reduced.
[0077]
Although the object detection device 2 and the object detection device 18 are configured by computers here, it is of course possible to configure the object detection device 2 and the object detection device 18 by dedicated hardware. In addition, some of the functions according to the present invention are realized by loading a predetermined program into a computer, and other functions are realized by dedicated hardware, or are already incorporated in the computer. The object detection device 2 and the object detection device 18 can be configured by using a combination of programs.
[0078]
The computer-readable information recording medium is not limited to the above-described CD-ROM, but is a portable medium such as a floppy (registered trademark) disk, a magneto-optical disk, or a ROM, or a storage device such as a hard disk built in the computer. Is included.
Further, the computer-readable information recording medium dynamically holds the program for a short time like a communication line when transmitting the program via a communication network such as the Internet or a communication line such as a telephone line. In this case, there are those that hold a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case.
[0079]
【The invention's effect】
As described above, in the object detection apparatus of the present invention, the image input unit captures and holds the luminance value of each pixel of the captured image, and the plurality of reference point selection units includes a plurality of reference pixels for each target pixel of the image. And the luminance values of the selected reference pixels and the target pixel are acquired for each target pixel from the image input means. Then, the background model creating means outputs the luminance values of the plurality of pixels in the background image respectively corresponding to the target pixel selected by the plurality of reference point selecting means and the plurality of reference pixels corresponding to the same pixel from the background image holding means. The illumination variation background model is acquired for each set of the pixel of interest and each reference pixel corresponding to the pixel when the background luminance changes due to the illumination variation.
[0080]
In addition, in the luminance space, the distance calculation unit acquires the illumination variation background model created by the background model creation unit for each pair of the target pixel and the reference pixel, and the plurality of reference point selection units acquires the target pixel for each target pixel. The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
The object determining unit determines, for each pair of the target pixel and each reference pixel, whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculating unit, and determines each group A logical operation is performed on the result to finally determine whether the target pixel is a pixel representing an object.
[0081]
As described above, in the object detection device of the present invention, a plurality of reference pixels are set for one target pixel, object detection is performed for each pair of the target pixel and each reference pixel, and finally a logical operation is performed. Since it is determined whether or not the target pixel is a pixel of the object, the position of the object can be reliably specified even if the target pixel and the reference pixel are set apart.
Accordingly, the reference pixel to be selected for each target pixel can be set apart from the target pixel by more than the size of the object, and as a result, the target pixel and the reference pixel are set close to each other. Thus, only the contour portion of the object is not detected.
In addition, as in the prior art described above, since whether or not the pixel of interest represents an image is determined based on the distance between the illumination variation background model and the luminance point in the luminance space, the illumination variation is large and frequently occurs In addition, an object can be detected with high accuracy and stability.
[0082]
In the object detection device of the present invention, the image input unit captures and holds the luminance value of each pixel of the captured image, and the horizontal reference point selection unit includes a plurality of pixels arranged in the horizontal direction in the captured image. For each horizontal pixel column, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is acquired from the image input unit. Further, the vertical reference point selection unit selects a plurality of reference pixels included in the target vertical pixel column for each target vertical pixel column including a plurality of pixels arranged in the vertical direction in the captured image, and brightness of each reference pixel. The value is acquired from the image input means.
Then, the horizontal background model creation means acquires the brightness values of the plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selection means from the background image holding means, and the background brightness is An illumination variation background model in the case of changing under the influence of illumination variation is created, and the vertical background model creation means is a plurality of reference pixels selected by the vertical reference point selection means, each of which corresponds to the plurality of reference pixels in the background image. A luminance value background model is created when the luminance value of the pixel is obtained from the background image holding means and the luminance of the background changes due to the influence of the illumination variation.
[0083]
Thereafter, in a multi-dimensional luminance space, the horizontal distance calculation unit is a distance between the illumination variation background model created by the horizontal background model creation unit and the point represented by the plurality of luminance values acquired by the horizontal reference point selection unit. The vertical distance calculation means is a multi-dimensional luminance space between the illumination variation background model created by the vertical background model creation means and the points represented by the plurality of brightness values obtained by the vertical reference point selection means. Calculate the distance.
Then, the object determining means determines whether or not a pixel representing the object is included in the target horizontal pixel column and the target vertical pixel column based on the distance calculated by the horizontal distance calculating unit and the vertical distance calculating unit, A logical operation is performed on the discrimination results relating to both pixel columns to identify a rectangular area including the object.
[0084]
In the object detection device of the present invention, calculation processing such as creation of a background model for each horizontal pixel column and calculation of the distance between the background model and the luminance point is performed, and similar calculation processing is performed for each vertical pixel column. Therefore, the total number of times of calculation processing is a value obtained by adding the number of pixels in the horizontal direction of the image to the number of pixels in the vertical direction of the image.
On the other hand, when calculation processing such as creating a background model for each pixel and calculating the distance between the background model and the luminance point is performed, the total number of calculation processing is the number of pixels in the horizontal direction of the image. , Multiplied by the number of pixels in the vertical direction of the image.
Therefore, in the present invention, it is possible to detect an object existence area with high accuracy and stable even when the illumination fluctuation is large and frequently occurs, and the calculation amount is greatly reduced. This is very advantageous for detection and cost reduction of the apparatus.
[0085]
In the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, in the image input step, the luminance value of each pixel of the photographed image is captured and held in the first storage means. In the multiple reference point selection step, a plurality of reference pixels are selected for each target pixel of the image, and the selected plurality of reference pixels and the luminance value of the target pixel are acquired from the first storage unit for each target pixel. . In the background model creation step, the second storage means stores the luminance values of the plurality of pixels in the background image respectively corresponding to the target pixel selected in the plurality of reference point selection step and the plurality of reference pixels corresponding to the pixel. Thus, an illumination variation background model when the background luminance changes under the influence of illumination variation is created for each set of the target pixel and each reference pixel corresponding to the pixel.
[0086]
In addition, in the distance calculation step, in the luminance space, the illumination variation background model created for each pair of the target pixel and the reference pixel in the background model creation step and the target reference pixel acquired in the multiple reference point selection step The distance between each of the luminance values of the plurality of reference pixels and the point represented by the set of the luminance value of the target pixel is calculated.
In the object determining step, for each set of the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated in the distance calculating step. A logical operation is performed on the result to finally determine whether the target pixel is a pixel representing an object.
[0087]
As described above, in the present invention, a plurality of reference pixels are set for one target pixel, object detection is performed for each pair of the target pixel and each reference pixel, and finally, the target pixel is detected as an object by logical operation. Since it is determined whether or not the pixel is a pixel, the position of the object can be reliably specified even if the target pixel and the reference pixel are set apart from each other.
Accordingly, the reference pixel to be selected for each target pixel can be set apart from the target pixel by more than the size of the object, and as a result, the target pixel and the reference pixel are set close to each other. Thus, only the contour portion of the object is not detected.
In addition, as in the prior art described above, since whether or not the pixel of interest represents an image is determined based on the distance between the illumination variation background model and the luminance point in the luminance space, the illumination variation is large and frequently occurs In addition, an object can be detected with high accuracy and stability.
[0088]
In the object detection method of the present invention and the object detection program recorded on the computer-readable information recording medium of the present invention, the luminance value of each pixel of the photographed image is captured and stored in the first storage means in the image input step. In the horizontal reference point selection step, for each target horizontal pixel column composed of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the target horizontal pixel column are selected and the luminance of each reference pixel is selected. A value is obtained from the first storage means. Further, in the vertical reference point selection step, a plurality of reference pixels included in the target vertical pixel column are selected for each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, and the luminance of each reference pixel is selected. A value is obtained from the first storage means.
In the horizontal background model creation step, the luminance values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selection step are obtained from the first storage means, and the background luminance is obtained. A lighting fluctuation background model is created when the lighting changes due to the influence of lighting fluctuation, and in the vertical background model creation step, a plurality of background pixels in the background image respectively corresponding to the plurality of reference pixels selected in the vertical reference point selection step The luminance value of the pixel is obtained from the second storage means, and an illumination variation background model is created when the background luminance changes under the influence of illumination variation.
[0089]
Thereafter, in the horizontal distance calculation step, in a multidimensional luminance space, the distance between the illumination variation background model created in the horizontal background model creation step and the point represented by the plurality of luminance values acquired in the horizontal reference point selection step In the vertical distance calculation step, in the multidimensional luminance space, between the illumination variation background model created in the vertical background model creation step and the points represented by the plurality of luminance values obtained in the vertical reference point selection step Calculate the distance.
Then, in the object determination step, based on the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, it is determined whether or not the pixel representing the object is included in the target horizontal pixel column and the target vertical pixel column, A logical operation is performed on the discrimination results relating to both pixel columns to identify a rectangular area including the object.
[0090]
Thus, in the present invention, calculation processing such as creation of a background model and calculation of the distance between the background model and the luminance point is performed for each horizontal pixel column, and similar calculation processing is performed for each vertical pixel column. Therefore, the total number of times of calculation processing is a value obtained by adding the number of pixels in the horizontal direction of the image to the number of pixels in the vertical direction of the image.
On the other hand, when calculation processing such as creating a background model for each pixel and calculating the distance between the background model and the luminance point is performed, the total number of calculation processing is the number of pixels in the horizontal direction of the image. , Multiplied by the number of pixels in the vertical direction of the image.
Therefore, according to the present invention, the fluctuation of the illumination is large and stable even when it frequently occurs, and the existence area of the object can be detected with high accuracy, and the calculation amount is greatly reduced. This is very advantageous for detecting an object and reducing the cost of the apparatus.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an object detection apparatus according to a first embodiment.
FIG. 2 is a flowchart showing an operation of the object detection apparatus of FIG.
3A is an explanatory diagram showing a positional relationship between a target point and a reference point on an image plane, FIG. 3B is a schematic diagram when an object has entered a shooting area, and FIG. 3C is a schematic diagram showing a detection result; FIG. 4D is another explanatory diagram showing the positional relationship between the attention point and the reference point on the image plane, FIG. 4E is another schematic diagram showing the detection result, and FIG. 4F is a schematic diagram showing the final detection result. FIG.
4A is a schematic diagram showing an original image of a black car running on a road, FIG. 4B is an explanatory diagram showing a detection result of a vehicle according to the prior art, and FIG. 4C is an embodiment of the present invention. It is explanatory drawing which shows the detection result of a vehicle.
FIG. 5 is a block diagram showing an object detection apparatus according to a second embodiment.
6 is a flowchart showing an operation of the object detection apparatus of FIG.
7A and 7B are diagrams for explaining the operation of the second embodiment, in which FIG. 7A is an explanatory diagram showing detection of a possible area of an object in the vertical direction, and FIG. 7B is an object in the horizontal direction; (C) is an explanatory view showing the detection result.
FIG. 8 is an explanatory diagram showing a background image.
FIG. 9 is an explanatory diagram showing a two-dimensional luminance space formed by luminance values of a target point and a reference point.
FIG. 10 is an explanatory diagram illustrating a captured image when an object enters the imaging region of the surveillance camera.
11A is an explanatory diagram illustrating an image plane, FIG. 11B is a schematic diagram when an object has entered a shooting area, and FIG. 11C is a schematic diagram illustrating a detection result;
[Explanation of symbols]
2 ... Object detection device, 4 ... Image input means, 6 ... Multiple reference point selection means, 8 ... Background image holding means, 10 ... Background model creation means, 12 ... Distance calculation means, 14 ... Object Discriminating means, 16 ... background creating means, 18 ... object detecting device, 20 ... multiple reference point selecting means, 22 ... background model creating means, 24 ... distance calculating means, 26 ... object discriminating means, 28 ... ... background creation means, 102 ... white dots, 104 ... straight lines, 106 ... white circle dots, 108 ... objects, 110 ... black dots.

Claims (4)

A computer-readable information recording medium recording an object detection program for detecting an object using a captured image,
An image input step of capturing the luminance value of each pixel of the photographed image and holding it in the first storage means;
Multiple reference point selection step of selecting a plurality of reference pixels for each target pixel of the photographed image, and acquiring the selected reference pixels and luminance values of the target pixel from the first storage unit for each target pixel. When,
A background image holding step of holding the luminance value of each pixel of the background image forming the background of the photographed image in the second storage means;
The luminance values of the plurality of pixels in the background image respectively corresponding to the target pixel selected in the plurality of reference point selection step and the plurality of reference pixels corresponding to the pixel are acquired from the second storage unit, A background model creation step of creating a lighting variation background model for each set of the pixel of interest and each reference pixel corresponding to the pixel when the luminance of the background changes under the influence of illumination variation;
In a luminance space, the illumination variation background model created for each set of the target pixel and reference pixel in the background model creation step, and the plurality of reference pixels acquired for each target pixel in the multiple reference point selection step A distance calculation step of calculating a distance between each of the brightness values of the pixel and a point represented by the set of the brightness values of the target pixel,
For each set of the target pixel and each reference pixel, it is determined whether the target pixel is a pixel representing an object based on the distance calculated in the distance calculation step, and a logical operation is performed on the determination result for each set. And an object discrimination step for finally discriminating whether or not the target pixel is a pixel representing an object. A computer-readable information recording medium recording an object detection program. A luminance value of a pixel of the background image corresponding to each of the target pixel determined not to be a pixel representing an object in the object determination step and each of the plurality of reference pixels corresponding to the pixel is determined as the target pixel and the same pixel. 2. A computer-readable computer-readable recording object detection program according to claim 1, further comprising a background creation step of updating the plurality of reference pixels corresponding to the pixels based on luminance values and storing the background in the first storage means. Information recording medium. A computer-readable information recording medium recording an object detection program for detecting an object using a captured image,
An image input step of capturing the luminance value of each pixel of the photographed image and holding it in the first storage means;
For each target horizontal pixel column composed of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the target horizontal pixel column are selected, and the luminance value of each reference pixel is stored in the first storage. A horizontal reference point selection step acquired from the means;
For each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the photographed image, a plurality of reference pixels included in the target vertical pixel column are selected, and the luminance value of each reference pixel is stored in the first storage. A vertical reference point selection step acquired from the means;
A background image holding step of holding the luminance value of each pixel of the image constituting the background of the captured image in the second storage means;
The luminance values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selection step are obtained from the second storage means, and the background luminance is affected by illumination variation. A horizontal background model creation step for creating a lighting fluctuation background model when the
The luminance values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the vertical reference point selection step are obtained from the second storage means, and the background luminance is affected by illumination variation. Vertical background model creation step to create a lighting fluctuation background model when changed
In a multidimensional luminance space, a horizontal distance calculation that calculates a distance between the illumination variation background model created in the horizontal background model creation step and a point represented by the plurality of luminance values acquired in the horizontal reference point selection step Steps,
Vertical distance calculation for calculating the distance between the illumination variation background model created in the vertical background model creation step and the point represented by the plurality of luminance values obtained in the vertical reference point selection step in a multidimensional luminance space Steps,
Based on the distance calculated in the horizontal distance calculation step and the vertical distance calculation step, it is determined whether or not the pixel representing the object is included in the target horizontal pixel column and the target vertical pixel column, and both pixel columns A computer-readable information recording medium having an object detection program recorded thereon, comprising: an object determination step of performing a logical operation on the determination result related to the above and identifying a rectangular area including the object. Based on the luminance value of each pixel constituting the target horizontal pixel column and the target vertical pixel column determined not to include the pixel representing the object in the object determination step, the luminance value of the pixel of the corresponding background image is updated. 4. A computer-readable information recording medium having recorded thereon an object detection program according to claim 3, further comprising a background creation step to be held in said second storage means.

Images