JP3567114B2 - Image monitoring apparatus and image monitoring method - Google Patents

Description

ここで、背景との差を表す値Ｄは、上記のような減算によって求めても良いし、背景画素の明度値Ｂで除算して背景との変化比として求めても良い等、任意の演算によって求めることができる。
【００３３】
時刻ｔでの特徴分布Ｆ（ｔ）は、画像中に設定されている検知領域Ｒに属する各画素（ｘ，ｙ）について特徴量を求め、この統計的な分布を求めることにより算出される。検知領域Ｒは、画像全体でも良いし、図３に示すように任意の形状に設定することができる。
【００３４】
ここでは、特徴分布Ｆを単純に差分値Ｄ（特徴量）のヒストグラムとして求めると、図４のグラフが得られる。ヒストグラムとして特徴分布を求める場合を記述すれば、以下の通りである。
【００３５】
【数２】

である。
【００３６】
このようにして、任意の時刻ｔでの特徴分布Ｆ（ｔ）が得られることになる。
【００３７】
図４では、検知領域が矩形に設定されていて、その検知領域内に画像が図示されている。また、撮影した画像が背景画像に比べて変化がない場合であるため、特徴分布Ｆは単峰で平均０の正規分布に近い形となっている。
【００３８】
（４）画像変化追跡部１３では、特徴分布算出部１２において上記のように算出された任意時刻ｔでの特徴分布の時間的な変化を解析し、その結果を次の結果判定部１４に伝える（画像変化算出ステップ）。
【００３９】
今、図５に示すように、２つの時刻ｔ１、ｔ２で特徴分布Ｆ（ｔ１）、Ｆ（ｔ２）が得られている場合を例にして、画像変化追跡部１３での処理について説明する。
【００４０】
図５では、時刻ｔ１では人物や車両等の侵入物はなく、背景との変化はほとんどない。これに対して時刻ｔ２においては、雲が晴れる等の日照強度の変動があって画面全体で明るくなるという環境変動があるとともに、人物（背景より明度が暗い）の侵入がある場合を示している。
【００４１】
時刻ｔ１では、時刻ｔ１における撮像画像と、背景画像との差（明度差）がほとんどないため、前述した図４と同様に特徴分布Ｆ（ｔ１）は平均０の単峰の分布となる。
【００４２】
これに対して時刻ｔ２で得られる特徴分布Ｆ（ｔ２）は、人物以外の背景領域の各画素の明度値が一律に大きくなる（明るくなる）ため、背景部分からくる分布の平均値は０よりも大きくなり（ｐ２２）、人物領域による分布は背景よりも暗い部分に小さくできることになり（ｐ２１）、特徴分布Ｆ（ｔ２）はこれら２つの分布を合わせた二峰性のものとして得られる。
【００４３】
ここで、従来の単純な撮像画像と背景画像との差分処理によって画像変化を検出する。
【００４４】
つまり各画素でのＤの値の大小で変化を検出する場合では、時刻ｔ２では背景そのものの明度変化が侵入人物による明度変化よりも大きいため、単に変化量の大きさだけで判定した場合には、侵入人物だけでなく背景全体を画像変化として誤って検出してしまう等、侵入人物だけを正しく検出できない。
【００４５】
このような背景差分における弊害を避けるため、画像変化追跡部１３では、背景領域による特徴分布のピーク（峰）を各時刻で算出することにより、背景変動による画像変化と物体の侵入による画像変化とを区別している。
【００４６】
まず、各時刻での特徴分布のピークを求める。時刻ｔ１では１つしかないため簡単にピークが得られるが、時刻ｔ２ではピークが２つ（複数）あるため、各々のピークを区別する必要がある。
【００４７】
ピークを区別して求める方法としては、最も単純には特徴分布での極大値となるところをピークと見做す方法がある。また、複雑な分布を複数の正規分布等の和で近似するＥＭアルゴリズムといった方法がある（参考文献１、２参照方）。
【００４８】
［参考文献１］…Ａ． Ｄｅｍｐｓｔｅｒ， Ｎ． Ｌａｉｒｄ， ａｎｄ Ｄ． Ｒｕｂｉｎ， Ｍａｘｉｍｕｍ ｌｉｋｅｌｉｈｏｏｄ ｆｒｏｍ ｉｎｃｏｍｐｌｅｔｅ ｄａｔａ ｖｉａ ｔｈｅ ＥＭ ａｌｇｏｒｉｔｈｍ， Ｊｏｕｒｎａｌ ｏｆ ｔｈｅ Ｒｏｙａｌ Ｓｔａｔｉｓｔｉｃａｌ Ｓｏｃｉｅｔｙ， ３９ （Ｓｅｒｉｅｓ Ｂ）： １−３８， １９７７．
［参考文献２］…Ｎ． Ｆｒｉｅｄｍａｎ， Ｓ． Ｒｕｓｓｅｌｌ， Ｉｍａｇｅ Ｓｅｇｍｅｎｔａｔｉｏｎ ｉｎ ｖｉｄｅｏ ｓｅｑｕｅｎｃｅｓ： Ａ ｐｒｏｂａｂｉｌｉｓｔｉｃ ａｐｐｒｏａｃｈ， Ｐｒｏｃ． Ｔｈｉｒｔｅｅｎｔｈ Ｃｏｎｆ． ｏｎ Ｕｎｃｅｒｔａｉｎｔｙ ｉｎ Ａｒｔｉｆｉｃｉａｌ Ｉｎｔｅｌｌｉｇｅｎｃｅ （ＵＡＩ）， １９９７
この方法を用いると、平均μ、標準偏差σの正規分布ｐ（ｘ）を以下のように表し、
【数３】

ｐ２１、ｐ２２に相当する正規分布の平均、標準偏差、重みといったパラメータをμ２１、σ２１、ｗ２１と、μ２２、σ２２、ｗ２２とで表せば、特徴分布Ｆ（ｔ２）は、以下のように２つの正規分布の重み付き和で近似でき、ｐ２１とｐ２２とを区別することができる。
【００４９】
【数４】

ここで、時刻ｔ２において人物侵入領域だけを正しく検知できるようにするには、複数（この例では２つ）の分布のうちのどちらが背景領域によるものかを判定する必要がある。
【００５０】
このために、以前の時刻ｔ１での背景領域による分布（この例の場合ｐ１しかない）との分布の近さを求め、近いもの同士が背景領域による分布であると認定して、各時刻での背景領域の分布を算出する。分布の近さＳとしては、例えば各分布でのパラメータ同士のユークリッド距離を以下のように算出することができる。
【００５１】
【数５】

分布の近さＳとしては、パラメータ間の距離だけでなく、各分布を用いての任意の演算によって算出することが可能である。
【００５２】
このようにして近さＳをｐ２１、ｐ２２で求め、ｐ１と近い方が背景領域による分布であることがわかる（この例ではｐ２２となる。図５右側の下向の矢印）。
【００５３】
以上のようにして、画像変化追跡部１３では、各時刻での背景領域の分布を追跡し、その結果を次の結果判定部１４に送る。
【００５４】
以上述べた実施形態では、背景での環境変動がある場合について説明したが、他にもカメラ等の撮像手段が持つＡＧＣ等の自動調整機能により画像が変化を起こす場合がある。
【００５５】
図６では、各時刻ｔ１〜ｔ４での特徴分布列Ｆ（ｔ１）〜Ｆ（ｔ４）がそれぞれ撮像された画像の右側（図中）に求められ記載されているが、この例では環境変動（撮像領域の変更）がないにもかかわらず、検知領域内に白色の車両が侵入することにより、画像全体に明度変化が発生する場合を示している。
【００５６】
侵入前の時刻ｔ１では、撮影した画像と背景画像との間に大きな変化がないため、特徴分布Ｆ（ｔ１）は、図４と同様に単峰の分布として得られている。
【００５７】
続いて、時刻ｔ２、ｔ３、ｔ４と経過するにつれて車両が進入してくると、車両が白色であって、その撮像される領域（車両の部分の画素）が背景に比べて明るく、かつ大きな領域として捉えられているため、特徴分布のプラスの部分にもうひとつのピーク（峰）ができ始めて徐々に大きくなり、双峰性の特徴分布が得られる。
【００５８】
このとき、明るい（白色）車両領域の侵入により、カメラのＡＧＣ機能が働いて画像全体を暗くする方向に明度を補正してしまうため、本来環境変化のない背景領域においても、明度が暗くなってしまう。このようにカメラの自動調整機能が働く場合においては従来の監視装置では、画像変化から侵入する物体だけを正しく検知することができなかったが、本発明の画像変化追跡部１３を用いることによって、背景が変動している場合でも特徴分布を時間的に追跡した上で（図６グラフの下向の矢印）、次の工程で詳細を説明する結果判定を行うことにより、侵入物の領域か背景領域かを区別して検出することができる。
【００５９】
（５）最後に、結果判定部１４では、時系列的に前の画像変化追跡部１３にて求められた画像変化の追跡情報を用いて画像処理を行った後に、撮影された画像中に人物や物体等の侵入があるかどうかを判定する（判定ステップ）。
【００６０】
図５（時刻ｔ２において背景変動と人物侵入がある場合）における結果判定を説明する。
【００６１】
画像変化算出手段３では、背景領域の特徴分布が追跡されているため（図５中ｐ１→ｐ２２）、この情報を用いて時刻ｔ２での画像を補正することにより、背景変動による明度（特徴量）変化の影響を取り除く。
【００６２】
つまり、時刻ｔ２での背景領域の分布ｐ２２の平均値μ２２を用いて、この値だけ画像の各画素の明度値を減算して背景変動分を補正すると、図７下段に示すように背景部分の明度は予め作成されている背景画像と同じにすることができる。このとき背景領域の分布はｐ２２´となり、平均値もμ２２´となる。なお、補正前の時刻ｔ２における画像は図７中段の通りである。
【００６３】
ここで、画像を補正する方法としては、本実施形態に示す通り、特徴量として撮像された画像と背景画像との差Ｄを用いる場合には、背景分布の平均値だけ各画素の明度値から減算することもでき、また撮像された画素の明度値と背景画素の明度値との変化比など他の特徴量を用いる場合には、その特徴算出方法に応じた任意の方式により画像を補正することができる。
【００６４】
画像を補正することなく背景画像との差分処理を行えば、前述したとおり、背景全体が画像変化として検出される恐れがある。しかしながら、本実施形態のように、補正した後の画像を用いて背景画像との差分処理を行えば、図８に示すように背景変動は検出されず、侵入した物体の領域のみが検出されることになる。
【００６５】
このようにして、日照変化といった環境変動やカメラの自動調整機能により背景部分の画像が変化してしまった場合でも、監視装置本来の検知目的である人物や物体の侵入だけを正しく検知することができ、使用者に対して誤って警告を発する等の誤動作を大幅に減少させることができる。
【００６６】
次に、本発明の画像監視方法の第２実施形態の構成、動作について図９を参照して説明する。
【００６７】
第２実施形態の特徴は、差分処理の結果から環境条件や撮像条件の変化を推定することにより、検出された画像変化が検知目的であるか否かの判定を行い信頼性を向上させることである。
【００６８】
図９は、種々の条件において撮像された画像から算出される特徴分布図である。
【００６９】
撮像された画像と背景画像との差分処理によって画像中の変化を検出し、人物や物体の侵入を検知しようとする場合、撮影した画像が背景画像に対してほとんど変化していない時には、特徴分布（ここでは、これまでの例と同様に、背景画像との明度差を特徴量として用いる場合について説明を行う）は、平均値０の単峰の正規分布状になり（図９上段）、日照変動等の環境変動だけの影響により撮影した画像全体の明度が変わる（例えば暗くなる）場合には、特徴分布の形は保ったままで平均値が０から例えば右側にずれる（図９中段）。
【００７０】
ただし、図９上段、中段の図では、侵入物はないものと考え、特徴分布は背景領域の特徴分布と等しいものとする。
【００７１】
また、カメラ等の撮像手段の設置条件が悪く、自動車や電車等の車両が近くを通過した場合や、強い風の影響等によりカメラ自体が振動する場合には、画像中に撮影されている背景物体の位置がずれることにより、特徴分布の平均値は同じままで分散が大きくなる。
【００７２】
これらのことから、撮影した画像において求めた特徴分布の変化から逆に、その時刻での環境条件や撮影条件の変化を特徴分布の変化から推定することができる。
【００７３】
例えば、背景領域の特徴分布の平均値だけが変化した場合には日照強度の変動など環境条件が変化しており、また平均値はそのままで分散だけが変化した場合にはカメラが振動するなど撮影条件が変化していると判定することができる。
【００７４】
このようにして、背景差分等の画像処理によって画像変化を検出する場合、特徴分布からその時刻での環境条件や撮影条件の変化を推定することにより、検出された画像変化が真に人物や物体の侵入によるものなのか否か、環境条件や撮影条件の変化によるものなのか否かを判断するための指標として用いることができる。
【００７５】
また、環境条件や撮影条件の変化が起こっていると判定される場合には、使用者に対して警告を発しないようにするなど、画像監視装置の誤動作を大幅に減少させることができる。
【００７６】
次に、本発明の画像監視方法の構成、動作について図１０を参照して説明する。
【００７７】
第３実施形態の特徴は、差分処理の結果から画像を撮影している時刻における天候や日照状態を推定し、真の検知目的である対象物の抽出の信頼性を向上させたことである。
【００７８】
図１０は、様々な天候や日照状態（時刻）において算出された特徴分布図である。
【００７９】
画像変化算出手段３において、複雑な形状の特徴分布であっても複数の正規分布の和で近似できる方法があることは第１実施形態でも述べたが、このようにして特徴分布の各ピークを区別することにより、画像を撮影している時刻での天候や日照状態を推定することができる。
【００８０】
図１０上段図は、画像中に車両の侵入がある場合の特徴分布の例を示しているが（背景変動はないものとする）、第１実施形態でも述べたように、物体の侵入がある場合には背景領域によるピークと侵入物領域によるピークとの２つ（複数）の分布が足し合わされたような多峰性の特徴分布になる。
【００８１】
この上段図では、天候が曇りである場合を想定しているが、図１０中段図のように、天候が晴れである場合には、侵入物体の「影」が画像中に映り、この影の領域による分布が特徴分布内の背景による分布よりも暗いところに現れる。
【００８２】
また、時刻が変わって夜間になった場合、車両がヘッドライトを点灯するため、このヘッドライトの映り込み領域の分布が他の分布よりも明るいところに現れる。
【００８３】
さらに、図示しないが、雨や雪が降っているような天候であれば、雨滴等による小さな画像変化が多数現れることにより、観測される背景領域による特徴分布が晴れや曇りのときに比べて、その分散が大きく観測される等、特徴分布の性質が変化する可能性がある。
【００８４】
このようにして、特定の特徴分布のピークや性質を計測することにより、画像を撮影した時刻が昼間であるか夜間であるかといった時間帯（日照条件）に関する情報や、その時刻において撮像領域が晴れているか曇っているか雨や雪が降っているか霧が出ているかといった天候状態を推定することができる。
【００８５】
また、他の各種センサや人手による撮像環境などの入力作業といった手段に頼らずとも、撮影した画像から天候や日照状態を推定でき、推定結果に基づいて画像監視装置の条件を最適に設定することにより（カメラの感度の上げ下げや、検知領域の再設定等）、誤動作を大幅に減少させ検知目的対象物の検知の信頼性を向上させることができる。
【００８６】
尚、本発明は上記実施形態には限定されず、その主旨を逸脱しない範囲で種々変形して実施できることは言うまでもない。例えば、撮像装置を複数設けることにより、検知対象物を特定する信頼性を向上させることができる。
【００８７】
【発明の効果】
以上説明したように本発明によれば、画像監視装置の検知性能を向上させることができる。
【図面の簡単な説明】
【図１】本発明の画像監視装置の第１実施形態のブロック図。
【図２】本発明の画像監視方法の第１実施形態のフローチャート。
【図３】画像内での検知領域の設定例を示す図。
【図４】特徴分布の算出方法の説明図。
【図５】条件の異なる２つの時刻において算出された特徴分布の説明図。
【図６】撮像装置の自動調整機能が働いた場合の特徴分布の算出の説明図。
【図７】画像変化算出手段からの結果を用いて画像を補正する方法の説明図。
【図８】補正後の画像において背景差分処理を行った場合の説明図。
【図９】本発明の画像監視方法の第２実施形態を説明する種々の条件において撮像された画像から算出される特徴分布図。
【図１０】本発明の画像監視方法の第３実施形態を説明する様々な天候や日照状態において算出された特徴分布図。
【符号の説明】
１１ 画像入力部
１２ 特徴分布算出部
１３ 画像変化追跡部（画像変化算出部）
１４ 結果判定部
１０１ 撮像装置
１０２ 表示装置
１０３ 通信装置[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image monitoring apparatus and an image monitoring method for automatically monitoring a desired target (area) using an imaging unit such as an ITV (Intelligent Television) camera.
[0002]
[Prior art]
Image processing technology that automatically detects intrusion of a suspicious person or a vehicle from an image in a monitoring area obtained by an imaging means such as an ITV camera has been used mainly in the security field until now. There was a cost reduction effect such as a reduction in long-term monitoring work and a reduction in the number of personnel required for monitoring.
[0003]
As an image processing method for detecting a change in an image in such an automatic monitoring apparatus, a method of detecting a change between a plurality of images (frames) photographed continuously in time series and a method of comparing a background image created in advance with a current image is used. There is a method of calculating a difference between the two.
[0004]
As described above, as a result of calculating a difference between a plurality of screens and detecting a change in an image, when an arbitrary change is observed in the monitoring area (in many cases, when there is a large change), the image is displayed in the monitoring area. It is a conventional image monitoring device to issue a warning signal that some kind of intrusion or abnormality has occurred.
[0005]
However, when a simple criterion such as issuing a warning when a large image change occurs in the monitoring area is used, an image change due to time fluctuations such as weather or sunshine is erroneously regarded as “intrusion detected, warning issued”. In some cases.
[0006]
Further, the position of the intruder could not be correctly detected due to the influence of an automatic adjustment function such as AGC (Auto Gain Control) of the camera itself as the imaging device.
[0007]
Also, depending on the installation conditions of the camera, the image changes when a vehicle such as an automobile or a train passes near the installation site of the camera, or when the camera itself vibrates due to the influence of a strong wind, etc. Was detected incorrectly.
[0008]
Due to such erroneous detection, only the image change due to intrusion of a person or an object, which is the original detection purpose, cannot be correctly detected.
[0009]
Furthermore, environmental conditions for photographing include weather, such as sunny, cloudy, rain, or fog, and sunshine, such as day or night. These environmental conditions are factors that greatly affect the detection performance of the automatic monitoring device. Despite these factors, various conditions could not be determined just by photographing with a camera.
[0010]
Therefore, in order to reduce the malfunction of the automatic monitoring device and improve the performance, information about the weather and sunshine conditions is obtained by relying on other sensors or other means such as manual input operation, and the detection conditions of the automatic monitoring device are changed. There is no other choice but to set the optimal style in time, and there is a problem that labor and installation costs are high.
[0011]
[Problems to be solved by the invention]
In the conventional image monitoring apparatus described above, since the detection method and the determination standard of the image change are simple, the intrusion is caused by fluctuations in environmental conditions such as weather and sunshine conditions, vibration of the camera itself, and automatic adjustment function. There was a problem that it was not possible to correctly detect only. In addition, there is a problem that environmental conditions that greatly affect the detection performance can be obtained only by relying on other means.
[0012]
In view of the above, the present invention has been made in view of the above-described conventional problems. Based on the statistical distribution of feature amounts obtained from an image, image change due to intrusion of a person or an object, which is the original detection object, and fluctuation of environmental conditions Automatically judge changes in images due to camera vibration and automatic adjustment function, and automatically estimate the weather and sunshine conditions to reduce false detections other than intrusion of people and objects, and greatly improve detection performance. It is an object of the present invention to provide an image monitoring device and an image monitoring method that can be improved.
[0013]
[Means for Solving the Problems]
In order to achieve the above object, an image monitoring device according to the present invention includes:PredeterminedImage input unit to which an image obtained from an imaging device is inputAnd the pictureImage input by image input unitMin of featuresFeature distribution calculator for finding clothAnd specialCalculated by the signature distribution calculatorFrom the distribution of the feature values obtained, the peak of the feature value distribution due to the background region is obtained, and this peak is tracked at a predetermined time and imaged.Image change to calculate image change stateTrackingDepartmentAnd the pictureImage changeTrackingIs the calculation result of the change state obtained by the partThe fluctuation of the peak of the feature distribution due to the background area trackedImage changes due to fluctuations in environmental conditions and shooting conditionsAnd a determination unit that determines the distribution of the other feature amounts as an image change different from the image change.Composed ofIt is. Alternatively, an image input unit to which an image obtained from a predetermined imaging device is input, a feature distribution calculation unit that obtains a distribution of a feature amount by a background region of the image input by the image input unit, and a feature distribution calculation unit An image change tracking unit that calculates a change state of an image from the average value and the distribution state of the feature amount distribution obtained by the background region, and only the average value is obtained from the calculation result of the change state obtained by the image change tracking unit. If it changes, it is determined that the environmental condition has changed.If only the variance has changed, it is determined that the shooting condition has changed. And a result determination unit that determines that the image has entered the image.
[0014]
Further, the image monitoring method of the present invention,PredeterminedImage input step in which an image obtained from an imaging device is inputIn this image input stepEmpowered imageMin of featuresFeature distribution calculation step for finding clothAnd this featureIn the distribution calculation stepFrom the distribution of the feature values obtained, the peak of the feature value distribution due to the background region is obtained, and this peak is tracked at a predetermined time and imaged.Image change to calculate image change stateTrackingStepsAnd this pictureImage changeTrackingIs the calculation result of the change state obtained in the stepIt is determined that the change of the peak of the distribution of the feature amount due to the tracked background region is determined as an image change due to the change of the environmental condition or the imaging condition, and the distribution of the other feature amount is determined as the image change different from the above image change.Fixed step. Alternatively,An image input step in which an image obtained from a predetermined imaging device is input, a feature distribution calculating step of obtaining a distribution of a feature amount by a background region of the image input in the image input step, and a feature distribution calculating step An image change tracking step of calculating an image change state from the average value and the distribution state of the feature amount distribution by the obtained background region, and only the average value is obtained from the calculation result of the change state obtained by the image change tracking step. If it changes, it is determined that the environmental condition has changed.If only the variance has changed, it is determined that the shooting condition has changed. And a result determination step of determining that the intrusion has occurred in the image.
[0015]
An image input function for inputting an image obtained from an arbitrary image pickup means on a computer-readable recording medium storing a program for operating the computer of the present invention, and an image input function for inputting an image obtained by the image input means. A feature distribution calculation function for calculating a statistical distribution of the feature amount, an image change calculation function for calculating a change state of an image from the feature distribution obtained by the feature distribution calculation means, A program consisting of a result determination function for determining the intrusion of a person or an object for detection from the calculation result of the changed state is stored. It is characterized in that a change is detected separately from an image change caused by intrusion of a detection target person or object.
[0016]
Various information obtained by each function can be recorded on a recording medium such as an optical disk or a magnetic recording medium, and can be appropriately displayed on a display unit such as a display.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a configuration of an embodiment of the present invention will be described with reference to the drawings.
[0018]
FIG. 1 is a block diagram of an image monitoring apparatus according to a first embodiment of the present invention, FIG. 2 is a flowchart of the image monitoring method according to the first embodiment of the present invention, and FIG. FIG. 4 is a diagram illustrating a setting example of a region, FIG. 4 is a diagram illustrating a method of calculating a feature distribution, FIG. 5 is a diagram illustrating a feature distribution calculated at two times under different conditions, and FIG. FIG. 7 is an explanatory diagram of a feature distribution calculation when the automatic adjustment function of FIG. 3 is activated, FIG. 7 is an explanatory diagram of a method of correcting an image using a result from an image change calculating unit, and FIG. FIG. 9 is an explanatory diagram when a background difference process is performed.
[0019]
In the image monitoring apparatus, an image input unit 11, a feature distribution calculation unit 12, an image change tracking unit 13 (image change calculation unit), and a result determination unit 14 are cascaded.
[0020]
An arbitrary image pickup device 101 such as an ITV camera is connected to the image input unit 11. In addition, the result determination unit 14 includes a display device 102 such as a display that displays the determination result, and a communication device 103 that outputs (transmits) the determination result to the outside of the image monitoring device. Note that the communication device 103 can transmit information to a predetermined place wirelessly or online.
[0021]
The operation of the embodiment having such a configuration will be described.
[0022]
(1) A predetermined monitoring area is imaged by the imaging device 101. The captured image is sent to the image input unit 11 (image input step).
[0023]
(2) The video signal continuously output from the imaging device 101 is sequentially A / D converted in the image input unit 11. The converted video signal is stored as a digital image or an image sequence in a storage unit provided in the image input unit 11. Further, it outputs the pixels of the entire image at an arbitrary time or the pixels of an arbitrary area at an arbitrary time to the feature distribution calculating means 2 at the subsequent stage (image input step).
[0024]
(3) From information output from the image input unit 11, a feature amount preset for an arbitrary time or an arbitrary region of an image or an image sequence stored in the image input unit 11 is calculated, and the statistical distribution is calculated. (Characteristic distribution calculation step).
[0025]
Here, the feature amount indicates arbitrary information that can be obtained in each pixel or an arbitrary region. For example, from simple values such as lightness and color information for each pixel, the results of differentiation or integration by space or time, the result of superposition of an arbitrary image filter, statistics such as average and variance, and the difference from a background image created in advance Any scalar or vector quantity that can be calculated from the image, such as the result of performing any object detection or area division processing, such as processing, and the fillet diameter, area, and motion amount of the area selected from the result, as well as any features It is possible to use a combination of quantities.
[0026]
In the present embodiment, the difference between the brightness value at each time and each pixel of the image and the brightness value at each pixel in the background image created in advance is used as the feature amount.
[0027]
That is, the method is based on a method of detecting an image change by a difference process between a captured image and a background image, and the image is composed of brightness information at each pixel (shade image). Is created.
[0028]
Any method can be used as a method of creating the background image. For example, (a) one time (frame) or an image preceding by a specific time is used as a background image as it is (in this case, a continuous difference method). (A) It is known that there is no intrusion of a person or an object. One image is extracted from the image sequence and used. (C) The image is created by performing any operation such as calculating the brightness or the average or a weighted sum of any characteristic amount at each pixel of the image sequence. By performing an arbitrary operation such as applying a median filter or a Kalman filter to the brightness of each pixel or a time series of an arbitrary feature amount in an image sequence (it may be unknown whether there is an intrusion or not). That is.
[0029]
The background image created in this way may be updated according to the time, or the background image created once may not be used and updated.
[0030]
Now, the brightness value at the pixel (x, y) of the image captured at time t is I (x, y, t), and the brightness value of the pixel at the same position in the background image at that time is B (x, y, t) (if the background image is updated temporally, if t changes, the background imageImage brightness value BAlso changes).
[0031]
The difference value D (x, y, t) from the background image, which is the feature amount, is obtained by the following equation (1).
[0032]
(Equation 1)

Here, the value D representing the difference from the background may be obtained by the above-described subtraction, or may be obtained by dividing by the brightness value B of the background pixel to obtain the change ratio with respect to the background. Can be determined by:
[0033]
The feature distribution F (t) at time t is calculated by obtaining a feature amount for each pixel (x, y) belonging to the detection region R set in the image, and obtaining this statistical distribution. The detection region R may be the entire image or may be set to any shape as shown in FIG.
[0034]
Here, the feature distribution F is simplyOf the difference value D (feature amount)When obtained as a histogram, the graph of FIG. 4 is obtained. The case where a feature distribution is obtained as a histogram is described as follows.
[0035]
(Equation 2)

It is.
[0036]
In this manner, a feature distribution F (t) at an arbitrary time t is obtained.
[0037]
In FIG. 4, the detection area is set to a rectangle, and an image is shown in the detection area. In addition, since the captured image has no change compared to the background image, the characteristic distribution F has a shape that is close to a normal distribution with a single peak and an average of 0.
[0038]
(4) The image change tracking unit 13 analyzes the temporal change of the feature distribution at the arbitrary time t calculated by the feature distribution calculation unit 12 as described above, and outputs the result as follows.Judgment(Step of calculating image change).
[0039]
Now, as shown in FIG. 5, an example in which the characteristic distributions F (t1) and F (t2) are obtained at two times t1 and t2 will be described.TrackingThe processing in the unit 13 will be described.
[0040]
In FIG. 5, there is no intruder such as a person or a vehicle at time t1, and there is almost no change from the background. On the other hand, at time t2, there is a change in the sunshine intensity such as clearing of the clouds and the like, and there is an environmental change such that the entire screen becomes bright, and a person (brighter than the background) enters. .
[0041]
At time t1, since there is almost no difference (brightness difference) between the captured image at time t1 and the background image, the characteristic distribution F (t1) is a single-peak distribution with an average of 0 as in FIG.
[0042]
On the other hand, in the feature distribution F (t2) obtained at time t2, since the brightness value of each pixel in the background area other than the person becomes uniformly large (brighter), the average value of the distribution coming from the background part is 0 or more. Becomes larger (p22), and the distribution of the person region can be reduced to a portion darker than the background (p21), and the feature distribution F (t2) is obtained as a bimodal combination of these two distributions.
[0043]
Here, an image change is detected by a conventional simple difference processing between a captured image and a background image.
[0044]
That is, in the case where the change is detected based on the magnitude of the value D in each pixel, the brightness change of the background itself is larger than the brightness change caused by the intruding person at the time t2. In this case, only the intruding person cannot be correctly detected, for example, the whole background is erroneously detected as an image change in addition to the intruding person.
[0045]
To avoid such adverse effects in background subtraction,TrackingThe unit 13 distinguishes between an image change caused by a background change and an image change caused by an intrusion of an object by calculating a peak (peak) of a feature distribution based on a background region at each time.
[0046]
First, the peak of the feature distribution at each time is obtained. Since there is only one peak at time t1, a peak can be easily obtained. However, at time t2, since there are two (plural) peaks, it is necessary to distinguish each peak.
[0047]
As a method of obtaining the peak by distinguishing it, the simplest is a method in which the local maximum value in the feature distribution is regarded as the peak. Further, there is a method such as an EM algorithm for approximating a complicated distribution by a sum of a plurality of normal distributions (see References 1 and 2).
[0048]
[Reference Document 1] ... A. Dempster, N.M. Laird, and D.M. Rubin, Maximum likelihood from complete data via the EM algorithm, Journal of the Royal Statistical Society, 39 (Series B): 191-38.
[Reference Document 2] ... N. Friedman, S.M. Russell, Image Segmentation in video sequences: A probabilistic approach, Proc. Thirdenth Conf. on Uncertainty in Artificial Intelligence (UAI), 1997.
Using this method, the normal distribution p (x) with mean μ and standard deviation σ is expressed as follows:
(Equation 3)

If parameters such as the mean, standard deviation, and weight of the normal distribution corresponding to p21 and p22 are represented by μ21, σ21, w21 and μ22, σ22, w22, the characteristic distribution F (t2) becomes two normal distributions as follows. It can be approximated by a weighted sum of distributions, and p21 and p22 can be distinguished.
[0049]
(Equation 4)

Here, in order to correctly detect only the human intrusion area at time t2, it is necessary to determine which of the plurality (two in this example) of distributions is due to the background area.
[0050]
For this purpose, the closeness of the distribution with the distribution by the background region at the previous time t1 (in this example, there is only p1) is determined, and the close ones are compared with the background region.ToThen, the distribution of the background area at each time is calculated. As the distribution closeness S, for example, the Euclidean distance between parameters in each distribution can be calculated as follows.
[0051]
(Equation 5)

The distribution closeness S can be calculated not only by the distance between the parameters but also by any calculation using each distribution.
[0052]
In this way, the closeness S is obtained from p21 and p22, and it can be seen that the distribution closer to p1 is the distribution based on the background area (p22 in this example; a downward arrow on the right side of FIG. 5).
[0053]
As described above, the image change tracking unit 13 tracks the distribution of the background region at each time, and sends the result to the next result determination unit 14.
[0054]
In the above-described embodiment, the case where there is an environmental change in the background has been described. However, an image may be changed by an automatic adjustment function such as an AGC included in an imaging unit such as a camera.
[0055]
In FIG. 6, the feature distribution sequences F (t1) to F (t4) at the respective times t1 to t4 are obtained and described on the right side (in the drawing) of the captured image, respectively. This shows a case where the brightness of the entire image changes due to the intrusion of a white vehicle into the detection area despite the fact that there is no change in the imaging area.
[0056]
At time t1 before the intrusion, there is no significant change between the captured image and the background image, and thus the characteristic distribution F (t1) is obtained as a single-peak distribution as in FIG.
[0057]
Subsequently, when the vehicle enters as time t2, t3, and t4 elapses, the vehicle is white, and the imaged area (pixels of the vehicle) is brighter and larger than the background. Therefore, another peak (peak) starts to be formed in the plus part of the feature distribution and gradually increases, and a bimodal feature distribution is obtained.
[0058]
At this time, the invasion of a bright (white) vehicle area causes the AGC function of the camera to work, thereby correcting the brightness in a direction to darken the entire image. Therefore, the brightness becomes dark even in the background area where there is no environmental change. I will. In the case where the automatic adjustment function of the camera works as described above, the conventional monitoring device cannot correctly detect only the intruding object from the image change, but by using the image change tracking unit 13 of the present invention, Even if the background is fluctuating, the feature distribution is tracked over time (downward arrow in the graph of FIG. 6), and the result is determined in the next step to determine the details. It can be detected by distinguishing between areas.
[0059]
(5) Finally, the result determination unit 14 performs image processing using the image change tracking information obtained by the previous image change tracking unit 13 in chronological order, and then adds a person in the captured image. It is determined whether there is an intrusion of an object or an object (determination step).
[0060]
The result determination in FIG. 5 (when there is a background change and a person invasion at time t2) will be described.
[0061]
Since the image distribution calculation unit 3 tracks the feature distribution of the background region (p1 → p22 in FIG. 5), the image at time t2 is corrected using this information, and the brightness (feature amount) ) Remove the effects of change.
[0062]
That is, using the average value μ22 of the background area distribution p22 at the time t2 and subtracting the brightness value of each pixel of the image by this value to correct the background variation, as shown in the lower part of FIG. The brightness can be the same as a previously created background image. At this time, the distribution of the background region is p22 ', and the average value is also μ22'. The image at time t2 before the correction is as shown in the middle part of FIG.
[0063]
Here, as a method of correcting the image, as shown in the present embodiment, when the difference D between the captured image and the background image is used as the feature amount, the average value of the background distribution is calculated from the brightness value of each pixel. If another feature amount such as a change ratio between the brightness value of a captured pixel and the brightness value of a background pixel is used, the image is corrected by an arbitrary method according to the feature calculation method. be able to.
[0064]
If the difference processing with the background image is performed without correcting the image, the entire background may be detected as an image change as described above. However, if the difference processing from the background image is performed using the corrected image as in the present embodiment, the background change is not detected as shown in FIG. 8 and only the area of the intruding object is detected. Will be.
[0065]
In this way, even if the background image changes due to environmental changes such as changes in sunlight or the camera's automatic adjustment function, it is possible to correctly detect only the intrusion of a person or object, which is the primary detection purpose of the monitoring device. It is possible to greatly reduce erroneous operations such as erroneously issuing a warning to the user.
[0066]
Next, the configuration and operation of an image monitoring method according to a second embodiment of the present invention will be described with reference to FIG.
[0067]
The feature of the second embodiment is that by estimating a change in an environmental condition or an imaging condition from a result of the difference processing, it is determined whether or not the detected image change is a detection purpose, thereby improving reliability. is there.
[0068]
FIG. 9 is a feature distribution diagram calculated from images captured under various conditions.
[0069]
If the difference between the captured image and the background image is detected to detect a change in the image to detect the intrusion of a person or an object, and if the captured image hardly changes from the background image, the feature distribution (Here, the case where the brightness difference from the background image is used as the feature amount will be described, as in the previous examples.) The result is a single-peak normal distribution with an average value of 0 (upper part in FIG. 9), and When the brightness of the entire captured image changes (for example, becomes darker) due to only the environmental change such as the change, the average value shifts from 0 to, for example, the right side while maintaining the shape of the feature distribution (the middle part in FIG. 9).
[0070]
However, in the upper and middle diagrams of FIG. 9, it is assumed that there is no intruder, and the feature distribution is equal to the feature distribution of the background region.
[0071]
In addition, when the installation condition of the imaging means such as a camera is poor and a vehicle such as an automobile or a train passes nearby, or when the camera itself vibrates due to the influence of a strong wind or the like, the background captured in the image is When the position of the object shifts, the variance increases while the average value of the feature distribution remains the same.
[0072]
From these facts, it is possible to infer from the change in the characteristic distribution the change in the environmental conditions and the shooting conditions at that time, contrary to the change in the characteristic distribution obtained in the captured image.
[0073]
For example, if only the average value of the feature distribution in the background area changes, environmental conditions such as changes in sunlight intensity have changed, and if the variance only changes while the average value remains the same, the camera vibrates. It can be determined that the condition has changed.
[0074]
In this way, when detecting an image change by image processing such as background subtraction, by estimating a change in an environmental condition or a shooting condition at that time from a feature distribution, the detected image change is truly a person or an object. It can be used as an index for determining whether or not the change is due to a change in environmental conditions or photographing conditions.
[0075]
Further, when it is determined that a change in the environmental condition or the photographing condition has occurred, a malfunction of the image monitoring apparatus can be significantly reduced, for example, by not giving a warning to the user.
[0076]
Next, the configuration and operation of the image monitoring method of the present invention will be described with reference to FIG.
[0077]
The feature of the third embodiment is that the weather and the sunshine state at the time when the image is being captured are estimated from the result of the difference processing, and the reliability of the extraction of the target that is the true detection purpose is improved.
[0078]
FIG. 10 is a feature distribution diagram calculated in various weather conditions and sunshine conditions (time).
[0079]
As described in the first embodiment, the image change calculation unit 3 has a method that can approximate a feature distribution having a complicated shape by a sum of a plurality of normal distributions. By distinguishing, it is possible to estimate the weather and the sunshine state at the time of capturing the image.
[0080]
The upper part of FIG. 10 shows an example of a feature distribution in a case where a vehicle enters the image (assuming that there is no background change). However, as described in the first embodiment, there is entry of an object. In this case, a multimodal feature distribution is obtained in which two (plural) distributions of a peak due to the background region and a peak due to the intruding object region are added.
[0081]
In the upper diagram, it is assumed that the weather is cloudy. However, as shown in the middle diagram of FIG. 10, when the weather is clear, the “shadow” of the intruding object is reflected in the image, and The distribution due to the region appears darker than the distribution due to the background in the feature distribution.
[0082]
Also, when the time changes and it becomes nighttime, the vehicle turns on the headlights, so that the distribution of the reflection area of the headlights appears brighter than other distributions.
[0083]
Furthermore, although not shown, in the case of rainy or snowy weather, a large number of small image changes due to raindrops and the like appear. The characteristics of the feature distribution may change, such as a large variance being observed.
[0084]
In this way, by measuring the peaks and properties of a specific feature distribution, information about the time zone (sunshine condition) such as whether the time when the image was taken is daytime or nighttime, and the imaging area at that time. It is possible to estimate weather conditions such as whether it is sunny, cloudy, rainy, snowy, or fogy.
[0085]
In addition, it is possible to estimate the weather and the sunshine state from a captured image without relying on means such as an input operation of an imaging environment or the like by various other sensors or humans, and to optimally set conditions of the image monitoring device based on the estimation result. Thereby (raising or lowering the sensitivity of the camera, resetting of the detection area, etc.), malfunctions can be greatly reduced, and the reliability of detection of the detection target object can be improved.
[0086]
It is needless to say that the present invention is not limited to the above embodiment, and can be variously modified and implemented without departing from the gist thereof. For example, by providing a plurality of imaging devices, the reliability of identifying a detection target can be improved.
[0087]
【The invention's effect】
As described above, according to the present invention, the detection performance of an image monitoring device can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram of a first embodiment of an image monitoring apparatus according to the present invention.
FIG. 2 is a flowchart of a first embodiment of the image monitoring method of the present invention.
FIG. 3 is a diagram showing an example of setting a detection area in an image.
FIG. 4 is an explanatory diagram of a feature distribution calculation method.
FIG. 5 is an explanatory diagram of a feature distribution calculated at two times with different conditions.
FIG. 6 is an explanatory diagram of calculation of a characteristic distribution when an automatic adjustment function of an imaging device is activated.
FIG. 7 is an explanatory diagram of a method of correcting an image using a result from an image change calculation unit.
FIG. 8 is a diagram illustrating a case where background subtraction processing is performed on an image after correction.
FIG. 9 is a feature distribution diagram calculated from images taken under various conditions for explaining a second embodiment of the image monitoring method of the present invention.
FIG. 10 is a feature distribution diagram calculated in various weather and sunshine states for explaining a third embodiment of the image monitoring method of the present invention.
[Explanation of symbols]
11 Image input section
12 Feature distribution calculator
13 Image change tracking unit (image change calculation unit)
14 Result judgment part
101 Imaging device
102 Display device
103 Communication device

Claims (8)

An image input unit into which an image obtained from a predetermined imaging device is input,
A feature distribution calculating unit for determining the distribution of the feature amount of the input image by the image input unit,
The peak of the distribution of the feature amount by the background area from the distribution of the feature amount obtained by the feature distribution calculating unit, and an image change tracking unit for calculating a change state of track and picture image of this peak every predetermined time,
Calculation result whether these changes state determined by the image change tracking unit, the variation of the peak of the distribution of the feature amount by the tracked background region is determined as the image changes due to variations in environmental conditions and shooting conditions, other features of the A result determination unit that determines the amount distribution as an image change different from the image change,
An image monitoring device, comprising: In the image change tracking unit, obtains the peak position of the distribution of the feature amount before Symbol background area, corrected according to the image background variation in accordance with the peak positions, the difference process and the corrected image and the background image image monitoring apparatus according to claim 1, wherein that you perform. In the image change tracking unit, the image monitoring apparatus according to claim 2, wherein users update the background image using the image subjected to pre-Symbol differential processing. An image input unit into which an image obtained from a predetermined imaging device is input,
  A feature distribution calculation unit that determines a distribution of a feature amount based on a background region of the image input by the image input unit;
An image change tracking unit that calculates a change state of an image from an average value and a distribution state of the distribution of the feature amount by the background region obtained by the feature distribution calculation unit,
From the calculation result of the change state obtained by the image change tracking unit, if only the average value changes, it is determined that the environmental condition has changed, and if only the variance has changed, shooting is performed. A determination unit that determines that the condition has changed, and determines that the intrusion into the image has occurred when both have changed;
An image monitoring device, comprising: An image input step in which an image obtained from a predetermined imaging device is input,
A feature distribution calculating step of obtaining a distribution of the feature amount of the input image in the previous SL image input step,
The peak of the distribution of the feature amount by the background area from the distribution of the feature amount obtained by the feature distribution calculating step, an image change tracking step of calculating a change state of track and picture image of this peak every predetermined time,
Calculation result whether these changes state obtained by the image change tracking step, the variation in the peak of the distribution of the feature amount by the tracked background region is determined as the image changes due to variations in environmental conditions and shooting conditions, other features of the and results determination Priority determination step of determining a distribution of amount as the image variation different image changes,
An image monitoring method, comprising: In the image change tracking step, a peak position of the distribution of the feature amount of the background region is obtained, corrected in accordance with the amount of image background variation in accordance with the peak position, and a difference process is performed between the corrected image and the background image. The image monitoring method according to claim 5, wherein the method is performed. The image monitoring method according to claim 6, wherein in the image change tracking step, a background image is updated using the image on which the difference processing has been performed. An image input step in which an image obtained from a predetermined imaging device is input,
A feature distribution calculating step of obtaining a feature amount distribution by a background region of the image input by the image input step;
An image change tracking step of calculating a change state of an image from an average value and a distribution state of the distribution of the feature amount by the background region obtained in the feature distribution calculation step,
From the calculation result of the change state obtained in the image change tracking step, if only the average value changes, it is determined that the environmental condition has changed, and if only the variance has changed, shooting is performed. A determination step of determining that the condition has changed, and determining that the condition is an intrusion into the image when both are changing;
An image monitoring method, comprising:

Images