JP3933453B2 - Image processing apparatus and moving body monitoring apparatus - Google Patents

Description

以下、このエッジ量比率変化を利用し、侵入者等と蛍光灯のフリッカーとを識別して検知する方法を図４、図５のプログラムフロー図を用いて説明する。
【００３６】
プログラムをスタートさせ、先ず、ステップＳ１１で領域分割手段２３１により画像を８×８のブロックに分割する領域アドレスを設定する。次いで、ステップＳ１２で抽出手段２３２に時刻ｔ−１における基準（背景）画像が取込まれ、ステップＳ１３で前記領域アドレスにしたがって分割されたブロック毎（以下、単に「ブロック毎」という。）に輝度ヒストグラムを４つに分割し、分布Ｂ１〜Ｂ４の画素数が算出され、背景データとして記憶される。その際、ヒストグラムが図６に示すような分布をしている場合、４つの分布Ｂ１〜Ｂ４における分布画素数がほぼ同じになるように輝度閾値Ｔ_１、Ｔ_２、Ｔ_３が決められ、Ｔ_１、Ｔ_２、Ｔ_３が記憶される。なお、分布Ｂ１〜Ｂ４の分布画素数は、照明状態によって変化するので、照明が変化した時、又はタイマーで定期的に計算をし直して、分布Ｂ１〜Ｂ４における分布画素数が等しくなるよう輝度閾値Ｔ_１、Ｔ_２、Ｔ_３が変更される。
【００３７】
ステップＳ１４において時刻ｔにおける判定対象の画像が抽出手段２３２に取込まれ、ステップＳ１５で、ブロック毎に輝度閾値Ｔ_１、Ｔ_２、Ｔ_３にて区分された分布画素数Ｂ１、Ｂ２、Ｂ３、Ｂ４が算出され、ステップＳ１６で、各ブロックのヒストグラム変化画素数Ｈが以下の計算式により前回データ（時刻：ｔ−１）と比較して求められる。
Ｈ=（abs（B1_ｔ−B1_t-1）＋abs（B2_ｔ−B2_t-1）＋abs（B3_ｔ−B3_t-1）＋abs（B4_ｔ−B4_t-1））/２
なお、absは絶対値を示す記号である。
また、今回のデータと前回のデータとの比較だけでなく、今回と前々回、今回と前前々回との比較を行って、最も大きな変化をヒストグラム変化Ｈとしても良い。比較回数が多いほどゆっくりとした移動物体の検知が可能となる。
ステップＳ１７において、この値Ｈが全画面においてヒストグラム変化が所定値、例えば３０％以上の画素で発生したとき、照明異常と判定される。
【００３８】
照明異常と判定された場合は、フローの最初に戻りステップＳ１２の基準画像の取込みから始め、各ブロックのヒストグラム変化を算出して、この異常照明の変化に対応される。
【００３９】
異常照明でないと判断された場合は、ステップＳ１８において、所定の判定休止期間、例えば６回判定を休止する。この判定休止期間６回は、本来異常照明が検出された場合には、以下に説明するように、前前々回のデータ（時刻：ｔ−３）を使用しているために、３回の判定休止で済むが、照明変化が落ち着くまでの余裕を持たせこの回数休止する。
【００４０】
ステップＳ１９で、ブロック毎に各周波数におけるエッジ量が算出されるが、ブロック毎に８種類の周波数成分を選定し、うち４種類を使用する。このように選定した理由はハード的な制限からであり、４種類に限定されるものではない。
【００４１】
本実施形態では、４種類のカットオフ周波数Ａ、Ｂ、Ｃ、Ｄが、例えば周波数の高い順に８種類のうちから１つ置きに４種類Ａ、Ｂ、Ｃ、Ｄが選定されている。時刻t、t−１、t−２、t−３で各周波数Ａ〜Ｄにおけるエッジ量Ｅａ、Ｅｂ、Ｅｃ、Ｅｄが算出され、その結果は表２のとおりとなる。
【表２】

ステップＳ２０で、各周波数Ａ〜Ｄのエッジ量比ａ、ｂ、ｃ、ｄが算出される。その算出方法は、例えば周波数Ａについて示せば、そのエッジ量比率は以下の計算式により各時刻ｔ、ｔ−１、ｔ−２、ｔ−３でのエッジ比率が求められる。
【００４２】
ａ＝Ｅａ/(Ｅａ+Ｅｂ+Ｅｃ+Ｅｄ)
また、他の周波数Ｂ、Ｃ、Ｄにおけるエッジ量の比率ｂ、ｃ、ｄも同様にして算出される。その結果は、表３のとおりとなる。
【表３】

ステップＳ２１では、ブロック毎に周波数Ａのエッジ量Ｅａの増減率Ｅが以下の計算式で算出される。
Ｅ＝abs(Ea_ｔ-Ea_t-1)/Max（Ea_ｔ，Ea_t-1）
なお、Maxはそれに続く括弧内の数値のうち最も大きい数値を示す。また、このエッジ量の増減率は、周波数Ａでなくともよい。ここでもヒストグラム変化算出と同じ様に、上式の今回と前回、今回と前々回、今回と前前々回を比較し、最も変化の大きいものをエッジ増減率Ｅとしても良い。
【００４３】
ステップＳ２２で、ブロック毎にエッジ周波数比率の最大変化値Ｆが算出される。この算出は、実際はゆっくり移動する物体を検出するため、下記の算出式により、前前々回（時刻：ｔ−３）、前々回（時刻：ｔ−２）、前回（時刻：ｔ−１）の３回に分けて最大変化値Ｆが求められる。
【００４４】
前回との比較：
Ｆ１＝Max(abs(a_t-a_t-1),abs(b_t-b_t-1),abs(c_t-c_t-1),abs(d_t-d_t-1))
前々回との比較：
Ｆ２＝Max(abs(a_t-a_t-2),abs(b_t-b_t-2),abs(c_t-c_t-2),abs(d_t-d_t-2))
前前々回との比較：
Ｆ３＝Max(abs(a_t-a_t-3),abs(b_t-b_t-3),abs(c_t-c_t-3),abs(d_t-d_t-3))
そして、最終的には３回分の値Ｆ１〜Ｆ３から最大値が求められる。
Ｆ＝ Max（Ｆ１，Ｆ２，Ｆ３）
【００４５】
ステップＳ２３では、ステップＳ２２で算出されたブロック毎のＦと所定の閾値との比較がなされ、ブロック毎の検知状態が判定される。この判定は、図５のサブプログラムフローに基づいて行われる。すなわち、この判定は全てのブロック６４個（８×８ブロック）について、個別に識別番号ｋ＝１〜６４を付与して次のようにして行われる。
【００４６】
先ずステップＳ２３１で、ブロックｋのステップＳ２２で算出されたエッジ周波数比率の最大変化値Ｆ_Ｋがエッジ周波数比率変化抑制閾値Ｆ_Ｔ（第一の閾値）より大きいか否かが判定される。
【００４７】
Ｆ_Ｋ≦Ｆ_Ｔの場合は、ヒストグラム分布変動やエッジ量増減が大きくても、照明変化によって変動したものと判断し、そしてステップＳ２３３を介して、全部のブロックについてこの判定が終了していなければ（ｋ＜６４）ステップＳ２３１へ戻って次のブロックの判定を継続し、全部のブロックについての判定が終了している場合（ｋ≧６４のとき）には次のステップＳ２４へ進む。
【００４８】
なお、ここではエッジ周波数比率変化抑制閾値Ｆ_Ｔは一定値として処理を行ったが、エッジ周波数比率変化は背景エッジ量により明るさの変化の影響度合いがわずかに異なり、背景エッジ量が非常に少ない時と多い時は照明環境が変動しても各周波数のエッジ量の比率にあまり変化はないが、ほどほどに背景エッジ量があると照明変動の影響を受けやすいので、前記エッジ周波数比率変化抑制閾値Ｆ_Ｔは背景エッジ量によって変えても良い。
【００４９】
また、Ｆ_Ｋ＞Ｆ_Ｔの場合は、ステップＳ２３２において、ステップＳ１６で算出されたヒストグラム変化画素数Ｈ_ｋとその所定の閾値Ｈ_ｔｈとの大小、ステップＳ２１で算出されたエッジ量の増減率Ｅ_ｋとその所定の閾値Ｅ_ｔｈとの大小、及び、ステップＳ２２で算出されたエッジ周波数比率の最大変化値Ｆ_ｋとその所定の閾値Ｆ_ｔｈ（第２の閾値）との大小が判定され、
Ｆ_ｔｈ≧Ｆ_ｋ＞Ｆ_Ｔのとき、
Ｈ_ｋ＞Ｈ_ｔｈ又はＥ_ｋ＞Ｅ_ｔｈである場合には、ブロックｋの画像は何らかの異常状態にあると判定し、検知ブロック数をインクリメント（＋１）してステップＳ２３３へ進み、
Ｈ_ｋ≦Ｈ_ｔｈ及びＥ_ｋ≦Ｅ_ｔｈである場合には、ヒストグラム変化もエッジ量変化も認められなく、しかもエッジ周波数比率変化も中途半端な範囲であるので、ノイズ等何らかの他の要因によるものと判断してそのままステップＳ２３３へ進み、
Ｆ_ｋ＞Ｆ_ｔｈの場合には、Ｈ_ｋ及びＥ_ｋの値の如何に拘わらず、ブロックｋの画像は何らかの異常状態にあると判定し、検知ブロック数をインクリメント（＋１）してステップＳ２３３へ進み、全てのブロックについて判断が終了していなければ（ｋ＜６４）再度ステップＳ２３１に戻って次のブロックの判断を継続し、全てのブロックについての判断が終了した場合（ｋ≧６４の場合）には次のステップＳ２４へ進む。
【００５０】
なお、Ｈ_ｔｈはブロック毎の輝度のヒストグラム変化画素数の閾値、Ｅ_ｔｈはブロック毎の各周波数におけるエッジ量の増減率の閾値であり、それぞれ実験的に誤判定が少なくなるように選択・設定され、また、Ｆ_ｔｈは、ブロック毎のエッジ周波数比率変化の第二の閾値であって、エッジ周波数比率変化抑制閾値Ｆ_Ｔよりも大きい範囲で実験的に誤判定が少なくなるように選択・設定される。また、本発明では述べなかったが各ブロック毎に違う感度を設定することも可能である。
【００５１】
次いで、ステップＳ２４で、前記検知ブロック数が所定の閾値と比較され、この閾値を越える検知ブロック数があると、ステップＳ２５において、コントローラ２へ異常が通報され、モニタ３にその旨を表示するとともにスピーカ４で警報音を発するようになされる。
【００５２】
また、ステップＳ２６において、ステップＳ１５で算出されたブロック毎のヒストグラム分布画素数Ｂ１〜Ｂ４、ステップＳ２１で算出されたブロック毎のエッジ量増減率Ｅａ、及び、ステップＳ２０で算出されたエッジ量の比率ａ、ｂ、ｃ、ｄについて、時刻ｔのデータを前回のデータとし、前回のデータを前々回のデータとし、前々回データを前前々回データとして記憶する。
【００５３】
そして、ステップ２７において、予め定めたヒストグラム輝度閾値更新時間を経過したか否かを判断し、この時間を超えていた場合には、ステップＳ１２に戻って新たに取込んだ画像を基に輝度閾値Ｔ_１、Ｔ_２、Ｔ_３を更新して基準画像データとして取り込み、また、前記時間を超えていない場合には、ステップＳ１４で再度判定すべき画像を取り込んで、上記と同様の判定を繰り返す。
【００５４】
この更新は、朝日あるいは夕焼け時に、徐々に明るくなるあるいは徐々に暗くなるような状態の際に、この明暗の変化によりＢ１、Ｂ２、Ｂ３、Ｂ４の画素数が変わってしまう可能性があるので、これらの画素数がほぼ同じになるように輝度閾値Ｔ_１、Ｔ_２、Ｔ_３を一定間隔で変えて背景データの入れ換えを行うものである。
【００５５】
なお、上記の具体例では、各ブロックの異常を判定するためのパラメータとして、エッジ周波数比率の最大変化値Ｆ_ｋを使用したが、これは判定精度を向上させるためであり、ブロック毎の各周波数エッジ量の比率ａ、ｂ、ｃ及びｄをそのまま判定に使用することもできる。
【００５６】
また、上記の具体例では、各ブロックの異常を判定するためのパラメータとして、エッジ周波数比率の最大変化値Ｆ_ｋのみでなく、輝度ヒストグラム変化画素数Ｈ_ｋ及びエッジ量の増減率Ｅ_ｋをも使用したが、これは侵入者検知に際し、少なくとも侵入者があった場合にそれを見逃すことがないようにするためであり、単にエッジ周波数比率の最大変化値Ｆ_ｋとエッジ周波数比率変化の第２の閾値Ｆ_ｔｈとの大小に基づいてブロック毎の異常を検知することもできる。
【００５７】
さらに、上記の具体例では、画像を８×８＝６４個のブロックに分割して各ブロックの異常を判定し、その異常と判定された検知ブロックの数から画像全体の異常について判定する例について記載したが、画像全体を１つのブロックとして異常を検知することもでき、その場合は画素数ｋ＝１として処理してエッジ周波数比率変化Ｆが上記第２の閾値Ｆ_ｔｈを越えているか否かを判定すればよい。
【００５８】
【発明の効果】
本発明によれば、以下の効果を奏する。
【００５９】
第１及び第２の発明によれば、基準画像のエッジ量比率から判定画像のエッジ量比率が変化したことを検出することにより、照明の影響を受けずに判定画像が基準画像から変化していることが判定できるようになる。
【００６０】
また、第３の発明によれば、少なくとも領域毎に照明の影響を受けずに判定画像が基準画像から変化していることが判定できるので、画像の中の移動している部分を判定する精度が向上する。
【００６１】
また、第４及び第５の発明によれば、蛍光灯のフリッカー、太陽光、影などの環境変動があっても基準画像のエッジ量比率と判定画像のエッジ量比率はほとんど変化しないことから、環境変化による画像変化と移動体による画像変化とを誤って判定することがなくなる。
【００６２】
また、第６の発明によれば、基準画像のエッジ量比率と判定画像のエッジ量比率がほとんど変化していない場合には環境変動によるものと判定することで移動体の誤った検出を防止することができるとともに、基準画像のエッジ量比率と判定画像のエッジ量比率との差から精度良く移動体を検出することが可能となる。
【００６３】
さらに、第７の発明によれば、輝度のヒストグラム変化及びエッジ増減率からも移動体の存在を判定しえるので、移動体が存在しているにも拘わらず存在していないとする重大な誤判定が少なくなり、第８の発明によれば、判定結果の精度が向上するために、より照明の影響を受けずに移動体の判定ができるようになり、第９の発明によれば、少なくとも領域毎に照明の影響を受けずに判定画像が基準画像から変化していることが判定できるので、より照明の影響を受けずに移動体の判定ができるようになる。
【図面の簡単な説明】
【図１】 本発明の監視システムの全体構成図である。
【図２】 図１における監視カメラの構成図である。
【図３】 図２の移動体検出手段の構成を示すブロック図である。
【図４】 図２の移動体検出手段によって実行されるエッジ周波数変化による侵入者検知フローチャートである。
【図５】 図４のブロック毎の検知状態を判定するフローチャートである。
【図６】 図４に関連する画像を説明する輝度ヒストグラム図である。
【図７】 図４に関連し抽出される周波数成分を例示した図である。
【図８】 背景が一様で輝度変化があまりないブロックにおいて、明るさが変わった場合のヒストグラム変化を説明する図であり、（Ａ）は明るい場合、（Ｂ）は暗い場合を示す。
【図９】 あるブロックの所定周波数でのエッジ強度が明るいときと暗いときとでどのように変化するかを説明する図であり、（Ａ）は画像を表し、（Ｂ）はその画像の特定ブロックの所定周波数におけるエッジ強度を示す図である。
【符号の説明】
１・・・・・監視カメラ
１０・・・・撮像手段
２０・・・・制御手段
２１・・・・信号増幅手段
２３・・・・移動体検出手段
２３０・・・画像入力手段
２３１・・・領域分割手段
２３２・・・抽出手段
２３３・・・エッジ量算出手段
２３４・・・比率算出手段
２３５・・・基準比率記憶手段
２３６・・・判定手段
２３７・・・出力手段[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and a moving body monitoring apparatus that detect, for example, an intrusion of a suspicious person into a convenience store, a department store, a building, or an intruding object by image processing. In particular, the present invention relates to an image processing apparatus and a moving object monitoring apparatus that efficiently distinguish between changes that appear in an image due to illumination fluctuations, diurnal fluctuations of sunlight, changes in shadows, and the like, and changes caused by intruders.
[0002]
[Prior art]
Conventionally, an image processing apparatus that extracts a change area by calculating a difference between a reference image such as a background and a determination image such as a monitored object, and detects an intruder or the like from the area or shape of the extracted change area It has been known. Such an image processing apparatus distinguishes it from an intruder as an illumination variation when the entire image changes suddenly. That is, as a result of the difference calculation between the reference image and the determination image, if a luminance change appears on the entire surface of the image, it is determined that the illumination variation is present. In addition, there is also known one that determines that the illumination variation occurs when the edge amount in the change region does not change much.
[0003]
[Problems to be solved by the invention]
However, the amount of edge of an input image, that is, the amount of edge in an image of a specific frequency varies depending on the flicker of fluorescent lamps or sunlight when the clouds block the clouds due to automatic exposure control of the surveillance camera. This variation in the edge amount often occurs indistinguishable from the variation in the edge amount caused by an intruder or the like.
[0004]
For this reason, in a conventional image processing apparatus, if a determination threshold value that can reliably detect an intruder or the like is set, a flicker or the like of a fluorescent lamp is erroneously detected as an intruder or the like. If a determination threshold value that does not reliably detect an intruder is set, an intruder may not be detected. In particular, when an image is determined by being divided into a plurality of blocks, it is extremely difficult to distinguish from partial fluctuations due to spot light.
[0005]
Hereinafter, these conventional techniques will be illustrated and supplementarily described.
[0006]
FIG. 8 is a luminance histogram distribution of a reference image according to the prior art. FIG. 8A shows the distribution when the illumination is bright and FIG. 8B shows the distribution when it is dark.
[0007]
In this reference image, when the background image has a little change in luminance, for example, a wall having a small pattern, and the illumination is slightly dark from a bright state, the histogram distribution has a threshold T as shown in FIG. ₁ The distribution of the number of pixels in the vicinity changes greatly. However, if this change is detected and an object to be monitored is determined, the intruder may change the same tendency even though this change is caused by the brightness of the illumination. It is extremely difficult to determine whether it is caused by a person or due to a change in lighting. T ₂ ~ T ₃ Are other luminance thresholds.
[0008]
FIG. 9 shows a determination method based on edge information of a predetermined frequency in the prior art. FIG. 9A shows an image, FIG. 9B shows edge strength in this image, and a thick dotted line indicates When the illumination is bright, the solid line indicates the edge intensity at a predetermined frequency when the illumination is dark.
[0009]
In this determination method, a determination is made by determining a pixel that is equal to or more than a dotted line (shown in the diagram (B)) with a thin edge strength as an edge region, and a region below the dotted line as a non-edge region. According to this method, as the illumination becomes brighter, the edge becomes stronger and thicker. As a result, a new edge appears where it was not recognized as an edge. Therefore, if the edge area increases beyond a certain level, it is always determined to be abnormal, causing a malfunction.
[0010]
Therefore, in the determination method based on the change rate of the edge amount using the histogram change and the frequency edge as described above, the determination should be made when there is a relatively small brightness variation such as flicker of a fluorescent lamp or blockage by a cloud of sunlight. Then, it was difficult to determine whether it was due to an intruder or a change in lighting.
[0011]
SUMMARY OF THE INVENTION The present invention provides an image processing apparatus and a moving body monitoring apparatus that can efficiently distinguish an intruder or the like from a flicker of a fluorescent lamp or a cloud block.
[0012]
[Means for Solving the Problems]
The inventors of the present invention have conducted various experiments on changes in background images due to flickering of fluorescent lamps, diurnal fluctuations of sunlight, shadows, light, automatic exposure control of cameras, and image changes due to intruders, etc. The brightness value of the image changes due to flickering of fluorescent lamps, diurnal fluctuations of sunlight, shadows, light, camera automatic exposure control, etc., but edges appearing at each frequency when multiple frequency components are extracted from the image The ratio of the amount hardly changes, and on the other hand, it is found that the luminance value of the image is changed by the moving body such as an intruder, and the ratio of the edge amount appearing in a plurality of frequency components is also changed, thereby completing the present invention. It's just a thing.
[0013]
According to a first aspect of the present invention, there is provided an image processing device for determining a change in an image input from an imaging device,
(1) extraction means for extracting a plurality of frequency components from an image input from the imaging means;
(2) edge amount calculating means for calculating the respective edge amounts from the extracted frequency components;
(3) Ratio calculating means for calculating the respective edge amount ratios from the calculated edge amounts of the respective frequency components;
(4) The edge amount ratio of each frequency of the reference image input from the imaging means is compared with the edge amount ratio of each frequency of the determination image to be determined, and the change in the edge amount ratio of both frequencies is detected. A ratio change detecting means for detecting;
(5) Determination means for determining a change between the determination image and the reference image based on a change in the edge amount ratio of each frequency detected by the ratio change detection means;
An image processing apparatus is provided.
[0014]
According to this configuration, by detecting that the edge amount ratio of the determination image has changed from the edge amount ratio of the reference image, it is possible to determine how the determination image has changed from the reference image without being affected by illumination. Can be judged.
[0015]
According to the second aspect of the present invention, in the first aspect, the determining means in (5) determines based on a maximum value in absolute value among changes in the edge amount ratio of each frequency. There is provided an image processing apparatus for discriminating a change between an image and a reference image. According to such a configuration, since the accuracy of the determination result is improved, it is possible to determine how the determination image is changed from the reference image without being influenced by illumination.
[0016]
Furthermore, according to the third aspect of the present invention, in the first or second aspect, the image processing apparatus further includes means for dividing the image input from the imaging means into a plurality of regions, The means (1) to (5) are configured to perform respective processing for each image in each region, and
(6) An output unit that calculates the number of areas to which the determination output is sent from the determination unit of (5), and determines and outputs a change between the determination image and the reference image based on the number of the areas;
Is provided.
[0017]
According to such a configuration, it is possible to determine that the determination image has changed from the reference image without being affected by illumination at least for each region, so that the accuracy of determining the changing portion in the image is improved.
[0018]
Moreover, according to another 1st and 2nd aspect of this invention, it is a moving body monitoring apparatus which detects a moving body from the image | photographed image,
(1) extraction means for extracting a plurality of frequency components from an image input from the imaging means;
(2) edge amount calculating means for calculating the respective edge amounts from the extracted frequency components;
(3) Ratio calculating means for calculating the respective edge amount ratios from the calculated edge amounts of the respective frequency components;
(4) The edge amount ratio of each frequency of the reference image input from the imaging means is compared with the edge amount ratio of each frequency of the determination image to be determined, and the change in the edge amount ratio of both frequencies is detected. A ratio change detecting means for detecting;
(5) Judgment means for judging whether or not the change in the edge amount ratio of each frequency detected by the ratio change detection means exceeds a threshold value for distinguishing between image fluctuations due to the moving body and other fluctuations;
A mobile monitoring device comprising:
The threshold value of (5) in the mobile object monitoring device is a first threshold value for distinguishing between image fluctuations due to the environment and other fluctuations as the change in the ratio detected by the ratio change detection means, or image fluctuations due to the mobile object and others There is provided a moving body monitoring apparatus that is a second threshold value that is larger than the first threshold value that distinguishes the fluctuations of the moving object.
[0019]
According to such a configuration, the edge amount ratio of the reference image and the edge amount ratio of the judgment image hardly change even if there are environmental fluctuations such as flicker of fluorescent lamps, sunlight, and shadows. An image change caused by the body is not erroneously determined.
[0020]
According to another third aspect of the present invention, in the other first aspect, the change in the ratio detected by the ratio change detection means by the determination means in (5) is caused by an image fluctuation caused by the environment. A first threshold value that distinguishes between the image and other fluctuations, and a second threshold value that is greater than the first threshold value that distinguishes image fluctuations caused by the moving object from other fluctuations, and There is provided a moving body monitoring device which is a judging means for prohibiting sending of a moving body judgment output when a threshold value is not exceeded, and sending a moving body judgment output when a second threshold value is exceeded.
[0021]
According to such a configuration, when the edge amount ratio of the reference image and the edge amount ratio of the determination image are hardly changed, it is possible to prevent erroneous detection of the moving body by determining that it is due to environmental fluctuations. The moving object can be detected with high accuracy from the difference between the edge amount ratio of the reference image and the edge amount ratio of the determination image.
[0022]
Further, according to another fourth aspect of the present invention, in the other third aspect, the determination means of (5) further includes that at least one of a luminance histogram change and an edge increase / decrease rate has a predetermined threshold value. There is provided a moving body monitoring apparatus that has a means for determining whether or not it exceeds, and that outputs a determination output even when at least one of a luminance histogram change and an edge increase / decrease rate exceeds a predetermined threshold.
[0023]
According to such a configuration, since the presence of the moving object can be determined also from the luminance histogram change and the edge increase / decrease rate, there is less serious misjudgment that the moving object exists but does not exist. .
[0024]
According to another fifth aspect of the present invention, in the other first to fourth aspects, the determining means (5) is an absolute value among the changes in the edge amount ratio of each frequency. There is provided a moving body monitoring apparatus that makes a determination based on the maximum value. According to this configuration, since the accuracy of the determination result is improved, it is possible to determine the moving body without being affected by the illumination.
[0025]
According to another sixth aspect of the present invention, in any one of the other first to fourth aspects, the image processing apparatus further converts the image input from the imaging unit into a plurality of regions. Means for dividing, and the means of (1) to (5) are configured to perform respective processing for each image in each region; and
(6) An output unit that calculates the number of areas to which the determination output is transmitted from the determination unit of (5), and transmits a moving body detection output based on the number of the areas;
Is provided.
[0026]
According to such a configuration, since it can be determined that the determination image has changed from the reference image without being affected by illumination at least for each region, the accuracy of the determination result is improved, so that it is less affected by illumination. The mobile object can be determined.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. The present invention is not limited to this.
[0028]
FIG. 1 shows the overall configuration of the monitoring system of the present invention, FIG. 2 is a configuration diagram of the monitoring camera body in FIG. 1, and FIG. 3 is a block diagram of the moving body detection means 23 in FIG.
[0029]
The surveillance system includes a plurality of surveillance camera bodies 1 that are appropriately installed in places where surveillance is required, these surveillance camera bodies 1 are connected by cables, and record / playback video sent from these surveillance camera bodies 1. Alternatively, it includes a controller 2 for sending an alarm signal, a monitor 3 for monitoring a video, and a speaker 4 for sending a sound such as an alarm when the video is monitored and an abnormal state is detected. In this monitoring system, each surveillance camera body 1 is equipped with a moving body detection means (described later) for detecting a moving body such as an intruder.
[0030]
Each surveillance camera body 1 includes an imaging unit 10 that captures an image of the monitored object, a control unit 20 that inputs a signal from the imaging unit 10 and performs predetermined signal processing, and performs focus and exposure control of the imaging unit 10. A display unit 30 for displaying an image or the like based on a signal processed by the control unit 20, a monitor I / F 40 for sending a monitor or adjustment signal to a remote location, and similarly processing the captured image data such as compression to a remote location It consists of a communication I / F 50 to be transmitted and a storage unit 60 for storing image data and the like. Among these, the imaging unit 10 includes a lens and an imaging element, and the control unit 20 includes a signal amplification unit 21, a display / output unit 22, a moving body detection unit 23, and a communication unit 24.
[0031]
In FIG. 2, a solid line with an arrow from the communication I / F 50 is a LAN (Local Area Network), and the arrow indicates transmission / reception of a signal. This LAN is connected to the daisy chain, and transmits various control signals and image data to the controller 2 or other cameras.
[0032]
The moving object detection means 23 in the control unit 20 includes an image input means 230 for capturing a reference image such as a background and a determination image of the monitored object, and an area dividing means for dividing the image area into predetermined blocks, for example, 8 × 8 blocks. 231, an extraction unit 232 that extracts a plurality of frequency components from each image, for example, four frequency components by filtering, an edge amount calculation unit 233 that calculates an edge amount from each extracted frequency component by filtering, and an edge amount of each frequency component Calculate the edge ratio from Ratio calculation means 234, a reference ratio storage means 235 for storing the calculated edge amount ratio value of the reference image, and further comparing the edge amount ratio in the reference image with the edge amount ratio value in the determination image to determine whether or not a predetermined threshold value is exceeded. Determination means 236, and output means 237 for sending a predetermined output based on the result of the determination means 236.
[0033]
In the moving body detection unit 23, first, the reference image and the determination image are taken into the image input unit 230, and these images are divided into 8 × 8 blocks by the region dividing unit 231, and four frequency values are obtained from each block image. Components are extracted by the extraction means 232. Next, the edge amount calculation unit 233 calculates the edge amount in each frequency component of the reference image and the determination image, and then the ratio calculation unit 234 calculates the ratio of the edge amount in each frequency component. Then, the determination unit 236 compares the determination image edge amount ratio in each frequency component with the reference image edge amount ratio in each frequency component stored in the reference ratio storage unit 235, and compares the comparison difference with a predetermined threshold value. Then, a predetermined output is sent from the output means 237.
[0034]
According to various experiments by the present inventors, the ratio of the edge amount appearing at each frequency when extracting a plurality of frequency components from the image is as follows:
(1) The ratio of the edge amount hardly changes even though the luminance value of the image changes due to flicker of fluorescent lamps, diurnal fluctuation of sunlight, shadow, light, automatic exposure control of the camera, etc.
(2) The luminance value of the image is changed by the moving body such as an intruder, and the ratio of the edge amount is also changed.
It was found.
[0035]
For example, FIGS. 7A to 7D show an experimental example of how the edge intensities of four frequency components A to D in one region change between when the image is bright and when it is dark. It is a figure, a dotted line shows edge strength when bright, and a solid line shows edge strength when dark. Table 1 summarizes the experimental results, and shows the results of determining the ratio of the edge amount and the edge amount for the four frequency components A to D when the illumination is bright and dark. Based on this table, the difference in edge amount ratio between bright and dark is calculated: 1.1% at frequency A, 0.7% at frequency B, 0.9% at frequency C, and 0. 8%. From this result, it can be seen that the difference in the edge amount ratio of only the background image is an extremely small value of about 1% even at the most changing frequency.
[Table 1]

Hereinafter, a method for identifying and detecting an intruder or the like and a flicker of a fluorescent lamp using this edge amount ratio change will be described with reference to the program flow diagrams of FIGS.
[0036]
The program is started. First, in step S11, an area address for dividing the image into 8 × 8 blocks is set by the area dividing means 231. Next, in step S12, the reference (background) image at time t-1 is taken into the extraction unit 232 in step S12, and in step S13, the luminance is divided for each block (hereinafter simply referred to as “each block”) divided according to the area address. The histogram is divided into four, and the number of pixels of the distributions B1 to B4 is calculated and stored as background data. At this time, when the histogram has a distribution as shown in FIG. 6, the luminance threshold T is set so that the number of distribution pixels in the four distributions B1 to B4 is substantially the same. ₁ , T ₂ , T ₃ Is decided, T ₁ , T ₂ , T ₃ Is memorized. Since the number of distribution pixels in the distributions B1 to B4 changes depending on the illumination state, the luminance is set so that the number of distribution pixels in the distributions B1 to B4 becomes equal when the illumination changes or by periodically recalculating with a timer. Threshold T ₁ , T ₂ , T ₃ Is changed.
[0037]
In step S14, the image to be determined at time t is taken into the extraction unit 232, and in step S15, the luminance threshold T is set for each block. ₁ , T ₂ , T ₃ The distribution pixel numbers B1, B2, B3, and B4 divided by are calculated. In step S16, the histogram change pixel number H of each block is compared with the previous data (time: t-1) by the following formula. Desired.
H = (abs (B1 _t −B1 _t-1 ) + Abs (B2 _t −B2 _t-1 ) + Abs (B3 _t −B3 _t-1 ) + Abs (B4 _t −B4 _t-1 )) / 2
Here, abs is a symbol indicating an absolute value.
Further, not only the comparison between the current data and the previous data but also the comparison between the current time and the previous time, and the current time and the previous time, the largest change may be set as the histogram change H. As the number of comparisons increases, a slower moving object can be detected.
In step S <b> 17, when this value H is a histogram change in a predetermined value, for example, 30% or more of pixels on the entire screen, it is determined that illumination is abnormal.
[0038]
If it is determined that the illumination is abnormal, the process returns to the beginning of the flow and starts from the capture of the reference image in step S12, and the histogram change of each block is calculated to cope with the abnormal illumination change.
[0039]
If it is determined that the illumination is not abnormal, in step S18, the determination is suspended for a predetermined determination suspension period, for example, six times. In the determination pause period of 6 times, when abnormal illumination is originally detected, as described below, since the data of the previous time (time: t-3) is used, 3 determination pauses are performed. However, it is paused this number of times to allow for the lighting change to settle down.
[0040]
In step S19, the edge amount at each frequency is calculated for each block. Eight types of frequency components are selected for each block, and four of them are used. The reason for such selection is due to hardware limitations and is not limited to four types.
[0041]
In the present embodiment, four types of cut-off frequencies A, B, C, and D are selected from every other eight types, for example, in descending order of frequency, for example, four types A, B, C, and D are selected. Edge amounts Ea, Eb, Ec, Ed at the respective frequencies A to D are calculated at times t, t-1, t-2, and t-3, and the results are as shown in Table 2.
[Table 2]

In step S20, the edge amount ratios a, b, c, and d of the frequencies A to D are calculated. As for the calculation method, for example, for frequency A, the edge ratio at each time t, t-1, t-2, and t-3 can be obtained by the following calculation formula.
[0042]
a = Ea / (Ea + Eb + Ec + Ed)
The ratios b, c, and d of edge amounts at other frequencies B, C, and D are calculated in the same manner. The results are shown in Table 3.
[Table 3]

In step S21, the increase / decrease rate E of the edge amount Ea of the frequency A is calculated for each block by the following calculation formula.
E = abs (Ea _t -Ea _t-1 ) / Max (Ea _t , Ea _t-1 )
In addition, Max shows the largest numerical value among the numerical values in the following parentheses. Further, the increase / decrease rate of the edge amount may not be the frequency A. Here, as in the case of the histogram change calculation, the present time and the previous time, the current time and the previous time, the current time and the previous time in the above formula may be compared, and the edge change rate E may be the largest change.
[0043]
In step S22, the maximum change value F of the edge frequency ratio is calculated for each block. In this calculation, in order to detect an object that moves slowly, the previous calculation (time: t-3), the previous measurement (time: t-2), the previous calculation (time: t-1), and the previous calculation (time: t-1) The maximum change value F is obtained separately.
[0044]
Comparison with last time:
F1 = Max (abs (a _t -a _t-1 ), abs (b _t -b _t-1 ), abs (c _t -c _t-1 ), abs (d _t -d _t-1 ))
Comparison with last time:
F2 = Max (abs (a _t -a _t-2 ), abs (b _t -b _t-2 ), abs (c _t -c _t-2 ), abs (d _t -d _t-2 ))
Comparison with previous time:
F3 = Max (abs (a _t -a _t-3 ), abs (b _t -b _t-3 ), abs (c _t -c _t-3 ), abs (d _t -d _t-3 ))
Finally, the maximum value is obtained from the values F1 to F3 for three times.
F = Max (F1, F2, F3)
[0045]
In step S23, F for each block calculated in step S22 is compared with a predetermined threshold value, and the detection state for each block is determined. This determination is made based on the subprogram flow of FIG. That is, this determination is performed as follows by assigning identification numbers k = 1 to 64 individually to all 64 blocks (8 × 8 blocks).
[0046]
First, in step S231, the maximum change value F of the edge frequency ratio calculated in step S22 of block k. _K Is the edge frequency ratio change suppression threshold F _T It is determined whether or not it is greater than (first threshold).
[0047]
F _K ≦ F _T In this case, even if the histogram distribution fluctuation or the edge amount increase / decrease is large, it is determined that the fluctuation is caused by the illumination change, and if this determination is not completed for all the blocks via step S233 (k <64). Returning to step S231, the determination of the next block is continued, and if the determination for all the blocks has been completed (when k ≧ 64), the process proceeds to the next step S24.
[0048]
Here, the edge frequency ratio change suppression threshold F _T Is processed as a constant value, but the edge frequency ratio changes slightly differently depending on the amount of background edge, even when the lighting environment changes when the background edge amount is very small or large The ratio of the edge amount of each frequency does not change much, but if there is a moderate amount of background edge, it is easily affected by illumination fluctuations. _T May be changed depending on the amount of the background edge.
[0049]
F _K > F _T In this case, in step S232, the number H of histogram change pixels calculated in step S16. _k And its predetermined threshold H _th The edge amount increase / decrease rate E calculated in step S21. _k And its predetermined threshold E _th And the maximum change value F of the edge frequency ratio calculated in step S22. _k And its predetermined threshold F _th (The second threshold value) is determined,
F _th ≧ F _k > F _T When,
H _k > H _th Or E _k > E _th If it is, it is determined that the image of the block k is in some abnormal state, the number of detected blocks is incremented (+1), and the process proceeds to step S233.
H _k ≦ H _th And E _k ≦ E _th In this case, since neither a histogram change nor an edge amount change is recognized, and the edge frequency ratio change is in the middle of the range, it is determined that it is caused by some other factor such as noise, and the process proceeds to step S233 as it is.
F _k > F _th In the case of _k And E _k Regardless of the value of, the image of block k is determined to be in some abnormal state, the number of detected blocks is incremented (+1), and the process proceeds to step S233. If the determination has not been completed for all blocks (k <64) Returning to step S231 again, the determination of the next block is continued, and when the determination for all the blocks is completed (when k ≧ 64), the process proceeds to the next step S24.
[0050]
H _th Is the threshold value of the number of pixels of histogram change in luminance for each block, E _th Is a threshold value of the rate of increase / decrease of the edge amount at each frequency for each block, which is selected and set experimentally so as to reduce misjudgment. _th Is the second threshold value of the edge frequency ratio change for each block, and the edge frequency ratio change suppression threshold F _T Is selected and set so as to reduce erroneous determination experimentally within a larger range. Although not described in the present invention, it is possible to set a different sensitivity for each block.
[0051]
Next, in step S24, the number of detected blocks is compared with a predetermined threshold value. If there is a detected block number exceeding this threshold value, an abnormality is notified to the controller 2 in step S25, and that fact is displayed on the monitor 3. An alarm sound is emitted from the speaker 4.
[0052]
In step S26, the histogram distribution pixel numbers B1 to B4 for each block calculated in step S15, the edge amount increase / decrease rate Ea for each block calculated in step S21, and the ratio of the edge amount calculated in step S20. For a, b, c, and d, the data at time t is the previous data, the previous data is the previous data, and the previous data is stored as the previous data.
[0053]
In step 27, it is determined whether or not a predetermined histogram brightness threshold update time has passed. If this time has been exceeded, the process returns to step S12 to return the brightness threshold based on the newly captured image. T ₁ , T ₂ , T ₃ Is updated and captured as reference image data. If the time has not been exceeded, an image to be determined again is acquired in step S14, and the same determination as above is repeated.
[0054]
In this update, the number of pixels of B1, B2, B3, and B4 may change due to this change in brightness when it becomes brighter or darker in the morning or sunset. The luminance threshold T is set so that the number of pixels is substantially the same. ₁ , T ₂ , T ₃ Is changed at regular intervals to replace the background data.
[0055]
In the above specific example, the maximum change value F of the edge frequency ratio is used as a parameter for determining the abnormality of each block. _k This is to improve the determination accuracy, and the ratios a, b, c, and d of each frequency edge amount for each block can be used for the determination as they are.
[0056]
In the above specific example, the maximum change value F of the edge frequency ratio is used as a parameter for determining the abnormality of each block. _k As well as the luminance histogram change pixel number H _k And edge rate change rate E _k This is to prevent an intruder from being missed at least when there is an intruder, and is simply the maximum change F of the edge frequency ratio. _k And second threshold value F of edge frequency ratio change _th It is also possible to detect an abnormality for each block based on the size.
[0057]
Furthermore, in the above specific example, the image is divided into 8 × 8 = 64 blocks, the abnormality of each block is determined, and the abnormality of the entire image is determined from the number of detected blocks determined to be abnormal. Although described, abnormality can also be detected with the entire image as one block. In that case, processing is performed with the number of pixels k = 1, and the edge frequency ratio change F is the second threshold value F. _th What is necessary is just to determine whether it is over.
[0058]
【The invention's effect】
The present invention has the following effects.
[0059]
According to the first and second aspects, by detecting that the edge amount ratio of the determination image is changed from the edge amount ratio of the reference image, the determination image is changed from the reference image without being affected by illumination. It can be determined that
[0060]
Further, according to the third invention, it is possible to determine that the determination image has changed from the reference image without being affected by the illumination at least for each region. Therefore, the accuracy of determining the moving portion in the image Will improve.
[0061]
In addition, according to the fourth and fifth inventions, the edge amount ratio of the reference image and the edge amount ratio of the determination image hardly change even when there is an environmental variation such as flicker, sunlight, and shadow of a fluorescent lamp. An image change caused by an environmental change and an image change caused by a moving object are not erroneously determined.
[0062]
Further, according to the sixth invention, when the edge amount ratio of the reference image and the edge amount ratio of the determination image are hardly changed, it is determined that the change is caused by the environmental change, thereby preventing erroneous detection of the moving object. In addition, the moving object can be detected with high accuracy from the difference between the edge amount ratio of the reference image and the edge amount ratio of the determination image.
[0063]
Furthermore, according to the seventh aspect, since the presence of the moving object can be determined from the luminance histogram change and the edge increase / decrease rate, it is a serious error that the moving object exists but does not exist. According to the eighth aspect, the accuracy of the determination result is improved, so that the moving object can be determined without being affected by the illumination. According to the ninth aspect, at least Since it can be determined that the determination image has changed from the reference image without being affected by the illumination for each region, the moving object can be determined without being affected by the illumination.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a monitoring system according to the present invention.
2 is a configuration diagram of the surveillance camera in FIG. 1. FIG.
3 is a block diagram illustrating a configuration of a moving object detection unit in FIG. 2;
4 is an intruder detection flowchart according to edge frequency change executed by the moving body detection means of FIG. 2. FIG.
FIG. 5 is a flowchart for determining a detection state for each block in FIG. 4;
6 is a luminance histogram diagram illustrating an image related to FIG. 4; FIG.
FIG. 7 is a diagram illustrating frequency components extracted in association with FIG. 4;
FIGS. 8A and 8B are diagrams for explaining a histogram change when the brightness changes in a block having a uniform background and little change in luminance. FIG. 8A shows a case where the brightness is bright and FIG. 8B shows a case where the brightness is dark.
FIGS. 9A and 9B are diagrams for explaining how the edge intensity at a predetermined frequency of a certain block changes between when it is bright and when it is dark. FIG. 9A shows an image, and FIG. It is a figure which shows the edge strength in the predetermined frequency of a block.
[Explanation of symbols]
1 ... Surveillance camera
10... Imaging means
20... Control means
21... Signal amplification means
23... Moving object detection means
230 ... Image input means
231 ... Area dividing means
232 ... Extraction means
233... Edge amount calculation means
234... Ratio calculation means
235: Reference ratio storage means
236 ... Determination means
237 ... Output means

Claims (9)

An image processing device for determining a change in an image input from an imaging device,
(1) extraction means for extracting a plurality of frequency components from an image input from the imaging means;
(2) edge amount calculating means for calculating the respective edge amounts from the extracted frequency components;
(3) Ratio calculating means for calculating the respective edge amount ratios from the calculated edge amounts of the respective frequency components;
(4) The edge amount ratio of each frequency of the reference image input from the imaging means is compared with the edge amount ratio of each frequency of the determination image to be determined, and the change in the edge amount ratio of both frequencies is detected. A ratio change detecting means for detecting;
(5) Determination means for determining a change between the determination image and the reference image based on a change in the edge amount ratio of each frequency detected by the ratio change detection means;
An image processing apparatus comprising: 2. The image according to claim 1, wherein the determination unit of (5) determines a change between a determination image and a reference image based on a maximum value of absolute values among changes in the edge amount ratio of each frequency. Processing equipment. The image processing apparatus further includes means for dividing the image input from the imaging means into a plurality of areas, and the means (1) to (5) perform each process for each image in each area. And
(6) An output unit that calculates the number of areas to which the determination output is sent from the determination unit of (5), and determines and outputs a change between the determination image and the reference image based on the number of the areas;
The image processing apparatus according to claim 1, further comprising: A moving body monitoring device that detects a moving body from a captured image,
(1) extraction means for extracting a plurality of frequency components from an image input from the imaging means;
(2) edge amount calculating means for calculating the respective edge amounts from the extracted frequency components;
(3) Ratio calculating means for calculating the respective edge amount ratios from the calculated edge amounts of the respective frequency components;
(4) The edge amount ratio of each frequency of the reference image input from the imaging means is compared with the edge amount ratio of each frequency of the determination image to be determined, and the change in the edge amount ratio of both frequencies is detected. A ratio change detecting means for detecting;
(5) determination means for determining whether or not the change in the edge amount ratio of each frequency detected by the ratio change detection means exceeds a predetermined threshold;
A moving body monitoring apparatus comprising: The predetermined threshold value of (5) is a first threshold value that distinguishes image fluctuations due to environment and other fluctuations, or a second threshold value that is greater than the first threshold value that distinguishes image fluctuations caused by a moving object and other fluctuations. The moving body monitoring device according to claim 4. The determination means of (5) has a first threshold value for distinguishing between image fluctuations due to the environment and other fluctuations, and image fluctuations caused by the moving body and other fluctuations. It is determined whether or not a second threshold value that is larger than the first threshold value is exceeded, and if the first threshold value is not exceeded, transmission of the determination output of the moving object is prohibited and the second threshold value is exceeded. 5. The moving body monitoring apparatus according to claim 4, wherein the moving body monitoring apparatus is a determination means for sending a determination output of the moving body. The determination means of (5) further includes means for determining whether or not at least one of the luminance histogram change and the edge increase / decrease rate exceeds a predetermined threshold, and at least one of the luminance histogram change and the edge increase / decrease rate is 7. The moving body monitoring apparatus according to claim 6, wherein a determination output is also sent when a predetermined threshold value is exceeded and a change in the ratio detected by the ratio change detection means exceeds the first threshold value. The moving body monitoring apparatus according to any one of claims 4 to 7, wherein the determination means (5) determines based on a maximum value in absolute value among changes in the edge amount ratio of each frequency. The image processing apparatus further includes means for dividing the image input from the imaging means into a plurality of areas, and the means (1) to (5) perform each process for each image in each area. And
(6) An output means for calculating the number of areas to which a determination output is transmitted from the determination means in (5), and transmitting a moving body detection output when the number of the areas exceeds a predetermined number;
The moving body monitoring apparatus according to any one of claims 4 to 8, further comprising:

Images