JP3684672B2 - Remote monitoring method - Google Patents

Description

【００２０】
ここで、関数ＢＯＯＬ（）は、（）内の式が真のとき１、偽のとき０なる関数である。
【００２１】
つまり、動きベクトルの絶対値の総和をとることにより、監視画像全体の動き量を知ることができる。
【００２２】
自動発呼制御部（コンピュータ）１４では、侵入物検出部１３で検出された監視画像全体の動き量が定数ｄ₁より大きい場合は、何か侵入物（異常）が検出されたとして、あらかじめ指定されたオペレータ側（ホスト局）に自動発呼をし、子局側のコーデックと接続させ、その侵入物が検出された監視地点の映像を伝送する。
【００２３】
例えば、この時オペレータ側では、監視ポイントからの呼を自動着信し、送られてくる監視画像を自動的にディスプレイ上に表示することで、オペレータにその異常を知らせることができる。
【００２４】
このように、監視ポイントが異常を感知して自動的に発呼することによって、オペレータ側の監視業務は大きく軽減され、かつ、異常の発生を見落とすことがなくなる。
【００２５】
（第２の実施形態）
上記の第１の実施形態では、マクロブロック単位の絶対値の総和（（１）式参照）をとって監視画像全体の動き量を算出している。その為、監視画像内に写しだされている物体に微振動などがあった場合にも、（１）式による計算の結果、監視画像の動き量が大きいと判断され、侵入物があったものとしてとらえてしまう可能性がある。
【００２６】
例えば、風に木の葉が揺れている楊合を考える。図４に示すように、一つ一つのマクロブロック単位の動きベクトルを見ると方向性が一定でない場合が多い。このような場合、（１）式により算出される動きベクトルの総和から、実際の画像の動きよりも多くの動きがあったと認識されてしまい、結果としてオペレータに誤報することになる。
【００２７】
よって、本実施形態では、マクロブロック単位の動きベクトルの総和の絶対値を、監視画像全体の動き量とする。この演算は、次の（２）式になる。
【００２８】
【数２】

【００２９】
この（２）式を用いて監視画像全体の動き量を求めることにより、監視画像内に写し出されている物体に微振動などがあった場合でも、侵入物があったと認識されず、オペレータへの誤報を減らすことができる。
【００３０】
（第３の実施形態）
上記までの第１及び第２の実施形態では、個々の動きベクトルに注目していたが、本実施形態では、互いに隣り合う動きベクトルの相関に注目する。動きベクトルの相関度を計る指標として、互いの動きベクトルの内積値を利用する。
【００３１】
つまり、二つの動きベクトルルＭＶ_xy，ＭＶｘ’ｙ’（ＭＶ_xy，ＭＶｘ’ｙ’は互いに隣り合う（上下左右）ベクトル）の相関度Ｃｍｎ−ｍ’ｎ’は次式で表される。
【００３２】
【数３】

【００３３】
ここで、全てのマクロブロックに対して式（３）を当てはめ、各動きベクトル間の相関度Ｃｍｎ−ｍ’ｎ’をすべて算出する（図５参照）。
【００３４】
さらに、相関度Ｃｍｎ−ｍ’ｎ’が何等かの意昧を持った動きとして捉えるために、次式を設定する。
【００３５】
【数４】

【００３６】
ここでｄは、予め設定する定数で、値が大きければ検出感度は鈍くなり、小さければ検出感度は敏感になる。
【００３７】
次に、（４）式を満たす相関度Ｃｍｎ−ｍ’ｎ’の数を数える関数Ｆ（Ｃｍｎ−ｍ’ｎ’）を用意し、それが次式を満たすとき、何等かの意昧を持った動きが入力された連続画像に検出されたと判断する。
【００３８】
【数５】

【００３９】
ここで、ｅは、予め設定する定数で、値が大きければ検出感度は鈍くなり、小さければ検出感度は敏感になる。
【００４０】
（第４の実施形態）
上記までの各実施形態における動物体を検出する方法は、互いに独立しているため、それらを単独で用いるだけでなく、組み合わせて利用することができる。つまり、次式を用いることで、複数の検出関数を組み合わせた検出をすることができる。
【００４１】
【数６】

【００４２】
ここで、ｄ₄は、値１、２、３をとる。
【００４３】
よって、式（６）により、パラメータｄ₁、ｄ₂、ｄ₃、ｄ₄、ｅ₃を調節することで、検出対象、検出地域に適応した検出レベルをより詳細に設定することができる。
【００４４】
（第５の実施形態）
以上までの各実施形態では、監視カメラが固定されている場合において有効である。カメラの移動操作（パン、チルト）が行われる場合においては、監視画像内の動きにカメラ自身の動きが重畳されるため、侵入物がない場合でも監視画像全体の動き量が大きくなり誤判定を起こしてしまう可能性がある。
【００４５】
例えば、監視カメラがパンしている場合を考える。図６のように各マクロブロックの動きベクトルルＭＶ_xyは、監視カメラの移動（パン、チルト）に起因して発生するベクトル成分Ｖ_cameraと、監視画像固有の動きに起因するベクトル成分ＭＶ_realxyを用いて、次式のように表せる。
【００４６】
【数７】

【００４７】
従って、実際には動物体がないマクロブロック（ＭＶ_realxy＝０）においても動き量が検出され、監視画像全体の動き量が増加し、侵入物が検出されたとしてオペレータに誤報することになる。
【００４８】
よって、本実施形態では、監視画像の各マクロブロックの動きベクトルから監視カメラの移動に起因するベクトル成分（Ｖ_camera）を推定し、これを減算することによって、カメラの移動操作（パン、チルト）の影響をキャンセルし、真の動きベクトルＭＶ_realxyを求める。
【００４９】
ＭＶ_realxyは、上記の（７）式から以下の通り求めることができる。
【００５０】
【数８】

【００５１】
ＭＶ_xyは、コーデックからの出力であり、既知の値である。よって、Ｖ_cameraを推定することでＭＶ_realxyを知ることができる。
【００５２】
監視カメラの移動操作（パン、チルト）が行われている場合、各マクロブロックの動きベクトルは監視カメラの移動に起因するベクトル成分にほぼ等しい値が大多数を占めると考えられる。すなわち、カメラをパン、チルト操作したときはその動きベクトル（ｍｎ×Ｖ _camera ）は真の動きベクトル（ＭＶ _realxy ）よりも十分に大きく（ｍｎ×Ｖ _camera ≫ΣΣＭＶ _realxy ）、各マクロブロックＭＶ _xy を総和したベクトルΣΣＭＶ _xy がカメラのパン、チルト操作に基づく動きベクトルの総和（ｍｎ×Ｖ _camera ）に近いと見なすことができる。
【００５３】
従って、全マクロブロックの動きベクトルの平均値Ｖ_cameraestを求めることで、監視カメラの移動（パン、チルト）に起因するベクトル成分Ｖ_cameraに置き換えることができる。ここで、Ｖ_cameraestは次式により求めることができる。
【００５４】
【数９】

【００５５】
よって、ＭＶ_realxyは、次式で導かれる。
【００５６】
【数１０】

【００５７】
先の実施形態で上げた（１）、（２）、〈５）式のＭＶ_xyをＭＶ_realxyで置き換えることにより、監視カメラの移動操作（パン、チルト）が行われている場合でも、カメラの動きに惑わされること無く、動物体を正確に検出することができる。
【００５８】
（第６の実施形態）
第４及び第５の実施形態では、監視カメラの移動（パン、チルト）に起因するベクトル成分Ｖ_cameraを各マクロブロックの動きベクトルの乎均値にほぼ等しいと仮定している。その為、動物体を表示しているマクロブロックの数が少なくてもその動きベクトルが非常に大きい場合、推定した値が真の値と大きく異なり、うまくキャンセルできない可能性がある。
【００５９】
よって、本実施形態では、図７に示すように、各マクロブロックの動きベクトルのｘ、ｙ成分について２次元ヒストグラムを作成し、最も頻度の高いベクトル値を監視カメラの移動に起因したベクトル成分の推定値（Ｖ_cameraest）として用いる。すなわち、第５の実施形態における全マクロブロックの動きベクトルの平均値（Ｖ _cameraest ）に代えて、最も頻度の高いベクトル値そのものをＶ _camera として採用する。なお、第５の実施形態と同様に、カメラをパン、チルト操作したときはその動きベクトル（ｍｎ×Ｖ _camera ）は真の動きベクトル（ＭＶ _realxy ）よりも十分に大きくなることから、仮に動物体がマクロブロックに存在する場合でもそれに影響されることは少ない。
【００６０】
この推定値を各マクロブロックの動きベクトルＭＶ_xyから減算することにより、第４及び第５の実施形態の場合と同様に（ｌ０）式を使って、真の動きベクトルＭＶ_realxyを求めることができる。
【００６１】
この場合、カメラの動きに惑わされること無く、かつ、動物体の動き量の大きさに関係なく、動物体を正確に検出することができる。
【００６２】
【発明の効果】
以上のとおり、本発明によれば、動画像コーデックが持っている動き補償予測機能を利用して、非通信時にマクロブロック単位の動きベクトルの絶対値の総和等から監視映像による侵入物検出を行い、その検出信号を利用して監視室側に自動発呼で伝送するようにしたため、ポーリングによる監視の非通信時にも侵入者や侵入物があると自動的にオペレータ側で検出できるし、侵入物等の見落としもなくなる。
【図面の簡単な説明】
【図１】本発明の実施形態を示すシステム構成。
【図２】Ｈ.２６１における動画像コーデックの機能ブロック。
【図３】Ｈ.２６１におけるマクロブロック単位の符号化。
【図４】マクロブロック単位の動きベクトル例。
【図５】動きベクトル間の相関度例。
【図６】監視カメラがパンしている場合のマクロブロック単位の動きベクトル例。
【図７】マクロブロック単位の動きベクトルのヒストグラム例。
【図８】遠隔監視システムの例。
【符号の説明】
１₁、１₂、１₃…監視カメラ
２₁、２₂、２₃…動画像コーデック部
１１…動画像コーデック
１２…ネットワークインタフェース部
１３…侵入物検出部
１４…自動発呼制御部[0001]
BACKGROUND OF THE INVENTION
The present invention uses a motion compensation prediction circuit possessed by a moving image codec in a remote monitoring system using a moving image codec to detect an intruder with a monitoring image during non-communication and further uses the detected signal The present invention relates to a remote monitoring system that improves the efficiency of monitoring work by automatically calling a host station.
[0002]
[Prior art]
FIG. 8 shows a remote monitoring system including a moving image codec. Code A, B, provided in each point of the C, etc. perform photographing monitoring site by the camera 1 ₁ to 1 ₃ are position and orientation control, the moving picture codec unit 2 ₁ to 2 ₃ a video signal from each camera The compressed video data is transmitted to the center of a monitoring room or the like through the digital network 3 such as ISDN, and the compressed video data is decoded by a video codec unit (not shown) at the center, 4 is displayed as a monitoring image.
[0003]
When a remote place is monitored with the configuration shown in the figure, a monitor (hereinafter referred to as an operator) periodically connects (polling) to an arbitrary monitoring point and monitors a transmitted image.
[0004]
[Problems to be solved by the invention]
In the conventional system, there are the following inconveniences in detecting an intruder or an intruder in the monitored area.
[0005]
(1) Since it is necessary to perform polling periodically even when no intruder or intruder exists, the communication cost increases.
[0006]
(2) If an abnormality occurs at the monitoring point between polling, the operator cannot find the abnormality or the discovery is delayed.
[0007]
(3) A burden is placed on the operator's work.
[0008]
An object of the present invention is to provide a remote monitoring system that reliably and easily detects an intruder or an intruder without polling.
[0009]
[Means for Solving the Problems]
(First invention)
In a remote monitoring system comprising a surveillance camera for photographing a monitoring point and a moving image codec that has a motion compensation prediction function and encodes and compresses the photographed image of the camera and transmits it to the surveillance room,
Whether or not the degree of correlation between motion vectors adjacent to each other in macroblocks calculated by the motion compensated prediction function of the video codec is obtained for all macroblocks, and whether or not the degree of correlation between the motion vectors is greater than a certain value Intruder detection unit that detects whether there is an intruder or an intruder at the monitoring point,
And an automatic call control unit that automatically transmits an image from the moving image codec to the monitoring room when the intruder detection unit detects an intruder or an intruder.
[0012]
( Second invention)
The intruder detection unit includes, in the “degree of correlation”, “the sum of absolute values of motion vectors in units of macroblocks calculated by the motion compensation prediction function of the video codec” and “motion vector of the entire surveillance image” Intruders or intruders are detected in combination with “the absolute value of the sum of” .
[0013]
( Third invention)
The intruder detection unit estimates a vector component resulting from the movement of the camera from an average value of motion vectors of the macroblock, and subtracts the vector component from the motion vector to detect an intruder or an intruder. It is characterized by.
[0014]
( Fourth invention)
The intruder detection unit estimates a vector component resulting from the movement of the camera from the highest vector value in the two-dimensional histogram of the motion vector of the macroblock, and subtracts the vector component from the motion vector to enter. It is characterized by detecting an object or an intruder.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment)
FIG. 1 is a system configuration diagram showing an embodiment of the present invention. The system shown in the figure shows the configuration of a codec unit at one monitoring point, and includes a moving image codec 11 that implements H.261 according to ITU-T recommendation, a network interface unit 12 for ISDN, an intruder detection unit 13, an automatic generation It is constructed by a call control unit (computer) 14. Each component will be described below.
[0016]
The moving image codec 11 that realizes H.261 has the functional block configuration shown in FIG. 2, the encoding unit is a macro block (16 × 16 pixel block) as shown in FIG. 3, and the luminance is 16 × 16 pixels. The signal has a function of performing motion compensation prediction between the current frame and the previous frame.
[0017]
The motion compensation prediction circuit calculates a motion vector MVxy (see FIGS. 2 and 3) for each macroblock. Normally, the switches 1 and 2 in FIG. 2 are not connected during non-communication, but in this system, a circuit is connected to the switch 1 even during non-communication, and the motion vector MV _xy always relates to the input video from the camera. Can be calculated.
[0018]
The intruder (intruder) detection unit 13 receives a motion vector calculated from a moving image codec according to H.261. Based on this motion vector, a change in the moving image is detected by the following equation (1).
[0019]
[Expression 1]

[0020]
Here, the function Boolean () is a function that is 1 when the expression in () is true and 0 when it is false.
[0021]
That is, by calculating the sum of absolute values of motion vectors, the amount of motion of the entire monitoring image can be known.
[0022]
In the automatic call control unit (computer) 14, if the movement amount of the entire monitoring image detected by the intruder detection unit 13 is larger than the constant d _1, it is designated in advance that an intruder (abnormality) has been detected. The call is automatically made to the operator side (host station), connected to the codec on the slave station side, and the video of the monitoring point where the intruder is detected is transmitted.
[0023]
For example, at this time, the operator can automatically receive a call from a monitoring point, and automatically display the sent monitoring image on the display, thereby notifying the operator of the abnormality.
[0024]
As described above, when the monitoring point senses an abnormality and automatically makes a call, the monitoring work on the operator side is greatly reduced and the occurrence of the abnormality is not overlooked.
[0025]
(Second Embodiment)
In the first embodiment described above, the motion amount of the entire monitoring image is calculated by taking the sum of absolute values in macroblock units (see equation (1)). For this reason, even if there is slight vibration or the like in the object imaged in the monitoring image, it is determined that the movement amount of the monitoring image is large as a result of the calculation by the formula (1), and there is an intruder. There is a possibility of catching as.
[0026]
For example, consider the combination of leaves swaying in the wind. As shown in FIG. 4, when looking at the motion vector of each macroblock unit, the directionality is often not constant. In such a case, it is recognized from the sum of the motion vectors calculated by equation (1) that there is more motion than the actual motion of the image, and as a result, the operator is misreported.
[0027]
Therefore, in this embodiment, the absolute value of the sum of motion vectors in units of macroblocks is used as the motion amount of the entire monitoring image. This calculation is expressed by the following equation (2).
[0028]
[Expression 2]

[0029]
By calculating the amount of movement of the entire monitoring image using this equation (2), even if there is a slight vibration or the like in the object imaged in the monitoring image, it is not recognized that there was an intruder, False alarms can be reduced.
[0030]
(Third embodiment)
In the first and second embodiments described above, attention is paid to individual motion vectors, but in this embodiment, attention is paid to the correlation between motion vectors adjacent to each other. As an index for measuring the degree of correlation of motion vectors, the inner product value of each other's motion vectors is used.
[0031]
That is, the correlation degree Cmn−m′n ′ of two motion vectors MV _xy and MVx′y ′ (MV _xy and MVx′y ′ are adjacent to each other (upper and lower left and right) vectors) is expressed by the following equation.
[0032]
[Equation 3]

[0033]
Here, Equation (3) is applied to all the macroblocks, and all the correlation degrees Cmn-m′n ′ between the motion vectors are calculated (see FIG. 5).
[0034]
Further, the following equation is set so that the correlation degree Cmn−m′n ′ is regarded as a motion with some significance.
[0035]
[Expression 4]

[0036]
Here, d is a constant set in advance. If the value is large, the detection sensitivity becomes dull, and if it is small, the detection sensitivity becomes sensitive.
[0037]
Next, a function F (Cmn−m′n ′) that counts the number of correlation degrees Cmn−m′n ′ satisfying the expression (4) is prepared, and when it satisfies the following expression, it has some meaning. It is determined that the detected motion is detected in the input continuous image.
[0038]
[Equation 5]

[0039]
Here, e is a preset constant. If the value is large, the detection sensitivity becomes dull, and if it is small, the detection sensitivity becomes sensitive.
[0040]
(Fourth embodiment)
Since the methods for detecting moving objects in each of the embodiments described above are independent of each other, they can be used not only independently but also in combination. That is, by using the following equation, detection combining a plurality of detection functions can be performed.
[0041]
[Formula 6]

[0042]
Here, d ₄ takes values 1, 2, and 3.
[0043]
Therefore, by adjusting the parameters d ₁ , d ₂ , d ₃ , d ₄ , and e _{3 according} to the equation (6), the detection level adapted to the detection target and the detection area can be set in more detail.
[0044]
(Fifth embodiment)
Each of the above embodiments is effective when the surveillance camera is fixed. When a camera movement operation (panning or tilting) is performed, the movement of the camera itself is superimposed on the movement in the monitoring image, so that even if there is no intruder, the amount of movement of the entire monitoring image increases, resulting in an erroneous determination. There is a possibility of waking up.
[0045]
For example, consider the case where the surveillance camera is panning. As shown in FIG. 6, the motion vector MV _xy of each macroblock uses a vector component V _camera generated due to the movement (pan, tilt) of the surveillance _camera and a vector component MV _realxy caused by the motion unique to the surveillance image. And can be expressed as:
[0046]
[Expression 7]

[0047]
Accordingly, the amount of motion is detected even in a macroblock (MV _realxy = 0) that actually does not have a moving object, the amount of motion of the entire monitoring image increases, and the operator is falsely _notified that an intruder has been detected.
[0048]
Therefore, in this embodiment, a vector component (V _camera ) resulting from the movement of the surveillance camera is estimated from the motion vector of each macroblock of the surveillance image, and this is subtracted, thereby moving the camera (pan, tilt). And the true motion vector MV _realxy is obtained.
[0049]
MV _realxy can be obtained from the above equation (7) as follows.
[0050]
[Equation 8]

[0051]
MV _xy is an output from the codec and is a known value. Therefore, MV _realxy can be known by estimating V _camera .
[0052]
When the surveillance camera moving operation (pan, tilt) is performed, it is considered that the motion vector of each macroblock occupies the majority almost the same value as the vector component resulting from the surveillance camera movement. That is, when the camera is panned and tilted, the motion vector (mn × V _camera ) is sufficiently larger than the true motion vector (MV _realxy ) (mn × V _camera >> _{ΣΣMV realxy} ), and each macro block MV _xy is It can be considered that the summed vector ΣΣMV _xy is close to the sum of motion vectors (mn × V _camera ) based on the pan and tilt operations of the _camera .
[0053]
Therefore, by _obtaining the average value V _cameraest of the motion vectors of all macroblocks, it can be replaced with the vector component V _camera resulting from the surveillance camera movement (pan, tilt). Here, V _cameraest can be obtained by the following equation.
[0054]
[Equation 9]

[0055]
Therefore, MV _realxy is derived by the following equation.
[0056]
[Expression 10]

[0057]
By replacing MV _xy in the above-described embodiments (1), (2), and <5) with MV _realxy , even when a surveillance camera moving operation (pan, tilt) is performed, The moving object can be accurately detected without being confused by the movement.
[0058]
(Sixth embodiment)
In the fourth and fifth embodiments, it is assumed that the vector component V _camera resulting from the movement (pan, tilt) of the surveillance camera is substantially equal to the average value of the motion vectors of each macroblock. Therefore, if the motion vector is very large even if the number of macroblocks displaying a moving object is small, the estimated value is significantly different from the true value, and there is a possibility that it cannot be canceled successfully.
[0059]
Therefore, in the present embodiment, as shown in FIG. 7, a two-dimensional histogram is created for the x and y components of the motion vector of each macroblock, and the most frequent vector value is determined as the vector component resulting from the movement of the surveillance camera. _Used as an estimated value (V _cameraest ). That is, instead of the average value (V _cameraest ) of motion vectors of all macroblocks in the fifth embodiment , the most frequently used vector value itself is adopted as V _camera . As in the fifth embodiment, when the camera is panned and tilted, the motion vector (mn × V _camera ) is sufficiently larger than the true motion vector (MV _realxy ). Even if is present in a macroblock, it is less affected by it.
[0060]
By subtracting this estimated value from the motion vector MV _xy of each macroblock, the true motion vector MV _realxy can be obtained using equation (10) as in the fourth and fifth embodiments. .
[0061]
In this case, the moving object can be accurately detected without being confused by the movement of the camera and regardless of the amount of movement of the moving object.
[0062]
【The invention's effect】
As described above, according to the present invention, using the motion compensation prediction function of the video codec, intruder detection is performed by monitoring video from the sum of absolute values of motion vectors in units of macroblocks during non-communication. Because the detection signal is used for automatic transmission to the monitoring room, the operator can automatically detect that there is an intruder or intruder even during non-communication of monitoring by polling. There are no oversights.
[Brief description of the drawings]
FIG. 1 is a system configuration showing an embodiment of the present invention.
2 is a functional block diagram of a video codec in H.261.
FIG. 3 shows encoding in units of macroblocks in H.261.
FIG. 4 shows an example of a motion vector in units of macroblocks.
FIG. 5 shows an example of the degree of correlation between motion vectors.
FIG. 6 shows an example of a motion vector in units of macroblocks when the surveillance camera is panning.
FIG. 7 is a histogram example of motion vectors in units of macroblocks.
FIG. 8 shows an example of a remote monitoring system.
[Explanation of symbols]
1 ₁ , 1 ₂ , 1 ₃ ... Surveillance camera 2 ₁ , 2 ₂ , 2 ₃ ... Moving image codec unit 11... Moving image codec 12... Network interface unit 13.

Claims (4)

In a remote monitoring system comprising a surveillance camera for photographing a monitoring point and a moving image codec that has a motion compensation prediction function and encodes and compresses the photographed image of the camera and transmits it to the surveillance room,
Whether or not the degree of correlation between motion vectors adjacent to each other in macroblocks calculated by the motion compensated prediction function of the video codec is obtained for all macroblocks, and whether or not the degree of correlation between the motion vectors is greater than a certain value Intruder detection unit that detects whether there is an intruder or an intruder at the monitoring point,
A remote monitoring system comprising: an automatic call control unit that automatically transmits an image from a moving image codec to the monitoring room when the intruder detection unit detects an intruder or an intruder. . The intruder detection unit includes, in the “degree of correlation”, “the sum of absolute values of motion vectors in units of macroblocks calculated by the motion compensation prediction function of the video codec” and “motion vector of the entire surveillance image” The remote monitoring method according to claim 1, wherein an intruder or an intruder is detected in combination with an “absolute value of the sum of the two” . The intruder detection unit estimates a vector component resulting from the movement of the camera from an average value of motion vectors of the macroblock, and subtracts the vector component from the motion vector to detect an intruder or an intruder. The remote monitoring system according to claim 1 or 2, wherein The intruder detection unit estimates a vector component resulting from the movement of the camera from the highest vector value in the two-dimensional histogram of the motion vector of the macroblock , and subtracts the vector component from the motion vector to enter. 3. The remote monitoring method according to claim 1, wherein an object or an intruder is detected .

Images