JP3917148B2

JP3917148B2 - Moving object extraction device

Info

Publication number: JP3917148B2
Application number: JP2004151283A
Authority: JP
Inventors: 利和藤岡; 基孫中; 健児北村; 武久田中; 和史水澤; 真人森
Original assignee: Tokyo Electric Power Co Inc; Panasonic Corp; Matsushita Electric Industrial Co Ltd
Current assignee: Tokyo Electric Power Co Inc; Panasonic Corp; Panasonic Holdings Corp
Priority date: 2004-05-21
Filing date: 2004-05-21
Publication date: 2007-05-23
Anticipated expiration: 2015-09-11
Also published as: JP2004240997A

Description

本発明は、監視装置等において動画像から動領域を検出して移動物体を抽出し追跡するための移動物体抽出装置に関する。 The present invention relates to a moving object extraction apparatus for detecting a moving area from a moving image and extracting and tracking a moving object in a monitoring apparatus or the like.

動画像から、フレーム間または背景画像との差分を用いて移動物体を抽出する際に特に問題となるのは、一つには移動物体と背景との間の輝度・色度信号差が出ないことによる検出動領域の分断であり、もう一つには移動物体どうしまたは移動物体と静止物体との間で起こる遮蔽である。これら両者の問題について解決を試みた例が知られている（例えば、非特許文献１参照）。この方式の処理の流れについて図面を参照しながら説明する。図８において、ステップ１２０１は時系列画像から動きのあった領域を検出して変化領域を抽出する処理であり、ステップ１２０２は変化領域を画像内および時間方向でラベリングする時空間ラベリング処理であり、ステップ１２０３はラベリングされた各領域を時間方向にグルーピングするグルーピング処理であり、ステップ１２０４はグルーピングされた各移動物体に時間方向に対応付けて同定する移動物体の対応付け処理である。上記方式は、このようにして検出動領域の分断並びに遮蔽の問題を解決することを試みたものである。
『時系列差分画像を用いた複数移動物体の抽出及び追跡』（嶺・八木・谷内田、情報処理学会研究報告、CV-81-7 、51-55) One problem that arises when moving objects are extracted from moving images using differences between frames or background images is that there is no difference in luminance and chromaticity signals between moving objects and the background. In other words, the detection moving area is divided, and the other is shielding that occurs between moving objects or between a moving object and a stationary object. An example of trying to solve both of these problems is known (see, for example, Non-Patent Document 1). The processing flow of this method will be described with reference to the drawings. In FIG. 8, step 1201 is a process for detecting a region that has moved from a time-series image and extracting a change region, and step 1202 is a spatio-temporal labeling process for labeling the change region in the image and in the time direction. Step 1203 is a grouping process for grouping each labeled region in the time direction, and step 1204 is a moving object association process for identifying each grouped moving object in association with the time direction. The above method attempts to solve the problem of the division and shielding of the detection moving area in this way.
"Extraction and tracking of multiple moving objects using time-series difference images" (Tatsumi, Yagi, Taniuchida, Information Processing Society of Japan, CV-81-7, 51-55)

しかしながら、上記従来の技術では、対象物体の影などによる検出動領域の変形を考慮しておらず、検出動領域に影などが含まれた場合に正しい対象物体像を追跡できないという欠点があった。 However, the above conventional technique does not take into account the deformation of the detection moving area due to the shadow of the target object, and has a drawback that a correct target object image cannot be tracked when the detection moving area includes a shadow or the like. .

本発明は、上記従来の問題点に鑑みてなされたもので、その目的は、検出動領域から対象物体の影等の部分を除去して正しい対象物体像を抽出できる移動物体抽出装置を提供することである。 The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a moving object extraction apparatus capable of extracting a correct target object image by removing a portion such as a shadow of the target object from a detection moving region. That is.

上記目的を達成するために本発明の移動物体抽出装置は、映像信号をサンプリングして原画像を出力するサンプリング手段と、前記原画像から動領域を検出して動領域２値画像を出力する動領域検出手段と、前記動領域２値画像の連結成分をラベリングして動領域ラベル画像を出力するラベリング手段と、対象とする形状パターン画像が存在しない動領域に対しては前記動領域ラベル画像を出力する動領域整形手段と、前記動領域整形手段から出力される移動物体画像を用いて各移動物体の形状パターン画像を形成及び更新する形状パターン画像生成手段とを備え、前記動領域整形手段は、前記形状パターン画像を用いて前記動領域ラベル画像を整形して移動物体画像を出力する構成となっている。 In order to achieve the above object, the moving object extracting apparatus of the present invention includes a sampling means for sampling a video signal and outputting an original image, and a moving object for detecting a moving area from the original image and outputting a moving area binary image. An area detection means; a labeling means for labeling the connected components of the moving area binary image to output a moving area label image; and for the moving area where the target shape pattern image does not exist, the moving area label image. A moving area shaping means for outputting, and a shape pattern image generating means for forming and updating a shape pattern image of each moving object using the moving object image outputted from the moving area shaping means, the moving area shaping means The moving region label image is shaped using the shape pattern image and a moving object image is output.

この構成により、動領域整形手段から出力される移動物体画像は、移動物体とともに動く影や床面による鏡面反射像などの出現頻度の低い画像を含む形状パターン画像が加味されて整形されるので、影や床面による鏡面反射像、あるいは移動物体の偶発的な変形などによらず、対象移動物体の形状を安定して抽出することができる。 With this configuration, the moving object image output from the moving area shaping means is shaped by adding a shape pattern image including an image with low appearance frequency such as a shadow moving with the moving object and a specular reflection image by the floor surface, The shape of the target moving object can be stably extracted regardless of the specular reflection image by the shadow or the floor, or the accidental deformation of the moving object.

本発明は、動領域整形手段から出力される移動物体画像を用いて各移動物体の形状パターン画像を形成及び更新する形状パターン画像生成手段を備え、動領域整形手段は、形状パターン画像を用いて動領域ラベル画像を整形して移動物体画像を出力するので、検出動領域から対象物体の影等の部分を除去して正しい対象物体像を抽出できる移動物体抽出装置を提供することができる。 The present invention includes shape pattern image generation means for forming and updating a shape pattern image of each moving object using the moving object image output from the moving area shaping means, and the moving area shaping means uses the shape pattern image. Since the moving area label image is shaped and the moving object image is output, it is possible to provide a moving object extracting apparatus capable of extracting a correct target object image by removing a shadow or the like of the target object from the detected moving area.

以下、本発明の実施の形態について図面を参照しながら説明する。まず、本実施の形態における形状パターン画像について図面を参照しながら説明する。図７において、１３０１、１３０２、１３０３は連続する適当な数のフレームを表しており、１３０４、１３０５、１３０６はそれぞれのフレームの中で検出された動領域を表している。形状パターン画像は、１３０７に示すように同一移動物体に対応する各フレームの動領域を平行移動によって最新フレームでの位置に合わせて重ね合わせ、１３０８に示すように各画素の出現頻度を画像にしたものである。位置合わせのための基準点としては、動領域の重心を用いてもよいし、また動領域の外接矩形の中心もしくは四隅のいずれかを用いてもよいし、また重心のＸ座標もしくはＹ座標と外接矩形の四隅のいずれかのＹ座標もしくはＸ座標とをそれぞれ組み合わせた点を用いてもよいし、また動領域と形状パターン画像とから算出した移動ベクトル分だけ動領域の重心をずらした点を用いてもよい。出現頻度としては、例えば移動物体出現時からの出現回数の平均値を用いてもよいし、また各フレーム毎に重み付けをした平均値を用いてもよい。 Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, the shape pattern image in the present embodiment will be described with reference to the drawings. In FIG. 7, 1301, 1302, and 1303 represent an appropriate number of consecutive frames, and 1304, 1305, and 1306 represent moving areas detected in the respective frames. The shape pattern image is obtained by superimposing the moving areas of the frames corresponding to the same moving object in accordance with the position in the latest frame by parallel movement as indicated by 1307, and making the appearance frequency of each pixel an image as indicated by 1308 Is. As a reference point for alignment, the center of gravity of the moving area may be used, or the center or four corners of the circumscribed rectangle of the moving area may be used, and the X coordinate or Y coordinate of the center of gravity may be used. A point obtained by combining any of the Y coordinates or X coordinates of the four corners of the circumscribed rectangle may be used, or a point obtained by shifting the center of gravity of the moving area by the movement vector calculated from the moving area and the shape pattern image. It may be used. As the appearance frequency, for example, an average value of the number of appearances since the appearance of the moving object may be used, or an average value weighted for each frame may be used.

１３０８において濃淡のハッチングで示したのは出現頻度を模式的に表したものである。１３０８に一例として示したのは人物の例であるが、この場合一般に、相対的な動きの激しい手や足の部分は出現頻度が比較的低く、相対的な動きの少ない頭や胴体など人物像の中心軸付近では出現頻度が比較的高くなる。また、移動物体とともに動く影や床面による鏡面反射像などは現れたり消えたりを繰り返すため、出現頻度は低くなる。このように、形状パターン画像を形成することにより、影や床面による鏡面反射像、あるいは移動物体の偶発的な変形などによらず、対象移動物体の形状を安定して抽出することが可能となる。 In 1308, shaded hatching schematically represents the appearance frequency. An example of a person is shown in 1308 as an example. In this case, however, in general, hand and foot portions with relatively strong movements are relatively infrequent, and human images such as heads and trunks with relatively little movement. The appearance frequency is relatively high near the center axis of. In addition, since the shadow moving with the moving object and the specular reflection image by the floor repeatedly appear and disappear, the appearance frequency is low. In this way, by forming a shape pattern image, it is possible to stably extract the shape of the target moving object, regardless of a specular reflection image by a shadow or a floor, or an accidental deformation of the moving object. Become.

（実施の形態１）
以下、上記形状パターン画像を用いた本発明の第１の実施の形態について図１を参照しながら説明する。図１は本発明の実施の形態１の基本構成を示すブロック図である。図１において、サンプリング手段１０１は入力された映像信号２１１をサンプリングして原画像信号２１２を出力する。 (Embodiment 1)
Hereinafter, a first embodiment of the present invention using the shape pattern image will be described with reference to FIG. FIG. 1 is a block diagram showing a basic configuration of Embodiment 1 of the present invention. In FIG. 1, a sampling unit 101 samples an input video signal 211 and outputs an original image signal 212.

動領域検出手段１０２は上記原画像信号２１２を受けて動領域を検出し、動領域２値画像２１３を出力する。動領域検出方式としては、前フレームまたは数フレーム前と現フレームとの差分をとる方式でもよいし、予め用意された背景画像との差分をとる方式でもよい。またこれらを組み合わせた方式でもよいし、フレーム間差分を数フレーム時間にわたり累積する方式でもよい。ラベリング手段１０４は、上記動領域２値画像２１３を受けてその連結成分をラベリングし、動領域ラベル画像２１５を出力する。 The moving area detecting means 102 receives the original image signal 212 to detect a moving area, and outputs a moving area binary image 213. As a moving region detection method, a method of taking a difference between the previous frame or several frames before and the current frame may be used, or a method of taking a difference from a background image prepared in advance may be used. Moreover, the system which combined these may be sufficient and the system which accumulate | stores the difference between frames over several frame time may be sufficient. The labeling means 104 receives the moving region binary image 213, labels the connected components, and outputs a moving region label image 215.

動領域整形手段１１５は、上記動領域ラベル画像２１５と後述する形状パターン画像生成手段１１６が各動物体毎に出力する形状パターン画像２１８とを受けて、動領域ラベル画像２１５の各動領域に整形を施して移動物体画像２１４を出力する。なお、対応する形状パターン画像の存在しない動領域に対しては、新しく出現した移動物体としてそのまま出力する構成としてもよい。形状パターン画像生成手段１１６は、上記動領域整形手段１１５からの移動物体画像２１４を受けて、各移動物体の形状パターン画像２１８を生成及び記憶、更新し、出力する。 The moving region shaping unit 115 receives the moving region label image 215 and the shape pattern image 218 output from the shape pattern image generating unit 116 described later for each moving object, and shapes the moving region label image 215 into each moving region. And a moving object image 214 is output. It should be noted that a moving region in which no corresponding shape pattern image exists may be output as it is as a newly appearing moving object. The shape pattern image generation unit 116 receives the moving object image 214 from the moving area shaping unit 115, generates, stores, updates, and outputs the shape pattern image 218 of each moving object.

次に、上記形状パターン画像を用いた本実施の形態の具体的構成について図２を参照しながら説明する。図２において、サンプリング手段１０１は入力された映像信号２１１をサンプリングして原画像信号２１２を出力する。動領域検出手段１０２は上記原画像信号２１２を受けて動領域を検出し、動領域２値画像２１３を出力する。動領域検出方式としては、前フレームまたは数フレーム前と現フレームとの差分をとる方式でもよいし、予め用意された背景画像との差分をとる方式でもよい。またこれらを組み合わせた方式でもよいし、フレーム間差分を数フレーム時間にわたり累積する方式でもよい。ラベリング手段１０４は、上記動領域２値画像２１３を受けてその連結成分をラベリングし、動領域ラベル画像２１５を出力する。 Next, a specific configuration of the present embodiment using the shape pattern image will be described with reference to FIG. In FIG. 2, a sampling means 101 samples an input video signal 211 and outputs an original image signal 212. The moving area detecting means 102 receives the original image signal 212 to detect a moving area, and outputs a moving area binary image 213. As a moving region detection method, a method of taking a difference between the previous frame or several frames before and the current frame may be used, or a method of taking a difference from a background image prepared in advance may be used. Moreover, the system which combined these may be sufficient and the system which accumulate | stores the difference between frames over several frame time may be sufficient. The labeling means 104 receives the moving region binary image 213, labels the connected components, and outputs a moving region label image 215.

動領域整形手段１１５は、形状パターン画像検索手段１１７と、形状パターン画像選択手段１１８と、動領域ラベル整形手段１１９とから成り、上記動領域ラベル画像２１５と形状パターン画像生成手段１１６が各動物体毎に出力する形状パターン画像２１８とを受けて、動領域ラベル画像２１５の各動領域に整形を施して移動物体画像２１４を出力する。動領域ラベル整形手段１１９は、形状パターン２値化手段１２０と、ＡＮＤ回路１２１とから構成されている。なお、対応する形状パターン画像２１８の存在しない動領域に対しては、新しく出現した移動物体としてそのまま出力する構成としてもよい。 The moving area shaping means 115 includes a shape pattern image searching means 117, a shape pattern image selecting means 118, and a moving area label shaping means 119. The moving area label image 215 and the shape pattern image generating means 116 are each animal body. In response to the shape pattern image 218 output every time, each moving area of the moving area label image 215 is shaped and a moving object image 214 is output. The moving area label shaping unit 119 includes a shape pattern binarizing unit 120 and an AND circuit 121. It should be noted that a moving region in which the corresponding shape pattern image 218 does not exist may be output as it is as a newly appearing moving object.

本実施の形態では１２１をＡＮＤ回路としているが、これをＯＲ回路とする構成でもよい。図３には、動領域ラベル整形手段１１９を形状パターン２値化手段１２０と、ＯＲ回路１２４とから構成した例を示す。また、ＡＮＤ回路１２１とＯＲ回路１２４とを並列に併せ備え、これらＡＮＤ回路１２１とＯＲ回路１２４のうちいずれかの出力を切り替え出力するようにしてもよい。すなわち、図４に示すように、動領域ラベル整形手段１１９を形状パターン２値化手段１２０と、ＡＮＤ回路１２１と、ＯＲ回路１２４と、セレクタ１２５とから構成し、各形状パターン画像に対応する移動物体が初めて出現してからの時間情報２１９に応じてＡＮＤ回路１２１とＯＲ回路１２４とを切り替える構成としてもよい。 In this embodiment, 121 is an AND circuit, but this may be an OR circuit. FIG. 3 shows an example in which the moving region label shaping unit 119 is configured by a shape pattern binarizing unit 120 and an OR circuit 124. Further, the AND circuit 121 and the OR circuit 124 may be provided in parallel, and one of the outputs of the AND circuit 121 and the OR circuit 124 may be switched and output. That is, as shown in FIG. 4, the moving area label shaping means 119 includes a shape pattern binarization means 120, an AND circuit 121, an OR circuit 124, and a selector 125, and moves corresponding to each shape pattern image. The AND circuit 121 and the OR circuit 124 may be switched according to the time information 219 from when the object first appears.

形状パターン画像検索手段１１７は、上記動領域ラベル画像２１５に含まれる各動領域毎に、形状パターン画像生成手段１１６で生成された形状パターン画像２１８の中から上記動領域に対応するものを検索する。検索方法としては、一例として、動領域の重心における各移動物体の形状パターン画像の画素値、すなわち出現頻度が一定のしきい値を越えるものを選ぶ方式を採ってもよい。また、形状パターン画像の重心が動領域に含まれるものを選ぶ方式を採ってもよい。 The shape pattern image search means 117 searches the shape pattern image 218 generated by the shape pattern image generation means 116 for each moving area included in the moving area label image 215 and searches for one corresponding to the moving area. . As a search method, for example, a method of selecting a pixel value of the shape pattern image of each moving object at the center of gravity of the moving region, that is, a method in which the appearance frequency exceeds a certain threshold value may be adopted. Further, a method of selecting the shape pattern image whose center of gravity is included in the moving region may be adopted.

形状パターン画像選択手段１１８は、上記動領域ラベル画像２１５に含まれる各動領域毎に、上記形状パターン画像検索手段１１７の出力する一つまたは複数の形状パターン画像２１８の中から、上記動領域と同一の移動物体に相当するものを選択して出力する。選択方法としては、一例として動領域の重心と形状パターン画像２１８の重心とが最も近いものを選ぶ方式を採ってもよい。また、動領域と各形状パターン画像２１８とをパターンマッチングして、最も近い形状であると判定されたものを選ぶ方式を採ってもよい。また、一時的に分断されていた動領域が再統合された場合など、複数の形状パターン画像２１８が単一の動領域対応する場合に上記複数の形状パターン画像を重ね合わせたものを新たな形状パターン画像２１８として生成して出力する方式を採ってもよい。また、一時的に重なり合っていた移動物体が再び離れた場合など、複数の動領域が単一の形状パターン画像に対応し、かつその状態が一定フレーム数以上にわたって継続する場合には、上記複数の動領域を各々新たな形状パターン画像２１８として生成して出力する方式としてもよい。 The shape pattern image selecting unit 118 selects, for each moving region included in the moving region label image 215, the moving region from the one or more shape pattern images 218 output by the shape pattern image searching unit 117. Select and output the one corresponding to the same moving object. As an example of the selection method, a method may be used in which the center of gravity of the moving region and the center of gravity of the shape pattern image 218 are selected closest. Further, a method may be adopted in which a moving region and each shape pattern image 218 are subjected to pattern matching and an image determined to have the closest shape is selected. In addition, when a plurality of shape pattern images 218 correspond to a single movement region, such as when a moving region that has been temporarily divided is reintegrated, a new shape is formed by superimposing the plurality of shape pattern images. A method of generating and outputting the pattern image 218 may be adopted. In addition, when a plurality of moving regions correspond to a single shape pattern image and the state continues for a certain number of frames or more, such as when a moving object that has temporarily overlapped again, A moving area may be generated and output as a new shape pattern image 218, respectively.

形状パターン画像２値化手段１２０は、上記形状パターン画像選択手段１１８が出力する画像情報を受けて、これに２値化処理を施し、形状パターン２値画像を出力する。２値化処理のしきい値は撮影環境によって予め決定しておいてもよいし、上記形状パターン画像に相当する移動物体の輝度や監視領域の照度によって変動させる構成としてもよい。 The shape pattern image binarization unit 120 receives the image information output from the shape pattern image selection unit 118, performs binarization processing on the image information, and outputs a shape pattern binary image. The threshold value of the binarization process may be determined in advance depending on the shooting environment, or may be configured to vary depending on the luminance of the moving object corresponding to the shape pattern image and the illuminance of the monitoring area.

動領域ラベル整形手段１１９を構成するＡＮＤ回路１２１は、上記動領域ラベル画像２１５と上記形状パターン画像２値化手段１２０からの形状パターン２値画像とを受けて、動領域ラベル画像２１５の各ラベル毎に対応する形状パターン２値画像とのＡＮＤをとり、移動物体画像２１４を出力する。 The AND circuit 121 constituting the moving area label shaping means 119 receives the moving area label image 215 and the shape pattern binary image from the shape pattern image binarizing means 120 and receives each label of the moving area label image 215. An AND operation is performed on the corresponding shape pattern binary image and a moving object image 214 is output.

形状パターン画像生成手段１１６は、形状パターン画像記憶手段１２２と形状パターン画像更新手段１２３とから成り、上記動領域整形手段１１５からの移動物体画像２１４を受けて、各移動物体の形状パターン画像２１８を生成及び記憶、更新し、出力する。形状パターン画像更新手段１２３は、動領域整形手段１１５からの移動物体画像２１４の各動領域を用いて形状パターン画像を生成するとともに、動領域整形手段１１５が出力する移動物体画像２１４と形状パターン画像記憶手段１２２に記憶された形状パターン画像２１８とを受けて、上記形状パターン画像２１８を更新する。更新の方式としては、一例として次のような方式を採用してもよい。上記動領域整形画像の中の各動領域ごとに、対応する形状パターン画像が存在しない場合は動領域自体を新しく出現した移動物体に相当する形状パターン画像として出力し、そうでない場合には形状パターン画像と動領域とを両者の重心で位置合わせして重み付け加算したものを出力する。 The shape pattern image generation means 116 includes a shape pattern image storage means 122 and a shape pattern image update means 123. The shape pattern image generation means 116 receives the moving object image 214 from the moving area shaping means 115 and obtains the shape pattern image 218 of each moving object. Generate, store, update, and output. The shape pattern image update unit 123 generates a shape pattern image using each moving area of the moving object image 214 from the moving area shaping unit 115, and the moving object image 214 and the shape pattern image output from the moving area shaping unit 115. The shape pattern image 218 is updated in response to the shape pattern image 218 stored in the storage unit 122. As an example of the update method, the following method may be adopted. For each moving area in the moving area shaped image, if there is no corresponding shape pattern image, the moving area itself is output as a shape pattern image corresponding to a newly appearing moving object, otherwise the shape pattern The image and the moving area are positioned at the center of gravity of both and are weighted and added.

形状パターン画像記憶手段１２２は、上記形状パターン画像更新手段１２３から出力される各移動物体ごとの形状パターン画像２１８を記憶し、次フレームで再び動領域整形手段１１５に出力する。なお、長時間にわたる監視などで多数の移動物体が検出される場合などに記憶容量を節約するため、記憶された形状パターン画像２１８が定められた時間以上にわたって更新されなかった場合に、この形状パターン画像２１８に相当する移動物体が監視領域から外に出たものとしてこの形状パターン画像を消去する構成としてもよい。 The shape pattern image storage unit 122 stores the shape pattern image 218 for each moving object output from the shape pattern image update unit 123 and outputs the shape pattern image 218 to the moving area shaping unit 115 again in the next frame. It should be noted that this shape pattern is used when the stored shape pattern image 218 has not been updated over a predetermined time in order to save the storage capacity when a large number of moving objects are detected by monitoring over a long period of time. A configuration may be adopted in which the shape pattern image is erased assuming that the moving object corresponding to the image 218 has moved out of the monitoring area.

このように、本実施の形態１によれば、映像信号をサンプリングして原画像を出力するサンプリング手段１０１と、原画像から動領域を検出して動領域２値画像を出力する動領域検出手段１０２と、動領域２値画像の連結成分をラベリングして動領域ラベル画像を出力するラベリング手段１０４と、対象とする形状パターン画像が存在しない動領域に対しては動領域ラベル画像を出力する動領域整形手段１１５と、動領域整形手段１１５から出力される移動物体画像を用いて各移動物体の形状パターン画像を形成及び更新する形状パターン画像生成手段１１６とを備え、動領域整形手段１１５は、形状パターン画像生成手段１１６からの形状パターン画像を用いて、ラベリング手段１０４からの動領域ラベル画像を整形して移動物体画像を出力するので、動領域整形手段１１５から出力される移動物体画像は、移動物体とともに動く影や床面による鏡面反射像などの出現頻度の低い画像を含む形状パターン画像が加味されて整形されるので、検出動領域に影や床面による鏡面反射像、もしくは持ち物等による偶発的な変形などが生じた場合にも、正しい対象物体像を抽出・追跡でき、検出動領域と移動物体との対応関係を常に記憶・更新しつつ処理を行なうことができる。 As described above, according to the first embodiment, the sampling means 101 that samples the video signal and outputs the original image, and the moving area detection means that detects the moving area from the original image and outputs the moving area binary image. 102, a labeling unit 104 that labels the connected components of the moving area binary image and outputs a moving area label image, and a moving area that outputs a moving area label image for a moving area that does not have a target shape pattern image. The area shaping unit 115 and the shape pattern image generation unit 116 that forms and updates the shape pattern image of each moving object using the moving object image output from the moving area shaping unit 115, and the moving area shaping unit 115 includes: Using the shape pattern image from the shape pattern image generation means 116, the moving area label image from the labeling means 104 is shaped to output a moving object image. Therefore, the moving object image output from the moving region shaping unit 115 is shaped by adding a shape pattern image including an image with low appearance frequency such as a shadow moving with the moving object and a specular reflection image by the floor surface. Even when a specular reflection image due to a shadow or floor surface in the detection moving area, or accidental deformation due to belongings, correct target object images can be extracted and tracked, and the correspondence between the detection moving area and the moving object can be determined. Processing can be performed while always storing and updating.

（実施の形態２）
次に、本発明の第２の実施の形態について図５を用いて説明する。図１に示した実施の形態１では、動領域整形手段１１５からの動領域整形画像を形状パターン画像生成手段１１６に入力し、動領域整形画像を用いて形状パターン画像を生成及び更新する構成としたが、図５に示す実施の形態２では、ラベリング手段１０４からの動領域ラベル画像を形状パターン画像生成手段２１６に入力し、動領域ラベル画像を用いて形状パターン画像２１８を生成及び更新する構成としている。 (Embodiment 2)
Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment shown in FIG. 1, the moving area shaped image from the moving area shaping means 115 is input to the shape pattern image generating means 116, and the shape pattern image is generated and updated using the moving area shaped image. However, in the second embodiment shown in FIG. 5, the moving region label image from the labeling unit 104 is input to the shape pattern image generating unit 216, and the shape pattern image 218 is generated and updated using the moving region label image. It is said.

図６はラベリング手段１０４からの動領域ラベル画像２１５を形状パターン画像生成手段２１６に入力し、動領域ラベル画像２１５を用いて形状パターン画像２１８を生成および更新する構成の詳細を示している。このように、図６に示した構成は、図２に示した構成に対して形状パターン画像生成手段２１６を構成する形状パターン画像記憶手段２２２および形状パターン画像更新手段２２３への画像信号の入力の仕方を変更した点が異なるのみである。 FIG. 6 shows details of a configuration in which the moving region label image 215 from the labeling unit 104 is input to the shape pattern image generating unit 216, and the shape pattern image 218 is generated and updated using the moving region label image 215. In this way, the configuration shown in FIG. 6 is the input of image signals to the shape pattern image storage means 222 and the shape pattern image update means 223 constituting the shape pattern image generation means 216 with respect to the configuration shown in FIG. The only difference is how it was changed.

形状パターン画像生成手段２１６は、形状パターン画像記憶手段２２２と形状パターン画像更新手段２２３とから成り、ラベリング手段１０４からの動領域ラベル画像２１５を受けて、各移動物体の形状パターン画像２１８を生成及び記憶、更新し、出力する。形状パターン画像更新手段２２３は、ラベリング手段１０４からの動領域ラベル画像２１５を用いて形状パターン画像２１８を生成するとともに、ラベリング手段１０４からの動領域ラベル画像２１５と形状パターン画像記憶手段２２２に記憶された形状パターン画像２１８とを受けて、上記形状パターン画像２１８を更新する。更新の方式としては、一例として次のような方式を採用してもよい。上記動領域整形画像の中の各動領域ごとに、対応する形状パターン画像が存在しない場合は動領域自体を新しく出現した移動物体に相当する形状パターン画像として出力し、そうでない場合には形状パターン画像と動領域とを両者の重心で位置合わせして重み付け加算したものを出力する。 The shape pattern image generation unit 216 includes a shape pattern image storage unit 222 and a shape pattern image update unit 223. The shape pattern image generation unit 216 receives the moving area label image 215 from the labeling unit 104 and generates and generates a shape pattern image 218 of each moving object. Store, update, and output. The shape pattern image update unit 223 generates the shape pattern image 218 using the moving region label image 215 from the labeling unit 104 and is stored in the moving region label image 215 and the shape pattern image storage unit 222 from the labeling unit 104. The shape pattern image 218 is updated in response to the received shape pattern image 218. As an example of the update method, the following method may be adopted. For each moving area in the moving area shaped image, if there is no corresponding shape pattern image, the moving area itself is output as a shape pattern image corresponding to a newly appearing moving object, otherwise the shape pattern The image and the moving area are positioned at the center of gravity of both and are weighted and added.

形状パターン画像記憶手段２２２は、上記形状パターン画像更新手段２２３から出力される各移動物体ごとの形状パターン画像２１８を記憶し、次フレームで動領域整形手段１１５に出力する。なお、長時間にわたる監視などで多数の移動物体が検出される場合などに記憶容量を節約するため、記憶された形状パターン画像２１８が定められた時間以上にわたって更新されなかった場合に、この形状パターン画像２１８に相当する移動物体が監視領域から外に出たものとしてこの形状パターン画像を消去する構成としてもよい。 The shape pattern image storage unit 222 stores the shape pattern image 218 for each moving object output from the shape pattern image update unit 223 and outputs it to the moving region shaping unit 115 in the next frame. It should be noted that this shape pattern is used when the stored shape pattern image 218 has not been updated over a predetermined time in order to save the storage capacity when a large number of moving objects are detected by monitoring over a long period of time. The shape pattern image may be erased assuming that the moving object corresponding to the image 218 has moved out of the monitoring area.

動領域整形手段１１５は、形状パターン画像検索手段１１７と、形状パターン画像選択手段１１８と、動領域ラベル整形手段１１９とから成り、上記動領域ラベル画像２１５と形状パターン画像生成手段２１６が各動物体毎に出力する形状パターン画像２１８とを受けて、動領域ラベル画像２１５の各動領域に整形を施して移動物体画像２１４を出力する。動領域ラベル整形手段１１９は、形状パターン２値化手段１２０と、ＡＮＤ回路１２１とから構成されている。なお、対応する形状パターン画像２１８の存在しない動領域に対しては、新しく出現した移動物体としてそのまま出力する構成としてもよい。 The moving area shaping means 115 includes a shape pattern image searching means 117, a shape pattern image selecting means 118, and a moving area label shaping means 119. The moving area label image 215 and the shape pattern image generating means 216 are provided for each moving object. In response to the shape pattern image 218 output every time, each moving area of the moving area label image 215 is shaped and a moving object image 214 is output. The moving area label shaping unit 119 includes a shape pattern binarizing unit 120 and an AND circuit 121. It should be noted that a moving region in which the corresponding shape pattern image 218 does not exist may be output as it is as a newly appearing moving object.

本実施の形態２においても、実施の形態１における動領域整形手段１１５に対してなされた図３および図４に示す変更が、図６の構成に対してもそのまま適用することができる。その他、実施の形態１に用いられた種々の技術が本実施の形態２にも採用することができる。 Also in the second embodiment, the changes shown in FIGS. 3 and 4 made to the moving region shaping means 115 in the first embodiment can be applied to the configuration of FIG. 6 as it is. In addition, various techniques used in the first embodiment can also be adopted in the second embodiment.

このように、本実施の形態２によれば、映像信号をサンプリングして原画像を出力するサンプリング手段１０１と、原画像から動領域を検出して動領域２値画像を出力する動領域検出手段１０２と、動領域２値画像の連結成分をラベリングして動領域ラベル画像を出力するラベリング手段１０４と、動領域ラベル画像を用いて各移動物体の形状パターン画像を生成及び更新する形状パターン画像生成手段２１６と、形状パターン画像生成手段２１６からの形状パターン画像を用いて、ラベリング手段１０４からの動領域ラベル画像を整形して移動物体画像を出力する動領域整形手段１１５とを備えているので、動領域整形手段１１５から出力される移動物体画像は、移動物体とともに動く影や床面による鏡面反射像などの出現頻度の低い画像を含む形状パターン画像が加味されて整形されるので、検出動領域に影や床面による鏡面反射像、もしくは持ち物等による偶発的な変形などが生じた場合にも、正しい対象物体像を抽出・追跡でき、検出動領域と移動物体との対応関係を常に記憶・更新しつつ処理を行なうことができる。 Thus, according to the second embodiment, the sampling means 101 that samples the video signal and outputs the original image, and the moving area detection means that detects the moving area from the original image and outputs the moving area binary image. 102, labeling means 104 for labeling the connected components of the moving region binary image and outputting a moving region label image, and shape pattern image generation for generating and updating the shape pattern image of each moving object using the moving region label image Since it includes the means 216 and the moving area shaping means 115 for shaping the moving area label image from the labeling means 104 using the shape pattern image from the shape pattern image generating means 216 and outputting the moving object image. The moving object image output from the moving area shaping unit 115 is an image with a low appearance frequency such as a shadow moving with the moving object or a specular reflection image by the floor surface. Since the shape pattern image is included and shaped, the correct target object image can be extracted and tracked even if the detection motion area is subject to shadows, specular reflections from the floor, or accidental deformations due to belongings, etc. It is possible to perform processing while always storing and updating the correspondence between the detection moving area and the moving object.

以上説明したように、本発明に係る移動物体抽出装置は、検出動領域から対象物体の影等の部分を除去して正しい対象物体像を抽出できるという効果を有し、監視装置等において動画像から動領域を検出して移動物体を抽出し追跡するための移動物体抽出装置等に有用である。 As described above, the moving object extraction device according to the present invention has an effect that a correct target object image can be extracted by removing a portion such as a shadow of the target object from the detection moving region. This is useful for a moving object extraction device or the like for detecting a moving area from an object and extracting and tracking a moving object.

本発明の実施の形態１における移動物体抽出装置の基本構成を示すブロック図1 is a block diagram showing a basic configuration of a moving object extraction device in Embodiment 1 of the present invention. 本発明の実施の形態１における動領域整形手段および形状パターン画像生成手段をより詳細に示したブロック図The block diagram which showed the moving area shaping means and the shape pattern image generation means in Embodiment 1 of this invention in detail 本発明の実施の形態１における動領域整形手段の変更例を示すブロック図The block diagram which shows the example of a change of the dynamic region shaping means in Embodiment 1 of this invention 本発明の実施の形態１における動領域整形手段のさらに他の変更例を示すブロック図The block diagram which shows the further another example of a change of the moving area shaping means in Embodiment 1 of this invention. 本発明の実施の形態２における移動物体抽出装置の基本構成を示すブロック図The block diagram which shows the basic composition of the moving object extraction apparatus in Embodiment 2 of this invention. 本発明の実施の形態２における動領域整形手段および形状パターン画像生成手段をより詳細に示したブロック図The block diagram which showed the movement area shaping means and the shape pattern image generation means in Embodiment 2 of this invention in detail 本発明の実施の形態１、２における形状パターン画像の説明図Explanatory drawing of the shape pattern image in Embodiment 1, 2 of this invention 従来の移動物体抽出装置の一例の処理の流れを示すフロー図Flow diagram showing the flow of an example of a conventional moving object extraction device

Explanation of symbols

１０１サンプリング手段
１０２動領域検出手段
１０３動領域対応付け手段
１０４ラベリング手段
１１５動領域整形手段
１１６、２１６形状パターン画像生成手段
１１７形状パターン画像検索手段
１１８形状パターン画像選択手段
１１９動領域ラベル整形手段
１２０形状パターン画像２値化手段
１２１ＡＮＤ回路
１２２、２２２形状パターン画像記憶手段
１２３、２２３形状パターン画像更新手段
１２４ＯＲ回路
１２５セレクタ
１３０１、１３０２、１３０３複数の連続したフレーム画像の一例
１３０４フレーム１３０１で検出された動領域
１３０５フレーム１３０２で検出された動領域
１３０６フレーム１３０３で検出された動領域
1 ３０７一連のフレーム１３０１〜１３０３で検出された動領域をフレーム１３０３の動領域１３０６に重ね合わせて得られる画像
１３０８フレーム１３０１〜１３０３で検出された動領域から形成される形状パターン画像 DESCRIPTION OF SYMBOLS 101 Sampling means 102 Moving area detection means 103 Moving area correlation means 104 Labeling means 115 Moving area shaping means 116, 216 Shape pattern image generation means 117 Shape pattern image search means 118 Shape pattern image selection means 119 Moving area label shaping means 120 Shape Pattern image binarization means 121 AND circuit 122, 222 Shape pattern image storage means 123, 223 Shape pattern image update means 124 OR circuit 125 Selector 1301, 1302, 1303 An example of a plurality of consecutive frame images 1304 Detected in frame 1301 Moving region 1305 Moving region detected in frame 1302 1306 Moving region detected in frame 1303
1 307 Image obtained by superimposing moving areas detected in a series of frames 1301 to 1303 on moving area 1306 of frame 1303 1308 Shape pattern image formed from moving areas detected in frames 1301 to 1303

Claims

Sampling means for sampling a video signal and outputting an original image, moving area detecting means for detecting a moving area from the original image and outputting a moving area binary image, and labeling connected components of the moving area binary image Labeling means for outputting a moving area label image and outputting the moving area label image for a moving area in which no target shape pattern image exists, and if there is a target shape pattern image, the shape A moving area shaping means for shaping the moving area label image using a pattern image and outputting a moving object image ;
When the moving object image output from the moving region shaping means is used , the label corresponding to the moving object among the labels of the moving object image is superimposed according to the position in the latest frame of the moving object, A shape pattern image of each moving object is formed with the frequency of label appearance at each pixel in the frame as a pixel value, and the shape pattern image stored for each moving object is moved from the moving area shaping means. A shape pattern image generating means for performing an update process using each moving region of the object image ,
The shape pattern image generating means outputs the moving area itself as a shape pattern image when there is no shape pattern image corresponding to each moving area of the moving object image from the moving area shaping means. A moving object extraction apparatus characterized by storing and updating a result of weighted addition of an image and a moving region after positioning and updating as a shape pattern image of each moving object .

Sampling means for sampling a video signal and outputting an original image, moving area detecting means for detecting a moving area from the original image and outputting a moving area binary image, and labeling connected components of the moving area binary image Labeling means for outputting a moving region label image,
Using the moving region label image, the label corresponding to the moving object among the labels of the sequentially input moving region label images is overlapped in accordance with the position of the moving object in the latest frame. A shape pattern image generating means for generating and updating a shape pattern image of each moving object having a pixel value as an appearance frequency of a label in each pixel at
A moving area shaping means for shaping the moving area label image using the shape pattern image and outputting a moving object image ;
The shape pattern image generation means outputs the moving area itself as a shape pattern image when there is no shape pattern image corresponding to each moving area of the moving area label image from the labeling means. Object extracting apparatus for storing and updating a result of weighted addition by aligning a moving area and a moving area as a shape pattern image of each moving object .

The shape pattern image generation means generates the shape pattern image using a shape pattern image storage means for storing a shape pattern image for each moving object and each moving area of the moving object image from the moving area shaping means. And shape pattern image updating means for performing update processing on the shape pattern image stored in the shape pattern image storage means using each moving area of the moving object image from the moving area shaping means. The moving object extraction device according to claim 1 .

The shape pattern image generation means generates the shape pattern image using a shape pattern image storage means for storing a shape pattern image for each moving object and each moving area label of the moving area label image from the labeling means. And shape pattern image updating means for performing update processing on the shape pattern image stored in the shape pattern image storage means using each moving area label of the moving area label image from the labeling means. The moving object extraction device according to claim 2 .

It said shape pattern image generating means according to any one of claims 1 to 4, characterized in that to erase the shape pattern image when the stored shape pattern image has not been updated over time determined Moving object extraction device.

As a moving region alignment point when aligning the shape pattern image and the moving region, a movement vector of the corresponding moving object is obtained from the moving region and the corresponding shape pattern image, and the barycentric point of the shape pattern image is obtained. moving the object extracting apparatus according to any one of claims 1 to 5, characterized in that a point shifted by the moving vector.

As a shape pattern image alignment point when aligning the shape pattern image and the moving area, a movement vector of the corresponding moving object is obtained from the moving area corresponding to the shape pattern image, and the center of gravity point of the moving area is moved. moving the object extracting apparatus according to any one of claims 1 to 6, characterized in that a point returned by vector.

The moving area shaping means receives one or more shape pattern images from the shape pattern image generating means and a moving area label image from the labeling means, and the moving area label image is selected from the shape pattern images. A shape pattern image search means for searching whether or not there is a corresponding to each moving area and outputting the result, and a shape pattern image selection for selecting and outputting the corresponding shape pattern image if it exists And a moving area label shaping means for shaping a moving area where a corresponding shape pattern image exists among the moving areas of the moving area label image using a shape pattern image and outputting a moving object image. moving the object extracting apparatus according to any one of claims 1 to 7, characterized in that the.

9. The moving object according to claim 8, wherein the moving area label shaping means outputs a logical product or logical sum of a shape binary pattern obtained by binarizing the shape pattern image and the moving area. Extraction device.

The time from when the moving object corresponding to the shape pattern image first appeared is the logical product or logical sum of the shape binary pattern obtained by binarizing the shape pattern image and the moving region by the moving region label shaping means. The moving object extraction apparatus according to claim 8 , wherein switching is performed according to

The moving region label shaping means, according to claim 8 to 10, characterized in that the binarization process by a threshold that varies in accordance with the illuminance of the luminance and monitoring area of the moving object corresponding to the input shape pattern The moving object extraction device according to any one of the above.

The shape pattern image search means outputs an image in which the pixel value of the shape pattern image at the moving region representative point is equal to or greater than a certain threshold among the shape pattern images stored in the shape pattern image generation means. The moving object extracting device according to claim 8 , wherein the moving object extracting device is a moving object extracting device.

The shape pattern image search means outputs an image in which shape pattern image representative points are included in a moving region among the shape pattern images stored in the shape pattern image generation means. The moving object extraction device according to any one of claims 8 to 11 .

The shape pattern image search means has a pixel value of a shape pattern image at a moving region representative point that is greater than or equal to a certain threshold value among the shape pattern images stored in the shape pattern image generation means, or The moving object extraction device according to claim 8 , wherein an image indicating whether the shape pattern image representative point is included in the moving region is output.

The shape pattern image search means has a pixel value of a shape pattern image at a moving region representative point that is greater than or equal to a certain threshold among the shape pattern images stored in the shape pattern image generation means, and the shape 12. The moving object extraction apparatus according to claim 8 , wherein an image in which the pattern image representative point is included in the moving area is output.

The shape pattern image search means has a threshold value at which a pixel value of the shape pattern image is constant in a predetermined number or more of pixels included in the moving region among the shape pattern images stored in the shape pattern image generation means. 12. The moving object extraction device according to claim 8 , wherein the image is output as described above.

The shape pattern image search means performs pattern matching between each shape pattern image stored in the shape pattern image generation means and a moving region, and outputs a shape pattern image determined to be the closest pattern. The moving object extraction device according to claim 8 , wherein the moving object extraction device is a moving object extraction device.

The shape pattern image selecting means selects and outputs the image of the shape pattern image representative point closest to the moving area representative point on the image among the shape pattern images output by the shape pattern image search means. The moving object extraction apparatus according to claim 8 , wherein the moving object extraction apparatus is a moving object extraction apparatus.

The shape pattern image selecting unit selects and outputs an image having the largest pixel value of the shape pattern image at the moving region representative point from among the shape pattern images output by the shape pattern image searching unit. Item 18. The moving object extraction device according to any one of Items 8 to 17 .