JPH06274625A - Moving object tracking method for monitor image - Google Patents

Abstract

PURPOSE:To enable efficient image monitor by tracking a moving object while deciding this moving object as a target object when a change area extracted from a differential binary is in contact with a previously set appearance side area. CONSTITUTION:The differential image is calculated by performing inter-picture element difference arithmetic between an image stored in an image memory 12 and an image fetched before the stored image, and a differential binarizing circuit 13 converts that image to a differential binary image for which the changed picture element is expressed by '1'. After a CPU 15 calculates the change area and judges it as the moving object, it is registered on a person dictionary 16 as a gradation pattern. After the next frame, a predictive window is set for the moving direction of the moving object by referring to a moving picture element table 17 corresponding to the size of the object, matching processing is performed by using a mask pattern previously registered on a mask pattern memory 18, and the moving object is tracked while deciding the object is moved into an area with the highest degree of similarity. Thus, the object inside the monitor picture can be accurately tracked and the performance of deciding whether it is a moving object or not can be improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a monitoring object moving object tracking method for tracking a moving object in a monitoring image in an image monitoring apparatus for monitoring a moving object in a monitoring area.

[0002]

2. Description of the Related Art An example of this type of image monitoring apparatus is shown in FIGS. 17 and 18, for example. In the figure, an ITV camera 1 as an image pickup means picks up an image in a monitoring area 2 and converts it into an electric signal. The image picked up by the ITV camera 1 is sent to the processing unit 4 and the video tape recorder (hereinafter simply referred to as VTR) 5 as an image recording unit by the transmission line 3. VT
R5 records the monitoring image captured by the ITV camera 1 continuously or when a moving object (hereinafter, also referred to as an intruding object) is detected.

On the other hand, the image signal sent to the processing device 4 is
The sampling pulse having a predetermined cycle output from the sampling pulse generation circuit 10 is converted into a digital signal by the A / D converter 11, and is stored in the image memory 12 as image data at time t = ti, for example.

Then, in the differential binarization circuit 13, the image data at the time t = ti is extracted before the image memory 12 in order to extract a change area, as will be described later.
The difference image is obtained by performing the inter-pixel difference calculation with the image data at time t = ti-1 that was captured in the difference binary image in which the changed pixel is represented by "1". And is stored in the differential binarized image memory 14.

A CPU (Central Processing Unit) 15 has a difference 2 in a difference binarized image memory 14.
When the changed area is analyzed using the binarized image and it is determined that the changed area is a moving object (intruding object), the alarm device 6 is sounded, or the supervisor moves the moving object on the screen of the display device 7. Is checked, and if the VTR 5 is not operating, it is operated to record a monitoring image. If it is determined that the object is not a moving object, the image is continuously captured and the above processing is repeated.

By the way, as an inter-pixel calculation for extracting a change area, in general, a difference from a background image when a moving object does not exist, or a time t before a predetermined time Δt.
Either one of the time-series differences from the image of = ti-1 is used. With the difference from the background image, it can be detected even when the moving object is stopped, but when the environment such as the brightness is changing, such as outdoors, the moving object exists unless the background image is updated. There is a feature that a change area is generated even if it is not.

In the case of the time series difference, since the difference is generally made at a relatively short time interval, the followability to the environmental change is excellent, but it cannot be detected when the moving object stops, and further, Even if the moving object moves, the difference between the two screens is characterized in that both the disappearing portion and the occurring portion of the moving object are detected.

In order to eliminate the latter drawback of the time-series difference, in addition to the two screens of t = ti and t = ti-1,
= Ti + 1 consecutive three screens are used to perform the difference calculation on the front two screens and the rear two screens respectively, and by the logical product of the two difference images obtained, the moving object in the image t = ti In some cases, the method of extracting is taken.

In addition, when the CPU 15 analyzes a change area from a binary difference image, the value is "as a basic method of image processing in order to obtain a predetermined grouped area.
A serial number is given (labeling) to the closed area that is 1 ", and the area and distance of each partial area are calculated to extract a grouped area.
In many cases, the area such as 1 "is calculated and analyzed.
Performing this operation over the entire image area is very time consuming.

Therefore, for example, as disclosed in Japanese Patent Laid-Open No. 62-147891, a method has been proposed in which a changed area is roughly divided from a difference binary image. This is as shown in FIG. 19 in the difference binarized image.
The projection histogram is obtained in the X and Y axis directions, and in the obtained histogram, a segmented area whose height of the predetermined value Xth or Yth or more is equal to or more than the predetermined width Xw or Yw is obtained, and the inside is analyzed. Is a technique.

That is, the divided areas A11 to A shown in FIG.
Only 22 is examined to analyze whether the change area is a moving object or not. At the time of analyzing the change area, as described above, the analysis with labeling can be performed, and as in the case of the divided areas A11 and A22 in FIG. 19, the divided areas are often circumscribed rectangles of the moving object. When the divided area has a predetermined size, it may be analyzed by a method such as directly inspecting its shape or obtaining the area of a portion whose value is "1" without performing processing such as labeling. .

The method of image difference and the method of analyzing a moving object have been described above. However, since the captured image contains a noise component, it is common to perform spatial filtering or logical filtering processing.

[0013]

In the conventional image monitoring apparatus, after the change area is determined to be a moving object, an alarm is sounded, or a video recording is performed on the VTR, and the like. It is determined whether or not the moving object moves from where and how it is a suspicious person. Therefore, compared with the initial image monitoring device in which the monitor always looks at the screen of the display device to find the moving object, the work is reduced, but it takes a considerable time for the monitor to make a judgment in the post-processing. Spend labor.

Therefore, not only the detection of the moving object but also the improvement of tracking the moving object and sounding the alarm only to a truly suspicious person can be expected to further reduce the work.

Therefore, recently, a method of tracking a moving object between time-series frames has been considered for the purpose of improving the performance of determining whether or not the object is a moving object. As an example of the moving object tracking method, there is a method using a grayscale pattern. This moving object tracking method obtains a moving object from the difference image,
Further, the grayscale pattern of the original image of the moving object is registered as a person dictionary, and the matching process between the original image and the grayscale pattern dictionary is performed within the range of the prediction window set in advance in the next frame to determine the similarity. Tracking is performed by determining a high area as the moving object. In addition, the grayscale pattern dictionary is updated every frame so as to cope with the change in shape of the moving object and the change in size due to perspective.

However, when the grayscale pattern dictionary of a moving object is compared with the original image, if the target is a human, for example,
The shape of the moving object changes in the next frame in which the dictionary is created because the user is moving by moving his hand or foot. for that reason,
At the time of matching, it is necessary to consider that the grayscale pattern dictionary of the previous frame is slightly modified.

Further, the object size of the captured image changes depending on the difference in perspective where a person exists. Even if the same person is moving at the same speed, the size of the extracted object and the moving pixels change, and there is a problem that matching time is required if the object size is large and the moving pixels are large.

Further, since the moving pixel varies depending on the specifications of the surveillance camera, the height and the angle at which it is installed, when trying to accurately obtain the moving pixel, all the above conditions are input according to the installation location, and the moving pixel is calculated. Need to ask by.

Also, updating the grayscale pattern dictionary for each frame is effective in dealing with changes in the shape of moving objects and changes in size due to perspective, but there are multiple moving objects within the monitoring area. Then, even if moving objects seem to overlap, if you update the dictionary,
Since the information of the person on the upper side is included in the dictionary on the lower side, the grayscale information of the original tracking object disappears, and another moving object may be tracked.

Further, when images are captured at a predetermined capture interval, and moving objects are extracted and tracked in each captured image, if the capture interval is lengthened in order to improve the extraction of moving objects, the moving distance is reduced. There is a contradiction that the extraction performance of the moving object is deteriorated when the capturing interval is shortened in order to increase the matching time and lengthen the matching time.

On the other hand, when the monitoring area is outdoors, all the extracted change areas are not moving objects of interest, but shaking of trees due to the wind, smoke, small animals, or changes in the shade information due to changes in the weather. In some cases, it is necessary to determine whether the change area is the target moving object.

Further, when tracking an intruding object or the like, the place where the intruding object appears can be predicted to some extent, such as the frame of the captured image, the edge or the entrance of the building, so that all area changes in the monitoring area are detected. If so, it means that he is looking at a useless area where an intruding object should not appear.

Therefore, according to the present invention, it is possible to accurately track a moving object in a surveillance image, improve the performance of determining whether or not the object is a moving object, and prevent erroneous detection of the moving object to detect the moving object. It is an object of the present invention to provide a method of tracking a moving object in a monitoring image, which enables time reduction and efficient image monitoring.

[0024]

According to the method of tracking a moving object in a surveillance image of the present invention, an image in the surveillance area is picked up by an image pickup means and inputted, and the inputted image in the surveillance area is set at predetermined time intervals. In the image monitoring apparatus for monitoring a moving object in the monitoring area, a differential image of the digitized image is obtained by continuously capturing and digitizing the digitized image and calculating a plurality of the digitized images. Then, the calculated difference image is binarized to generate a difference binarized image,
A change area is extracted from the generated difference binarized image, and when the extracted change area is in contact with a preset appearance prediction area in which a moving object may appear, the moving object is regarded as an object. It is characterized in that the object is determined to be an object to be traced and is tracked.

Further, in the method of tracking a moving object of a surveillance image according to the present invention, the image in the surveillance area is picked up by the image pickup means and inputted,
The input image within the monitoring area is continuously captured at a predetermined time interval and digitized, and a plurality of digitized images are calculated to monitor a moving object within the monitoring area. In the image monitoring device, a differential image of the digitized image is calculated, the calculated differential image is binarized to generate a differential binarized image, and a change area is extracted from the generated differential binarized image. Change area extracting means, change area extracted by the change area extracting means, and a light and shade pattern extracting means for extracting a light and shade pattern of the change area from the original image digitized by the image pickup means, and the light and shade pattern Mask pattern generating means for generating a pattern for masking a predetermined range of the grayscale pattern extracted by the extracting means,
A reference pattern generating means for obtaining a reference light and shade pattern from the output of the mask pattern generating means and the output of the light and shade pattern extracting means, a reference pattern obtained from the reference pattern generating means, and the digitized original image By providing a similarity calculation unit that obtains a region having a high similarity with the reference pattern in the original image by comparing, the image in the monitoring region is taken in every predetermined time, and the change region is extracted by the change region extraction unit. At the time of extraction, a reference pattern corresponding to the changed area is obtained by the reference pattern generating means, and thereafter, each time an image is captured, the similarity calculation means uses the reference pattern to determine an area having a high degree of similarity in the captured image. The position of the moving object after a predetermined time has been obtained is determined, and the position of the moving object is further determined using the high similarity region and the mask pattern. By updating the reference pattern in a quasi-pattern generating means and to track moving objects within the monitored image.

Further, in the method of tracking a moving object of a surveillance image of the present invention, an image in the surveillance area is picked up by an image pickup means and inputted,
The input image within the monitoring area is continuously captured at a predetermined time interval and digitized, and a plurality of digitized images are calculated to monitor a moving object within the monitoring area. In the image monitoring device, a difference image of the digitized images captured at a predetermined time interval t1 is calculated, and the calculated difference image is binarized to obtain a difference 2
A changed area extracting means for generating a binarized image and extracting a changed area from the generated difference binarized image, a changed area extracted by the changed area extracting means, and input from the image pickup means and digitized. A light and shade pattern extracting means for extracting a light and shade pattern of the changed region from the original image, and a mask pattern generating means for generating a pattern for masking a predetermined range of the light and shade patterns extracted by the light and shade pattern extracting means. A reference pattern generating means for obtaining a reference light and shade pattern from the output of the mask pattern generating means and the output of the light and shade pattern extracting means, a reference pattern obtained from the reference pattern generating means, and a predetermined time shorter than the time interval t1. By comparing the original digitized image captured at time interval t2 of A quasi-pattern and a similarity calculation means for obtaining an area having a high degree of similarity, the image in the monitoring area is taken in every predetermined time, and when the change area is extracted by the change area extraction means, the reference pattern generation means is used. The reference pattern corresponding to the change area is obtained, and thereafter, every time an image is captured, the similarity calculation means uses the reference pattern to obtain a highly similar area in the captured image, and the moving object after a predetermined time elapses. It is characterized in that the moving object is tracked in the monitoring image by determining the position and further updating the reference pattern by the reference pattern generating means using the area having high similarity and the mask pattern.

Further, in the method of tracking a moving object of a surveillance image according to the present invention, an image in the surveillance area is picked up by an image pickup means and inputted, and the inputted image in the surveillance area is continuously taken at predetermined time intervals. In the image monitoring device that monitors a moving object in the monitoring area by calculating a digitized image by calculating the difference image of the digitized image, A difference area extracting unit that binarizes the calculated difference image to generate a difference binarized image, and extracts a change area from the generated difference binarized image; and a change area extracted by the change area extracting unit. Of the grayscale patterns extracted by the grayscale pattern extraction means, the grayscale pattern extraction means for extracting the grayscale pattern of the change region portion from the original image input from the image pickup means and digitized. Mask pattern generating means for generating a pattern for masking a predetermined range, and reference pattern generating means for obtaining a reference light and shade pattern from the output of the mask pattern generating means and the output of the light and shade pattern extracting means. And a similarity calculation means for obtaining a region having a high similarity with the reference pattern in the original image by comparing the reference pattern obtained from the reference pattern generating means with the digitized original image, and the similarity calculation means. After obtaining a region having a high degree of similarity by the degree calculation means, the change in the shading pattern inside or outside the mask pattern is examined, and it is determined whether the moving object is a target object according to the magnitude of the change. A change area is extracted by the change area extraction means by capturing an image in the monitoring area at predetermined time intervals. , A reference pattern corresponding to the changed area is obtained by the reference pattern generating means, and thereafter, every time an image is captured, the similarity calculation means uses the reference pattern to obtain a highly similar area in the captured image, and the predetermined area is determined. The position of the moving object after a lapse of time is determined, and the moving object is tracked in the monitoring image by updating the reference pattern by the reference pattern generating means using the high similarity region and the mask pattern,
It is characterized in that whether or not the object is a target object is determined by the change in the grayscale pattern of the tracked moving object.

[0028]

According to the present invention, when a moving object is tracked by a monitoring device, a region is extracted from a difference image and a mask pattern that is not easily influenced by motion information is used to generate a gray pattern in a dictionary and a prediction window. By tracking by matching with the original image, the appearance location and destination of the moving object can be known, and it can be accurately determined whether it is a true moving object or not depending on where it appears and in which direction. It is possible to improve the performance of determining whether or not the object is a moving object. In addition, since the tracking is performed using a grayscale pattern, it is possible to handle multiple moving objects that intersect and move, and many moving objects that move while separating or merging. Even if they move in different directions, they can be determined to be the same object by matching with the original image. In the matching with the original image, a prediction window corresponding to the speed and direction of the moving object is provided to perform matching by limiting the range, thereby preventing erroneous detection of the moving object and detecting the moving object. Can be shortened.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. First, an outline of the present invention will be described before giving a specific description.

In order to analyze the movement of a moving object, it is considered that the moving object is extracted from an image at a certain time t = ti, and the corresponding region of the moving object is extracted from the images at the subsequent times. To be That is, the “tracking” can be divided into two steps: extraction of a moving object and extraction of a corresponding portion (hereinafter referred to as tracking processing). here,
The moving object extraction processing, which is the first step, can be realized by the method described in the conventional example. By adding the tracking process of the second step described in the present invention to the first step, a more effective image monitoring device can be realized as compared with the conventional example.

As described above, in order to track a moving object, it is necessary to correspond to the moving object with images captured at different times. In order to take this correspondence, time t = t
An area (hereinafter referred to as a prediction window) in the image in which the moving object at i is considered to have moved at time t = ti + 1 is set, and corresponds to the moving object at time t = ti within the prediction window. The area may be extracted. Tracking a moving object is often done by tracking a human, and the person moves by moving their hands or feet.Therefore, when comparing corresponding parts, all dictionaries of the previous frame are compared with the original image of the next frame. , There are places that do not match in moving parts such as hands and feet.

Therefore, by preparing a mask pattern from a plurality of person patterns that is not easily affected by movement and using it for matching, an error is less likely to occur and the tracking performance is improved (the first aspect of the present invention). (1, 2 points). For example, when a person who holds his / her hand down as shown in FIG. 2 (a) raises his / her hand as shown in FIG. By comparing and collating with the original image at the common portion of the mask pattern 31 as shown in c), the collation can be performed without being affected by the movement.

Next, a method of creating a mask pattern will be described. The first method extracts a grayscale pattern of a person of a specific person from the time-series image, corrects the size thereof, and then obtains a variance value of each point. A set of points whose variance value is smaller than a predetermined threshold value is considered to be less likely to be affected by movements.Therefore, a point whose variance value is smaller than the predetermined threshold value is extracted from a plurality of persons. Then, by selecting the common part and using it as a mask pattern, it is possible to create a mask pattern that is not easily affected by movement (the third point of the present invention). In addition, since the shape of the movement is different between the vertical walking and the horizontal walking, an effective matching result can be obtained by using different mask patterns.

In the second method, a plurality of binarized human patterns are size-corrected and then overlapped to create a number distribution. Since points with a large distribution are considered to be less susceptible to movement, a mask pattern is created by using a set of points with a number distribution greater than a predetermined threshold as a mask pattern. It is possible (the 4th point of the present invention).

The mask pattern is a pattern of a predetermined rectangular size which has information as to whether or not it is used for matching, and is used by enlarging / reducing according to the extracted rectangular size at the time of actual matching. Further, at the time of registration in the memory, it is possible to efficiently perform the matching by registering the horizontal star and position of the pixel used for the matching and the length thereof (the fifth point of the present invention).

FIG. 6 shows an example of the number distribution of the rectangular size “16 × 37” obtained from, for example, 65 person dictionaries. FIG. 7 is an example of a mask pattern created by selecting points whose number is “50” or more, and FIG. 8 is an example of registering the mask pattern of FIG. 7 on a memory.

In practice, the mask pattern of which the size is corrected at the time of registering the light and shade pattern and the light and shade pattern of the common portion of the original image are registered as a dictionary, and the light and shade pattern of the dictionary is checked at the time of matching the dictionary and the original image. And the method of matching the original image with the size correction, and all the light and shade patterns of the moving area obtained from the difference at the time of dictionary registration are registered as a dictionary, and the size-corrected mask pattern and the light and shade pattern of the dictionary at the time of matching are registered. A method of matching the common part of the image with the original image can be considered.

Also, depending on the installation location of the camera, FIG.
As shown in, the object size apparently changes in the captured image depending on whether the object is near or far from the camera. The same person may appear several times larger in the same image due to the difference in perspective where an object exists. For this reason, if all the dictionary points of the same person are targeted for collation, the closer to the camera, the larger the dictionary size and the prediction window, and the collation time is considerably long.

It is not necessary to use all the dictionary points for collation, regardless of whether the person is distant or near, unless the characteristics of the shade pattern of the person are lost. Therefore, it is similar to reducing the dictionary to a predetermined dictionary size. In addition, by thinning out the dictionary points and performing the matching, it is possible to shorten the matching calculation time (sixth point of the present invention).

The dictionary size that does not lose the characteristics of the shade pattern of a person is experimentally obtained in advance, and when a moving object is extracted larger than the predetermined dictionary size, the dictionary points are thinned out and the dictionary size is determined in advance. When a moving object smaller than the dictionary size is extracted, the time can be shortened by matching the grayscale pattern with the original image using the mask pattern with the extracted object size,
Tracking can be done efficiently.

Further, as shown in FIG. 9, if the object size is different depending on the distance, the moving pixels of the moving object are apparently different even if the moving speed of the moving object is the same. The size of the prediction window depends on the moving pixel of the moving object, and the moving pixel can be calculated from the specifications and installation conditions of the camera if the actual speed of the moving object is known. However, since the operability is not good if the installation condition is input every time the camera is installed and the moving pixel of the person is calculated, the moving pixel having the speed corresponding to the size of the person in the Y direction as a parameter (X direction). , Y direction) table is created in advance, and after the initial extraction of the moving object, the moving pixel table is drawn to obtain the moving pixel, whereby the prediction window can be set easily (seventh aspect of the present invention). point).

After that, the prediction window is determined according to the direction vector during the tracking process, but when the value of the direction vector is small, the prediction window is determined using the moving pixel table described above.
FIG. 10 shows a table of the size and moving pixels in the Y direction created by considering the speed of a person as 80 m / min. When the moving object is extracted for the first time, the moving object is assumed to be a human, and the moving pixel table of FIG. 10 is subtracted from the size in the Y direction to obtain the moving pixel.

As shown in FIG. 11, for example, when the Y size of the extracted object 91 is 60 pixels and the capturing interval of the collation image is 0.1 sec, 5 pixels are arranged on the left and right sides of the extraction frame, and 1 is arranged on the upper and lower sides.
A prediction window 92 having an enlarged size is set to the size of the added pixels in consideration of some variation. In the subsequent frames, since the direction of motion and the moving pixel (moving vector) are obtained, it is sufficient to determine the prediction window 92 that is large in the vector direction and small in the other directions using the direction vector. When the direction vector is small, the moving pixel is obtained from the moving pixel table corresponding to the predicted speed (normal walking speed), the prediction window 92 is set and the result is collated. , Size corresponding to running speed) By resetting and comparing the prediction window 92, an effective search can be performed.

Since the moving pixels in the Y direction vary considerably depending on the height and depression angle of the camera as compared with the X direction, the moving pixel table should be set to a large size in order to cope with all installation conditions. Need to be kept.

Further, when tracking a moving object, the grayscale pattern dictionary is updated as needed after matching with the original image. However, when there are a plurality of moving objects in the area, there is a possibility that the moving objects will appear to overlap. There is. If the grayscale pattern dictionary is updated when the moving objects overlap each other, the grayscale patterns of the moving objects on the front side are mixed in the dictionary of the grayscale patterns of the moving objects on the back side. If two people update both dictionaries when they overlap exactly, both the back side and the front side will have the same light and shade pattern, and if you follow that dictionary, both will track the same person. .

Whether or not the moving object overlaps can be determined from the position and size information of the moving object being tracked by using a difference image or the like. Therefore, it is determined that the moving object overlaps another moving object. At times, it is preferable not to update the gradation pattern dictionary (the eighth point of the present invention). Here, the determination as to whether or not the moving objects overlap can be made by determining, for example, whether the diagonal corners of the two rectangles are included in another rectangle.

Further, as in the present invention, a grayscale pattern dictionary is created by extracting a change area from an image captured at a predetermined time interval using a difference, and after a predetermined time, the original image and the grayscale pattern dictionary are created. When a moving object is tracked by collating with, the extraction performance is better if the image capturing interval for extracting the change region is longer to some extent. In particular, as shown in FIG. 12A, in the case of vertical walking, if the capture interval is short, the person rectangle may not be extracted as the change region when the difference is taken, and only a part may be extracted (FIG. 1).
2b).

On the other hand, when the original image is compared with the grayscale pattern dictionary, the influence of the difference in size due to movement or perspective is reduced, so it is clear that the image capturing interval should be short. Therefore, an image is captured from the camera 1 at a time interval t0, a captured image at a time interval (t0 × m) is used to generate a difference image for detecting a changed region, and an original image is compared with a grayscale pattern dictionary. Effective tracking can be realized by using captured images at (t0 × k) time intervals (m> k ≧ 1) (ninth point of the present invention).

For example, as shown in the flow charts of FIGS. 3 to 5, images are taken in at intervals of 0.1 sec, images at intervals of 0.5 sec are used to extract change regions, and images at intervals of 0.1 sec are used for collation. To use.

When creating a mask pattern, for example,
When humans are to be tracked, a plurality of humans are sampled to extract a part having little movement except hands and feet.
In other words, it can be said that the variation of the movement is small in the area inside the mask pattern and the variation of the movement is large in the area other than the mask pattern. By examining the variation in the grayscale information inside and outside the mask pattern, the moving object is a human. It can be determined whether or not. That is, the change of the light and shade pattern outside the mask pattern is examined, and if there is no change or a small change, it is determined that the moving object is not a human (10th and 11th points of the present invention).

If the change in the shading pattern inside the mask pattern is examined and the change is large, it is determined that the moving object is not a human (the twelfth point of the present invention). For example, when one plate having a density similar to a human figure moves, the variation of the light and shade pattern outside the mask pattern becomes small, so that it can be determined that the person is not a person.

Further, the difference between the images captured at a predetermined time interval is used when extracting the change area. Especially, when the monitor area is outdoors, if the change area is extracted over the entire monitor area, smoke or swaying of trees may occur. In some cases, an object other than the actual moving object may be extracted as a moving object due to changes in the weather. It is good if it can be determined that the shape of the detection area is not a person, but if the shape is similar to that of a person, the detection is erroneous.

The area in which a person appears can be roughly limited. As shown in FIG. 11, in addition to the outer frame of the captured image, a building or a shadow can be assumed, and it is considered that a person suddenly appears in an area with no shadow. I can't. Therefore, the appearance prediction areas 93 and 94 of the intruding object are set in advance from the monitoring area, the change area is extracted from the difference image, and if the change area and the appearance prediction areas 93 and 94 are in contact with each other, the change area is detected. Can be prevented from being erroneously detected by determining that the object is an intruding object and tracking the object (13th point of the present invention). Of course, the performance is further improved by adding the size judgment.
Further, the processing time can be shortened by extracting the change area only around the appearance prediction area. Next, one embodiment of the present invention will be specifically described.

FIG. 1 shows the configuration of an image monitoring apparatus for carrying out the present invention. The difference between the method of the present invention and the conventional method is that t = ti stored in the image memory 12.
Difference image is obtained by performing the difference calculation between the pixels of the image of No. 1 and the image captured before that, and the difference 2 which can be represented by "1" for the changed pixel in the difference binarization circuit 13. After converting to a binarized image, the CPU 15 obtains a change area and determines that the moving object is a moving object, the moving object is registered (stored) in the person dictionary (storage means) 16 in a light and shade pattern, and in the next frame and thereafter. A prediction window is set by referring to the moving pixel table 17 according to the size of the moving object in the Y direction in the moving direction of the moving object, and the dictionary of the moving object and the original image in the prediction window are set to the mask pattern memory 18 By performing matching processing using a mask pattern that is not easily affected by motion information registered in advance, it is determined that the moving object has moved to the area with the highest degree of similarity, and tracking of the moving object is performed. That.

The mask pattern created by analyzing a plurality of person dictionaries as described above is registered in the mask pattern memory 18 in advance. Further, the position of the moving object and the size of the prediction window in each frame are stored in the tracking table 19. Not only the detection of a moving object, but the history of the moving object is obtained so that the appearance location and the destination of the moving object can be known, and even when two or more people cross each other, they can be tracked. It will be possible.

Next, the procedure for tracking a moving object will be described with reference to the flow charts shown in FIGS. The processing is roughly divided into new object extraction processing and tracking processing. First, a rough flow will be described with reference to the flow chart of FIG. 3. For example, an image is input every 0.1 sec (S1), and the difference binarization circuit 13 uses the time series difference or the background difference method to obtain the difference 2 A binarized image is generated (S2). If there is a tracking object in the previous frame (S
3) The tracking process of the moving object is performed by collating the grayscale pattern dictionary with the original image (S4).

Next, a new object extraction process is performed. As described above, it is better that the new object is extracted at a relatively long capture interval. In this case, every 5 frames (0.5 s).
A new object is extracted (at ec intervals) (S5, S
6), post-processing is performed (S7), and the process ends (S8). In the post-processing, the information stored in the tracking table 19 is used to check whether a person who has failed in tracking is extracted as a new object, a case where tracking is successful, but a case where it is determined as another person, and the result is checked. Fix it. Here, an example of the tracking table 19 is shown in FIG.

First, the new object extracting process will be described with reference to the flow chart of FIG. A moving object is extracted by projecting in the X and Y directions or extracting a closed region using the difference binarized image generated in step S2 (S11).
If there is a new moving object (S12), it is checked whether the new object is in contact with the edge of the frame set in advance according to the monitoring area or the appearance prediction area such as the shadow (S13). ), The person dictionary pattern of the shade pattern of the moving area extracted as described above is generated and registered in the person dictionary 16 (S14).

Since it is possible to predict the next moving direction of the extracted moving object depending on the appearance location, the moving window table 17 is used to set the prediction window which is the collation range in the moving predicted direction. It is set by subtracting and obtaining the moving pixel, and is stored in the tracking table 19 together with a new object extraction flag and position information for tracking in the next frame and later (S15).

Considering the case where there is no shadow of a building or the like in the frame, the predicted movement direction is that the person is expected to appear from four frame edges (or two edges such as the upper right) on the upper, lower, left and right sides. (0, -1), (0, 1), (1, 0), (-1, 0) on the left and right respectively
A movement prediction direction of ((X, Y): X direction, Y direction) is set ((-1, -1) in the case of upper right).

In FIG. 14, since the moving object 122 has emerged from the lower part of the frame 121, the predicted moving direction 123
(0, 1) is given and the prediction window 124 is set. As described above, the size of the prediction window 124 is obtained by subtracting the moving pixel table 17 from the size of the moving object in the Y direction in the initial extraction.

The direction vector from the position of the previous frame to the position of the next frame is used as the predicted movement direction when the moving object is tracked in the next frame. For example, in FIG.
If the moving object 122 of No. moves to the position of 131 in FIG. 15 in the next frame, the predicted movement direction is an arrow 133. Further, since the edges that can appear can be limited as described above, instead of obtaining the changed areas in all the areas in the frame, the changed areas are extracted only around the edges that can appear, thereby reducing the processing time. Is shortened.

Next, the tracking process will be described with reference to the flowchart of FIG. The flag (new object extraction or tracking success) stored in the tracking table 19 is checked, and if there is a tracking object (S3), tracking processing is performed by the following method.

First, the prediction window 132 set for all frames
The grayscale pattern in the inside and the grayscale pattern registered in the person dictionary 16 (the portion 122 in FIG. 14) are used to determine the matching range using a mask pattern registered in the mask pattern memory 18 that is not easily affected by movement. The comparison is limited (S21, first matching process). Here, for example, using the template matching method, the prediction window 132
A region having the highest degree of similarity with the grayscale pattern registered in the person dictionary 16 is extracted.

However, since the shape and density of the moving object may change as the moving object moves, or the size may change slightly due to the difference in distance from the camera 1, a region having a high degree of similarity does not necessarily mean a moving object (person). Does not mean for that reason,
As a method for determining whether or not the extracted area is a moving object, the position of the extracted area in the difference binarized image and the area that is considered to be a person extracted using the original image in the prediction window as described above. The determination is made by comparing the above (S2
2).

In addition to the determination using the above-described difference binary image, the moving direction of the moving object up to that point (123 in FIG. 14) and the direction from the position of the previous frame to the currently extracted position It may be possible to compare the vectors (133 in FIG. 15) for reference, but since the moving object does not always move in the same direction at the same speed, if there are other candidates, the same direction is prioritized. The usage of the degree is good. However, the determination using this direction is effective when a similar person moves from the opposite direction and crosses and passes.

In the first matching process (S21), the comparison result is determined based on the difference between the line omission in the differential area, the ratio of the moving area, and the direction vector. If the matching fails, the size of the prediction window is changed (S2).
3). Normally, the smaller the prediction window, the shorter the calculation time for matching, so referencing the direction vector of the previous frame,
Although the movement direction is particularly emphasized, the collation range is further widened and the prediction window is set to be large (134 in FIG. 15).
Matching processing is performed on the original image in the difference area (S24,
Second matching process).

The evaluation method of the second matching processing is also the first
Similar to the matching process of No. 2, and if the second matching process also fails, it is possible to search in another range, but here, the tracking is terminated. The moving object that has failed to be tracked should be extracted as a new moving object in the new object extracting process (S11 and subsequent steps) to be performed later, and the post-processing (S
Tracking can be continued by associating the tracking failure object with the new object in 7).

As described above, if the matching with the dictionary pattern is successful using the mask pattern and the moving object can be traced, the overlapping with other moving objects is checked (S25). ), If it does not overlap with another moving object, the dictionary pattern is updated (S26),
It is registered in the person dictionary 16, the prediction window and the tracking success flag used in the next frame are set (S27), and the result is stored in the tracking table 19. On the other hand, if the matching with the dictionary pattern fails, a tracking failure flag is set and stored in the tracking table 19 (S28).

After the tracking processing is completed, a new moving object is further extracted from the image obtained by deleting the tracking object extracted by the tracking processing from the difference binary image (S6). FIG.
FIG. 16B shows a difference binarized image of a ti frame in FIG.
An example of the difference binarized image in the (ti + 1) frame is shown in FIG. 16C, and an example of [the difference binarized image in the ((ti + 1) frame-tracking object] is shown in FIG. 16C. The range of the tracking object to be deleted may be a person shape excluding the background or a rectangle including a person. Here, a rectangle including a person is used. After that [differential binary image of (ti + 1) frame-tracking object]
The new object extraction process performed on the image is as described above.

The tracking processing has been described above. However, not only a moving object but also an intruding object (intruder) heading for a dangerous area or an unauthorized area by tracking the moving object by the above method, or It is possible to detect an suspicious intruding object and sound an alarm when the suspicious intruding object is detected, so that effective monitoring can be performed. Further, even when a plurality of intruders cross each other in a frame and walk in, it can be tracked because the matching process is performed using the original image.

As described above, according to the above-described embodiment, when the moving object is tracked by the image monitoring apparatus, the area is extracted from the difference image and the gray level is changed by using the mask pattern which is not easily influenced by the motion information. By tracking the pattern dictionary against the original image in the prediction window, it is possible to know where the moving object has moved. By having a gradation pattern dictionary, even if there are multiple moving objects, they can be associated, and even if they move in different directions, they are judged to be the same object by matching with the original image. be able to.
In matching with the original image, a search window can be shortened by providing a prediction window corresponding to the speed and direction of the moving object and limiting the range to perform matching processing.

A mask pattern is generated by analyzing a plurality of person dictionaries in advance and selecting points with a small variance in time series, or by generating a number distribution by selecting points with a large distribution, and When matching the dictionary of patterns with the original image, it is less affected by movement and good tracking can be realized. In addition, the mask pattern can be saved by expressing the mask pattern in a predetermined rectangular size with the horizontal start position and the length from the start position and registering the mask pattern, and the collation efficiency can be improved.

Further, since the mask pattern is generated by selecting an area which is not easily affected by the movement of the tracking object, when the dictionary of the gradation pattern is collated with the original image, the fluctuation of the gradation information inside and outside the mask pattern is changed. If the variation in the grayscale information outside the mask pattern is small or if the variation in the grayscale information inside the mask pattern is large, it is possible to determine that the moving object is not the target object, and When the target object is used, it is possible to prevent erroneous detection due to fluctuations in the grayscale information due to swaying of trees, small animals, changes in the weather, or the like.

Further, in order to deal with the change in the person size due to the distance, a collation dictionary size is set in advance so as not to reduce the characteristics of the person, and the moving object exists at a close position and the person size is large. In this case, when the grayscale pattern dictionary and the original image are collated, the collation time is reduced by thinning out the collation points. Also, when a moving object is detected for the first time, the speed of the moving object is unknown, so the size of the moving object in the Y direction and the moving pixel (X direction,
It is possible to set an effective prediction window by creating a table (Y direction) in advance and determining the size using the moving pixel table when setting the prediction window.

Further, there are a plurality of moving objects, and it is checked from the position and size information whether or not the moving object overlaps with other moving objects. If it is determined that they overlap, the dictionary is not updated. By doing so, false tracking can be prevented. Even if another person is mistakenly tracked, it is possible to correct it by using the difference image and the direction vector in the table storing the tracking information up to that point.

On the other hand, an image captured at the shortest possible interval in consideration of the matching time is used for matching, and every other several images are used for detection, so that an effective image for detection and matching can be obtained. can get. Further, when a moving object is detected, the moving object is detected by referring to a preset appearance prediction region, which can prevent erroneous detection and shorten the moving object detection time.

From the above, it is possible to realize an efficient image monitoring apparatus by tracking a moving object as in the present invention as compared with the conventional apparatus for detecting a moving object. it can.

[0079]

As described above in detail, according to the present invention, it is possible to accurately track a moving object in a surveillance image, improve the performance of determining whether or not the object is a moving object, and, moreover, detect an erroneous detection of the moving object. It is possible to provide a method of tracking a moving object of a monitoring image, which can prevent the detection time of the moving object and shorten the detection time of the moving object, thereby enabling efficient image monitoring.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image monitoring apparatus to which an embodiment of the present invention is applied.

FIG. 2 is a diagram illustrating the movement of a person.

FIG. 3 is a flowchart schematically illustrating an overall process.

FIG. 4 is a flowchart illustrating a new object extraction process.

FIG. 5 is a flowchart illustrating a tracking process.

FIG. 6 is a diagram showing an example of the number distribution of a person dictionary when creating a mask pattern.

FIG. 7 is a diagram showing an example of a mask pattern.

FIG. 8 is a diagram showing an example in which a mask pattern is registered.

FIG. 9 is a diagram showing a difference in object size depending on perspective.

FIG. 10 is a diagram showing an example of an object size and moving pixels.

FIG. 11 is a diagram showing an example of a setting method of a prediction window.

FIG. 12 is a diagram illustrating an example in which only a part is extracted from a difference image.

FIG. 13 is a diagram showing an example of a tracking table in tracking processing.

FIG. 14 is a diagram showing an example of a ti frame image in a tracking process.

FIG. 15 is a diagram showing an example of an image of a (ti + 1) frame in tracking processing.

FIG. 16 is a diagram for explaining a new object extraction process in the tracking process.

FIG. 17 is a configuration diagram showing an example of an image monitoring device.

FIG. 18 is a block diagram showing a configuration of a conventional image monitoring device.

FIG. 19 is a diagram illustrating a region extraction method using projection.

[Explanation of Codes] 1 ... ITV camera, 2 ... monitoring area, 3 ... transmission path,
4 ... Processing device, 5 ... VTR, 6 ... Alarm device, 7 ...
Display device, 10 sampling pulse generation circuit, 1
1 ... A / D converter, 12 ... Image memory, 13 ... Difference binarization circuit, 14 ... Difference binarized image memory, 15 ...
... CPU, 16 ... person dictionary, 17 ... moving pixel table, 18 ... mask pattern memory, 19 ... tracking table.

Claims (4)

[Claims] 1. An image in a surveillance area is picked up by an image pickup means and inputted, and the inputted image in the surveillance area is continuously taken in at a predetermined time interval and digitized, and the digitized plurality of images are obtained. In the image monitoring apparatus that monitors the moving object in the monitoring area by calculating the image of the difference image, the difference image of the digitized image is calculated, and the calculated difference image is binarized to obtain the difference binary value. When a changed image is generated, a change area is extracted from the generated difference binarized image, and the extracted change area is in contact with a preset appearance prediction area in which a moving object may appear, A method of tracking a moving object in a surveillance image, wherein the moving object is determined to be a target object and is tracked. 2. An image in the surveillance area is picked up by an image pickup means and inputted, the inputted image in the surveillance area is continuously taken in at a predetermined time interval to be digitized, and the digitized plurality of images are obtained. In the image monitoring apparatus that monitors the moving object in the monitoring area by calculating the image of the difference image, the difference image of the digitized image is calculated, and the calculated difference image is binarized to obtain the difference binary value. A changed area extracting unit that generates a changed image and extracts a changed area from the generated difference binary image, a changed area extracted by the changed area extracting unit, and an original digitized by input from the image pickup unit. A light / dark pattern extracting means for extracting a light / dark pattern of the change area portion from the image, and a pattern for masking a predetermined range of the light / dark pattern extracted by the light / dark pattern extracting means. A mask pattern generating means for generating a turn, a reference pattern generating means for obtaining a reference light and shade pattern from an output of the mask pattern generating means and an output of the light and shade pattern extracting means, and a reference pattern obtained from the reference pattern generating means. And by comparing the digitized original image with
And a similarity calculation unit that obtains a region having a high similarity with the reference pattern in the original image, captures an image in the monitoring region at predetermined time intervals, and extracts the change region by the change region extraction unit, A reference pattern corresponding to the changed area is obtained by the reference pattern generation means, and thereafter, every time an image is captured, the similarity calculation means uses the reference pattern to obtain a highly similar area in the captured image and a predetermined time elapses. It is possible to track the moving object in the monitoring image by determining the position of the moving object after that, and updating the reference pattern by the reference pattern generating means using the high similarity area and the mask pattern. A method for tracking a moving object in a characteristic surveillance image. 3. An image in the surveillance area is picked up by an image pickup means and inputted, the inputted image in the surveillance area is continuously taken in at a predetermined time interval to be digitized, and the digitized plurality of images are obtained. In the image monitoring apparatus for monitoring the moving object in the monitoring area by calculating the image of the above, the difference image of the digitized images captured at the predetermined time interval t1 is calculated, and the calculated difference is calculated. The image is binarized to generate a differential binarized image, and a changing region extracting unit that extracts a changing region from the generated differential binarizing image, a changing region extracted by the changing region extracting unit, and the imaging unit. The grayscale pattern extraction means for extracting the grayscale pattern of the changed region from the original image digitized by inputting the grayscale pattern and the grayscale pattern extracted by the grayscale pattern extraction means are predetermined. Mask pattern generating means for generating a pattern for masking the range, reference pattern generating means for obtaining a reference light and shade pattern from the output of the mask pattern generating means and the output of the light and shade pattern extracting means, and the reference pattern By comparing the reference pattern obtained from the generating means with the digitized original image captured at a predetermined time interval t2 shorter than the time interval t1,
And a similarity calculation unit that obtains a region having a high similarity with the reference pattern in the original image, captures an image in the monitoring region at predetermined time intervals, and extracts the change region by the change region extraction unit, A reference pattern corresponding to the changed area is obtained by the reference pattern generation means, and thereafter, every time an image is captured, the similarity calculation means uses the reference pattern to obtain a highly similar area in the captured image and a predetermined time elapses. It is possible to track the moving object in the monitoring image by determining the position of the moving object after that, and further updating the reference pattern by the reference pattern generating means using the high similarity area and the mask pattern. A method for tracking a moving object in a characteristic surveillance image. 4. An image in the surveillance area is picked up by an image pickup means and inputted, the inputted image in the surveillance area is continuously taken in at a predetermined time interval and digitized, and the digitized plurality of images are obtained. In the image monitoring apparatus that monitors the moving object in the monitoring area by calculating the image of the difference image, the difference image of the digitized image is calculated, and the calculated difference image is binarized to obtain the difference binary value. A changed area extracting unit that generates a changed image and extracts a changed area from the generated difference binary image, a changed area extracted by the changed area extracting unit, and an original digitized by input from the image pickup unit. A light / dark pattern extracting means for extracting a light / dark pattern of the change area portion from the image, and a pattern for masking a predetermined range of the light / dark pattern extracted by the light / dark pattern extracting means. A mask pattern generating means for generating a turn, a reference pattern generating means for obtaining a reference light and shade pattern from an output of the mask pattern generating means and an output of the light and shade pattern extracting means, and a reference pattern obtained from the reference pattern generating means. And by comparing the digitized original image with
Similarity calculating means for obtaining an area having a high degree of similarity with the reference pattern in the original image, and after obtaining an area having a high degree of similarity with this similarity calculating means, the change of the shading pattern inside or outside the mask pattern is calculated. And a determination unit that determines whether or not the moving object is a target object that is a target based on the magnitude of the change, captures an image in the monitoring area at predetermined time intervals, and the change area is extracted by the change area extraction unit. When extracting the reference pattern corresponding to the change area by the reference pattern generating means, thereafter, the similarity calculation means uses the reference pattern every time the image is captured, the area having a high degree of similarity in the captured image. To determine the position of the moving object after a lapse of a predetermined time, and further, by using the high similarity area and the mask pattern, the reference pattern generation means Movement of the monitoring image characterized by tracking the moving object in the monitoring image by updating the turn, and determining whether or not the target object is the target based on the change in the gradation pattern of the tracked moving object. Object tracking method.