JP2003169319A - Image-monitoring apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image-monitoring apparatus for accurately detecting only a mobile object by appropriately segregating background variation and the mobile object from images to be monitored. <P>SOLUTION: The image-monitoring apparatus comprises an image-inputting means 1 for inputting images to be monitored, an image change-extracting means 12 for setting a change region by extracting change in an image that is inputted by the image-inputting means 1, and a movement-judging means 13 that arranges a correlation block at the change region that is set by the image change-extracting means 12 and extracts the mobile object from images that are inputted from the image-inputting means 1 by a correlation operation to be made for each correlation block. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention processes an image obtained from a surveillance camera to detect a moving object such as an intruder or a vehicle,
The present invention relates to an image monitoring device that automatically discriminates background fluctuations such as tree swings and irregular reflections on the water surface, and accurately extracts moving objects such as intruders and vehicles.

[0002]

2. Description of the Related Art FIG. 10 shows Toshihiko Morita, "Motion Detection and Tracking by Local Correlation Calculation," IEICE Transactions D-II.
Vol. J84-D-II No. 2 pp. 299-
It is a block diagram which shows the structure of the conventional video surveillance system published in 3092001 February. In the figure, 1 is an image input unit, 4a is a detection region setting unit, 5a is a correlation calculation block arrangement unit, 6 is a past image storage unit, 7 is a correlation calculation execution unit, 8 is an intra-region motion vector integration unit, and 9 is a space. Dynamic motion uniformity determining means, 10 is a motion vector integrating means within a fixed time, and 11 is temporal motion uniformity determining means.

Next, the operation will be described. The image is input to the detection area setting means 4a by the image input means 1 and this image is divided into a plurality of detection areas. The size of this detection area is fixed at the time of division and is not changed in the subsequent processing. Each detection area divided by the detection area setting means 4a is input to the correlation calculation block arrangement means 5a, and each detection area is further divided into a plurality of correlation blocks. The image output from the image input means 1 is the past image storage means 6
Is input to and sequentially consecutive images are accumulated. The correlation calculation execution means 7 inputs the image sent from the image input means 1 and the image stored in the past image storage means 6, and performs the correlation calculation between the images continuous from the past for each correlation block,
The motion vector of each correlation block is calculated. In this way, the motion vector of each correlation block in the detection area obtained by the correlation calculation executing means 7 is integrated by the intra-area motion vector integrating means 8 to obtain the motion vector representing the detection area. When the motion vector of each correlation block in the detection area is spatially almost uniform in a certain detection area, the motion vector representing the detection area shows the average motion of the motion vectors of each correlation block. . On the contrary, when the motion vector in each correlation block in the detection region is not spatially uniform, the motion vectors in each correlation block cancel each other out by the integration of the intra-region motion vector integration means 8, and finally the motion is zero. Converge to.

Spatial motion uniformity determining means 9 correlates the motion vector representing the detection area with the correlation calculation block arranging means 5.
a, and the detection area is tracked based on the motion vector representing this detection area, and the correlation calculation block arrangement means 5
a, the correlation calculation executing means 7, and the intra-region motion vector integrating means 8 are caused to obtain the motion vector representative of the next detection region, and the motion vector representative of the plurality of detection regions thus obtained within a certain period of time is obtained. Store. In addition, the presence / absence of motion is determined for the motion vector representative of each detection area, and the determination result and the stored motion vector representative of the detection area within a fixed time are sent to the motion vector integration means 10 within a fixed time. Output.

The motion vector integration means 10 within a fixed time is
The temporal integration of the motion vectors representative of the plurality of detection areas obtained within the input fixed time is performed using the determination result of the presence or absence of motion.

The movement within the detection area is represented by a plurality of motion vectors that represent detection areas having substantially the same size and the same direction within a fixed time, and movements that represent the detection area obtained within such a fixed time. When the vectors are integrated, for example, the average motion vector of the motion vectors representing the detection area is obtained. On the other hand, when reciprocating motion or random motion is performed in the detection area, when the motion vectors representing the detection area within a certain time are integrated, they cancel each other and finally the motion vector converges to zero.

The motion vectors integrated within a certain time period are input to the temporal motion uniformity determining means 11, and the determination is made as follows. If the motion vector integrated within a certain period of time indicates an average motion of the motion vector representing the detection area before integration, this detection area indicates a moving object such as an intruder or a vehicle. Judge,
If it is any other motion vector, it is determined that the fluctuation of the tree in the background fluctuation or the sparkle of the water surface is indicated, and only moving objects are discriminated and extracted from various moving objects in the image.

[0008]

Since the conventional video monitoring apparatus is configured as described above, when a plurality of moving objects and background fluctuations exist in the initially set detection area, and the moving directions are different from each other. Had a problem that they would cancel each other's motion vectors and failed to extract moving objects.

Further, since the size of the correlation blocks to be arranged is constant, for example, when the moving object is small, the area of the detection area becomes small, and a small number of correlation blocks are arranged. Since the number is small, there is a problem that the error included in the correlation calculation of the motion vector has a great influence, the accuracy of the determination deteriorates, and the extraction of the moving object finally fails.

Further, when the moving object displayed in the image is large, it is dealt with by the image degeneracy process, but the extraction of the moving object requiring the image process with the largest degeneracy rate is prioritized. However, when a plurality of moving objects of different sizes are present in the same image at the same time, there is a problem that extraction of a small moving object is likely to fail.

The present invention has been made in order to solve the above problems. Even when a plurality of moving objects having different movements approach each other and appear in the detection area, whether or not each moving object is a moving object individually. It is an object of the present invention to obtain a video surveillance device that judges and extracts.

Even if the size of the moving object in the image is small, such as several times as large as the size of the correlation block, optimum correlation blocks are arranged according to the size of the moving object shown in the image. It is an object of the present invention to provide a video monitoring device that can be extracted without oversight and that can be accurately extracted even when a moving object in an image is large.

Further, it is another object of the present invention to provide an image monitoring apparatus which calculates a correct motion vector individually and accurately detects a moving object even when a plurality of moving objects having different sizes exist in the detection area.

[0014]

An image monitoring apparatus according to the present invention includes image input means for inputting a monitoring image, and image change extraction for setting a change area by extracting a change in the image input by the image input means. Unit and a plurality of correlation blocks are arranged in the change area set by the image change extraction unit, correlation calculation is performed for each correlation block, and a moving object is detected from the image input by the image input unit based on the result of the correlation calculation. And a motion determining means for performing the motion determination.

In the video monitoring apparatus according to the present invention, the image change extraction means removes the background image from the image input by the image input means, the background difference means for updating the background image, and the background difference means. And a changing area extracting means for setting a changing area in the image from which the background image has been removed.

In the video monitoring apparatus according to the present invention, the background updating means updates the background image based on the information when the moving object acquired from the motion determining means is detected.

In the video monitoring apparatus according to the present invention, the motion determination means arranges a plurality of correlation blocks in the change area set by the image change extraction means, and the image input by the image input means. A past image storage means for storing the following, a correlation calculation execution means for performing a correlation calculation for each correlation block arranged in the change area to obtain a motion vector, and an intra-region motion vector integration means for integrating the motion vector for each change area. A spatial motion uniformity determining unit that stores the intra-region motion vector integrated by the intra-region motion vector integrating unit and determines the motion indicated by the intra-region motion vector, and a determination result of the motion indicated by the intra-region motion vector By using, the constant time to integrate the in-region vectors stored in the spatial motion uniformity determination means within the constant time A motion vector integrating means and a temporal motion uniformity determining means for determining a temporal motion uniformity of the intra-time motion vector integrated by the constant time motion vector integrating means and extracting a moving object are provided. It is a thing.

In the video monitoring apparatus according to the present invention, the correlation calculation block arranging means obtains the difference between the present image input by the image input means and the past image accumulated in the past image accumulating means, and this image is obtained. The correlation block is arranged based on the difference of

In the video monitoring apparatus according to the present invention, the correlation calculation block arrangement means preferentially arranges the correlation blocks from the portion of the image input by the image input means having a large number of edge components.

In the video monitoring apparatus according to the present invention, the correlation calculation execution means obtains the correlation score data by the correlation calculation performed on the correlation block, and the integration processing performed by the intra-region motion vector integration means and the intra-region motion vector integration means are performed. Correlation score data is used for the integration processing to be performed.

In the video monitoring apparatus according to the present invention, the correlation calculation executing means obtains the correlation score data by the correlation calculation performed on the correlation block, and the determination processing and the temporal motion uniformity performed by the spatial motion uniformity determining means. Correlation score data is used for the determination process performed by the determination means.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below. Embodiment 1. 1 is a block diagram showing a configuration of a video surveillance apparatus according to Embodiment 1 of the present invention. In the figure, 1 is an image input means for inputting an image picked up by a camera, 2 is a background difference means for outputting a difference result image in which the difference result between the input image and the background image is the pixel value of each pixel. is there. Reference numeral 3 denotes a background updating unit that stores a background image and updates the stored background image based on a predetermined condition. Reference numeral 4 denotes a changing area extracting means for extracting a changing portion in the image from the output result of the background subtracting means 2 and setting this as a changing area. Reference numeral 5 is a correlation calculation block arrangement means for arranging a plurality of correlation blocks in the change region, 6 is a past image accumulation means for accumulating the input image, 7 is a correlation calculation execution means for performing a correlation calculation for each arranged correlation block, 8 Is an intra-region motion vector integrating means for obtaining a motion vector representative of a change region obtained by integrating the result of the correlation calculation obtained for each correlation block for each change region; and 9 is a motion representative of the change region obtained within a fixed time. Spatial motion uniformity determining means for storing vectors, 10
Is a motion vector integration means within a constant time, which integrates the motion vectors stored in the spatial motion uniformity determination means 9 and which represents a change region within a fixed time, to obtain a temporal motion vector,
Reference numeral 11 is a temporal motion uniformity determining means for determining whether the temporal motion vector indicates a moving object. Reference numeral 12 is an image change extracting means composed of the background subtracting means 2, the background updating means 3, and the changed area extracting means 4. Reference numeral 13 denotes a correlation calculation block placement means 5, a past image storage means 6, a correlation calculation execution means 7, an intra-region motion vector integration means 8, a spatial motion uniformity determination means 9, a constant time motion vector integration means 10, and a time. It is a motion determination means configured by the dynamic motion uniformity determination means 11.

Next, the operation will be described. The image input means 1, the past image storage means 6, the correlation calculation execution means 7, the intra-region motion vector integration means 8, the spatial motion uniformity determination means 9, and the fixed time within the fixed time period which compose the video monitoring apparatus according to the first embodiment. The basic operations of the motion vector integration means 10 and the temporal motion uniformity determination means 11 are the same as those of the conventional video surveillance device shown in FIG. 10 as described later.

FIG. 2 is an explanatory diagram showing an image monitored by the image monitoring apparatus according to the first embodiment. For example, in FIG.
A case will be described in which an image including a pedestrian, a tree swaying in the wind, and light irregularly reflected on the water surface shown in FIG. When the image shown in FIG. 2A is input to the video change extraction means 12 by the image input means 1, the background difference means 2 is stored in the image input from the image input means 1 and the background update means 3. The difference between the background image and the existing background image is obtained and the image from which the background image is removed is output. For example, when the background image is not stored in the background updating unit 3 such as when the video monitoring device is started, the image output from the image input unit 1 is stored in the background updating unit 3, and the difference from this image is stored. Is calculated by the background difference means 2. That is, the operation of the background difference means 2 at this time is to perform difference processing on images that are sequentially input, and this image is appropriately stored in the background update means 3 as a background image.

As a method of obtaining the difference between images, for example,
A description will be given of a method of obtaining the average of a plurality of input images and obtaining the background image based on the average image. When the input images sampled in a time series are synthesized, the moving object or the oscillating object in the image is deviated in outline and overlapped, and the appearance becomes unclear. An average image is obtained by superimposing a predetermined number of images, and an object whose appearance is unclear is removed from this image and stored in the background updating means 3 as a background image.

When the background image is stored in the background updating means 3, the background difference means 2 compares the input image sequentially input from the image inputting means 1 and the background image stored in the background updating means 3 with each other. The difference is calculated, and the background image is removed from the input image, and an image in which a changeable object is extracted is output. The background image stored in the background updating means 3 is appropriately updated, for example, when an average image is obtained from a predetermined number of input images.

FIG. 2B shows an example of an image showing a pedestrian, a tree swaying in the wind, and light diffusely reflected on the water surface, which is output by the background subtraction unit 2 after removing the background image. This image is input to the change area extracting means 4, and image areas in which pedestrians, trees swaying in the wind, and light which is diffusely reflected on the water surface are extracted, and these are set as change areas. In the processing up to this point, a change area other than a pedestrian, which is a true moving object, is also set, and even if there is no true moving object only by the processing of the image change extracting unit 12, the light sways or is reflected irregularly on the water surface. In response, detection of moving objects may be triggered. The image processing performed by the image change extraction unit 12 is performed for each pixel forming an image.

The correlation calculation block arranging means 5 constituting the motion judging means 13 arranges a plurality of correlation blocks on each change area set by the change area extracting means 4. The image input by the image input unit 1 is stored in the past image storage unit 6. After that, the correlation calculation block arranging means 5 associates the correlation block arranged in the change area with the image input from the image input means 1 and the correlation calculation executing means together with the past images accumulated in the past image accumulating means 6. Input to 7. The correlation calculation executing means 7 performs correlation calculation between the past and present images for each correlation block, calculates the motion vector of each correlation block, and outputs this to the intra-region motion vector integrating means 8. After that, the intra-region motion vector integration means 8 integrates the input motion vectors of the correlation blocks for each change region to obtain a motion vector representing the change region (hereinafter, referred to as an intra-region motion vector).

The spatial motion uniformity determining means 9 stores the intra-region motion vector at each time point obtained within a predetermined time (for example, about 1 second), and is represented by these intra-region motion vectors. Judgment is made regarding "presence or absence of movement". This determination result is input to the motion vector integration means 10 within a certain time together with the stored intra-region motion vector, and is used for integration processing described later.

Further, the spatial motion uniformity determining means 9 inputs the intra-region motion vector to the correlation calculation block arranging means 5 and sequentially shifts and tracks the changed regions. The correlation calculation block arranging unit 5 arranges the correlation blocks for the changed regions that have been tracked, and then the correlation calculation executing unit 7 and the intra-region motion vector integrating unit 8 perform the same processing as that described so far. As described above, the intra-region motion vector obtained within a certain time as described above is sequentially stored in the spatial motion uniformity determining means 9.

The motion vector integration means 10 within a fixed time
A plurality of intra-regional motion vectors, which are stored in the spatial motion uniformity determining means 9 and are obtained within a certain time, are input and integrated as intra-time motion vectors. If multiple intra-region motion vectors input within a certain time have the same magnitude and direction vector amount, the integrated intra-region motion vector is the average motion of the intra-region motion vectors before integration. Will be shown. Also, when reciprocating motion or random motion is performed within the change region within a fixed time, the motion vectors within each region cancel each other out, and this vector amount finally converges to zero.

The motion vector integrated by the motion vector integration means 10 within a fixed time is the temporal motion uniformity determination means 1
1 is input, and if the intra-time motion vector satisfies a predetermined condition, it is determined that the change area related to the intra-time motion vector indicates a moving object. In addition, when the intra-time motion vector is substantially zero, it is determined that the change area related to the intra-time motion vector does not indicate a moving object. The video monitoring apparatus according to the first embodiment extracts a moving object from the image thus input, and issues a predetermined alarm or the like as needed to detect the moving object.

Next, detailed operations of the correlation calculation block placement means 5 and the correlation calculation execution means 7 which mainly constitute the motion determination means 13 will be described. Excluding the image change area from the configuration of the video monitoring device, do not set the image change area,
When a motion vector is obtained by arranging a correlation block in a range fixed to a certain area as in a conventional video surveillance apparatus, as shown in FIG. 2C, a motion showing two different moving directions in one area is obtained. It happens when a vector exists. In addition, there are cases where multiple motion vectors that indicate the same direction cannot be covered in the same area, and if the correlation blocks are uniformly arranged without making a judgment as to what range each area should be set in this way, it will be The motion vector will be obtained for a smaller area or a larger area than that, and as a result, the background fluctuation cannot be extracted as a moving object, or the true moving object cannot be extracted from the image. Occasionally you may miss a report about what you are monitoring.

Therefore, by providing the image change extracting means 12 in the video monitoring apparatus, the image change extracting means 12 causes the image change extracting means 12 to intrude in the image (pedestrian in FIG. 2A), as shown in FIG. 2D. Is set to a portion that is likely to be), and the motion determination unit 13 determines whether or not the change area is moving in the same direction and speed for a certain period of time. It is possible to extract only moving objects that move in the same direction and speed within a fixed time.

In this way, when the change area in the image is correctly set and the motion vector is determined, the motion determining means 1 is set for the change area set by the image change extracting means 12.
3 determines an intra-region motion vector, and determines the presence or absence of a moving object in the change region based on this intra-region motion vector. The contents of this judgment are items (a) to (d) shown below.
Can be classified into FIG. 3 is an explanatory diagram showing processing and determination performed by the video monitoring apparatus according to Embodiment 1 on a change area. Items (a) to (d) will be described with reference to FIG.

(A) Moving objects such as intruders and vehicles are shown in FIG.
As shown in (a), if the motion vector distribution in the change region (for each correlation block) is substantially “uniform”, the intra-region motion vector integrated by the intra-region motion vector integration means 8 becomes a spatial motion equal. "There is motion" by the characteristic determination means 9
Is determined. Further, when the temporal distribution of the intra-region motion vector is also almost uniform within the constant time, if the intra-region motion vector is temporally integrated by the intra-region motion vector integrating means 10. A motion vector in time indicating "linear motion" is obtained. If this intra-time motion vector is a motion vector indicating "linear motion", the temporal motion uniformity determination means 11 determines that the final determination result is "moving object such as intruder, vehicle, etc." It is determined that

(B) As shown in FIG. 3 (b), the distribution of motion vectors in the change region (for each correlation block) of a tree, a shake of a shadow, a halo due to direct light, etc. is almost "uniform". , The intra-region motion vector output from the intra-region motion vector integration means 8 to which these intra-variation-region vectors are input,
The spatial motion uniformity determining means 9 determines that there is a motion. However, the distribution of motion vectors within a region is not uniform, and it consists of multiple motion vectors within a region that indicate "reciprocating motion, random motion", etc. The temporal integration by means 10 causes them to cancel each other out to show zero motion. From this, the temporal motion uniformity determining unit 11 determines that the change area indicates “shadow, sway of tree, halo of direct light” or the like as a final determination result.

(C) Shaking of grass and leaves, diffuse reflection on the water surface, etc. are within the change region (each correlation block), as shown in FIG. 3 (c).
The distribution of the motion vectors of the regions is not uniform but “non-uniform”, and the spatial integration result by the intra-region motion vector integration unit 8 indicates that the motion vector representative of the change region indicates “motion zero”. Become. Therefore, the intra-region motion vector indicating "zero motion" is set as the motion vector integration means 1
With 0, even if they are integrated within a certain period of time, the obtained motion vector becomes “motion zero”, that is, a still state, and the temporal motion uniformity determining unit 11 determines that the change region concerned is the final determination result. "Sway of grass, leaves, irregular reflection of water surface"
Etc. are determined to be indicated.

(D) As shown in FIG. 3 (d), the motion vector in the change region (for each correlation block) is "all zeros" for abrupt sunshine fluctuations, etc. As a result, the motion vector that is representative of the change area also indicates “motion zero”. Therefore, the integration result of the intra-region motion vectors within a certain time also indicates “stillness” (complete stillness) of which the motion vector is zero, and the temporal motion uniformity determining means 11 determines that the change region concerned is the final determination result. Is determined to indicate "rapid sunshine fluctuation".

As described above, according to the first embodiment, a moving object such as an intruder or a vehicle can be discriminated from a background fluctuation such as a sway of a tree or a sparkle of the water surface, and the moving object can be accurately measured. There is an effect that only one can be extracted.

Since the intra-region motion vector is obtained for each change region and whether or not the change region is a moving object is determined based on this intra-region motion vector, each change region is contacted unless the change region touches. The moving area can be easily determined, and even if multiple moving objects are present close to each other in the same image, or even if the moving object and background fluctuation are present close to each other, the moving object can be accurately extracted. There is an effect that can be.

Embodiment 2. 4 is a block diagram showing the configuration of a video surveillance device according to Embodiment 2 of the present invention.
The same or corresponding parts as those shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted. The video monitoring apparatus shown in FIG. 4 is obtained by adding the information about the moving object extracted in the previous process, which is output from the temporal motion uniformity determining means 11, to the input of the background updating means 3 shown in FIG. so,
Other configurations are the same.

Next, the operation will be described. Embodiment 2
The background updating unit 3 of FIG. 3 not only extracts the image input from the image input unit 1 and the background difference result output from the background difference unit 2, but also finally extracts the moving object by the temporal motion uniformity determining unit 11. Enter information about the situation
It controls the background image update process and generates an appropriate background image. The other operations are the same as those of the video surveillance apparatus according to the first embodiment, and the description thereof will be omitted.

When the temporal motion uniformity determining means 11 finally extracts the moving object in the previous moving object extraction processing, a background image that correctly represents the background seen when the moving object is removed is used. It can be guessed that it was done.
In this way, the background image in which only the moving object is extracted is one in which the factors causing the background fluctuation and the moving object can be separated,
Even in the processing on a new input image, there is a high possibility that only the moving object can be extracted by eliminating the influence of the background fluctuation.
For this reason, the background updating unit 3 acquires, for example, the information about the pixel representing the moving object extracted from the image, as the information obtained from the temporal motion uniformity determining unit 11 in executing the background updating. However, among the stored background images, the current display is maintained without updating the pixels of the background image for the pixels indicated by the above information, and the pixels representing the other backgrounds are the current background image. For example, the contents of the background image are updated according to a predetermined background image updating procedure such as averaging in time series. As described above, since the portion which is once recognized as the background is not updated, a more faithful background image can be generated and an appropriate change area can be set.

As described above, according to the second embodiment, the background updating means 3 provided in the image change extracting means 12 is provided.
In the above, since the background image is automatically updated / generated by using the input image, the background difference result, and the final moving object extraction result, the background image is updated in consideration of the past change area. There is an effect that a more accurate background image can be generated.

Embodiment 3. The configuration of the video surveillance apparatus according to Embodiment 3 of the present invention is the same as that of the video surveillance apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. The following description will be given using the reference symbols.

Next, the operation will be described. Correlation calculation block placement means 5 of the video surveillance apparatus according to the third embodiment
When arranging the correlation blocks in the change area set by the image change extracting means 12, the blocks are not evenly arranged in the change area, but the pixels forming the past image and the current image are formed. The correlation block is arranged with emphasis on the portion where many pixels remain as the difference. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

The correlation calculation block arranging means 5 inputs the past image stored in the past image accumulating means 6 and the current image sent from the image inputting means 1, and calculates the difference between the pixels of the two images. (Difference between frames) is calculated. In these two images, the magnitude of the inter-frame difference value reflects the change in the pixel, that is, the presence or absence of movement, and the inter-frame difference value becomes high at the part where the movement occurs, and conversely the part where the movement does not occur. Then, the inter-frame difference value becomes a low value. In this way, since the magnitude of the inter-frame difference value reflects the change of pixels, that is, the presence or absence of movement, for example, a different state, that is, the distribution of pixels changed between frames is obtained from the difference between two screens. The correlation block is arranged in the portion where the number of changed pixels is large. This arrangement is preferentially performed from the portion that can cover the most pixels in one correlation block. In this way, the correlation block is arranged in the change area, and the presence / absence of a moving object in the change area is determined while reducing the operation load of the motion determination means 13.

As described above, according to the third embodiment, since the correlation blocks are preferentially arranged in the portion where the change in the image is large, a large number of correlation blocks are arranged due to the restriction of the calculation processing speed and the like. Even if it is not possible, there is an effect that the correlation blocks can be efficiently arranged and the moving object can be accurately extracted.

Fourth Embodiment The configuration of the video monitoring apparatus according to Embodiment 4 of the present invention is the same as that of the video monitoring apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. 1 or the corresponding parts are the same. The following description will be given using the reference symbols.

Next, the operation will be described. Correlation calculation block arrangement means 5 of the video surveillance apparatus according to the fourth embodiment
In the case of arranging the correlation blocks, instead of arranging the correlation blocks uniformly in the change area, the correlation blocks are arranged with emphasis on the portion where the edge component exists in the image. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

Since the edge component of the image is the part that most clearly represents the motion, the correlation calculation block arrangement means 5
Correlation blocks are arranged by detecting edge components of the input image. This arrangement is carried out preferentially from the portion where one correlation block can cover the most edge components so that various motions can be extracted by covering as many edge components as possible. In this way, the correlation block is arranged in the change region, and the operation load of the motion determination means 13 is reduced,
The presence or absence of a moving object is determined for the change area.

As described above, according to the fourth embodiment, since the correlation blocks are preferentially arranged in the area having the edge component, when a large number of correlation blocks cannot be installed due to the restriction of the calculation processing speed and the like. However, there is an effect that the correlation blocks can be efficiently arranged to accurately extract the moving object.

Embodiment 5. The configuration of the video monitoring apparatus according to the fifth embodiment of the present invention is the same as that of the video monitoring apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. The following description will be given using the reference symbols.

Next, the operation will be described. Correlation calculation block arrangement means 5 of the video surveillance apparatus according to the fifth embodiment
When the area of the change area is smaller than the size of the correlation block, when arranging the correlation blocks, not only the correlation blocks are arranged evenly in the change area, but also the correlation blocks evenly arranged in the change area. In addition, the correlation blocks are displaced by half the size of the correlation blocks, that is, the correlation blocks arranged in two rows are arranged so as to overlap each other with the center line in the longitudinal direction of the correlation blocks evenly arranged in the change region as a boundary. is there. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

FIG. 5 is an explanatory diagram showing the correlation blocks arranged in the change region by the correlation calculation block arrangement means 5 of the fifth embodiment. For example, when arranging the correlation blocks in the image of a pedestrian, a plurality of correlation blocks are arranged so as to form a vertically long rectangular shape. As shown in FIG. 5A, when the area of the change region that covers the image of the pedestrian is as small as several block sizes, the number of obtained motion vectors is also several (the one shown in FIG. 3), and the accuracy of the determination of a moving object using these is not sufficient. Therefore, for example, the correlation blocks are arranged horizontally in the vertical direction in the drawing so that the correlation blocks are overlapped by half in the horizontal direction in the drawing, and two correlation blocks are arranged in the upper right and the upper left. By performing such a process on all the correlation blocks evenly arranged in the change area, the number of motion vectors obtained from the change area increases as shown in FIG. 5B. The determination unit 13 can accurately perform the motion determination.

As described above, according to the fifth embodiment, since the correlation blocks are evenly arranged in the change area and the correlation blocks are arranged so as to overlap each other by shifting by a predetermined amount, the area of the change area is reduced. Even when the size of the correlation block is small, as many motion vectors as possible can be calculated, and the moving object can be extracted with high accuracy.

Sixth Embodiment The configuration of the video monitoring apparatus according to Embodiment 6 of the present invention is the same as that of the video monitoring apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. 1 or the corresponding parts are the same. The following description will be given using the reference symbols.

Next, the operation will be described. A moving object displayed in a large size in the image moves at a high speed in the image, that is, it moves quickly among a large number of arranged correlation blocks, and may exceed the set range of the change region in the correlation calculation. However, the accurate movement determination cannot be performed because the background fluctuation that appears in the image is also the same. Therefore, the correlation calculation block arranging means 5 of the video monitoring apparatus according to the sixth embodiment is effective when the area of the change region is large with respect to the size of the correlation calculation block (a moving object is shown in a large size in the image). Is to reduce the image and arrange the correlation block. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

FIG. 6 is an explanatory diagram showing the correlation blocks arranged in the change area by the correlation calculation block arrangement means 5 of the sixth embodiment. As shown in FIG. 6, the correlation calculation block arrangement means 5 of the sixth embodiment reduces the entire image to 1 / N before arranging the correlation block when, for example, a pedestrian is greatly reflected in the image. Then, the correlation block is arranged in the change region for this degenerated image. The correlation block is processed by the correlation calculation executing means 7 described in the first embodiment to obtain the motion vector,
If this motion vector is multiplied by N, a true motion vector can be obtained.

As described above, according to the sixth embodiment, when the area of the change region is large with respect to the size of the correlation block, the entire image is degenerated to obtain the motion vector, and this motion vector is calculated. Since the image is multiplied by the magnification corresponding to the degeneracy, there is an effect that even a moving object that is greatly reproduced in the image can be accurately extracted.

Embodiment 7. The configuration of the video surveillance apparatus according to Embodiment 7 of the present invention is the same as that of the video surveillance apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. The following description will be given using the reference symbols.

Next, the operation will be described. Correlation calculation block arrangement means 5 of the video monitoring apparatus according to the seventh embodiment
When the area of the change area is small, medium, or large for the correlation block, perform appropriate processing individually for each change area, and perform these processing in parallel. This is what you do. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

FIG. 7 is an explanatory diagram showing a state in which the correlation calculation block arrangement means 5 of the seventh embodiment arranges the correlation blocks in the change areas of various areas. When the area of the change region is smaller than that of the correlation block as shown in FIG. 7, the correlation calculation block arrangement means 5 of the seventh embodiment is similar to the correlation calculation block arrangement means 5 described in the fifth embodiment. , The correlation blocks arranged in two columns are shifted so as to overlap the correlation blocks evenly arranged in the change region by half. When the area of the change region is medium with respect to the size of the correlation block, that is, when the area is appropriate,
Similar to the correlation calculation block placement means 5 described in the first embodiment, the correlation block is placed in the change area. Further, when the area of the change region is larger than the area of the correlation block, the entire image is degenerated to 1 / N as in the correlation calculation block arrangement means 5 described in the sixth embodiment, and this degenerated image is reduced. Then, the correlation block is arranged in the change area. By performing these placement processes in parallel and applying them to varying areas of various areas that are mixed in the same image, moving objects can be accurately extracted from the same image without being affected by the difference in the areas of the varying areas. I do.

As described above, according to the seventh embodiment, when changing regions having various areas are mixed in the same image, correlation block arrangement processing suitable for each changing region is performed. Therefore, there is an effect that a moving object can be accurately extracted from an image in which any change area is set.

In the above fifth to seventh embodiments, the number of correlation blocks to be arranged in the change area is given within an appropriate range in consideration of the monitoring accuracy and the like by the monitoring image. The fact that the change region is small with respect to the area of the correlation block means that the correlation block cannot be arranged in the change region up to its proper range, and in that case, the fifth embodiment described above.
Etc., the correlation blocks are arranged so as to partially overlap, and the number of blocks is increased. On the other hand, a large change area with respect to the area of the correlation block means that the correlation block must be arranged in excess of the number of blocks in the above-mentioned appropriate range, and in that case, the image of the change area is degenerated. Let As described above, even when changing regions having various areas are mixed, an appropriate number of correlation blocks can be arranged in each changing region, and a moving object can be accurately extracted.

Embodiment 8. The structure of the video monitoring apparatus according to Embodiment 8 of the present invention is the same as that of the video monitoring apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. The following description will be given using the reference symbols.

Next, the operation will be described. The intra-region motion vector integration means 8 of the video monitoring apparatus according to the eighth embodiment also uses the intra-region motion vector integration means 10.
Is to add score data of correlation calculation results obtained for each correlation block arranged in the change region and perform each integration process. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

FIG. 8 is an explanatory diagram showing the processing for obtaining the correlation score data performed by the correlation calculation executing means 7 of the eighth embodiment. Correlation calculation executing means 7 of the eighth embodiment represents each correlation block as a template (block term) shown in FIG.
The image of the size of this template is cut out from the current image input from the image input unit 1, and the same portion as this template image is past (previous) in the past image storage unit 6. Search from the image. At this time, as shown in FIG. 8, the template image is displayed on the past image ('search area' in the figure).
Search while moving with. The sum of the absolute values of the pixel value differences obtained by this search is calculated as correlation score data.

FIG. 9 is an explanatory diagram showing the shape when the correlation score data (each correlation score value) is displayed on a map. The correlation score data is expressed by the shape shown in FIG. 9A, and the vector from the point (0, 0) where the motion is zero to the minimum value (xm, ym) of the correlation score data is the motion vector. As shown in FIG. 9B, the integration of motion vectors is performed by adding the correlation score data represented by each motion vector (in FIG. 9B and FIG. The peaks are shown inverted above, but in reality the peaks are generated downwards).

As described above, according to the eighth embodiment, the intra-region motion vector integrating means 8 and the intra-predetermined time motion vector integrating means 10 correlate the correlation calculation results obtained for each correlation block. Since each integration process is performed by adding score data, there is an effect that a moving object can be accurately extracted.

Ninth Embodiment The configuration of the video monitoring apparatus according to Embodiment 9 of the present invention is the same as that of the video monitoring apparatus shown in FIG. 1, and the description thereof is omitted. The same parts as those shown in FIG. The following description will be given using the reference symbols.

Next, the operation will be described. The spatial motion uniformity determining means 9 and the temporal motion uniformity determining means 11 of this Embodiment 9 make respective determinations using correlation score data. This correlation score data is stored in the correlation calculation executing means 7 as described in the eighth embodiment.
The operation processing obtained by the above is the same as the content described in the eighth embodiment, and the description thereof will be omitted here. Hereinafter, an example of the determination using the correlation score data will be described with reference to FIG. The other operations are the same as those of the video monitoring apparatus according to the first embodiment, and the description thereof will be omitted.

For example, the peak shape in the vicinity of the minimum value (xm, ym) of the added correlation score data is examined, and the minimum value (xm,
The linear correlation score value towards ym) decreases monotonically,
Moreover, as shown in FIG. 9B, the shape of the integrated peak is sharp, and if the sharpness of the peak is equal to or larger than a predetermined threshold value, the final movement is uniform (moving in the same direction and speed. Yes). Further, the linear correlation score value from the point (0, 0) of zero movement to the minimum value (xm, ym) of the correlation score data is not monotonically decreasing, or FIG.
As shown in (c), when the shape of the integrated peak is gentle and the sharpness of this peak is smaller than a predetermined threshold value, the final movement is not uniform (not moving in the same direction and speed). To determine.

As described above, according to the ninth embodiment, the spatial motion uniformity determining means 9 and the temporal motion uniformity determining means 11 add the correlation score data by integrating the motion vectors. Since the correlation score data obtained as a result is determined, the moving object can be accurately extracted.

[0076]

As described above, according to the present invention, the image input means for inputting the monitor image, and the image change extracting means for extracting the change of the image input by the image input means and setting the change area. A motion in which a plurality of correlation blocks are arranged in a change region set by the image change extraction means, correlation calculation is performed for each correlation block, and a moving object is detected from the image input by the image input means based on the result of the correlation calculation. Since the determination means is provided, it is possible to distinguish a moving object such as an intruder or a vehicle from a background fluctuation such as a sway of trees or a sparkle of the water surface, and it is possible to accurately extract the moving object. is there.

According to the present invention, the image change extracting means includes:
Background subtraction means for removing the background image from the image input by the image input means, background updating means for updating the background image, and change area extracting means for setting a change area in the image from which the background image has been removed by the background subtracting means. Since it is provided with, there is an effect that the moving object can be accurately extracted by separating the moving object from the background fluctuation.

According to the present invention, the background updating means updates the background image based on the information when the moving object acquired from the motion determining means is detected. There is an effect that the background image can be updated and a more accurate background image can be generated.

According to the present invention, the motion determining means stores the correlation calculation block arranging means for arranging a plurality of correlation blocks in the change area set by the image change extracting means, and the image input by the image inputting means. Past image storage means, correlation calculation executing means for performing a correlation calculation for each correlation block arranged in the change area to obtain a motion vector, intra-area motion vector integration means for integrating the motion vector for each change area, and intra-area motion Using the spatial motion uniformity determination means that stores the intra-region motion vector integrated by the vector integration means and determines the motion indicated by the intra-region motion vector and the motion determination result indicated by the intra-region motion vector, Intra-time motion vector integration that integrates intra-regional motion vectors stored in the spatial motion uniformity determination means in time And the temporal motion uniformity determining means for determining the temporal motion uniformity of the temporal motion vector integrated by the constant time motion vector integrating means and extracting the moving object. There is an effect that a moving object such as a person or a vehicle can be discriminated from a background fluctuation such as a sway of a tree or a sparkle of the water surface, and the moving object can be accurately extracted.

According to the present invention, the correlation operation block arrangement means obtains the difference between the present image input by the image input means and the past image accumulated in the past image accumulating means, and uses this difference as the difference between the images. Since the correlation blocks are arranged based on this, even if a large number of correlation blocks cannot be arranged due to restrictions such as calculation processing speed, it is possible to arrange the correlation blocks efficiently and accurately extract a moving object. effective.

According to the present invention, the correlation calculation block arranging means preferentially arranges the correlation blocks from the portion of the image input by the image input means having many edge components. Thus, even when a large number of correlation blocks cannot be arranged, there is an effect that the correlation blocks can be arranged efficiently and a moving object can be accurately extracted.

According to the present invention, the correlation calculation executing means is
Since the correlation score data is obtained by the correlation calculation performed on the correlation block, and the correlation score data is used for the integration process performed by the intra-region motion vector integration means and the integration process performed by the motion vector integration means within a certain period of time, the accuracy is high. There is an effect that a moving object can be extracted.

According to the present invention, the correlation calculation executing means is
Since the correlation score data is obtained by the correlation calculation performed on the correlation block, the correlation score data is used for the determination process performed by the spatial motion uniformity determination means and the determination process performed by the temporal motion uniformity determination means. There is an effect that a moving object can be accurately extracted.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a video surveillance device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an image monitored by the image monitoring apparatus according to the first embodiment.

FIG. 3 is an explanatory diagram showing processing and determination performed by the video monitoring apparatus according to the first embodiment on a change area.

FIG. 4 is a block diagram showing a configuration of a video surveillance device according to a second embodiment of the present invention.

FIG. 5 is an explanatory diagram showing correlation blocks arranged in a change region by a correlation calculation block arrangement means according to the fifth embodiment.

FIG. 6 is an explanatory diagram showing correlation blocks arranged in a change region by a correlation calculation block arrangement means according to the sixth embodiment.

FIG. 7 is an explanatory diagram showing a state in which the correlation calculation block arrangement means according to the seventh embodiment arranges correlation blocks in changing regions having various areas.

FIG. 8 is an explanatory diagram showing a process of obtaining correlation score data performed by the correlation calculation executing means according to the eighth embodiment.

FIG. 9 is an explanatory diagram showing a shape when the correlation score data is displayed as a map.

FIG. 10 is a block diagram showing a configuration of a conventional video surveillance device.

[Explanation of symbols]

1 image input means, 2 background subtraction means, 3 background update means, 4 change area extraction means, 4a detection area setting means,
5 correlation calculation block arrangement means, 6 past image accumulation means, 7 correlation calculation execution means, 8 intra-region motion vector integration means, 9 spatial motion uniformity determination means, 10 constant time motion vector integration means, 11 temporal motion Uniformity determination means, 12 Image change extraction means, 13 Motion determination means.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/14 H04N 5/14 Z (72) Inventor Kazuhiko Washi 2-3-2 Marunouchi, Chiyoda-ku, Tokyo Sanryo Electric Co., Ltd. F-term (reference) 5B057 AA19 CD05 DA11 DA15 DB02 DB09 DC16 DC34 5C021 PA58 PA72 PA78 RA01 RA11 RB06 YC03 5C054 AA01 CA04 CC02 EA01 EG07 FC01 FC13 GA04 GB15 HA18 5C084 AA02 AA07 FF08 FF02 FF02 CC17 DD12 GG52 GG57 GG68 GG78 5L096 AA06 BA02 CA04 EA03 FA34 FA59 FA69 GA08 GA17 HA04 JA09

Claims (8)

[Claims] 1. A video monitoring device for monitoring a video and detecting a moving object, comprising: an image input means for inputting a monitoring image; and a change area set by extracting a change in the image input by the image input means. Image change extracting means, and a plurality of correlation blocks are arranged in the change area set by the image change extracting means, a correlation operation is performed for each of the correlation blocks, and the image input means inputs the result of the correlation operation. And a motion determination unit that detects a moving object from the captured image. 2. The image change extracting means includes a background difference means for removing a background image from an image input by the image input means, a background updating means for updating the background image, and a background difference means for calculating the background image by the background difference means. 2. The video monitoring apparatus according to claim 1, further comprising a change area extracting unit that sets a change area in the removed image. 3. The image monitoring apparatus according to claim 2, wherein the background updating means updates the background image based on information obtained when the moving object acquired from the motion determining means is detected. 4. The motion determination means includes a correlation calculation block arrangement means for arranging a plurality of correlation blocks in the change area set by the image change extraction means, and a past image accumulation means for accumulating the image input by the image input means. And a correlation calculation executing unit that calculates a motion vector by performing a correlation calculation on each correlation block arranged in the change region, an intra-region motion vector integrating unit that integrates the motion vector for each change region, and the intra-region motion By using a spatial motion uniformity determination unit that stores the intra-region motion vector integrated by the vector integration unit and determines the motion indicated by the intra-region motion vector, and the determination result of the motion indicated by the intra-region motion vector, Within a fixed time, the motion vector within a fixed time that integrates the intra-regional motion vectors stored in the spatial motion uniformity determining means. Tor integration means, and temporal motion uniformity determination means for determining temporal motion uniformity of the intra-time motion vector integrated by the constant time motion vector integration means and extracting a moving object. The video surveillance device according to claim 1, wherein 5. The correlation calculation block arrangement means obtains a difference between the current image input by the image input means and the past image stored in the past image storage means, and the correlation block is based on the difference between the images. The video monitoring device according to claim 4, wherein the video monitoring device is provided. 6. The video monitoring apparatus according to claim 4, wherein the correlation calculation block arranging means arranges the correlation blocks preferentially from a portion of the image input by the image input means having a large number of edge components. 7. The correlation calculation execution means obtains correlation score data by a correlation calculation performed on a correlation block, and the correlation processing is performed in the integration processing performed by the intra-region motion vector integration means and the integration processing performed by the intra-region motion vector integration means. The video monitoring device according to claim 4, wherein score data is used. 8. Correlation calculation execution means obtains correlation score data by correlation calculation performed on a correlation block, and determination processing performed by spatial motion uniformity determination means and determination processing performed by temporal motion uniformity determination means. 5. The video surveillance device according to claim 4, wherein the correlation score data is used for.