JP2000341679A - Picture recognizing device and monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the detecting precision of an object under an environment in which a background fluctuates. SOLUTION: Background pictures 110-116 in each time phase are held corresponding to the temporal fluctuation of a background. A differential picture between a present picture 120 obtained by an image pickup means and each background picture is obtained, and a picture commonly appearing in them is detected as a non-background image. In detail, differential binarized pictures 130-136 between the present picture and the background pictures are added so that a synthetic picture 140 can be obtained, and the presence or absence of a region having a data value equal to a number N of the background pictures is detected in this synthetic picture. This detection is operated by binarizing the synthetic picture by using the number N as a threshold. A person image 152 being an object to be detected commonly appearing in all the differential binarized pictures 130-136 is formed in a synthetic binarized picture 150.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognizing apparatus and a monitoring apparatus for detecting a detection target such as an intruder appearing in a monitoring image, and more particularly to detecting a detection target with high accuracy even when the background changes with time. About things.

[0002]

2. Description of the Related Art In recent years, monitoring devices using imaging devices such as industrial television cameras (ITV cameras) have been installed in various places.
It is effective in detecting intruders and abnormalities. Such a monitoring device acquires an image of a monitoring target area using an ITV camera installed on a ceiling or the like. Even at present, there are monitoring systems in which a monitoring person monitors a monitoring image to detect an abnormality or the like, but in recent years, an image recognition device is combined to automatically detect a detection target image appearing in the monitoring image. Monitoring devices have been developed to detect.

In this automatic monitoring apparatus, when a detection target is detected, for example, a report is sent to a supervisor, and the supervisor receives the report and goes to the site to take necessary measures such as checking. . In the device for automatically recognizing the detection target as described above, the burden on the supervisor is greatly reduced.

A conventional automatic monitoring device or an image recognition device used therein detects a temporal change in a monitoring image,
Further, a feature amount such as a size or a change rate of the changed image is extracted, and it is determined whether or not the change is due to the detection target. A representative method for detecting a change in the monitoring image is a difference method for obtaining a difference (difference image) between the reference image and the monitoring image. As a reference image, there is a method of using a monitoring image in which only the background is captured, and a method of using a monitoring image obtained earlier by a predetermined time than the present time as a reference image.

[0005]

However, in the above-described conventional automatic monitoring apparatus and image recognition apparatus, when the background itself fluctuates with time, the fluctuation itself may be erroneously detected as a detection target. Was. For example, in a commercial store, an illumination such as a Christmas tree is often used. In addition, there is a case where infrared communication is used for communication between various devices. When an area where such a device is arranged is monitored by infrared light, an infrared light source which blinks in response to communication is visualized. For example, when these blinking light sources sequentially emit light at different positions on the image, an image change similar to that when a detection target such as a human moves in the monitoring area may occur, and erroneous detection may occur. The probability of such an event occurring may not be very high, but the increased burden on supervisors and responders due to false detections has been recognized as a problem, because automatic monitoring devices are widely used due to their convenience. .

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and has as its object to provide a highly reliable monitoring apparatus in which erroneous detection that may occur due to steady blinking light or the like is suppressed. .

[0007]

According to the present invention, there is provided an image recognition apparatus comprising: background registration means for storing monitoring images at a plurality of timings as background images; and an image for comparing a current monitoring image with each of the background images. Means of comparison;
Determining means for determining the presence or absence of the detection target based on a comparison result of the current monitoring image with the image comparing means.

According to the present invention, background images at different timings are registered. Therefore, a background image in each time phase of the change is retained for a time-varying background such as when there is a constantly blinking light source. The image comparing means compares each of the background images with the current monitoring image. When only the background is reflected in the monitoring image, some of the background images stored in the background registration unit have a high degree of coincidence with the monitoring image. Depending on the definition of the level of the degree of coincidence, there may be one or more background images having such a high degree of coincidence. On the other hand, when the surveillance image includes something other than the background, the surveillance image basically does not match any of the background images stored in the background registration unit. Depending on the definition of the level of coincidence, the result of comparison by the image comparing means may indicate that the background registering means matches some of the background images learned in advance even if something other than the background is shown. However, the number of matching background images is basically smaller than when only the background is shown. As described above, when a plurality of comparison results obtained by comparing the surveillance image with each background image are comprehensively captured, there are cases where only the background is shown in the surveillance image and cases where other backgrounds are shown. There is a difference between The judging means detects a case where something other than the background appears in the monitoring image based on the difference. That is, erroneous determination that the detection target exists when the background itself fluctuates is suppressed.

[0009] In another aspect of the present invention, the image comparing means includes, as the comparison result of each of the background images, a comparison result image based on a difference image between the background image and the current monitoring image. And the determining means comprises:
The determination is performed based on an image that appears in common among the plurality of comparison result images.

According to the present invention, in principle, when the monitoring image shows only the background, no image appears in at least one comparison result image as a result of the difference. If the image reflects something other than the background, an image having portions overlapping each other appears in all the comparison result images. In practice, even in the former case, a common image will appear in some comparison result images, and in the latter case, a common image will not necessarily appear in all comparison result images. However, the number of comparison result images in which a common image appears is basically larger in the latter case than in the former case. Therefore, by determining an appropriate threshold value for the number, the two can be distinguished, and the presence or absence of the detection target can be determined based on the two.

[0011] In the monitoring apparatus according to the present invention, the image capturing means acquires the monitor images at a plurality of timings as background images, respectively, and registers the background images as a background image set. Background updating means for acquiring a background image, gradually updating the background image set using the new background image, image comparing means for comparing a current monitoring image with each of the background images, Determining means for determining the presence or absence of the detection target based on a comparison result of the monitoring image with the comparing means.

According to the present invention, a background image set including a plurality of background images according to the use environment of the apparatus is registered before the operation starts. Further, the background image constituting the background image set is a background image newly obtained from the monitoring image,
Updated as appropriate. In other words, changes in background fluctuations,
The background image set is updated in response to the overall change of the background due to sunshine or the like. As a result, it is possible to reliably and continuously remove the case where only the background appears in the monitoring image.

According to another aspect of the present invention, when the background updating means selects a new background image set from a plurality of background images obtained by adding the new background image to the current background image set, A criterion for reducing the degree of similarity between the respective background images to be included in the new background image set is used.

According to the present invention, background images that are not similar to each other are registered in the background image set. In the present invention, it is preferable that a background image similar to the background at that timing be stored in the background registration unit regardless of the timing at which the monitoring image is obtained. That is, regardless of the time phase of the fluctuation,
That is, it is preferable that the information is uniformly stored in the background registration unit. This requirement is achieved by ensuring that the background images registered in the background image set do not resemble any other background image. The background updating unit updates the background image set while satisfying this requirement, so that the accuracy of discriminating between the image including only the background and the image including the detection target is always kept good.

[0015]

Next, a preferred embodiment of an intruder monitoring apparatus according to the present invention will be described with reference to the drawings.

FIG. 1 is a schematic block diagram of a monitoring apparatus according to the present invention. This apparatus includes an imaging device 2 that can capture a monitoring area, and an image recognition device 4 that determines whether or not there is a shadow in a current monitoring image output from the imaging device 2. The imaging device 2 is, for example, an industrial camera or the like using a CCD imaging device, and may be an infrared camera or a visible light camera depending on the monitoring purpose. Here, the case where the imaging apparatus 2 is fixedly installed toward the monitoring target area will be described. However, the imaging apparatus 2 may be configured to swing in a predetermined angle range to scan and monitor a wide range.

The image recognition device 4 has a processing unit 6 and a storage unit 8. The processing unit 6 includes a background registration module 10, a background update module 12, an image comparison module 14, and a determination module 16. The processing unit 6 can be configured using, for example, a central processing unit (CPU: Central Processing Unit), and each of the modules 10 to 16 can be realized as a program executed on the CPU.

The storage unit 8 can use a magnetic recording device such as a hard disk device.
The processing speed can be increased by using a semiconductor memory such as an om Access Memory. The storage unit 8 stores a background image set by the background registration module 10 as described later. The background image set registered in the storage unit 8 can be gradually updated by the background update module 12.

The image recognition device 4 includes an image comparison module 1
4, the current image, which is a monitoring image currently obtained from the imaging device 2, is compared with each of the plurality of background images included in the background image set stored in the storage unit 8. The presence or absence of an intruder in the current image is determined based on the comparison result.

FIG. 2 is a processing flowchart showing a schematic processing of the present apparatus. The operation of this device is largely in initialization mode,
It consists of three operation modes, a monitoring mode and a background image update mode. The processing of this apparatus is started, for example, by turning on the power switch of the apparatus (S30). When the apparatus is started, first, a background image registration process S32 is performed by the background registration module 10 as an initialization mode. In the background image registration processing S32, images of a predetermined number N of frames are recorded in the storage unit 8 as a background image set,
Thereafter, updating of the background image set is continued until the mode shifts to the monitoring mode.

The transition from the initialization mode to the monitoring mode is as follows.
For example, it is performed by a remote operation using a communication line from a monitor who remotely monitors the apparatus, an instruction by a person who installs the apparatus, and a switch operation. When detecting the instruction (S34), the processing unit 6 starts the monitoring processing S36. This monitoring process S36 is specifically performed by the image comparison module 1
4. Processing performed by the determination module 16.

In step S34, if no instruction to shift to the monitoring process is detected, the process returns to the background image registration process S32 as described above, and updating of the background image set is continued until shifting to the monitoring mode. .

Each time the monitoring process S36 is completed once, it is determined whether or not a monitoring termination instruction has been issued (S3).
8) If instructed, for example, stop the operation of the device. On the other hand, if the monitoring end instruction has not been issued, it is subsequently determined whether or not it is time to update the background image set (S40). If it is not the update timing, the process returns to the monitoring process S36. On the other hand, if it is the update timing, the mode shifts to the background image update mode, the background image update processing S42 performed by the background update module 12 is executed, and thereafter, the process returns to the monitoring processing S36.

Subsequently, each of the background image registration processing S32, the monitoring processing S36, and the background image update processing S42 will be described in detail. FIG. 3 is a processing flowchart of the background image registration processing S32. In the background image registration process S32, first, the current image is obtained from the imaging device 2 (S50). Until the number of background images already registered in the storage unit 8 reaches N frames defining the background image set (S5
2) The current images sequentially obtained from the imaging device 2 are added to the background image set in the storage unit 8 as background images (S5).
4). In this way, the background images of N frames are stored in the storage unit 8, and the background image set is completed.

In the state where the background image set is completed (S52), difference images between the current image obtained in the process S52 and each background image included in the registered background image set are calculated. A difference binarized image obtained by binarizing the difference image with an appropriate threshold is calculated (S56).
The threshold for binarization can be set by a monitor, for example, as a parameter. The threshold value used here can be determined, for example, based on the average value of each pixel value of the difference image. Among the N difference binary images generated corresponding to the N frame background images, the one having the smallest difference is selected, and the corresponding registered background image and the current image are replaced. That is, the background image set is updated by registering the current image as a new background image instead of the corresponding background image (S58). Thereafter, the process returns to the main routine shown in FIG. Here, the minimum difference can be determined based on, for example, a criterion that the sum of squares of each pixel value (that is, “0” or “1”) of the difference binary image is the minimum. Incidentally, replacing the background image with the smallest difference means replacing the background image determined to be most similar to the current image. Here, from the viewpoint of easiness of the sum of squares calculation, the background image similar to the current image is selected based on the difference binary image, but similar to the current image based on the simple difference image. A process for selecting a background image can also be performed.

The processing returned from the background image registration processing S32 shifts to the judgment processing S34. If the shift to the monitoring mode is not determined in the process S34, the background image registration process S32 is executed again. In this case, since the background image set has already been completed, the process S52 is performed in the process S5.
6, the process branches to S58, and only the background image set is updated with the current image acquired in process S50.

As described above, in the background image registration processing S32, a background image that can fluctuate over time can be recorded at various time phases. In the case where the fluctuation is stationary, that is, in a state where the pattern that the background can take is within a limited range, the background pattern set defined by N is sufficient to cover the background pattern sufficiently. It is possible to As described above, the present apparatus learns a change in the background in a situation where there is a light source that constantly blinks in the background, such as an environment where a device for performing illumination or infrared communication of a store is arranged. By the way, as will be apparent from the following description, the present device can be used as long as the background pattern fluctuates within a limited range, and these patterns need not necessarily appear periodically.

FIG. 4 is a flowchart of the monitoring process S36. A specific example of this processing will be described later with reference to another drawing. In the monitoring process S36, the current image is acquired from the imaging device 2 (S70), and a difference binary image is calculated as a comparison result image between the current image and each of the background images included in the registered background image set. (S72).

By calculating the difference between the background image and the background image in the current image, there is a high possibility that the data value of that portion will be "0". Here, the reason that the possibility does not become 100% is that the apparatus can be used in a situation where the background of the present apparatus constantly fluctuates with time as described above. That is, in the comparison between the current image and the background image of a time phase different from the background time phase of the current image, the data value of the background portion is not necessarily “0” even in the background portion. That is, by setting an appropriate binarization threshold,
In some cases, the background portion and the non-background portion of the current image cannot be distinguished.

However, the background image of the time phase corresponding to the current image is basically learned in the background image set. Therefore, the background portion of the difference image between the corresponding background image and the current image is basically “0” or a sufficiently small value. In this case, the binarized pixel value of the background portion is set to “0” and the binarized pixel value of the non-background portion is set to “1” by an appropriate binarization threshold value.
It can be.

Next, the N difference binary images generated corresponding to the N frame background images are added to each other to generate a composite image (S74). Each pixel value of this composite image can take an integer in the range of 0 to N. For example, the pixel value takes the maximum value N when all the N difference binary images have data “1”. Now, each of the difference binarized images obtained corresponding to each background image has a region where the binarized pixel value is “1” in a non-background portion of the current image. Incidentally, the case where there is no region where the differential binarized pixel value is “1” in a portion corresponding to the non-background portion of the current image means that the region corresponding to the non-background portion of the current image includes the current image and the background image. This is a case where the pixel values are equal to each other, and such a case is generally unlikely to occur. Therefore, the data value of the non-background portion in the composite image is basically “N”.
Becomes

On the other hand, as for the background portion of the current image, as described above, the data value of the corresponding portion of any of the differential binarized images is "0", and the data value of the composite image of the corresponding portion is "N". ".

Therefore, the binarization process S7 for the composite image
In 6, the threshold value can be set so that the data value of the area corresponding to the background portion of the current image is set to “0”. In the above example, the threshold value is set to N, and is set to “1” when the data value of the composite image is N, and is set to “0” when the data value is less than N. If the current image is the background image itself in any of the time phases, an image having a pixel value of “0” over the entire image is obtained by the binarization processing S76. on the other hand,
When a non-background portion is included in the current image, an image having an area with a pixel value “1” corresponding to the non-background portion is obtained.

The perspective transformation S78 is a process for converting the size of an image appearing on the image in consideration of the perspective of the image.
This is a process conventionally performed. As a result, if the size of the image appearing in the binarized image after the synthesis is estimated to be a detection target, for example, a person (S80), the monitoring center is notified, for example, via a communication line that the person is an intruder. (S82), and thereafter, the process returns to the main process shown in FIG. On the other hand, when it is determined that there is no detection target in the binarized image after synthesis, the process returns to the main process without performing notification.

FIG. 5 is a flowchart of the background image updating process S42. This processing is started when it is determined in the main processing that it is the update timing. For example, the update timing can be set after a lapse of a predetermined time. In the process S42, first, the current image is obtained (S9).
0). The update is performed by registering the acquired current image instead of any background image in the registered background image set. Here, two criteria are used for selecting which background image is selected. The first criterion is a criterion of selecting a background image registered in the background image set, if a predetermined time has elapsed, that is, if there is a sufficiently old one (S92). If there are a plurality of items that have passed the predetermined time, the oldest one is selected. If there is a background image that meets this criterion, the background image is deleted from the background image set, and the current image is newly registered (S94).

The second criterion is a criterion for selecting, from among the background images registered in the background image set, the one most similar to the current image. For example, a difference binarized image between the current image and each background image is obtained (S96), and a background image that is most similar to the current image can be defined, for example, under the condition that the sum of squares of each pixel value is minimized. Yes (S98).

According to the second criterion, the background images included in the background image set are maintained in a mutually dissimilar relationship. Therefore, it is expected that the background image set evenly includes the background image of each time phase of the background fluctuation.

On the other hand, according to the second criterion, for some reason,
Once an irregular image different from the background is registered,
The image is less likely to be removed from the background image set. First
According to the standard, the inconvenience of the second standard can be corrected.

Next, a specific example of the monitoring process S36 of the present apparatus will be described. FIG. 6 is a schematic diagram illustrating the principle of the above-described device. In the following example, the number N of images constituting the background image set is set to 4. In the storage unit 8, a background image set including background images 110 to 116 at different time phases is registered. The current image 120 is input from the imaging device 2 to the image recognition device 4. Here, for convenience of illustration, the background images 110 to 116 and the current image 120 are respectively binarized images, and a black portion represents a data “1” portion and a white portion represents a data “0” portion. For example, each black circular area in the background image represents a separate flashing light source image. Since they are blinking, they basically do not appear in common in background images of different phases. The current image represents an image in which a person image 122 is reflected in a background image 114, and a black circular image 124 is an image of a light source appearing in the background image 114.

The current image 120 and each of the background images 110 to 116
By performing the difference binarization process with each of them, each background image 11
0 to 116 corresponding to each of the difference binarized images 130
To 136 are obtained. By adding these, a composite image 140 is obtained. Data values are described in each image of the composite image 140. For example, the data value of the area corresponding to the image 124 is “3”. Also, the data value of the person image 122 is basically “4”, but the data value of the area corresponding to the light source (light source appearing in the background image 116) overlapping the human image 122 is “3”. ing. The result of performing the binarization process S76 on the composite image 140 with the threshold value 4 is the composite binarized image 150. In this way, the non-zero area 152 is formed in the area corresponding to the human image in the binarized image 150 after synthesis.

As described above, according to the present apparatus, even in a situation where the background image changes with time, it is possible to detect the presence of a person to be detected by eliminating the influence of the background image.

FIG. 6 shows a process for one current image at a certain time. FIG. 7 is a schematic diagram showing a binarized image after synthesis according to the temporal progress of the current image. In the figure, the time axis is, for example, from top to bottom, and a series 170 of images on the left side is a series of current images as time passes. The series 180 of images on the right is a series of binarized images after synthesis corresponding to each of these current images. The current image at the same time and the binarized image after synthesis are shown side by side. Human images 171 to 173 moving from the right to the left of the screen appear on the current image at times t5 to t7. Only in this case in the present apparatus, images 181 to 183 corresponding to the moving human image are generated in the binarized image after synthesis. That is, conventionally, for example, it was not possible to remove the influence of the light source that emits light at different positions and at different timings. Therefore, for example, an image sequence such as the current image at times t1 to t3 is detected as a moving object, In some cases, such an erroneous recognition is suppressed in the present apparatus.

In the above-described processing, the threshold for generating a binarized image after synthesis is set to be the same as the number N of constituent images of the background image set. However, the threshold can be set lower than that. By lowering the threshold value, for example, even if a situation occurs in which the detection target image coincides with one of the background images, detection omission hardly occurs. That is, the detection sensitivity is improved. On the other hand, there is a possibility that the possibility of erroneously detecting an image other than the detection target, for example, a pseudo image generated by a change in the background may increase. The threshold can be selected by such a trade-off between the detection sensitivity and the detection accuracy.

[0044]

According to the monitoring device and the image recognition device of the present invention, the object detection process is performed based on a plurality of comparison result images obtained by comparing a plurality of background images and the current image. In addition, even in a situation where the background fluctuates steadily with time, an effect is obtained that the presence other than the background can be identified and the detection target can be accurately detected.

Also, by gradually updating the background image set, for example, even when the background image changes over a long period of time, it is possible to obtain an effect that the influence of the background can be satisfactorily eliminated and the detection target can be identified. Can be In addition, at the time of the update, by maintaining the dissimilarity between the background images, the background image set reflects the time change of the background well, the influence of the background is removed well, and the detection target can be detected. The identification accuracy is improved.

[Brief description of the drawings]

FIG. 1 is a schematic block configuration diagram of a monitoring device according to the present invention.

FIG. 2 is a process flowchart showing a schematic process of the present apparatus.

FIG. 3 is a process flowchart of a background image registration process.

FIG. 4 is a process flowchart of a monitoring process.

FIG. 5 is a processing flowchart of a background image updating process.

FIG. 6 is a schematic diagram illustrating the principle of a determination module 16 of the present apparatus.

FIG. 7 is a schematic diagram showing a binarized image after synthesis according to a temporal progress of a current image.

[Explanation of symbols]

2 imaging device, 4 image recognition device, 6 processing unit, 8 storage unit, 10 background registration module, 12 background update module, 14 image comparison module, 16 judgment module.

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Hiroshi Masuda 6-11-23 Shimorenjaku, Mitaka City, Tokyo Inside Secom Co., Ltd. (72) Inventor Toshio Hoshi 1410 Inada, Hitachinaka City, Ibaraki Prefecture Hitachi, Ltd.Digital Corporation Media Products Division (72) Inventor Yoshiki Kobayashi 7-1-1, Omika-cho, Hitachi City, Ibaraki Prefecture F-term in Hitachi Research Laboratory, Hitachi, Ltd. F-term (reference) 5B057 AA19 DA02 DB02 DC32 DC39 5C054 AA01 AA05 CA04 CC02 EA05 ED17 EH01 FC11 FE12 FE25 FF03 GB14 GD05 HA18 5C084 AA02 AA07 BB04 BB40 CC19 DD11 GG43 GG51 GG56 GG57 GG78 5L096 BA02 GA08 HA07 JA03 KA03 KA15 9A001 HH20 HH23

Claims (4)

[Claims] 1. An image recognition device for detecting a detection target appearing in a monitoring image, a background registration unit for storing the monitoring images at a plurality of timings as background images, respectively, and a current monitoring image for each of the background images. An image recognition apparatus, comprising: an image comparison unit that performs comparison; and a determination unit that determines the presence or absence of the detection target based on a comparison result of the current monitoring image with the image comparison unit. 2. The image recognition device according to claim 1, wherein the image comparison unit determines, as the comparison result of each of the background images, a comparison result based on a difference image between the background image and the current monitoring image. An image recognition apparatus, wherein an image is generated, and the determination unit performs the determination based on an image that appears in common among the plurality of comparison result images. 3. A monitoring apparatus comprising: an imaging unit for acquiring a monitoring image; and an image recognizing unit for detecting a detection target appearing in the monitoring image, wherein the imaging unit sets a background of the monitoring image at a plurality of timings. A background registration unit that acquires an image and registers it as a background image set; and acquires a new background image from a monitoring image output by the imaging unit, and gradually updates the background image set using the new background image. Background updating means, image comparing means for comparing a current monitoring image with each of the background images, and determining means for determining the presence or absence of the detection target based on a comparison result of the current monitoring image with the image comparing means. A monitoring device, comprising: 4. The monitoring device according to claim 3, wherein the background updating unit is configured to select a new background image set from a plurality of background images obtained by adding the new background image to the current background image set. A criterion for reducing the degree of similarity between the respective background images to be included in the new background image set.