JP5142416B2 - Object detection device - Google Patents

Description

  The present invention relates to a technology for detecting a voyeur camera or a forgotten item installed in a monitoring area.

  Damage to the installation of voyeur cameras at ATMs has occurred. In ATM, since a user inputs a personal identification number, such an act must be thoroughly eliminated. There are also cases where forgotten items are found in ATMs, lobby, counters, etc. If you leave a forgotten item, the possibility of returning to the owner's hand over time decreases. It can also be taken away by a malicious person. Therefore, early detection of forgotten items is important.

  As a method of automatically detecting these illegal installations and forgotten items, there is a method of analyzing a captured image. A camera is installed in the monitoring area, and an illegal installation or forgotten object is detected by taking a difference between a captured image acquired sequentially and a reference image.

  In Patent Document 1, a background image of a monitoring area where a monitoring target does not exist is acquired in advance, and a stationary object is monitored by acquiring a difference between the background image and an image at an arbitrary time.

JP-A-6-105312

  As described above, a method has been proposed in which a difference image between a background image and a current image is acquired and the left object is detected by analyzing the difference image. However, in these methods, the accuracy of the background image affects the accuracy of object detection.

  As in Patent Document 1, the method of preparing a background image in advance has a problem that it is vulnerable to environmental fluctuations. When the environment changes, such as changes in lighting, the background image in the monitoring area will change. However, if the background image prepared in advance is used fixedly, the change in the environment will be May be detected.

  On the other hand, there is a method of periodically updating the background image. As a result, the background image is also updated following the environmental change. However, it is difficult to update the background image in places where there are many people such as bank ATMs. Although there is a method of obtaining an average image of background images acquired from a plurality of frame images, there is a problem in accuracy in a place where people go in and out.

  SUMMARY OF THE INVENTION In view of the above problems, the present invention has an object to provide a technique for accurately detecting an abandoned object or the like in a monitoring area where image changes frequently occur, such as an area where many people come and go. .

In order to solve the above-described problem, the invention described in claim 1 is an image pickup apparatus that picks up an image of a monitoring area, and an update unit that acquires a background image from an image output from the image pickup apparatus and periodically updates the background image. A person detection unit that detects a person from difference information between the current image output from the imaging device and the latest background image; and a difference between the current image output from the imaging device and the latest background image A means for removing isolated points from an image, and a logical image generated by obtaining a plurality of difference images from which isolated points have been removed at intervals longer than the frame interval and taking the logical product of the plurality of differential images and means for extracting the object region, wherein the updating means, while the person is detected by the person detecting means in the image output from the imaging device, it does not update the background image And features. Invention of Claim 2 is the object detection apparatus of Claim 1, Comprising: The means which determines the presence or absence of an object by comparing the hue of a background color image and the present color image about the said object area | region, Is further provided.

  The object detection apparatus of the present invention acquires a plurality of difference images from which isolated points are removed at intervals longer than the frame interval, and obtains an object region from a logical product image generated by calculating a logical product of the plurality of difference images. To extract. The noise component in the two-dimensional direction is removed by the isolated point removal, and the noise component in the time direction is removed by the AND operation. Further, since the logical product images of frames with a time interval are used, the influence of luminance unevenness can be eliminated.

It is a figure which shows the object detection apparatus installed in ATM. It is a figure which shows the imaging range of the camera installed in the position facing ATM. It is a block diagram of an object detection apparatus. It is a main flowchart of an image analysis process. It is an image figure of the calculation process of a background image. It is a figure which shows the update timing of a background image. It is a flowchart of an object detection process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram of a bank ATM room in which an object detection device 1 according to the present embodiment is installed. The ATM 5 is surrounded by a partition 6 as shown in FIGS. Thus, by enclosing the ATM 5 with the partition 6, when the user 7 operates the ATM 5, the password input operation or the like is not visible from the outside.

  And the object detection apparatus 1 of this Embodiment is comprised from the camera 2 and the control apparatus 3, and the camera 2 is arrange | positioned in the position facing ATM5 as shown in FIG.1 and FIG.2. The position of the camera 2 is adjusted so that the ATM 5 is imaged from the front. In other words, when the user 7 is operating the ATM 5, the camera 2 takes an image from the back side of the user 7, and the position and angle are adjusted so that the user 7 and the AMT 5 are within the imaging range. Yes.

  The installation position and installation angle of the camera 2 shown in FIGS. 1 and 2 are examples. Since this camera 2 only needs to put one booth of this ATM 5 in the imaging range, for example, even if the camera 2 is installed directly above the ATM 5 and the camera 2 images the ATM 5 from above, Good.

  FIG. 3 is a block diagram of the control device 3. The control device 3 is installed in a place different from the room in which the ATM 5 is installed. The control device 3 and the camera 2 are connected by a video cable.

  As shown in the figure, the control device 3 includes an image processing unit 31, a buffer memory 32, a background image update unit 33, a person detection unit 34, an object detection unit 35, a background image memory 36, a notification unit 37, and a work memory 38. ing. The processing units 31, 33, 34, and 35 related to the image analysis may be configured by hardware, or may be realized by software operating on the CPU. Further, a part of the processing unit may be configured by hardware.

  The image processing unit 31 inputs an image signal output from the camera 2 via a video cable. The image processing unit 31 subjects the analog image signal output from the camera 2 to image processing, and then performs A / D conversion. Further, image processing is performed on the digital image signal to generate captured image data 41, and the captured image data 41 is stored in the buffer memory 32. The captured image data 41 is color image data. For example, color image data having RGB color components. The captured luminance image data 42 is data generated from the luminance component of the captured image data 41. For example, when the captured image data 41 is RGB image data, the RGB image data is converted into YCrCb image data, and a luminance component (Y component) is acquired.

  The background image update unit 33 generates background image data 43 and background luminance image data 44 from the captured image data 41 and the captured luminance image data 42 stored in the buffer memory 32 and stores them in the background image memory 36. The background image data 43 is color image data. The background image update unit 33 periodically creates background image data 43 and background luminance image data 44 (collectively referred to as background images) at a predetermined time interval, and a background image memory. The background image stored in 36 is updated.

  However, as will be described in detail later, the background image update unit 33 does not update the background image data 43 and the background luminance image data 44 while the control device 3 detects a person in the ATM room. Even when the person is no longer detected, the background image is not updated during the object detection process.

  The person detection unit 34 detects a person within the imaging range of the camera 2 from the captured image data 41 stored in the buffer memory 32 and the background image data 43 stored in the background image memory 36.

  The human detection algorithm is not particularly limited. In the present embodiment, the person detection unit 34 generates difference image data between the captured luminance image data 42 and the background luminance image data 44, and detects a person based on the distribution state of the difference image data. That is, the variance is calculated for the luminance value of each pixel of the difference image data, and it is determined that a person exists when the variance exceeds a predetermined threshold. Alternatively, difference image data (for a certain color component) between the captured image data 41 and the background image data 43 may be generated, and a person may be detected from the distribution state of the difference image data.

  In the present embodiment, as described above, the camera 2 takes a large image of one ATM 5 booth. That is, the entire ATM room (if a plurality of ATMs are arranged, the whole) is not imaged, but the range in which one ATM 5 is contained is set as the imaging range. Therefore, when the user 7 stands in front of the ATM 5, the difference image data is greatly dispersed, so that a person can be detected with high accuracy.

  The object detection unit 35 receives the captured image data 41 and the captured brightness image data 42 stored in the buffer memory 32 and the background image data 43 and the background brightness image data 44 stored in the background image memory 36 in the ATM room. Detects illegally installed voyeur cameras or misplaced objects. The object detection algorithm of the object detection unit 35 will be described in detail later.

  The notification unit 37 is connected to the center 8 via a network, and transmits object detection information to the center 8 when the object detection unit 35 detects an object. Or the notification part 37 transmits object detection information in the bank store adjacent to bank ATM. Thereby, it is possible to collect illegally installed voyeur cameras and forgotten items at an early stage.

  A background update process, a person update process, and an object detection process performed by the object detection apparatus 1 configured as described above will be described with reference to FIGS. FIG. 4 is a main flowchart of image analysis processing executed in the object detection apparatus 1.

  First, the object detection apparatus 1 inputs a captured image from the camera 2 (step S11). For example, the camera 2 outputs captured images at a frame rate such as 30 frames / second. This captured image is a color image such as RGB. The image processing unit 31 performs image processing on the input image signal and stores the captured image data 41 in the buffer memory 32. The buffer memory 32 stores the captured image data 41 for the past predetermined time (the predetermined number of frames).

  Next, the person detection unit 34 performs person detection processing from the captured luminance image data 42 stored in the buffer memory 32 and the background luminance image data 44 stored in the background image memory 36 (step S12). As described above, in the present embodiment, the person detection unit 34 determines the presence or absence of a person by calculating the variance of the difference image between the captured luminance image data 42 and the background luminance image data 44. Alternatively, the variance of the difference image between the captured image data 41 and the background image data 43 is calculated to detect the presence or absence of a person.

  If no person is detected by the person detection unit 34 (NO in step S13), it is determined whether it is the update timing of the background image (step S14). If it is not yet the update timing of the background image, the process returns to step S11. If the background image update timing has elapsed, the background image update unit 33 performs update processing of the background image data 43 and the background luminance image data 44 (step S15).

  FIG. 5 is a processing image diagram of background image update processing by the background image update unit 33. In the present embodiment, the background image update unit 33 generates background luminance image data 44 using the captured luminance image data 42 of a plurality of frames.

  Specifically, the luminance values (density) of each pixel position of the plurality of imaged luminance image data 42, 42,... Are sorted, and background luminance image data 44 having the intermediate value as the luminance value of the pixel is generated. It is. That is, with respect to the captured luminance image data 42 of a plurality of frames, attention is paid to a certain pixel position (coordinate position), and the luminance values at the pixel positions are rearranged in ascending order. Then, a luminance value taking an intermediate value among the sorted luminance values is adopted as the background luminance value for the pixel.

  The background luminance image data 44 created in this way is stored in the background image memory 36. However, instead of calculating using all the captured image data 41 output from the camera 2 (for example, the captured image data 41 of 30 frames / second), the captured image data 41 is extracted at a predetermined frame interval. Luminance image data 42 is generated, and background luminance image data 44 is generated from the thinned captured luminance image data 42. In this way, the processing speed is increased. Further, the number of frames of the captured image data 41 used for generating the background luminance image data 44 may use all the frames stored in the buffer memory 32, or may use some of the frames. Also good.

  Note that the method of generating the background luminance image data 44 is not limited to the above method. Since it is determined in step S13 that no person is detected, the latest captured luminance image data 42 may be updated as the background luminance image data 44 as it is.

  As for the background image data 43, the latest captured image data 41 may be employed as it is. In this case, the latest captured image data 41 is stored as background image data 43 in the background image memory 36. Alternatively, the background image data 43 may be created by obtaining an average value of the captured image data 41 of a plurality of past frames. Further, the background image data 43 may be created by performing the same process as the process for creating the captured luminance image data 44. That is, not only the luminance component but also the color component may be obtained by obtaining an intermediate value of pixels of a plurality of frames, and the background image data 43 may be created using the intermediate value.

  If it is determined in step S13 that a person is detected, the person detection unit 34 repeats the determination until no person is detected (step S16). Then, when no person is detected, the process proceeds to the object detection process (step S17).

  As described above, the object detection apparatus 1 according to the present embodiment updates the background image data 43 and the background luminance image data 44 only when no person is detected, and the background image data 43 while the person is detected. The background luminance image data 44 is not updated, and the object detection process is not started. FIG. 6 is a diagram showing person detection and background image update timing. As shown in the figure, the background image is updated at predetermined time intervals before the person is detected. During the person detection, the background image data 43 and the background luminance image data 44 are not updated even when the background image update timing elapses. That is, the background image data 43 and the background luminance image data 44 before the person detection are saved as the latest. In the person detection process, the latest background image in which no person is detected is used. Thereafter, the object detection process (step S17) is executed, but the background image is not updated during this time.

  Next, the flow of object detection processing executed by the object detection unit 35 will be described with reference to FIG. First, the object detection unit 35 creates a luminance edge image for background (step S21). For example, an 8-adjacent Laplacian filter is applied to the background luminance image data 44 stored in the background image memory 36 to create a luminance edge image.

  Next, the luminance edge image created in step S21 is binarized (step S22). That is, the luminance value of each pixel of the luminance edge image is compared with a predetermined threshold value, and “0” is assigned to a pixel having a luminance value equal to or lower than the threshold value, and “1” is assigned to a pixel having a luminance value exceeding the threshold value.

  Next, a brightness edge image is created for the latest captured brightness image data 42 stored in the buffer memory 32 (step S23). For example, an 8-adjacent Laplacian filter is applied to the captured luminance image data 42 to create a luminance edge image.

  Next, the luminance edge image created in step S23 is also binarized (step S24). The contents of the process are the same as in step S22.

  Then, a difference image between the background binarized image created in step S22 and the current binarized image created in step S24 is created (step S25). Here, the background luminance image data 44 has been updated before the person is detected. Therefore, when a voyeur camera or a forgotten object that did not exist before the person detection is left in the ATM, these objects are included in this difference image.

  Next, the object detection unit 35 performs isolated point removal from the difference image (step S26). Thereby, the noise component is removed from the difference image. In step S <b> 26, the created difference image from which isolated points have been removed is stored in the work memory 38. The work memory 38 stores a differential image from which isolated points for a past predetermined frame have been removed.

  Next, the object detection unit 35 creates a logical product image of the difference images from which the isolated points have been removed for the past three frames stored in the work memory 38 (step S27). As described above, since the difference image is a difference image of the binarized image, the difference image is also a binarized image (even if the difference is a negative sign, it is converted to a positive sign. ). Accordingly, in step S27, for the pixel at each pixel position, “1” is assigned as the luminance value of the logical product image only for the pixels whose luminance values are “1” in all three frames.

  In the present embodiment, the three-frame difference image used to create the logical product image is a difference image having a predetermined time interval. That is, since the processing from step S21 to step S26 is performed on the latest captured luminance image data 42 stored in the buffer memory 32, the difference image stored in the work memory in step S26 is also captured image data. 41 is stored at the same frame rate. However, when the object detection unit 35 creates a logical product image, it does not process the difference image of successive frames, but creates a logical product image for the difference image thinned out of the frames. is there. In this way, the logical product image is created from the difference images having time intervals, so that the influence of luminance unevenness and the like can be eliminated, and only the difference component that exists regularly can be extracted.

  Further, as described above, the object detection unit 35 creates a logical product image after performing isolated point removal processing. Therefore, the noise component in the two-dimensional region is removed by the isolated point removal process, and the noise component in the time direction is removed by the logical product image creation process.

  Next, the object detection unit 35 performs segmentation of the logical product image and extracts an object region (step S28). That is, in the logical product image, an area where pixels having a luminance value of “1” continue is extracted. For example, if a round object is left in the ATM, a pixel set having a luminance value “1” is extracted as a circular area in the logical product image. In this way, the region of the object that did not exist before the person detection is detected.

  Further, the object detection unit 35 compares the hues of the captured image data 41 and the background image data 43 for the object region extracted in step S28 (step S29). That is, the hues of the background and the current image are compared using the captured image data 41 and the background image data 43 which are color image data. If a change in hue is detected above a certain level, it is determined that an object exists in the object region obtained in step S28 (step S30). For example, if the color image is an RGB image, the difference between pixel values of segmented regions for red (R) and blue (B) pixel components is obtained, and the object is detected when the pixel value difference is greater than or equal to a threshold value. Judge that it exists. Alternatively, if the color image is a YCrCb image, the comparison may be made using Cr and Cb components.

  As described above, after the object region is detected using the luminance component in steps S21 to S28, the presence of the object is determined by comparing the color components of the object region. Thereby, it can prevent detecting an object accidentally by the change of illumination, etc. In other words, in the determination of only the luminance component, there is a possibility that the object region may be detected due to a change in luminance such as lighting, sunlight, etc., but the object is detected more accurately by comparing the color component for this region. I am doing so.

  If an object is detected (YES in step S30), the notification unit 37 notifies the center 8 of object detection information. Alternatively, the notification unit 37 notifies the object detection information in the adjacent bank store. Thereby, for example, when a voyeur camera is installed, it is possible to remove the voyeur camera immediately. It is also possible to find forgotten items at an early stage.

  Again, it returns to the flowchart of FIG. When the object detection process (step S17) ends, the object detection apparatus 1 returns to step S11 again and starts the process. If no person is detected, the background image is updated. As described above, during the person detection and the object detection process, the background image is not updated, and the background image is updated after the object detection process is completed (see FIG. 6). Thus, a background image is not created from an image including a person or an object, and an appropriate background image can be acquired and updated.

  As described above, by using the object detection apparatus 1 according to the present embodiment, it is possible to accurately detect an object that is illegally installed or a forgotten object in a monitoring area such as an ATM. In particular, when a person is detected, the update of the background image is interrupted and the object detection process is not performed. The object detection process is started when no person is detected. Also, the background image is not updated during the object detection process. As a result, an appropriate background image can be acquired in the absence of a person or object.

  Further, since the object detection process is executed after the person is detected, the efficiency is high. That is, when an object is left in the monitoring area, a person who left the object must have visited before that. Therefore, while the person is not detected, only the background image is updated and the object detection process is not performed. Thereby, the execution of the object detection process can be kept to the minimum necessary.

  In the above embodiment, object detection is performed using a single ATM 5 as an imaging range. However, this is only an example, and for example, the object detection processing may be performed by putting two ATMs 5 in the imaging range. In this case, for example, the captured image data may be divided into two at the center of the screen, and the same processing as in the above embodiment may be executed for each region. Alternatively, four ATMs 5 may be put in the imaging range, and the image may be divided into four to execute similar processing.

DESCRIPTION OF SYMBOLS 1 Object detection apparatus 2 Camera 3 Control apparatus 5 ATM
6 Partition 7 User

Claims (2)

An imaging device for imaging the monitoring area;
An update unit that acquires a background image from an image output from the imaging device and periodically updates the background image;
A person detecting means for detecting a person from difference information between the current image output from the imaging device and the latest background image;
Means for removing isolated points from the difference image between the current image and the latest background image output from the imaging device;
Means for acquiring a plurality of differential images from which isolated points have been removed at intervals longer than the frame interval and extracting an object region from a logical product image generated by taking a logical product of the plurality of differential images;
Equipped with a,
The update means does not update a background image while a person is detected by the person detection means in an image output from the imaging device. Means for determining the presence or absence of an object by comparing the hue of a background color image and a current color image for the object region;
The object detection device according to claim 1, further comprising:

Images