JP7355553B2 - Image processing device and image processing program - Google Patents

Description

The present invention relates to an image processing device and an image processing program.

Conventionally, image processing devices are known that detect moving objects from captured images by comparing captured images captured by a camera or other imaging unit with background images prepared in advance based on past captured images. . In such an image processing apparatus, the background image is updated as needed in order to respond to changes in the brightness of the imaging area (monitoring space), such as the movement of the sun or the lighting of lights.

In order to make it easier for intruders to enter the building by feigning changes in the filming environment, there are schemes to place shields in advance so that the entrance route cannot be seen from the filming department, and employees to arbitrarily place luggage in front of the filming department. There were cases where the background image was updated with the image taken of the obstruction. In other words, the background image may include a shielding object, making it difficult to detect (monitor) a moving intruder.

In order to detect such situations where monitoring is difficult, conventional methods store a reference image that cannot be updated automatically and do not include moving objects, and then compare the reference image with the target image to be determined, thereby determining the current monitoring space. It is determined whether or not the situation is difficult to monitor (for example, Patent Document 1).

Patent No. 3986193

When determining whether the monitoring space is in a difficult-to-monitor situation by comparing the reference image and the target image, if an environmental change in the monitoring space occurs that does not cause a problem in the monitoring situation, it can be determined that there is a problem in the monitoring situation in the first place. There were cases where it was incorrectly determined that there was a problem with the monitoring status even though there was no problem. Environmental changes in the monitoring space that do not cause problems in the monitoring situation include, for example, small objects that are within the monitoring space (that is, included in the reference image) and that do not affect the monitoring situation even if moved. (For example, moving stationery, small cardboard boxes, etc.)

An object of the present invention is to provide an image processing device that determines whether a monitoring space is in a difficult-to-monitor situation by comparing a reference image and a target image. The purpose of the present invention is to enable highly accurate determination even in cases where

The present invention is an image processing device that monitors whether or not a moving object exists in a monitoring space based on a photographed image of the monitoring space, wherein the mobile object is present in the monitoring space in the photographed image. Reliability setting means for setting reliability for each image area of the reference image according to similarity between a reference image taken at a time when the reference image is not used and a comparison image based on the taken image other than the reference image; , the degree of difference between the reference image and the target image, which is the photographed image to be determined, is calculated for each image region, and based on the degree of difference for each image region and the weight according to the reliability, the monitoring space is monitoring status determining means for determining whether the presence of a moving object is in a situation where it is possible to monitor the presence of the moving object, and the image area having a higher reliability is given a larger weight than the image area having a lower reliability. This is an image processing device.

Preferably, the monitoring status determining means determines the degree of difference based on a comparison of edges extracted from the reference image and the target image, respectively.

Preferably, the reliability setting means sets the reliability in accordance with feature amounts of edges extracted from each of the image regions of the reference image, in addition to the determination of similarity.

Preferably, the monitoring status determining means calculates an evaluation value indicating a degree of mismatch between the reference image and the target image from the weight for each image region and the degree of difference, and based on the evaluation value, the The present invention is characterized in that it is determined whether the monitoring space is in a situation where the presence of the mobile object can be monitored.

Preferably, the reliability setting means cumulatively updates the reliability for each image region according to the similarity obtained by sequentially comparing the reference image and the plurality of comparison images, The present invention is characterized in that the higher the current reliability of the image area, the smaller the amount of decrease in the reliability when the reference image and the comparison image are dissimilar.

Preferably, the monitoring status determining means further determines the weight by determining whether the weight is homogeneous, which is an area where no edge exists in the reference image or an area where the number of pixels corresponding to the edge is less than a predetermined number, among the edges extracted from the reference image. It is characterized in that it is determined whether the monitoring space is in a situation where the presence of the moving object can be monitored based on the weight that is greater in the surrounding neighboring region than in the surrounding neighboring region of the region and the degree of difference. shall be.

Preferably, the monitoring status determining means determines that the monitoring space is in a situation where the presence of the moving body cannot be monitored if the average reliability of the plurality of image areas is less than a predetermined value. .

Preferably, the reliability setting means cumulatively updates the reliability for each image region according to the similarity obtained by sequentially comparing the reference image and the plurality of comparison images, The monitoring status determining means determines that in the cumulative update of the reliability, if the average of the reliability of the plurality of image regions is on a downward trend, the monitoring space is in a situation where the presence of the moving object cannot be monitored. It is characterized by determining that.

The present invention also provides an image processing device that causes a computer to monitor whether or not a moving object exists in a monitoring space based on a photographed image of the monitoring space, wherein the mobile object is detected in the photographed image. Reliability is set for each image area of the reference image depending on the similarity between a reference image taken when the reference image does not exist in the monitoring space and a comparison image based on the taken image other than the reference image. Reliability setting means, determining the degree of difference for each image region between the reference image and the target image which is the photographed image to be determined, and based on the degree of difference for each image region and a weight according to the reliability; monitoring status determining means for determining whether the monitoring space is in a situation where the presence of the moving body can be monitored; This is an image processing program characterized by increasing the weight of an image area.

According to the present invention, in an image processing device that determines whether a monitoring space is in a difficult-to-monitor situation by comparing a reference image and a target image, environmental changes in the monitoring space that do not cause problems in the monitoring situation occur. Even in the case of a high-precision determination, it is possible to make a highly accurate determination.

1 is a schematic configuration diagram of a security system according to the present embodiment. FIG. 1 is a schematic configuration diagram of an image sensor according to the present embodiment. FIG. 3 is a diagram showing an example of a reference image. FIG. 3 is a diagram showing an example of a stable image. FIG. 3 is a diagram showing an example of an image area. FIG. 2 is a first diagram showing an example of a target image. FIG. 2 is a second diagram showing an example of a target image. FIG. 7 is a first diagram showing an example of another target image. FIG. 7 is a second diagram showing an example of another target image. 7 is a flowchart showing the flow of reliability setting processing. 3 is a flowchart showing the flow of monitoring status determination processing.

Embodiments of the present invention will be described below.

FIG. 1 is a schematic diagram of the configuration of a security system 10 according to this embodiment. The security system 10 includes security devices 14 installed at each monitored property 12 such as a store, office, apartment, warehouse, or house, and a security center connected to each security device 14 via a communication network 16 such as a public telephone line. It is configured to include a device 18 and a user device 20. Furthermore, the security system 10 determines whether there is a moving object (a human image in this embodiment) to be detected in the monitoring space of the monitoring object 12 based on the photographed image of the monitoring space of the monitoring object 12. The image sensor 22 includes one or more image sensors 22 (image processing devices) for monitoring (detecting) whether or not the image sensor 22 is in use, and a recording device 24 that records images captured by the image sensor 22. The image sensor 22 and the recording device 24 are communicably connected to the security device 14.

When the security device 14 receives an alarm signal from the image sensor 22 connected to itself via the campus LAN, etc., the security device 14 receives the alarm signal and the identification signal of the security device 14 itself, or detects the object to be monitored 12 or an abnormality. The identification signal of the image sensor 22 is transmitted to the security center device 18. To this end, the security device 14 includes a communication interface for communicating with the image sensor 22, a communication interface for communicating with the security center device 18 and the user device 20, and a control unit for controlling them.

Security device 14 may be operable in multiple security modes. For example, it may be possible to operate in a security set mode in which the surveillance space is monitored and a security release mode in which the surveillance space is not monitored.

The security center device 18 is composed of a so-called computer, and includes a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a notification section composed of a buzzer, LED, etc. . When the security center device 18 receives an alarm signal from the security device 14 via the communication network 16, the security center device 18 sends information about the object to be monitored 12 where the security device 14 that sent the alarm signal is installed and the details of the detected abnormality to the notification unit. Notification will be sent to the monitor through the display device.

The user device 20 is also composed of a so-called computer, and includes a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a keyboard, mouse, etc., for remotely controlling the security device 14. A user interface for inputting operation commands is provided. When the user device 20 performs an operation to observe a monitored object 12 that has been registered in advance via the user interface, the user device 20 displays the current image capturing information for the security device 14 installed at the registered monitored object 12. Various image request signals requesting the user device 20 to transmit the captured images inside or the captured images recorded in the recording device 24 to the user device 20 are transmitted. Then, upon receiving the photographed image from the security device 14, the user device 20 displays the requested photographed image on the display unit.

The recording device 24 includes a recording medium that can be attached to and detached from the recording device 24, such as a magnetic disk device such as an HDD, a magnetic tape such as a DAT, and an optical recording medium such as a DVD-RAM, and accesses these recording media to store data. Consists of devices that read and write data. The recording device 24 receives the photographed image taken by the image sensor 22 from the security device 14, and records it in association with the photographing time.

FIG. 2 is a schematic diagram of the configuration of the image sensor 22 as an image processing device.

The communication unit 30 is an input/output interface that transmits and receives various setting signals and control signals between the image sensor 22 and the security device 14 via a communication network such as a campus LAN. It consists of communication interface circuits and driver software that drives them. Specifically, when a moving object (an intruder) is detected by a signal processing unit 50, which will be described later, the communication unit 30 outputs an intrusion alarm signal indicating that an intruder has been detected to the security device 14.

The photographing unit 32 photographs the monitoring space in the monitored object 12 to obtain a photographed image. The photographing unit 32 according to this embodiment is a fixed camera, and photographs a predetermined monitoring space at a constant angle of view.

Furthermore, the photographing unit 32 has two photographing modes: a daytime mode in which the surveillance space is photographed using visible light, and a night mode in which the surveillance space is photographed by illuminating near-infrared light. Photographed images can be obtained using this method. Specifically, in the daytime mode, an IR cut filter is inserted into the optical path of the imaging optical system to cut near-infrared light, and the two-dimensional detector of the imaging unit 32 acquires a visible light image. On the other hand, in the night mode, the IR cut filter is removed from the optical path of the imaging optical system, and the two-dimensional detector acquires a near-infrared image. In addition, the sensitivity wavelength range of the photoelectric converter can be changed by separately installing a visible camera that acquires visible light images and a near-infrared camera that acquires near-infrared images, and by changing the voltage applied to the organic thin film that has a stacked structure. Visible images may be captured during the daytime mode, and near-infrared images may be captured during the nighttime mode, using a method of controlling the above. In this way, the imaging unit 32 includes a photoelectric converter such as a CCD that is sensitive to visible light and infrared light, an imaging optical system that forms an image of the monitoring space on the photoelectric converter, and an imaging optical system. The camera is configured to include an IR cut filter (infrared cut filter) provided in the camera, an infrared image sensor for detecting the temperature of the object to be photographed, and the like.

In the present embodiment, the photographing unit 32 acquires photographed images by photographing at regular time intervals (for example, 1/5 second), but the photographing interval of the photographing unit 32 is not limited to this. The acquired photographed image is stored in the storage unit 38. Note that in this embodiment, the image capturing section 32 is included in the image sensor 22, but the image capturing section 32 may be provided outside the image sensor 22.

The illuminance sensor 34 includes, for example, a photodiode. The illuminance sensor 34 detects the illuminance (brightness) of the monitoring space and outputs an illuminance signal indicating the detected illuminance to a signal processing section 50, which will be described later. Note that various known sensors can be used as the illuminance sensor 34.

The illumination unit 36 includes, for example, a near-infrared LED. The illumination unit 36 emits near-infrared light toward the monitoring space when the photography mode of the photography unit 32 is night mode. On the other hand, when the photographing mode of the photographing section 32 is daytime mode, the illumination section 36 is turned off.

The storage unit 38 is composed of a semiconductor memory, a magnetic disk (HDD), or an optical disk drive such as a CD-ROM/DVD-RAM and its recording medium. The storage unit 38 stores an image processing program for operating each part of the image sensor 22. Further, as shown in FIG. 2, the storage unit 38 stores a photographed image 40 acquired by the photographing unit 32. Furthermore, the storage unit 38 stores a background image 42, a reference image 44, a stable image 46, and a reliability image 48, which will be described below.

The photographed image 40 is an image obtained by the photographing unit 32 sequentially photographing the monitoring space. The storage unit 38 stores several tens of frames (images) worth of captured images 40, and when a new captured image 40 is captured, the most captured image 40 is selected from among the plurality of captured images 40 stored in the storage unit 38. After deleting the older photographed image 40, the latest photographed image 40 is stored.

The background image 42 is an image that does not include an image corresponding to a moving object, and is created or selected by the signal processing unit 50 based on one or more captured images 40 stored in the storage unit 38. For example, the signal processing unit 50 calculates the difference between two chronologically adjacent captured images 40, and calculates the average value of the absolute values of the brightness differences between corresponding pixels between the two images. If there is no moving object in both images, the average value of the absolute value of the brightness difference between the two images will be considerably small. Therefore, the signal processing unit 50 selects, as the background image 42, one of the two captured images 40 whose average value is smaller than the predetermined reference.

The background image 42 is updated to accommodate changes in the brightness of the monitored space due to changes in lighting conditions, diurnal changes in the sun, and the like. In this embodiment, the difference between the average brightness value of a plurality of pixels included in a newly acquired captured image 40 and the average brightness value of a plurality of pixels included in an already stored background image 42 is greater than a predetermined value. When the background image 42 also becomes larger, the background image 42 is updated. The predetermined value is set in advance by an administrator or the like. Note that various "predetermined values" appear in this specification, and these are also set in advance by an administrator or the like. Note that whether or not to update the background image 42 is determined by comparing the exposure amount when a new captured image 40 is captured with the exposure amount of the background image 42 that has already been stored, rather than comparing the average brightness values. You may. Furthermore, it may be determined whether or not to update the background image 42 based on the illuminance of the monitoring space detected by the illuminance sensor 34. Alternatively, the background image 42 may be updated at regular intervals (for example, every 10 minutes).

A plurality of background images 42 may be stored in the storage unit 38, or one background image 42 may be stored in the storage unit 38.

The reference image 44 is an image among the photographed images 40 photographed by the photographing unit 32, which is photographed when the moving object to be detected is not present in the monitoring space. In other words, the reference image 44 is a photographed image 40 that does not include an image corresponding to a moving object. A reference image 44 is selected from among the plurality of captured images 40 by an administrator or the like visually checking the plurality of captured images 40 . Alternatively, since there is often no moving object in the monitoring space when the image sensor 22 is started up (when monitoring the monitoring space starts), the photographed image 40 taken when the image sensor 22 is started up is used as the reference image 44. It may be done as follows. Alternatively, the inter-frame difference may be performed within a predetermined time after the image sensor 22 is started up, and an image with no difference, that is, an image in which no moving object exists, may be set as the reference image 44. FIG. 3 shows a reference image 44 in this embodiment.

The stable image 46 as a comparison image is an image created or selected based on the plurality of captured images 40. The details of the stable image 46 will be explained together with the explanation of the processing of the stable image extracting means 52, which will be described later. Further, the reliability image 48 is an image created based on the reference image 44. Details of the reliability image 48 will be explained together with a description of the processing of the reliability setting means 54, which will be described later.

Note that in this embodiment, the captured image 40, background image 42, reference image 44, stable image 46, and reliability image 48 are stored in the storage unit 38 within the image sensor 22, but these images are The information may be stored in a storage unit provided outside the image sensor 22 and accessible from the image sensor 22 .

The signal processing unit 50 includes a built-in microprocessor unit, memory such as ROM/RAM, and peripheral circuits thereof, and executes various signal processing in the image sensor 22. As shown in FIG. 2, the signal processing section 50 functions as a stable image extracting means 52, a reliability setting means 54, a monitoring situation determining means 56, and a moving object detecting means 58. By performing these means, the signal processing unit 50 determines with high accuracy whether or not the monitoring space is in a situation where monitoring of a moving object is possible, and then a moving object is detected from the photographed image 40. Each means included in the signal processing section 50 will be explained below.

The stable image extraction means 52 extracts a stable image 46 based on the plurality of captured images 40 and stores it in the storage unit 38. In the present embodiment, the stable image extraction means 52 performs a difference extraction process (hereinafter referred to as "background difference processing") between the captured image 40 and the background image 42. (less than a quantitative amount), and the difference is small (difference amount) as a result of difference extraction processing (hereinafter referred to as "inter-frame difference processing") between the captured image 40 and other captured images 40 captured before and after the captured image 40. is less than a predetermined amount) is extracted as a stable image 46. Desirably, among the plurality of captured images 40, the captured image 40 that satisfies the above conditions and that does not have any illumination fluctuations in comparison with the background image 42 and comparison with other frame images is stabilized. It may also be extracted as the image 46. Note that whether or not an illumination variation has occurred can be determined based on the average luminance value of both images to be compared (the average of the luminance values of a plurality of pixels included in each image) or the difference in exposure amount.

FIG. 4 shows an example of the stable image 46 in this embodiment. Compared to the reference image 44 shown in FIG. 3, the shadow of the pillar appears due to sunlight, the object in front of the pillar has moved, and the pen on the desk is missing. Although the environment of the monitoring space is changing, the background image 42 is updated sequentially as described above, and an image similar to the image shown in FIG. 4 can be stored as the background image 42. Therefore, an image like the one shown in FIG. An image that is different from the standard image 44 can also be extracted as a stable image 46.

Further, the stable image extracting means 52 performs inter-frame difference processing between a plurality of photographed images 40 taken during a predetermined period of time, and as a result, there is a small change continuously for a predetermined period of time (the amount of change is a predetermined amount). In the case where the number of captured images 40 is less than 1), any one of the plurality of captured images 40 may be used as the stable image 46.

In addition, the stable image extracting means 52 does not use one photographed image 40 as a stable image 46, but divides the photographed image 40 into a plurality of image regions, and performs background subtraction processing or interframe inter-frame processing from one photographed image 40. A process of extracting only the image regions determined to have little change in the difference processing is executed on the plurality of captured images 40, and a stable image 46 is created by combining the image regions with little change extracted from the plurality of captured images 40. You may also create one.

The stable image 46 is used for reliability setting processing in a reliability setting means 54, which will be described later. As described later, the stable image extracting means 52 extracts a new stable image 46 every time the reliability setting means 54 updates the reliability (described later).

The reliability setting means 54 sets the reliability for each image area of the reference image 44 based on the reference image 44 and the stable image 46. Here, the image area is a certain area within the predetermined reference image 44. In FIG. 3, an example of an image area 60 is shown. Also shown in FIG. 4 is an image area 60 in the stable image 46. In the examples shown in FIGS. 3 and 4, the image area 60 is a rectangular area delimited by dashed lines. One image area may be composed of a plurality of pixels or may be one pixel. Further, when the image area is composed of a plurality of pixels, the image area 60 does not have to be rectangular.

In this embodiment, first, the reliability setting means 54 sets the initial reliability of all image regions 60 to the median value. For example, if the reliability takes a value from 0 to 255, the reliability setting means 54 sets the initial reliability of the entire image area 60 to 128. Note that the reliability does not need to be a numerical value. As an example, reliability may be indicated in multiple ranks. For example, reliability may be expressed in five ranks from rank A, which is the highest reliability rank, to rank E, which is the lowest reliability rank. Further, the initial reliability of the entire image area 60 may be set to the lowest value (0 in this embodiment) or the highest value.

The reliability setting unit 54 may extract edges from the reference image 44 and set the initial reliability of each image region 60 according to the edge feature amount of the edge extracted from the reference image 44. Here, an edge means a pixel whose pixel value difference is greater than or equal to a predetermined value between any adjacent pixel, and an edge feature amount (edge feature amount) is a pixel whose pixel value difference is greater than or equal to a predetermined value. means size. For example, the larger the edge feature amount of the reference image 44 included in the image region 60, the higher the initial reliability may be set. Alternatively, an object included in the reference image 44 may be detected from the shape of the extracted edge, and the initial reliability may be set higher for the image region 60 in which a larger object is detected.

Next, the reliability setting means 54 compares the reference image 44 and the stable image 46, and determines whether or not the reference image 44 and the stable image 46 are similar for each image region 60. Determine whether it is true or different. In this embodiment, normalized correlation is performed for each pixel between the reference image 44 and the stable image 46 for each image region 60, and if the correlation value is greater than or equal to a preset first threshold, the image region 60 is If it is determined that they are similar and the correlation value is less than a preset second threshold, it is determined that the image regions 60 are different and the correlation value is greater than or equal to the second threshold and less than the second threshold. In this case, it is determined that the image area 60 is in an intermediate state that does not correspond to either similarity or difference. The normalized correlation here is a process of calculating a correlation value based on whether the pixel values (luminance values and color values) of each corresponding pixel in the reference image 44 and the stable image 46 are similar. be. Note that, instead of comparing the pixel values of each pixel, the correlation value may be calculated by also taking into account the pixel values of pixels in the vicinity of the pixel for which the correlation value is to be calculated. Further, in this embodiment, the first threshold value and the second threshold value are different values, but the first threshold value and the second threshold value may be the same. In this case, there is no case where an intermediate state is determined, and each image region 60 is determined to be either similar or different (in this case, it may be expressed as dissimilar).

Whether each image area 60 of the reference image 44 and the stable image 46 is similar, different, or in an intermediate state (hereinafter collectively referred to as "similarity") is determined based on the plurality of images included in the image area 60. The determination may be made from the average brightness value or the distributed brightness value of the pixels. In this case, it is determined that the image areas 60 are similar when the difference between the average brightness value or the distributed brightness value of both images is less than a predetermined value, and the difference between the average brightness value or the distributed brightness value of both images is determined to be similar. If the value is greater than or equal to the value, it is determined that the image areas 60 are different.

Alternatively, edges may be extracted from each of the reference image 44 and the stable image 46, and the similarity of the image regions 60 may be determined by comparing the positions of the edges included in the image regions 60.

Then, the reliability setting unit 54 increases the reliability of image regions 60 that are determined to be similar, and lowers the reliability of image regions 60 that are determined to be different. The reliability of the image region 60 determined to be in an intermediate state that does not correspond to either similarity or difference is not changed. The amount of increase in reliability when it is determined that they are similar, and the amount of decrease in reliability when it is determined that they are different may be determined in advance. The amount of rise and the amount of fall may be the same or different. Note that when the reliability of the image region 60 is already the highest value (256 in this embodiment), and it is further determined that the reference image 44 and the stable image 46 are similar in the image region 60, the reliability The temperature shall not rise any higher. Furthermore, if the reliability of the image region 60 is already at the lowest value, and it is further determined that the reference image 44 and the stable image 46 are different in the image region 60, the reliability will not decrease any further. shall be taken as a thing. Note that the reliability of the image region 60 from which the difference is extracted by performing inter-frame difference on the captured image 40 taken a short time before the stable image 46 determined to be similar to the reference image 44 is not changed. You can also do this. Further, the reliability may be set for each shooting mode. In other words, the reliability may be set by comparing the stable image 46 taken in the daytime mode and the reliability set by comparing the stable image 46 taken in the nighttime mode.

The reliability setting means 54 intermittently repeats the determination of similarity between the reference image 44 and the stable image 46 in each image region 60 and the increase/decrease of the reliability of each image region 60. As described above, the stable image extracting means 52 extracts a new stable image 46 each time the reliability is updated, so in the reliability updating process by the reliability setting means 54, the reference image 44 is a different stable image each time. It will be compared with image 46. In this way, the reliability setting means 54 sets each image according to the plurality of similarities between the reference image 44 and the plurality of stable images 46 obtained by sequentially comparing the reference image 44 and different stable images 46. The reliability of the area 60 is cumulatively updated.

In this embodiment, as described above, the stable image extracting means 52 extracts the stable image 46 based on the plurality of captured images 40, but the stable image extracting means 52 extracts the stable image 46 based on the reliability setting means 54. One captured image 40 acquired at the timing of the degree update process may be used as the stable image 46. In this case, the stable image extraction means 52 determines whether the captured image 40 acquired at the timing of the reliability update process can become the stable image 46 (that is, the background image 42 and the inter-frame difference process). If it is determined that the photographed image 40 can be a stable image 46, the photographed image 40 is determined as the stable image 46. If it is determined that the photographed image 40 cannot be the stable image 46, the stable image extracting means 52 attempts to determine whether the photographed image 40 taken next can become the stable image 46.

In this embodiment, the reliability setting unit 54 updates the reliability of each image area 60 every 10 minutes, but the timing of updating the reliability is not limited to this. For example, the reliability may be updated at the timing when the difference becomes less than a predetermined value by performing inter-frame difference processing between the newly captured image 40 and the immediately previous captured image 40. Alternatively, the reliability may be updated at the timing when the environment of the monitoring space changes. An environmental change in the monitoring space can be determined, for example, based on a comparison of the average brightness value or exposure amount between the newly captured image 40 and the immediately previous captured image 40. An environmental change in the monitoring space may be determined based on the detection result of the illuminance sensor 34. Furthermore, in times when environmental changes in the monitoring space are unlikely to occur (for example, late at night), the update interval may be made longer than in other times. Note that the reliability may not be updated periodically, but may be set or updated before a determination is made by the monitoring status determining means 56, which will be described later. That is, when the monitoring situation determining means 56 makes a determination, the reliability may be set using the stable image 46 stored in the storage unit 38 before the determination.

By comparing each stable image 46 with a different environment of the monitoring space and the reference image 44, it is possible to determine the image area 60 of the reference image 44 that is less susceptible to changes in the environment of the monitoring space and the monitoring space. Image areas 60 that are susceptible to environmental changes can be identified. In other words, the reliability is an index indicating the degree to which the monitoring space is resistant to environmental changes. Environmental changes in the monitoring space here include the movement of small items within the monitoring space that do not affect the monitoring situation (for example, stationery, small cardboard boxes, etc.); (Insertion of sunlight, turning off and on of lights, generation of shadows, etc.) may also be included.

Through the above-described processing, the reliability image 48 is an image in which reliability is set in each image area 60 with respect to the reference image 44, and the reliability image 48 is stored in the storage unit 38. An example of the reliability image 48 is shown in FIG. In the example of FIG. 5, a shaded image area 60 is an area with relatively low reliability. As mentioned above, reliability can be expressed in multiple stages, but for simplicity in FIG. Image regions 60 with high reliability are represented without shading, and image regions 60 with low reliability are represented with shading. Comparing the reference image 44 shown in FIG. 3 and the stable image 46 shown in FIG. The image area 60 including the latter position, the image area 60 including the window where the sun appeared, and the image area 60 including the desk top where the pen is missing are image areas 60 with low reliability.

Preferably, in addition to the above-described determination of similarity in the reliability update process, the reliability setting means 54 extracts edges from the reference image 44 and the stable image 46, and extracts edges from the reference image 44 for each image region 60. The reliability is set according to at least one of the feature amount of the extracted edge and the degree of similarity between the reference image 44 and the stable image 46.

In this embodiment, the update reliability of the image region 60 in which the reference image 44 and the stable image 46 are determined to be similar is calculated using the following equation 1.
Update reliability = current reliability + (edge feature amount of the reference image 44 included in the image area 60 x coefficient A x similarity of the image area 60 between the reference image 44 and the stable image 46) (Formula 1)
The edge feature amount of the reference image 44 included in the image region 60 is the total value of the edge feature amount of a plurality of edge pixels (pixels corresponding to edges) included in the image region 60. That is, the clearer the edges included in the image region 60 are, or the more pixels are defined as edges (for example, the longer the edges are), the larger the edge feature amount becomes.

According to Equation 1, the larger the edge feature amount of the reference image 44 included in the image region 60, the greater the reliability will be when the reference image 44 and the stable image 46 are similar. This is because pixels corresponding to the outline of an object tend to have large edge features, and in the image region 60 where the reference image 44 has a large edge feature, if the reference image 44 and the stable image 46 are similar, This is because it is clearer that there was no movement of the object between the two images in the image area 60, and such an image area 60 can be said to have high reliability.

Furthermore, the greater the degree of similarity between the reference image 44 and the stable image 46 in the image region 60, the greater the reliability will be when the reference image 44 and the stable image 46 are similar. Note that the degree of similarity here may be calculated by the above-mentioned normalized correlation, or it may be calculated based only on the edges extracted from the reference image 44 and the stable image 46 (for example, the degree of matching of edge positions in both images). You may. In addition, in the above (Formula 1), the update reliability is set using the edge feature amount, coefficient A, and similarity, but the reliability may be set using one or both of them. Good too.

On the other hand, in this embodiment, the update reliability of the image region 60 in which the reference image 44 and the stable image 46 are determined to be dissimilar (different) is calculated by the following equation 2.
Update reliability = current reliability - coefficient B (Formula 2)
As shown in Equation 2, in this embodiment, the reliability of the image region 60 in which the reference image 44 and the stable image 46 are determined to be dissimilar decreases by a certain value (coefficient B). Note that even when it is determined that the reference image 44 and the stable image 46 are dissimilar, the edge feature amount of the reference image 44 included in the image area 60 and the degree of similarity between the reference image 44 and the stable image 46 are The reliability may be lowered accordingly. For example, the reliability may be reduced as the edge feature amount is larger, and the reliability may be reduced as the similarity is smaller.

Incidentally, the coefficient A included in Equation 1 and the coefficient B included in Equation 2 may be determined in advance. Coefficient A and coefficient B may be the same value or may be different values.

Further, the reliability setting means 54 may set the reliability setting unit 54 to reduce the amount of decrease in the reliability when the reference image 44 and the stable image 46 are dissimilar as the current reliability is higher for each image region 60. good. In other words, the lower the current reliability, the greater the amount of decrease in the reliability when the reference image 44 and the stable image 46 are dissimilar.

Giving more importance to the edges extracted from the reference image 44, the reliability setting means 54 sets the reliability to the lowest value (this embodiment Then, it may be fixed to 0) and the reliability of the image area 60 may not be updated. In particular, when the image region 60 is one pixel, the reliability setting means 54 may set the reliability only for edge pixels extracted from the reference image 44. Note that when edge pixels are continuous, the reliability may be set by considering the continuous edge pixels as one image area 60.

As described above, the reliability setting means 54 cumulatively updates the reliability of each image area 60, and stores in the storage unit 38 an update history of reliability for each image area 60. It's okay. Based on the reliability update history, it is possible to grasp temporal changes in the reliability of each image region 60. Here, the reliability setting means 54 determines that the image area 60 that repeatedly increases and decreases as the reliability changes over time, and other areas (that is, the reliability does not repeat increases and decreases, in other words, the reliability generally increases or decreases). The amount of decrease in reliability may be made larger when the reference image 44 and the stable image 46 are dissimilar than in the image region 60 (where there is a tendency). For example, in an image area 60 in which branches or leaves of a plant are shaking, or in an image area 60 in which shadows repeatedly appear and disappear due to lights being turned off, the reliability level may repeatedly increase and decrease over time. become. Such an image region 60 may be determined to be similar when comparing the reference image 44 and a specific stable image 46, and the reliability may temporarily increase; however, when comparing a plurality of stable images 46, If you look, there is a change in the image, that is, there is an image area 60 where the reliability should be lowered. Therefore, for the image region 60 where the reliability repeatedly increases and decreases over time, in order to further reduce the reliability, the amount of decrease in the reliability when the reference image 44 and the stable image 46 are dissimilar is increased. good.

The monitoring status determination means 56 calculates an evaluation value indicating the degree of dissimilarity between the target image, which is the photographed image 40 to be determined (for example, the newly acquired photographed image 40), and the reliability image 48, and calculates the evaluation value. Based on this, it is determined whether the monitoring space is in a situation where the presence of a moving object can be monitored.

Specifically, first, the monitoring status determining means 56 calculates the degree of difference between the target image and the reliability image 48 (the content of the image is the same as the reference image 44) for each image area 60. The degree of difference is calculated based on the number of edge disappearances, which will be described later.

Next, the monitoring status determining means 56 sets a weight for each image region 60 based on the reliability set for each image region 60 by the reliability setting means 54. Specifically, the higher the reliability, the larger the weight, and the lower the reliability, the smaller the weight. As a result, the weight of the low-reliability image area, which has lower reliability than the high-reliability image area, is set smaller than the weight of the high-reliability image area, which has relatively high reliability.

Then, the product of the degree of dissimilarity and the weight is calculated for each image area 60, and the evaluation value is calculated by summing the multiple products obtained from the multiple image areas 60. When the calculated evaluation value is equal to or higher than a predetermined value, the monitoring situation determining means 56 determines that the monitoring space is in a situation where the presence of a moving object cannot be monitored (that is, the situation is difficult to monitor), and the calculated evaluation value is is less than a predetermined value, it is determined that the monitoring space is in a situation where the presence of a moving object can be monitored. When the monitoring situation determining means 56 determines that the monitoring space is not in a situation where the presence of a moving body can be monitored, it notifies the security center device 18.

As described above, the monitoring status determining means 56 determines whether or not the monitoring space is in a monitoring possible condition based on the evaluation value indicating the degree of difference between the target image and the reliability image 48 (reference image 44). However, in the calculation process of the evaluation value, since the image region 60 with low reliability is given a small weight, the degree of similarity between the target image and the reliability image 48 is not much considered (ultimately it is ignored). ). That is, in the monitoring space, the evaluation value is calculated without giving much consideration to areas that are susceptible to environmental changes. Therefore, even if there is a dissimilarity between the target image and the reliability image 48 due to environmental changes within the monitoring space, even though the monitoring space is originally in a situation where monitoring is possible. The monitoring situation determining means 56 can correctly determine that the monitoring space is in a situation where monitoring is possible. Note that the evaluation is performed without calculating the product of dissimilarity and weight (without setting weights), ignoring the judgment results of low-reliability image areas, and using only the judgment results of high-reliability image areas. A value may also be calculated. In this case as well, it can be said that the evaluation value is calculated by giving greater weight to the high reliability image area than to the low reliability image area.

For example, when determining the target image 40a shown in FIG. 6, compared to the reference image 44 (reliability image 48) shown in FIG. An object is missing, and a box is placed in place of the pen that was on the desk. The monitoring space indicated by the target image 40a is originally a situation in which a moving object can be monitored. However, if the conventional method, that is, the determination method based on the degree of difference between the target image 40a and the reliability image 48 without using the reliability, is used, since there is a difference between the two images, the surveillance space can be monitored. There was a risk that it would be determined that the situation was not suitable.

According to this embodiment, as shown in FIG. 7, the shaded image area is a low-reliability image area, and its reliability is low, that is, its weight is small. Therefore, the evaluation value indicating the degree of difference between the target image 40a and the reliability image 48 is calculated without much consideration (ultimately ignored) of the low reliability image area. Therefore, the monitoring situation determining means 56 can correctly determine that the monitoring space is in a situation where monitoring is possible even when an environmental change in the monitoring space occurs that does not cause a problem in the monitoring situation.

On the other hand, FIG. 8 shows a target image 40b photographed with a shield placed between the photographing unit 32 and the monitoring space. The monitoring space indicated by the target image 40b is a situation in which monitoring of a moving object is truly impossible. In the present embodiment, an evaluation value indicating the degree of difference between the target image 40a and the reliability image 48 is calculated with sufficient consideration given to the high reliability image area that is not shaded. Therefore, when comparing the target image 40b and the reference image 44 in FIG. 3, there are image regions 60 (for example, the image region 60a in FIG. 9) that have differences among the high-reliability image regions. Based on the difference in the image area 60, the monitoring situation determining means 56 can correctly determine, based on the target image 40b, that the monitoring space is in a situation where monitoring of a moving object is impossible. Note that the target image 40b is an example in which the monitoring space is in a situation where it is impossible to monitor a moving object due to an obstruction, but according to the present embodiment, for example, the image sensor 22 is coated with paint such as spray. It is possible to detect even if the monitoring space becomes unmonitorable due to factors other than obstructions, such as being blocked by objects.

Preferably, the monitoring status determination means 56 extracts edges from each of the target image and the reliability image 48, and compares the position of the edge extracted from the target image with the position of the edge extracted from the reliability image 48. The evaluation value may be calculated based on the above. Note that the target image 40a in FIG. 7 is an example in which it is determined that the monitoring space is in a situation where monitoring is possible even when an environmental change occurs due to sunlight or the movement of small objects. According to this method, even if there is a change in the environment due to other factors, it can be detected that the situation is monitorable. For example, even if smoke from a cigarette or the like hits the image sensor 22 and the photographed image 40 becomes temporarily cloudy, edges are extracted from the photographed image because the clouding is due to smoke. Therefore, by comparing the edges extracted from the target image in the high reliability image area and the edges extracted from the reliability image 48, the degree of similarity between the high reliability image areas becomes high, so that the monitoring space can be monitored. It can be determined that Note that if the captured image 40 becomes cloudy for a predetermined period of time due to tar or oil from cigarette smoke or the like adhering to the image sensor 22, monitoring becomes impossible because edges cannot be extracted due to the tar or other objects adhering to the image sensor 22. It can be determined that the situation is

In this embodiment, the monitoring status determining means 56 first classifies each image region 60 into a plurality of reliability ranks according to the reliability set by the reliability setting means 54. For example, the images are classified into low reliability image regions (reliability values 0 to 110), medium reliability image regions (reliability values 111 to 180), and high reliability image regions (reliability values 181 to 255). Furthermore, in each image region 60, the number of edge disappearances (dissimilarity), which is the number of pixels that were edge pixels in the reliability image 48 but are no longer edge pixels in the corresponding pixels of the target image, is determined.

Then, for each image region 60, pixels that were edge pixels in the reliability image 48 but are no longer edge pixels in the corresponding pixel of the target image, for the number of edge pixels included in the reliability image 48. Find the discrepancy rate (evaluation value), which is the ratio of numbers. Specifically, in the present embodiment, the mismatch rate is calculated using Equation 3 below.
Discrepancy rate = (Number of edge disappearances in low reliability image area x 1 + Number of edge disappearances in medium reliability image area x 2 + Number of edge disappearances in high reliability image area x 3) / Edge pixels extracted from reliability image 48 Total number x 6... (Formula 3)

In Equation 3, the number of edge disappearances in low reliability image areas is multiplied by a small coefficient (1), and the image area 60 with higher reliability is multiplied by a larger coefficient, so the above discrepancy rate also increases. It can be said that the calculation is performed in consideration of the weight according to the reliability of each image region 60.

The monitoring status determining means 56 uses the calculated discrepancy rate as an evaluation value, and determines that the monitoring space is in a situation where monitoring is impossible when the discrepancy rate is equal to or higher than a predetermined value, and when the discrepancy rate is less than a predetermined value. determines that the monitoring space is in a situation where it can be monitored. Note that in the above method, edges newly appearing in the target image at positions that did not exist in the reliability image 48 are not used for determination. Although the mismatch rate increases due to the movement or appearance of small objects, newly appearing edges may also be used as the number of edge disappearances to calculate the mismatch rate.

For example, when there is an environmental change in the monitoring space due to light or shadow, the brightness of the entire or most part of the target image changes uniformly with respect to the reliability image 48 (reference image 44). Edge extraction can be performed with high precision. Therefore, by calculating the evaluation value using edges, it is possible to improve the processing accuracy of the monitoring status determining means 56.

In addition, in calculating the mismatch rate, in this embodiment, the number of edge disappearances in the low reliability image area and the medium confidence image area is considered, but the mismatch rate is calculated based only on the number of edge disappearances in the high confidence image area. may be calculated. In that case, the mismatch rate is calculated using the following equation 4, for example.
Mismatch rate = number of edge disappearances in high reliability image area/total number of edge pixels extracted from reliability image 48... (Formula 4)
Equation 4 can also be viewed as Equation 3 in which the weight of the low reliability image area and the medium reliability image area is set to 0 (lowest), and the weight of the high reliability image area is set to the maximum. Furthermore, it is not necessary to calculate the mismatch rate from the product of the number of edge disappearances and the weight. For example, the weight of the number of edge disappearances in high-reliability image areas is greater than the number of edge disappearances in low-reliability image areas (ignoring the low-reliability image areas), and the evaluation value is calculated from the number of edge disappearances. Good too.

Furthermore, the monitoring status determining means 56 extracts edges from the reliability image 48, and detects areas in the reliability image 48 where no edges exist or where the number of pixels corresponding to edges (edge pixels) is less than a predetermined value. A certain homogeneous area is detected, and an evaluation value is calculated by placing more emphasis on the comparison result of the edge position between the reliability image 48 and the target image in the vicinity of the homogeneous area than in areas other than the surrounding area of the homogeneous area. You may also do so. Incidentally, the homogeneous area, its surrounding neighborhood area, and areas other than the surrounding neighborhood area are different concepts from the image area 60 described above. In this case, for example, one image area 60 is divided into a surrounding neighborhood area and two other divided image areas, and the weight set for the image area 60 by the reliability setting means 54 is set to correspond to the surrounding neighborhood area. After increasing the weight of the divided image area and decreasing the weight of the divided image area corresponding to areas other than the surrounding neighborhood area, the evaluation value is calculated by the above-described process. In other words, the evaluation value focuses on the edges of objects that are not easily affected by environmental changes in the monitoring space, such as floors, walls, columns, desks, etc., and the edges of objects that are likely to become impossible to photograph if obstacles are installed. Perform calculations. As a result, even if an obstacle similar in color and shape to a floor or wall is installed so as to overlap the floor or wall, it can be determined with high accuracy that the situation is difficult to monitor.

Further, the monitoring status determining means 56 may determine whether or not the monitoring space is in a situation where monitoring is possible based on the reliability itself set in the reliability image 48. Specifically, in the reliability image 48, if the average reliability of the plurality of image regions 60 is less than a predetermined value, it may be determined that the monitoring space is in a situation where the presence of a moving object cannot be monitored.

Alternatively, the monitoring status determining unit 56 may determine whether or not the monitoring space is in a monitoring possible condition based on the reliability update history stored in the storage unit 38 by the reliability setting unit 54. . Specifically, if the average reliability of the plurality of image regions 60 is on a downward trend, it may be determined that the monitoring space is in a situation where the presence of a moving object cannot be monitored. Here, the average reliability is on a downward trend not only when the average reliability decreases continuously over a predetermined number of updates, but also when the average reliability may increase slightly during updates. This concept also includes a case where the average reliability of a predetermined amount decreases after a predetermined number of updates.

In the present embodiment, the degree of dissimilarity is calculated from the number of edge disappearances (whether the target image and reliability image match or do not match for each image region), but is not limited to this. For example, the target image and the reliability image 48 may be compared to extract a difference (difference in average brightness value, etc.), and the degree of difference may be determined according to the value of the extracted difference. For example, if there is no difference, the degree of difference may be determined to be 0, and the larger the difference, the higher the degree of difference may be requested.

In the present embodiment, the monitoring status determining means 56 determines intermittently (for example, at predetermined intervals) whether or not the monitoring space is in a status that allows monitoring, but the timing at which the determination is made is not limited to this. For example, the determination may be made every time the captured image 40 is acquired. Furthermore, for example, it may be determined at the timing when the security mode of the security device 14 is changed whether or not the monitoring space is in a situation where the presence of a moving body can be monitored. Alternatively, it may be determined whether the monitoring space is in a state where the presence of a moving object can be monitored at the timing when the shooting mode of the shooting unit 32 is changed between daytime mode and nighttime mode. Further, as a result of the reliability update process of the reliability setting means 54, the monitoring space is moved at the timing when the reliability of the high reliability image area (the image area 60 whose reliability is equal to or higher than the predetermined value) decreases by a predetermined amount. It may also be determined whether or not the presence of the body can be monitored.

The monitoring status determining unit 56 may determine whether or not to determine the status of the monitoring space depending on the number of times the reliability setting unit 54 has performed the reliability update process. For example, if the reliability update process by the reliability setting unit 54 has been performed a predetermined number of times (for example, 10 times) or more, the monitoring status determining unit 56 determines the status of the monitored space, but the reliability update process is If the number of times is less than a predetermined number, the situation of the monitoring space may not be determined. Further, if the average reliability of the plurality of image regions 60 in the reliability image 48 is greater than or equal to a predetermined value, or if there are a predetermined number of image regions 60 with reliability greater than or equal to a predetermined value, the situation of the monitoring space is determined. In other cases, the situation of the monitoring space may not be determined.

When the monitoring situation determining means 56 determines that the monitoring space is in a situation where the presence of a moving object can be monitored, the moving object detecting means 58 compares the object image with the background image 42 to detect a moving object from the object image. Detect the body. Regarding the process of detecting a moving object by comparing the target image and the background image 42, a conventional technique may be used, so a detailed explanation will be omitted here. When a moving object is detected from the target image, an intrusion alarm signal is transmitted from the communication unit 30 to the security device 14.

The flow of the reliability setting process by the reliability setting means 54 will be explained below according to the flowchart shown in FIG.

In step S10, the reliability setting means 54 sets the initial reliability for the entire image area 60 of the reference image 44. In this embodiment, as described above, the reliability setting unit 54 sets the initial reliability of all image regions 60 to the median value.

In step S12, the reliability setting means 54 selects one image area 60 for which reliability is to be set (updated).

In step S14, the reliability setting means 54 compares the reference image 44 and the stable image 46 for the image region 60 selected in step S12, and determines their similarity in step S16.

If it is determined in step S16 that they are similar, the reliability setting means 54 increases the reliability of the image area 60 in step S18. On the other hand, if it is determined in step S16 that there is a difference, the reliability setting means 54 decreases the reliability of the image area 60 in step S20. The increasing and decreasing methods are as described above.

In step S22, the reliability setting means 54 determines whether the processing of steps S14 to S20 for all image regions 60 included in the reference image 44 has been completed. If there is an unprocessed image area 60, the process returns to step S12, and the reliability setting means 54 selects the unprocessed image area 60 in step S12 again, and repeats the process from step S14 onwards.

If the processing for all image regions 60 included in the reference image 44 has been completed, the process advances to step S24, and in step S24, the reliability setting means 54 sets the reliability of each image region 60 to the updated reliability. The image 48 is stored in the storage unit 38.

The flow of the monitoring status determination process by the monitoring status determining means 56 will be described below according to the flowchart shown in FIG.

In step S30, the monitoring status determining means 56 determines whether the security mode of the security device 14 has been changed to a security set mode for monitoring the surveillance space. The process waits until the security mode is changed to the security set mode, and if the security mode is changed to the security set mode, the process advances to step S32.

In step S32, the monitoring status determining means 56 determines whether the reliability of the reliability image 48 has been updated a predetermined number of times or more by the reliability setting means 54. If the number of times the reliability of the reliability image 48 has been updated is less than the predetermined number of times, the monitoring status determining means 56 ends the monitoring status determining process without performing the processes from step S34 onwards. If the number of times the reliability of the reliability image 48 has been updated is equal to or greater than the predetermined number of times, the process advances to step S34.

In step S34, the monitoring status determining means 56 calculates an evaluation value indicating the degree of difference between the target image and the reliability image 48, taking into account the reliability of each image area 60 set by the reliability setting means 54. . Specifically, as described above, the monitoring status determining means 56 sets a larger weight for each image region 60 as the reliability is higher, and sets a smaller weight as the reliability becomes lower. The evaluation value is calculated by calculating the product of the degree of difference and the weight, and summing the multiple products obtained from the multiple image regions 60.

In step S36, the monitoring status determining means 56 determines whether the evaluation value calculated in step S34 is less than a predetermined value. If the evaluation value is equal to or greater than the predetermined value, in step S38, the monitoring situation determining means 56 determines that the monitoring space is not in a situation where the presence of a moving object can be monitored. On the other hand, if the evaluation value is less than the predetermined value, in step S40, the monitoring situation determining means 56 determines that the monitoring space is in a situation where the presence of a moving object can be monitored.

Although the embodiments according to the present invention have been described above, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention.

10 security system, 12 monitored object, 14 security device, 16 communication network, 18 security center device, 20 user device, 22 image sensor, 24 recording device, 30 communication section, 32 photographing section, 34 illuminance sensor, 36 lighting section , 38 storage section, 40 photographed image, 40a, 40b target image, 42 background image, 44 reference image, 46 stable image, 48 reliability image, 50 signal processing section, 52 stable image extraction means, 54 reliability setting means, 56 Surveillance status determination means, 58 Moving object detection means, 60, 60a Image area.

Claims (9)

An image processing device that monitors whether a moving object exists in a monitoring space based on a captured image of the monitoring space,
The reference image is determined according to the similarity between a reference image that is captured when the mobile object is not present in the monitoring space among the captured images, and a comparison image that is based on the captured image that is different from the reference image. Reliability setting means for setting reliability for each image area;
The degree of difference between the reference image and the target image, which is the photographed image to be determined, is calculated for each image region, and the monitoring space is moved according to the degree of difference for each image region and a weight according to the reliability. Surveillance status determination means for determining whether the presence of the body can be monitored;
Equipped with
giving greater weight to the image region with the higher reliability than the image region with the set lower reliability;
An image processing device characterized by: The monitoring situation determining means determines the degree of difference based on a comparison of edges extracted from the reference image and the target image, respectively.
The image processing device according to claim 1, characterized in that: In addition to the determination of similarity, the reliability setting means sets the reliability according to a feature amount of an edge extracted from each of the image regions of the reference image.
The image processing device according to claim 2, characterized in that: The monitoring situation determining means calculates an evaluation value indicating a degree of mismatch between the reference image and the target image from the weight for each image region and the degree of difference, and based on the evaluation value, determines whether the monitoring space is determining whether the presence of the mobile object can be monitored;
The image processing device according to any one of claims 1 to 3. The reliability setting means cumulatively updates the reliability for each image region according to the similarity obtained by sequentially comparing the reference image and the plurality of comparison images, and cumulatively updates the reliability for each image region. Among them, the higher the current reliability, the smaller the amount of decrease in the reliability when the reference image and the comparison image are dissimilar;
The image processing device according to any one of claims 1 to 4. The monitoring situation determining means further determines the weight around a homogeneous area, which is an area where no edge exists in the reference image or an area where the number of pixels corresponding to an edge is less than a predetermined number, among the edges extracted from the reference image. determining whether the monitoring space is in a situation where the presence of the moving body can be monitored based on the degree of difference and a weight that is greater in the surrounding neighborhood area than in areas other than the neighborhood area;
The image processing device according to claim 2 or 3, characterized in that: The monitoring situation determining means determines that the monitoring space is in a situation where the presence of the moving body cannot be monitored if the average reliability of the plurality of image areas is less than a predetermined value.
The image processing device according to any one of claims 1 to 6. The reliability setting means cumulatively updates the reliability for each image region according to the similarity obtained by sequentially comparing the reference image and the plurality of comparison images,
The monitoring status determining means determines that in the cumulative update of the reliability, if the average of the reliability of the plurality of image regions is on a downward trend, the monitoring space is in a situation where the presence of the moving object cannot be monitored. It is determined that
The image processing device according to any one of claims 1 to 7. computer,
An image processing device that monitors whether a moving object exists in a monitoring space based on a captured image of the monitoring space,
The reference image is determined according to the similarity between a reference image that is captured when the mobile object is not present in the monitoring space among the captured images, and a comparison image that is based on the captured image that is different from the reference image. Reliability setting means for setting reliability for each image area;
The degree of difference between the reference image and the target image, which is the photographed image to be determined, is calculated for each image region, and the monitoring space is moved according to the degree of difference for each image region and a weight according to the reliability. Surveillance status determination means for determining whether the presence of the body can be monitored;
function as an image processing device equipped with
giving greater weight to the image region with the higher reliability than the image region with the set lower reliability;
An image processing program characterized by:

Images