JP2011217201A - Leaving or carrying-away determination apparatus and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine whether a differential (foreign substance) image obtained from a background image and a current image is caused by leaving or carrying-away.SOLUTION: A differential image creating means 201 in a determination control device 200 of a determination apparatus 20 creates a differential image [(background differential value)=(current image)-(background image)] between a background image in a monitored area and a current image in the monitored area captured by an electronic camera 10 and a leaving or carrying-away determining means 202 determines, based on a positive or negative code of a pixel of the created differential image, whether the differential image is caused by leaving or carrying-away.

Description

  In the present invention, for example, when a person leaves an object in a certain place, such as a forgotten object, or removes the object, the object determined as a foreign object is either an object left or removed. The present invention relates to a leaving or carrying-out determination device, a determination method, and a program for identifying the above.

A method and an apparatus for determining when an object is placed or taken away at a certain place are already known. For example, if you take a background image of the place with a photographic device, store the background image in memory, compare it with the current image, and create the difference image, someone left or removed it. Since the object is obtained as a difference image, the object can be specified.
In addition, when there are people or things that move constantly, such as at stations, it is difficult to distinguish them from other moving objects even if a difference image is simply created from images taken after a while. Even in such a case, an invention (change area calculation method and apparatus) that can determine whether an object is placed or taken away at a certain place is already known (see Patent Document 1).

  In this change area calculation method and apparatus, a plurality of captured images captured at a predetermined time interval are averaged over the same imaging area, and the pixel values of the pixels of each image are averaged to shorten the time. The influence of what enters and exits is diluted so that it can be extracted as foreign matter only when the foreign matter in the image stays for a certain period of time or longer.

However, in each of the above conventional methods, since only the location changed from the background is determined, it is possible to identify an object that has been left or removed, but whether the specified object has been left or It is not possible to determine whether it has been taken away.
Therefore, even if it is desired to construct a system based on the determination of leaving or taking away, such a request cannot be met. For example, in the case of a system for determining a lost article, generally, the determination of leaving is necessary, but it is not necessary to determine whether the lost article is removed after the lost article is collected. However, the conventional apparatus and method cannot construct a system that distinguishes such leaving or taking away.

JP 2007-108965 A

  The present invention has been made in view of conventional problems, and an object of the present invention is to distinguish between leaving and taking away with a single camera.

The leaving or taking away determination device of the present invention includes a storage device that stores a background image of a monitoring area photographed by a photographing device, and a difference that creates a difference image between the background image and a current image of the monitoring area photographed by the photographing device. Image creation means, and a leaving or removal determination means for determining whether the difference image is due to leaving or removal based on a positive or negative sign of a pixel of the created difference image.
The leaving or taking away determination method of the present invention is a leaving or taking away judging method in a leaving or take away judging device, which stores a background image of a monitoring area photographed by a photographing device, and the background image and photographing. A difference image creation step for creating a difference image with the current image of the monitoring area photographed by the device, and determining whether the difference image is due to leaving or taking away based on the positive or negative sign of the pixel of the created difference image And a step of determining whether to leave or take away.
The program according to the present invention causes the computer of the leaving or removing determination apparatus of the present invention to function as each unit of the leaving or removal determining apparatus.

  According to the present invention, it is possible to realize an application corresponding to each use by distinguishing between leaving or taking away and determining. That is, the present invention can be applied to a system that requires determination of only leaving or taking away or both of them.

It is a figure explaining the principle of this invention. It is the block diagram which showed schematically the structure of the determination apparatus which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the leaving or removal determination by a determination apparatus. It is a flowchart which shows the fixed pixel determination procedure performed by the determination process of the fixed pixel in the flowchart of FIG. Examples showing exceptions to the first embodiment An example corresponding to another exception of the first embodiment is shown. It is the block diagram which showed schematically the structure of the determination apparatus which concerns on another embodiment. It is a flowchart which shows the procedure of the leaving or the removal determination process by the determination apparatus of 2nd Embodiment. It is a flowchart explaining the process of the log | history determination in the flowchart of FIG. It is the block diagram which showed schematically the structure of the determination apparatus which concerns on another embodiment. It is a flowchart which shows the procedure of the leaving or carrying out determination implemented with the determination apparatus which concerns on embodiment of FIG. It is a figure explaining the opening process and closing process in the dense location detection of a background edge.

Hereinafter, the abandonment or removal determination according to the embodiment of the present invention will be described.
First, the principle of the present invention will be described.
A difference image (background difference value = current image−background image) is created from the background image captured by the camera (electronic camera) and the current image, and the difference value of the pixel value for each pixel of both images is taken. Whether the difference image exists only in the current image or only in the background image can be determined based on the sign of the background difference value + or −. That is, if it is + (the pixel value of the current image is larger), it can be imagined that the pixel value has increased as a result of some object (foreign matter) entering the pixel. Conversely, if it is-(the pixel value of the current image is smaller), it can be imagined that the pixel value has decreased because there is no object (foreign matter) at that pixel in the current image.

FIG. 1 is a diagram for explaining the principle of the present invention.
That is, as shown in FIG. 1A, when there is no foreign object in the background image (FIG. 1A (1)) and there is a foreign object in the current image (FIG. 1A (2)), the edge image of each of the background image and the current image is displayed. Background difference value (pixel value) = current image-background image is + difference, and as shown in FIG. 1B, the background image (FIG. 1B (1)) contains a foreign object, and the current image (FIG. 1B (2)). If there is no foreign object, the background image difference value (pixel value) of the edge image = current image−background image−difference.
Therefore, when the value is +, it can be determined as a left-out pixel (pixel), and when it is −, it can be determined as a left-out pixel (pixel). The present invention uses this principle to determine whether the foreign object detected in the differential image is left behind or removed.

Hereinafter, a leaving or taking away determination apparatus and method will be described with respect to an embodiment of the present invention. First, a leaving or leaving determination device (hereinafter simply referred to as a determination device) will be described.
FIG. 2 is a diagram schematically showing a determination device according to an embodiment of the present invention.
The determination device 20 is connected to a camera (electronic camera) 10 installed in a monitoring area, and is a leaving or leaving determination control device (hereinafter simply referred to as a determination control device) 200 and a storage device 22 that stores image data. And a display device 24 having, for example, a liquid crystal display means (LCD) for displaying the output of the determination device 20 and the input result of the input device 26, and the input device 26 having a keyboard and touch input means. Yes. Reference numeral 30 denotes a bus for connecting the devices.
The determination control apparatus 200 includes, for example, a microcomputer, and functions as means for realizing a program of the microcomputer include a differential image creation unit 201, a left or removed determination unit 202, a non-moving pixel determination unit 203, a labeling unit 204, and a background image update unit. 205.

FIG. 3 is a flowchart showing a procedure for determining whether to leave or take away by the determination device 20.
With reference to the process flow diagram shown in FIG. 3, the determination of leaving or taking away by the determination device according to the embodiment of the present invention will be described.
That is, first, an image captured by the electronic camera 10 is input to the determination device 20 according to the present embodiment (S101), and then an edge image is created (extracted) from the input image (in this case, an RGB image or a luminance image). (S102), the difference image creating means 201 calculates a background difference value (background difference value = current image−background image) for each pixel (pixel) of the obtained edge image, and creates a signed background difference image. (S103). Next, the leaving or taking away determination unit 202 performs leaving or taking away determination based on the sign of the signed background difference of the pixel value of the obtained edge image (S104), that is, (i) the background difference value is determined. Positive value, and a predetermined threshold value is set here. If the positive value is greater than or equal to the threshold value, it is left behind. (Ii) If the background difference value is negative and less than the threshold value, it is left away. (Iii) Otherwise, it is determined that there is no foreign object.

In the present embodiment, the non-moving pixel determining unit 203 further determines that a pixel that has been left or carried away for a certain period of time or more is a non-moving pixel (S105). At this time, the leaving stationary pixel and the leaving stationary pixel are distinguished and determined.
Next, the non-moving pixel detected by the labeling unit 204 is labeled (S106), and then the determination processing result is output (S107). The background image updating unit 205 updates the corresponding pixel of the background image with the non-moving pixel. (S108) The next abandonment or removal determination process is performed.
As described above, in this embodiment, even if there is a difference between leaving and taking away, it is not immediately determined as a foreign object to be determined, and noise is first removed by limiting the difference to a certain threshold or more. Furthermore, it is treated as a difference image (foreign matter) only when the state of leaving or taking away continues for a certain time or longer. Further, the storage device 22 is made to store whether the foreign matter is left behind or taken away as pixel information of the background image. That is, since the background image is updated with the information of the non-moving pixels, it is possible to exclude a person or an object that enters or exits the monitoring area in a short time from the determination target and perform a highly accurate leaving or carrying out determination. In addition, after performing the leaving or removal determination, it is possible to continue to perform the next removal or removal determination using the updated background image.

FIG. 4 is a flowchart showing the stationary pixel determination procedure executed in the stationary pixel determination process in steps S105 and S106.
In this process, first, the presence or absence of a leaving difference is determined (S201). When it is determined that there is a leaving difference (S201, YES), the leaving timer is added, and the leaving timer is subtracted (S202). It is determined whether or not the set value is exceeded (S203). When the leaving timer is set to be the set value or more (S203, YES), it is determined that the pixel is a leaving immovable pixel (S204), and the process is performed. finish.

In step S203, if the leave timer is less than the set value (S203, NO), it is determined whether or not the leave timer is equal to or greater than the set value (S208). It is determined that the pixel is a fixed pixel that has been taken away (S209), and the process ends. Further, if the subtraction value of the take-away timer is less than the set value (S208, NO), both the leave timer and the take-off timer are less than the set value, and thus, as it is, that is, without making a leave or take-away determination. The process ends.
In step S201, if there is no leftover difference (S201, NO), it is determined whether there is a leftover difference (S205). If there is a leftover difference (S205, YES), the removed timer is added and the leftout timer is set. Subtraction is performed (S206), and the process proceeds to step S203 and subsequent steps.
In step S205, when there is no take-away difference (S205, NO), the leaving timer and the take-out timer are subtracted (S207), and the process proceeds to step S203 and subsequent steps.
In the above processing, the set value is the time for which it is decided to leave or take away. Accordingly, when the device is left for more than the set value, it is determined whether it is left or removed.

In most cases, as described above, the object leaving is + difference, the object removal is -difference, and it is possible to distinguish between the object leaving and the object leaving. For example, by determining only the object leaving It is possible not to determine whether the object is taken away.
In practice, however, there are some exceptions where the above leave or take away determination is not applicable. Next, this point will be described.

5 and 6 show examples corresponding to the above exception.
First, the first exception will be described.
As shown in FIG. 5A, when the background image has an edge portion (for example, a crack in the floor, which is the background image), and the object is placed on the edge portion, the edge of the object is applied by applying the above determination method. Is determined to be left behind, and the background edge is determined to be removed.
That is, for example, in FIG. 5A (1), the background image has a + pattern edge here, and in the current image in FIG. 5A (2), an object is placed on the edge. At this time, if the pixel leaving and removal information (that is, difference information) is determined by the background difference value of the edge image, the object edge is determined to be left as shown in FIG. 5A (3), and + Since the pattern edge is determined to be carried away, it cannot be determined accurately. For example, when a system is constructed that detects only the removal of an object and does not detect the removal, the removal is detected even when the object is left.

  As shown in FIG. 5B (1) to (3), when the object is placed on the edge of the + pattern in the background image and the object is taken away, the object as shown in FIG. It is determined that the edge of the pattern is removed and the edge of the + pattern is determined to be left behind. In other words, when an object placed on the original background edge is taken away, the edge image of the object is taken away and the original edge of the eyelid background is left behind, so an accurate judgment cannot be made. . For example, when a system is constructed that detects only the object leaving and not the object leaving, the object is detected even when the object is left.

Next, exception 2 will be described.
Edges are densely present in the background image as shown in FIGS. 6A (1) and 6B (1), and when the background edge strength of the pixel> the object edge strength, the object leaving determination itself may be difficult. is there.
That is, in FIG. 6A, the background image has an edge as shown in FIG. 6A (1), and the edge image of the current image with an object placed thereon as shown in FIG. When the above-mentioned leaving or carrying-out determination is performed in the case of the object edge strength, in the leaving determination image shown in FIG. 6A (3), the leaving determination is performed at a portion surrounded by a rectangle, and the portion where the object edge is broken (background image) In the portion where the edge overlaps the object edge), the removal determination is made.

  Also, as shown in FIG. 6B (1), when the edges of the background image are more dense, the current image with objects placed thereon becomes the image of FIG. 6B (2), and the background edge strength of the pixel In the case of> object edge strength, if it is determined by the background difference value of the edge image, all the edges of the object are determined to be carried away, and there is no portion that can be left behind. Therefore, in this case, it is not possible to determine whether the object has been left behind.

Next, countermeasures for the cases of exceptions 1 and 2 will be described.
Embodiment corresponding to Exception 1 (Embodiment 2)
In the present embodiment, as shown in FIG. 7, in addition to the first embodiment, the determination control device 200 of the determination device 20 includes a history information addition unit 206 that is a unit realized as a computer function by a program. , When past removal or removal information (history information) is added to each pixel and stored in the storage device, and when the background edge hidden by the removed object is restored by the removal of the object The history information is read out so that it is not determined to be left behind. In addition, the background edge hidden by the left object is not taken away and judged in the same manner.
The other configuration of the determination apparatus 20 according to this embodiment is the same as that of the first embodiment shown in FIG.

Next, the procedure of the leaving or carrying-out determination process by the determination apparatus 20 of this embodiment is demonstrated with reference to FIG.
That is, first, a captured image of the electronic camera 10 is input to the leaving or carrying-out determination device 20 according to the present embodiment (S301), and stored in the storage device 22, and then an edge image of the input image is created (extracted) ( S302). A signed background difference image (background difference image = current image−background image) is created for the obtained edge image (S303). Subsequently, the leaving or removal determining unit 202 determines whether to leave or remove based on the signed background difference of the pixel value of the obtained edge image (S304), that is, (i) the background difference value is a positive value. If it is equal to or greater than the threshold, it is left behind, (ii) if the background difference value is negative and less than the threshold, it is left away, and (iii) otherwise, it is determined that there is no foreign object.

  Next, if there is a pixel to which history information is added by the history information adding means 206, this is determined as a stationary pixel with history information. At this time, leaving and taking away are distinguished and determined (S305). Here, the history of the stationary pixels with history information is determined. That is, here, the history information of the image change is added for each pixel, that is, for example, for each pixel, Info = 0 (initial value) and Info = 1 (there was left in the past). Since any history information of Info = 2 (has been taken away in the past) is added, the history information is determined (S306). Subsequently, the determined immovable pixel with history information is labeled (S307), the determination result is output (S308), the corresponding pixel of the background image is updated with the immovable pixel (S309), and the process returns to the initial state.

FIG. 9 is a flowchart for explaining the history determination processing in step S306.
That is, when it is determined that the pixel is a stationary pixel with history information (S401, YES) and the stationary pixel with the history information is determined to be left behind (S402, YES), the history information is removed (Info = 2: held in the past). If there has been a leave) (S403, YES), the history information is initialized (Info = 0) and changed to take-away determination (S404). For example, in FIG. 5B, the + background edge of the original background is determined to be left behind, but if the history information is left away (FIG. 5A), it is changed to the removal determination.
In step S403, if the history information is not carried away (S403, NO), the history information is updated (Info = 1: there has been left in the past), and the leaving determination is held (S405). For example, in FIG. 5A, the object edge is determined to be left behind, but if the history information is not left away, the determination to leave is held.

In step S402, the immovable pixel with history information is not left behind (S402, NO), that is, if the immovable pixel with history information is taken away and the history information is left behind (S406, YES), the history information is initialized. (Info = 0), the take-away determination is held (S407). For example, in FIG. 5B, the edge of the object is determined to be taken away, but if the history information is left behind (FIG. 5A), the removal determination is held.
In step S406, if the history information is not left behind (S406, NO), the history information is updated with (Info = 2: there has been taken away in the past), and is changed to a left-over determination. For example, in FIG. 5A, the + pattern edge of the background is determined to be removed, but if the history information is not left, it is changed to determination to leave.

Note that what is output in the stationary pixel determination process with history information is a stationary pixel image with history information.
Here, the non-moving pixel image with history information is, for example, a ternary image including any one of a leaving determination pixel (pixel) = 1, a leaving determination pixel (pixel) = 2, and an initial value = 0 for each pixel. It is. This immovable pixel image with history information is labeled with a 1 pixel and a 2 pixel in a labeling process, and finally a determination result is output.

Embodiment for exception 2 (third embodiment)
In this embodiment, as shown in FIG. 10, in addition to the first embodiment and the second embodiment, the determination control apparatus 200 has a pixel opening processing means 207 as a function realizing means by a computer program. And a closing processing unit 208, automatically determine a portion where the intensity of the background edge is strong and dense, create a binary image of the dense area of the edge and the area where the edge is not dense, and perform the following determination. That is,
(1) In the “leave or take away” flow (FIG. 3) of the first embodiment for dense pixels,
(I) If the background difference value is a positive value and equal to or greater than the threshold value, “take away” is set.
(Ii) If the background difference value is a negative value and less than the threshold value, “leave left” is set.
(Iii) Otherwise, it is not a foreign object.

(2) Regarding the dense pixels, the following changes are made in the flow of “determination of non-moving pixels with history information” in the second embodiment (FIG. 9).
(I) The immovable pixels determined to be left at the end of FIG. 9 are changed to “take away”.
(Ii) The immovable pixels determined to be removed at the end of FIG. 9 are changed to “leave”.
Note that there are the following two methods for defining dense pixels when using a binary image of an edge dense area.
(1) All pixels in the dense area of the edge are set as dense pixels.
(2) Pixels in the edge dense area and whose edge intensity is greater than or equal to the threshold are designated as dense pixels, but any of them may be used.

Hereinafter, a procedure of creating a binary image of a background edge dense area and an area other than the background edge dense area implemented by the determination apparatus 20 according to the present embodiment will be described with reference to a flowchart shown in FIG.
First, a captured image of the electronic camera 10 is input to the leaving or carrying-out determination device 20 according to the present embodiment (S501), and an edge image of the input image (background image) is created (S502). Next, the edge image intensity (pixel value intensity) is binarized in comparison with the threshold value (S503), and closing, that is, the binarized edge image is expanded n times and then contracted n times. Then, the process of combining the dense edges is performed (S504), and then the opening, that is, the closed edge image is contracted n times to remove isolated thin edges, and then the nth expansion process is performed to determine the size based on the size. Perform processing to return.
The third embodiment is realized using the binary image created by the above processing. Here, the binary image creation process is a process performed once when there is no foreign object in the video, and is not performed every frame.

FIG. 12 is a diagram for explaining the opening process and the closing process.
FIG. 12A shows a background edge binary image. The illustrated edge shows a non-crowded place and a crowded place. Next, each edge binary image is subjected to a closing process, that is, the edge binary image is expanded n times to combine the dense edges. FIG. 12B shows a binary image after expansion n times. Subsequently, the obtained binary image is shrunk n times to restore the original size, and the binary image shown in FIG. 12C is obtained. Here, opening, that is, removal of non-congested edges (isolated edges) that have contracted n times is performed. FIG. 12D is a binary image after removing isolated edges. Finally, the binary image after edge removal is expanded n times to return to the original size, and the process is terminated.
With the above processing, even if the edges of the background image are dense and the background image edge strength is larger than the object edge strength, the “removal or removal determination flow” is performed in the first embodiment for dense pixels. (FIG. 3) In the second embodiment, by changing the processing of the “non-moving pixel determination flow with dense information” (FIG. 9), it is determined whether the pixel (that is, a foreign object) is due to leaving or being removed. It can be judged correctly.

Next, another embodiment will be described.
Fourth Embodiment Using the background edge binary image created after the binarization of the flow shown in FIG. 11 (S503), regarding the background edge pixel, the “Leave or take away determination flow” of the first embodiment. (FIG. 3), the following changes are made.
(i) When the background difference value is a positive value and equal to or greater than the threshold value, it is determined as “not a foreign object”.
(ii) If the background difference value is negative and less than the threshold value, it is determined as “not a foreign object”.
(iii) In other cases, it is assumed that it is not a foreign object.
According to the above processing, pixels such as the + pattern edge (FIG. 5A (1)) of the floor existing in the original background are always determined not to be a foreign object, so in FIG. 5A, only the object edge leaving determination occurs. In FIG. 5B, only the object edge removal determination occurs, so that the problem of exception 1 can be solved.
Here, the background edge binary image creation processing is performed only once when there is no foreign object in the image, and is not performed every frame.

Fifth Embodiment In the fourth embodiment, the problem of exception 1 can be solved. However, if the edges are dense in the original background as shown in FIGS. 6A (1) and 6B (1). When an object is placed at a location, it may be impossible to detect the object because the pixel is always determined not to be a foreign object.
At this time, in addition to the fourth embodiment, the first implementation is performed with respect to the dense pixels by using the binary image of the dense area of the background edge and the area that is not created at the end of the flow shown in FIG. The following changes are made in the “Leave or take away determination flow” (FIG. 3) of the form. When the changes in the fourth embodiment are also described,
(1) First, using the background edge binary image,
(i) When the background difference value is a positive value and equal to or greater than the threshold value, it is determined as “not a foreign object”.
(ii) If the background difference value is negative and less than the threshold value, it is determined as “not a foreign object”.
(iii) In other cases, it is assumed that it is not a foreign object.
(2) Then, using the binary image of the area where the background edge is dense and the area where it is not,
(i) If the background difference value is a positive value and equal to or greater than the threshold value, “take away” is set.
(ii) If the background difference value is negative and less than the threshold value, “leave left” is assumed.
(iii) In other cases, it is assumed that it is not a foreign object.
According to the above processing, pixels such as the + pattern edge (FIG. 5A (1)) of the floor existing in the original background are always determined not to be a foreign object, so in FIG. 5A, only the object edge leaving determination occurs, In FIG. 5B, only the object edge removal determination occurs, and even when the object is placed at a location where the edges in FIGS. 6A and 6B are dense, it is correctly determined whether the object is due to leaving or removal. Therefore, exception 1 and exception 2 can be solved.
Here, the background edge binary image creation processing and the binary image creation processing of the area where the background edge is dense and the background edge are processing that is performed only once when there is no foreign object in the image, and is not performed every frame. .

  DESCRIPTION OF SYMBOLS 10 ... Electronic camera, 20 ... Determination apparatus, 200 ... Determination control apparatus, 201 ... Difference image creation means, 202 ... Abandonment or removal determination means, 203 ... Immovable pixel determination means 204 ... Labeling means, 205 ... Background image updating means, 206 ... History information adding means, 207 ... Opening processing means, 208 ... Closing processing means, 22 ... Storage device, 24 ... display device, 26 ... input device, 30 ... bus.

Claims (8)

A storage device for storing a background image of the surveillance area photographed by the photographing device;
A difference image creating means for creating a difference image between the background image and the current image of the monitoring area photographed by the photographing device;
A leaving or removal determining means for determining whether the difference image is due to leaving or removal based on the positive or negative sign of the pixel of the created difference image;
A device for determining whether to leave or take away. In the leaving or taking away judging device according to claim 1,
A leaving or carrying determination apparatus comprising background image updating means for updating the background image based on a determination result of the leaving or taking determination means. In the leaving or taking away judging device according to claim 2,
The background image includes a non-moving pixel determining unit that determines a pixel that has been left or removed by the leaving or taking-out determining unit as being a non-moving pixel when the pixel is left or left for a certain time or longer. The update means updates the corresponding pixel of the background image with the non-moving pixel, and the leaving or removal determining device. In the leaving or taking-away determination device according to claim 2 or 3,
When an edge image of a background image is included, history information adding means for adding history information of leaving or taking away to pixels of the edge image,
The leaving or removing determination unit is configured to determine whether a pixel has been removed or removed with reference to the history information of the stationary pixels. In the leaving or taking away judging device according to claim 4,
When the history information is taken away when the immovable pixel is left behind, the determination is changed to take away, and the history information is not taken away when the immovable pixel is left behind. If the history information is left behind in the determination that the non-moving pixel is taken away, the determination result of the leaving or removal determining means is maintained regardless of the history information, and the history information is determined when the non-moving pixel is determined to be removed. A device for determining whether to leave or take away when the device is left behind. A method for determining whether a device is left behind or taken away by a device for determining whether left behind or removed,
A storage step of storing a background image of the surveillance area captured by the imaging device;
A difference image creating step for creating a difference image between the background image and the current image of the monitoring area photographed by the photographing device;
A leaving or leaving determination step of determining whether the difference image is due to leaving or removal based on the positive or negative sign of the pixel of the created difference image;
A method for determining whether to leave or take away. In the leaving or carrying out judging method described in claim 6,
A method for determining whether to leave or take away, comprising: a background image updating step for updating the background image based on a determination result of the leaving or taking-out determining means.   6. A program for causing a computer of a leaving or removing determination apparatus according to claim 1 to function as each means of the leaving or removal determining apparatus.

Images