JP2009265827A - Object detection device and method, object detection system, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve highly precise object detection in response to the change of the environment condition around a camera including illumination conditions. <P>SOLUTION: A prediction region where the existence of an object is predicted and a non-prediction region where the absence of the object is predicted are specified in an input frame image. Then, a background image for contrast is generated, which is different in generation system between a region where difference from the prediction region is calculated and a region where difference from the non-prediction region is calculated. Especially, the region at a position equal to the non-prediction region of the background image for contrast is generated by selecting a small block background model image which is the most similar for each small block. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection apparatus, an object detection method, and an object detection system that detect an object using an acquired image of a camera. Moreover, it is related with the program corresponding to them.

  The detection and extraction processing of an object using an acquired image of a camera is generally realized by comparing two images. For this comparison, a difference between a background image prepared in advance and a currently acquired image is obtained and compared with a predetermined threshold value, or a difference between two temporally continuous images is obtained and compared with a predetermined threshold value. Differences between them are used. However, when this type of object detection method is used, if there is a change other than the object to be detected between the two images to be compared, highly accurate object detection becomes difficult. A typical factor is a change in lighting in the shooting environment of the camera. The shooting environment of the camera is illuminated by sunlight, illumination light of a lighting device, or the like, but the brightness and color of the acquired image change due to the change in illumination of the shooting environment. For example, when the brightness between two images to be compared differs due to a change in illumination, a difference value equal to or greater than a predetermined threshold value can be obtained not only for the object region that should be originally detected but also for the background region. This makes it difficult to distinguish the object from the background.

  Specifically, for example, if the lighting device is turned on / off while the room is being photographed, the illuminance of the illumination in the photographing environment can change significantly and instantaneously. When a sudden change in illuminance occurs in this way, not only when the background difference is used for detecting the object, but also when the difference between frames is used, highly accurate object detection is possible depending on the speed of change. become unable.

  In Patent Document 1, whether the result obtained by detecting the difference value is a steady change (for example, fluctuation of a tree or irregular reflection of the water surface) or an unsteady change (for example, movement of a person). ) Has been proposed based on the occurrence probability for each region, and an object detection method for detecting is proposed.

Japanese Patent Laid-Open No. 8-44874

  However, even if this method is used, unless the change due to turning on / off of the lighting device as described above is constant (for example, one that is frequently turned on / off at a predetermined cycle), There is a possibility of erroneous detection. And since the change of a normal imaging | photography environment tends to become a non-stationary change, erroneous detection may occur frequently and detection accuracy may fall.

  Accordingly, the present invention provides an object detection device, an object detection system, an object detection method, and a program that can realize highly accurate object detection in response to changes in ambient environmental conditions of the camera including illumination conditions. The purpose is to do.

  In order to achieve the above object, an object detection apparatus according to the present invention sequentially acquires detected images based on camera shooting, and captures the camera based on a difference result between the detected image and a background image for comparison. An object detection apparatus for detecting an object in an area, wherein a prediction area where the object in the detected image is predicted to exist and a non-prediction area which is an area other than the prediction area are specified The background image for comparison with different generation methods for a mask generation unit that generates a mask, an area where a difference is obtained from the prediction area specified by the mask, and an area where a difference is obtained from the non-prediction area A background image generation unit for comparison.

  This makes it possible to generate a contrast background image suitable for each region for each region. Therefore, it is possible to realize highly accurate object detection. The comparison background image suitable for each region indicates a comparison background image that enables appropriate difference processing to be performed. Further, the appropriate difference process indicates a difference process in which erroneous detection corresponding to a change in the ambient environment condition of the camera is unlikely to occur and the detection accuracy of the target object is high.

  Specifically, for example, the comparison background image generation unit, based on the non-prediction area of the detected image, an area in which a difference from the prediction area of the comparison background image is obtained, and the non-prediction area; Each of the regions where the difference is calculated is generated.

  Therefore, when the comparison background image is generated based on the non-predicted region of the detected image, it is possible to generate the comparison background image that effectively reflects the ambient environment conditions of the camera.

  Furthermore, specifically, for example, an image holding unit that holds a plurality of images shot in a state where the object is not included in the shooting region, and a plurality of images from the plurality of images held in the image holding unit. A model generation unit that generates the model image, and a model holding unit that holds the model image generated by the model generation unit, wherein the comparison background image generation unit is held in the model holding unit The comparison background image is generated using the model image, and at least the non-prediction region of the comparison background image is generated using the model image similar to the non-prediction region of the detected image. And a generation method for generating an area for which a difference is obtained.

  Thereby, a contrast background image is generated using a model image similar to the non-predicted region of the detected image. Therefore, it is possible to generate a background image for comparison that makes it possible to realize highly accurate object detection.

  And, for example, a plurality of small block background model images corresponding to each of small blocks that are subdivided areas of the detected image are held in the model holding unit, and the comparison background image generation unit is used for the comparison When generating a region in which a difference from the non-predicted region of the background image is obtained, the most similar small block background model image is obtained from the model holding unit for each small block of the non-predicted region of the detected image. A generation method is used in which each is selected and a region in which a difference is obtained from the non-predicted region of the comparison background image is generated using the selected small block background model image.

  If comprised in this way, the small block background model image most similar to the non-prediction area | region of a to-be-detected image can be selected for every small block. The small block background model image is included in the above model image.

  More specifically, for example, a plurality of background model images corresponding to the whole of the detected image are held in the model holding unit, and the comparison background image generation unit is connected to the prediction region of the comparison background image. When generating a region for which a difference is calculated, the background model image having the most similar region at the same position as the non-predicted region is selected from the model holding unit, and is equal to the predicted region from the selected background model image A generation method for generating a region where a difference from the non-predicted region of the background image for comparison is obtained by extracting and using the region of the position is used.

  As described above, when a background model image similar to the non-predicted region of the detected image is selected and only the region at the same position as the predicted region is extracted and used, the difference between the predicted region of the background image for comparison is obtained. It is possible to effectively reflect the surrounding environmental conditions of the camera on the area. The background model image is included in the above model image.

  Alternatively, for example, the comparison background image is selected and combined from the plurality of small block background model images held in the model holding unit by the comparison background image generation unit, A model synthesis history holding unit that holds a synthesis history that is a combination history of the small block background model images used in the background image for comparison output from the background image generation unit for comparison; When the image generation unit generates an area in which a difference is obtained from the prediction area of the comparison background image, the synthesis history and the non-prediction area of the comparison background image form an area in which a difference is obtained. The small block used for a region for which a difference is obtained from the prediction region of the background image for comparison based on a combination of small block background model images. Using the generation method of generating select a click background model image.

  Note that when determining a combination of a small block background model image in a region for which a difference is obtained from the prediction region of the background image for comparison, a small block background model image that forms a region in which a difference is obtained from the non-predicted region of the background image for comparison As a condition, a combination of small block background images with the highest possibility (occurrence of occurrence probability) may be selected from the synthesis history.

  Further, for example, the model synthesis history holding unit obtains a correlation about the selection result of the small block background image between the small blocks based on the synthesis history, and includes this information in the synthesis history, and for the comparison The background image generation unit selects the small block background image when selecting the small block background model image in order to generate an area in which a difference is obtained from the prediction area of the comparison background image. Generating by selecting and generating the small block background model image in a region where a difference is obtained from the prediction region of the background image for comparison based on the small block background model image used for the small block having a high correlation with the block Use the method.

  If comprised in this way, it will become possible to select a small block background model image based on the small block background model image selected in the small block with a high correlation regarding the small block which wants to select a model. . Therefore, it is possible to reduce the data to be held and the amount of calculation.

  In order to achieve the above object, an object detection system according to the present invention includes a camera that obtains an image to be detected by photographing, and the object detection device described above.

  In order to achieve the above object, an object detection method according to the present invention is used in a system including a camera that obtains a detected image by photographing and a display device that displays an image based on the detected image, and An object detection method for sequentially acquiring the detected images from a camera and detecting an object in a shooting region of the camera based on a difference result between the detected image and a background image for comparison, When generating a background image, the prediction region in which the target object in the detected image is predicted to exist and a non-prediction region that is a region other than the prediction region are specified, and the comparison background image The generation method is different between a non-predicted region and a region in which a difference is obtained, and a region in which a difference is obtained from the predicted region.

  In order to achieve the above object, a program according to the present invention detects an object in a shooting area of the camera based on a difference result between a detected image sequentially obtained by shooting with the camera and a background image for comparison. A program for causing a computer to realize a function, the function of specifying a prediction region where the target in the detected image is predicted to exist and a non-prediction region other than the prediction region, A function of generating the contrast background image by using a generation method different between the non-predicted region and a region in which a difference is obtained and a region in which a difference is obtained from the predicted region. And

  According to the present invention, it is possible to realize highly accurate object detection in response to changes in the ambient environmental conditions of the camera including illumination conditions.

  The significance or effect of the present invention will become more apparent from the following description of embodiments. However, the following embodiment is merely one embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the following embodiment. .

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle. First, the configuration and operation of the object detection system according to the embodiment of the present invention will be described, and then specific examples of each component will be described.

<< Overall structure >>
First, the overall configuration of an object detection system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall block diagram of an object detection system according to an embodiment of the present invention. The object detection system includes a camera 1, a detection processing unit 2, and a display unit 3, and a detection target that can exist in a shooting area of the camera 1 (in other words, a field of view of the camera 1) based on a captured image of the camera 1. Detect objects. In the following description, it is assumed that the detection target (hereinafter simply referred to as “target”) is mainly a person.

  As the camera 1, a camera using a CCD (Charge Coupled Devices) or a camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor is used. The detection processing unit 2 is formed using an integrated circuit or the like. The display unit 3 is a display device formed from a liquid crystal display panel or the like.

  The camera 1 sequentially captures images at a predetermined frame period. Each image obtained for each frame period by this sequential shooting is hereinafter referred to as a “frame image”. Image data representing the frame image is sent to the detection processing unit 2. It is assumed that the shooting area of the camera 1 is fixed. In the present embodiment, it is assumed that the shooting area of the camera 1 is a room such as a house, and the light source that illuminates the shooting area of the camera 1 includes a lighting device such as a fluorescent lamp and sunlight. Assume a case. However, the imaging region of the camera 1 can be outdoor, and the number and type of light sources are arbitrary. In some cases, the photographing region of the camera 1 is temporarily illuminated by sunlight and outside light (for example, neon light) other than illumination light of a lighting device provided indoors. It can be seen as a type of light source that illuminates the shooting area.

  The detection processing unit 2 includes each part referred to by reference numerals 11 to 14. The function of each part of the detection processing unit 2 will be schematically described. Each part in the detection processing unit 2 can freely refer to each data generated in another part in the detection processing unit 2 as necessary.

  The image acquisition unit 11 acquires each frame image by sequentially reading image data output from the camera 1. The difference processing unit 12 performs a difference process between the frame image and the comparison background image supplied from the comparison background image supply unit 14 to extract an object in the frame image. The display control unit 13 generates and outputs display image data so that an image in which the extraction result is reflected in the frame image is displayed on the display unit 3. The “contrast background image” is an image that shows only the background of the shooting area of the camera 1 (that is, an image that does not include an object), and is an image for performing difference processing with a frame image.

  FIG. 2 shows an example of a difference result image, which is an example of an extraction result obtained by performing difference processing on the frame image, the contrast background image, and both images. A person as an object is extracted from the difference result image. For example, a difference result image can be obtained using a so-called background difference method. In the frame image of FIG. 2, a person standing upright along the corner of the room, the floor of the room, the two walls of the room, and one of the walls is depicted. In addition, this difference result image is represented by binary values (each value is expressed in white and black), and the region that is one value (white region) is the region of the object and the other value. An area (black area) indicates an area that is not an object. Hereinafter, when a difference result image is shown, it will be shown by the same method as this difference result image.

  FIG. 3 shows an example of a display image displayed on the display unit 3. In FIG. 3, reference numeral 200 is a display image, and a broken-line square frame to which reference numeral 201 is attached is a frame that surrounds the object area that is displayed in a superimposed manner. The display image is generated by the display control unit 13 based on the extraction result input from the difference processing unit 12 and the frame image input from the image acquisition unit.

  A frame image from which an object is to be extracted is particularly referred to as a “detected image”. Each frame image is sequentially sent to the difference processing unit 12 and handled as a detected image. In the following description, a frame image that is currently handled as a detected image is referred to as a current frame image or a current frame image, and a frame from which the frame image is obtained is referred to as a current frame or a current frame. The descriptions of the past, the previous time, and the next time, which will be described later, are based on the current frame. In this specification, the term “frame” is not interpreted as a word meaning an image but as a word representing time. The frame is also referred to as a shooting timing or a shooting section for obtaining a frame image. For example, when a subject at a certain focused timing is photographed by the camera 1 and the frame image obtained by the photographing is the current frame image, the term “current frame” and the term “current frame” are described above. Means. In addition, the object detection system according to the embodiment of the present invention includes a comparison background image supply unit that generates and outputs a comparison background image using different generation methods. The “generation method” indicates a certain method for generating. The comparison background image supply unit will be described below.

<< Contrast Background Image Supply Unit >>
First and second configuration examples of the background image supply unit for comparison will be described with reference to the drawings. First, a basic configuration common to each configuration example will be described, and then each configuration example will be described.

<Basic configuration of contrast background image supply unit>
FIG. 4 is a block diagram showing a basic configuration of the comparison background image supply unit of FIG. As shown in FIG. 4, the comparison background image supply unit 14 includes a mask generation unit 15, an image holding unit 16, a model generation unit 17, a model holding unit 18, and a comparison background image generation unit 19. Prepare.

  The mask generation unit 15 generates and outputs a “mask” based on the extraction result (for example, the difference result image shown in FIG. 2) output from the difference processing unit 12. This mask is shown in FIG. FIG. 5 is a schematic diagram of a mask output from the mask generation unit of FIG. 4 and shows a mask image which is an example of a mask. Further, the mask image shown in FIG. 5 corresponds to the difference result image shown in FIG.

  As shown in FIG. 5, the mask image 210 includes a prediction area 212 and a non-prediction area 211. The prediction area 212 is an area where an object detected in the difference result of the current frame image is predicted to exist in the next frame image. In short, the prediction area 212 is set to be slightly larger than the area of the object detected from the previous frame image. On the other hand, the non-prediction area 211 is an area other than the prediction area 212 in the mask image 210. That is, it is an area where the object detected in the previous frame image is predicted not to exist in the current frame image. Here, “prediction” means, for example, determining that the possibility is high.

  Further, in order to simplify the description of the prediction region 212 and the non-prediction region 211, the mask has been described as the mask image 210, but the mask is not limited to the mask image 210. In particular, as long as the contrast background image generation unit 19 can identify the non-predicted region and the predicted region in the current frame image based on the input mask, any format of data is possible. It doesn't matter. However, in the following description, in order to simplify the description, it is assumed that the mask is the mask image 210.

  In addition, the mask generation unit 15 may be an extraction result output from the difference processing unit 12 other than the difference result image (an image other than the image obtained by binarizing the object region and the other region). It is possible to generate a mask. However, for the sake of simplicity, the following description will be made assuming that the extraction result is expressed as a difference result image and the mask is obtained based on the difference result image.

  The model generation unit 17 generates a “statistic model” based on the sample background image held in the image holding unit 16. This statistical model will be described with reference to FIG. FIG. 6 is a diagram for explaining the statistical model. In order to create a statistical model, a number of frame images are acquired in advance by the camera 1 and held in the image holding unit 16 prior to actual detection of an object. Each frame image acquired in advance is referred to as a sample background image. It is assumed that no object is included in the shooting area of the camera 1 when shooting all the sample background images. On the other hand, it is assumed that a plurality of sample background images obtained include a plurality of sample background images having different illumination conditions around the camera 1 (hereinafter simply referred to as illumination conditions). However, a plurality of sample background images captured under the same illumination conditions can also be included in many sample background images.

  In order to realize this, for example, a sample background image acquisition period is provided prior to actual detection of an object, and during that period, the camera 1 acquires a large number of frame images as sample background images. During this period, power on / off of the illumination device that illuminates the imaging region of the camera 1, changes in the illuminance of sunlight that illuminates the imaging region of the camera 1, and the like have occurred. The illumination condition means the state of the light source that illuminates the shooting area of the camera 1, and is also referred to as an illumination state. If the illumination conditions are different, the illuminance and color temperature of the light source that illuminates the imaging region of the camera 1 are different, and the brightness and color of the obtained frame image are also different.

  As shown in FIG. 6, a plurality of statistical models are generated, and the generated plurality of statistical models are held in the model holding unit 18. The number of statistical models to be generated is represented by N (N is an integer of 2 or more). Each statistical model is formed from one background model image obtained by modeling a background image and a deviation list referred to in the difference processing in the difference processing unit 12. A background model image of one statistical model is basically generated by collecting and averaging similar sample background images, and the brightness and color of the background model image differ between different statistical models.

  Note that the background image means an image photographed by the camera 1 in a state where no object is present in the photographing region of the camera 1, in other words, a frame image not including the object. The background model is a combination of a background model image obtained by modeling image information about a shooting region that does not include an object and other information. The statistical model is a kind of background model.

  Further, the model generation unit 17 generates a small block statistical model in addition to the statistical model described above. This small block statistical model will be described with reference to FIG. FIG. 7 is a diagram for explaining the small block statistical model. As shown in FIG. 7, the small block statistical model is generated based on the sample images held in the image holding unit 16 classified into small blocks.

  The small block is a frame image that is subdivided, and is composed of, for example, 8 × 8 pixels. In this example, if the frame image is composed of 320 × 240 pixels, the total number of small blocks is 40 × 30 = 1200. Various processes can also be performed in units of small blocks (specific examples of processes performed in units of small blocks are shown in the examples described later). Therefore, if the prediction area 212 and the non-prediction area 211 of the mask image 210 described above are divided in units of small blocks, it may be preferable for the convenience of processing. Hereinafter, in order to simplify the description, it is assumed that the prediction area 212 and the non-prediction area 211 are divided in units of small blocks.

  A plurality of small block statistical models are generated in the same manner as the statistical model, and the generated small block statistical models are held in the model holding unit 18. Here, if the frame image is divided into M small blocks and P small block statistical models are created for each small block, a total of M × P small block statistical models are generated. (M and P are integers of 2 or more). Each small block statistical model is formed from one small block background model image obtained by modeling a certain small block of the background image, and a deviation list referred to in the difference processing in the difference processing unit 12. The Similar to the statistical model, the background model image of one small block statistical model is basically generated by collecting and averaging small blocks of similar sample background images, and the small block background between different small block statistical models. The model image brightness and color are different.

  The comparison background image generation unit 19 obtains a statistical model and a small block statistical model from the model holding unit 18 and uses at least one of the background model image and the small block background model image included in the statistical model and the small block background model image. Is generated. At this time, the non-predicted region and the predicted region of the current frame image are identified based on the mask input from the mask generating unit 15, and a comparison background image is generated by using a different generation method for each region. To do. Thereby, it is possible to generate a background image for comparison suitable for each region. Note that the comparison background image suitable for each area indicates a comparison background image that enables appropriate difference processing to be performed. Further, the appropriate difference process indicates a difference process in which erroneous detection corresponding to a change in the ambient environment condition of the camera is unlikely to occur and the detection accuracy of the target object is high.

  The comparison background image generation unit 19 selects the model image based on the non-prediction region of the current frame image, and generates a comparison background image. The non-predicted area of the current frame image is an area that is highly likely to be the same as the held background, and is an area that is suitable for grasping the surrounding environmental conditions (background state) of the current camera. Therefore, by using this area, it is possible to generate a background image for comparison that can perform appropriate difference processing. A specific example of the generation method used when generating the comparison background image and a specific configuration example of the comparison background image supply unit will be described below.

<First Configuration Example of Comparison Background Image Supply Unit>
A first configuration example of the contrast background image supply unit will be described with reference to the drawings. FIG. 8 is a block diagram illustrating a first configuration example of the comparison background image supply unit of FIG. As shown in FIG. 8, the configuration of the comparison background image supply unit 14a is the same as the basic configuration of FIG. 4 described above. However, the comparison background image generation unit 19 a includes a prediction region synthesis unit 191 and a non-prediction region synthesis unit 192. Note that the prediction region synthesis unit 191 and the non-prediction region synthesis unit 192 may be integrated without being separated, but are separated for convenience of explanation, and each operation will be described below.

  The operation of the comparison background image supply unit 14a shown in FIG. 8 will be described with reference to FIG. FIG. 9 is a schematic diagram for explaining the operation of the first configuration example of the comparison background image supply unit in FIG. 8. In particular, it shows a method for generating a comparison background image 314 for the t-th frame image 310 when the difference result image 301 and the mask image 302 are obtained by the object detection processing of the t-1th frame image 300. is there.

  As shown in FIG. 9, when the mask image 302 is obtained by the object detection process of the (t−1) -th frame image 300, the non-prediction region 311 (hereinafter, “t”) of the t-th frame image is obtained by the non-prediction region 303 of the mask image 302. And the prediction region (hereinafter referred to as the t-th frame image [prediction region]) are specified. Then, the non-predictive region synthesizing unit 192 is based on the t-th frame image [non-predictive region] 311, and the difference between the non-predictive region image and the t-th frame image [non-predictive region] 311 is obtained in the subsequent difference processing A region 312 (hereinafter referred to as a contrast background image [non-predicted region]) is generated.

  At this time, the non-predictive region synthesizing unit 192 selects a small block background model image (FIG. 7) that is most similar to the t-th frame image [non-predictive region] 311 for each small block. An image [non-predicted region] 312 is generated. Then, one of the P small block background model images is selected for each small block. The comparison background image [non-predicted region] 312 shown in FIG. 9 is a combination of the selected small block background model images. Note that “similar” means that the pixel values of the respective pixels of the two images to be compared (for example, various signal values such as a luminance signal, a color difference signal, and an RGB signal) are close. For example, a difference between pixel values of two pixels to be compared is compared with a certain threshold value, and when the difference is smaller than a certain threshold value, the pixel values are close. That is, it shall be similar.

  On the other hand, based on the t-th frame image [non-predicted region] 311, the prediction region synthesis unit 191 determines a region 313 (in the background image for comparison in which a difference from the t-th frame image [prediction region] is obtained in the subsequent difference processing. Hereinafter, a contrast background image [prediction area] is generated. In particular, the prediction region synthesis unit 191 generates the comparison background image [prediction region] 313 by a generation method of selecting a background model image (FIG. 6) that is most similar to the t-th frame image [non-prediction region] 311. At this time, a similar determination is made for the region at the same position as the non-predicted region 303 of the background model image and the t-th frame image [non-predicted region] 311, and the most similar among the N background model images. One will be selected. A comparison background image [prediction area] 313 shown in FIG. 9 shows an extracted area of the selected background model image at a position equal to the prediction area 304.

  As described above, the background image for comparison 314 is obtained by combining the background image for comparison [non-prediction region] 312 and the background image for prediction [prediction region] 313 obtained by different generation methods. Then, the difference processing unit 12 performs a difference process between the comparison background image 314 and the t-th frame image 310, thereby performing an object detection process. Then, a difference result image 315 is generated and output by the difference processing unit 12 as an extraction result, and the mask generation unit 15 generates a mask image 316 based on the difference result image 315.

  The object detection process for the t + 1-th frame image 320 is the same as the object detection process for the t-th frame image 310. In the object detection process of the (t + 1) -th frame image 320, the prediction area 318 and the non-prediction area 317 of the previous (t-th) mask image 316 are used.

  The comparison background image [non-prediction area] 312 is obtained by selecting and combining the small block background model images most similar to the t-th frame image [non-prediction area] 311 for each small block. For this reason, it is possible to prevent the object from being erroneously detected from this region when the object detection process is performed. For example, even if the surrounding environment conditions of the camera change and become partially bright, the most similar small block background model image is allocated to that portion, so the changes in the surrounding environment conditions of the camera Is erroneously detected as an object. Therefore, by using the comparison background image 314 generated by this method, it is possible to cope with a partial change in the ambient environmental conditions of the camera.

  On the other hand, the contrast background image [prediction region] 313 is obtained by selecting the statistical model most similar to the t-th frame image [non-prediction region] 311. In other words, since there is a possibility that an object exists, the object of the t-th frame image 310 is selected for an area (predicted area) in which a similar small block statistical model cannot be selected and used as described above. This is achieved by estimating the background state of the t-th frame image 310 from a region that is predicted not to exist (non-predicted region) and selecting a statistical model corresponding to the estimated state. Therefore, a background model image suitable for the t-th frame image 310 can be used as the comparison background image [prediction region] 313. Therefore, it is possible to generate a comparison background image 314 that can perform highly accurate object detection.

<Second Configuration Example of Comparison Background Image Supply Unit>
Next, a second configuration example of the comparison background image supply unit will be described with reference to the drawings. FIG. 10 is a block diagram illustrating a second configuration example of the comparison background image supply unit of FIG. As shown in FIG. 10, the configuration of the comparison background image supply unit 14b is the same as the basic configuration of FIG. 4 and the first configuration example of FIG. However, in this configuration example, the model synthesis history holding unit 20 is provided. Note that the non-predictive region synthesis unit 192 of the comparison background image generation unit 19b performs the same operation as that in the first configuration example, and thus is denoted by the same reference numeral. On the other hand, since the prediction region synthesis unit 193 operates differently from the prediction region synthesis unit 191 of the first configuration example, different numbers are assigned.

  The model synthesis history holding unit 20 holds a history of the configuration of the comparison background image output from the comparison background image generation unit 19b (hereinafter referred to as a synthesis history). Then, based on the synthesis history held in the model synthesis history holding unit 20, the prediction area synthesis unit 193 generates a comparison background image area in which the difference between the prediction area of the detected image and the difference is obtained. Details of this operation will be described below with reference to FIG.

  FIG. 11 is a schematic diagram for explaining the operation of the second configuration example of the comparison background image supply unit 14b of FIG. 10, and corresponds to FIG. 9 shown in the first configuration example. Since FIG. 9 and FIG. 11 are different only in the generation method used for generating the background image for comparison, this portion will be mainly described. Further, the detailed description of other parts is omitted because it is the same.

  As described above, the operation of the non-predictive region synthesis unit 192 is the same as that shown in the first configuration example. On the other hand, the prediction region synthesis unit 193 selects the small block background model image of the comparison background image [prediction region] 413 based on the combination of the small block background model images selected in the comparison background image [non-prediction region] 312. The comparison background image [prediction region] 413 is generated according to the generation method to be selected. Further, the model synthesis history holding unit 20 holds a history (combination history) of the combination of the small block background model images of the background image for comparison output previously, and the prediction region synthesis unit 193 also refers to the synthesis history. . For example, the prediction region synthesis unit 193 recognizes the combination of the small block background model images selected in the comparison background image [non-prediction region] 312 and refers to the synthesis history to generate a blank prediction region. Then, a generation method for selecting a combination of small block background model images that has the highest occurrence probability is used.

  The comparison background image 414 is obtained by combining the comparison background image [prediction area] 413 obtained by the above generation method and the comparison background image [non-prediction area] 312. The comparison background image 414 obtained by this method is constituted by a small block background model image. Then, similarly to the first configuration example, the object detection process is performed using the comparison background image 414.

  Since the method of generating the comparison background image [non-predictive region] 312 by the non-predictive region synthesis unit 192 is the same as that in the first configuration example, the same effect can be obtained in this configuration example. That is, by using the comparison background image 414 generated by this method, it is possible to cope with a partial change in the ambient environment conditions of the camera.

  On the other hand, the comparison background image [prediction region] 413 is a comparison background image [based on the synthesis history of the comparison background image output before the t-th image and the t-th frame image [non-prediction region] 411. Non-prediction area] 412 and a small block background model image. Therefore, as in the first configuration example, the comparison background image [predicted region] 413 is obtained based on the region (non-predicted region) where the target object of the t-th frame image 310 is predicted not to exist. Therefore, it is possible to obtain a comparison background image [prediction region] 413 suitable for the t-th frame image 310.

  Further, the background image for comparison [prediction region] 413 is formed by selecting the small block background model image so that the combination is most likely (with a high occurrence probability) with reference to the synthesis history. Therefore, it is possible to generate a combination of small block background model images even in the prediction region. Then, by accumulating the history, it is possible to increase the possibility of obtaining the comparison background image 414 more suitable for the t-th frame image 310.

  In this configuration example, based on the fact that a certain small block background model image is selected for a certain small block of the comparison background image [prediction region] 413, It is assumed that the small block background model image of the small block is selected. Even in such a case, these are sequentially selected based on the combination of the small block background model images selected in the comparison background image [non-prediction area] 312 and the synthesis history. It can be said that it was selected based on.

  Further, only the combination of the small block background model images used in the comparison background image [non-prediction area] 312 may be recorded in the synthesis history, and the comparison background image 414 including the estimated and selected ones may be recorded. All combinations of the small block background model images used in the above may be recorded. In the former case, since only the combination obtained by directly comparing with the actually obtained image can be recorded, the information stored in the synthesis history can be made more accurate. On the other hand, in the latter, since data corresponding to the entire frame image is always stored, there is no missing combination. Therefore, it is possible to store all the mutual relationships between the small blocks in the synthesis history, and to increase the data amount of the synthesis history in a short time.

<< Example >>
Hereinafter, first to sixth examples will be described as examples for explaining more detailed operations and configurations of the above-described object detection system. The matters described in one embodiment can be applied to other embodiments as long as no contradiction arises.

<First embodiment>
First, the first embodiment will be described. In the first embodiment, an example of a method for setting the prediction region 212 of the mask image 210 illustrated in FIG. 5 will be described.

  For example, when the mask image 210 shown in FIG. 5 is created from the difference result image shown in FIG. 2, first, an area (for example, a pixel group having a certain pixel value) indicating that an object is detected in the difference result image. The position of the area itself or the area surrounding the area with a rectangular area or the like is specified as the position of the object area. On the other hand, the position of the area other than the object area is specified as the position of the background area. And the prediction area | region 212 and the non-prediction area | region 211 are set based on the position of the target object area | region and background area which were identified in this way. In particular, the prediction area 212 is set as an area obtained by slightly increasing the object area specified as described above.

  The prediction area 212 can be set in consideration of the moving speed (for example, walking speed) of an object (for example, a person). In this case, an area that can exist even if the object moves at a predetermined moving speed from the object area specified in the current frame image is set as the prediction area 212. It is also possible to set the prediction region 212 using an optical flow (apparent speed on the screen). The optical flow of the region including the target object is, for example, matching a part (or all) of the target object in a plurality of temporally continuous frame images, or extraction output from the difference processing unit 12 It can be obtained by using the result. Then, an area where the object can exist in the next frame image is obtained by the optical flow of the object of the current frame image (or the current frame image and the past frame image), and set as the prediction area 212. In addition, the setting method of these prediction area | regions 212 is only an example, and you may set the prediction area | region 212 by another method.

  If the prediction region 212 is set by the method as described above, it is possible to increase the possibility that an object may exist in the prediction region 212 of the next frame image. Also, it is possible to prevent the prediction region 212 from being unnecessarily enlarged and the non-prediction region 211 from becoming small.

  Note that the mask image 210 is not generated when the object processing apparatus is activated. In this case, for example, for a frame image that is input first after activation, all regions may be set as non-predicted regions. However, when setting in this way, it is assumed that there is no object in the imaging area of the camera 1 at the time of activation.

<Second embodiment>
Next, a second embodiment will be described. In the second embodiment, an example of a method for generating the statistical model and the small block statistical model shown in FIGS. 6 and 7 will be described.

  As described above, a large number of sample background images are acquired in advance by the camera 1. The model generation unit 17 performs clustering and classifies the large number of sample background images to obtain the statistical model shown in FIG. On the other hand, by classifying the sample background image for each small block and classifying the sample background image for each small block for each corresponding small block, the small block statistical model shown in FIG. 7 is obtained.

  As a clustering method, for example, hierarchical clustering can be employed. In other words, processing is performed in which similar sample background images are grouped to form clusters, and similar clusters are merged until the total number of clusters is N for a statistical model and P for a small block statistical model. Repeat. Thereby, each sample background image is classified into one of N (P) clusters. A collection of sample background images from which a statistical model is generated is collectively referred to as a sample background image group. A collection of sample background images classified for each small block that is a generation source of the small block statistical model is also referred to as a sample background image group for each small block. Note that the statistical model and the small block statistical model can be clustered by the same method. Therefore, the small block statistical model will be mainly described below, and the detailed description of the statistical model will be omitted.

  Although any known method can be used as the clustering method, one method is exemplified below. FIG. 12 shows a clustering state of a certain small block statistical model when ten small block sample background images are provided for each small block and P = 4. First, each of ten predetermined small block sample background images is considered as a cluster composed of one element, and the similarity between different clusters is evaluated. Treats the distance between clusters in a multidimensional vector space whose vector values are determined by image features as an index of similarity, and sequentially merges the clusters with the shortest distance between clusters (that is, clusters with the highest similarity). The clustering is advanced. The degree of similarity can be evaluated based on a feature vector, which will be described later in the third embodiment, or can be evaluated by employing any known method for evaluating the degree of similarity between images.

For example, clustering is performed using the Ward method. The Ward method is a method of assigning a weight to the number of elements of a cluster included in the focused cluster as an element when evaluating the degree of similarity between the clusters. For example, the similarity d _xc between the cluster C and the cluster X including the cluster A and the cluster B as elements is calculated according to the following equation (1). Here, d _xa, d _xb and d _ab represent respectively, the similarity of the clusters X and cluster A, the similarity of the similarity and the cluster A and the cluster B of the cluster X and Cluster B, n _x, n _a, n _b and n _c are the number of cluster elements included in the cluster X, the number of cluster elements included in the cluster A, and the number of cluster elements included in the cluster B, respectively. And the number of cluster elements included in the cluster C as elements (in this example, 2).

  When the clustering advances until the total number of clusters reaches P, the clustering is terminated and P small block statistical models are created. For example, when P = 4 and the first to fourth clusters are finally generated, the kth small block statistical model is based on all small block sample background images included in the kth cluster. (Where k is an integer from 1 to 4). Each small block statistical model has one small block background model image and a deviation list as described with reference to FIG.

A method for generating a small block background model image and a deviation list forming a small block statistical model will be described by focusing on one small block statistical model. For the sake of concrete explanation, it is assumed that all the small block sample background images included in the cluster corresponding to the focused small block statistical model are composed of the first to third small block sample background images. The small block background model image of the focused small block statistical model is an image obtained by averaging the first to third small block sample background images. That is, as shown in FIG. 13, the R, G, and B signal values of the pixel at a certain coordinate position in the first small block sample background image are represented by R ₁ , G _1, and B ₁ , respectively, and , The R, G, and B signal values of the pixels in the second small block sample background image at the same coordinate position are denoted by R ₂ , G _2, and B ₂ , respectively, and at the same coordinate position, When the R, G, and B signal values of the pixels in the three small block sample background images are represented by R ₃ , G _3, and B ₃ , respectively, the R, G, and B of the pixels in the background model image at the same coordinate position The G and B signal values are (R ₁ + R ₂ + R ₃ ) / 3, (G ₁ + G ₂ + G ₃ ) / 3 and (B ₁ + B ₂ + B ₃ ) / 3, respectively.

In accordance with the generation of the small block background model image, the model generation unit 17 sets the standard deviation of the pixel values of the first to third small block sample background images for each pixel and each of the R, G, and B signal values. To calculate. For example, for the pixel at the coordinate position noted above, the standard deviations σ _R , σ _G, and σ _B for the R, G, and B signal values are determined according to the following equations (2a), (2b), and (2c). Calculated.

A list obtained by calculating and listing the standard deviations σ _R , σ _G, and σ _B for all the pixels in the small block (in other words, for all the coordinate positions in the small block) is called a deviation list. A combination of a small block background model image based on the first to third small block sample background images and a deviation list is generated as a small block statistical model (see FIG. 7). It should be noted that the background model image and the standard deviation provided in the statistical model whose description is omitted can also be generated by the same method as described above.

<Third embodiment>
Next, a third embodiment will be described. In the third embodiment, for example, it is used when the most similar small block background model image is selected for each small block of the t-th frame image [non-predicted region] 311 shown in FIGS. 9 and 11. An example of a comparison method that can be used will be described. The following comparison method can also be used when selecting the most similar background model image for the t-th frame image [non-predicted region] 311 shown in FIG. In the following, the comparison method using the small block background model image will be mainly described, and the detailed description of the comparison method using the background model image will be omitted as the same thing.

"First comparison method"
The first comparison method will be described. As shown in FIGS. 9 and 11, when generating the comparison background image [non-predicted region] 312, the t-th frame image [non-predicted region] 311 and the P small images held in the model holding unit 18 are used. Each of the block background model images is compared for each small block. Now, paying attention to one small block statistical model of a certain small block, the focused small block statistical model, and an image of a certain small block in the t-th frame image [non-predicted region] 311 corresponding thereto, A comparison method will be described.

  First, an image of a certain small block in the t-th frame image [non-predicted region] 311 and a small block background model image of the focused small block statistical model are displayed for each pixel and for each R, G, and B signal value. Compare. Then, a pixel having a relatively large difference in pixel value (that is, a different pixel) and a pixel having a relatively small difference (that is, a non-different pixel) are discriminated for each pixel.

Specifically, the R, G, and B signal values of the pixel at a certain coordinate position of a certain small block in the t-th frame image [non-predicted region] 311 are respectively R _CF , G _CF, and B _CF. When the R, G and B signal values of the pixels in the small block background model image of the small block statistical model at the same coordinate position are represented by R _BM , G _BM and B _BM , respectively, the following 3 It is determined whether or not two inequalities (3a), (3b), and (3c) hold. If all the three inequalities are satisfied, it is determined that the pixel at the focused coordinate position is a non-difference pixel. If any one of the three inequalities is not satisfied, the pixel at the focused coordinate position is determined. It is determined that the pixel is a different pixel. At this time, σ _R , σ _G, and σ _B in the inequalities (3a), (3b), and (3c) are R, G, and B signals for the coordinate coordinates of interest based on the deviation list of the focused small block statistical model. The standard deviation of values. Α ₁ is a predetermined positive coefficient. As described above, after classifying all the pixels into different pixels or non-different pixels, the number of different pixels is counted.

  The number of different pixels is counted between a certain small block in the t-th frame image [non-predicted region] 311 and each of the P small block statistical models held in the model generation unit 17. The small block statistical model having the smallest number is a small block statistical model including a small block background model image that is most similar to a certain small block in the t-th frame image [non-predicted region] 311. to decide.

In addition, also when comparing a background model image, it can compare for every corresponding pixel by the same method. In this case, it is not restricted to the method of comparing for every small block as mentioned above. It is also possible to compare the entire frame images. Also, the present invention can be applied to the difference processing between the current frame image and the contrast background image in the difference processing unit 12. When applied to the difference processing performed by the difference processing unit 12, the value of α ₁ shown in the inequalities (3a), (3b), and (3c) may be different. Further, R _BM , G _BM and B _BM indicating the signal values of the pixels in the small block background model image, and the standard deviations σ _R , σ _G and σ _B of the small block statistical model are used as appropriate. .

"Second comparison method"
The second comparison method will be described. Note that when the second comparison method is adopted, the statistical model shown in FIGS. 6 and 7 and the deviation list in the small block statistical model are not used. Similar to the first comparison method, the small block statistical model is mainly described in this comparison method, and the detailed description of the statistical model is omitted as it is the same. In addition, the second comparison method will be described using the same example as that used in the description of the first comparison method.

  In the second comparison method, a certain small block in the t-th frame image [non-predicted region] 311 and each of the P small block statistical models held in the model holding unit 18 are handled as an operation image. The feature amount and feature vector of the image to be computed are calculated as follows. Refer to FIG. For the sake of concrete explanation, it is assumed that a small block is formed of 8 × 8 pixels as shown in FIG.

  First, an R signal value histogram, a G signal value histogram, and a B signal value histogram in a small block are obtained. Although it is possible to define the feature amount from each frequency in the 256-level histogram, in order to ignore a small signal value difference and reduce the amount of calculation, the resolution in the histogram is lowered to reduce the 64-level histogram (that is, , A histogram obtained by classifying signal values of 0 to 255 into 64 levels). 14 and FIG. 15 described later, the horizontal axis represents the classification step number in the histogram, and the vertical axis represents the frequency. In this way, each frequency of the 64-stage histogram obtained for each small block and for each R, G, and B signal value is defined as a feature amount of the operation image, and a vector having all feature amounts as elements is defined as: It is obtained as a feature vector of the operation image.

  When comparing the small block background model images, the necessary feature vectors are 64 × 3 = 192 dimensional vectors. On the other hand, when comparing the background model images, if the number of small blocks included in the t-th frame image [non-predicted region] 311 in FIG. 9 is β, the required feature vector is β × 64 × 3 = 192 β dimension. It becomes a vector. Further, as shown in the above-described example, the frame image is composed of 320 × 240 pixels, and in the case where the entire frame image is compared with the background model image, the necessary feature vector is 1200 × 64. × 3 = 230 400-dimensional vector.

  As described above, after calculating the feature vectors of a certain small block in the t-th frame image [non-predicted region] 311 and the small block background model image, the similarity is evaluated based on the feature vector. To do. Hereinafter, a method for evaluating the similarity between a certain small block in the t-th frame image [non-predicted region] 311 and one focused small block background model image will be described.

  First, a 64-step histogram of a small block in the t-th frame image [non-predicted region] 311 is compared with a 64-step histogram of a small block of the focused small block background model image. The histograms of R, G, and B signal values for the former small block are referred to by reference numerals 251R, 251G, and 251B, respectively, and the histograms of the R, G, and B signal values for the latter small block are respectively reference numerals 252R, 252G. And 252B. FIG. 15 shows these histograms.

And it is judged that the similarity of the image in the compared small block is so high that the area which overlapped when the histogram regarding the signal value of the same color was overlapped is large. Specifically, the frequencies of the i-th stage of the histograms 251R and 252R are represented by X _Ri and Y _Ri , respectively, the frequencies of the i-th stage of the histograms 251G and 252G are represented by X _Gi and Y _Gi , respectively, and the histograms 251B and 252B When the frequency at the i-th stage is expressed by X _Bi and Y _Bi , the block evaluation value d _b ² is calculated by the following formula (4). The block evaluation value d _b ² represents the similarity between the small blocks to be compared, and the block evaluation value d _b ² decreases as the similarity between the images in the small blocks to be compared increases.

Block evaluation values d _b ² for a certain small block in the t-th frame image [non-predicted region] 311 and each small block background model image are sequentially calculated. Then, it is determined that the small block background model image having the minimum block evaluation value d _b ² is the small block background model image most similar to a certain small block.

Note that the t-th frame image [non-predicted region] 311 shown in FIG. 9 is compared with the background model image, and the t-th frame image [non-predicted region] 311 includes a plurality of small blocks. The comparison uses a similarity evaluation value S that is the sum of the block evaluation values d _b ² . Then, the similarity evaluation value S is obtained for each statistical model background model image, and the background model image having the minimum similarity evaluation value S is determined to be the most similar background model image.

Further, as in the above example, the frame image can be subdivided into 40 × 30 = 1200 small blocks, and when comparing the entire frame image and each background model image, the similarity evaluation shown in Expression (5) is performed. The value S is determined. In the equation (5), the block evaluation value d _b ² described above is shown as d _xy ² . These x and y indicate the position of the small block in the frame image, and satisfy 1 ≦ x ≦ 40 and 1 ≦ y ≦ 30. In addition, x represents a position in the horizontal direction of the frame image, and y represents a position in the vertical direction. As x increases, the horizontal position moves to the right, and as y increases, the vertical position moves downward. And

<Fourth embodiment>
Next, a fourth embodiment will be described. In the fourth embodiment, an example of a method of selecting a small block background model image used for the comparison background image [prediction region] 413 illustrated in FIG. 11 will be described with reference to FIG. FIG. 16 shows a simple example for explaining the selection method. The case where the small block background model image used for the small block E is selected based on the small block background model image selected in the small blocks A and B will be described with reference to FIG. For the sake of specific description, in FIG. 16 and the following description, there are three small block background model images held in the model holding unit 18 for each small block, which are indicated by IDs 1 to 3. Shall be. In addition, it is assumed that the synthesis results are recorded in the synthesis history, and that A = 1 and B = 3 are currently obtained.

  As shown in FIG. 16, combinations of small block background model images used for small blocks A and B from 10 combinations (model synthesis history list) of small block background model images used for small blocks A, B, and E. And the small block background model image used for the small block E (combination appearance probability list) and the probability of the small block background model image used for each of the small blocks A, B, and E (prior probability list P ( A), P (B) and P (E)). Note that all of these tables may be held in the model synthesis history holding unit 20 as a synthesis history, or a part (for example, a model synthesis history list or a combination appearance probability list) is held, and the rest are stored as needed. It is also possible to calculate as follows.

In the fourth embodiment, Bayesian estimation as shown in the following formula (6) is performed using these values. In Equation (6), for example, P (E ₁ ) represents the probability when E = 1. This probability can be obtained from the prior probability list of FIG. 16, for example. For example, P (A ₁ , B ₃ | E ₁ ) indicates a conditional probability of A = 1 and B = 3 when E = 1. This conditional probability can be obtained from, for example, the combined appearance probability list in FIG.

From Equation (6), the probability that E = 1 is estimated under the condition of A = 1 and B = 3 is obtained. When the values shown in FIG. 16 are actually substituted, P (E ₁ | A ₁ , B ₃ ) = 0.03 / 0.13≈23.1%. Similarly, P (E ₂ | A ₁ , B ₃ ) ≈30.8%, P (E ₃ | A ₁ , B ₃ ) ≈46.1%, and it can be seen that the probability of E = 3 is the highest. . In this way, it is possible to estimate and select a small block background model image to be used for the comparison background image [prediction region] 413.

In the above example, for the sake of simplicity, the small block background model image assigned to the small block E is selected based on the two small blocks A and B. However, based on small blocks other than the small blocks A and B, It is also possible to select a small block background model image to be allocated to the small block E. However, if the number of small blocks to be referred to indefinitely is increased, the amount of calculation increases and smooth operation is hindered. Further, if the number of small blocks to be used is u and the number of small block statistical models for one small block is P, the cell of the combined appearance probability list is ^Pu . Therefore, if the number u of small blocks to be used is increased, the amount of data to be held and the amount of calculation increase exponentially. Therefore, it is preferable to obtain in advance a small block having a high correlation with the small block to be obtained, for example, by the method described below, because it is possible to hold useless data and suppress an increase in the amount of calculation.

  An example of a method for obtaining a small block correlation will be described with reference to FIG. Further, for the sake of concrete description, a case where the correlation between the small blocks A and E shown in FIG. 16 is obtained will be described as an example. FIG. 17 is a correlation table showing a combination of a small block background model image allocated to the small block A and a small block background model image allocated to the small block E. Using this correlation table, the correlation between nominal scales is obtained. Specifically, it is performed by obtaining and evaluating the Kramer coefficient.

First, the expected value for each category is calculated as shown in the following equation (7). In the following, E _ij represents an expected value of category (i, j), that is, an expected value of (ID of A small block background model image, ID of E small block background model image). Further, 1 ≦ i ≦ 3 and 1 ≦ j ≦ 3. O _ij is the value of category (i, j), n _i is the sum when A = i, n _j is the sum when E = j, and n is the total sum (ie, 10). .

Also, using the obtained E _ij , the chi-square distribution χ ₀ ² in this table is obtained using the following equation (8).

Using the value of the obtained chi-square distribution χ ₀ ² , the attribute correlation coefficient φ is obtained using the following equation (9).

  Using the value of the attribute correlation coefficient φ obtained, the Kramer coefficient V is obtained using the following equation (10). In the following formula, t is the size of the table (the smaller number of values that A and E can take. In this example, A and E take three values, so t = 3).

The correlation can be compared by the Kramer coefficient V obtained in this way. When obtaining a small block background model image to be assigned to a certain small block, for example, a small block background model image assigned to five small blocks having a large correlation (a large Kramer coefficient V), and a synthesis history It can be determined as described above. If the number of small blocks is limited to five in advance, even if the number P of small block statistical models for one small block is 30, the above-described cell can be about 30 ⁵ ≈24 million. .

  Further, it is preferable to select a small block having a large correlation as described above in advance. Moreover, it is preferable to update regularly. For this reason, for example, it is more preferable that the detection is automatically performed once a day in a time zone (such as midnight) in which the object detection system is not operated so much and the system is hardly loaded.

  Further, if a small block having a large correlation is selected as described above, as shown in the combination appearance probability list of FIG. 16, an area that includes only those having a probability of 0 depending on the combination (for example, A = 2 , B = 1) may appear. In this case, even if it is calculated based on this table, it cannot be estimated. Such a case can be dealt with by the following method, for example.

  When such a phenomenon occurs, two situations can be considered. One is a case where a small block background model image that has never been observed is allocated to one of the small blocks to be referred to (a small block background model image in which the prior probability list in FIG. 16 is 0). If). In this case, since it cannot be calculated unless this small block is removed, for example, the small block having the next highest correlation after the small block being referenced is replaced with this small block, and this small block is removed. .

  The other case is a combination that has been observed in each small block but has not been observed so far in combination (for example, when A = 2 and B = 1 as described above). In this case, for example, a block having the lowest correlation among the small blocks referred to may be replaced with a small block having the second highest correlation after the small block referred to. Further, the small block having the lowest correlation may be ignored. Then, the above-described Bayesian estimation may be performed, and these operations may be repeated until the estimation can be performed.

  Note that the above countermeasures are merely examples, and other countermeasures may be used.

<Fifth embodiment>
In the fifth embodiment, an example of an object detection method using an “estimated model” that is a model other than the statistical model and the small block statistical model described above will be described with reference to the drawings. First, the estimation model will be described with reference to FIG.

  FIG. 18 is a diagram for explaining the estimation model. The estimation model is generated using frame images for a predetermined number of frames in the latest past viewed from the current frame. For example, when the predetermined number is 30, an estimated model is generated using frame images for a total of 30 frames from 30 frames before to the previous frame starting from the current frame. The estimation model has the same configuration as the statistical model described above, and is formed from a background model image and a deviation list that is referred to in the difference processing in the difference processing unit 12. The background model image included in the statistical model and the background model image included in the estimation model are different in generation method, but are common in that the background image is modeled. For this reason, both are called background model images and are distinguished by indicating which model is provided. The estimation model is also included in the background model described above.

  The background model image of the estimation model is generated by, for example, averaging the frame images for a predetermined number of frames in the past past. As an averaging method, a method of averaging corresponding pixels of a plurality of images (a plurality of most recent frame images) as described with reference to FIG. 13 in the second embodiment is used. be able to. The deviation list is the same, and can be generated using the method shown in the second embodiment.

  In addition, an example of an object detection system configured to use an estimation model in addition to the statistical model and the small block statistical model described above will be described with reference to FIG. Note that FIG. 19 corresponds to FIG. 1 showing the object detection system according to the embodiment of the present invention, and the same reference numerals are given to parts having the same configuration, and detailed description is omitted. .

  The object detection system of the present embodiment includes an estimation model generation unit 21 that generates an estimation model, an estimation model holding unit 22 that holds an estimation model generated by the estimation model generation unit 21, and a differential background image for comparison. A comparison background image selection unit 23 to be supplied to the unit 12. The comparison background image selection unit 23 includes at least a comparison background image generated from the estimation model supplied from the estimation model holding unit 22, and a statistical model and a small block statistical model supplied from the comparison background image supply unit 14. It is selected whether to use a contrasting background image generated from one. The comparison background image obtained by the selection is supplied to the difference processing unit 12. Thus, in the present embodiment, the comparison background image based on the estimation model generated by the estimation model generation unit 21 may be used in the difference processing unit 12. An example of the operation of this object detection system will be described with reference to the flowchart of FIG.

  FIG. 20 is a flowchart showing the overall operation flow of the object detection system according to this embodiment. A loop process including steps S1 to S12 is executed for each frame image. The frame period is, for example, 1/3 to 1/5 second. The process of step S1 is executed by the image acquisition unit 11, the processes of steps S2 to S6 are executed by the comparison background image selection unit 23, and the processes of steps S7, S8, and S10 are executed by the difference processing unit 12. The process of step S9 is executed by the estimated model generation unit 21, the process of step S11 is executed by the display control unit 13 and the display unit 3, and the process of step S12 is performed by the mask generation unit 15 provided in the comparison background image supply unit 14. Executed by.

  An outline of each step will be described. In step S1, a current frame image is acquired from the camera 1. In subsequent step S2, each of the N statistical models held in the model holding unit 18 of the comparison background image supply unit 14 is compared with the current frame image. In step S3, the most similar to the current frame image is compared. One statistical model including the background model image to be searched is searched.

  In step S4, it is determined whether the search result for the current frame image is the same as the search result for the previous frame image. That is, it is determined whether the statistical model searched for the current frame image is the same as the statistical model searched for the previous frame image. If they are the same, it is presumed that the illumination condition at the time of capturing the previous frame image is the same as that at this time, so that the estimation model can be used for detection of the object. Therefore, if the two search results are the same, it is estimated that the illumination condition has not changed between the previous and current frames, and the process proceeds to step S5, where a comparison background image is generated from the estimated model in step S5. On the other hand, if the search result for the current frame image is different from the search result for the previous frame image, it is presumed that the illumination condition at the time of shooting the previous frame image is different from that of the current frame image. It is not appropriate to use. Therefore, if the two search results are different, it is presumed that the illumination condition has changed between the previous and current frames, and the process proceeds to step S6. In step S6, the comparison background image is obtained from at least one of the statistical model and the small block statistical model. Is generated. That is, the comparison background image 314 shown in FIG. 9 and the comparison background image 414 shown in FIG. 11 are generated. It should be noted that the fact that there is no change in illumination conditions includes not only a state where there is no change, but also a state where the change is small enough to be ignored.

  When the comparison background image is generated in step S5 or S6, the process proceeds to step S7, and difference processing is performed between the comparison background image and the current frame image. Based on the result of the difference processing, the object area and the background area are extracted from the entire area of the current frame image. The extraction of the object area and the extraction of the background area are executed in steps S10 and S8, respectively. The object area means an area where the object is drawn, and the background area means an area where a background not including the object is drawn. Therefore, the entire area of the frame image corresponds to a composite area of the object area and the background area.

  After the target area and the background area are extracted, the estimation model is updated based on the extraction result in step S9, and the extraction result of the target area is displayed on the display screen of the display unit 3 in step S11. The For example, the display control unit 13 in FIG. 1 generates a display image in which a frame surrounding the target region is superimposed on the current frame image so that the target region and the background region can be visually recognized separately. 3 is displayed on the display screen. That is, a display image as shown in FIG. 3 is generated.

  Further, the mask generation unit 15 generates a mask image 210 as shown in FIG. 5 from the difference result image extracted and output by the difference processing unit 12 in step S10. When the processes of steps S9, S11, and S12 are completed, the process returns to step S1, and the processes of steps S1 to S12 are performed on the next frame image.

  While the illumination conditions (particularly the illuminance) of the camera 1 change relatively slowly due to changes in the illuminance of sunlight during the day, the illumination conditions can change drastically and instantaneously by turning on / off the illumination device. Since the former change state is sequentially reflected in the estimation model, it is possible to detect an object suitable for the former change by using the estimation model. On the other hand, when the estimation model is used, the latter change state may not be fully supported. Therefore, when it is estimated that a significant instantaneous change has occurred in the lighting conditions, the object detection processing is performed using a statistical model or a small block statistical model statistically created from a large number of sample background images. . Therefore, by adopting the configuration of the present embodiment, it is possible to estimate a change in the illumination condition of the camera 1 and use the estimation model, the statistical model, and the small block statistical model properly according to the estimation result. Therefore, it is possible to detect an object with high accuracy corresponding to both a gradual change in illumination conditions and a large instantaneous change.

  Hereinafter, specific examples of the above steps will be described.

  When comparing each statistical model in step S2 with the current frame image, the first comparison method and the second comparison method shown in the second embodiment can be applied. Further, as described above, a method such as the “first comparison method” of the second embodiment can be applied to the difference processing in step S7. Furthermore, when extracting the object region and the background region from the difference result image obtained thereby (steps S8 and S10), the object region and the background region when generating the mask image shown in the first embodiment are used. A specific method may be used.

  A specific example of updating the estimation model in step S9 will be described with reference to FIG. By combining the remaining image when the object area is extracted from the current frame image 220 and the image in the object area in the comparison background image 221, a stock as a current background image without a pseudo defect is obtained. A background image 225 is generated. Specifically, the partial image located in the object area (the object area specified based on the difference result image 222) is removed from the current frame image 220, and the remaining image after the removal is used as the object removal image 223. Generate as In FIG. 21, the area surrounded by the broken line in the object removal image 223 represents the removed area. On the other hand, a partial image located in the object area (the object area specified based on the difference result image 222) is extracted from the comparison background image 221, and this extracted image is used as the object position background image 224. Then, both the images are combined by fitting the object position background image 224 into the object removal image 223, and the combined image is used as the stock background image 225.

  The stock background image 225 is added to the estimated model generation unit 21 and held as a component of the stock background image group. The upper limit number of stock background images to be held is determined. If the upper limit number of stock background images are already stored, the oldest stock background image is deleted and a new stock background image 225 is stored. The estimated model generation unit 21 holds it. As a result, the stock background image group is updated, and a new estimation model is recreated based on the updated stock background image group. The newly created estimated model (that is, the estimated model updated in step S9 this time) is held in the estimated model holding unit 22 and used as an estimated model for the next frame image.

  In this way, by updating the estimation model based on the sequentially acquired frame images, it is possible to detect an object adapted to a relatively gradual change in illuminance.

  If it is determined in step S4 that the search result for the current frame image is different from the search result for the previous frame image and the process proceeds to step S6, the illumination condition is presumed to have changed abruptly. The estimated model used is not suitable for the object detection process. Therefore, in this case, all the stock background image groups may be once deleted in step S9. At the same time, the estimation model held at that time may be discarded.

  In this case, an image similar to the object position background image 224 of FIG. 21 is generated using the comparison background image generated in S6, and an image similar to the object removal image 223 is generated from the current frame image. Alternatively, an alternative stock background image may be generated.

  Further, in this case, once the determination condition that the search result for the current frame image is the same as the search result for the previous frame image is not satisfied once in step S4 in FIG. 20, the determination is made for the subsequent frame images. Even if the condition is satisfied, exceptional processing may be performed. The exceptional process is a process forcibly moving to step S6 without moving from step S4 to step S5, and using the contrast background image based on the statistical model, the difference process, the object area and the background area extraction process, and This means that an alternative stock background image generation process is performed. By executing exceptional processing, the number of stock background images increases. However, this exceptional process is not performed when the number of stock background images held in the estimated model holding unit 22 reaches a specified number. In other words, when the number of stock background images has not reached the specified number, exceptional processing is performed regardless of whether or not the above judgment condition is satisfied (thus, exceptionally immediately after the object detection system is activated). Processing is performed). When the number of stock background images reaches the specified number, in step S9, the estimated model generation unit 21 in FIG. 19 generates an estimated model using the stock background images for the specified number of images, and holds the estimated model. After that, if the above judgment condition is satisfied, the process proceeds from step 4 to step S5 as a rule, and the comparison background image generation process and difference process using the estimation model are performed.

  For example, in the case where the specified number of times is 30, the case where the above-described determination condition is once satisfied and then the determination condition is continuously satisfied for the subsequent frame images will be considered. In this case, exceptional processing is performed on the frame images for 30 frames after the next time, and 30 stock background images are generated using the frame images for 30 frames. An estimated model is generated. After that, if the above judgment condition is satisfied, the process proceeds from step S4 to step S5 as a rule, and the comparison background image generation process and difference process using the estimation model are performed. The newly generated estimated model is a background model that matches the lighting conditions at that time.

<< Sixth Example >>
Next, a sixth embodiment will be described. In the sixth embodiment, the system configuration when the object detection system of FIG. 1 is applied to an actual system is illustrated. FIG. 22 is an overall block diagram of the object detection system according to the sixth embodiment. The object detection system of FIG. 22 has at least an imaging module 50 and a monitoring device 60, to which a terminal device 70 and a network 80 such as the Internet network are connected. The terminal device 70 is a terminal device having a communication function, and is a personal computer, a mobile phone, a portable information terminal, or the like. It is also possible to regard the terminal device 70 and the network 80 as components of the object detection system.

  The imaging module 50 is provided with each part referred by the codes | symbols 51-53, and is installed indoors etc. in which a target object may exist. The monitoring device 60 includes each part referred to by reference numerals 61 to 65 and is installed in the monitoring person's room. The terminal device 70 is provided with each part referred by the codes | symbols 71-73. The user of the terminal device 70 can also be regarded as one of the supervisors.

  The camera 51 and the display unit 63 in FIG. 22 are the same as the camera 1 and the display unit 3 in FIG. 1 and function as the camera 1 and the display unit 3, respectively. The communication unit 53 is a part for connecting the imaging module 50 to the network 80, the communication unit 61 is a part for connecting the monitoring device 60 to the network 80, and the communication unit 71 is a part for connecting the terminal device 70 to the network 80. 80 for connecting to 80. The communication units 53, 61 and 71 are connected via the network 80 to the module side processing unit 52 connected to the communication unit 53, the monitoring side processing unit 62 connected to the communication unit 61, and the terminal side connected to the communication unit 71. Data exchange with the processing unit 72 is realized.

  The module processor 52 and the monitoring processor 62 cooperate to form the detection processor 2 of FIG. That is, a part of processing to be performed by the detection processing unit 2 is executed by the module side processing unit 52, and the remaining processing to be performed by the detection processing unit 2 is executed by the monitoring side processing unit 62.

  For example, each part referred to by reference numerals 11, 12, and 14 in FIG. 1 is provided in the module side processing unit 52, and the display control unit 13 in FIG. 1 is provided in the monitoring side processing unit 62. For convenience of explanation, each member (reference numerals 15 to 19) provided in the comparison background image supply unit 14 may be provided in a different place. Further, the image data of the sample background image group for generating the statistical model and the small block statistical model is stored in the storage unit 65, and the monitoring side processing unit 62 determines each statistical data based on the storage contents of the storage unit 65. Generate a model. Of course, this is an example of how the processing to be performed by the detection processing unit 2 is allocated to the module side processing unit 52 and the monitoring side processing unit 62, and the allocation method can be arbitrarily modified.

  Further, the extraction result of the object region with respect to the image data of the frame image obtained by the camera 51 is processed on the terminal side from the communication unit 53 of the imaging module 50 or the communication unit 61 of the monitoring device 60 via the network 80 and the communication unit 71. By transmitting to the unit 72, an image equivalent to the image displayed on the display unit 63 (for example, the display image 200 in FIG. 3) reflecting the extraction result can be displayed on the display unit 73. . Of course, a display image such as the display image 200 in FIG. 3 is generated on the imaging module 50 side or the monitoring device 60 side, and image data of the display image is sent to the terminal device 70, so that the display image is displayed. You may display on the display part 73. FIG. The terminal-side processing unit 72 also functions as a display control unit that controls the display content of the display unit 73, and the display control unit can be equivalent to the display control unit 13 in FIG.

  The specific numerical values shown in the above description are merely examples, and as a matter of course, they can be changed to various numerical values. In addition, modifications or annotations of the above-described embodiment are listed below.

  The above-described object detection system can be used as a system for detecting whether or not there is a person as an object in the imaging region of the camera (1 or 51). In particular, for example, the camera (1 or 51) is formed so that a stereo image can be generated. Then, by detecting an object region in a stereo image and analyzing the detected image of the object region in time series, the posture of a person as the object can also be detected. It is also possible to determine whether or not a person has fallen from the detection result of the posture. In addition, the above-mentioned object detection system can also be used as a security monitoring system by regarding a person as an object as a suspicious person. In addition, a system for counting the number of persons in the photographing region of the camera (1 or 51) can be formed by counting the number of persons as objects.

  Further, although the embodiment of the present invention has been described assuming that the object is a person, the object can be any object other than a person (note that a person is also included in the object). An object including a person can be interpreted as a moving object, or can be interpreted as an object entering the imaging region of the camera (1 or 51).

  In the above-described embodiment, the detection result of the object is notified to the monitor located outside the object detection system using the display unit. However, the detection result may be notified by voice. Good. In this case, for example, whether or not the object is detected from the frame image where the object is to be detected may be output as a sound using a sound output unit (not shown).

  In addition, the image characteristics (brightness and hue) of the frame image obtained by photographing with the camera 1 vary depending on various factors. This change is mainly caused by a change in the illumination condition of the camera 1 (that is, the state of the light source that illuminates the imaging region of the camera 1). However, for example, if the color or gloss of an object (such as a wall of a room) located in or around the shooting area of the camera 1 is different or the arrangement position of the object is different, the light source itself is not changed. However, the image characteristics of the obtained frame image can change. That is, the illumination condition is a kind of ambient environment condition of the camera 1 that affects the image characteristics of the obtained frame image, and the color, gloss, and arrangement position of the object are also a kind of the ambient environment condition. Therefore, the illumination conditions described in the above embodiment can be read as the ambient environment conditions of the camera 1. In other words, the above-described object detection system is considered to be a system that outputs a background image for comparison that allows appropriate difference processing to be performed in accordance with changes in the surrounding environmental conditions of the camera 1 including illumination conditions. You can also

  The detection processing unit 2 in FIG. 1 can be realized by hardware or a combination of hardware and software. When the detection processing unit 2 is configured using software, a block diagram of a part realized by software represents a functional block diagram of the part. All or part of the arithmetic processing that needs to be performed by the detection processing unit 2 is described as a program, and the program is executed on the program execution device to realize all or part of the arithmetic processing. May be.

  Similarly, each of the module side processing unit 52 and the monitoring side processing unit 62 in FIG. 22 can be realized by hardware or a combination of hardware and software. When configuring them using software, a block diagram of a part realized by software represents a functional block diagram of the part. By describing all or part of the arithmetic processing that needs to be performed by the module side processing unit 52 and the monitoring side processing unit 62 as a program and executing the program on the program execution device, all of the arithmetic processing or You may make it implement | achieve a part.

  For example, it can be considered as follows. A generation unit that generates a mask for specifying a prediction region and a non-prediction region in the detected image is formed by the mask generation unit 15 in FIG. 4, and the comparison background image generation unit that generates the comparison background image 4 comparison background image generation units 19. For example, a portion formed by including the mask generation unit and the comparison background image generation unit and the difference processing unit 12 of FIG. 1 forms an object detection device (or object detection device). This object detection device (or object detection device) is considered to further include all or any of the image acquisition unit 11, the display control unit 13, and the model generation unit 17 of FIG. It is also possible.

  When the functions realized by the object detection apparatus are described as a program and the computer (not shown) that reads the program realizes the functions, FIG. 1, FIG. 4, FIG. 8, FIG. The processing contents of the difference processing unit 12, the mask generation unit 15, and the comparison background image generation units 19, 19a and 19b, the schematic diagrams shown in FIGS. 9 and 11, and the flowchart shown in FIG. 20 are described as a program. good.

  As mentioned above, although each embodiment in the present invention was described, the range of the present invention is not limited to this, and can be carried out by adding various changes without departing from the gist of the invention.

1 is an overall block diagram of an object detection system according to an embodiment of the present invention. It is a figure for demonstrating the principle of the detection method of the target object by the target object detection system of FIG. It is a figure which shows the example of the image displayed on the display part 3 of FIG. It is a block diagram which shows the basic composition of the contrast background image supply part of FIG. It is a schematic diagram of the mask output from the mask production | generation part of FIG. It is a figure for demonstrating the statistical model produced | generated in the model production | generation part of FIG. It is a figure for demonstrating the small block statistical model produced | generated in the model production | generation part of FIG. It is a block diagram which shows the 1st structural example of the contrast background image supply part of FIG. It is a schematic diagram for demonstrating operation | movement of the 1st structural example of the contrast background image supply part of FIG. It is a block diagram which shows the 2nd structural example of the contrast background image supply part of FIG. It is a schematic diagram for demonstrating operation | movement of the 2nd structural example of the contrast background image supply part of FIG. It is a figure showing a mode that many sample background images are clustered. It is a figure for demonstrating the production | generation method of the background model image contained in a statistical model. It is a figure which shows that the feature-value and feature vector of an image are expressed with a histogram. It is a figure which shows the histogram of one image which should be contrasted, and the histogram of the other image. To describe a method of estimating a combination of small block background model images used for a comparison image [prediction region] based on a comparison image [non-prediction region] and a synthesis history according to the fourth embodiment of the present invention. FIG. It is a figure for demonstrating the method of calculating | requiring the correlation of a small block. It is a figure for demonstrating an estimation model. It is a whole block diagram of the target object detection system which concerns on 5th Example of this invention. It is a flowchart showing the flow of the whole operation | movement of the target object detection system which concerns on 5th Example of this invention. It is a figure which shows the mode of the difference process with the background image for a comparison based on the present frame image and an estimation model. It is a whole block diagram of the target object detection system which concerns on 6th Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera 2 Detection processing part 3 Display part 11 Image acquisition part 12 Difference processing part 13 Display control part 14 Background image supply part 14 for comparison 15 Mask generation part 16 Image holding part 17 Model generation part 18 Model holding part 19, 19a, 19b Comparison Background image generator

Claims (10)

A target detection apparatus that sequentially acquires detected images based on camera shooting, and detects a target in a shooting region of the camera based on a difference result between the detected image and a contrast background image,
A mask generation unit that generates a mask that identifies a prediction region in which the object in the detected image is predicted to exist and a non-prediction region that is a region other than the prediction region;
A background image generation unit for comparison that generates the background image for comparison by using different generation methods for the region that is different from the prediction region specified by the mask and the region that is different from the non-prediction region. When,
An object detection apparatus comprising: The comparison background image generation unit, based on the non-prediction area of the detected image, an area where the difference from the prediction area of the comparison background image is obtained, and an area where the difference is obtained from the non-prediction area; Each of these is produced | generated, The target object detection apparatus of Claim 1 characterized by the above-mentioned. An image holding unit that holds a plurality of images shot in a state where the object is not included in the shooting region;
A model generation unit that generates a plurality of model images from the plurality of images held in the image holding unit;
A model holding unit for holding the model image generated by the model generating unit;
Further comprising
The comparison background image generation unit generates the comparison background image using the model image held in the model holding unit, and is similar to the non-predicted region of the detected image. The object detection apparatus according to claim 2, wherein a generation method is used to generate at least a region in which a difference from the non-predicted region of the comparison background image is obtained using an image. A plurality of small block background model images corresponding to each of the small blocks that are subdivided areas of the detected image are held in the model holding unit,
When the comparison background image generation unit generates an area in which a difference is obtained from the non-predicted area of the comparison background image,
For each small block in the non-predicted region of the detected image, the most similar small block background model image is selected from the model holding unit, and the selected small block background model image is used for the comparison. The object detection apparatus according to claim 3, wherein a generation method for generating an area in which a difference from the non-predicted area of the background image is obtained is used. A plurality of background model images corresponding to the entire detected image are held in the model holding unit,
When the comparison background image generation unit generates an area in which a difference is obtained from the prediction area of the comparison background image,
By selecting the background model image having the most similar region at the same position as the non-predicted region from the model holding unit, and extracting and using the region at the same position as the predicted region from the selected background model image, The object detection apparatus according to claim 4, wherein a generation method for generating an area in which a difference from the non-predicted area of the comparison background image is obtained is used. The comparison background image is selected and combined from the plurality of small block background model images held in the model holding unit by the comparison background image generation unit,
A model synthesis history holding unit that holds a synthesis history that is a combination history of the small block background model image used in the background image for comparison output from the background image generation unit for comparison;
When the comparison background image generation unit generates an area in which a difference is obtained from the prediction area of the comparison background image,
Based on the combination history and the combination of the small block background model image that forms a region where a difference is obtained from the non-predicted region of the comparison background image, the difference between the prediction region and the prediction region of the comparison background image is obtained. The object detection apparatus according to claim 4, wherein a generation method for selecting and generating the small block background model image used for a region to be used is used. The model synthesis history holding unit obtains a correlation about the selection result of the small block background image between the small blocks based on the synthesis history, and includes this information in the synthesis history.
When the background image for comparison generation unit selects the small block background model image to generate an area for which a difference is obtained from the prediction area of the background image for comparison,
Based on the small block background model image used for the small block having a high correlation with the small block for which the small block background image is to be selected, the prediction region of the comparison background image and the region in which the difference is obtained. The object detection apparatus according to claim 6, wherein a generation method of selecting and generating a small block background model image is used. A camera for obtaining a detected image by shooting;
The object detection device according to any one of claims 1 to 7,
An object detection system comprising: Used in a system including a camera that obtains an image to be detected by photographing and a display device that displays an image based on the image to be detected, and sequentially acquires the image to be detected from the camera, and a background for comparison with the image to be detected An object detection method for detecting an object in an imaging region of the camera based on a difference result with an image,
When generating the contrast background image,
Identifying a predicted region where the object in the detected image is predicted to exist and a non-predicted region that is a region other than the predicted region;
An object detection method, wherein a generation method is different between a region where a difference from the non-predicted region of the background image for comparison is obtained and a region where a difference is obtained from the prediction region. A program for causing a computer to realize a function of detecting an object in a shooting region of the camera based on a difference result between a detected image sequentially obtained by camera shooting and a background image for comparison,
A function for specifying a prediction region where the object in the detected image is predicted to exist and a non-prediction region other than the prediction region;
A function of generating the comparison background image with different generation methods in the non-predicted region and the region in which the difference is obtained, and in the prediction region and the region in which the difference is obtained;
A program for causing the computer to realize the above.