JP6275022B2 - Image monitoring device - Google Patents

Description

  The present invention relates to an image monitoring apparatus that sequentially processes images captured in a monitoring area and determines whether or not a detection target exists in the monitoring area.

  2. Description of the Related Art Conventionally, there is an image monitoring apparatus that captures an image of a monitoring area with a camera and detects a detection target such as an intruder from the captured image. In such an image monitoring apparatus, a sequentially changed image (input image) is compared with a reference image (background image) stored in advance, and an area with a change (moving object area) is extracted. Then, the presence of an intruder (human) in the monitoring area is detected based on the moving object area. However, it is not only humans that are extracted as moving object regions. For example, changes due to sunlight or illumination light, planting swaying in the wind, shadows thereof, changes in small animals, insects, and the like are also extracted as moving object regions. For this reason, in the image monitoring apparatus, it is necessary to distinguish between a change caused by these disturbances and a change caused by a human being to be detected.

  In Patent Document 1, the captured image information is processed to calculate human attribute information that indicates humanity in multiple values and disturbance attribute information that indicates multiplicity of factors other than humans. An image sensor that determines the presence or absence of a human based on a ratio of a cumulative value with disturbance attribute information is disclosed. In particular, the image sensor of Patent Literature 1 improves the detection accuracy by calculating human attribute information and disturbance attribute information for each frame and comparing cumulative values of a plurality of frames of each attribute information.

JP 2001-243475 A

  By the way, the human attribute information has a high value when the size and shape of the moving object region are human. Also, paying attention to the fact that “human beings exist differently from shadows and light”, the edges of background objects reflected in the background image are hidden in the moving object area, or the brightness value of the background image When the correlation of is low, the human attribute information has a high value.

  However, even when a moving object other than a human being is imaged, the human attribute information may be calculated higher than the disturbance attribute information (non-human attribute information). The cause of this is that the moving object on the input image looks like a human depending on the position of the moving object (it appears human as a result of how the light hits and the position of the camera), or behind the moving object It is conceivable that the calculation accuracy of the attribute information calculated using the characteristics of the background image is lowered due to the influence of the texture of the existing background object.

  The advantage of determining whether or not a person is a person using the accumulated value of attribute information as in Patent Document 1 is that even if attribute information cannot be calculated with high accuracy in a certain frame, the attribute information with other frames has high accuracy. If calculated, the detection accuracy can be ensured.

  However, as described above, if the moving object stays at a position on the image where it is difficult to accurately calculate the attribute information, the attribute information with low accuracy will greatly affect the accumulated value (the sum of the attribute information for each frame). As a result, there is a problem that the threshold value for determining whether or not the accumulated value is human is exceeded by attribute information with low accuracy.

  The present invention has been made in view of the above problems, and an object thereof is to provide an image monitoring apparatus with improved detection accuracy of a detection target.

  One aspect of the present invention is an image monitoring apparatus that inputs an image obtained by sequentially capturing a monitoring area and determines whether or not a detection target exists in the image, the moving object area extracting from the image An object area extracting means, a tracking means for sequentially tracking a moving object area by the same moving object on the image, an attribute information calculating means for calculating attribute information indicating the likelihood of detection for the moving object area, In a moving object region by the same moving object, a region association means for associating a moving object region having a predetermined positional relationship to form one group and forming the other moving object regions independently as a group; For each group, representative attribute information calculating means for calculating one representative attribute information using attribute information of the moving object area constituting the group, and the detection target using the representative attribute information And determining target determining means for determining whether or not there is an image monitoring apparatus comprising: a.

  Here, it is preferable that the area association means uses the predetermined positional relation as a relation located in a mutual position where the attribute information of the moving object area hardly changes.

  Further, it is preferable that the region association means sets the relationship in which the moving object region is located in a predetermined range on the image as the predetermined positional relationship.

  In addition, the region association unit sets the position of the tracking start point of the moving object region as a reference position, associates the relationship where the moving object region is located within a predetermined range from the reference position as the predetermined positional relationship, When the moving object region is outside the predetermined range from the reference position, it is preferable to set the position of the moving object region at that time as a new reference position.

  Further, it is preferable that the target determination unit determines whether or not the detection target exists based on at least a cumulative value or an average value of the representative attribute information.

  ADVANTAGE OF THE INVENTION According to this invention, the detection precision of a detection target can be raised in the image monitoring apparatus which processes the image which imaged the monitoring area | region sequentially, and determines whether a detection target exists in a monitoring area | region.

It is a figure which shows the condition of the monitoring area | region in embodiment of this invention. It is a functional block diagram which shows the structure of the image monitoring apparatus in embodiment of this invention. It is a figure which shows an input image and a moving object area | region when a person penetrates into the monitoring area | region in embodiment of this invention. It is a figure which shows an input image when a bug exists in the monitoring area | region in embodiment of this invention, and a moving object area | region. It is a figure which shows the membership function for calculating the attribute information in embodiment of this invention. It is a figure explaining the association process of the moving object area | region in embodiment of this invention. It is a figure explaining the image determination based on the attribute information and representative attribute information in an embodiment of the present invention. It is a flowchart which shows the image monitoring process in embodiment of this invention. It is a flowchart which shows the area | region correlation process in the Example of this invention.

  Embodiments of the present invention will be described with reference to the drawings. The image monitoring apparatus of the present invention detects a detection target from an image obtained by capturing an indoor or outdoor monitoring area. In the present embodiment, a person who has entered the monitoring area from the outside of the monitoring area is detected in the outdoor monitoring area.

  FIG. 1 is an image diagram showing various situations in the monitoring area. FIG. 1A and FIG. 1B show the situation at different times in the same monitoring area. A tree 11 exists as a background object in the monitoring area. The camera 2 is installed at a predetermined height, and images a monitoring area at a predetermined depression angle from above to below. FIG. 1A shows a situation where a person 9 is walking in the monitoring area toward the building 12 at time T. Moreover, FIG.1 (b) has shown the condition where the spider 10 exists in the vicinity of the camera 2 in the monitoring area | region at the time t.

  In the present embodiment, as an example of image monitoring processing, a case will be described in which a person 9 that is a detection target and a spider 10 that is not a detection target are distinguished in the monitoring region.

  FIG. 2 is a functional block diagram of the image monitoring apparatus 1 in the present embodiment. As shown in FIG. 2, the image monitoring apparatus 1 includes an image input unit 3, a storage unit 4, an image processing unit 5, an abnormality determination unit 6, an output unit 7, and a background image update unit 8. In the present embodiment, each unit is described as the image monitoring apparatus 1, but the image input unit 3, the storage unit 4, the image processing unit 5, the abnormality determination unit 6, the output unit 7, and the background image update unit 8 are separately provided. You may make it store in a housing | casing and communicate a required control signal etc. using a communication technique. In the present embodiment, the camera 2 is separated from the image monitoring apparatus 1 but may be incorporated in the same housing as other components. In addition, you may employ | adopt not only this but another housing structure.

  The camera 2 images the monitoring area. The camera 2 includes an optical system, an image sensor such as a CCD element or a C-MOS element, optical system parts, an analog / digital converter, and the like. The camera 2 is fixedly installed and the field of view is kept constant. The camera 2 sequentially captures images at regular time intervals (for example, every frame 1/5 second). The captured image is expressed as a digital image in which each pixel value has a pixel value of 0 to 255, for example. The camera 2 may generate a color image.

  The image input unit 3 is an interface that is connected to the camera 2, sequentially acquires images taken by the camera 2, and sequentially outputs the images to the image processing unit 5. Hereinafter, an image that is acquired by the image input unit 3 and is determined by the image processing unit 5 to determine whether or not a detection target exists is referred to as an input image 40.

  In the present embodiment, the image input unit 3 sequentially acquires images in accordance with the imaging timing of the camera 2. However, the present invention is not limited to this, and the image input unit 3 may store the acquired image in the storage unit 4 to be described later and sequentially output the input images 40 to the image processing unit 5 via the storage unit 4. Good. Further, the image input unit 3 is connected to a storage device (not shown) provided outside the image monitoring device 1, and the input images 40 captured by the camera 2 stored in the storage device are sequentially acquired. Also good.

  The storage unit 4 includes a semiconductor device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and a memory device such as a hard disk. The storage unit 4 is accessible from the image processing unit 5, the background image update unit 8, and the like.

  The main information stored in the storage unit 4 is the background image 41. The background image 41 is an image taken when there is no detection target in the monitoring area. The background image 41 is appropriately updated by a background image update unit 8 described later. Although not shown, the storage unit 4 also stores various programs and parameters for realizing each process of the image monitoring apparatus 1 and information such as the installation height and depression angle of the camera 2.

  The image processing unit 5 is configured by an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and reads and executes various programs stored in the storage unit 4. Perform image monitoring processing. In the image monitoring process, the input images 40 acquired by the image input unit 3 are sequentially processed. In the present embodiment, the input image 40 is processed in order of photographing time for each frame.

  The image processing unit 5 includes modules of a moving object region extraction unit 51, a position calculation unit 52, a tracking unit 53, an attribute information calculation unit 54, an area association unit 55, a representative attribute information calculation unit 56, and a target determination unit 57. Consists of. Hereinafter, each module will be described.

  The moving object region extraction unit 51 extracts a region (moving object region) where a change has occurred in the input image 40 acquired by the image input unit 3. Specifically, an absolute value of a difference between the input image 40 and the background image 41 stored in the storage unit 4 is obtained, and an area where the absolute value of the difference is equal to or greater than a predetermined threshold is a difference area (binary). Extract as Then, the moving object area extraction unit 51 labels the difference area extracted from one frame with the difference area based on the same moving object. Specifically, eight adjacent and adjacent pixels are grouped as a label area, and adjacent label areas having a certain size or positional relationship are the same moving object, and one label area and To do. Hereinafter, this label area is referred to as a “moving object area”.

  The extraction of the moving object region is not limited to the background difference described above, and a difference between a plurality of frames, a normalized correlation between the background image 41 and the input image 40, a region detected by a learning classifier, or the like may be used. .

  The position calculating unit 52 calculates the position on the input image 40 for the moving object region extracted by the moving object region extracting unit 51. Specifically, the barycentric coordinates of the moving object region are calculated. The center-of-gravity coordinates calculated for each moving object area are stored in the storage unit 4 and output to a tracking unit 53 and an area association unit 55 described later.

  The tracking unit 53 temporally tracks the moving object region extracted by the moving object region extracting unit 51. Specifically, tracking of the moving object region is realized by associating the moving object regions of the same moving object between the input images 40 captured at different times. The tracking unit 53 associates the moving object region of the previous frame with the moving object region of the current frame, and starts tracking the moving object region that is not associated with the moving object region of the previous frame as a new tracking target. To do. In addition, among the moving object areas that have been tracked up to the previous frame, for the moving object area that is not associated with the moving object area of the current frame, the tracking process for the moving object area ends. The association of the moving object areas may be performed by a known method using the barycentric coordinates calculated by the position calculation unit 52 or the area of the moving object area calculated separately.

  Here, with reference to FIG. 3 and FIG. 4, an image and a moving object region in the present embodiment will be described. 3 and 4 are diagrams showing various images and moving object areas in the present embodiment.

  FIG. 3 corresponds to a plurality of input images 40 obtained by capturing the monitoring area in the state of FIG. FIG. 4 corresponds to a plurality of input images 40 in which the monitoring area in the state of FIG. The background image 41 is the same image in FIGS. 3 and 4, and both show the trees 11 existing in the monitoring area.

  First, FIG. 3 will be described. An input image T obtained by photographing the monitoring area at the time T shown in FIG. 1A, an input image T-2 that is an input image for two frames before and after the input image T, and an input image T- 1, an input image T + 1, an input image T + 2, and an input image T + 3 are arranged in order of photographing time. These input images show a person 9 and a tree 11, and the person 9 moves to the right through the front of the tree 11 as time advances. The moving object area extracted using the input image of each frame and the background image 41 becomes a moving object area 13a shown in black. The moving object region 13a is started to be tracked as a new tracking target at the time of the input image T-2 and is tracked up to the time of the input image T + 3.

  Next, FIG. 4 will be described. An input image t obtained by photographing the monitoring area at the time t shown in FIG. 1B, an input image t-2 that is an input image for two frames before and after the input image t, and an input image t- 1, an input image t + 1, an input image t + 2, and an input image t + 3 are arranged in order of photographing time. In these input images, the spider 10 and the tree 11 are shown. As the time advances, the spider 10 sways in the wind and moves to the right across the front of the tree 11 shown in the center of the image. . At this time, since the spider 10 exists in the vicinity of the camera 2, it appears on the input image as large as the human 9 shown in FIG. 3. The moving object region extracted using the input image of each frame and the background image 41 becomes a moving object region 13b shown in black. The moving object region 13b is tracked as a new tracking target at the time of the input image t-2 and is tracked up to the time of the input image t + 3.

  The attribute information calculation unit 54 calculates attribute information indicating the likelihood of detection for each frame of the moving object region tracked by the tracking unit 53. In the present embodiment, human attribute information representing humanity is calculated. Further, the attribute information calculation unit 54 calculates non-human attribute information representing the non-human attribute for each frame of the moving object region tracked by the tracking unit 53. In the present embodiment, insect attribute information representing insect likeness is calculated as non-human attribute information. The attribute information calculation unit 54 stores the attribute information calculated for each frame in the storage unit 4 and outputs it to the representative attribute information calculation unit 56 described later. Hereinafter, calculation of human attribute information and insect attribute information will be described in detail.

  First, calculation of human attribute information will be described. The attribute information calculation unit 54 calculates a feature amount (human attribute element) for calculating human attribute information for each frame of the moving object region tracked by the tracking unit 53. In the present embodiment, the actual area of the moving object region (human attribute element 1), the background luminance correlation (human attribute element 2), and the background edge preservation ratio (human attribute element 3) are calculated.

  The “real area” is a feature amount indicating the validity of the size of the moving object region as a human size. For example, the attribute information calculation unit 54 calculates the actual value from the height x width in the real space obtained based on parameters such as the position in the input image 40 and the installation height and depression angle information of the camera 2 for the rectangle circumscribing the moving object region. Calculate the area. “Background luminance correlation” is a feature quantity that utilizes the fact that “human beings are different from light and shadow, and because they wear clothes, the texture is different from the background”. For example, the attribute information calculation unit 54 calculates the correlation between the luminance value of the background image 41 and the luminance value of the input image 40 in the moving object region. The lower the correlation value, the higher the human validity. The “background edge preserving rate” is a feature quantity that utilizes the fact that “human beings have a substance unlike light and shadow and therefore hide background objects”. For example, the attribute information calculation unit 54 calculates “the number of edges of the input image 40 and the edge of the background image 41” × 100 in the “number of edges of the background image 41” in the moving object region × 100. That is, as the “background edge preservation ratio” is lower, the background object is hidden by the entity and the edge of the background object disappears from the input image 40, and the validity as an actual person is higher.

  In the present embodiment, the three feature amounts (human attribute elements) have been described. However, the present invention is not limited to these, and some of the three feature amounts may be selected. The feature amount may be used.

The attribute information calculation means 54 calculates human attribute information using the calculated value of each human attribute element. Specifically, the value of each person attribute element is substituted into the membership function shown in FIG. 5 to obtain a value that becomes 1 for a person and 0 for a person. Then, the human attribute information is calculated by substituting the obtained value into the following equation.
(Formula 1)
Person attribute information = person attribute value element 1 × person attribute value element 2 × person attribute value element 3 (1)

  Next, calculation of insect attribute information will be described. The attribute information calculation unit 54 calculates a feature amount (insect attribute element) for calculating insect attribute information for each frame of the moving object region tracked by the tracking unit 53. In the present embodiment, the background difference minus point rate (insect attribute element 1), the luminance average value (insect attribute element 2), and the input edge strength average value (insect attribute element 3) are calculated.

  The “background difference minus point rate” is a feature quantity that utilizes the fact that “the insect appears in the darkness of the background image by blocking the ambient light because it appears in the immediate vicinity of the camera”. For example, the attribute information calculation unit 54 sets “the luminance value of the input image 40−the number of pixels in which the luminance value of the background image 41 is a negative value” × 100 in the “pixel area of the moving object region” in the moving object region × 100. Calculated as “background difference minus point rate”. The higher the “background difference minus point rate”, the darker the moving object region is than the background image, and the higher the relevance as an insect that is shielded from the ambient light. “Luminance average value” is a feature that uses the fact that “the insects appear in the immediate vicinity of the camera so that they are darker than the background image by blocking the ambient light, and that the insects themselves have a blackish color” It is. For example, the attribute information calculation unit 54 calculates an average value of luminance values in the moving object region. The lower the luminance average value, the more the moving object area is black, and the higher the validity as an insect that appears black in the input image. The “input edge intensity average value” is a feature amount using the fact that “the insect appears in the vicinity of the camera, so the focus is blurred”. The attribute information calculation unit 54 calculates, for example, “the total edge intensity of the input image 40” occupying the “pixel area of the moving object area” in the moving object area as the “input edge intensity average value”. That is, the lower the “input edge strength average value”, the more blurred the image is, and the higher the relevance of the insect that appears in the immediate vicinity of the camera.

  In the present embodiment, three feature amounts (insect attribute elements) have been described. However, the present invention is not limited to these, and some of the three feature amounts may be selected. The feature amount may be used.

The attribute information calculation means 54 calculates insect attribute information using the calculated value of each insect attribute element. More specifically, the value of each insect attribute element is substituted into the membership function shown in FIG. 5, and a value that becomes 1 when it seems to be an insect and 0 when it does not seem to be an insect is obtained. The insect attribute value information is calculated by substituting the obtained value into the following equation.
(Formula 2)
Insect attribute value information = insect attribute value element 1 × insect attribute value element 2 × insect attribute value element 3 (2)

  In the present embodiment, insect attribute information representing the nature of insects is used as attribute information other than a person, but the present invention is not limited to this. For example, what is necessary is just to obtain | require planting attribute value information etc. from the shadow attribute value information from the feature-value showing shadowiness, and the feature-value showing plantingness.

  The area association means 55 has a predetermined positional relationship among the moving object areas tracked by the tracking means 53 based on the position of each moving object area calculated for each frame by the position calculation means 52. Group moving object regions in association with each other. In addition, other moving object regions are grouped independently. The association processing is performed between the moving object regions that are tracked as the same moving object among the moving object regions extracted from the input image 40 captured at different times.

  In the present embodiment, the region association unit 55 groups the moving object regions whose positions are within a predetermined range based on the position of the moving object region calculated by the position calculating unit 52. Here, the predetermined range is a range in which it is possible to determine that the temporal change in the position of the moving object on the image is small, that is, the state where the moving object remains on the spot or the moving speed of the moving object is slow. To do. This is a state in which the attribute information of the moving object region is unlikely to fluctuate based on the knowledge that when the position on the image does not change, the way the moving object appears and the background object that exists behind the moving object does not change. It can be said. For this reason, in this embodiment, moving object regions located in a predetermined range are associated and grouped to group moving object regions located in a mutual position where the attribute information of the moving object is difficult to change. For example, the region association unit 55 sets a reference position, and two moving object regions in which the distance between the reference position and the position of the moving object region calculated by the position calculation unit 52 is within a predetermined distance R after the set time point. Are related to each other. Specifically, the region association unit 55 sets the center of gravity position of the moving object region of the frame at the start of the tracking process as a reference position, and assigns the moving object region to a new group. Then, the pixel distance between the center of gravity position of the moving object area of the next frame and the reference position is obtained, and when the pixel distance is within a predetermined distance R (for example, within 10 pixels), the moving object area and the reference position are set. Are associated with the moving object region and assigned to the same group. Further, the pixel distance between the center of gravity position of the moving object region and the reference position of the next frame is obtained. When the pixel distance is within a predetermined distance R (for example, within 10 pixels), the moving object region is set to the reference position. The moving object region in which the reference position is set in association with the set moving object region and the moving object region in the next frame and the moving object region in the next frame are assigned to the same group. The area associating means 55 sequentially repeats this process for each frame until tracking of the moving object area is completed, and associates all the moving object areas. If the pixel distance between the centroid position of the moving object region of the frame to be processed and the reference position is not within the predetermined distance R, the centroid position of the moving object region is set as a new reference position, and The moving object area is assigned to a new group and the process is repeated. Further, if the pixel distance between the center of gravity of the moving object area of the next frame of the frame for which the reference position is set and the reference position is not within the predetermined distance R, only the moving object area of the frame for which the reference position is set belongs. There can be one group.

  In the present embodiment, the predetermined range as the association condition is set by the pixel distance. However, the present invention is not limited to this, and an actual distance calculated from the installation height of the camera 2 or the depression angle may be used. Further, by using both the pixel distance and the actual distance, the pixel distance when the moving object exists in the far direction in the visual axis direction of the camera 2 and the pixel distance when the moving object exists in the vicinity of the visual axis direction of the camera 2 You may make it absorb a difference. For example, a pixel distance of 10 pixels or less and 5 pixels or a real distance of 50 cm or less may be associated. Further, a range of a predetermined shape on the image or the real space may be set as the predetermined range. Further, a range in which the moving object regions overlap each other may be set as the predetermined range.

  Here, with reference to FIG. 6, association of the moving object region in the present embodiment will be described. FIG. 6 is a diagram showing an example of association in the present embodiment. 6A corresponds to FIG. 3, and FIG. 6B corresponds to FIG.

  First, FIG. 6A will be described as an example. The black circle indicates the position of the center of gravity of the moving object region in each frame. For example, the center of gravity position of the moving object region 13a at time T-2 in FIG. 3 is the center of gravity position T-2 in FIG. The white circle is a circle having a predetermined distance R as a radius.

  For the moving object region 13a, the tracking process is started at time T-2. For this reason, the gravity center position T-2 is set as the reference position, and the moving object region 13a at time T-2 is assigned to the group 1a. Since the pixel distance between the center of gravity T-1 of the moving object region 13a extracted from the frame at time T-1 that is the next frame and the reference position is equal to or greater than the predetermined distance R, the moving object at time T-1 The area 13a is not associated with the moving object area 13a at time T-2, and the center of gravity position T-1 is set as a new reference position. As a result, only the moving object region 13a at time T-2 belongs to the group 1a, and the moving object region 13a at time T-1 belongs to the new group 2a. Since the pixel distance between the centroid position T of the moving object region 13a extracted from the frame at time T, which is the next frame, and the centroid position T-1, which is the reference position, is within the predetermined distance R, The moving object area 13a is associated with the moving object area 13a at time T-1 and assigned to the group 2a. Further, the pixel distance between the centroid position T + 1 of the moving object region 13a extracted from the frame at time T + 1, which is the next frame, and the centroid position T-1, which is the reference position, is within the predetermined distance R. The moving object region 13a is associated with the moving object region 13a at time T-1 and the moving object region 13a at time T, and is assigned to the group 2a. Since the pixel distance between the centroid position T + 2 of the moving object region 13a extracted from the frame at the time T + 2 that is the next frame and the centroid position T-1 that is the reference position is equal to or greater than the predetermined distance R, the moving object at the time T + 2 The region 13a is not associated with the moving object region 13a at time T-1, and the center of gravity position T + 2 is set as a new reference position. As a result, the moving object region 13a at times T-1, T, and T + 1 belongs to the group 2a, and the moving object region 13a at time T + 2 belongs to the new group 3a. Further, since the pixel distance between the centroid position T + 3 of the moving object area 13a extracted from the frame at the time T + 3 which is the next frame and the centroid position T + 2 which is the reference position is within the predetermined distance R, the moving object at the time T + 3 The region 13a is associated with the moving object region 13a at time T + 2 and assigned to the group 3a.

  In this way, with respect to the moving object area 13a in FIG. 3, the moving object area 13a at time T-2 is the group 1a, the moving object area 13a from time T-1 to time T + 1 is the group 2a, and the movement from time T + 2 to time T + 3. The object region 13a is associated as a group 3a.

  Next, FIG. 6B will be described. The black circle indicates the position of the center of gravity of the moving object region in each frame, as in FIG. For example, the center of gravity position of the moving object region 13b at time t-2 in FIG. 4 is the center of gravity position t-2 in FIG. 6B. Further, the white circle is a circle having a predetermined distance R as a radius as in FIG.

  The moving object region 13b starts tracking processing at time t-2. For this reason, the gravity center position t-2 is set as the reference position, and the moving object region 13b at time t-2 is assigned to the group 1b. Since the pixel distance between the center of gravity position t-1 of the moving object region 13b extracted from the frame at the time t-1 that is the next frame and the reference position is equal to or greater than the predetermined distance R, the moving object at the time t-1 The region 13b is not associated with the moving object region 13b at time t-2, and the center of gravity position t-1 is set as a new reference position. As a result, only the moving object region 13b at time t-2 belongs to the group 1b, and the moving object region 13b at time t-1 belongs to the new group 2b. The pixel distance between the center of gravity position t of the moving object region 13b extracted from the frame at time t, which is the next frame, and the center of gravity position t-1, which is the reference position, is within the predetermined distance R. The moving object area 13b is associated with the moving object area 13b at time t-1 and assigned to the group 2b. Further, since the pixel distance between the centroid position t + 1 of the moving object region 13b extracted from the frame at the time t + 1 that is the next frame and the centroid position t-1 that is the reference position is within the predetermined distance R, the pixel at the time t + 1 The moving object area 13b is assigned to the group 2b in association with the moving object area 13b at time t-1 and the moving object area 13b at time t. Since the pixel distance between the centroid position t + 2 of the moving object region 13b extracted from the frame at time t + 2, which is the next frame, and the centroid position t-1, which is the reference position, is greater than or equal to the predetermined distance R, the moving object at time t + 2 The region 13b is not associated with the moving object region 13b at time t-1, and the center of gravity position t + 2 is set as a new reference position. As a result, the moving object region 13b at time t-1, t, t + 1 belongs to the group 2b, and the moving object region 13b at time t + 2 belongs to the new group 3b. In addition, since the pixel distance between the centroid position t + 3 of the moving object region 13b extracted from the frame at time t + 3 which is the next frame and the centroid position t + 2 which is the reference position is within the predetermined distance R, the moving object at time t + 3 The region 13b is associated with the moving object region 13b at time t + 2 and assigned to the group 3b.

  In this way, with respect to the moving object region 13b in FIG. 4, the moving object region 13b at time t-2 is group 1b, the moving object region 13b from time t-1 to time t + 1 is group 2b, and the moving object region 13b from time t + 2 to time t + 3 is moved. The object region 13b is associated as a group 3b.

  The representative attribute information calculation unit 56 obtains one piece of representative attribute information representing the attribute information of the moving object area included in the same group for the moving object areas associated with each other as the same group by the area association unit 55. .

  In the present embodiment, the average value of the human attribute information and the insect attribute information calculated by the attribute information calculation means 54 is calculated for the moving object regions assigned to the same group, and the average values are used as the representative attribute information and the representative insect. Attribute information. The representative attribute information calculation unit 56 stores the representative attribute information calculated up to the current frame in the storage unit 4 and outputs it to the target determination unit 57 described later.

  In the present embodiment, the average value is calculated as the representative attribute information, but the present invention is not limited to this. The representative attribute information may be any information as long as each attribute information obtained from the moving object areas constituting the group is collected to a value that can be regarded as an attribute area obtained from one moving object area. For example, the representative attribute information calculation unit 56 may calculate by a method that can calculate a value that almost falls within the range from the minimum value to the maximum value of the attribute information of the moving object area that constitutes the group. Also good. Specifically, a median value, maximum value, minimum value, weighted average, a value obtained by correcting the accumulated value to be low by division, or the like may be calculated as representative attribute information.

  The target determination unit 57 determines whether or not the moving object region is a detection target based on the representative attribute information calculated by the representative attribute information calculation unit 56. In the present embodiment, the accumulated value of representative attribute information and the accumulated value of representative insect attribute information calculated from the start of tracking to the current frame are calculated, and the accumulated value of representative attribute information is equal to or greater than a threshold (for example, 3 or more) and the ratio of the representative attribute information to the representative insect attribute information is equal to or greater than a threshold (for example, 2.5 or more), it is determined that the moving object region is a detection target, that is, a human. The target determination unit 57 outputs a determination result for the moving object region to the abnormality determination unit 6 described later.

  In the present embodiment, the determination is performed using the ratio between the accumulated value of the representative attribute information and the accumulated value of the representative insect attribute information, but the determination method is not limited to this. For example, the determination may be made based on one representative attribute information of representative person attribute information and representative insect attribute information. Specifically, the accumulated value of representative attribute information, the accumulated value of representative insect attribute value information, the average value of representative attribute information and the average value of representative insect attribute information, the minimum and maximum values of representative attribute information, the representative The determination may be made according to the minimum value or the maximum value of the insect attribute information, or a plurality of these may be used for the determination. In this case, the attribute information calculation unit 54 and the representative attribute information calculation unit 56 may calculate attribute information necessary for the determination. In addition, when the accumulated value or the average value of the representative attribute value is used, it is preferable in that the representative attribute value in each group of the moving object region is treated equally and the determination by the target determination unit 57 does not become extreme.

  In the present embodiment, the determination is performed based on the representative attribute information calculated from the start of tracking to the current frame. However, the present invention is not limited to this, and the most recent predetermined number of groups (for example, for five groups). The determination may be made based on the representative attribute information.

  Here, with reference to FIG. 7, calculation of representative attribute information in the present embodiment will be described. FIG. 7 is a diagram showing each attribute information and each representative attribute information in the present embodiment. 7A corresponds to FIGS. 3 and 6A, and FIG. 7B corresponds to FIGS. 4 and 6B.

  As shown in FIG. 7A, the human attribute information and insect attribute information calculated for the moving object region 13a in FIG. 3 and the representative attribute information and representative insect attribute information for each group of moving object regions associated with each other are as shown in FIG. It becomes “3” and “0”, respectively. Therefore, the ratio of the representative attribute information to the representative insect attribute information is “∞”, which is a large value. Therefore, when the determination condition that the person is a human being is applied when the accumulated value of the representative attribute information is greater than or equal to a threshold (for example, 3 or more) and the ratio is greater than or equal to the threshold (for example, 2.5 or more), The object region 13a is determined to be a human.

  Also, the human attribute information and insect attribute information calculated for the moving object area 13b in FIG. 4 and representative attribute information and representative insect attribute information for each group of moving object areas associated with each other are shown in FIG. 7B. Thus, “3” and “2.2” are obtained. Therefore, the ratio of the representative attribute information to the representative insect attribute information is “1.4”. Therefore, when the determination condition that the person is a human being is applied when the accumulated value of the representative attribute information is equal to or greater than a threshold (for example, 3 or more) and the ratio is equal to or greater than the threshold (for example, 2.5 or more) It is determined that the object region 13b is not a person.

  Here, as shown in FIG. 7B, the human attribute information of the moving object region 13b is higher than the insect attribute information in the frames at times t-1, t, and t + 1. In other words, insects appear in the same size as humans when viewed in the immediate vicinity of the camera, and since there are entities similar to humans, it is difficult to distinguish humans and insects from human attribute information alone. Therefore, the determination is performed using the insect attribute information indicating the insect likeness. However, when the insect overlaps the background tree 11 at times t-1, t, t + 1, the insect attribute element (for example, background difference minus point rate) calculated using the luminance of the background image is accurately calculated. Can not do. This is because the trunk portion of the tree 11 is originally projected with a low luminance value, and therefore there are more portions where the luminance is lower than the luminance value of the insect. For this reason, if the attribute information calculated for each frame is simply accumulated as in the prior art, when the insect stays at a position where the attribute information cannot be accurately calculated, such as time t-1, t, t + 1, At the time of the frame of t + 1, the accumulated value becomes “4” and the ratio of attribute information becomes “2.5”, and each exceeds the above-described threshold. As a result, the moving object region 13b due to the insect is erroneously determined as “human”. According to the present invention, moving object regions of the same object at different times are associated and grouped based on their positions, and representative attribute information representing the group is calculated for each group and used for determination of a detection target. As a result, the attribute information calculated in the frames at times t−1, t, and t + 1 when the moving object region remains at a position where the attribute information cannot be accurately calculated can be combined into one, and as a result, movement by insects can be performed. It is possible to prevent the object region 13b from being erroneously determined as “human”.

  The abnormality determination unit 6 determines whether there is an abnormality in the monitoring area based on the determination result in the object determination unit 57 of the image processing unit 5. In the present embodiment, the abnormality determination unit 6 determines “abnormal” when there is a moving object region determined to be a human, and outputs a signal to that effect to the output unit 7.

  When the “abnormal” signal is input from the abnormality determination unit 6, the output unit 7 outputs an abnormal signal to an alarm unit (not shown), and notifies the occurrence of an abnormality to the surroundings by sounding a buzzer or displaying a warning light. To do. Alternatively, the monitoring center may be notified of the occurrence of an abnormality by outputting the monitoring center (not shown) to a monitoring center via a communication network such as the Internet.

  The background image update unit 8 updates the background image 41 stored in the storage unit 4 using the image (input image 40) taken by the camera 2 and the determination result of the target determination unit 57. Specifically, the background image 41 is not updated when the target determination unit 57 determines that there is a person, and the background image 41 is updated when there is no person.

  Note that the method of updating the background image 41 is not limited to this, and if there is a moving object region without a human being, the frequency of updating the background image may be reduced. Further, a method of updating at regular time intervals, a method of updating with an image determined to have no moving object region, a method of updating when an illumination change is detected, or the like may be applied.

  Hereinafter, image monitoring processing using the image monitoring apparatus 1 of the present embodiment will be described with reference to the flowcharts of FIGS. 8 and 9. FIG. 8 is a flowchart of the image monitoring process of the present embodiment. FIG. 9 is a flowchart of the area association process of this embodiment.

  When starting the image monitoring process, the image monitoring apparatus 1 sequentially acquires images taken by the camera 2 by the image input unit 3 as the input image 40 in step S1. The following steps are performed on the acquired input image 40 of one frame.

  In step S <b> 2, the moving object region extraction unit 51 performs threshold processing on the absolute value of the difference between the input image 40 acquired by the image input unit 3 and the background image 41 stored in the storage unit 4. The difference area (binary) is extracted.

  In step S3, the moving object area extracting means 51 labels the difference area extracted in step S2 to generate a label area. In this embodiment, this label area is a moving object area.

  In step S4, the position calculation means 52 calculates the gravity center position for each moving object area. The position calculation means 52 obtains the position of the center of gravity for each moving object area extracted from the input image 40 and stores it in the storage unit 4 and outputs it to the tracking means 53 and the area association means 55.

  In step S5, the tracking unit 53 associates the moving object area of the current frame with the moving object area of the previous frame. In addition, tracking is started for a moving object area that is not associated with the moving object area of the previous frame as a new tracking target. Also, the tracking of the moving object area that has not been associated with the moving object area of the current frame is terminated with respect to the moving object area that has been tracked up to the previous frame.

  The following steps S6 to S11 are executed for each moving object area by sequentially setting all the moving object areas being tracked in the input image 40 as the moving object area of interest.

  In step S7, the attribute information calculation means 54 calculates feature amounts (attribute elements) for calculating human attribute information and non-human attribute information (insect attribute information in the present embodiment) for the target moving object region for each frame. calculate.

  In step S8, the attribute information calculation unit 54 calculates the human attribute information and the insect attribute information for each frame using the feature amount calculated in step S7. The attribute information calculation unit 54 stores the calculated human attribute information and insect attribute information in the storage unit 4 and outputs them to the representative attribute information calculation unit 56.

  In step S9, the region association unit 55 associates and groups the moving object regions using the center of gravity position of the moving object region calculated by the position calculation unit 52. The area association means 55 stores information on the result of grouping the moving object areas in the storage unit 4. Details of the processing by the region association means 55 will be described later with reference to the flowchart of FIG.

  In step S10, the representative attribute information calculation means 56 calculates the average values of the human attribute information and insect attribute information calculated in step S7 for each group of moving object regions.

  In step S11, the target determination unit 57 determines whether or not the moving object region is a human. Specifically, the accumulated value of the average value of the human attribute information for each group calculated in step S10 and the accumulated value of the average value of the insect attribute information are calculated, and the ratio thereof is obtained. The ratio and the accumulated value of the average value of the human attribute information are compared with the respective threshold values to determine whether or not the moving object region is a human. Thereafter, the target determination unit 57 outputs the determination result to the abnormality determination unit 6. At this time, even if there are only human attribute information and insect attribute values calculated from one frame, the ratio and the accumulated value of the average value of the human attribute information are compared with the respective threshold values. What is necessary is just to determine whether it is a human.

  In step S12 and step S13, the abnormality determination unit 6 determines “abnormal” if there is a moving object region determined as “human” among the moving object regions extracted from the input image 40 based on the determination result. Then, a signal to that effect is output to the output unit 7, and the process proceeds to step S14. Based on the determination result, the abnormality determination unit 6 determines “no abnormality” when there is no moving object region determined as “human” among the moving object regions extracted from the input image 40, and the process proceeds to step S15. To migrate.

  In step S14, the output unit 7 outputs an abnormal signal to the outside of the image monitoring apparatus 1, and the process proceeds to step S15.

  In step S15, the background image update unit 8 determines whether to update the background image 41 in the storage unit 4, and then updates the background image 41. Thereafter, whether or not the background image 41 is updated, the process returns to step S1, the input image 40 of the next frame is acquired, and the image monitoring process is continued.

  Next, the region association processing according to the present embodiment will be described with reference to the flowchart of FIG.

  First, in step S91, the region association unit 55 determines whether or not the moving object region has been tracked by the tracking unit 53 from the current frame. That is, it is determined whether or not the moving object region appears for the first time. If the moving object region appears for the first time, the process proceeds to step S93. If the moving object area does not appear for the first time, the process proceeds to step S92.

  In step S92, the distance between the set reference position and the center of gravity position of the moving object region calculated by the position calculation means 52 is calculated, and it is determined whether or not the distance is equal to or greater than a predetermined distance R. If it is greater than or equal to the predetermined distance R, the process proceeds to step S93, and if not greater than the predetermined distance R, the process proceeds to step S95.

  In step S93, the moving object region is assigned to a new group. In step S94, the center of gravity position of the moving object area calculated by the position calculating unit 52 is set as a new reference position for the moving object area. On the other hand, in step S95, the moving object area is associated with the moving object area of the past frame in which the distance between the set reference position and the center of gravity position is within the predetermined distance R, and assigned to the same group. That is, it is assigned to the same group as the previous frame.

  When the process of step S94 or step S95 ends, the group determination process for the moving object region ends.

  As described above, according to the present embodiment, a plurality of moving object regions that are likely to be erroneously detected as moving objects other than the detection target are grouped in association with each other, and movements included in the group are included. It is possible to prevent the influence on the determination of the object region from becoming too large. Thus, the detection accuracy of the detection target can be increased in the image monitoring apparatus that sequentially processes the images obtained by capturing the monitoring area and determines whether or not the detection target exists in the monitoring area.

<Modification 1>
In the above embodiment, the region association unit 55 performs the process of associating the moving object region and assigning it to the group based on whether the distance from the reference position is within a predetermined distance. However, it is an object of the present invention to be frequently used for determining whether or not the attribute information calculated during the time when the moving object region remains at a position on the image where it is difficult to accurately calculate the attribute information. It is something to prevent. For this reason, an area in which attribute information may be difficult to calculate with high accuracy is set in advance on the background image, and the moving object area is associated with whether or not the position exists in the area. It may be. Instead of setting an area on the background image, an area that may be difficult to accurately calculate attribute information may be set on the input image.

  For example, the trunk portion of the tree 11 included in the background image 41 illustrated in FIGS. 3 and 4 may deteriorate the calculation accuracy when calculating the insect attribute information. For this reason, the area of the tree 11 of the background image 41 is set and stored in advance, and the moving object areas at the times t−1, t, and t + 1 where the moving object area exists at the position of the area are associated and associated. The moving object region may be determined using representative attribute information. In this case, the area of the tree 11 set in advance in the storage unit 4 may be stored, and the area association unit 55 may set the position in the area of the tree 11 as the reference position.

<Modification 2>
In the above-described embodiment and modification, processing is performed regardless of the number of moving object regions associated with each other, but the present invention is not limited to this. For example, the representative attribute information calculation unit 56 refers to the grouping result information associated with the moving object area, and whether or not only two or less moving object areas are assigned to all the groups created up to the present time. Determine. Then, when all the groups include only two or less moving object regions, the target determination unit 57 uses the attribute information calculation unit 54 instead of the representative attribute information calculated by the representative attribute information calculation unit 56 for each frame. It is only necessary to determine whether or not the moving object region is a detection target using the attribute information calculated in step (b).

<Modification 3>
In the embodiment and the modification described above, the object determination unit 57 determines whether or not the object is a detection object using the representative attribute information calculated by the representative attribute information calculation unit 56, and the abnormality determination unit 6 determines that the determination result is Based on this, the presence or absence of abnormality was determined. However, it is not limited to such processing. For example, the target determination unit 57 includes determination of a detection target based on the representative attribute information calculated by the representative attribute information calculation unit 56 and determination of a detection target based on the attribute information calculated for each frame by the attribute information calculation unit 54. Both are performed, and the abnormality determination unit 6 determines that the detection target exists in both the detection target determination result using the representative attribute information and the detection target determination result using only the attribute information calculated for each frame. In some cases, “abnormal” may be set.

  DESCRIPTION OF SYMBOLS 1 Image monitoring apparatus, 2 Cameras, 3 Image input part, 4 Storage part, 5 Image processing part, 6 Abnormality determination part, 7 Output part, 8 Background image update part, 9 Human, 10 Spider, 11 Tree, 12 Building, 13a , 13b Moving object area, 40 input image, 41 background image, 51 moving object area extracting means, 52 position calculating means, 53 tracking means, 54 attribute information calculating means, 55 area association means, 56 representative attribute information calculating means, 57 Object determination means.

Claims (5)

An image monitoring apparatus for inputting an image obtained by sequentially imaging a monitoring area and determining whether or not a detection target exists in the image,
Moving object region extraction means for extracting a moving object region from the image;
Tracking means for sequentially tracking a moving object region by the same moving object on the image;
Attribute information calculating means for calculating attribute information representing the likelihood of detection for the moving object region;
In the moving object region by the same moving object, region association means for associating the moving object regions having a predetermined positional relationship to form one group and forming the other moving object regions independently as a group ,
Representative attribute information calculating means for calculating one representative attribute information using the attribute information of the moving object region constituting each group;
Target determination means for determining whether or not the detection target exists using the representative attribute information;
An image monitoring apparatus comprising: The image monitoring apparatus according to claim 1,
The image monitoring apparatus according to claim 1, wherein the region association unit uses the predetermined positional relationship as a relationship where the attribute information of the moving object region is located at a mutual position where the moving object region hardly changes. The image monitoring apparatus according to claim 1 or 2,
The image monitoring apparatus according to claim 1, wherein the region association means uses the predetermined positional relationship as a relationship in which the moving object region is located in a range set in advance on the image. The image monitoring apparatus according to claim 1 or 2,
The area associating means sets the position of the tracking start time of the moving object area as a reference position, and associates the relationship where the moving object area is located within a predetermined range from the reference position as the predetermined positional relation
When the moving object area is out of the predetermined range from the reference position, the position of the moving object area at that time is set as a new reference position. The image monitoring device according to any one of claims 1 to 4,
The image monitoring apparatus according to claim 1, wherein the target determination unit determines based on at least an accumulated value or an average value of the representative attribute information.

Images