JP2007060468A - Image sensor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image sensor capable of accurately determining whether or not a temporal varying domain included in an image picking up a monitoring space is caused by a moving object to be monitored. <P>SOLUTION: An image sensor extracts a varying domain from a differential between an image acquired by an image pick-up 2 and a past image, and determines whether or not the varying domain is caused by disturbance. An image sensor 1 is configured to comprise a pixel group extractor 21 for extracting a group of pixels having a predetermined feature amount in the varying domain; a deviation calculator 22 for calculating a degree of eccentricity for determining whether or not a position of that group of pixels is eccentric in the varying domain; and a disturbance determination section 13 for determining that the varying domain is caused by disturbance, if the degree of eccentricity is judged small. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image sensor, and more particularly to an image sensor used in a monitoring device.

  2. Description of the Related Art Conventionally, as an image sensor used in an image monitoring device, a moving object in an input image from a change mode between an image of a monitoring space input from a camera as an imaging device and an image acquired in the past with respect to the monitoring space. Image sensors that determine the presence or absence of a human (such as a human) are known. As this image sensor, for example, a currently captured input image is compared with a previously stored past image (background image), and an area having a difference in luminance between both images is extracted as a fluctuation area. From the degree of change in texture (pattern, etc.) of the extracted area, image fluctuations due to disturbance caused by changes in the amount of illumination light or shadows reflected in the monitoring space, or images of humans entering the monitoring space Is known (see Patent Document 1).

  With regard to such an image sensor, particularly in Patent Document 1, a change in edge and a change in texture between a change area obtained from a difference between an input image and a background image and a corresponding area of a background image corresponding to the change area. An image sensor is described in which a feature amount such as the polarity direction of luminance change is calculated to determine whether the fluctuation region corresponds to a human or due to a disturbance such as a shadow other than a human.

  For example, when the change area shows a negative brightness change (when the brightness value of the pixel in the change area is lower than the brightness value of the pixel in the corresponding area in the background image), or when the edge change is small, or When the texture change is small, the “shadowiness” of the fluctuation area is calculated to be high, and it is determined that the fluctuation area is a shadow.

However, in the conventional image sensor described above, when a person and a shadow extending from the person are extracted as a fluctuation region, the presence of the person may not be detected.
This is the case when the shadow is extended for a long time (for example, when the angle between the line connecting the light source and the person and the surface on which the shadow is projected is very narrow, as in the case just before sunset) The ratio of the shadow part in the area extracted as the fluctuation area increases, and the image features (edge similarity, texture similarity, etc.) of the shadow part dominate the whole fluctuation area. This is because “likeness” is calculated to be high.

  This is shown in FIG. If the human a1 and its shadow a2 and the simple shadow b are shown in the input image I, the difference between the human a1 and its shadow a2 and the simple shadow b2 are obtained by performing a difference from the background image B that does not exist. The part b can be extracted as a region where the luminance value of each pixel fluctuates, that is, as a fluctuating region.

  Here, when the shadow is extended for a long time from the person who appears black (that is, with low brightness) in the input image I in the monitoring space due to backlight or when the person is wearing black clothes. The fluctuation area (a1 + a2) including the “black” human and the shadow shows a negative luminance change with respect to the corresponding area of the background image B, and the edge change degree or the texture change degree calculated from the shadow part a2. Therefore, the “shadowiness” of the fluctuation region is generally calculated to be high.

  For this reason, the image feature calculated for the variation region (a1 + a2) is not significantly different from the image feature calculated for the variation region b including only the shadow. Therefore, in conventional image sensors, it is difficult to distinguish between a person with a shadow and a simple shadow, and a person with a shadow is determined as a simple shadow. There was a risk of not detecting humans.

  In addition, in order to deal with this problem that humans may not be detected, even if the “shadowiness” is calculated as high as possible, if there is a possibility of being a human, it is determined to be a human. There was a problem that a simple shadow was judged as a human and the possibility of misreporting increased.

JP-A-2001-243475

  In view of the above problems, the present invention provides an image sensor that can accurately determine whether or not a moving object is captured even when a moving object such as a human and its shadow are reflected in the monitoring space. The purpose is to do.

  In order to achieve the above object, the image sensor according to the present invention extracts a fluctuation region from a difference between an image acquired by an imaging unit and a past image, and determines whether or not the fluctuation region is caused by a disturbance. Determine. The image sensor includes a pixel group extraction unit that extracts a pixel group having a predetermined feature amount in the fluctuation region, and a deviation that calculates a degree of bias for determining whether the position of the pixel group is a biased position in the fluctuation region. When it is determined that the degree of bias is small, the fluctuation region includes a disturbance determination unit that determines that the fluctuation region is caused by a disturbance.

  As described above, by checking whether the pixel having the characteristic in the fluctuation area exists at a position biased with respect to the whole fluctuation area, a moving object with a long shadow appears in the image obtained by imaging the monitoring space. Even when the moving object is present, it is possible to accurately determine the presence or absence of the moving object without being affected by the shadow area.

  Note that the above disturbance refers to changes in the amount of illumination light that illuminates the monitoring space (light from light sources such as the sun, streetlights, and illuminating lights), weather changes, or car shadows that cross the vicinity of the monitoring space. This is caused by objects other than objects (including humans and animals) that have intruded. The past image is an image of the monitoring space taken before the captured image from which the variable region is extracted. Specifically, it can be an image obtained by photographing the monitoring space in advance at the time of installation or maintenance of the image sensor according to the present invention, or an image photographed by the image sensor according to the present invention a predetermined time ago.

  Further, the deviation calculation unit preferably calculates the degree of deviation of the position of the pixel group in the variation area based on the ratio of the distance between the center of gravity position of the variation area and the center of gravity position of the pixel group and the size of the variation area. . Since it is possible to accurately grasp the positional relationship of the pixel group having a predetermined feature amount occupying in the fluctuation region, it is possible to accurately check whether or not the position of the pixel group is biased with respect to the fluctuation region. Is possible.

  Furthermore, the pixel group is preferably a pixel group including any one of a low luminance pixel having a luminance equal to or lower than a predetermined threshold and a high luminance pixel having a luminance equal to or higher than the predetermined threshold. Furthermore, the pixel group extraction unit binarizes the fluctuation region with a threshold value that is equal to or lower than the luminance average value of the fluctuation region, extracts low luminance pixels in the fluctuation region, and sets a pixel group including the low luminance pixels as the pixel group described above. It is preferable to extract. This is because it is possible to easily and accurately extract pixel groups having different characteristics in the fluctuation region.

  In addition, the disturbance determination unit determines that the fluctuation region is caused by the disturbance when it is determined that the fluctuation region has lower luminance than the corresponding region in the past image and the degree of bias is small, and the fluctuation region is the past. When the brightness is higher than the corresponding area in the image, or when it is determined that the fluctuation area is lower in brightness than the corresponding area in the past image and the degree of bias is large, the fluctuation area is not caused by disturbance. Is preferably determined.

  Here, the determination as to which luminance is higher between the fluctuation area and the corresponding area in the past image is performed by comparing the average luminance value in the fluctuation area and the corresponding area.

  In addition, when the disturbance determination unit determines that the fluctuation region is not caused by the disturbance, the moving object determination unit further determines whether the fluctuation region is caused by the moving object based on another determination criterion. It is preferable to provide. By providing such a moving object determination unit, it is possible to accurately determine whether the fluctuation region in the monitoring space is caused by the moving object that is the monitoring target or the other object. It is.

  According to the present invention, it is possible to provide an image sensor that can accurately determine whether or not a moving object is reflected even when the moving object and its shadow are reflected in the monitoring space.

Hereinafter, the present invention will be described in detail with reference to the drawings.
An image sensor according to an example of the present invention, in an image obtained by capturing a space to be monitored (monitoring space), a moving object reflected in a captured image, particularly a luminance variation by a human, a shadow or a light amount change across the monitoring space, and the like. This is an image sensor equipped with a detection logic that distinguishes luminance fluctuations caused by external disturbances.

  In the following description, an example will be described in which a human (intruder) who has entered the monitoring space is detected and output as a moving object to be monitored.

FIG. 1 is a functional block diagram of an image sensor according to an example of the present invention.
An image sensor 1 according to an example of the present invention includes an imaging unit 2 that acquires a captured image of a monitoring space, a storage unit 3 that stores information used in various processes of the image sensor 1, and an intrusion into the monitoring space from the captured image. An image processing unit 4 for determining whether there is a person and an output unit 5 for outputting the determination result.

Hereinafter, each part will be described in detail.
The imaging unit 2 includes an imaging device such as a CCD camera. The imaging unit 2 converts the optical image of the monitoring space into a digitized image composed of discrete pixel groups, and a captured image I (x, y) (where each pixel is represented by a luminance value) (however, , X, and y represent the horizontal coordinate and vertical coordinate of the pixel in the image data, respectively, and the same applies to the acquired image, and outputs the captured image I (x, y) to the image processing unit 4. Further, the imaging unit 2 acquires one frame of the captured image I (x, y) at a predetermined time interval (for example, every 1/5 second) and outputs the acquired image to the image processing unit 4.
As an example, the captured image I (x, y) has a gray scale of 768 pixels horizontally, 494 pixels vertically (approximately 380,000 pixels) per frame, and a luminance value of 0 to 255 (256 gradations) per pixel. The brightness value has a larger value as it is brighter.

  The storage unit 3 includes a semiconductor memory such as a read only memory (ROM) and a random access memory (RAM), a magnetic disk (HDD), an optical disk drive such as a CD-ROM and a DVD-RAM, and a medium thereof. In addition, the storage unit 3 stores information used in various processes of the image sensor 1, for example, past images to be compared with the captured image, various threshold values, and the like. Then, necessary information is provided in response to a request from the image processing unit 4 or the like.

  The image processing unit 4 includes a calculation device such as a CPU and a numerical calculation processor, and software read from the storage unit 3 so as to be executable by the calculation device. Then, the image processing unit 4 executes processing such as fluctuation region extraction processing, feature calculation processing, and disturbance determination processing based on the digitized captured image I (x, y) acquired from the imaging unit 2. To do. Details of the image processing unit 4 will be described later.

The output unit 5 is means for outputting the abnormality determined by the image processing unit 4 to the outside, and includes an LED, a buzzer, and the like. When an abnormal signal is input from the image processing unit 4, the output unit 5 lights or blinks an LED or sounds a buzzer to notify the occurrence of an abnormality.
The output unit 5 is connected to a security device or a remote monitoring center (not shown) by wired or wireless communication, and is configured as a communication interface that outputs an abnormal signal input from the image processing unit 4 to them. Also good.

Hereinafter, the image processing unit 4 will be described in detail.
The image processing unit 4 extracts, from the captured image I (x, y), a variation region extraction unit 11 that extracts a variation region that is a pixel region having a difference in luminance value from the corresponding pixel in the past image, and the extracted variation region A feature calculation unit 12 that calculates the inside feature, a disturbance determination unit 13 that determines whether or not the fluctuation region is caused by a disturbance based on the feature calculated by the feature calculation unit 12, and the fluctuation region When it is determined that it is not caused by the moving object determination unit 14, the moving object determination unit 14 determines whether or not the fluctuation region is caused by an intruder.

Next, processing contents performed in the image processing unit 4 will be described with reference to FIG.
In general, in a region including a human and a shadow extending from the human (a shadow region including a human), the human is located at the end of the region, and the shadow extends toward the opposite end. Further, in a shadow area including a human, a luminance difference is generated between the human and the shadow. For example, even when a shadow extends from a human who appears black, such as when the subject is backlit, in the shadow area including this human, a difference in brightness occurs between the human and the shadow, and the appearance position of dark pixels is biased. This is because the reflectance of the ground on which the shadow is projected is different from the reflectance of human clothes, skin, and hair.

On the other hand, in a shadow-only area, dark pixels are distributed throughout the area. In addition, regarding a region where only a simple shadow is shown, for example, when the texture of the background part of the region where the shadow is projected is stored (visible through) because the shadow is pale, The texture appears almost uniformly, and the dark pixel group does not appear at a biased position in the region. In addition, when the shadow is strong, the texture of the background part of the area where the shadow is projected cannot be seen at all and becomes a dark area uniformly, and the dark pixel group does not appear at a biased position in the area. .
That is, in a shadow area including a human, a dark pixel group appears biased toward the edge of the area, whereas no bias is observed in a shadow-only area.

  Therefore, a difference image S of luminance values between corresponding pixels between a past image (hereinafter referred to as background image B) captured in the absence of an intruder and an input image I captured by the imaging unit 2 in the monitoring space. As a result, a large difference luminance value can be obtained for the intruder (region a1) and its shadow (region a2), or a region having only a shadow (region b). Therefore, regions where pixel groups having large difference luminance values are continuous are extracted as variable regions c (here, region a1 + a2 and region b). Next, a distribution obtained by averaging the luminance values of the pixels of the captured image included in each variation region c is used as a threshold, and the distribution of pixels having luminance values equal to or lower than the threshold is examined. Then, in the region (a1 + a2), pixels having a luminance value equal to or less than the threshold value are concentrated on one end in the region (a1 + a2), whereas in the region b, pixels having a luminance value equal to or less than the threshold value are included in the region b. Spread throughout. Therefore, by examining whether the degree of bias as the degree of increase of the pixel group having a luminance value less than or equal to the threshold is closer to one end in the fluctuation area, whether the fluctuation area c is caused by an intruder, a shadow, or the like It can be discriminated whether it is due to the disturbance.

The degree of bias is a level indicating whether or not the position of the pixel group is biased in the variation area c, and is a value that increases as the pixel group is biased to one end in the variation area. Say. In addition, when the degree of bias is small, an arbitrary straight line orthogonal to a straight line connecting the two most distant points in the fluctuation region c, and when the fluctuation region is divided into two, the pixels occupying each of the divided fluctuation regions This means that there is no significant difference in the ratio of pixels belonging to the group.
On the other hand, when the degree of bias is large, the pixel group occupying each of the divided variable areas when the variable area is divided into two by a certain straight line orthogonal to a straight line connecting the two most distant points in the variable area. This means that there is a significant difference in the ratio of pixels belonging to.

Hereinafter, each part of the image processing unit 4 will be described in detail.
The variable region extraction unit 11 calculates a luminance difference between pixels corresponding to each other in the captured image I (x, y) acquired from the imaging unit and the reference background image B (x, y), and the captured image I A difference image S (x, y) (= I (x, y) −B (x, y)) between (x, y) and the background image B (x, y) is obtained.

  The background image B (x, y) is a past image captured by the imaging unit 2 in a state where no intruder exists in the monitoring space, and is acquired in advance and stored in the storage unit 3. Then, the variation area extraction unit 11 expresses, from the difference image S (x, y), a pixel 2 (a variation pixel) in which the absolute value of the difference luminance value is equal to or greater than a predetermined value, and a difference 2 that represents other pixels as 0. A value image Sb (x, y) is generated. Here, as the predetermined value, a value of about 2 to 10 (for 256 gradations) is used according to the situation of the monitoring space in order to avoid the adverse effect due to noise and to appropriately extract the fluctuation region.

  When the difference binary image Sb (x, y) is generated, a substantially continuous variation pixel group is extracted as one variation region c (x, y). In this case, in order to remove noise, an opening operation of a morphological basic operation is performed on the difference binary image Sb (x, y), and the extracted variation pixel (difference binary image Sb (x, y)) is extracted. It is preferable to delete an isolated point of a pixel having a value of 1. Similarly, a closing operation is performed on the differential binary image Sb (x, y), and non-variable pixels (pixels having a value of 0 in the differential binary image Sb (x, y)) surrounded by the varying pixels. Is preferably deleted. Alternatively, a known filtering operation used for removing a minute area may be performed to delete the isolated point of the variable pixel. The fluctuation region c (x, y) is a region in which pixels having a pixel value of 1 are substantially continuous in the differential binary image Sb (x, y) subjected to the above-described noise removal processing.

  When the change area c (x, y) is obtained, the total number of pixels Ctotal included in the change area c (x, y) is calculated. Then, based on the minimum size that can be considered as an intruder, it is compared with a predetermined threshold value Thc. If the total number of pixels Ctotal is larger than the threshold value Thc, it is determined that a variable region exists. When it is determined that the fluctuation region c (x, y) does not exist, the next captured image is acquired by the imaging unit 2, and the fluctuation region is extracted again. On the other hand, when it is determined that a fluctuation region exists, the generated difference image S (x, y) and difference binary image Sb (x, y) are stored in the storage unit 3.

  The background image B (x, y) may be configured to be updated with an image captured every predetermined time such as every 10 minutes or every hour. In addition, by using a captured image acquired before a predetermined frame as a background image, a difference image between frames can be obtained as the above S (x, y) and used for extracting the fluctuation region c (x, y). Good.

  The feature calculation unit 12 checks the feature amount in the extracted fluctuation region c (x, y), and checks whether or not there is a bias in the position of a pixel showing a predetermined luminance in the fluctuation region. Therefore, the feature extraction unit 12 extracts a pixel group composed of low-luminance or high-luminance pixels in the extracted fluctuation region c (x, y), and the bias of the appearance position of the pixel group. And a deviation calculating unit 22 for calculating the degree.

  Hereinafter, a case where the degree of bias of the appearance position of the low luminance pixel group is calculated will be described as an example. When the degree of bias of the appearance position of the high brightness pixel group is calculated, the degree of bias of the high brightness pixel group is used in the subsequent processing instead of the degree of bias of the low brightness pixel group.

  The pixel group extraction unit 21 extracts a low-brightness pixel group in the fluctuation region c (x, y) (when using the degree of bias of the high-brightness pixel group, a high-brightness pixel group is extracted). The extraction of the low-luminance pixel group in the fluctuation area c (x, y) is performed by binarizing the fluctuation area c (x, y) in the captured image I (x, y) with a predetermined luminance threshold to obtain a low-luminance pixel. And high luminance pixels. Specifically, the following procedure is used.

First, an average value (luminance average value ca) for the luminance values of the changing pixels included in the changing region c (x, y) is calculated from the captured image I (x, y). Next, the luminance threshold value Th1 is calculated based on the following formula using the calculated luminance average value ca.
Th1 = ca * p
Here, the parameter p is a value set for extracting dark (low luminance) pixels while reducing the influence of high luminance noise, and is set to 0.9 as an example. By setting the parameter p to less than 1, dark pixels can be extracted more strictly. On the other hand, by setting the parameter p to be larger than 1, it is possible to extract more strictly bright (high luminance) pixels. Thus, the parameter p is used as an adjustment factor for extracting pixels to be extracted by binarization under more severe conditions. However, it is not always necessary to use the parameter p, and the value of the luminance average value ca itself may be used as the threshold value Th1.

  When the luminance threshold Th1 is calculated, the luminance threshold Th1 is compared with the luminance value of each pixel corresponding to the fluctuation region c (x, y) in the captured image I (x, y). Then, the pixel value of a pixel having a luminance value less than the luminance threshold Th1 is set to 1, and the pixel value of a pixel having a luminance value equal to or higher than the luminance threshold Th1 is set to 0, and extracted as a low luminance pixel group l (x, y). A pixel of the captured image I (x, y) corresponding to the pixel represented by the pixel value 1 in the low-luminance pixel group l (x, y) is a low-luminance pixel (a pixel that is not a low-luminance pixel (luminance threshold Th1) The high luminance pixel in the fluctuation region having the above luminance is represented as a pixel value 0 in the low luminance pixel group l (x, y)). Conversely, among the pixels corresponding to the fluctuation region c (x, y) in the captured image I (x, y), pixels having a luminance value equal to or higher than the luminance threshold Th1 are set as the high luminance pixel group h (x, y). May be extracted as The pixel of the captured image I (x, y) corresponding to the pixel represented by the pixel value 1 in the high luminance pixel group h (x, y) is a high luminance pixel (in addition, a pixel that is not a high luminance pixel (in the fluctuation region) Is represented as a pixel value of 0 in the high-luminance pixel group h (x, y)).

  Note that the method of extracting a low-luminance pixel group or a high-luminance pixel group is not limited to the above. For example, a person is experimentally arranged in advance in a monitoring space, a plurality of sample images taken by the imaging unit 2 are prepared, and an area where a person is reflected and an area where a shadow is reflected are analyzed by analyzing the sample images. By experimentally finding the best separable luminance threshold value and binarizing it into a pixel having a luminance value equal to or lower than the experimentally found luminance threshold value and other pixels, a low luminance pixel group or a high luminance value is obtained. A pixel group can be extracted.

The deviation calculation unit 22 is a degree of the appearance position of the low-luminance pixel group l (x, y) extracted by the pixel group extraction unit 21 with respect to the entire variation region c (x, y) (bias degree R). ) Is calculated. The degree of deviation R is the center-of-gravity position cg (cgx, cgy) of the fluctuation region c (x, y) (where cgx is the horizontal coordinate value in the captured image I (x, y), and cgy is the captured image I (x, y). y) representing the vertical coordinate value) and the barycentric position lg (lgx, lgy) of the low luminance pixel group l (x, y) (where lgx is the horizontal coordinate value in the captured image I (x, y)) lgy is calculated based on the distance to the vertical coordinate value in the captured image I (x, y). In other words, the deviation calculating unit 22 calculates the ratio between the distance d between the centers of gravity of the variation region c (x, y) and the low luminance pixel group l (x, y) in the variation region and the size of the variation region. This is output as the degree of bias R of the low luminance pixel group l (x, y). This is shown in FIG.
Here, the distance d between the centers of gravity is calculated based on the following formula (1).

As the size of the fluctuation area c (x, y), the length of the long side of the circumscribed rectangle of the fluctuation area c (x, y) is used. For example, when it is assumed that the center of gravity cg (cgx, cgy) of the fluctuation region is near the center of the circumscribed rectangle of the fluctuation region c (x, y), the low luminance pixel group l (in the fluctuation region c (x, y)) When the position of x, y) is greatly deviated toward one end of the fluctuation area, the distance d between the centers of gravity of the fluctuation area c (x, y) and the low luminance pixel group l (x, y) in the fluctuation area is , Approaches the distance r that divides the long side distance of the circumscribed rectangle into two. Therefore, in the present embodiment, the ratio between the distance d and the distance r is obtained based on the following equation (2), and the obtained value is stored as the degree of bias R of the low-luminance pixel group l (x, y). To remember.

The circumscribed rectangle used in the above calculation is preferably a circumscribed rectangle whose long and short sides are parallel to the major and minor axes of the ellipse when the variable region c (x, y) is approximated to an ellipse. Alternatively, a circumscribed rectangle having the smallest area may be used.
Thereby, it is possible to calculate the degree of bias of the appearance position of the pixel group exhibiting low luminance in the fluctuation region c (x, y).

The disturbance determination unit 13 is based on the image feature amount of the variation area calculated by the feature calculation unit 12, that is, the bias degree R calculated by the deviation calculation unit 22 in this embodiment, and the variation region is caused by a change in the amount of light. It is determined whether or not it is due to disturbance.
The disturbance determination unit 13 performs determination according to the following procedure.
First, from the difference image S (x, y) stored in the storage unit 3, an average value of difference luminance values (difference luminance average value ea) of each pixel in the fluctuation region c (x, y) is calculated. Then, based on the difference luminance average value ea, the fluctuation area c (x, y) extracted from the captured image I (x, y) is darker than the luminance average value of the corresponding area of the background image B (x, y). It is determined whether or not. Here, when only one variable region is extracted, the difference luminance average value ea of this variable region is calculated by the following equation (3), and when a plurality of variable regions are extracted. The difference luminance average value ea can be calculated by obtaining and adding the difference luminance value of each pixel in the region from the difference image and dividing by the total number of pixels in the region for each variable region.

Here, m is the number of pixels in the horizontal direction of the captured image I (x, y), and n is the number of pixels in the vertical direction. When the calculated luminance average value ea shows a positive value or 0, that is, when the fluctuation area c (x, y) is brighter than the corresponding area of the background image B (x, y), the fluctuation area c (x, It is determined that y) is not caused by a disturbance such as a shadow due to a change in the amount of light, and the disturbance determination unit 13 ends the process. In such a case, the fluctuation region is likely to be caused by a moving object existing in the monitoring space.

  On the other hand, when the difference luminance average value ea shows a negative value, that is, when the fluctuation area c (x, y) is darker than the corresponding area of the background image B (x, y), the fluctuation area c (x, y). May be a shadow.

  As described above, when a shadow area including a human being is extracted as the fluctuation area c (x, y), a dark pixel group appears evenly toward the edge of the fluctuation area c (x, y). On the other hand, when a region having only a shadow is extracted as the variation region c (x, y), such a bias is not seen.

  Therefore, the disturbance determining unit 13 calculates the deviation calculation unit 22 when the fluctuation region c (x, y) is darker than the corresponding region in the background image B (x, y) and there is a possibility of a shadow. Based on the biased degree R, it is determined whether the fluctuation region is a simple shadow (a region with only a shadow) or a region that may include a human being. Then, the disturbance determination unit 13 determines that the fluctuation region may include a human when the degree of bias R is equal to or greater than the predetermined threshold Th2, and the fluctuation region c (x, y) is determined not to be due to disturbance. To do. On the other hand, if the degree of bias is less than the threshold Th2, it is determined that the fluctuation region c (x, y) is due to disturbance. Here, the predetermined threshold Th2 is set to, for example, 0.0625 (the distance d between the centers of gravity is equivalent to ¼ of the length r that is ½ of the long side of the circumscribed rectangle).

  By making such a determination, even if a person is extracted as a fluctuation region together with a shadow, it can be distinguished from a simple shadow due to a disturbance such as a change in the amount of light. In particular, even if a shadow is long extending from a human being reflected in black due to backlight or black clothes, it is possible to prevent it from being mistakenly judged as a mere shadow, and an intruder into the surveillance space. It is possible to accurately determine whether or not there is.

  When the disturbance determination unit 13 ends the determination process, the disturbance determination unit 13 sends the determination result J to the moving body determination unit 14. The determination result J is set to 1 when it is determined that the fluctuation region c (x, y) is due to disturbance, and is set to have a value of 0 otherwise.

  When the moving body determination unit 14 receives the determination result J from the disturbance determination unit 13, the moving body determination unit 14 refers to the determination result J. If the determination result J is determined that the fluctuation region c (x, y) is not due to disturbance (J = 0), the fluctuation results based on the determination criterion using other additional information. It is determined whether the region c (x, y) is due to an intruder.

  When the moving body determination unit 14 determines that the fluctuation region c (x, y) is caused by a human, the moving body determination unit 14 outputs an abnormal signal to the output unit 5. In addition, as other additional information used for the determination of the moving body determination unit 14, for example, “the size of the variation region c (x, y)”, “the shape of the variation region c (x, y)”, or “ Movement information "can be used.

  Here, “the size of the variation region c (x, y)” can be expressed as the total number of pixels included in the variation region c (x, y), and is used to determine whether or not the size of a human being is satisfied. Can be used for Whether or not a human-like size is satisfied can be determined empirically based on the size of the monitoring space shown in the captured image I (x, y). The “shape of the fluctuation region c (x, y)” can be the aspect ratio of the fluctuation region c (x, y), and can be used to determine whether or not a human-like shape is satisfied. . For the calculation of the aspect ratio, the ratio between the distance between the upper end coordinate and the lower end coordinate of the fluctuation region c (x, y) and the distance between the left end coordinate and the right end coordinate can be used. Furthermore, the “movement information” refers to the center of gravity position of the variable region c (x, y) between a plurality of temporally adjacent frames, and the variable region c (x, x, calculated based on the difference between the center of gravity positions. The movement speed of y) can be obtained. This moving speed is compared with a threshold value corresponding to a limit of a normal speed at which a human can move. If the moving speed is equal to or higher than the threshold value, it can be determined that the human cannot.

The operation procedure of the image sensor 1 according to an example of the present invention will be described below with reference to FIG.
FIG. 4 shows an operation flowchart of the image sensor 1 according to an example of the present invention.
First, the captured image I (x, y) obtained by capturing the monitoring space is acquired by the imaging unit 2 (step S01).

  Next, the captured image I (x, y) is transmitted to the image processing unit 4, and a difference image S (x, y) representing a difference luminance value between corresponding pixels with the background image B (x, y) is calculated. Based on the difference image S (x, y), it is determined whether or not the fluctuation region c (x, y) exists. If the total number of pixels Ctotal included in the fluctuation area c (x, y) is equal to or greater than the predetermined value Thc, the fluctuation area c (x, y) exists, otherwise, the fluctuation area c (x, y). Is determined not to exist. (Step S02). When it is determined that the fluctuation region c (x, y) does not exist, the captured image is acquired again. On the other hand, when it is determined that the fluctuation region c (x, y) exists, the low luminance pixel group l (x, y) in the fluctuation region c (x, y) is extracted (step S03).

Next, the center of gravity in the fluctuation region c (x, y) and the center of gravity of the low luminance pixel group l (x, y) are obtained, and the distance d between them is calculated (step S04).
When the distance d is calculated, based on the ratio of the distance d to r which is 1/2 of the long side distance of the circumscribed rectangle obtained by ellipse approximation of the variation region c (x, y), the variation region c (x, The bias degree R of y) is calculated (step S05).

  Thereafter, it is determined whether or not the fluctuation area c (x, y) is darker than the corresponding area of the background image. When the fluctuation area c (x, y) is brighter than the corresponding area of the background image, the fluctuation area c (x, y) is likely to exist in the monitoring space as a moving object having a higher reflectance than the background. For this reason, it is preferable to always determine whether or not this fluctuation area is an intruder without determining whether or not this fluctuation area is caused by a disturbance. This determination is performed by calculating an average value ea of the difference luminance value between the luminance value of each pixel in the fluctuation region in the captured image and the luminance value of the corresponding pixel in the background image B (x, y), and ea is negative. The determination is made based on whether or not the value is reached (step S06). When ea <0, it is determined whether or not the fluctuation region c (x, y) is a region caused by a disturbance (step S07). On the other hand, when ea ≧ 0, it is determined that the fluctuation region c (x, y) is not a region caused by disturbance, and a moving body determination process is performed (step S08).

  In determining whether or not the variation region is a region caused by disturbance (step S07), the degree of bias R of the low-luminance pixel group l (x, y) in the variation region c (x, y) calculated in step S05 is determined. It is determined whether or not the fluctuation region c (x, y) is a region caused by a disturbance by checking whether or not the threshold value Th2 is greater than or equal to the predetermined threshold value Th2. If the degree of bias R is greater than or equal to the predetermined threshold Th2, it is determined that the fluctuation region is not a region due to disturbance, and then a moving body determination process is performed (step S08). On the other hand, when the degree of bias R is less than the threshold value Th2, the fluctuation region c (x, y) is determined as a region caused by disturbance. Then, a captured image of the monitoring space is acquired again.

  In the moving body determination process (step S08), the variable area c (x, y) is caused by an intruder using information such as the size, shape, and speed of movement of the variable area c (x, y). It is determined whether it is a thing. When it is determined that the fluctuation area c (x, y) is caused by an intruder, an output process is performed (step S09), and a series of processing procedures of the image sensor is completed. On the other hand, when it is determined that the fluctuation region c (x, y) is not caused by an intruder, a captured image of the monitoring space is acquired again.

  In the operation procedure of the image sensor 1 described above, the step (step S06) of determining whether or not the variation region c (x, y) is darker than the corresponding region of the background image B (x, y) is performed. After the step of determining whether or not x, y) exists (step S02), the step of extracting the low luminance pixel group l (x, y) in the fluctuation region c (x, y) (step S03). You may go before. In this case, if it is determined No in step S06, the process proceeds to step S03, and if it is determined Yes, it is determined that the region is not caused by a disturbance, and the process proceeds to step S08. Thereby, if the fluctuation area c (x, y) is brighter than the corresponding area of the background area B (x, y), the procedure for obtaining the degree of bias in the fluctuation area can be omitted.

  The image sensor according to the present invention is not limited to the above-described embodiment. For example, a camera having a high pixel number such as 1 million pixels or 2 million pixels may be used as the camera used in the imaging unit, or a camera capable of color photography may be used. Furthermore, a camera equipped with a C-MOS sensor can be used as the image sensor instead of the CCD.

  Further, the image processing unit 4 can calculate the degree of bias in the fluctuation region based on the feature amount other than the luminance. For example, the feature calculation unit 12 can calculate a texture feature quantity such as entropy based on the density co-occurrence matrix and use it for calculating the degree of bias instead of extracting a pixel group of low luminance or high luminance. In this case, the pixel group extraction unit 21 divides the variable region into a plurality of small regions, calculates texture feature amounts and the like in each small region, and whether the calculated feature amount is equal to or greater than a predetermined threshold value. No, each small region can be classified into two or more pixel groups. Also, the difference between the edge distribution based on the luminance difference between neighboring pixels in the fluctuation area and the similar edge distribution in the corresponding area of the background image, or the difference between the texture feature quantity in the fluctuation area and the texture feature quantity in the corresponding area of the background image. By using this, it is possible to classify the fluctuation region into two or more pixel groups depending on whether or not this difference is greater than or equal to a threshold value.

  Further, the deviation calculating unit 22 may use the square root of R described above or the reciprocal as an index of the degree of bias. Further, instead of elliptically approximating the fluctuation region, the upper end coordinate tp, the lower end coordinate bp, the left end coordinate lp, and the right end coordinate rp of the fluctuation region are simply obtained, and the point (lp, tp) is set to the upper left end and the point (rp, bp). The above-mentioned r may be obtained as a value based on the size of the fluctuation area, which is used for calculating the degree of bias R based on the rectangle having the right lower end as a lower limit. In addition, the total number of pixels in the variable region may be r. On the other hand, for the distance d used in the calculation of the degree of bias R, for example, the center point of the ellipse obtained by ellipse approximation of the fluctuation region c (x, y) and the low luminance pixel group l (x, y) (or high luminance) The distance from the center point of the ellipse obtained by ellipse approximation of the pixel group h (x, y)) can be used. Further, as the degree of bias R, the ratio between the distance between the two most distant points in the fluctuation region c (x, y) and the distance between the two most distant points in the low luminance pixel group l (x, y). May be used. Needless to say, when the indexes listed here are used as the degree of bias R, the threshold Th2 also needs to be optimized according to these indexes. Regardless of which index is used, the threshold Th2 can be optimized experimentally.

  Furthermore, in the above example, the degree of bias of the pixel group having different image characteristics in the fluctuation region is used to determine whether or not the fluctuation region is caused by disturbance. Instead, the degree of homogeneity in the fluctuation region is used. May be used. This degree of homogeneity and the above-mentioned degree of bias are in an integrated relationship, and when using the degree of homogeneity in the fluctuation region, contrary to the case of using the degree of bias, the higher the degree of homogeneity, the more the fluctuation region becomes more disturbing. It is more likely to be attributed.

It is a functional block diagram of an image sensor concerning an example of the present invention. It is explanatory drawing about the principle of determination whether the change area | region is based on a disturbance by this invention. It is a figure explaining the bias | inclination degree of the low-intensity pixel group in a fluctuation area. It is an operation | movement flowchart in one Embodiment of this invention. It is explanatory drawing of the human presence determination process in the monitoring space by the image sensor of a prior art example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image sensor 2 Imaging part 3 Recording part 4 Image processing part 5 Output part 11 Fluctuation area extraction part 12 Feature calculation part 13 Disturbance determination part 14 Moving body determination part 21 Pixel group extraction part 22 Deviation calculation part

Claims (6)

An image sensor that extracts a fluctuation region from a difference between an image acquired by an imaging unit and a past image, and determines whether the fluctuation region is caused by a disturbance,
A pixel group extraction unit that extracts a pixel group having a predetermined feature amount in the variable region;
A deviation calculating unit for calculating a degree of bias for determining whether the position of the pixel group is a biased position in the fluctuation region;
When it is determined that the degree of bias is small, a disturbance determination unit that determines that the fluctuation region is caused by a disturbance,
An image sensor comprising:   The deviation calculating unit calculates a degree of deviation of the position of the pixel group in the variation area based on a ratio between the center position of the variation area and the center position of the pixel group and the size of the variation area. The image sensor according to claim 1.   The image sensor according to claim 1, wherein the pixel group is a pixel group including any one of a low luminance pixel having a luminance equal to or lower than a predetermined threshold and a high luminance pixel having a luminance equal to or higher than the predetermined threshold.   The pixel group extraction unit binarizes pixels included in the variation area with a threshold value equal to or less than a luminance average value of the variation area, extracts a low luminance pixel in the variation area, and a pixel group including the low luminance pixels The image sensor according to any one of claims 1 to 3, wherein a pixel group is extracted as the pixel group.   The disturbance determination unit determines that the fluctuation area is caused by disturbance when the fluctuation area is lower in brightness than the corresponding area in the past image and the degree of bias is small. If the region is higher in luminance than the corresponding region in the past image, or if the variation region is lower in luminance than the corresponding region in the past image and the degree of bias is large, the variation The image sensor according to claim 3, wherein the region is determined not to be caused by a disturbance.   When the disturbance determination unit determines that the fluctuation region is not caused by disturbance, a moving object determination unit that determines whether the fluctuation region is caused by a moving object based on another determination criterion. The image sensor according to claim 1, further comprising: