JP2009010594A - Image sensor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image sensor for detecting light imprinted into a monitor image. <P>SOLUTION: The image sensor includes a light candidate pixel extracting means 10 for extracting pixels that are likely as being the light caught in monitor images, based on the monitor images captured at a predetermined frequency. The light candidate pixel extracting means 10 includes a determining means for determining whether brightness values of corresponding pixels between a plurality of monitor images captured at the predetermined frequency increase or decrease; a cumulative change quantity calculating means for calculating a cumulative quantity of change for each pixel, based on a temporal continuity of determination of an increase determined by the determining means and a temporal continuity of determination of a decrease judged by the determining means; and the light candidate pixel extracting means 10 extracts the pixels whose cumulative change quantity is a predetermined threshold or higher as the light candidate pixels. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image sensor that detects a moving object that has entered a monitoring area, and in particular, recognizes the light as a moving object by using a process for recognizing strong light reflected in a monitoring image obtained by photographing the monitoring area. The present invention relates to an image sensor that can prevent this.

  In recent years, in security systems, an image sensor that detects an intruder by performing image processing on an image captured by an imaging device such as a camera has been provided. In such a system, a reference background image in which an intruder is not captured is stored, and a difference image is generated by background difference processing between a monitoring image captured at any time and the background image. Based on the image feature amount obtained from the change area where the difference signal is a predetermined value or more in the difference image, whether the change area is due to a moving object that has entered the monitoring area or other than moving objects such as light and shadow To determine whether it is due to the cause of

Examples of the image feature amount obtained from the change area include the similarity of the texture of the object existing in the monitoring area. When the change area is due to disturbance light or shadow, the texture is similar between the monitoring image and the background image corresponding to the change area.
On the other hand, the texture of the object existing in the monitoring space is not similar in the change area generated by the moving object shielding the object originally existing in the monitoring space. Therefore, a moving object and a disturbance can be distinguished by using a change in the similarity of the texture of the object.

  For example, in the image sensor disclosed in Patent Literature 1, a change area is extracted by background difference processing, and the degree of similarity between the texture of the area corresponding to the change area in the background image and the texture of the area corresponding to the change area in the monitoring image If is high, the attribute value indicating the light or shadowiness is calculated to be high. A change region having a high light attribute value or shadow attribute value is determined to be due to disturbance.

JP 2001-243475 A

  However, with regard to a change area that occurs when a strong directional light such as a headlight or a searchlight of a car is inserted into the monitoring area at night, the texture of the monitoring image and the background image area corresponding to the changing area The similarity may be low. For example, due to the strong light of the headlight, some luminance values in the surveillance image will be saturated beyond the dynamic range of the image, and the change in light and shade of the part irradiated with light will disappear, or the light from the headlight Shadows such as unevenness on the ground may become dark, and textures corresponding to the unevenness may be emphasized. In such a case, the texture similarity between the monitoring image corresponding to the change area generated by the light of the headlight and the background image area is low. For this reason, the light attribute value indicating the lightness is lowered, and the erroneous detection that the change area is not determined to be a light object but is determined to be a moving object occurs.

Accordingly, an object of the present invention is to provide a moving object detection device capable of preventing erroneous detection due to light even when intense light is irradiated in a monitoring region.
Another object of the present invention is to provide an image sensor for detecting light reflected in a monitoring image.

  As one form of the present invention for solving such a problem, a light candidate for extracting, as a light candidate pixel, a pixel having a high possibility of light reflected in the monitoring image based on the monitoring image captured in a predetermined cycle. An image sensor having pixel extraction means is provided. In such an image sensor, the light candidate pixel extraction unit includes: a determination unit that determines whether or not the luminance value of the corresponding pixel between a plurality of monitoring images captured in a predetermined cycle is increased or decreased; A cumulative change frequency calculating means for calculating the cumulative change frequency for each pixel based on the temporal continuity of the determination of increase or the temporal continuity of the determination of decrease, and the cumulative change frequency is a predetermined threshold value The above pixels are extracted as light candidate pixels.

  In the image sensor according to the present invention, the determination unit increases the luminance value of the target pixel when the luminance value of at least half of the pixels in the vicinity of the target pixel for which the cumulative change frequency is to be calculated is increased or decreased. Or it is preferable to determine that it is decreasing.

  Furthermore, in the image sensor according to the present invention, the cumulative change frequency calculating means adds a first predetermined value when the determining means determines increase, and subtracts a second predetermined value that is greater than the first predetermined value when decreasing. Means for counting the positive change frequency; and means for counting the negative change frequency for adding a first predetermined value when the determination means determines decrease, and subtracting a second predetermined value greater than the first predetermined value when increasing. It is preferable that the total value of the positive change frequency and the negative change frequency be the cumulative change frequency.

  Further, in the image sensor according to the present invention, the cumulative change frequency calculation means sets a lower limit value for the positive change frequency and the negative change frequency, and when the corresponding luminance value between the plurality of monitoring images does not change, the positive change frequency It is preferable to subtract the third predetermined value from the negative change frequency.

  In addition, an image sensor according to the present invention includes a storage unit that stores an image in which a moving object is not captured as a background image, a difference area extraction unit that performs difference processing between the monitoring image and the background image, and extracts a difference area, It is preferable to further include moving object determination means for reducing the possibility that the difference area is due to a moving object when all or part of the candidate pixels are included in the difference area.

ADVANTAGE OF THE INVENTION According to this invention, even when strong light was irradiated in the monitoring area | region, it became possible to provide the moving object detection apparatus which can prevent the misdetection by the light.
Further, according to the present invention, it is possible to provide an image sensor that detects light reflected in a monitoring image.

Hereinafter, an embodiment in which a moving object detection device according to the present invention is applied to an intruder detection device will be described with reference to the drawings.
An intruder detection apparatus to which the present invention is applied is an apparatus that detects an intruder based on a difference area between a monitoring image obtained by photographing a monitoring area and a background image. In particular, this intruder detection device erroneously detects the irradiated light as an intruder by examining the possibility that a strong light having a certain degree of directivity, such as a headlight, was applied to the obtained difference area. Is to prevent.

  FIG. 1 is a diagram showing an overall system configuration of an intruder detection apparatus 1 to which the present invention is applied. As illustrated in FIG. 1, the intruder detection device 1 includes an imaging unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and an alarm unit 6. Hereinafter, each part of the intruder detection device 1 will be described in detail.

  The imaging unit 2 captures a monitoring area where an intruder is detected and acquires a monitoring image. For this purpose, the imaging unit 2 includes a two-dimensional detector composed of a photoelectric converter such as a CCD or C-MOS sensor, and an imaging optical system that forms an image of a monitoring region on the two-dimensional detector. Composed. The imaging unit 2 performs imaging at regular time intervals (for example, 1/5 second). Here, in the monitoring image I (x, y) (where x and y represent the horizontal and vertical coordinates of the pixels in the image data, respectively), each pixel has a luminance value of 0 to 255, for example. Expressed as digital image data. The monitoring image acquired by the imaging unit 2 is sent to the image processing unit 5 through the communication unit 3.

  The communication unit 3 is an input / output interface that transmits and receives various signals between the image processing unit 5 and the imaging unit 2 or an external device. Various communication such as USB, SCSI, RS232C, Ethernet (registered trademark), and the like. It is composed of an interface circuit and driver software for driving them. Then, the communication unit 3 sends the monitoring image acquired by the imaging unit 2 to the image processing unit 5. The communication unit 3 also outputs an abnormality occurrence signal indicating that an intruder has been detected by the image processing unit 5 to a security device or a remote monitoring center (not shown).

  The storage unit 4 includes a semiconductor memory such as a read only memory (ROM) and a random access memory (RAM), a magnetic disk (HDD), an optical disk drive such as a CD-ROM and a DVD-RAM, and a recording medium thereof. The storage unit 4 stores software for controlling the operation of the image processing unit 5, information used in various processes of the image processing unit 5, for example, past images to be compared with the monitoring image, various threshold values, and the like. . The storage unit 4 provides necessary information in response to a request from the image processing unit 5.

  In addition, the storage unit 4 stores a monitoring image obtained by capturing a monitoring area in a state where there is no intruder as a background image that serves as a reference for intruder detection. Specifically, when the intruder detection device 1 is installed, for example, a monitoring image that is first taken after the intruder detection device 1 is initialized is used as a background image. In addition, while the intruder detection device 1 is operating, the monitoring acquired by the imaging unit 2 at regular intervals in order to cope with the change in brightness in the monitoring region and the change in shadow position over time. Of the images, an image determined to have no background difference area or a monitoring image in which no intruder has been detected may be updated as a background image and stored in the storage unit 4. An image obtained by averaging the monitoring images acquired over several frames (for example, 5 frames) may be used as the background image.

  The image processing unit 5 includes an arithmetic device such as a CPU or a numerical arithmetic processor, and software read from the storage unit 4 so as to be executable by the arithmetic device. Then, the image processing unit 5 determines whether there is an intruder in the monitoring area based on the monitoring image I (x, y) acquired from the imaging unit 2.

Here, the image processing unit 5 refers to the luminance change that occurs over several frames, and examines the possibility that the luminance change that has occurred in the monitoring image is caused by light such as a headlight.
Therefore, with reference to FIG. 2, a difference in luminance change between when the light such as a headlight is irradiated in the monitoring area and when there is an intruder will be described. FIG. 2A shows the luminance change of the pixel of interest accompanying the movement of the irradiated light, and FIG. 2B shows the luminance change of the pixel of interest accompanying the movement of the intruder. First, as illustrated in FIG. 2A, it is assumed that the light 202 moves from the left side to the right side in the monitoring image 201 obtained by capturing the monitoring area from time t to (t + 2). Usually, a strong light having a certain degree of directivity, such as a headlight, has the brightest center, and gradually becomes darker as it moves away from the center. Therefore, when the change in luminance of the target pixel 203 over time is examined, the light 202 gradually becomes brighter as the light 202 approaches the target pixel 203 as shown in the graph 204 shown in the lower part of FIG. It becomes darker as it gets away from the pixel of interest 203.

  On the other hand, as shown in FIG. 2B, it is assumed that the intruder 212 moves from the left side to the right side in the monitoring image 211 obtained by photographing the monitoring area from time t to (t + 2). In this case, on the monitoring image 211, only the area corresponding to the intruder 212 changes in luminance due to, for example, reflection of illumination light that illuminates the monitoring area. Therefore, when the change in luminance with time in the target pixel 213 is examined, the brightness is brightened only during a period in which the intruder 212 and the target pixel 213 overlap as shown in the graph 214 shown in the lower part of FIG. Then, when the intruder 212 overlaps the target pixel 213 and when the target pixel 213 does not overlap the intruder 212, the luminance change of the target pixel 212 is steep.

Thus, the relationship between the elapsed time and the luminance change is different between the light irradiated in the monitoring area and the intruder. Therefore, by examining this relationship and using it for the determination of the presence or absence of an intruder, the possibility of erroneously recognizing the light irradiated in the monitoring area as an intruder can be reduced.
For this purpose, the image processing unit 5 performs a difference process between the background image obtained by capturing the monitoring area in a state where there is no intruder and the monitoring image to obtain a background difference area, and from the background difference area, a human attribute representing humanity Find the value. On the other hand, the image processing unit 5 extracts light candidate pixels that are considered to have become brighter by the light irradiated to the monitoring area by examining temporal changes in luminance, and the light candidate pixels are included in the background difference area. Based on the ratio, the light attribute value representing the lightness is obtained. Then, the image processing unit 5 determines whether there is an intruder in the monitoring area based on the human attribute value and the light attribute value. Details of individual processing of the image processing unit 5 will be described later.

  The alarm unit 6 includes an LED, a buzzer, and the like. When the intruder is detected by the image processing unit 5, the alarm unit 6 notifies that the intruder has been detected by turning on or blinking an LED or sounding a buzzer.

  Hereinafter, the image processing unit 5 will be described in detail. As illustrated in FIG. 1, the image processing unit 5 includes a light candidate pixel extraction unit 10, a background difference area extraction unit 20, a labeling unit 30, a tracking unit 40, a feature amount calculation unit 50, and an intruder determination unit 60.

  The light candidate pixel extraction unit 10 extracts light candidate pixels that are considered to be included in the region irradiated with light in the monitoring region. As described above, when light passes through the pixel of interest, the luminance value of the pixel of interest gradually increases with time and then gradually decreases. Therefore, the light candidate pixel extraction unit 10 stores the luminance change of each pixel over time as a positive change counter and a negative change counter. The positive change counter is a counter (cumulative change frequency) that counts up as the luminance value of the pixel of interest continuously increases. The negative change counter is a counter that counts up as the luminance value of the pixel of interest continues to darken. is there. A plus change counter and a minus change counter are provided corresponding to each pixel of the monitoring image, and points according to the luminance change at each pixel position are added and subtracted, and the count result is stored in the storage unit 4. Then, a pixel in which the added value of the plus change counter and the minus change counter is equal to or greater than a predetermined threshold is set as a light candidate pixel.

  The operation of the light candidate pixel extracting unit 10 will be described with reference to the flowchart shown in FIG. First, the light candidate pixel extraction unit 10 calculates a difference between pixels corresponding to each other between the latest monitoring image acquired from the imaging unit 2 and the monitoring image of the previous frame stored in the storage unit 4. A difference image is generated (step ST021).

  The light candidate pixel extraction unit 10 compares the luminance value of each pixel of the inter-frame difference image generated in step ST021 with a predetermined threshold Th1. Then, the light candidate pixel extraction unit 10 sets a pixel whose luminance value is Th1 or more, that is, a pixel that is brighter than the previous frame (plus change pixel) to 1, and a pixel whose luminance value is −Th1 or less, that is, A ternary value in which a pixel that has become darker than the previous frame (a minus change pixel) is -1, and a pixel whose luminance value is between -Th1 and Th1, that is, a pixel that does not change in luminance (a pixel without change) is 0. Are generated (step ST022). The predetermined threshold Th1 can be set to 2, for example.

  In addition, as described above, when light such as a headlight passes over the target pixel, such light irradiates an area having a certain size. The pixel of interest becomes brighter at the same time as it becomes brighter, and it becomes darker at the same time as the pixel of interest becomes darker.

Therefore, next, the light candidate pixel extraction unit 10 sets only the pixels included in the region where the direction of the luminance change is unified as the plus change pixel or the minus change pixel in the plus / minus change image generated in step ST022. It corrects (step ST023).
Specifically, if the number of positive change pixels is not less than a predetermined number (at least half, for example, 4 pixels or more) in the peripheral pixels (for example, the vicinity of 8) that are positive change pixels, the light candidate pixel extraction unit 10 The pixel is corrected to the unchanged pixel (pixel value 0). The correction result is reflected in the update plus / minus change image obtained by copying the plus / minus change image.

The light candidate pixel extraction unit 10 also has a negative change pixel equal to or more than a predetermined number (at least half) of the negative change pixels in the peripheral pixels (for example, 8 neighborhoods) of the target pixel among the negative change pixels, similarly to the positive change pixels. If there are no more than four pixels, for example, the pixel of the update plus / minus change image corresponding to the target pixel is corrected to a non-change pixel (pixel value 0).
When the light candidate pixel extraction unit 10 finishes the above processing for all the positive change pixels and the negative change pixels, the light candidate pixel extraction unit 10 uses the update positive / negative image reflecting the correction result as a positive / negative change image for the subsequent processing.

  Next, the light candidate pixel extraction unit 10 updates the value of the plus change counter and the value of the minus change counter for each pixel from the updated plus / minus change image. Then, by adding the values of the positive change counter and the negative change counter of each pixel and performing threshold processing, light candidate pixels are extracted, and a light candidate pixel image representing the position of the light candidate pixel is generated (step ST024).

  FIG. 4 is a flowchart showing details of the process in step ST024 described above. First, the light candidate pixel extraction unit 10 sets an arbitrary pixel on the plus-minus change image as a target pixel (step ST24-1). Next, the light candidate pixel extraction unit 10 determines whether or not the target pixel is a positive change pixel (step ST24-2). When the pixel of interest is a positive change pixel, that is, when the value of the pixel of interest is 1, the light candidate pixel extraction unit 10 increments the value of the positive change counter corresponding to the pixel of interest by two, and the value of the negative change counter Is counted down by 4 (step ST24-4).

  On the other hand, if the target pixel is not a positive change pixel in step ST24-2, the light candidate pixel extraction unit 10 determines whether the target pixel is a negative change pixel (step ST24-3). When the pixel of interest is a negative change pixel, that is, when the value of the pixel of interest is −1, the light candidate pixel extraction unit 10 counts down the value of the positive change counter corresponding to the pixel of interest by 4 and sets the value of the negative change counter to Counts up by 2 (step ST24-5). On the other hand, if the target pixel is not a minus change pixel in step ST24-3, that is, if the value of the target pixel is 0, the light candidate pixel extraction unit 10 avoids remaining light candidate pixels when there is no luminance change. Therefore, the values of the plus change counter and the minus change counter corresponding to the target pixel are both counted down by 1 (step ST24-6).

  Thereafter, the light candidate pixel extraction unit 10 causes the value of the plus change counter to be extremely reduced when the luminance change has not occurred for a long time, and is not extracted as a light candidate pixel when actual light is irradiated. Is compared with the lower limit value set in order to avoid the negative change counter value (step ST24-7). Then, when the value of the plus change counter is below the lower limit value, the light candidate pixel extraction unit 10 sets the value of the plus change counter to the lower limit value (for example, 0) (step ST24-8). Similarly, the light candidate pixel extraction unit 10 compares the value of the minus change counter with the lower limit value (step ST24-9). If the value of the minus change counter is below the lower limit value, the light candidate pixel extraction unit 10 sets the value of the minus change counter to the lower limit value (step ST24-10).

  Next, the light candidate pixel extraction unit 10 calculates the sum of the plus change counter and the minus change counter, and compares the sum with a predetermined threshold Th2 (step ST24-11). The predetermined threshold value Th2 can be set to 8, for example. If the total value of the counter is equal to or greater than the threshold Th2, the light candidate pixel extraction unit 10 sets the target pixel as a light candidate pixel (step ST24-12). On the other hand, when the total value of the counter is less than the threshold Th2 in step ST24-11, the target pixel is not a light candidate pixel (step ST24-13). Whether or not it is a light candidate pixel is represented by a light candidate pixel image that is a binary image having the same size as the monitoring image. When the target pixel is a light candidate pixel, the corresponding position of the light candidate pixel image Is set to 1; otherwise, the pixel value is set to 0.

  Thereafter, the light candidate pixel extraction unit 10 checks whether or not the processing in steps ST24-2 to ST24-13 has been performed on all the pixels (step ST24-14). If pixels that have not been processed remain, the process returns to step ST24-1, and a new pixel of interest is set from the unprocessed pixels. On the other hand, when those processes are performed for all the pixels, the light candidate pixel extraction unit 10 ends the extraction of the light candidate pixels. Then, the light candidate pixel image is stored in the storage unit 4.

The increase / decrease value of each counter and the threshold value Th2 are adjusted to appropriate values by experiment or experience. Further, in steps ST24-4 and ST24-5, when the decrease value is made larger than the increase value per frame of each counter, the change in luminance is not continuously increased or decreased. By rapidly counting down the counter value, it is possible to prevent extraction of luminance changes due to causes other than light passing through the monitoring area. Further, instead of counting down, a lower limit value (for example, 0) of the counter may be substituted.
In step ST24-6, instead of decrementing the value of each counter by 1, the lower limit value of the counter may be substituted.

Furthermore, instead of setting the positive change counter and the negative change count separately, a change counter that represents both positive and negative luminance changes as one index may be prepared. In this case, the light candidate pixel extraction unit 10 determines that the target pixel of the plus / minus change image obtained one frame before is a plus change pixel, and the corresponding pixel of the plus / minus change image obtained at the current time is also a plus change pixel. When the change counter is counted up. Similarly, the light candidate pixel extraction unit 10 determines that the target pixel of the plus / minus change image obtained one frame before is a minus change pixel, and the corresponding pixel of the plus / minus change image obtained at the current time is also a minus change pixel. Sometimes the change counter is counted up. In other cases, the light candidate pixel extraction unit 10 counts down the change counter or sets it to a lower limit value (for example, 0).
In this case, in step ST24-11, instead of the total value of the plus change counter and the minus change counter, the value of the change counter is directly compared with the threshold Th2.

  In order to facilitate understanding, FIG. 5A to FIG. 5E are used to determine the luminance change when light passes through the pixel of interest, the value of each counter, and whether or not it corresponds to the light candidate pixel. An example is shown. FIG. 5A shows a change in luminance value of the target pixel of the monitoring image. The horizontal axis represents time (number of frames), and is divided every two frames for convenience. The vertical axis represents the luminance value, and the value of one scale is 2. A graph 501 represents a change in luminance value over time. FIG. 5B shows the time change of the value of the plus change counter, and FIG. 5C shows the time change of the value of the minus change counter. FIG. 5D shows the time change of the total value of the plus change counter and the minus change counter. 5B to 5D, the horizontal axis represents time (number of frames) and corresponds to FIG. 5A. The vertical axis represents the value of each counter or its total value. Graphs 502 to 504 represent the change over time of the positive change counter, the negative change counter, and their total values, respectively. FIG. 5E shows whether the target pixel corresponds to a light candidate pixel in each frame. In FIG. 5E as well, the horizontal axis represents time (the number of frames) and corresponds to FIG. When the numerical value corresponding to each frame is “1”, it indicates that the target pixel is a light candidate pixel, and when the numerical value is “0”, it indicates that the target pixel is not a light candidate pixel.

Hereinafter, in order from the first frame, the luminance change and the change in the value of each counter are followed. In FIG. 5A, the luminance does not change until the second frame, but the luminance gradually increases in 3 to 10 frames (represented as 510 in the figure). Thereafter, there is no luminance change in the 11th to 14th frames, and the luminance gradually decreases in the 15th to 20th frames (represented as 520 in the figure). The luminance does not change after 21 frames.
As shown in FIG. 5B, the positive change counter value 502 increases by 2 during 3 to 10 frames in which the luminance of the target pixel gradually increases. Thereafter, during the 11th to 14th frames in which the luminance of the target pixel does not change, the value 502 of the plus change counter decreases by one. Then, in 15 to 20 frames when the luminance of the pixel of interest becomes low, the value 502 of the plus change counter rapidly decreases by -4 and becomes constant after reaching the lower limit value.
On the other hand, as shown in FIG. 5C, the negative change counter value 503 is constant at the lower limit value until 14 frames when the luminance of the target pixel becomes constant or high. Then, the minus counter value 503 increases by 2 during 15 to 20 frames when the luminance value decreases. Then, after 21 frames when the luminance value becomes constant, the minus counter value 503 decreases by one.
As a result, as shown in FIG. 5D, the total value 504 of both counters increases as the luminance value of the target pixel increases, and becomes equal to or greater than the threshold Th2 in the sixth frame. Then, up to the 24th frame in which at least one of the plus change counter and the minus change counter has a relatively large value, the total value 504 is equal to or greater than the threshold Th2.
Therefore, as shown in FIG. 5E, the pixel of interest is a light candidate pixel from 6 frames to 24 frames.

  In the setting example of FIG. 5, since the light candidate pixel is extracted based on the total value of the plus change counter and the minus change counter, the target pixel is determined as the light candidate pixel during a period substantially equal to the period during which light is irradiated. be able to. However, the light candidate pixel may be extracted using only the plus change counter or the minus change counter by appropriately adjusting the increase / decrease value of the counter or the threshold value Th2.

  The background difference area extraction unit 20 extracts a background difference area in which a luminance variation occurs between the monitoring image and the background image by an intruder who has entered the monitoring area. For this purpose, the background difference area extraction unit 20 calculates a difference value between corresponding pixels of the latest monitoring image and the background image acquired from the imaging unit 2, and generates a difference image. Then, the background difference area extraction unit 20 compares the luminance value of each pixel included in the generated difference image with a predetermined threshold value, and extracts the difference 2 obtained by extracting the background difference area in which the fluctuation of the luminance value is equal to or greater than the threshold value. A value image B (x, y) is generated. Here, the predetermined threshold is, for example, the luminance value when the cumulative histogram from the minimum value is a predetermined ratio (for example, 70%) with respect to the cumulative histogram of the entire differential image with respect to the luminance value of the differential image. It can be. The predetermined threshold may be a value obtained by adding a predetermined bias value to an average value of luminance values of the difference image.

  Furthermore, it is preferable that the background difference area extraction unit 20 performs the expansion / contraction process of the morphological operation or the filtering process of excluding the small area on the difference binary image B (x, y) to remove the noise.

The labeling means 30 performs a labeling process on the difference binary image B (x, y) obtained by the background difference area extraction means 20, and labels each independent background difference area, that is, each background difference area not connected to each other. Is attached. Since the labeling process can be performed using a known technique, detailed description is omitted here. Thereafter, the labeling means 30 calculates the area, that is, the number of pixels included in the area, for each independent background difference area. Then, the labeling means 30 manages a background difference area whose calculated area is equal to or larger than a preset area threshold by assigning a management number as an “object candidate area” to be processed later. On the other hand, the labeling means 30 determines that the background difference area whose area is less than the area threshold is an area caused by noise, deletes it, and does not set it as a subsequent processing target.
The labeled difference binary image B (x, y) is stored in the storage unit 4 for use in calculating the human attribute value and the light attribute value, and as a reference for tracking processing for the next acquired monitoring image. Remembered.

Tracking means 40, and the object candidate region of interest in the latest binary difference image _{B t (x, y),} 1 frame before time t-1 in the binary difference was calculated from the obtained monitored image to the image B _t- Among object candidate areas included in ₁ (x, y), object candidate areas that are considered to be the same object are related. For this purpose, the tracking unit 40 obtains the center-of-gravity position G _t (x, y) and area of the object candidate region of interest from the latest difference binary image B _t (x, y). Next, the tracking unit 40 calculates the center of gravity G _mt-1 (x, y) of each object candidate area m included in the difference binary image B _t-1 (x, y) one frame before (where m is each This is a label number representing an object candidate area, for example, m = 1, 2,. Then, the tracking unit 40 obtains a distance ΔG _m between G _t (x, y) and each G _mt−1 (x, y). Tracking means 40, one frame before the binary difference image _{B t-1 (x, y} ) of the object candidate region included in, than the maximum moving distance value of the distance .DELTA.G _m is estimated from the moving speed of the person is small, and the one closest to the object candidate area in which the area of interest, and by the same object and object candidate region of interest of the latest binary difference image B _t (x, y), their object candidate region Are given the same reference numbers.
The tracking unit 40 may associate the object candidate area for the latest monitoring image and the object candidate area for the monitoring image acquired in the past by the same object using another tracking method.

  The feature amount calculating unit 50 represents the ratio of the light candidate pixels extracted by the light candidate pixel extracting unit 10 with respect to the object candidate region monitored by the labeling unit 30 and represents the light attribute value. Calculate as Specifically, the feature amount calculation unit 50 calculates a logical product between corresponding pixels of the difference binary image obtained by the background difference region extraction unit 20 and the light candidate pixel image obtained by the light candidate pixel extraction unit 10, The total number of pixels whose logical product is 1 in the object candidate region of interest is obtained. Then, a value obtained by dividing the total by the number of pixels included in the object candidate region to be noticed is set as a light attribute value representing the proportion of light candidate pixels in the object candidate region. This value is 1 when all the pixels included in the target object candidate region are light candidate pixels, and is 0 when none of the pixels included in the target object candidate region is a light candidate pixel. The light attribute value indicates that the closer the value is to 1, the greater the degree of “lightness”. When a plurality of object candidate areas are included in the binary difference image, the feature amount calculating unit 50 calculates a light attribute value for each object candidate area.

  Further, the feature amount calculating means 50 calculates a human attribute value representing “humanity” for the object candidate region. When a plurality of object candidate areas are included in the binary difference image, the feature amount calculating unit 50 calculates a human attribute value for each object candidate area.

The feature amount calculating means 50 calculates a plurality of types of parameters representing the degree of “humanity” from different viewpoints. The feature quantity calculation means 50 calculates the human attribute value by weighting the plurality of parameters and then normalizing the values obtained by addition or integration.
Here, the feature amount calculation unit 50 calculates, for example, the similarity of edges in the object candidate region, the similarity of texture, and the like as parameters representing the degree of “humanity”. Hereinafter, an example of a specific calculation method of these parameters will be described. In the following example, for any parameter, the greater the value, the greater the degree of “humanity”.

First, a parameter representing the similarity of edges in the object candidate area will be described. The feature amount calculating unit 50 examines the distribution of edge pixels in the corresponding region of the latest monitoring image and background image for the object candidate region of interest. For this purpose, the feature amount calculation unit 50 obtains an edge image in which the edge strength is calculated using, for example, a Laplacian filter, for each pixel in the object candidate region of interest. Then, the feature amount calculating unit 50 calculates a total E _total of pixels having edge strengths (for example, 50) that are equal to or higher than a predetermined value among the edge images calculated for the latest monitoring image. In addition, the feature amount calculation unit 50 calculates the absolute value of the difference in edge strength between the edge image obtained for the latest monitoring image and the corresponding pixel of the edge image obtained for the background image for a pixel having an edge strength equal to or higher than a predetermined value. Ask for. The characteristic amount calculating unit 50, the absolute value of the difference between the edge strength is a predetermined threshold value (e.g., 30) obtains the cumulative value E _s of the pixel is greater than or equal. Then, the feature quantity calculating means 50 sets (1-E _s / E _total ) as a parameter value representing the similarity of edges. The value of this parameter approaches 1 as the edge distribution of the latest monitoring image and the edge distribution of the background image differ for the object candidate region of interest, and approaches 0 as the edge distribution is similar.

ここでαは定数である。 Next, a parameter representing the texture similarity in the object candidate area will be described. As the texture index, for example, a median or mode value of a histogram of luminance values, a luminance dispersion value, an intensity of a predetermined frequency of a Fourier power spectrum, a value of a predetermined element of a density co-occurrence matrix, or the like can be used. The feature amount calculating means 50 calculates the value of at least one of the texture indices for each of the corresponding regions of the latest monitoring image and background image for the object candidate region of interest. The feature amount calculating means 50 is composed of the index values T of the texture calculated for monitoring image, based on the index value T _B of the texture calculated for the background image, a value close to 0 as both values match Using such a function, the value of the parameter representing the texture similarity is calculated. As the above function, for example, the following functions can be used.

Here, α is a constant.

In addition to the above parameters, the feature amount calculating unit 50 may calculate parameters based on, for example, the polarity distribution of luminance change, the shape and area of the object candidate region, and the like.
When each parameter value is calculated, the feature amount calculating means 50 weights each parameter value using a weighting factor optimized based on experiments or the like. Then, the feature quantity calculating means 50 normalizes the values obtained by adding or integrating the weighted parameter values in the range of 0-1. The feature quantity calculation means 50 uses the normalized value as the human attribute value. The obtained human attribute value represents that the closer to 1, the higher the possibility of being a person. Further, the feature quantity calculating means 50 may obtain the human attribute value from the value of each parameter based on fuzzy inference.

The intruder determination unit 60 determines whether or not the object candidate region is an intruder based on the human attribute value and the light attribute value calculated by the feature amount calculation unit 50. For this purpose, the intruder determination unit 60 determines the obtained human attribute value and optical attribute for the object candidate region related by the tracking unit 40 as being due to the same object over a predetermined number of consecutive frames (for example, 5 frames). When the determination value calculated from the value is equal to or greater than a predetermined threshold, it is determined that the object candidate region is due to an intruder. Then, an abnormality occurrence signal is transmitted to the monitoring center or the like through the alarm unit 6 or the communication unit 3. On the other hand, if the number of consecutive frames whose determination value is equal to or greater than the predetermined threshold is less than the predetermined number, the intruder determination unit 60 determines that the object candidate area is not from an intruder.
The intruder determination means 60 calculates a determination value used for the determination according to the following formula, for example.
Judgment value = human attribute value / (1 + light attribute value)
The predetermined threshold value can be experimentally determined according to the application. Note that the determination value calculation method is not limited to the above. For example, (person attribute value−light attribute value) may be used as the determination value. Alternatively, the intruder determination unit 60 may use the human attribute value and the light attribute value directly as the determination value, and may evaluate each separately. In this case, for example, when a predetermined number of frames in which the human attribute value is higher than a predetermined threshold value (for example, 0.8) and the light attribute value is less than a separately determined threshold value (for example, 0.2) are consecutive. The intruder determination means 60 determines that the object candidate area is due to an intruder. Furthermore, by appropriately adjusting the threshold value, when any one of the above determination values is equal to or greater than the threshold value even in one frame, the intruder determination unit 60 determines that the object candidate area is an intruder. Good.

The operation of the intruder detection device 1 to which the present invention is applied will be described below with reference to the flowchart shown in FIG.
First, when power is turned on to the intruder detection device 1, the imaging unit 2 captures a monitoring area at regular time intervals and acquires a monitoring image. Then, the imaging unit 2 transmits the monitoring image to the image processing unit 5 via the communication unit 3 (step ST01). And the image processing part 5 memorize | stores the acquired monitoring image in the memory | storage part 4 so that it may arrange according to the order of the acquisition time. When a background image is not set, such as immediately after the intruder detection device 1 is activated, or when a certain time has elapsed since the background image was set, the image processing unit 5 uses the acquired monitoring image as the background image. Set and store in the storage unit 4.
Next, the image processing unit 5 uses the light candidate pixel extraction unit 10 to extract light candidate pixels (step ST02). Details of extraction of light candidate pixels are as described above.

  Thereafter, the image processing unit 5 performs a difference between the corresponding pixels of the latest monitoring image and the background image in the background difference area extraction unit 20, and extracts a background difference area having a relatively large luminance variation (step ST03). . Then, the image processing unit 5 uses the labeling unit 30 to label an area having a predetermined area or larger among the independent background difference areas, thereby forming an object candidate area (step ST04). Further, the tracking means 40 associates the object candidate area for the latest monitoring image, which is considered to be due to the same object, with the object candidate area for the monitoring image acquired in the past (step ST05).

Next, the image processing unit 5 calculates the human attribute value and the light attribute value for each object candidate region in the feature amount calculating unit 50 (step ST06). Then, in the intruder determination unit 50, the image processing unit 5 continues a predetermined number of frames in which the determination value calculated based on the human attribute value and the light attribute value is equal to or greater than a predetermined threshold for each object candidate region. It is determined whether or not the object candidate area is an intruder (step ST07). If the image processing unit 5 determines that none of the object candidate areas is due to an intruder, the process returns to step ST01 and continues monitoring.
On the other hand, if the image processing unit 5 determines that any one of the object candidate areas is caused by an intruder, the image processing unit 5 transmits an abnormality occurrence signal to the alarm unit 6. Notification of abnormalities such as blinking of light and oscillation of buzzer sound. Further, the image processing unit 5 transmits an abnormality occurrence signal to an external security device or a monitoring center via the communication unit 3 (step ST08). Then, the process returns to step ST01.

  As described above, when a strong light having directivity such as a headlight of a car is irradiated on the monitoring area, the brightness gradually decreases as the distance from the center of the light increases. Therefore, the luminance gradually changes as the light approaches or leaves. Therefore, the intruder detection apparatus 1 to which the present invention is applied extracts pixels whose luminance gradually changes with time as light candidate pixels. Then, based on the object candidate area extracted as the possibility of an intruder and the degree of overlap between the light candidate pixels and the like, it is determined whether or not the object candidate area is an intruder. It is possible to prevent the detected light from being mistakenly detected as an intruder.

The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. The present invention may be used to detect intrusion of an object other than a human, such as a vehicle or animal, into a monitored area. Moreover, the light candidate pixel extraction means 10 of the image processing unit 5 may be included and used as an image sensor that processes a monitoring image acquired from the imaging unit and extracts a light portion reflected in the monitoring image.
As described above, various modifications can be made within the scope of the present invention according to the embodiment to be implemented.

It is a functional block diagram of the intruder detection device to which the present invention is applied. (A) is a schematic diagram showing the luminance change of the pixel of interest accompanying the movement of the irradiated light, and (b) is a schematic diagram showing the luminance change of the pixel of interest accompanying the movement of the intruder. It is a flowchart which shows the operation | movement which extracts a light candidate pixel. It is a flowchart which shows the operation | movement which extracts a light candidate pixel. (A)-(e) is a figure which shows the relationship between the luminance change of an attention pixel, and each counter value. It is an operation | movement flowchart of the intruder detection apparatus to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Intruder detection apparatus 2 Imaging part 3 Communication part 4 Storage part 5 Image processing part 6 Alarm part 10 Light candidate pixel extraction means 20 Background difference area extraction means 30 Labeling means 40 Tracking means 50 Feature-value calculation means 60 Intruder determination means

Claims (5)

An image sensor having light candidate pixel extraction means for extracting pixels having a high possibility of light reflected in a monitoring image as light candidate pixels based on a monitoring image captured at a predetermined cycle,
The light candidate pixel extraction means includes
Determination means for determining whether or not the luminance value of a corresponding pixel between a plurality of monitoring images captured in the predetermined cycle is increased or decreased, and temporal continuity of determination of increase by the determination means Or a cumulative change frequency calculating means for calculating the cumulative change frequency for each pixel based on the temporal continuity of the determination of decrease,
An image sensor characterized in that a pixel whose cumulative change frequency is equal to or greater than a predetermined threshold is extracted as the light candidate pixel. The determination means includes
The method according to claim 1, wherein when the luminance value of at least half of the pixels in the vicinity of the target pixel for which the cumulative change frequency is to be calculated has increased or decreased, it is determined that the luminance value of the target pixel has increased or decreased. The image sensor described. The cumulative change frequency calculation means includes:
Means for adding a first predetermined value when the determination means determines increase, and counting a positive change frequency for subtracting a second predetermined value greater than the first predetermined value when decreasing;
Means for adding a first predetermined value when the determination means determines to decrease, and counting a negative change frequency for subtracting a second predetermined value greater than the first predetermined value when determined to increase;
The image sensor according to claim 1, wherein a total value of the positive change frequency and the negative change frequency is the cumulative change frequency. The cumulative change frequency calculation means includes:
Furthermore, a lower limit is set for the positive change frequency and the negative change frequency,
The image sensor according to claim 3, wherein a third predetermined value is subtracted from the positive change frequency and the negative change frequency when corresponding luminance values between the plurality of monitoring images do not change. Furthermore, a storage unit that stores an image in which a moving object is not captured as a background image;
Difference area extraction means for extracting a difference area by performing a difference process between the monitoring image and the background image;
The moving object determination unit according to any one of claims 1 to 4, further comprising a moving object determination unit that reduces a possibility that the difference area is caused by the moving object when all or a part of the light candidate pixels are included in the difference area. The image sensor according to item.

Images