JP4491360B2 - Image signal processing device - Google Patents

Description

  The present invention relates to an image signal processing apparatus suitable for detecting that a water droplet has adhered to a lens portion in an imaging apparatus.

  In recent years, security systems that detect intruders or the like that have entered a monitoring space have been widely used. For example, security systems using image sensors have become widespread. A system using an image sensor includes a camera unit that captures a monitoring space to be monitored, an image acquisition unit that receives an image captured by the camera unit, and a storage unit that stores and holds image data used for processing. A processing unit that processes the image data and detects a moving object existing in the monitoring space. The processing unit extracts a variation area in the image based on a difference image composed of a difference between the input image and a background image captured in the past, and detects an attribute value of the area of the input image corresponding to the variation area When the object has a characteristic characteristic, processing such as issuing an alarm as the detection object exists in the monitoring space is performed. For example, when the detection target is “intruder”, the attribute value indicating “humanity” such as the size, aspect ratio, and luminance information of the input image data corresponding to the variable region satisfies a predetermined condition. When the condition is satisfied, processing such as issuing an alarm that an intruder exists is performed.

  Moreover, the system which monitors an intruder etc. using an image sensor is further provided with the illumination part which irradiates light to monitoring space for nighttime monitoring. Then, an image obtained by capturing the monitoring space in a state where the monitoring space is illuminated by the illumination unit is acquired, and the acquired image data is processed to detect a moving object and determine whether the moving object is a detection target. Thus, it is determined whether or not there is a detection object (for example, an intruder) in the monitoring space at night.

  By the way, in order to detect an intruder or the like who has entered an outdoor monitoring space, the above system may be used by installing a camera unit outdoors. In this case, since the camera unit is installed outdoors in a system for monitoring an outdoor monitoring space, a situation may occur in which water drops adhere to the lens portion of the camera unit during rainy weather. Here, when a water droplet adheres to a lens part or the like, an image region in which the water droplet is imaged is reflected in the image data in a blurred state, so that it is difficult to detect a detection target in the blurred water droplet image region. For example, when there is an intruder in the water drop image area in the image, the intruder cannot be detected clearly because the intruder's image is hidden in the water drop image area. May occur. Such misreporting is particularly problematic in security systems that detect intruders.

  Therefore, when water drops adhere to the lens part of the camera unit, in order to prevent the occurrence of the above-mentioned problems, measures such as detecting the water drop adhering and removing it from the lens part are taken immediately. There is a need.

  Conventionally, when detecting the adhesion of water droplets, for example, as a technique for detecting raindrops adhering to the windshield of a vehicle, the illuminance in this region is calculated from image data obtained by imaging the set raindrop detection region Then, based on the obtained illuminance, there is a technique for calculating a threshold of change in illuminance for identifying the outline of the raindrop and detecting the outline of the raindrop using this threshold.

JP 05-284501 A Japanese Patent Laid-Open No. 10-148681

  In a system that uses an image sensor to detect intruders, etc., if the water droplets adhere to the lens part of the camera unit, the image area that captured the water droplets is reflected in the image data in a blurred state. In some cases, it is difficult to detect the detection target in the image area.

  Moreover, in the system using the image sensor, water droplets adhering to the lens portion of the camera unit have been a cause of false alarms.

  In a system using an image sensor, the point of detection is whether or not an attribute value that represents “likeness of a detection target” of a fluctuation region extracted from a difference image between an input image and a background image satisfies a predetermined condition. That is, even when the fluctuation region is not caused by the detection object, a detection error (false report) is issued when the condition of “detection object likelihood” is satisfied. Such false alarms are particularly problematic in security systems that detect intruders.

  Here, in such a system, when the monitoring space is imaged in a state where the monitoring space is illuminated by the illumination unit in a dark time zone such as at night, and an image of the monitoring space is acquired, a water droplet image in the captured image thus captured The area is characterized in that almost the entire area of the water drop image area becomes bright due to the influence of illumination light reflected by the water drop. This brightly reflected water drop image area is extracted as a variable area from the difference image between the input image and the background image, and if the area or shape of this variable area satisfies the condition of “humanity”, there is an intruder There was a misinformation.

  An object of the present invention is to provide an image signal processing device capable of increasing the detection accuracy of water droplets adhering to an imaging unit even when imaging is performed by applying illumination light at night or the like.

  The image signal processing apparatus of the present invention determines whether or not an image region due to water droplets exists in an input image captured by the imaging unit while illuminating the monitoring space, and attaches water droplets to the imaging unit. An image signal processing apparatus having a water droplet detection unit for detecting, wherein the water droplet detection unit is configured to detect a background image captured by illuminating a monitoring space including the image capturing unit, and to capture the image capturing unit after capturing the background image. A difference image generation process that takes a difference from an input image captured by illuminating a monitoring space including the image and generates a positive difference image in which the luminance of the input image is higher than that of the background image; and the edge strength in the input image is An input weak edge image generation process that generates an input weak edge image that satisfies a condition that the input image is less than or equal to a first threshold value, and compares the edge strength in the input image with the edge strength in the background image, and the input image Edge reduced image generation processing for generating an edge reduced image that satisfies the condition that edge strength is reduced compared to that in the background image, an image obtained by logical sum of the input weak edge image and the edge reduced image, and the difference image And a water droplet candidate image generation process for generating a water droplet candidate image with a high possibility of the presence of a water droplet, and a water droplet determination process for determining the presence or absence of a water droplet based on the water droplet candidate image and detecting the water droplet , Is performed.

  Here, in the image signal processing device having the above-described configuration, the water droplet detection unit has a condition that an edge intensity is not less than a second threshold in the input image and an edge intensity is not more than a third threshold in the background image. The difference strong edge image generation process for generating a difference strong edge image that satisfies the above is further performed, and in the water droplet candidate image generation process, after performing the logical product, further performing a negative logical product process with the difference strong edge image, It is preferable to generate a water droplet candidate image.

  Further, in the image signal processing device having the above configuration, the water droplet detection unit generates a large edge increased image that satisfies a condition that an edge intensity in the input image is greater than or equal to a fourth threshold value than in the background image. It is preferable to further perform an augmented image generation process, and in the water droplet candidate image generation process, after performing the logical product, further perform a negative logical product process with the edge greatly increased image to generate a water droplet candidate image. .

  Furthermore, in the image signal processing device having the above-described configuration, the water droplet detection unit generates a water droplet candidate vicinity image that generates a water droplet candidate vicinity image indicating a peripheral region of a region composed of pixels or pixel groups that satisfy each feature of the water droplet candidate image. Compare the generation strength with the edge strength in the input image and the edge strength in the background image, and generate an edge-enhanced image that satisfies the condition that the edge strength in the input image is higher than that in the background image. Edge-enhanced image generation processing, neighborhood strong edge image generation processing for generating a neighborhood strong edge image obtained by logical product of the water droplet candidate neighborhood image and the difference strong edge image generated by the difference strong edge image generation processing And near edge augmented image generation processing for generating a near edge augmented image obtained by a logical product of the water droplet candidate nearby image and the edge augmented image In the water droplet determination process, the water droplet candidate image generated in the water droplet candidate image generation process has a ratio of the near strong edge image to the water droplet candidate vicinity image equal to or greater than a first predetermined area ratio, and If there is a portion that satisfies the condition that the ratio of the near edge augmented image to the near water drop candidate image is equal to or greater than a second predetermined area ratio, it is preferable to determine that there is a water drop.

  Furthermore, in the image signal processing device configured as described above, the water droplet detection unit may perform exclusive OR of a water droplet candidate image in the latest input image and a water droplet candidate image in an image captured before imaging the latest input image. A process of calculating an area of a pixel or a group of pixels determined as an inter-frame difference area is performed. In the water droplet determination process, the water droplet candidate image is caused by a staying water droplet based on the inter-frame difference area. It is preferable to determine whether or not it is an image.

  Further, in the image signal processing device having the above configuration, the water droplet detection unit calculates a position of the center of gravity of each of the latest water droplet candidate image in the latest input image and the past water droplet candidate image captured before the input image is captured. In the water droplet determination process, it is preferable to further determine whether or not the water droplet candidate image is an image caused by a flowing water droplet based on the difference in the center of gravity between the latest water droplet candidate image and the past water droplet candidate image. It is.

  ADVANTAGE OF THE INVENTION According to this invention, the precision of the detection process of a water droplet in the case of imaging a monitoring space by illuminating illumination light outdoors at night etc. improves. Therefore, even when monitoring in the dark outdoors such as at night, it is possible to quickly take measures such as detecting the adhesion of water droplets and removing them from the lens portion, etc., and falsely report that the water droplet image area is the detection target. It can suppress emitting.

  As shown in FIG. 1, the image sensing device 100 according to the embodiment of the present invention includes an imaging unit 10, an illumination unit 20, an illumination control unit 30, an intruder detection unit 40, a water droplet detection unit 50, and a determination unit 60. The image processing unit 70 and the output unit 80 can be configured by a computer that can execute a program for executing processing described below. In the present embodiment, the detection target is “intruder”, but is not limited to this. Therefore, although the component 40 is named the intruder detection unit, the name can be changed depending on the detection target, and the image sensing device 100 performs a process of detecting a desired detection target. It ’s fine.

  The imaging unit 10 is a camera that includes a photoelectric conversion element such as a CCD element or a C-MOS element that can acquire the monitoring space as two-dimensional image data, and a process that performs processing such as analog / digital conversion on the captured image. And an interface (not shown) for outputting image data. The imaging unit 10 captures an image of the monitoring space at a predetermined time period Ti (for example, 0.1 second) and outputs it as image data.

  The illumination unit 20 has a configuration in which, for example, a plurality of light emitting diode (LED) elements, lamps, and the like are arranged in a matrix, for example. The illumination unit 20 can acquire an image captured by the imaging unit 10 in a state where the monitoring space is irradiated at night or the like. In addition, the imaging part 10 and the illumination part 20 are accommodated in the housing | casing provided with the waterproof performance for outdoors. In the present invention, water droplets attached to the front glass are detected.

  The illumination control unit 30 calculates the average luminance of the entire input image by the imaging unit 10, and if it is equal to or less than a predetermined value, irradiates the monitoring space on the assumption that it is difficult to monitor only with the brightness of natural light as in nighttime monitoring. The lighting unit 20 is controlled. On the other hand, if the average luminance of the entire input image is equal to or higher than a predetermined value, the illumination unit 20 is controlled so that the illumination unit 20 does not perform irradiation, assuming that the brightness by natural light can be sufficiently monitored as in daytime monitoring. .

  As a condition for determining the presence or absence of illumination, for example, in an AGC (Auto Gain Control) amplifier that amplifies an input image signal built in the imaging unit 10, if the input image signal exceeds the range in which the gain can be adjusted by the AGC amplifier, it is nighttime. It is also possible to set such a condition that it is regarded as being difficult (monitoring is difficult only by the brightness due to natural light), or if the level of the image signal is equal to or less than a predetermined value, it is regarded as nighttime (it is difficult to monitor only due to the brightness due to natural light). In the present embodiment, the illuminance of the monitoring space is determined using the input image data from the imaging unit 10, but an illuminance sensor is provided separately, and whether or not illumination is applied based on the illuminance acquired by the illuminance sensor. You may make it judge.

  The intruder detection unit 40 obtains input image data from the imaging unit 10 at a predetermined time period Ti, as in the conventional case, and includes a difference between the latest input image data and background image data captured in the past. A fluctuation area in the image is extracted based on the difference image data, and the attribute value of the area in the input image data corresponding to the fluctuation area is specific to the detection target (in this embodiment, “intruder”, that is, “person”). The presence of an “intruder” in the monitoring space is detected assuming that the detection target exists in the monitoring space. For example, an intruder is considered to be present when an attribute value representing “humanity” such as the size, aspect ratio, and luminance information of the input image data corresponding to the variable region satisfies a predetermined condition. ”Is detected. In addition, a signal indicating the detection result thus obtained is output to the determination unit 60.

  Similar to the intruder detection unit 40, the water droplet detection unit 50 acquires input image data at a predetermined time period Ti. As shown in FIG. 2, the water drop detection unit 50 includes a background image update unit 51, a background difference calculation unit 52, an edge extraction unit 53, a water drop candidate image generation unit 54, an interframe difference calculation unit 55, and a water drop candidate neighborhood image processing unit. 56, a water drop determination unit 57 and a storage unit 58. The storage unit 58 can be composed of a semiconductor memory, a hard disk device, an optical disk device, or the like. The storage unit 58 stores and holds background image data necessary for calculating a background difference image and past water droplet candidate image data necessary for calculating movement information of a water droplet candidate image described later. The past water droplet candidate image data is image data of a water droplet candidate image extracted as described later in image data captured a predetermined time before the imaging time of the newest input image (for example, one frame before). The Processing in components other than the storage unit 58 will be described later.

  The water droplet detection unit 50 generates difference image data including a difference between the latest input image data and background image data captured in the past. The input image data, the background image data, and the difference image data are subjected to various processes described later, and based on the image data obtained by the various processes, it is determined whether or not the input image is an image caused by water droplets. . If it is determined that the image is caused by a water droplet, a signal indicating that the adhesion of the water droplet has been detected is output to the determination unit 60.

  Based on the results detected by the intruder detection unit 40 and the water droplet detection unit 50, the determination unit 60 determines whether or not a detection target exists in the monitoring space, and a water droplet adheres to the lens portion of the imaging unit 10. Whether or not it is determined. The determination result obtained here is output to the output unit 80.

  As described above, in the present embodiment, the image processing unit 70 in the image sensing device 100 includes the intruder detection unit 40, the water droplet detection unit 50, and the determination unit 60.

  The output unit 80 outputs an alarm signal or the like to the outside of the apparatus when the determination unit 60 determines that a detection target exists in the monitoring space or that water droplets are attached to the lens portion or the like of the imaging unit 10. Used for. For example, an alarm device including an alarm lamp and an alarm buzzer may be used, and a network interface for transmitting an alarm signal to a remote monitoring room is also preferable.

  Next, processing for image data performed by the image sensing device 100 according to the present embodiment will be described with reference to FIG. In the present embodiment, a process in the case where an imaging is performed by illuminating the monitoring space by the illumination unit 20 such as at night will be described.

  In step S <b> 1, the latest input image data is acquired, and the latest image data of the monitoring space captured by the imaging unit 10 is input to the intruder detection unit 40 and the water droplet detection unit 50. At this time, the illumination control unit 30 controls the illumination unit 20 based on the input image data. When it is difficult to monitor only by the brightness of natural light such as at night, the illumination unit 20 sets the monitoring space. An image captured in the irradiated state is acquired as the latest input image data.

  In step S <b> 2, the intruder detection unit 40 detects the presence of “intruder” in the monitoring space. The intruder detection unit 40 extracts a fluctuation region in the image based on difference image data that is a difference between the latest input image data and background image data captured in the past. If the attribute value of the area in the input image data corresponding to the variable area has a characteristic characteristic of “intruder”, that is, “person”, for example, the size of the area in the input image data corresponding to the variable area If the attribute value indicating “humanity” such as aspect ratio and luminance information satisfies a predetermined condition, it is assumed that the “intruder” exists in the monitoring space and the presence of the “intruder” in the monitoring space. To detect. And the intruder detection part 40 outputs the signal which shows the detection result obtained in this way to the determination part 60 (step S3).

  In step S <b> 4, the water droplet detection unit 50 detects the adhesion of water droplets to the front surface of the imaging unit 10. The water droplet adhesion detection process performed by the water droplet detection unit 50 will be described with reference to FIG. The following image signal processing can be realized by executing an image signal processing program stored in advance in the storage unit 58 or the like.

  In step S4-1, background difference data is generated based on the latest input image data and background image data. The background difference calculation unit 52 reads the background image data from the storage unit 58 and calculates background difference data by calculating the difference between the latest input image data acquired by the imaging unit 10 and the background image data. Specifically, each pixel included in the latest input image data is set as the target pixel in order, and the difference data for the target pixel is obtained by subtracting the luminance value of the pixel of the background image data corresponding to the pixel from the luminance value of the pixel. Is calculated. If this difference data is a value greater than or equal to a predetermined threshold, the pixel value of the background difference data corresponding to the target pixel is set to “1”, and if the difference data is smaller than the predetermined threshold, the background difference data corresponding to the target pixel The binarized background difference data is generated by setting the pixel value of the pixel to “0”. That is, the pixel value of the image area where the input image data and the background image data fluctuated by more than the threshold value is “1”, and the pixel value of the image area where there was no fluctuation is “0”. It should be noted that an area constituted by a continuous pixel group among pixels having the pixel value “1” included in the background difference data forms one variable area.

  Here, in the process in step S4-1, the threshold TH1 (for example, about 160 when luminance information is set in 256 gradations) is set as the minimum luminance that can be regarded as an image region (water droplet image region) caused by a water droplet. ) Is set, pixels having a luminance value equal to or higher than the threshold TH1 are extracted from the input image data, and the pixel value of the above binarization process may be set to “1” only for the extracted pixels. According to this, there is a high possibility that an image region with too low luminance is not an image region caused by water droplets, and thus such an image region can be excluded from the processing target of the water droplet adhesion detection process. The background difference data generated in this way is passed to the water droplet candidate image generation unit 54.

  In step S4-2, the edge strength in the image is calculated for each of the latest input image data and background image data, and edge processing is performed. This process utilizes the feature that when a water droplet adheres to a lens portion or the like, an image area in which the water droplet is imaged is reflected in the image data in a blurred state due to the lens effect, and the change of the edge becomes remarkable. The edge extraction unit 53 calculates the edge strength in each image for the latest input image data acquired by the imaging unit 10 and the background image data in the storage unit 58 using an existing calculation method. For example, the edge strength is calculated by calculating the edge strength in the X direction (vertical direction of the image) and the edge strength in the Y direction (horizontal direction of the image), and adding the absolute values of the obtained edge strengths.

  The edge extraction unit 53 performs the above processing on the latest input image data and background image data, and generates the following image data using the input edge image data and background edge image data obtained thereby. First, image data (input weak edge image data) that satisfies the condition that the edge strength is equal to or less than a predetermined threshold TH2 (for example, about 2 to 3 when luminance information is set in 256 gradations) in the input edge image. Generate. This is for using the feature that the edge intensity is weak inside the waterdrop image area that appears blurred for waterdrop detection. The threshold value TH2 corresponds to an example of a first threshold value in the present invention.

  Also, image data (input strong edge image data) that satisfies the condition that the edge strength in the input edge image is equal to or higher than a predetermined threshold TH3 (for example, about 20 when luminance information is set with 256 gradations) is generated. . This is because the water droplet image area in the image captured with illumination at night etc. is almost entirely bright, so the edge strength is not strong inside the water droplet image area, but the vicinity of the periphery of the water droplet image area Then, it is for utilizing the feature that edge strength is strong for water droplet detection. The threshold value TH3 corresponds to an example of a second threshold value in the present invention.

  Furthermore, the edge strength in the input edge image is equal to or higher than the predetermined threshold value TH3, and in the background edge image, the edge strength is equal to or lower than the predetermined threshold value TH4 (for example, about 20 when luminance information is set with 256 gradations). Image data (difference strong edge image data) that satisfies the following condition is generated. As a procedure for generating the differential strong edge image, first, the condition that the edge intensity in the background edge image is greater than or equal to a predetermined threshold TH5 (for example, about 20 when the luminance information is set in 256 gradations) is greater than TH4. Image data (background strong edge image data) is generated. Next, difference strong edge image data is generated by taking a difference between the input strong edge image data and the background strong edge image data. This is because there is a feature that a strong edge does not newly appear inside the water drop image area, and in the water drop image area in an image captured with illumination at night, etc., the water drop is almost bright throughout. A strong edge is likely to be formed in the peripheral part due to light refraction, that is, the peripheral part of the water drop image area has many strong edges that do not exist in the background edge image but appear in the input edge image. Is used for water droplet detection. The threshold value TH4 corresponds to an example of a third threshold value in the present invention.

  Each edge image data generated by the edge extraction unit 53 as described above is passed to the water droplet candidate image generation unit 54.

  In step S4-3, the edge intensity of the corresponding pixels of the input edge image data extracted by the edge extraction unit 53 and the background edge image data are compared, and an image indicating a change in edge intensity between both edge images is extracted. The The water droplet candidate image generation unit 54 generates difference edge image data between the input edge image data and the background edge image data, and generates the following image data based on the difference edge image data.

  First, image data (edge-reduced image data) that satisfies the condition that the edge strength of the input edge image is lower than the background edge image by a predetermined value TH6 (for example, about 2 to 3 when luminance information is set in 256 gradations) or more. Is generated from the difference edge image. This is for using the feature that the edge intensity is reduced in the water droplet detection in the water droplet image region that appears blurred compared to the image region corresponding to the water droplet image region in the background image.

  Also, image data (edge-enhanced image data) that satisfies the condition that the edge strength of the input edge image is higher than a background edge image by a predetermined value TH7 (for example, about 2 to 3 when luminance information is set in 256 gradations) or more. Is generated from the difference edge image. This is for using the feature that the edge strength increases in the vicinity of the periphery of the water drop image area that appears blurred in comparison with the image area corresponding to the vicinity of the periphery of the water drop image area in the background image for water drop detection. Since the edge strength in the water drop image area in the input image is considered to be weaker than the image area corresponding to the water drop image area in the background image, the edge augmented image is generated when generating the water drop candidate image. It is effective to exclude.

  Here, for example, when a white wall is projected as a background image and the water droplet appears black due to light, the edge strength is increased inside the water droplet image region compared to the image region corresponding to the water droplet image region in the background image. To do. However, according to the definition of the edge-enhanced image, the edge-enhanced image resulting from such a water-drop image region is excluded when generating the water-drop candidate image, which is not desirable. Therefore, in order to cope with such a case, the threshold TH8 (for example, the luminance information 256 is set to the edge strength) that the edge intensity will not increase more than a certain amount in the water drop image area compared to the area corresponding to the water drop image area in the background image. When the gradation is set, about 20) is newly set, and image data (edge greatly increased image data) that satisfies the condition that the edge strength increase amount is equal to or greater than the threshold TH8 is generated from the difference edge image. The greatly increased edge image satisfies the condition that the edge strength of the input edge image is higher than the background edge image by a predetermined threshold TH8 or more (here TH8 ≧ TH7). The edge greatly increased image is excluded when the water droplet candidate image is generated. The threshold value TH8 corresponds to an example of a fourth threshold value in the present invention.

  In step S <b> 4-4, a water droplet candidate image for determining whether or not the input image is an image that may have occurred due to the attachment of water droplets to the imaging unit 10 is generated. The water droplet candidate image generation unit 54 uses the background difference data acquired from the background difference calculation unit 52, each edge image data acquired from the edge extraction unit 53, and each image data generated by the water droplet candidate image generation unit 54 to A candidate image is generated.

  Here, the water droplet candidate image in the case where the illumination unit 20 illuminates and captures a dark monitoring space such as at night is an image having the following characteristics. The feature 1 is an image (background difference data obtained in step S4-1) having a positive background luminance difference and having a luminance of a certain level or more in the input image. Feature 2 is an image with weak edge strength (input weak edge image obtained in step S4-2) or an image with reduced edge strength (edge reduced image obtained in step S4-3). The feature 3 is not an image in which there is no strong edge in the background image but in the input image (difference strong edge image obtained in step S4-2), and an image in which the edge strength has not increased significantly (step S4-). 3). The edge greatly increased image obtained in step 3). Therefore, the water droplet candidate image generation unit 54 uses the background difference data, the input weak edge image data, the edge reduced image data, the difference strong edge image data, and the edge greatly increased image data, as the water droplet candidate image. Generate.

  Here, the features 1 to 3 are listed as features to be considered when generating the water droplet candidate image. However, an image satisfying at least the features 1 and 2 can be generated as the water droplet candidate image. . By generating an image satisfying not only the feature 1 and the feature 2 but also the feature 3, it is possible to generate a water droplet candidate image with higher accuracy.

  When the water droplet candidate image is generated as described above, the water droplet determination unit 57 determines whether or not a water droplet exists based on the water droplet candidate image. For example, when a region composed of pixels or pixel groups satisfying the above features is present in the water droplet candidate image, it is determined that there is a water droplet, and such a pixel or region is present in the water droplet candidate image. If it does not exist, it is determined that there is no water droplet. The water droplet determination unit 57 outputs a signal indicating that a water droplet has been detected to the determination unit 60 when it is determined that a water droplet is present, and a signal indicating that no water droplet is detected when it is determined that no water droplet is present. . However, the detection accuracy of water droplet detection can be further increased by performing the processing after step S4-5 described below.

  In step S4-5, in the water droplet candidate image generated by the water droplet candidate image generation unit 54, a certain range in the vicinity (peripheral portion) of the region composed of pixels or pixel groups satisfying the characteristics of the water droplet candidate image is It is specified as the peripheral region of the pixel or region. The water drop candidate vicinity image processing unit 56 generates an image indicating the peripheral area as a water drop candidate vicinity image. This is because the edge intensity tends to be weaker than the corresponding part of the background image due to the lens effect inside the water drop image area, whereas the edge intensity is higher than the background image due to refraction in the peripheral part. It is intended to capture the phenomenon of strong edges that tend to increase, and that a strong edge is likely to appear around brightly reflected water droplets. The water droplet candidate vicinity image is a pixel in a region formed by expanding a predetermined number of pixels (for example, 4 pixels) of a region composed of pixels or pixel groups satisfying each feature of the water droplet candidate image. Alternatively, a portion not included in the region is shown as the peripheral region of the pixel or region.

  In step S4-6, the water drop candidate vicinity image processing unit 56 generates an image related to a strong edge included in the water drop candidate vicinity image. First, image data (neighboring strong edge image data) obtained by the logical product of the strong edge image acquired in step S4-2 and the water droplet candidate neighboring image is generated. Here, a differential strong edge image is used as a strong edge image with respect to a water droplet candidate image captured when illuminated at night or the like. This is because it can be assumed that a bright edge is likely to appear newly in the peripheral portion when compared with the background when it appears bright. Next, image data (neighboring edge augmented image data) obtained by the logical product of the edge augmented image acquired in step S4-3 and the water droplet candidate nearby image is generated.

  The above two types of image data are used to distinguish the water droplet candidate images because they may be due to other causes that are not actually water droplets. Other causes include, for nighttime, a bright light region created by inserting a flashlight light or an automobile headlight into the monitoring space, or a moving object existing in the monitoring range.

  Specifically, in the case of the light region, the area (or the total number of pixels) where the strong edge exists in the neighborhood strong edge image is small and the area of the portion where the edge strength is increased in the neighborhood edge increased image ( Another feature is that the total number of pixels) is large. This is due to the feature of the light region that the edge that becomes stronger than the background increases in the contour portion of the light region, but there are not many strong edges because the luminance changes gently like gradation. In the case of a moving object, the area (or the total number of pixels) where the strong edge exists in the near strong edge image is large, and the area (or the total number of pixels) where the edge strength increases in the near edge increased image. Is characterized by a small size. On the contrary, this is due to the feature of the moving object that the edge strength is high because the brightness changes sharply in the contour portion, but the amount of increase of the edge that is stronger than the background is not large. On the other hand, in the case of water droplets, the area (or total number of pixels) where the strong edge exists in the near strong edge image is large, and the area (or total pixel) where the edge strength increases in the near edge increased image. Number) tends to be large. Therefore, by using the near strong edge image data and the near edge augmented image data, it is possible to exclude images caused by other causes from the water droplet candidate images.

  As described above, after removing an image caused by another cause from the water droplet candidate image, the water droplet determination unit 57 determines whether or not a water droplet exists based on the water droplet candidate image after the exclusion process. The method of determination is the same as the determination process in step S4-6. The water droplet determination unit 57 outputs a signal indicating that a water droplet has been detected to the determination unit 60 when it is determined that a water droplet is present, and a signal indicating that no water droplet is detected when it is determined that no water droplet is present. . Thereby, water droplet detection can be performed with high detection accuracy. However, the detection accuracy of further water droplet detection can be further increased by performing the processing after step S4-7 described below.

  In step S <b> 4-7, a difference process with the past water droplet candidate image data in the storage unit 58 is performed on the water droplet candidate image generated by the water droplet candidate image generation unit 54. The inter-frame difference calculation unit 55 is a predetermined time before the latest input image capturing time stored in the storage unit 58 for the water droplet candidate image generated by the water droplet candidate image generation unit 54 (for example, 1 Inter-frame difference data is generated by calculating a difference from the previous water droplet candidate image data before the frame). Then, for example, an inter-frame difference area to be described later is calculated using the generated inter-frame difference data. Thereby, the information regarding the movement of the water droplet candidate image is acquired. This is because higher detection accuracy can be expected by taking in information on the motion of the water droplet candidate image and dividing the determination conditions for water droplets by distinguishing between the case where the water droplets do not move and the case where they flow down.

  The inter-frame difference calculation unit 55 determines the area (or the total number of pixels) of a pixel or a group of pixels determined by exclusive OR of a water droplet candidate image obtained from the current input image and a past water droplet candidate image. Is calculated as an inter-frame difference area. Further, the positions of the centers of gravity of the latest water droplet candidate image data and the past water droplet candidate image data are calculated.

  In step S4-8 and subsequent steps, regarding the water droplet candidate image generated by the water droplet candidate image generation unit 54, an image caused by a water droplet staying without flowing down (staying water droplet) or an image caused by a flowing water droplet (flowing water droplet). Determine whether. The water drop determination unit 57 is generated by the background difference calculation unit 52 from the water drop candidate image generated by the water drop candidate image generation unit 54 and the movement information of the water drop candidate image calculated by the inter-frame difference calculation unit 55. It is determined whether the background difference data thus obtained is an image caused by staying water droplets or an image caused by flowing water droplets.

  In step S4-8, determination information necessary for determining whether the water droplet candidate image is an image caused by a staying water droplet or an image caused by a flowing water droplet is calculated. Here, the following determination information is calculated by each process shown in FIG.

  In step S4-8-1, the area ratio F1, which is the ratio of the area composed of pixels or pixel groups satisfying the characteristics of the water droplet candidate image in one frame of the water droplet candidate image, is expressed as area ratio F1 = (each It is calculated by: (area of pixel or region satisfying feature) / (area of one frame of water drop candidate image). The area ratio F1 is for reflecting the feature that a pixel or a region satisfying each feature of the water drop candidate image should have a size of a certain level or more in the water drop candidate image. The area used here may be the total number of pixels.

  In step S4-8-2, using the inter-frame difference area obtained in step S4-7, the difference area ratio F2 is calculated as follows: difference area ratio F2 = (inter-frame difference area) / (latest and / or past). The area of the waterdrop candidate image that appears). The area of the water droplet candidate image that appears in at least one of the latest and the past is the area of an image generated by the logical sum of the latest water droplet candidate image and the past water droplet candidate image. The difference area ratio F2 is a water droplet candidate image that indicates how many pixels or groups of pixels that satisfy each feature of the water droplet candidate image appear or disappear at a predetermined time interval (for example, one frame). Expressed as a percentage of the total. The area used here may be the total number of pixels.

  In step S4-8-3, using the water droplet candidate vicinity image obtained in step S4-5, the neighborhood image area ratio F3 is calculated as follows: neighborhood image area ratio F3 = (pixel or region satisfying each feature of the water droplet candidate image) The area of the peripheral region of the image) / (the area of one frame of the water droplet candidate vicinity image). The near image area ratio F3 is such that when the water drop image area is projected out of the image frame and the surrounding area of the pixel or area satisfying each feature of the water drop candidate image is largely missing. Since the water droplet determination cannot be performed properly, the information is used for excluding the determination target. The area used here may be the total number of pixels.

  In step S4-8-4, the vicinity strong edge image ratio F4 is set to the vicinity strong edge image using the water droplet candidate vicinity image obtained in step S4-5 and the vicinity strong edge image obtained in step S4-6. Ratio F4 = (area of a region where a strong edge exists in the peripheral region of the pixel or region satisfying each feature of the water droplet candidate image) / (area of one frame of the water droplet candidate vicinity image). The neighborhood strong edge image ratio F4 represents how many strong edges are included in the water droplet candidate neighborhood image. The area used here may be the total number of pixels.

  In step S4-8-5, using the water drop candidate neighborhood image obtained in step S4-5 and the neighborhood edge augmented image obtained in step S4-6, the neighborhood edge augmented image ratio F5 is set to the neighborhood edge augmented image. The ratio F5 = (area of the region where the edge intensity is increased in the peripheral region of the pixel or region satisfying each feature of the water droplet candidate image) / (area of one frame of the water droplet candidate vicinity image). The near edge increased image ratio F5 represents how much the portion where the edge strength is increased in the input image as compared to the background image is included in the water droplet candidate vicinity image. The area used here may be the total number of pixels.

  In step S4-8-6, the position of the center of gravity of the latest water droplet candidate image data and the past water droplet candidate image data is calculated in the inter-frame difference data generated in step S4-7, and the coordinate value in the image is calculated. Is calculated as the amount of change in the center of gravity. The coordinate axes in the image can be arbitrarily set. For example, the upper left corner of the image is set as the origin, the horizontal axis is set as the X axis (right direction is positive), and the vertical axis is set as Y axis (down direction is positive). Then, the coordinates of the center of gravity of the latest water droplet candidate image data and the past water droplet candidate image data are calculated using the set coordinate axes, and the difference between the X coordinates is calculated as (X coordinate of the latest water droplet candidate image data) − (past water droplet candidate image data X coordinate), and a difference in Y coordinate is obtained from (Y coordinate of latest water droplet candidate image data) − (Y coordinate of past water droplet candidate image data). These indicate how much the water droplet image area has moved. If the difference in the X coordinate, that is, the movement amount ΔX in the X direction is positive, it indicates that the water droplet has moved in the right direction on the screen, and the difference in the Y coordinate. That is, if the movement amount ΔY in the Y direction is positive, it indicates that the water droplet has moved downward in the screen. In this way, the movement information of the water droplet is acquired.

  As described above, in step S4-8, each piece of determination information necessary for determining whether the water droplet candidate image is an image caused by a staying water droplet or an image caused by a flowing water droplet is calculated. When each determination information is calculated in this way, the process proceeds to step S4-9 in FIG.

  In step S4-9, the water droplet determination unit 57 determines whether or not the water droplet candidate image is an image caused by a staying water droplet. Specifically, it is determined whether or not the following conditions are satisfied. Each condition is <Residence condition 1> The area ratio F1 is not less than a predetermined threshold THF1-1 (for example, 2%) (because the staying water droplet has a certain area or more), <Residence condition 2> Difference area ratio F2 is equal to or less than a predetermined threshold value THF2 (for example, 5 to 10%) (due to less moving water droplets in a staying state), <Retention Condition 3> The neighborhood image area ratio F3 is equal to or more than a threshold value THF3 (for example, 2 to 3%) (If the area of the water droplet image region is not in a state of a certain area or more, the water droplet determination cannot be performed properly), and <Residual condition 4> The neighborhood strong edge image ratio F4 is a threshold THF4-1 (for example, 10 to 10). 50%) or higher (since there are many strong edges in the surrounding area of the water droplet), <Residual condition 5> The near edge increased image ratio F5 is equal to or higher than the threshold THF5 (for example, 10 to 50%) (the peripheral portion of the water droplet). so Since the area of the image edges is increased greater), are those such as, determines that satisfy these conditions, water droplets candidate image to be determined is an image due to retention water droplets. Note that the threshold THF4-1 corresponds to an example of a first predetermined area ratio in the present invention, and the threshold THF5 corresponds to an example of a second predetermined area ratio in the present invention.

  In step S4-10, the water droplet determination unit 57 determines whether or not the water droplet candidate image is an image caused by a flowing water droplet. Specifically, it is determined whether or not the following conditions are satisfied. Each condition is <flow condition 1> the area ratio F1 is equal to or greater than a predetermined threshold THF1-2 (for example, 10%) (because the flowing water droplet has a certain area or more), and <flow condition 2> neighboring image area The ratio F3 is equal to or greater than the threshold THF3 (for example, 2 to 3%) (because the water droplet image area cannot be properly determined unless the area of the water droplet image area is a certain area or more), <flow condition 3> strong in the vicinity The edge image ratio F4 is equal to or higher than the threshold THF4-1 (for example, 10 to 50%) (because there are many strong edges around the water droplet), and <flow condition 4> the near edge increased image ratio F5 is the threshold THF5 (for example, 10). ˜50%) (because the area of the image where the edge increases in the peripheral portion of the water droplet is large), <flow condition 5> the movement amount ΔY is greater than or equal to a predetermined threshold THΔY (for example, 5%) (flowing water <Flow condition 6> The amount of movement ΔY is greater than or equal to the absolute value of the amount of movement ΔX. If the movement amount in the downward direction should be larger than the movement amount), and these conditions are satisfied, it is determined that the water droplet candidate image to be determined is an image caused by the flowing water droplets.

  As described above, the water droplet determination unit 57 determines whether the water droplet candidate image is an image caused by a staying water droplet or an image caused by a flowing water droplet in steps S4-9 to S4-11, and the obtained determination result. Based on the above, it is possible to detect a water droplet in consideration of the state of the water droplet. And the water droplet determination part 57 outputs the signal which shows the obtained detection result to the determination part 60 in FIG. 1 (step S5). As the detection result, it is possible to indicate only the presence or absence of water droplets, or it is possible to include information on whether the water droplets are staying or flowing water droplets in the detection results and indicate that there is water droplet adhesion. Thus, the water droplet adhesion detection process in the water droplet detection unit 50 is completed, and then the process proceeds to step S6 in FIG.

  In step S <b> 6, the determination unit 60 determines whether there is a detection target in the monitoring space based on the detection result signals from the intruder detection unit 40 and the water droplet detection unit 50, and drops water on the lens portion of the imaging unit 10. It is determined whether or not the material is attached. When the determination unit 60 detects the presence of an intruder or determines that the attachment of water droplets has been detected, the process proceeds to step S7, and neither the presence of an intruder nor the attachment of water droplets is detected. The process is shifted to step S8.

  In step S7, an alarm signal is output from the output unit 60. When the output unit 60 is an alarm device including an alarm lamp and an alarm buzzer, the lamp is turned on and a buzzer sound is generated. At this time, if only the presence of an intruder is detected, if only the attachment of a water drop is detected, or if both an intruder and a water drop are detected, a different type of alarm signal is output, and a different type of lamp is lit. It is good to make it generate a buzzer sound of another kind. When the output unit 60 is a network interface, an alarm signal can be transmitted to a remote monitoring room via the Internet or a dedicated line. In this case, the alarm signal may be transmitted as another type of alarm signal according to the above three cases. At this time, it is preferable to transmit the input image data to be detected and determined together with the alarm signal. In step S7, when a signal indicating that a water droplet has been detected is output from the output unit 80, based on the output signal, a blower that blows warm air on the front surface of the imaging unit 10 or a wiper that removes attached water droplets. (Not shown) may be activated.

  In step S8, background image data update processing is performed. The background image update unit 51 updates the background image data held in the storage unit 58 with the latest input image data at a predetermined background update cycle Tr (for example, 10 seconds). It is preferable to determine the background update period Tr in consideration of fluctuations in sunlight with respect to the monitoring space. As another method, the background image data may be updated with the input image data determined that “person” does not exist, and the background is obtained by moving average image data of a plurality of input image data acquired in the past. The image data may be updated.

  In the present embodiment, the water droplet detection process is performed for the entire frame of the latest input image acquired by the imaging unit 10, but it is obtained by calculating the difference between the latest input image data and the background image data. The same water droplet detection process may be performed for each fluctuation region in the background difference data. In addition, the water droplet detection process for the entire frame of the latest input image and the water drop detection process for each fluctuation region in the background difference data may be performed in parallel.

  As described above, according to the present embodiment, it is possible to accurately detect a water droplet image area in a captured image even when the surveillance space is imaged by illuminating illumination light outdoors at night or the like. Thereby, generation | occurrence | production of the misreport and a misreport like the past can be suppressed.

It is a figure which shows the structure of the sensing apparatus in embodiment of this invention. It is a figure which shows the structure of the water droplet detection part in embodiment of this invention. It is a flowchart of the sensing process in embodiment of this invention. It is a flowchart of the water droplet adhesion detection process in the embodiment of the present invention. It is a flowchart of the water droplet determination information calculation process in embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Imaging part, 20 Illumination part, 30 Illumination control part, 40 Intruder detection part, 50 Water drop detection part, 51 Background image update part, 52 Background difference calculation part, 53 Edge extraction part, 54 Water drop candidate image generation part, 55 frames Difference calculation unit, 56 water droplet candidate vicinity image processing unit, 57 water droplet determination unit, 58 storage unit, 60 determination unit, 70 image processing unit, 80 output unit, 100 image sensing device.

Claims (6)

An image signal having a water droplet detection unit that determines whether or not an image region due to water droplets exists in an input image captured by the imaging unit while illuminating the monitoring space, and detects adhesion of the water droplets to the imaging unit A processing device comprising:
The water droplet detector is
The difference between the background image captured by illuminating the monitoring space including the imaging unit and the input image captured by illuminating the monitoring space including the imaging unit after capturing the background image is obtained as a background image. Difference image generation processing for generating a positive difference image with higher brightness of the input image,
An input weak edge image generation process for generating an input weak edge image that satisfies a condition that an edge strength in the input image is equal to or less than a first threshold;
Edge reduced image generation that compares the edge strength in the input image with the edge strength in the background image, and generates an edge reduced image that satisfies the condition that the edge strength is reduced in the input image than in the background image Processing,
A water droplet candidate image generation process for generating a water droplet candidate image with a high possibility of water droplets by logical product of an image obtained by logical sum of the input weak edge image and the edge reduced image and the difference image;
Water droplet determination processing for determining the presence or absence of water droplets based on the water droplet candidate image and detecting water droplets;
An image signal processing device characterized in that: The image signal processing apparatus according to claim 1,
The water droplet detector is
A differential strong edge image generation process is further performed to generate a differential strong edge image that satisfies the condition that the edge strength is not less than the second threshold in the input image and the edge strength is not greater than the third threshold in the background image. ,
In the water droplet candidate image generation process,
After performing the logical product, further performs a negative logical product process with the difference strong edge image to generate a water droplet candidate image,
An image signal processing apparatus. The image signal processing apparatus according to claim 1 or 2,
The water droplet detector is
Further performing an edge greatly increased image generation process for generating an edge greatly increased image that satisfies the condition that the edge intensity in the input image is equal to or higher than a fourth threshold than in the background image,
In the water droplet candidate image generation process,
After performing the logical product, further performs a negative logical product process with the edge greatly increased image, to generate a water droplet candidate image,
An image signal processing apparatus. In the image signal processing device according to any one of claims 1 to 3,
The water droplet detector is
Water droplet candidate vicinity image generation processing for generating a water droplet candidate vicinity image indicating a peripheral region of an area composed of pixels or pixel groups satisfying each feature of the water droplet candidate image;
Edge-enhanced image generation that compares the edge strength in the input image with the edge strength in the background image and generates an edge-enhanced image that satisfies the condition that the edge strength in the input image is higher than that in the background image Processing,
A neighborhood strong edge image generation process that generates a neighborhood strong edge image obtained by a logical product of the water droplet candidate neighborhood image and the difference strong edge image generated by the difference strong edge image generation process;
Near edge increased image generation processing for generating a nearby edge increased image obtained by a logical product of the water droplet candidate nearby image and the edge increased image;
And
In the water droplet determination process,
In the water droplet candidate image generated by the water droplet candidate image generation process, the ratio of the near strong edge image to the water droplet candidate vicinity image is equal to or higher than a first predetermined area ratio, and the vicinity edge increase relative to the water droplet candidate vicinity image is increased. If there is a part that satisfies the condition that the image ratio is equal to or greater than the second predetermined area ratio, it is determined that there is a water droplet.
An image signal processing apparatus. In the image signal processing device according to any one of claims 1 to 4,
The water droplet detector is
The area of the area composed of pixels or pixel groups determined by the exclusive OR of the water drop candidate image in the latest input image and the water drop candidate image in the image captured before the latest input image is captured. Process to calculate as
In the water droplet determination process,
Further, based on the inter-frame difference area, it is determined whether or not the water droplet candidate image is an image caused by a staying water droplet,
An image signal processing apparatus. In the image signal processing device according to any one of claims 1 to 5,
The water droplet detector is
Perform the process of calculating the position of the center of gravity of the latest water droplet candidate image in the latest input image and the past water droplet candidate image captured before imaging the input image,
In the water droplet determination process,
Further, based on the difference in the center of gravity position between the latest water droplet candidate image and the past water droplet candidate image, it is determined whether or not the water droplet candidate image is an image caused by flowing water droplets.
An image signal processing apparatus.

Images