JP4194208B2 - Image sensor - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image sensor, and more particularly to detection of a mask (monitoring shielding member) provided in the vicinity of a field of view of an imaging camera.
[0002]
[Prior art]
Image sensors are used for various purposes including crime prevention. The image sensor for crime prevention is installed on a ceiling or a wall, for example, and an intruder or the like is detected by imaging a monitoring area. In a conventional image sensor, a window member (cover) is provided on the front side (imaging side) of the imaging camera for concealing and protecting the camera. That is, monitoring is performed by the imaging camera through such a window member.
[0003]
[Problems to be solved by the invention]
However, if a mask such as a gum tape is affixed to the window member or the mask is hung in the vicinity of the window member, monitoring cannot be performed due to a visual field defect due to such a plan action. Even if it is not an act of planning, the same problem occurs when advertising paper or the like is inadvertently dropped from the ceiling near the window member in a store or the like.
[0004]
Conventionally, mask detection using the fact that there is little edge information in an image has been proposed. However, when a special mask including a striped pattern or the like is used, there is a problem that the determination accuracy is lowered.
[0005]
The present invention has been made in view of the above-described conventional problems, and an object thereof is to determine a mask existing in the vicinity of an imaging unit with high accuracy.
[0006]
[Means for Solving the Problems]
The present invention relates to an imaging unit, an irradiating unit that irradiates a field of view of the imaging unit with predetermined light, an irradiation image captured by the imaging unit during irradiation of the irradiation unit, and when the irradiation unit is not irradiated a difference calculating means for calculating a difference between the non-irradiation image imaged by the imaging means in, on the basis of at least the difference, and a determining means for determining a mask for blocking the field of view of the imaging means is Dressings containing.
[0007]
According to the above configuration, if there is a mask in the vicinity of the field of view of the imaging means, the difference between the image at the time of irradiation and the image at the time of non-irradiation, that is, the luminance difference becomes large, so that the presence of the mask can be determined. .
[0008]
Here, it is desirable that the predetermined light is weak light that substantially reaches only near the visual field. The reason is that if too much light is used, the original imaging target itself becomes bright and the accuracy of mask determination is lowered. For example, it is desirable to set the light amount so that it is smaller than the light amount for night illumination and reaches the field of view beyond the window member.
[0009]
In the above SL configuration, the determination means, the first determination condition brightness of irradiation image exceeds a first threshold value, and, the difference is based on the second determination condition exceeding the second threshold value, determining said mask . If the mask determination is performed on the image at the time of irradiation alone, there is a risk that the determination accuracy may decrease due to the change of the background (imaging target), the brightness of the monitoring area, etc. In addition to the first determination condition as described above, If the second determination condition based on the difference is used, it is possible to improve the determination accuracy together. That is, for example, when a mask is attached on the surface of the window member or when the mask is hung in the vicinity thereof, for example, the brightness of the image during irradiation increases due to light scattering or reflection by the mask, and at the same time, the difference also increases. . Therefore, both of these two phenomena are comprehensively considered as evaluation targets, thereby improving the determination accuracy. Note that, depending on the imaging environment, mask determination can be performed even with the difference alone.
[0010]
Desirably, in the said structure, the said determination means determines the said mask based on the magnitude | size of the range which satisfies both the said 1st determination conditions and the said 2nd determination conditions. According to this configuration, since the size of the range can be taken into account, it is possible to prevent erroneous reporting due to a simple failure.
[0011]
Desirably, in the said structure, the said determination means determines the said mask based on the temporal continuity of the period which satisfies both the said 1st determination conditions and the said 2nd determination conditions. According to this configuration, since the length of the period can be taken into account, it is possible to prevent erroneous reporting due to simple insect adhesion, window member or camera lens cleaning, and the like.
[0012]
Desirably, the said determination means compares the brightness | luminance value at the time of irradiation of the said predetermined light with a 1st threshold value for every predetermined unit in a captured image, and the said difference and a 2nd threshold value for every said predetermined unit. Number of units that simultaneously satisfy two determination conditions: a means for comparing, a first determination condition in which the luminance value at the time of irradiation of the predetermined light exceeds the first threshold, and a second determination condition in which the difference exceeds the second threshold Means for calculating the number of units, a means for comparing the number of units with a third threshold, a means for comparing the period when the number of units exceeds the third threshold with a fourth threshold, and the period exceeds the fourth threshold And a means for determining the mask based on the above.
[0013]
According to the above configuration, the captured image is divided into predetermined units, the first determination condition and the second determination condition are checked for each predetermined unit, and after considering a range and a period that satisfy both conditions, The mask can be determined. Therefore, highly accurate mask determination can be performed in consideration of various conditions. The predetermined unit corresponds to, for example, each block obtained by dividing a captured image into a plurality of blocks, and it is desirable to first obtain an average luminance (representative value) for each block and use it to make various comparison determinations. . According to this configuration, it is possible to enjoy advantages such as a reduction in calculation time, a reduction in memory capacity, and an improvement in noise performance. Of course, if each pixel constituting the image is set as a predetermined unit, more accurate determination can be performed.
[0014]
It should be noted that it is desirable to illuminate the entire area near the visual field during illumination. Therefore, an illuminator for night illumination may be used alone or in combination. However, in that case, it is desirable to control the irradiation light quantity to be a minute light quantity.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the drawings.
[0016]
FIG. 1 conceptually shows the overall configuration of a monitoring system having an image sensor according to the present invention. This monitoring system is roughly composed of a security center 8 and one or a plurality of monitoring units 10 connected thereto via a communication network 11 comprising a public telephone line or a dedicated line. The security center 8 is a facility that centrally manages each monitoring unit 10 outside.
[0017]
The monitoring unit 10 is installed, for example, for each building to be guarded or its floor. The monitoring unit 10 includes a plurality of image sensors 12 according to the present invention. Each image sensor 12 is installed in a place where security is required, specifically, installed in a ceiling, a wall, an elevator, or the like. The controller 14 receives image information, notification signals, and the like from each image sensor, and transmits them to the security center 8 via the communication network 11. The controller 14 manages each image sensor, processes signals from the fire sensor 16 and the emergency button 18 and executes various controls based on the signals. The controller 14 is connected to a telephone 20, a monitor (not shown), and other incidental equipment as necessary.
[0018]
Although the above configuration is the same in the conventional monitoring system, in the present embodiment, each image sensor 12 has various functions for ensuring the operation reliability. Describe.
[0019]
FIG. 2 shows each function of the image sensor 12 as a block diagram. The arithmetic processing unit 22 is configured by a microcomputer, for example, and a first irradiator 26 and a second irradiator 28 are connected to the arithmetic processing unit 22 via an irradiation control unit 30. Here, the irradiation control unit 30 controls the operations of the first irradiator 26 and the second irradiator 28 based on a command from the arithmetic processing unit 22. Here, in addition to the on / off control of the operation, the light amount control according to the ambient light level may be performed.
[0020]
As will be described later, the first irradiator 26 is a device that irradiates a monitoring area with relatively intense infrared light (near infrared light) in order to perform imaging at night. In the present embodiment, the second irradiator 28 irradiates weak infrared light (near infrared light) in order to detect various monitoring obstruction factors generated in association with the window member (monitoring window member) 13. Device. In addition, visible light can also be utilized for the light to irradiate. The weak infrared light passes through the window member 13 and is emitted to the imaging side. As shown in FIG. 3 later, one of the infrared light emitted to the outside by a light guide member (prism or diffusion member). Part or the whole is folded back to the center of the window member 13. That is, the central portion and the vicinity thereof are irradiated with weak infrared light. In the present embodiment, at the time of acquisition of an irradiation image, which will be described later, in addition to the second irradiator, the first irradiator 26 mainly intended for night illumination is also lit with a small amount of light. Thus, a minute amount of infrared light can be spread over a wide range of the visual field.
[0021]
The imaging unit 24 is configured by a CCD camera or the like having sensitivity to infrared light, and the imaging unit 24 is controlled by the arithmetic processing unit 22 via the imaging control unit 32. The image captured by the image capturing unit 24 is sent to the arithmetic processing unit 22 via the image capturing control unit 32.
[0022]
The image output unit 40 in the arithmetic processing unit 22 is a circuit that outputs a captured image to the controller 14 shown in FIG. The communication unit 38 is a circuit that performs various types of communication with the controller 14 illustrated in FIG.
[0023]
The memory 34 stores an irradiation-time image acquired in a state where the above-described weak infrared light is irradiated and a non-irradiation image acquired in a state where the irradiation is not performed. In FIG. 2, the storage areas are conceptually shown as an irradiation time image storage unit and a non-irradiation image storage unit.
[0024]
In the present embodiment, the image calculation unit 36 calculates the difference between the irradiation image and the non-irradiation image (difference image) and the presence of a mask based on the two pieces of image information of the difference image and the irradiation image. It has the function to judge.
[0025]
The determination result is output to the outside via the communication unit 38. A specific operation example of the arithmetic processing unit 22 will be described in detail later using a flowchart.
[0026]
The main features of this embodiment will be outlined. When the outer surface of the window member 13 and its vicinity are irradiated with infrared light, if there is a mask (shielding member) such as a gum tape on the surface, light scattering and light Reflection occurs, and the image becomes brighter when irradiated. Therefore, the presence / absence of a mask is determined by specifying such a change on the irradiation image and the difference image.
[0027]
FIG. 3 conceptually shows a structure corresponding to the reference numeral 100 shown in FIG. In the monitoring window 13, a first irradiator 26 including a plurality of light emitting elements 26 a and a second irradiator 28 including a plurality of light emitting elements are provided around the imaging unit 24. In addition, a light guide member 42 is provided around the central portion 13a on the outer surface of the window member 13, whereby infrared light from the second irradiator 28 is folded back toward the central portion 13a. That is, the weak light is refracted by the light guide member 42, and the weak infrared light is irradiated to the central portion 13a itself and the space near it. Here, the second irradiator 28 may be provided outside the window member 13.
[0028]
Further, at the time of mask determination, when acquiring the image at the time of irradiation, the first irradiator 26 as well as the second irradiator 28 is driven with a weak intensity as described above, whereby the outer surface of the window member 13 and its vicinity are driven. Irradiation is performed over the entire area.
[0029]
In FIG. 3, reference numeral 102 indicates the viewing angle of the imaging unit 24, and reference numeral 104 indicates the irradiation range of the first irradiator 26. Reference numeral 106 indicates an example of an optical path of light guided by the light guide member 42.
[0030]
FIG. 4 shows a normal state and a mask state. In the normal state shown in FIG. 4A, the monitoring area is appropriately imaged by the camera 24 through the window member 13. On the other hand, as shown in FIG. 4B, for example, when a mask is attached to the window member 13 with a tape or the like, the monitoring is hindered. In other words, intruders and abnormal situations can no longer be monitored. The same applies to the movable type imaging sensor 12 ′ as shown in FIG. Therefore, in the present embodiment, the above mask can be determined with high accuracy by the process (mask detection logic) shown in FIG.
[0031]
In FIG. 6, this process is repeatedly executed by the arithmetic processing unit 22 every 0.5 seconds, for example, but the time interval can be freely set. In S101, the outer surface of the window member 13 and the vicinity thereof are illuminated with weak light by the second illuminator 28 and the first illuminator 26, and in that state, an image during irradiation (irradiation image A) is acquired by the camera 24. . The amount of light from the first irradiator 26 is controlled by the irradiation controller 30.
[0032]
Next, in S102, in the state where no irradiation is performed, the non-irradiation image (non-light quantity image B) is acquired in the same manner as described above. Those images are temporarily stored in the memory 34 shown in FIG. In addition, although the order of S101 and S102 may be reversed, in order to avoid the influence by the change of the monitoring area itself, it is desirable to keep both close in time.
[0033]
In S103, for example, as shown in FIG. 7, the image A and the image B are each divided into, for example, 8 × 6 blocks. The number of divisions can be arbitrarily specified. In S104, the average value (average luminance) La (k) of the luminance value of each pixel belonging to each block in the image A is calculated, and the average luminance Lb (k) for each block is similarly calculated for the image B. ) Is calculated. Note that k indicates a block number.
[0034]
In S105, an average luminance difference Lc (k) is calculated for each block between the image A and the image B. In this case, only the absolute value of the difference or the plus side of the difference may be calculated.
[0035]
In S106, the number E of blocks satisfying the first determination condition that La (k) is equal to or greater than the first threshold (TH1) and satisfying the second determination condition that Lc (k) is equal to or greater than the second threshold (TH2) is calculated. Is done. In S107, it is determined whether or not the number of blocks E is greater than or equal to a third threshold value (TH3). If E is greater than or equal to TH3, a predetermined mask counter is incremented by 1 in S108, and if E is less than TH3, masking is performed. The counter is cleared to zero. The counter is reset to zero when the system is started up.
[0036]
In S110, it is determined whether or not the mask counter is equal to or greater than the fourth threshold value (T). If it is equal to or greater than T, the presence of the mask is determined in S111 and an alarm is output. On the other hand, if it is smaller than T, it is determined in S112 that the state is normal. The above process is repeatedly executed in a predetermined cycle.
[0037]
Therefore, according to the above configuration, the range and the period can be taken into account while using the two determination conditions related to the luminance, so that the mask determination can be performed with high accuracy.
[0038]
The TH1 is set in consideration of the light quantity of the two irradiators and the sensitivity of the camera. When this value is small, it is easy to determine that the mask is used when the image during illumination is bright. If this value is too small, it is likely to be erroneously reported in an originally bright image or a narrow room. For example, assuming that the pixel value can range from 0 to 255, 100 is set as TH1.
[0039]
The above TH2 is set in consideration of the light quantity of the two irradiators and the sensitivity of the camera. If this value is small, it is easy to determine that it is a mask. If this value is too large, it will be difficult to detect the masking action. For example, 60 is set.
[0040]
The above TH3 is a threshold value for determining what percentage of the screen the block determined to be a mask is to be determined as a mask. If it is too small, the alarm is issued only when there is an object near the camera, so the value is appropriately set according to the situation. For example, about 90% of the screen. In the case of 8 × 6 blocks, for example, 43 blocks are used.
[0041]
The above T is a threshold value for determining the temporal continuation of the mask state. Set appropriately so as not to notify when the camera is cleaned. For example, 10 seconds is set.
[0042]
According to the above process, as shown in the upper part of FIG.
The value of the difference image C (see 1C) as a difference between the image A (see 1A) and the image B (see 1B) is small. On the other hand, if the mask exists, a large luminance difference is generated in the difference image C (see 2C) as a difference between the image A (see 2A) and the image B (see 2B). Based on this principle, the presence or absence of a mask can be determined.
[0043]
According to the present embodiment described above, it is possible to determine the mask with high accuracy in consideration of the uniqueness of the mask among the monitoring faults, so that the reliability can be improved. Naturally, as an example, combining an intruder detection process using an image recognition process that has been conventionally adopted, an operation such as a sticking tape on the monitoring window member or a hanging curtain hanging in the vicinity thereof. In addition, in the case where a visual field defect in which advertising paper is hung in the vicinity thereof is detected, false alarms can be reduced if the intruder detection process is disabled so that the reliability of intruder detection is greatly improved.
[0044]
Further, in addition to this method, various existing determination methods of other examples may be used at the same time, whereby a highly reliable system can be constructed.
[0045]
【The invention's effect】
As described above, according to the present invention, it is possible to determine a mask as a monitoring failure with high accuracy and improve the reliability of the image sensor.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of a system including an image sensor according to the present embodiment.
FIG. 2 is a block diagram illustrating a specific configuration example of an image sensor.
FIG. 3 is a conceptual diagram illustrating a main configuration of an image sensor.
FIG. 4 is a diagram showing an example of a state in which a mask is attached.
FIG. 5 is a diagram showing another example of a state in which a mask is attached.
FIG. 6 is a flowchart showing mask detection logic.
FIG. 7 is a diagram for explaining block division;
FIG. 8 is a diagram for explaining mask detection by mask detection logic according to the embodiment.
[Explanation of symbols]
8 Security Center, 10 Monitoring Unit, 12 Image Sensor, 13 Window Member (Monitoring Window Member), 14 Controller, 22 Arithmetic Processing Unit, 24 Imaging Unit (Camera), 26 First Irradiator, 28 Second Irradiator, 30 Irradiation Control unit, 32 imaging control unit, 34 memory, 36 image calculation unit.

Claims (4)

Imaging means;
Irradiating means for irradiating predetermined light to the field of view of the imaging means;
A difference calculating means for calculating a difference between an irradiation image captured by the imaging means at the time of irradiation of the irradiation means and a non-irradiation image captured by the imaging means at the time of non-irradiation of the irradiation means;
Determination means for determining a mask that blocks the field of view of the imaging means based on at least the difference;
Only including,
The determination means determines the mask based on a first determination condition in which the luminance of the image during irradiation exceeds a first threshold and a second determination condition in which the difference exceeds a second threshold. Image sensor. The image sensor according to claim 1 .
The image sensor is characterized in that the determination unit determines the mask based on a size of a range that satisfies both the first determination condition and the second determination condition. The image sensor according to claim 1 .
The image sensor is characterized in that the determination unit determines the mask based on temporal continuity of a period that satisfies both the first determination condition and the second determination condition. Imaging means;
Irradiating means for irradiating predetermined light to the field of view of the imaging means;
A difference calculating means for calculating a difference between an irradiation image captured by the imaging means at the time of irradiation of the irradiation means and a non-irradiation image captured by the imaging means at the time of non-irradiation of the irradiation means;
Determination means for determining a mask that blocks the field of view of the imaging means based on at least the difference;
Including
The determination means includes
Means for comparing a luminance value at the time of irradiation of the predetermined light with a first threshold for each predetermined unit in the captured image;
Means for comparing the difference with a second threshold for each predetermined unit;
Means for calculating the number of units that simultaneously satisfy two determination conditions: a first determination condition in which the luminance value during irradiation of the predetermined light exceeds the first threshold; and a second determination condition in which the difference exceeds the second threshold; ,
Means for comparing the number of units with a third threshold;
Means for comparing a period in which the number of units exceeds the third threshold with a fourth threshold;
Means for determining the mask based on the period exceeding a fourth threshold;
An image sensor comprising:

Images