JP7252815B2 - Image processing device and image processing program - Google Patents

Description

The present invention relates to an image processing device and an image processing program for detecting visual field obstruction to an imaging device.

Conventionally, when a camera is used to monitor an intruder who enters a predetermined monitoring area (referred to as a predetermined area), the intruder irradiates the camera with a light such as a flashlight, obstructing the camera's field of view, and filming the monitored area. Techniques are disclosed for detecting obscured vision. For example, a brightness value is calculated for each predetermined region of an image captured by an imaging unit that captures an image of a monitoring region, and the number of regions whose calculated brightness value is equal to or greater than a predetermined brightness value is calculated, and the number of regions is used as a reference. A technique is disclosed in which it is determined that the field of view of an imaging means is obstructed when the value is greater than or equal to the value (Patent Document 1). Further, the number of edges of the current image is calculated by using the number of edges extracted from the image captured by the imaging unit and the gain value for adjusting the level of the image signal output from the imaging unit to a predetermined level. It is determined that the imaging unit is masked when the reference edge number obtained from the original image taken earlier is reduced by a predetermined threshold or more and the gain value of the current image is equal to or higher than the reference gain value of the original image. A technique has been disclosed (Patent Document 2).

JP-A-2000-222646 JP-A-2003-134504

However, when monochromatic light such as a laser pointer is irradiated onto the imaging device, it is difficult to determine that the brightness is high in the processing based on the luminance value, and there is a possibility that it is difficult to detect it as obstruction of the field of view of the imaging device. In other words, the luminance value obtained from monochromatic light (for example, (R, G, B)=(255, 0, 0)) is equivalent to that of white light such as a flashlight (for example, (R, G, B)=(255, 255, 255)), it is difficult to determine that the brightness is high, and there is a possibility that it is difficult to detect it as a visual disturbance. Further, when obtaining a high-brightness region in an image used for intruder detection, saturation of brightness due to environmental changes may also be determined as obstruction of the field of view of the imaging device. For example, in a photographed image for intruder determination, even sunlight or headlight light that does not hit the imaging device may be determined to have high brightness, and may be determined as obstructing the field of view of the imaging device.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus and an image processing program capable of detecting obstruction of a field of view to an imaging apparatus when a laser pointer is irradiated.

One aspect of the present invention is to provide a first photographed image photographed with a first exposure amount and the first photographed image photographed with a second exposure amount smaller than the first exposure amount, among photographed images sequentially photographed in a predetermined space by a photographing device. an acquisition means for acquiring a second photographed image photographed at the same time as the photographed image; and an RGB light source region composed of pixels in which only one of RGB component values of each pixel of the second photographed image is saturated. and when the RGB light source region is extracted, in the first captured image , the RGB light source region extracted from the second captured image captured at the same time as the first captured image A target image area other than the corresponding area, and the RGB light source area extracted from the second captured image captured at the same time as the first captured image in the reference image captured in the past with the first exposure amount. is compared with a reference image area other than the area corresponding to the target image area, and if the number of edges of the target image area is smaller than that of the reference image area by the first edge determination reference value or more, the field of view of the photographing device is obstructed. and visual field obstruction determination means for determining that the image processing apparatus is characterized by comprising:

Another aspect of the present invention is to provide a computer with a first photographed image photographed with a first exposure amount and a second exposure amount smaller than the first exposure amount among photographed images sequentially photographed in a predetermined space by a photographing device. Acquisition means for acquiring a second photographed image photographed at the same time as the first photographed image; extraction means for extracting an RGB light source region; and, when the RGB light source region is extracted, the RGB extracted from the second captured image captured at the same time as the first captured image in the first captured image. The target image area other than the area corresponding to the light source area and the second captured image captured at the same time as the first captured image in the reference image captured in the past with the first exposure amount A reference image area other than the area corresponding to the RGB light source area is compared, and if the number of edges of the target image area is smaller than that of the reference image area by a first edge determination reference value or more, the field of view of the photographing device An image processing program characterized by functioning as visual field obstruction determination means for determining that a is obstructed.

Here, the extraction means compares the values of the remaining two components with the values of the saturated components of the values of the RGB components of the pixels of the second captured image. It is preferable to extract an image area composed of sufficiently small pixels as the RGB light source area.

Another aspect of the present invention is to provide a first photographed image photographed with a first exposure amount and the first photographed image photographed with a second exposure amount smaller than the first exposure amount, among photographed images sequentially photographed in a predetermined space by a photographing device. an obtaining means for obtaining a second photographed image photographed at the same time as the photographed image; and a value obtained by summing any two RGB component values of each pixel of the second photographed image is equal to or greater than a saturation reference value. an extraction means for extracting an RGB light source area composed of certain pixels; and an area in the first captured image corresponding to the RGB light source area extracted from the second captured image captured at the same time as the first captured image. The extracted from the second captured image captured at the same time as the first captured image in the target image area other than the A reference image area other than the area corresponding to the RGB light source area is compared, and if the number of edges of the target image area is smaller than that of the reference image area by the edge determination reference value or more, the field of view of the photographing device is obstructed. and visual field obstruction determination means for determining that an image has been obstructed.

Another aspect of the present invention is to provide a computer with a first photographed image photographed with a first exposure amount and a second exposure amount smaller than the first exposure amount among photographed images sequentially photographed in a predetermined space by a photographing device. an acquisition means for acquiring a second captured image captured at the same time as the first captured image; an extracting means for extracting an RGB light source region composed of pixels having a value equal to or greater than the value of the RGB light source region; Extraction from the target image area other than the corresponding area and the second captured image captured at the same time as the first captured image in the reference image captured with the first exposure before the first captured image is compared with a reference image area other than the area corresponding to the RGB light source area, and if the number of edges of the target image area is smaller than that of the reference image area by an edge determination reference value or more, An image processing program characterized by functioning as visual field obstruction determination means for determining that the visual field is obstructed.

Here, the extracting means further extracts a luminance light source area made up of pixels whose luminance value is equal to or higher than a luminance reference value among the pixels of the second photographed image, and the visual field obstruction determining means determines whether the luminance light source area is When extracted, a target image region other than a region corresponding to the luminance light source region extracted from the second captured image captured at the same time as the first captured image in the first captured image, and the reference In the image , a reference image region other than the region corresponding to the luminance light source region extracted from the second captured image captured at the same time as the first captured image is compared, and the It is preferable to determine that the field of view of the photographing device is obstructed when the number of edges in the target image region has decreased by a second edge determination reference value that is larger than the first edge determination reference value.

In addition, the visual field obstruction determination means determines that, with respect to the luminance light source area, a luminance variance value of an image area surrounding the image area of the first captured image corresponding to the luminance light source area is less than a first variance determination reference value. In a certain case, it is determined that the field of view of the photographing device is obstructed, and for the RGB light source region, the variance value of the brightness of the image region surrounding the image region of the first captured image corresponding to the RGB light source region is the first It is preferable to determine that the field of view of the photographing device is obstructed when the variance is equal to or greater than a second variance criterion value, which is equal to or greater than one variance criterion value.

Further, when the size of the RGB light source area extracted for the first captured image is equal to or smaller than a predetermined area reference value, the visual field obstruction determination means determines that the number of edges of the target image area is larger than that of the reference image area. has decreased by more than a second edge criterion value which is higher than said first edge criterion value, it is preferably determined that the field of view of said imager is obstructed.

According to the present invention, it is possible to determine with high accuracy that there is obstruction of the field of view even when the imaging unit is irradiated with the laser pointer.

1 is a schematic configuration diagram of a security system according to an embodiment of the present invention; FIG. 1 is a schematic configuration diagram of an image processing apparatus according to an embodiment of the present invention; FIG. It is a figure which shows the example of the RGB component in embodiment of this invention. FIG. 4 is a diagram for explaining the distributed state of luminance in a luminance light source region and an RGB light source region according to the embodiment of the present invention; 4 is a flow chart showing the flow of visual field obstruction determination processing according to the embodiment of the present invention.

Security system 10 according to the present embodiment, as shown in FIG. It includes a security center device 18 connected to each security device 14 via and a user device 20 . Further, the security system 10 includes an image processing device 22 as one or more image processing devices for detecting abnormalities in the object 12 to be monitored based on a surveillance image obtained by photographing a monitoring area of the object 12 to be monitored, and image processing. It includes a recording device 24 for recording surveillance images captured by the device 22 . The image processing device 22 and the recording device 24 are communicably connected to the security device 14 .

When the security device 14 receives an alarm signal from the image processing device 22 connected to itself via a local LAN or the like, the alarm signal and the identification signal of the security device 14 itself, or the property 12 to be monitored or an abnormality is detected. The identification signal of the image processing device 22 is transmitted to the security center device 18 . Therefore, the security device 14 has a communication interface for communicating with the image processing device 22, a communication interface for communicating with the security center device 18 and the user device 20, and a control unit for controlling them.

The security center device 18 is composed of a so-called computer, and includes a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a notification unit composed of a buzzer, an LED, and the like. . When the security center device 18 receives an alarm signal from the security device 14 via the communication network 16, the security center device 18 notifies the property 12 to be monitored where the security device 14 that sent the alarm signal and the content of the detected abnormality are sent to the reporting unit and Notify the observer through the display device.

The user device 20 is also composed of a so-called computer, and includes a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a keyboard, mouse, etc. for remotely operating the security device 14. A user interface for inputting operation commands for When the user device 20 is operated to observe the object 12 to be monitored which has been registered in advance via the user interface, the user device 20 is currently photographing the security device 14 installed at the object 12 to be monitored which has been registered. It transmits various image request signals requesting transmission of the monitoring image inside or the monitoring image recorded in the recording device 24 to the user device 20 . Upon receiving the monitoring image from the security device 14, the user device 20 displays the requested monitoring image on the display device.

The recording device 24 includes recording media such as a magnetic disk device such as an HDD, a magnetic tape such as a DAT, and an optical recording medium such as a DVD-RAM. It consists of a device that reads and writes The recording device 24 receives the monitoring image captured by the image processing device 22 from the security device 14 and records it in association with the time of capturing.

FIG. 2 is a conceptual diagram of the configuration of the image processing device 22. As shown in FIG.

The communication unit 30 is an input/output interface for transmitting and receiving various setting signals and control signals between the image processing device 22 and the security device 14 via a communication network such as a local LAN. It consists of various communication interface circuits and driver software for driving them. Specifically, when an intruder is detected by the control unit 36 described later, the communication unit 30 outputs an intrusion alarm signal indicating that the intruder has been detected to the security device 14 .

The imaging unit 32 includes a two-dimensional detector such as a CMOS, which is composed of a photoelectric converter having sensitivity to visible light and near-infrared light, and an imaging unit that forms an image of a monitoring area on the two-dimensional detector. It consists of an optical system. The photographing unit 32 obtains a photographed image by photographing the monitoring area. The imaging unit 32 performs imaging at regular time intervals (for example, 1/5 second), but the imaging method of the imaging unit 32 is not limited to this. When an intruder exists in the monitoring area, the captured image acquired by the imaging unit 32 includes a human area corresponding to the person (intruder). The acquired captured image is stored in the storage unit 34 .

Further, in the present embodiment, the photographing unit 32 photographs the normal exposure image 40 with a normal exposure amount (first exposure amount suitable for intruder determination), and uses a low exposure amount (second exposure amount) that is smaller than the normal exposure amount. 2 exposure) to capture a low exposure image 42 . The normal exposure image 40 and the low exposure image 42 are shot at the same time. For example, the normal-exposure image 40 and the low-exposure image 42 may be alternately shot. The normal-exposure image 40 and the low-exposure image 42 may be captured alternately at regular time intervals (for example, 1/5 second). It is not necessary to take two images, and another image may be taken between the normal-exposure image 40 and the low-exposure image 42 . Note that the exposure amount of the photographing unit 32 is changed according to the photographing environment. In other words, the ratio of the exposure amounts of the normal-exposure image 40 and the low-exposure image 42 is not fixed, either.

The storage unit 34 is storage means configured by a semiconductor memory, a magnetic disk (HDD), or an optical disk drive such as a CD-ROM, DVD-RAM, and its recording medium. The storage unit 34 stores an image processing program for operating each unit of the image processing device 22 . The storage unit 34 stores a normal exposure image 40, a low exposure image 42, a reference image 44, a saturation reference value 46, a brightness reference value 48, a brightness determination reference value 50 (a first brightness determination reference value 50a, a second brightness determination reference value 50b), edge determination reference value 52 (first edge determination reference value 52a, second edge determination reference value 52b), and variance determination reference value 54 (first variance determination reference value 54a, second dispersion determination reference value 54b). Remember. That is, the storage unit 34 stores a normal exposure image storage means for storing the normal exposure image 40, a low exposure image storage means for storing the low exposure image 42, a reference image storage means for storing the reference image 44, and a saturation reference value. saturation reference value storage means, luminance reference value storage means for storing luminance reference values, luminance judgment reference value storage means for storing luminance judgment reference values, edge judgment reference value storage means for storing edge judgment reference values, variance judgment reference values It functions as a dispersion judgment reference value storage means for storing the .

A reference image 44 is an image that is not determined to be obstructing the field of view, out of the normal-exposure image 40 captured by the capturing unit 32 . Although one reference image 44 may be used, a plurality of reference images 44 may be created. Note that the exposure amount of the reference image 44 may be different from the exposure amount of the normal exposure image 40 captured in real time. Also, the reference image 44 may be created from a plurality of normal exposure images 40 .

The control unit 36 has a built-in microprocessor unit, memories such as ROM and RAM, and their peripheral circuits, and executes various signal processing of the image processing device 22 . As shown in FIG. 2, the control section 36 includes a light source area extraction means 60 and a visual field disturbance determination means 62, and functions as an image processing means as a whole. It is determined whether or not the field of view is obstructed by irradiating the imaging unit 32 with light by the control unit 36 functioning as these means. Each unit of the control unit 36 will be described below.

A light source region extracting means 60 extracts a light source region, which is an image region obtained by photographing a light source generated in the normal exposure image 40 when the photographing unit 32 is illuminated by a light source such as a laser pointer or a flashlight. A light source area extraction means 60 extracts an RGB light source area generated in the normal exposure image 40 by a laser pointer and a luminance light source area generated in the normal exposure image 40 by an LED light such as a flashlight.

First, extraction processing of the RGB light source area generated in the normal exposure image 40 by the laser pointer will be described.

As shown in FIG. 3, the light source region extracting means 60 extracts a red wavelength region component (R component), a green wavelength region component (G component), and a blue wavelength region component of each pixel of the low-exposure image 42. (B component) is extracted. Then, a pixel whose one of the RGB components is equal to or greater than the saturation reference value of 46 is extracted, and the extracted pixel is set in the RGB light source region. Only when any one of the RGB components of the pixel is completely saturated, the pixel may be regarded as having a saturation reference value of 46 or more, and may be regarded as the RGB light source region. If one component is equal to or greater than a predetermined saturation reference value 46, the pixel may be regarded as the RGB light source area.

For example, if one of the RGB components of a pixel in the low-exposure image 42 represented by 0 to 255 gradations is 255, the pixel may be set as the RGB light source area. Also, for example, if the saturation reference value 46 is set to 250, and any one of the RGB components of a pixel in the low-exposure image 42 represented by gradation 0 to 255 is 250 or more, the pixel is regarded as the RGB light source region. do it.

If the remaining two components are not saturated, they may be set as the RGB light source region, or if the value of one saturated component or a value sufficiently smaller than the saturation reference value 46 is set as the RGB light source region. May be set. Here, the sufficiently small value may be, for example, a value that is 10% or less of the saturation value of the RGB components.

For example, if the remaining two components are not saturated and set as the RGB light source region, the RGB components of a pixel in the low-exposure image 42 represented by 0 to 255 gradations are (R, G, B)=(255, 125, 100), the pixel is set as the RGB light source area. On the other hand, for example, if the remaining two components are set to be the RGB light source region if they are 10% or less of the saturation value, the RGB components of a pixel in the low-exposure image 42 represented by 0 to 255 gradations are (R, G, B) = (255, 125, 100), the pixel is not set in the RGB light source region, and (R, G, B) = (255, 25, 20), the pixel is set in RGB. Set to light source area.

Further, among the RGB components of the pixels of the low-exposure image 42, the sum of the values of any two components is equal to or greater than the predetermined saturation reference value 46, and the value of the remaining one component that has not been summed is the sum of the two components. A pixel having a value sufficiently smaller than the value of may be set in the RGB light source area. For example, in the case of a purple laser pointer, since the RGB components of the pixels of the low-exposure image 42 are (R, G, B)=(140, 0, 140), etc., the saturation reference value 46 is set to 255, The pixel may be set in the RGB light source area. Here, the sufficiently small value may be, for example, a value that is 10% or less of the total value of the two components, or a value that is 10% or less of the saturation reference value 46 .

Here, by using the low-exposure image 42 in the process of setting the RGB light source area, it is possible to set only the image area in which the light source emitting strong monochromatic light is captured as the RGB light source area. That is, it is preferable to use the low-exposure image 42 because there is a risk that even an object of a single color component that does not emit strong light in the normal-exposure image 40 will be extracted as the RGB light source region.

Next, extraction processing of a luminance light source region generated in the normal exposure image 40 by flashlight irradiation will be described.

The light source region extracting means 60 sets pixels having luminance values equal to or higher than a predetermined luminance reference value 48 among the pixels of the low exposure image 42 as luminance light source regions. Specifically, the luminance value of each pixel of the low-exposure image 42 is converted into an estimated luminance value when photographed with normal exposure from the ratio of the exposure amount of the normal-exposure image 40 and the low-exposure image 42, and the luminance value of each pixel is calculated. A pixel whose estimated luminance value is equal to or greater than a preset luminance reference value of 48 is set as a luminance light source region.

A luminance reference value 48 is a reference value representing a luminance value exceeding the dynamic range of the normal exposure image 40 . That is, the normal-exposure image 40 and the low-exposure image 42 are photographed as a set when the light of the flashlight is applied to the photographing unit 32, and the estimated luminance value necessary for detecting the obstruction of the visual field is set to the luminance reference value. Set as 48. At this time, it is preferable to set, as the luminance reference value 48, an estimated luminance value necessary for detecting obstruction of the field of view using various flashlights with different light intensities in the photographing section 32. FIG.

The brightness reference value 48 can be specifically obtained as follows. A set of normal-exposure images 40 and low-exposure images 42 are photographed in advance to obtain a plurality of sets of images. Then, an estimated luminance value obtained by converting the luminance value of each pixel of the low-exposure image 42 into the exposure amount of the normal-exposure image 40 based on the exposure amount ratio described later for each pair (when photographed with the exposure amount of the normal-exposure image 40 Estimated luminance value that can be obtained from Then, an estimated brightness value that can detect visual disturbance is specified, and the estimated brightness value is set as the brightness reference value 48 .

The exposure amount ratio can be calculated based on the shutter speed and gain of the imaging unit 32 . At this time, the aperture of the photographing unit 32 is fixed. However, the ratio may be obtained by adding an aperture or the like to the calculation below. The shutter speed and the gain of the imaging device when the normal exposure image 40 and the low exposure image 42 are captured by the imaging unit 32 are acquired. The shutter speed is converted into a decibel display [dB] according to Equation (1). For example, based on the shutter speed of 1/60 [msec], each shutter speed that can be set in the photographing unit 32 is converted into decibel display [dB]. When obtaining the exposure amount ratio, among the parameters such as shutter speed, gain, and aperture, the parameters obtained by photographing without being fixed by the photographing unit 32 may be used. For example, when shooting is performed with a fixed shutter speed and gain, the exposure ratio may be calculated from the gain.

20log (Out/In) = decibel display [dB] (1)

Here, Out: shutter speed used for shooting, In: standard shutter speed (eg, 1/60)

The exposure amount of the photographed image is obtained from the shutter speed and the gain after the decibel display [dB] conversion, based on the formula (2).

Exposure amount [dB] = gain [dB] + shutter speed [dB] (2)

For example, when the gain of the normal exposure image 40 is 6 [dB] and the shutter speed is 1/60 (0 [dB]), the exposure amount of the normal exposure image 40 is 6 [dB]. On the other hand, when the gain of the low-exposure image 42 is 0 [dB] and the shutter speed is 1/320 (-14 [dB]), the exposure amount of the low-exposure image 42 is -14 [dB]. When the exposure amounts of the normal-exposure image 40 and the low-exposure image 42 are obtained, the exposure difference [dB]=the exposure amount [dB] of the normal-exposure image 40−the exposure amount [dB] of the low-exposure image 42 is expressed in Equation (3). Based on this, the exposure amount ratio is calculated.

20log (exposure ratio) = exposure difference (3)

For example, if the exposure amount of the normal exposure image 40 is 6 [dB] and the exposure amount of the low exposure image 42 is -14 [dB], the exposure difference is 6-(-14)=20 [dB]. . When the exposure difference is 20 [dB], the exposure amount ratio is 10̂(20/20)=10. That is, it can be determined that the low-exposure image 42 was shot with an exposure amount 10% higher than that of the normal-exposure image 40 . Table 2 shows the relationship between the magnification of the exposure amount of the normal-exposure image 40 and the exposure amount of the low-exposure image 42 and the exposure difference.

The estimated brightness value is a value obtained by converting the brightness value of each pixel of the low-exposure image 42 into a brightness value when the image is captured with the exposure amount of the normal-exposure image 40 . Therefore, the estimated luminance value can be calculated by multiplying the luminance value of each pixel of the low-exposure image 42 by the exposure amount ratio. For example, when the low-exposure image 42 is shot with an exposure amount of 10% with respect to the normal-exposure image 40, the luminance value of each pixel of the low-exposure image 42 is multiplied by 10, which is the exposure ratio, to obtain the normal-exposure image 40. Calculates an estimated luminance value converted into a luminance value when photographing with an exposure amount of . When the luminance value of a pixel in the low-exposure image 42 is 60, the estimated luminance value is 600 (60 (luminance value of low-exposure image 42) x 10 (exposure ratio) = 600 (estimated luminance value)). Also, when the luminance value of a pixel in the low-exposure image 42 is 200, the estimated luminance value is 2000.

In this way, the estimated brightness value is calculated based on the exposure amount ratio, the estimated brightness value with which visual disturbance can be detected is specified, and the estimated brightness value is set as the brightness reference value 48 . For example, the brightness reference value 48 is set to 2000.

Note that if the same image area is determined to be both the RGB light source area and the luminance light source area, it is preferable to set it as the RGB light source area.

When the RGB light source area or the luminance light source area is set by the light source area extracting means 60, the visual field obstruction determination means 62 performs the following process for determining whether or not the visual field obstruction has occurred for the photographing section 32. FIG.

The visual field obstruction determining means 62 calculates the brightness value and the brightness variance value of the image area located in the vicinity of the light source area in the normal exposure image 40 according to the type of the light source area. At this time, for the luminance light source region, the average luminance value and variance value of the image region obtained by expanding the area of the luminance light source region by the first magnification are calculated. The first magnification is preferably set to 20%, for example. Also, for the RGB light source region, the average luminance value and variance value of the image region obtained by expanding the area of the RGB light source region by a second magnification lower than the first magnification with respect to the luminance light source region are calculated. The second magnification is preferably set to 10%, for example. However, instead of using different values for the first magnification and the second magnification according to the type of the light source region, the average luminance value and the variance value may be calculated uniformly. At this time, the image area for calculating the average luminance value and the variance value is calculated from the image area excluding the light source area in the expanded area.

Next, the visual field obstruction determination means 62 calculates the edge reduction rate. The number of edges of a reference image 44 photographed in the past and not determined to obstruct the field of view is compared with the number of edges of an ordinary exposure image 40, and the ratio of the number of edges decreased in the ordinary exposure image 40 from that of the reference image 44 is obtained. Here, the number of edges is the number of edges of the image area other than the image area obtained by expanding the image area set to the RGB light source area or the luminance light source area for both the normal exposure image 40 and the reference image 44 . Similar to the above, the expansion rate of the image area may be the first magnification for the luminance light source area and the second magnification for the RGB light source area. Alternatively, the number of edges may be obtained from an image area excluding pixels having luminance values equal to or higher than a predetermined luminance value in the normal exposure image 40 . The edge reduction rate may be a difference obtained by subtracting the number of edges between the normal exposure image 40 and the reference image 44, or {(the number of edges of the reference image 44−the number of edges of the normal exposure image 40)/the number of edges of the reference image 44}. may be Also, the reduction rate of the number of edges extracted with respect to the entire images of the normal exposure image 40 and the reference image 44 may be calculated. Note that the number of edges may be obtained from an image area other than the image area set as the RGB light source area or the luminance light source area for both the normal exposure image 40 and the reference image 44 .

It should be noted that the reference image 44 may be an image taken immediately before as long as it is a normal exposure image 40 that has been taken in the past and has not been judged as obstructing the field of view. For example, the normal exposure image 40 shot one frame before the normal exposure image 40 to be processed, or the normal exposure image 40 shot several frames before. Also, the reference image 44 may be a plurality of images instead of one. For example, edge number comparison processing may be applied to a plurality of reference images 44, and the result with the highest edge number reduction rate may be adopted.

The visual field obstruction determination means 62 performs processing for determining whether or not visual field obstruction has occurred with respect to the photographing section 32 according to the type of the light source region, based on the average luminance value, the variance value, and the edge reduction rate.

For the luminance light source region, the calculated average luminance value is equal to or greater than the first luminance determination reference value 50a, the calculated edge reduction rate is equal to or greater than the first edge determination reference value 52a, and the calculated variance value is equal to or greater than the first edge determination reference value 52a. If the value is less than the 1-dispersion criterion value 54a, it is determined that there is visual field obstruction. Further, for the RGB light source region, the calculated average luminance value is equal to or greater than the second luminance determination reference value 50b, the calculated edge reduction rate is equal to or greater than the second edge determination reference value 52b, and the calculated variance value is greater than or equal to the second dispersion criterion value 54b, it is determined that there is visual field obstruction.

Here, the second luminance criterion value 50b is set to a value lower than the first luminance criterion value 50a. Also, the second edge determination reference value 52b is set to a value smaller than the first edge determination reference value 52a. The second dispersion criterion value 54b may be set to a value equal to the first dispersion criterion value 54a, or may be set to a value different from the first dispersion criterion value 54a. When setting the first variance criterion value 54a and the second variance criterion value 54b to different values, the value of the second variance criterion value 54b should be higher than the value of the first variance criterion value 54a. .

In general, the luminance light source region of a flashlight tends to have a higher average luminance value than the RGB light source region of a laser pointer. Therefore, by setting the second luminance judgment reference value 50b to a value lower than the first luminance judgment reference value 50a, light that is not monochromatic light such as headlights that are not irradiated to the photographing unit 32 is mistaken as visual field obstruction. It is possible to detect the obstruction of the field of view by the laser pointer with high accuracy while suppressing misjudgment.

In addition, since the light of a flashlight or the like diffuses, the entire image of the normal exposure image 40 is often overexposed even if the light of the light is not pinpointed to the photographing unit 32 . On the other hand, when the entire image is not overexposed, it means that the light is applied to a part other than the camera lens, and in this case, an intruder who has entered the monitored area can be detected. On the other hand, since the light of the laser pointer generally does not diffuse, the entire image of the normal exposure image 40 will not be overexposed unless the light is applied to the photographing unit 32 at a pinpoint. When light is not applied to the photographing unit 32 at a pinpoint, only the light source area and the area in the vicinity of the light source area are overexposed, and the image area other than that area is not overexposed, and the luminance value is Sometimes it's just higher. Therefore, there is a possibility that the background object captured in the reference image 44 is also captured in the normal exposure image 40 as it is, and the edge extracted from such a background object changes between the reference image 44 and the normal exposure image 40. is small, the edge reduction rate is small, and there is a possibility that it will not be judged as obstruction of the visual field. On the other hand, unlike the light of the light, the luminance value of the entire image of the normal exposure image 40 is high, so the intruder may be buried in the area with high luminance, and the intruder may not be detected. . Therefore, by setting the second edge determination reference value 52b to a value smaller than the first edge determination reference value 52a, it is possible to detect, with high accuracy, visual disturbance caused by flashlights and laser pointers that affect intruder determination. .

In the case of a light source that is not monochromatic light, such as a flashlight, as shown in FIG. The brightness variance value in the region becomes smaller. On the other hand, in the case of a monochromatic light (single wavelength) light source such as a laser pointer, as shown in FIG. An image area with a high luminance value and an image area with a low luminance value appear due to interference. That is, in the normal-exposure image 40, the luminance variance value in the RGB light source area irradiated with light is larger than that in the luminance light source area. Therefore, visual field disturbance can be detected with high accuracy by determining visual field disturbance when the dispersion value is small in the luminance light source region and visual field disturbance when the dispersion value is large in the RGB light source region.

However, the presence or absence of visual field obstruction may be determined based only on the determination on the edge reduction rate without performing the determination on the average luminance value and the variance value. Alternatively, the presence or absence of visual field obstruction may be determined without determining the variance value without determining the edge reduction rate and the average luminance value. Further, the presence or absence of visual field obstruction may be determined without performing determination on the average brightness value by performing determination on the edge reduction rate and the variance value.

The flow of visual field obstruction determination processing for detecting visual field obstruction will be described below according to the flowchart shown in FIG.

The light source region extracting means defines pixels having luminance values equal to or higher than the luminance reference value 48 in each pixel of the low-exposure image 42 as luminance light source regions, and pixels having only one of the RGB components equal to or higher than the saturation reference value 46. It is extracted as an RGB light source area (step S10). If none of the light source areas are extracted, the visual field obstruction determination process is terminated. If the luminance light source area is extracted, the process proceeds to step S14, and if the RGB light source area is extracted, the process proceeds to step S26, and neither the luminance light source area nor the RGB light source area is extracted. If so, the process is terminated (step S12).

When the luminance light source area is extracted, the visual field obstruction determination means calculates the average luminance value of the image area surrounding the luminance light source area in the normal exposure image 40 (step S14). If the average brightness value is greater than or equal to the first brightness determination reference value, the process proceeds to step S18, otherwise the process ends (step S16). Subsequently, the luminance variance value of the image area around the luminance light source area is calculated (step S18). If the variance value is less than the first variance criterion value, the process proceeds to step S22, otherwise the process ends (step S20). Then, the edge reduction rate between the normal exposure image 40 and the reference image 44 is calculated (step S22). If the edge reduction rate is greater than or equal to the first edge determination reference value, the process proceeds to step S38 (step S24), and it is determined that the field of view of the imaging unit 32 has been obstructed (step S38). If the edge reduction rate is less than the first edge determination reference value, the process is terminated (step S24).

When the RGB light source area is extracted, the visual field obstruction determining means calculates the average brightness value of the image area around the RGB light source area in the normal exposure image 40 (step S26). If the average brightness value is greater than or equal to the second brightness determination reference value, the process proceeds to step S30, otherwise the process ends (step S28). Subsequently, the brightness variance value of the image area around the RGB light source area is calculated (step S30). If the variance value is greater than or equal to the second variance criterion value, the process proceeds to step S34, otherwise the process ends (step S32). Then, the edge reduction rate between the normal exposure image 40 and the reference image 44 is calculated (step S34). If the edge reduction rate is equal to or greater than the second edge determination reference value, the process proceeds to step S38 (step S36), and it is determined that the field of view of the imaging unit 32 has been obstructed (step S38). If the edge reduction rate is less than the second edge determination reference value, the process is terminated (step S36).

As described above, according to the present embodiment, even when the imaging unit 32 is irradiated with the laser pointer, it can be determined with high accuracy that the field of view is disturbed.

In the above embodiment, the light source area is divided into the RGB light source area and the luminance light source area. processing may be performed. For example, it is determined whether or not the size of the extracted light source region is equal to or greater than the region reference value. Also, the reference value for calculating the edge reduction rate is set low, and the visual field obstruction determination is performed. In other words, when the extracted light source region has a large area, visual disturbance due to the light source region occurs more than when the light source region is small. It is preferable to determine that On the other hand, when the light source area is small, the field of view obstruction due to the light source area is small. Therefore, by judging the edge reduction rate of the image area other than the light source area more strictly than when the light source area is large, the phenomenon that should be judged as obstruction of the field of vision can be avoided. can be determined with high accuracy. Note that there may be a plurality of reference values for determining the size of the light source region, and the size of the light source region may be divided into multiple levels.

In the above-described embodiment, when a set of captured images (normally exposed image 40 and low-exposed image 42) satisfies the conditions for determining visual field obstruction, it is determined that visual field obstruction has occurred. It may be determined whether or not visual field obstruction has occurred from the visual field obstruction determination result. For example, if there are 3 sets (3 frames) of captured images that satisfy the conditions for determining visual field obstruction among the 6 sets (6 frames), it may be determined that visual field obstruction has occurred.

Note that the image processing device 22 is not limited to processing images stored in the storage unit 34 within the device itself. For example, the configuration may be such that an image read from another external device is subjected to image processing, or an image received from an external camera is subjected to image processing. Alternatively, a plurality of cameras may be provided, and the normal-exposure image 40 and the low-exposure image 42 may be captured by each camera. In that case, the normal-exposure image 40 and the low-exposure image 42 may be shot simultaneously.

In the above-described embodiment, light source extraction and visual field obstruction determination are performed using light from a laser pointer, and light source extraction and visual field obstruction determination are performed using light from a light such as a flashlight. Only extraction of the light source and determination of obstruction of the field of vision by light of the laser pointer may be performed except for the configuration of extracting the light source and determining obstruction of the field of view using light such as the above.

Although the embodiments according to the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention.

10 security system, 12 object to be monitored, 14 security device, 16 communication network, 18 security center device, 20 user device, 22 image processing device, 24 recording device, 30 communication unit, 32 imaging unit, 34 storage unit, 36 control part, 40 normal exposure image, 42 low exposure image, 44 reference image, 46 saturation reference value, 48 luminance reference value, 50 luminance judgment reference value, 50a first luminance judgment reference value, 50b second luminance judgment reference value, 52 edge judgment reference value 52a first edge judgment reference value 52b second edge judgment reference value 54 dispersion judgment reference value 54a first dispersion judgment reference value 54b second dispersion judgment reference value 60 light source region extracting means 62 field of view Interference determination means.

Claims (7)

A first photographed image photographed with a first exposure amount and photographed at a second exposure amount smaller than the first exposure amount at the same time as the first photographed image among photographed images sequentially photographed of a predetermined space by a photographing device. Acquisition means for acquiring the second captured image;
extracting means for extracting an RGB light source region composed of pixels in which only one of RGB component values of each pixel of the second captured image is saturated;
When the RGB light source region is extracted, the target image other than the region corresponding to the RGB light source region extracted from the second captured image captured at the same time as the first captured image in the first captured image. and a reference other than the area corresponding to the RGB light source area extracted from the second captured image captured at the same time as the first captured image in the reference image captured in the past with the first exposure amount. and a visual field obstruction determining means for determining that the field of view of the photographing device is obstructed when the number of edges of the target image area is smaller than that of the reference image area by a first edge determination reference value or more. and,
An image processing device comprising: The image processing device according to claim 1,
The extraction means is configured such that only one of the RGB component values of each pixel of the second captured image is saturated, and the remaining two component values are sufficiently smaller than the saturated component values. An image processing apparatus, wherein an image area composed of pixels is extracted as the RGB light source area. The image processing device according to claim 1 or 2,
The extracting means further extracts a luminance light source region composed of pixels having a luminance value equal to or higher than a luminance reference value among the pixels of the second captured image,
When the luminance light source region is extracted, the visual field obstruction determination means determines, in the first captured image , the luminance light source region extracted from the second captured image captured at the same time as the first captured image. a target image area other than the corresponding area; and a reference image area other than the area corresponding to the luminance light source area extracted from the second captured image captured at the same time as the first captured image in the reference image. , and if the number of edges in the target image area is smaller than that in the reference image area by a second edge determination reference value that is larger than the first edge determination reference value, the field of view of the photographing device is obstructed. An image processing apparatus characterized by determining that The image processing device according to claim 3,
The visual field obstruction determination means includes:
With respect to the luminance light source region, if the variance value of the luminance of the image region surrounding the image region of the first captured image corresponding to the luminance light source region is less than a first variance determination reference value, the field of view of the photographing device is determined to be disturbed,
For the RGB light source region, furthermore, the luminance variance value of the image region surrounding the image region of the first captured image corresponding to the RGB light source region is equal to or greater than the first variance determination reference value, and is equal to or greater than the second variance determination reference value. An image processing apparatus that determines that the field of view of the imaging device is obstructed when . The image processing device according to claim 1 or 2,
The visual field obstruction determination means includes:
When the size of the RGB light source region extracted from the first captured image is equal to or less than a predetermined region reference value, the number of edges of the target image region is greater than that of the reference image region than the first edge determination reference value. and determining that the field of view of the photographing device is obstructed when the edge is also decreased by a second high edge determination reference value or more. A first photographed image photographed with a first exposure amount and photographed at a second exposure amount smaller than the first exposure amount at the same time as the first photographed image among photographed images sequentially photographed of a predetermined space by a photographing device. Acquisition means for acquiring the second captured image;
extraction means for extracting an RGB light source region made up of pixels for which a sum of values of any two of RGB component values of each pixel of the second captured image is equal to or greater than a saturation reference value;
A target image region in the first captured image other than the region corresponding to the RGB light source region extracted from the second captured image captured at the same time as the first captured image, and a region past the first captured image. A reference image area other than the area corresponding to the RGB light source area extracted from the second captured image captured at the same time as the first captured image in the reference image captured at the first exposure amount at the time of and, if the number of edges in the target image area is smaller than that in the reference image area by an edge determination reference value or more, a visual field obstruction determination means for determining that the visual field of the photographing device is obstructed;
An image processing device comprising: the computer,
A first photographed image photographed with a first exposure amount and photographed at a second exposure amount smaller than the first exposure amount at the same time as the first photographed image among photographed images sequentially photographed of a predetermined space by a photographing device. Acquisition means for acquiring the second captured image;
extracting means for extracting an RGB light source region composed of pixels in which only one of RGB component values of each pixel of the second captured image is saturated;
When the RGB light source region is extracted, the target image other than the region corresponding to the RGB light source region extracted from the second captured image captured at the same time as the first captured image in the first captured image. and a reference other than the area corresponding to the RGB light source area extracted from the second captured image captured at the same time as the first captured image in the reference image captured in the past with the first exposure amount. and a visual field obstruction determining means for determining that the field of view of the photographing device is obstructed when the number of edges of the target image area is smaller than that of the reference image area by a first edge determination reference value or more. and,
An image processing program characterized by functioning as

Images