JPH10289321A - Image monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent erroneous detection due to the periodical fluctuations of illumination for an image monitoring device which detects a mobile object via the image processing performed based on the graphic information that is photographed by a TV camera. SOLUTION: It is decided whether the graphic information is affected by the fluctuation of illumination based on the control information on a TV camera 100. If the graphic information is not affected by the fluctuation of illumination, the image processing is carried out based on the graphic information which is fetched at a certain time as a 1st input image to be processed and also the graphic information which is fetched at an earlier time than the 1st input image as a 2nd input image. If the graphic information is affected by the fluctuation of illumination, the image processing to be carried out based on the image graphic is passed and instead the graphic information of the same phase set on the illumination fluctuation cycle is fetched to carry out the image processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image monitoring apparatus, and more particularly to a method for monitoring a moving object in a store, a financial institution, a warehouse, a building, a house, or the like in an environment having periodic fluctuations in illumination intensity. The present invention relates to an image monitoring device for preventing erroneous detection due to various illumination fluctuations and for accurately identifying a moving object.

[0002]

2. Description of the Related Art In image monitoring for detecting a moving object by performing image processing based on video information obtained by photographing an environment having periodic illumination fluctuations, an image pickup control unit is provided to prevent erroneous detection. And a method of imaging at a point in time when the phases on the cycle of illumination fluctuation are the same (Japanese Patent Laid-Open No. 4-235489)
Reference).

[0003]

In a building, lighting that uses natural light and fluorescent light or incandescent light from a commercial power supply is used, and the brightness thereof is constantly changing.
In such an environment, when monitoring a moving object such as a suspicious person by performing image processing on video information obtained by photographing a monitoring target area with an ITV camera, for example, adjusting the aperture and shutter speed of the ITV camera to a certain brightness If it is set to a fixed value at that point, if the illuminance fluctuates, unclear video information (image)
Will be. Therefore, in order to obtain clear video information that follows a change in brightness, an ITV camera shoots an image by setting the control of the aperture, shutter speed, and the like to an automatic mode.

In the automatic mode of the ITV camera, the aperture, shutter speed and the like are controlled and changed for each frame (shooting scene) in accordance with the constantly fluctuating brightness.
For example, the shutter speed for bright illuminance is １ ／
500 s, 1/5000 s, etc., and 1/60 s, 1/100 s, etc. for relatively dark illuminance. Moreover,
Illumination variation of a fluorescent lamp generally changes periodically according to a variation of a power supply voltage of either 50 Hz or 60 Hz depending on a region.

When an ITV camera outputs a video information signal every 1/60 second at a shutter speed of 1/60 s,
In this ITV camera, the storage of video information inside is 1
/ 60 seconds, output of video information signal to outside is 1
/ 60 seconds, and the video information signal from the ITV camera is not actually the video at the moment at 1/60 seconds. Shutter speed 1 / 100s,
The same applies to the case of 1 / 2500s, 1 / 5000s and the like.

When the shutter speed is set to the automatic mode, the shutter speed is constantly changed to 1/60 s, 1/100 s, 1/250 s, 1/5 due to the change in the brightness of the environment.
00s, 1 / 1000s, 1 / 2500s, 1/500
0 s, 1/10000 s, etc. In the case of a fixed shutter speed, the initially set value, that is, 1/60
s, 1 / 100s, etc., and do not change. Therefore, regardless of the fixed shutter speed method or the automatic shutter speed method, flicker is always generated in the illumination of the fluorescent lamp whose illuminance is periodically changed.

However, in the case of relatively dark illuminance (for example,
Shutter speed 1 / 60s, 1 / 100s, 1/25
0s), a countermeasure process can be performed to suppress the occurrence of flicker of video information inside the ITV camera (processing time can be secured because the shutter speed is slow). However, in the case of bright illuminance (for example, when the shutter speed is 1/500 s, 1/1000 s, 1/25
00s, 1 / 5000s, 1 / 10000s, etc.)
In the ITV camera, a countermeasure process for suppressing the occurrence of flicker cannot be performed (the processing time cannot be secured because the shutter speed is high).

Therefore, at a shutter speed in which a countermeasure process for suppressing the occurrence of flicker cannot be performed inside the ITV camera, the monitoring target area is imaged and output in either the fixed shutter speed mode or the automatic shutter speed mode. Flicker occurs in the displayed video information (image), and the video information (image) to be input to the monitoring processing device.
Causes flickering.

In the method in which an imaging control unit is provided to perform imaging at the time when the phases on the illuminance fluctuation cycle are the same, video information at the same phase is taken in among the video information output from the ITV camera to the outside. The storage of video information inside the ITV camera is performed near the same phase,
Output to the outside of the TV camera is performed at the same phase.
Therefore, the actual video information signal does not necessarily output the video information itself at the moment stored at the same phase, so that in the case of fluorescent lamp illumination, the illuminance is not the same, and flicker due to the fluorescent lamp illumination occurs. However, there is a problem that the video information signal is input, and erroneous detection frequently occurs.

SUMMARY OF THE INVENTION An object of the present invention is to prevent a false detection caused by flicker or the like due to a periodic illumination fluctuation and to reliably identify a moving object online even in an environment where a periodic illumination fluctuation occurs. It is an object of the present invention to provide an image monitoring apparatus capable of performing the above.

Another object of the present invention is to detect a moving object in a shooting target area based on video information obtained by shooting a monitoring target area using an ITV camera having a function of preventing flickering of the video information. An object of the present invention is to provide an image monitoring apparatus capable of preventing erroneous detection due to flicker in a state where a flicker prevention countermeasure function does not function due to a high shutter speed of an ITV camera when monitoring.

[0012]

SUMMARY OF THE INVENTION According to the present invention, in an environment where periodic illumination fluctuations occur, an area to be monitored is photographed by a TV camera having a function for preventing flicker and a change in the area is monitored by an image. It is an image monitoring device.

This image monitoring apparatus obtains control information for determining whether or not there is an influence of periodic illumination fluctuation on video information output from an ITV camera. The control information uses, for example, shutter speed information of a TV camera, and is used as information for determining whether or not the influence of periodic illumination fluctuation is taken for each shooting scene based on the shutter speed information. Then, by referring to this control information, if the countermeasure function determines that the image information does not have the effect of the periodic illumination fluctuation due to the case, the image information captured at a certain time to be processed is determined. By the first
And the second input image based on the video information captured at the previous time is processed as it is to monitor the moving object.

On the other hand, if it is determined that the video information is affected by the periodic illumination fluctuation, the image processing of the photographed scene is omitted (passed) or the same phase in the illumination fluctuation cycle is used. The next video information is fetched and the image processing is performed, or the difference between the first input image of the video information fetched at a certain time and the video information fetched at the previous time is changed in the fluctuation cycle. It is determined that the video information captured when the phase is the same is selected and the image is processed as the second input image.

A difference between the first input image and the second input image is generated to generate difference image data, and a threshold for binarization is automatically determined from the density frequency distribution of the difference image to generate a binary image. Is created to extract the changed part. Using a binary image from which noise has been removed from the extracted changed portion, a binary image in an adjacent region is used.
Merge value images.

Finally, the extracted object is determined by calculating the characteristic amount of the integrated area, and a moving object is detected without disturbance. The detected moving object is displayed on the display device in real time by the display control unit, so that an image of the identified intended object can be monitored online on a display device such as a video monitor.

According to such an image monitoring apparatus, the environment to be monitored is photographed with an ITV camera in an environment where there is illuminance fluctuation in the background and periodic illumination fluctuation exists, and changes in the monitoring area are monitored. In an image monitoring system, an image processing method can be flexibly supported according to a demand for object detection accuracy. For example, when absolutely no false detection is performed, image processing based on video information that is affected by periodic illumination fluctuations is omitted, and image processing is performed using only video information that is not affected by periodic illumination fluctuations. By doing so, erroneous detection due to the influence of flicker or the like can be eliminated. In addition, when the moving object is detected so as not to be overlooked, image processing is performed using the video information input at the same cycle of the illumination fluctuation regardless of the presence or absence of the influence of the periodic illumination fluctuation. When the shutter speed of the ITV camera is in the automatic control mode, for example, 1/6
Since it changes from 0 to 1 / 10,000 at any time, the influence of the periodic illumination fluctuation rarely occurs in all the photographed scenes being monitored, so that the erroneous detection due to the influence of flicker or the like is considerably reduced. It becomes possible.

Furthermore, in the image processing, a difference between the first input image and the second input image is calculated to generate difference image data, and a binarization threshold value is determined from the density frequency distribution of the difference image. Automatically determine and create a binary image, remove noise, integrate binary images in adjacent areas, and identify the extracted object. Therefore, it is necessary to exclude disturbance and detect multiple objects separately. Therefore, highly accurate detection is possible.
Then, since the detected moving object is displayed on the display device in real time, the observer can online monitor the video in which the intended moving object is captured.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of an image monitoring apparatus showing one embodiment of the present invention. In this image monitoring apparatus, the ITV camera 100 shoots a monitoring target scene and outputs a video information signal. This ITV camera 100
Up to a shutter speed of 1/250 s, a flicker countermeasure processing function for performing flicker countermeasure processing on the video information signal is provided, and an automatic control function and a manual control function are provided for the shutter speed. Image input unit 800
Holds video information and shooting timing information output from the ITV camera 100. This image input unit 800
Is provided with a memory for accumulating and storing video information for at least two photographing scenes necessary for generating difference image data based on images of two preceding and succeeding video information and its photographing timing information.

The camera control information output unit 400 captures and outputs shutter speed information at the time of each photographing from the ITV camera 100.

The processing method determination unit 600 determines the shutter speed for each shooting scene, and the video information of the shooting scene is free from the effects of periodic illumination fluctuation such as flicker (flicker countermeasure processing has been performed). When it is determined that the image information input by the image input unit 800 is the video information captured at a certain time as the first input image to be processed and the video information captured at a previous time as the second input image, It is determined that the method of performing image processing using the image of the video information as an input image as it is is used, and the difference image creation unit 1000 creates difference image data by calculating the difference between the first input image and the second input image. Then, the image input unit 800 and the difference image creation unit 1000 are instructed. If it is determined that video information that has not been subjected to flicker countermeasures and is affected by periodic illumination fluctuations such as flicker has been output, the image input unit 8
The difference image data creation processing of the first input image and the second input image input at 00 is omitted (passed), and the ITV camera 1
Returning to the photographing step of acquiring a video information signal of the next photographed scene (preferably the same phase in the illumination fluctuation cycle) photographed in 00, or in the video information input and held by the image input unit 800 From, the video information captured when the difference between the capture time of the scene captured at a certain time as the first input image and the capture time of the second input image captured at the previous time becomes the same in the illumination fluctuation cycle is selected. Then, it is determined that a method of performing processing as an input image of two shooting scenes in which flicker or the like is reduced is used, and the difference image creating unit 100
At 0, an instruction is made to make a difference between the first input image and the second input image to create difference image data.

Next, the object extracting unit 2000 automatically determines a threshold for binarization from the density frequency distribution of the difference image data created by the difference image creating unit 1000, and creates a binary image. The object detection unit 3000 includes the object extraction unit 200
After removing the noise of the binary image created in step 0 and integrating the adjacent binary images, the feature amount of the integrated area is calculated, the extracted object is identified, and the intended moving object detection process is performed.

When an intended moving object is detected by the object detecting unit 3000, the display control unit 4000 stores the detected data in a memory or the like, and uses the stored data, such as image information and feature information, to display a display device. The control for displaying in real time at 5000 is performed.

These ITV camera 100, camera control information output unit 400, processing method determination unit 600, image input unit 800, difference image creation unit 1000, object extraction unit 200
0, the object detection unit 3000 sequentially and repeatedly executes the processing, so that the display control unit 4000
At 0, the detection result is displayed in real time, and the observer can monitor the image of the identified moving object on-line on a display device such as a monitor.

FIG. 2 is an explanatory diagram showing a timing of outputting video information when a periodic illumination variation and a shutter speed of an ITV camera change, and is an example of illumination variation by a fluorescent lamp. In an area where the supply power is 50 Hz (or 60 Hz), the fluorescent lamp periodically fluctuates in illuminance 220, and one cycle T from mountain to mountain (or from valley to valley).
Is 100 (or 1) for 50 Hz (or 60 Hz)
20) Period / sec.

Now, the shutter speed of the ITV camera is 1
The video information signal 230 in the case of / 60 s is output every 1/60 second, and the video information signal 240 in the case of 1/100 s
Is output every 1/100 second, and similarly, the video information signal 250 for 1/250 s is output every 1/250 second, and the video image signal 260 for 1/500 s is 1/5
It is output every 00 seconds, and the video information signal 270 for 1 / 1000s is output every 1/1000 seconds. Therefore,
If the shutter speed is low (low speed), the time from output at a certain time to the next output is long, and if it is high (high speed), the time is short. However, the higher the shutter speed, the smaller the number. For example, it is difficult to realize a process such as a measure against flicker for each video information signal at a high shutter speed.

FIG. 3 shows the camera control information output unit 400.
FIG. 2 is a block diagram illustrating an embodiment of the internal configuration of the embodiment. IT
When the V camera 100 shoots the monitoring target area and outputs a video information signal, the shutter speed mode determining unit 420 determines whether the shutter speed of the shooting scene is in the fixed mode or the automatic control mode, In the automatic control mode, the shutter speed information of the shooting scene is acquired from the ITV camera 100, and the shutter speed identification unit 4
40, and in the case of the fixed mode, the ITV camera 1
A value indicating a fixed value preset to 00 is output to shutter speed output section 460. Shutter speed identification unit 4
Reference numeral 40 denotes a shutter speed information of the shooting scene and an identification value (for example, “1” to 9 for 9 kinds of changes)
“9”) is output to the shutter speed output unit 460.

The processing method determining section 600 determines how to process video information according to the value output from the shutter speed output section 460.

FIG. 4 shows the shutter speed output unit 460.
6 is a flowchart showing an embodiment of a shutter speed acquisition processing procedure to be executed in FIG.

First, in step 500, the ITV camera 1
It is determined whether the shutter speed of 00 is a fixed mode or an automatic control mode.
A value preset in the ITV camera 100, for example, 1/6
If it is set to 0, it outputs 1 / 60s, and if it is set to 1/100, it outputs a value of 1 / 100s. In the case of the automatic control mode, the identification value of the shutter speed is determined in step 505. For example, if it is determined that the shutter speed changes to 9 types, if the identification value is “1”, 1 / 60s is determined in step 510,
If the discrimination value is “2”, 1 / 100s in step 520
If the identification value is “3”, 1/25 in step 530
0s, and if the identification value is “4”, 1 /
500 s, if the identification value is “5”, 1/1000 s in step 550; if the identification value is “6”, step 5
60, 1/2500 s. If the identification value is “7”, 1/5000 s in step 570. If the identification value is “8”, 1/10000 s in step 580. If the identification value is “9”, At step 590, a shutter speed value of 1/1500 s is obtained and output.

FIG. 5 is an explanatory diagram of a case where the shutter speed value of a certain photographic scene is displayed on the display device 5000 according to the present invention, in which the office where the OA equipment 480 is installed is illuminated by the fluorescent lamp 470. is there. The office where the OA equipment 480 such as a PC is installed is photographed with the ITV camera 100 while illuminating with the fluorescent lamp 470, and the image of the photographed scene and the shutter speed value at the time of photographing the scene are displayed at the lower right of the screen. A column 490 (for example, shutter speed = 1/500) is shown. The display form may be any display as long as the display shows the shutter speed.
An example in which the shutter speed is 1/500 is a bright shooting scene, and the output interval of the video information signal from the ITV camera 100 is short, so that it is not possible to perform the flicker countermeasure process. OA
The light is reflected brilliantly on the device 480, causing a change in luminance on the reflection surface.

FIG. 6 is a block diagram showing an embodiment of the internal structure of the processing method determining section 600. The shutter speed check unit 620 includes the camera control information output unit 400
The shutter speed is checked based on the value of the shutter speed information at the time of shooting of the monitoring target area output from. next,
The flicker measure determination unit 640 due to the periodic illumination fluctuation,
Using the check result, it is determined whether or not the image information of the processing target scene is affected by a periodic illumination change such as flicker. This criterion is, in this embodiment, IT
A shutter speed (1/25) at which the V-camera 100 can execute a flicker countermeasure process for preventing flicker due to periodic illumination fluctuations such as flicker on the video information signal.
0s). When the flicker countermeasure determination unit 660 determines that there is no influence due to periodic illumination fluctuation such as flicker, the processing method determination unit 660 determines whether the image information captured at a certain time as the first input image to be processed is The video information captured at the previous time as the second input image is determined as a method of performing image processing as the input image as it is.
On the other hand, if it is determined that there is an influence of periodic illumination fluctuation such as flicker, a method of omitting the image processing of the input video information and the video information captured before it is determined.
The video information of the next shooting scene (preferably the same phase on the illumination fluctuation cycle) is input from the camera 100, or
From the video information already input and stored, the difference between the video information captured at a certain time as the first input image and the capture time of the second input image captured at the previous time is the illumination fluctuation cycle. The video information fetched when the phase is the same as above is selected, and a method of processing the video information in which flicker or the like is reduced as an input image is determined. The latter processing method is
Since video information of a large number of past shooting scenes is required, it is necessary to provide a memory for storing a large number of video information in the image input unit 800 so that this method can be selected. . The processing technique determination unit 660 instructs the image input unit 800 and the difference image creation unit 1000 to perform processing for creating difference image data according to the determined technique.

FIG. 7 is a flowchart showing an embodiment of the determining procedure in the processing method determining section 660.
In step 700, the value of the shutter speed is checked, and if the shooting of the processing target scene is a shutter speed capable of outputting video information on which measures against flicker and the like have been processed,
That is, if it is 1/60, 1/100, or 1/250, a flag indicating that flicker countermeasure processing has been performed is set in step 710, and a processing method is determined in step 720 that uses the input video information at the captured time as it is. .

On the other hand, if the shooting of the processing target scene is a shutter speed at which countermeasure processing such as flicker can not be performed, that is, 1/500, 1/1000, 1/2500, 1/5.
If it is 000, 1/10000, 1/17500, in step 730, a flag indicating that the flicker countermeasure processing has not been performed is set. In step 740, it is determined whether or not to omit the image processing of the target shooting scene in which the influence of flicker or the like has occurred on the video information. If it is omitted, the process returns to step 700; A method of performing image processing using video information input at the same point in time on a periodic basis is determined.

When executing the processing by referring to the video information of the next target shooting scene and the shutter speed corresponding to the video information while omitting the image processing of the target shooting scene, the processing of the target shooting scene for which the image processing is omitted is performed. The video information is held in the image input unit 800 as the second input image, and the video information having the same phase on the illumination fluctuation cycle subsequent to the target shooting scene is selected and taken in as the first input image. By doing so, it is possible to shorten the interval until the next image processing and reduce the monitoring omission. In this selection processing, the cycle of the illumination fluctuation is determined from the commercial power frequency of the area where the device is installed, and the timer is used. Video information signals generated at the same phase may be taken in.

The image information at the same phase in the illumination fluctuation cycle is, for example, in a region of 50 Hz, the difference between the time of capturing the first input image and the time of capturing the second input image is 10 mm. This is a video input in integer multiples of seconds. It should be noted that the image processing of the target photographing scene is omitted in step 740 because image processing is performed only on video information that does not have the effect of periodic illumination fluctuation, thereby eliminating the influence of flicker and the like and never detecting erroneously. That's why. If the image processing of the target shooting scene is not skipped and the processing is performed, the influence of the periodic illumination fluctuation exists only in the shooting scene. However, the shutter speed of the ITV camera 100 is set to the automatic control mode. In such a case, since the light intensity changes as needed, for example, 1/60 to 1 / 10,000 or the like, the influence of the periodic illumination change rarely occurs in all monitoring scenes being monitored. It is possible to considerably reduce erroneous detection due to the influence of flicker and the like.
In this way, the processing method may be flexibly selected according to the requirement of the detection accuracy of the moving object. Here, the criterion for determining the presence or absence of the influence of flicker or the like is based on the ITV camera 10 to be used.
What is necessary is just to set suitably according to the value of a shutter speed according to the type of 0 (countermeasure processing function).

The processing of steps 740 and 750 can be performed not fixedly but fixedly by setting in advance an image processing method for video information for which flicker countermeasures have not been taken.

FIG. 8 is a block diagram showing an embodiment of the internal configuration of the image input unit 800. ITV camera 1
The image information signal which is captured and output from the monitoring target area by the A / D converter 810 is A / D converted, and the video information captured at one timing is used as a reference input image Fi-1 in the memory 820. And the video information captured at the other timing is stored in the memory 860 as an input image Fi to be processed. Memory 820,
The video information stored in 860 is sequentially updated with continuously input new video information.

FIG. 9 is a block diagram showing an embodiment of the internal configuration of the object extracting unit 2000. The density frequency distribution of the difference image is calculated by the density frequency distribution calculation unit 2100 for the difference image data generated by calculating the difference of each pixel between the images of the two pieces of video information by the difference image generation unit 1000, and noise is removed. The unit 2200 generates the density frequency distribution H _j (j
= 0, 1,... N) is smoothed by (Equation 1) to remove noise. n is the maximum value of the density.

H _j = (H _j-1 + 2H _j + H _{j + 1} ) / 4-1 where j = 1,..., N-1 H ₀ = (2H ₀ + H ₁ ) / 3-1 max = _{H n = (2H n + H} n-1) / 3-1 ...... ( Equation 1) binarization threshold calculation unit 2300, a noise removing unit 220
0 from the density frequency distribution created in step 0.
Is automatically calculated and supplied to the binarization unit 2400, and the binarization unit 24
Reference numeral 00 refers to the threshold value to binarize the difference image data to create a binary image. Thereby, the changed object region is extracted. Here, when the ITV camera 100 that cannot be extracted as difference image data by the difference image creation unit 1000 even when the monitoring place is dark and a moving object exists is used, an image information signal from which the difference image data can be extracted is output. The lighting may be turned on at the obtained brightness.

Here, if the maximum (maximum density) calculated by (Equation 1) is equal to or less than a predetermined value, it is determined that there is no change, and the noise removal unit 2200 and the binarization threshold calculation unit 23
The processing in the 00, binarization unit 2400 and the object detection unit 3000 may not be performed. For example, in video information obtained by photographing a CCD camera in an indoor daytime environment (or fluorescent light illumination) in an automatic shutter control mode, the image density ma
When x (maximum density) is about 6 to 12 gradations, it is determined that there is no change.

FIG. 10 shows the binarized threshold value calculating section 23.
FIG. 9 is an explanatory diagram showing an embodiment for calculating a binarization threshold value in 00.

The binarization threshold value calculation unit 2300 automatically calculates a binarization threshold value (th) from the maximum density value max2310 and the lower limit value thmin2340 obtained from the density frequency distribution smoothed by the noise removal unit 2200. Is calculated.

First, when there is a moving object or the like and there are many areas of the changing portion, the smoothed density frequency distribution is as shown in FIG. A median value 2320 between the maximum density max2310 and the lower limit value thmin2340 is calculated. 2 if the calculated median value 2320 is larger than the upper limit value 2330
The threshold value th of the binarization is set to the upper limit value 2330, and the upper limit value 233
If it is less than 0, the threshold value th of the binarization is set to the median value of 2320.
And

On the other hand, when the area of the changed portion is small, that is, when the density of the changed portion is small, the density frequency distribution is as shown in FIG. In this case, the binarization threshold value th2370 is calculated in the same manner as in FIG.
That is, the median value 2370 between the maximum density max 2350 and the lower limit value 2340 is calculated, and the median value 2370 is calculated as the upper limit value 2330.
If it is larger than the threshold value th of the binarization, the upper limit value 2330
If the value is less than the upper limit value 2330, the threshold value for binarization th
Is the median value 2370. Further, the case shown in FIG. 10A may be performed by a method of determining a binarization threshold value by a mode method in image processing.

Here, the lower limit value thmin 2340 is a maximum value of white noise in a normal state calculated in advance from a normal image to be monitored. Further, any value may be used as long as the value indicates the maximum noise density in the normal state of the monitoring target image.
Similarly, the upper limit value 2330 is a value obtained by adding a predetermined value to the maximum value of white noise in a normal state calculated in advance from a normal image to be monitored. In addition, any value may be used as long as a predetermined value is added to the value indicating the maximum noise density in the normal state of the monitoring target image. For example, in an image of video information obtained by shooting a daytime environment (or fluorescent light illumination) indoors using a CCD camera in an automatic shutter control mode, the upper limit value is 23.
30 is about 14 to 20 gradations, and the lower limit value is thmin 2340.
Is about 5 to 8 gradations.

FIG. 11 is a block diagram showing an embodiment of the internal configuration of the object detection unit 3000. The small area removing unit 3100 removes, as noise, a small area that is inappropriate for the extraction target from the binary image created by the binarizing unit 2400 in the object extracting unit 2000, and removes the binary image of only the changed area. create. Since the created binary image of only the change region is often separated from the extraction target object,
The value image integration processing unit 3200 integrates binary images at adjacent positions in these separated areas as one object. The feature amount calculation unit 3300 includes a binary image integration processing unit 3
In step 200, the feature amount of the binary image or the gray image in the integrated area is calculated. The object identification unit 3400 identifies a moving object based on the feature amount calculated by the feature amount calculation unit 3300. When the intended moving object is identified by the object identification unit 3400, the display control unit 40 determines that the intended moving object exists.
Output to 00.

Here, the reason why the small area removing unit 3100 creates a binary image of only the change area is, for example, an area having an area corresponding to a person when the extraction target is a person. Follow the field of view of the subject. Therefore, the size to be extracted may be set appropriately in accordance with the visual field range. The setting of the size may be automatically calculated in proportion to the visual field range, or may be manually designated in advance offline.
The small area to be removed by the small area removing unit 3100 as noise is about 0.15% to 0.25% of the set person size in the case of extracting a person.

FIG. 12 shows the binary image integration processing section 320.
0 is an explanatory diagram showing an embodiment when a rectangular image is generated as an object from a plurality of binary images.

For example, when the moving object is a person, the binary image extracted by the minute area removing unit 3100 excluding the minute area is based on the surrounding brightness, the color of the clothes of the person, and the background color. It is rare that a part of a person's head 3210, 3290, a part of a body 3220, or a part of a leg 3230 is concentrated and extracted in the vicinity of a rectangle 3240. Similarly, part 32 of another person's head
50, a part 3260 of the body, and a part 3270 of the leg are concentrated and extracted in the vicinity of the rectangle 3280. By the way, rectangle 324
0,3280 is the circumscribed shape of each extracted adjacent part.

FIG. 13 shows the binary image integration processing section 320.
0 is a flowchart illustrating an embodiment of a procedure for integrating a plurality of binary images into rectangles 3240 and 3280 as objects.

In step 3205, the small area removing unit 31
Labeling of the binary image created in 00 is performed. 1 to Ln
When the labels up to are added, the total number of labels becomes Ln.

In order to repeat the processing of the binary image having the total number of labels Ln, the label number (i) is initialized in step 3207.

Next, in step 3215, the label number is incremented by one (i is increased). In step 3217, it is checked whether or not the image processing has been completed for all the labels. , The process from step 3225 on is performed on the binary image of the i-th label.

In step 3225, the maximum coordinates and the minimum coordinates are calculated for the binary image of the i-th label.

In step 3227, it is checked whether the distance between the binary image of the ith label and the binary image of the (i + 1) th to Lnth labels in the X direction and the Y direction is within the allowable range. In this case, in step 3235, all the labels within the allowable range are added to the i-th label, and the added label is deleted. Then, in step 3245, the i-th to L-th
For the nth label, sort in ascending order, step 3
The process returns to 225, and the process is newly performed again from the i-th. As a result, the binary images of the labels within the allowable range are successively integrated.

If it is determined in step 3227 that the distance is larger than the allowable range, the flow returns to step 3215, in which the label number is increased by one (i is increased), and the processing is repeated until the processing for all the binary images of the labels is completed.

FIG. 14 shows an embodiment of the processing for calculating the maximum and minimum coordinates of the labeled binary image in step 3225 and the processing for checking in step 3227 whether the binary image is within the allowable range. FIG. For example, two labels, i.e., 2 with label 3210
An example of a binary image with a value image and a label 3290 is shown. First, upper left coordinates (l1x, l1y) and lower right coordinates (r1x, r1) of the circumscribed rectangle 3255 of the binary image to which the label 3210 is attached.
1y) and the upper left coordinates (l2x, l2y) and lower right coordinates (r2x, r2) of the circumscribed rectangle 3265 of the binary image to which the label 3290 is attached.
y) is calculated. Whether or not there is a binary image with the label 3290 within the allowable range of the binary image with the label 3210 is determined by the upper left coordinates (l1x, l1y) of the circumscribed rectangle 3255 of the binary image with the label 3210. ) And lower right coordinates (r1x, r
1y) and the circumscribed rectangle 3265 of the binary image of the label 3290
From the upper left coordinates (l2x, l2y) and lower right coordinates (r2x, r2y) of
The distance between the coordinates (l1x, r1y) of the closest circumscribed rectangle 3255 and the coordinates (r2x, l2y) of the circumscribed rectangle 3265 is calculated. If the distance 3275 in the X direction and the distance 3285 in the Y direction between the coordinates (l1x, r1y) and the coordinates (r2x, l2y) are within the allowable range, they are integrated as a binary image having the same label. The allowable range of the distance 3275 in the X direction is, for example, about 5 to 15 in the X direction, about 5 to 10 in the Y direction, or about 5 in the X direction when the moving object is a vertically long person. About 15 and the Y direction may be about 5 to 15. In any case, an appropriate setting may be made depending on how far the range is integrated.

FIG. 15 is a block diagram showing an embodiment of the internal configuration of the feature value calculation unit 3300. The feature amount calculation unit 3300 calculates a feature amount from the binary image and the grayscale image of the area integrated by the binary image integration processing unit 3200.

First, the feature amount is calculated by using the area calculated by the area calculation unit 3310 using the binary image and the aspect ratio calculation unit 332.
0, the moving distance from immediately before calculated by the immediately preceding moving distance calculating section 3330, and the starting point moving distance calculating section 3
It is the moving distance from the appearance start point calculated in 340, and the brightness distribution calculated in the brightness calculation unit 3350 using the grayscale image. These are the (i-1) -th, i-th, and i + 1
The calculation is performed in a time series by repeatedly repeating each time.

The area calculating section 3310 is provided with the small area removing section 3
The area is calculated by calculating the number of pixels based on the binary image data calculated in step 100. The aspect ratio calculation unit 3320 calculates an X projection distribution obtained by projecting in the X direction and accumulating the number of pixels for each position on the X axis, and a Y projection distribution obtained by projecting in the Y direction and accumulating the number of pixels for each position on the Y axis. The ratio is calculated as the aspect ratio. Immediately preceding moving distance calculating section 3330 and starting point moving distance calculating section 3340
Calculates the moving distance from immediately before calculated from the two centroids of the moved rectangular area and the moving distance from the appearance start point. The luminance calculator 3350 calculates the average luminance and the luminance variance of the grayscale image.

FIG. 16 is an explanatory diagram showing an embodiment of a calculation method for the immediately preceding moving distance calculating section 3330 and the starting point moving distance calculating section 3340. The immediately preceding movement distance calculation unit 3330 calculates
Rectangular area 328 integrated by binary image integration processing unit 3200
Using the center of gravity of 0, the center of gravity between the (i + 1) -th center of gravity (the center of gravity 3520 indicated by p3 is the third center of gravity) and the center of gravity of the i-th time (center of gravity 3510 indicated by p2 is the second center of gravity) The distance is calculated as the immediately preceding movement distance 3540 of the (i + 1) -th time (however, the first movement distance = 0).
The distance between the centroid (the centroid 3500 indicated by p1 and the first centroid) is calculated as the immediately preceding movement distance 3530 of the i-th time. Hereinafter, calculation is similarly performed for the (i + 2) th and subsequent times.

The starting point moving distance calculating section 3340 calculates the first point of gravity 3500 based on p1 (the first appearance position).
The moving distance 3530 to the second center of gravity, the moving distance 3550 to the third center of gravity, and the fourth and subsequent times are calculated in the same manner.

Here, when the intended detection target is a person, for example, when detecting a person moving at a normal speed that can be clearly confirmed visually in an indoor environment in the daytime, the immediately preceding movement distance is used. The threshold value may be about 20 to 30 pixels, and the aspect ratio may be about 3: 1. These threshold values may be appropriately changed off-line according to the state of movement of the posture in the monitoring target area.

FIG. 17 is a flowchart showing an embodiment of the identification procedure of the object identification section 3400. First, the area is checked in step 3600. If the area is equal to or smaller than a predetermined value, it is checked in step 3610 whether the shape of the object is vertically long using the aspect ratio. If the shape is vertically long, in step 3620 The immediately preceding moving distance is checked. If the moving distance from immediately before is equal to or less than a predetermined value, the moving distance from the starting point is checked in step 3630. If the moving distance from the appearance starting point is equal to or more than the predetermined value, the luminance is determined in step 3640. The distribution is checked, and if the luminance distribution is not uniform, it is determined in step 3650 that the person is a person.

If it is determined in step 3640 that the brightness distribution is uniform, it is checked in step 3660 whether the brightness distribution is bright. If the brightness distribution is bright, it is determined in step 3670 that the light is spot-like. Step 36
If it is determined in step 60 that the distribution is not a bright luminance distribution, it is determined in step 3680 that the object is a dark object such as a shadow.

If it is determined in step 3630 that the distance from the starting point is less than the predetermined value, it is checked in step 3700 whether the movement is a regular movement. Judge as an object that moves regularly. If it is determined in step 3700 that the movement is not regular, the determination in step 3640 is performed.

If it is determined in step 3620 that the moving distance from immediately before exceeds a predetermined value, step 372
If it is 0, it is determined that the object is a high-speed moving object.

If it is determined in step 3610 that the shape is not vertically long, and if it is determined in step 3740 that the shape is horizontally long, it is determined in step 3750 that other disturbance has occurred.
If it is determined in step 3740 that the shape is not horizontally long, the determination in step 3620 is performed.

If it is determined in step 3600 that the area is larger than the predetermined value, it is determined in step 3750 that another disturbance has occurred (assuming that there is no movement of the object of the image having an excessively large area).

FIG. 18 is a flowchart showing one embodiment of a luminance distribution identifying procedure in the luminance calculating section 3350. The brightness calculation unit 3350 includes a binary image integration processing unit 320
The luminance value is calculated by limiting (masking) the area extracted as a binary image of the rectangular area integrated with 0 (mask processing). That is, in step 3800, the average luminance and the luminance variance of the input image and / or the difference image are calculated only for the i-th label. In step 3810, the luminance variance calculated in step 3800 is checked, and if it is larger than a predetermined value, the average luminance of the image of the current label is compared with the average luminance of the image of the immediately preceding label in step 3820, and if the difference is larger than the predetermined value, In step 3830, it is determined that the i-th luminance distribution is not uniform, and if the difference is equal to or smaller than a predetermined value, step 3840
It is determined that the i-th luminance distribution is uniform. If the luminance distribution is equal to or smaller than the predetermined value in step 3810, it is determined in step 3840 that the i-th luminance distribution is uniform.

At step 3850, it is checked whether or not the processing has been completed for all the label images. If the processing has not been completed, the flow returns to step 3800; if completed, the processing proceeds to step 3860.

In step 3860, it is checked whether or not the luminance distribution is uniform for all the labels. If it is determined that the luminance distribution is uniform, it is finally determined in step 3880 that the luminance distribution is uniform. If it is determined in step 3860 that the data is not uniform, it is finally determined that the data is not uniform. In step 3860, whether or not the luminance distribution is uniform for all the labels is checked. However, not all of the labels but 80% or more of the labels may be used, or the labels may be appropriately designated off-line.

FIG. 19 is an explanatory diagram showing an embodiment in which the identification result is displayed on the display device 5000, and is an example of a person and a movement locus thereof. The display control unit 4000 controls the display of the detected intended moving object, and the object identification unit 3400
Is displayed online on the display device 5000. The object identification unit 3400 performs the (i−1) -th, i-th, i +
When the person is identified in time series at each first time, the display control unit 4
000 indicates the (i + 1) th detected person 3280 on the display device 5
000, and furthermore, the moving distance 39 from immediately before the (i−1) -th and i-th times calculated by the immediately preceding moving distance calculating unit 3330.
00 and the moving distance 39 from immediately before the i-th and i + 1-th times
10 is displayed, and furthermore, the (i + 1) th moving distance 39 from the starting point (the present time) calculated by the starting point moving distance calculating unit 3340.
20 is displayed on the display device 5000. Thus, for example, in the case of monitoring a person, the monitor can grasp the detected person and the moving state of the person in the form of an image on the display device 5000 online. The display device 500
When 0 is in a remote place, the fact that a person has been detected using a data line such as RS-232C is notified to a videophone or the like, and is displayed on a distant display device by communication via the telephone line or the like. You can also do so.

[0076]

According to the present invention, in an image surveillance apparatus for photographing an area to be monitored by a TV camera in an environment where periodic illumination fluctuations occur and monitoring the change in the monitored area with an image, an output from the TV camera is provided. Using control information, for example, shutter speed information, which can determine whether or not the influence of the periodic illumination fluctuation exists in the video information to be performed, the shutter speed information is determined for each video information, and the It is determined whether or not there is an influence of a large illumination variation, and if there is no influence, the video information captured at a certain time as the first input image to be processed and the previous input as the second input image The image information captured at the time is processed as an input image without any processing. On the other hand, if the shutter speed information indicates that the influence of the periodic illumination fluctuation exists, the image processing is performed. The image processing based on the information is omitted and the next video information is input to perform the image processing, or the video information captured at a certain time as the first input image and the second input captured at the previous time Image processing is performed by selecting image information captured when the difference between the image capturing times is the same in the illumination fluctuation cycle. If the detection of a moving object is performed by such image processing, erroneous detection due to the influence of periodic illumination fluctuation is reduced, and there is an effect that a moving object can be detected according to a request for object detection accuracy. For example, when absolute detection is not to be performed, the image processing based on the video information having the influence of the periodic illumination fluctuation is omitted and the image processing is performed with the subsequent video information having no influence of the periodic illumination fluctuation. By doing so, there is an effect that erroneous detection due to the influence of flicker or the like can be removed. In addition, when trying to detect a moving object without overlooking it, image processing using video information input at the same phase in the illumination fluctuation cycle is performed regardless of the presence or absence of periodic illumination fluctuation. To

When the shutter speed of the ITV camera used for photographing the environment to be monitored is in the automatic control mode, the shutter speed is, for example, 1/60 to 1/10 due to illuminance fluctuation.
000 s, etc., the influence of the periodic illumination fluctuation rarely occurs in all the photographed scenes being monitored, so that erroneous detection due to the influence of flicker or the like can be considerably reduced. effective.

Further, in the image processing, difference between the first input image and the second input image is made to generate difference image data, and a binarization threshold value is obtained from the density frequency distribution of the difference image data. Is automatically determined, a binary image is created, noise is removed, and the extracted objects are identified after integrating the binary images in adjacent areas. Therefore, disturbances are removed and multiple objects are separated and detected. Therefore, there is an effect that highly accurate object detection becomes possible.

When a moving object is detected, the image is displayed on the display device in real time by the display control unit, so that the observer can monitor the image of the identified moving object on-line on a display device such as a monitor. There is an effect that can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image monitoring apparatus according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a timing relationship of video information output when periodic illumination fluctuation and a shutter speed of an ITV camera change.

FIG. 3 is a block diagram showing an embodiment of an internal configuration of a camera control information output unit in the image monitoring device according to the present invention.

FIG. 4 is a flowchart illustrating an embodiment of a shutter speed calculation procedure calculated by a shutter speed output unit in the image monitoring device according to the present invention.

FIG. 5 is an explanatory diagram when a shutter speed value calculated from an output signal of a shooting scene in the image monitoring device according to the present invention is displayed on a display device.

FIG. 6 is a block diagram illustrating an embodiment of an internal configuration of a processing method determination unit in the image monitoring device according to the present invention.

FIG. 7 is a flowchart illustrating an embodiment of a determining procedure of a processing method determining unit in the image monitoring device according to the present invention.

FIG. 8 is a block diagram showing an embodiment of an internal configuration of an image input unit in the image monitoring device according to the present invention.

FIG. 9 is a block diagram showing one embodiment of an internal configuration of an object extracting unit in the image monitoring device according to the present invention.

FIG. 10 is an explanatory diagram showing an embodiment for calculating a binarization threshold in a binarization threshold calculator in the image monitoring device according to the present invention.

FIG. 11 is a block diagram showing an embodiment of an internal configuration of an object detection unit in the image monitoring device according to the present invention.

FIG. 12 is an explanatory diagram illustrating an embodiment when a binary image integration processing unit in the image monitoring apparatus according to the present invention generates a rectangle as an object from a plurality of binary images.

FIG. 13 is a flowchart showing an embodiment of a procedure in which the binary image integration processing unit in the image monitoring apparatus according to the present invention integrates a plurality of binary images into a rectangle as an object.

FIG. 14 illustrates an embodiment in which the image monitoring apparatus according to the present invention calculates maximum coordinates and minimum coordinates of a binary image and checks whether or not the distance between the binary images is within an allowable range. FIG.

FIG. 15 is a block diagram illustrating an embodiment of an internal configuration of a feature amount calculation unit in the image monitoring device according to the present invention.

FIG. 16 is an explanatory diagram showing one embodiment of distance calculation in the immediately preceding moving distance calculating unit and the starting point moving distance calculating unit in the image monitoring device according to the present invention.

FIG. 17 is a flowchart illustrating an embodiment of an identification procedure of an object identification unit in the image monitoring device according to the present invention.

FIG. 18 is a flowchart illustrating an embodiment of a luminance distribution identification procedure in the luminance calculation unit of the image monitoring device according to the present invention.

FIG. 19 is an explanatory diagram showing an embodiment in which the identification result is displayed on the display device in the image monitoring device according to the present invention.

[Explanation of symbols]

100: ITV camera, 400: camera control information output unit, 600: processing method determination unit, 800: image input unit, 1
000: difference image creation unit, 2000: object extraction unit, 30
00: object detection unit, 4000: display control unit, 3200:
Display device.

Claims (10)

[Claims] 1. A monitoring target area is photographed by a TV camera in an environment where illumination fluctuations occur, video information of an object in the monitoring target area is input, and a difference image between two input images based on two preceding and succeeding video informations. An image monitoring device that creates data and monitors the presence or absence of a moving object in the monitoring target area, wherein the control information for determining whether or not the video information from the TV camera is affected by the illumination variation is acquired. An information output unit, a processing method determining unit that determines a processing method based on the control information output from the illumination variation influence information output unit, and a difference that changes a method of creating difference image data according to a determination result of the processing method determining unit. An image monitoring apparatus, comprising: an image creation unit. 2. The method according to claim 1, wherein said difference image creating means is a processing target when there is no influence of illumination fluctuation.
The difference image data which is the difference between the two images is created by using the input image and the second input image captured immediately before, and the creation of the difference image data is omitted when there is an influence of illumination fluctuation. Image monitoring device. 3. The method according to claim 1, wherein the difference image creating means is a processing target when there is no influence of illumination fluctuation.
The difference image data that is the difference between the two images is created using the input image of the first and the second input images captured immediately before, and the creation of the difference image data is omitted when there is an influence of illumination fluctuation, and the first image is omitted. An image monitoring apparatus characterized in that the next image information is fetched at the same phase on the illumination fluctuation cycle in which the image information of the input image is fetched, and a difference image data creation process is performed between the fetched image information and the input image. 4. The apparatus according to claim 1, wherein said difference image creating means is a processing target when there is no influence of illumination fluctuation.
The difference image data which is the difference of both images is created using the input image of the above and the second input image just before the input image, and when there is an influence of the illumination variation, the input image previously captured at the same phase on the illumination variation cycle And generating the difference image data by selecting. 5. The difference image creation means according to claim 1, wherein said difference image creation means creates the difference image data by selecting an input image of video information taken in the same phase on an illumination variation cycle when there is an influence of illumination variation. Or an image monitoring apparatus characterized in that creation of the difference image data is omitted. 6. An image monitoring apparatus according to claim 1, wherein said illumination fluctuation influence information output means uses shutter speed information of said TV camera. 7. An image monitoring apparatus according to claim 1, wherein said processing method determining means changes a processing method for generating differential image data according to shutter speed information of a TV camera. 8. An image monitoring apparatus according to claim 1, further comprising a memory for storing video information of at least two photographing scenes generated by photographing with said TV camera. 9. A video camera which has a function of preventing flickering of video information, captures an image of an area to be monitored in an environment where illumination fluctuations exist, and inputs video information of an object in the area to be monitored. An image monitoring apparatus that creates difference image data of two input images based on two pieces of video information and monitors the presence or absence of a moving object in the monitoring target area, wherein the video information from the TV camera is not affected by illumination fluctuation. An illumination variation influence information output unit that obtains shutter speed information as control information for judging, and a processing method determination unit that determines an image processing method based on the shutter speed information output from the illumination variation influence information output unit. A difference image creating means for changing a method of creating difference image data in accordance with a result of the judgment by the processing method judging means. Image monitoring device. 10. An image monitoring apparatus according to claim 6, wherein said TV camera has an automatic shutter speed control function.