JP2987353B2 - Video surveillance method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video surveillance method for monitoring a state change using a video camera.

[0002]

2. Description of the Related Art In a conventional video surveillance system using a video camera, for example, the following monitoring methods (1) to (3) have been performed. (1) An observer monitors an image captured by a video camera on a monitor television. (2) Record by a recording device capable of recording for a long time, and play back and confirm as necessary. (3) Install an infrared sensor or the like in advance at the monitoring location,
When the sensor detects a change in the state, the recording device is activated to perform recording.

[0003]

However, the conventional monitoring method has the following problems. For example,
In the method of (1), it is necessary for the observer to constantly monitor,
Only applicable for large-scale monitoring systems. In addition, since there is a risk of oversight, it is necessary to use it in combination with the methods (2) and (3). In the method (2), an alarm cannot be output immediately when an abnormality occurs. The method (3) cannot be applied to a place where it is difficult to install an infrared sensor or the like. SUMMARY OF THE INVENTION The present invention solves the problems of the conventional technology, and provides an image monitoring method that does not require a sensor such as an infrared sensor and can immediately detect occurrence of an abnormality.

[0004]

According to a first aspect of the present invention, there is provided a video camera for photographing a monitored area, and display of screen information of an image photographed by the video camera. The following processing is performed in the video monitoring method of the video monitoring system that includes a display device that performs the following processing and monitors whether there is an abnormality based on the change in the screen information.
First, based on first and second points designated on the display screen of the display device, a monitoring line for setting a plurality of pixels forming a straight line connecting these first and second points as a monitoring line In the setting process, brightness data of a plurality of pixels constituting the monitoring line in the screen information captured by the video camera is collected for a certain period of time, and the pixels whose variation in brightness during the certain period exceeds a certain value are collected. A monitoring point extraction process is performed for extracting a monitoring point composed of only pixels whose variation width does not exceed the fixed value by deleting the monitoring point from the monitoring line.

[0005] Thereafter, input processing for sequentially inputting the screen information captured by the video camera, and the monitoring based on the latest N (where N is a plurality) screen information input from the video camera. Average brightness calculation processing for calculating an average value of the brightness of each pixel constituting the point; and for each pixel constituting the monitoring point, the brightness of the pixel of the monitoring point input from the video camera and the average A change number counting process of counting the number of changed pixels whose difference from the average value calculated in the brightness calculation process exceeds the first threshold, and a ratio of the changed pixel number to the total number of pixels constituting the monitoring point. A monitor information display process for calculating a change rate based on the change rate and displaying monitor information corresponding to the change rate on the display device together with the screen information;
When the threshold value is exceeded, output of an alarm or recording of the screen information, or alarm recording processing of outputting the alarm and recording the screen information is sequentially and repeatedly performed.

According to a second aspect, the following processing is performed in the video monitoring method of the video monitoring system similar to the first aspect. First, a monitoring line setting process and a monitoring point extraction process similar to those of the first invention are performed. Then, based on the input processing of sequentially inputting the screen information captured by the video camera, a plurality of pieces of the screen information input from the video camera, an average value of luminance of each pixel constituting the monitoring point , An average luminance calculation process for calculating the constant period, a change number counting process similar to the first invention, a monitoring information display process, and an alarm recording process are sequentially and repeatedly performed.

In a third aspect based on the first or third aspect, the setting of the first and second threshold values can be changed at any time. According to the first and third aspects of the present invention, since the video monitoring method is configured as described above, the following operation is performed. When the first and second points are designated on the display screen, a monitoring line setting process for setting a monitoring line composed of a plurality of pixels is performed based on the designation, and subsequently, a monitoring image is captured by a video camera. Based on the screen information, luminance data for each screen of the pixels constituting the monitoring line is collected for a certain period. By the monitoring point extraction processing, pixels in which the fluctuation width of the luminance data exceeds a certain value are deleted, and monitoring points whose fluctuation width is within a certain value are extracted. Thereafter, the screen information of the video captured by the video camera is sequentially input by the input process, and the average value of the luminance of the pixels at each monitoring point is calculated based on the latest N pieces of screen information.
The number of changed pixels in which the difference between the average value and the luminance of the pixel at the monitoring point given from the video camera exceeds the first threshold value is counted. Further, a change rate, which is a ratio of the number of changed pixels to the total number of pixels at the monitoring point, is calculated, and monitoring information corresponding to the changed rate is displayed on the display device together with the screen information. Further, when the rate of change exceeds the second threshold, an alarm is output or screen information is recorded, or an alarm is output and screen information is recorded.

According to the second and third aspects, the following operation is performed. When the first and second points are designated on the display screen, a monitoring line setting process for setting a monitoring line composed of a plurality of pixels is performed based on the designation, and subsequently, a monitoring image is captured by a video camera. Based on the screen information, luminance data for each screen of the pixels constituting the monitoring line is collected for a certain period. By the monitoring point extraction processing, pixels in which the fluctuation width of the luminance data exceeds a certain value are deleted, and monitoring points whose fluctuation width is within a certain value are extracted. Thereafter, the screen information of the video captured by the video camera is sequentially input by the input process, and an average brightness calculation process of calculating an average value of the brightness of the plurality of the screen information is performed at a constant cycle. The number of changed pixels in which the difference between the average value and the luminance of the pixel at the monitoring point given from the video camera exceeds the first threshold value is counted. Further, a change rate, which is a ratio of the number of changed pixels to the total number of pixels at the monitoring point, is calculated, and monitoring information corresponding to the changed rate is displayed on the display device together with the screen information. Further, when the rate of change exceeds the second threshold, an alarm is output or screen information is recorded, or an alarm is output and screen information is recorded.

[0009]

FIG. 2 is a block diagram showing an example of a video monitoring system to which a video monitoring method according to an embodiment of the present invention is applied. This video surveillance system includes a video camera 11 for photographing a monitored area. Video camera 11
Is, for example, NT color image information of 30 screens per second.
An analog / digital converter (hereinafter, referred to as “A / D”) 12 is connected to an output side of the video camera 11 by converting the video signal into a video signal of a National Television System Committee (SC) system. I have. A / D
Reference numeral 12 decomposes an NTSC video signal into analog signals corresponding to the three primary colors of red (R), green (G), and blue (B) light, and converts these analog signals into pixels constituting each screen. It has a function of sampling each time and converting it into R, G, B digital signals. A / D1
The image frame memory 13 is connected to the output side of the second. The image frame memory 13 is a memory for temporarily storing R, G, and B digital signal data corresponding to a plurality of pixels constituting one screen. For example, 600 dots × 400 lines × 3 colors (= 720 kbytes) Storage capacity. On the output side of the image frame memory 13,
A digital / analog converter (hereinafter referred to as “D / A”) 14 is connected. The D / A 14 sequentially reads out the data of the R, G, and B digital signals stored in the image frame memory 13 at regular intervals, and
The signal is converted into G and B analog signals. D / A1
The display unit 15 is connected to the output side of the display unit 4 so that the screen information stored in the image frame memory 13 is displayed.

A moving image control unit 16 is connected to the image frame memory 13. The moving image control unit 16 has a function of reading data in an arbitrary storage area of the image frame memory 13 and rewriting data in an arbitrary storage area of the image frame memory 13. Video controller 16
Are connected via a common bus 17 to a central processing unit (hereinafter referred to as “CP”).
U ”). The CPU 18 controls the entire video monitoring system and performs a video monitoring process. The common bus 17 includes an input unit 19 including a keyboard, a mouse, and the like, a read-only memory (hereinafter, referred to as a “ROM”) 20 in which programs and constants for controlling and processing the CPU 18 are stored, and control and processing. A random access memory (hereinafter, referred to as “RAM”) 21 for storing data for use with the camera, and a data file unit 22 for storing and storing screen information are connected.

FIG. 3 is a diagram showing a display example on the display unit 15 in FIG. As shown in FIG. 3, the display unit 15
A monitoring area display area 15a for displaying an image of the monitoring area captured by the video camera 11, a menu display area 15 for displaying an operation menu of the video monitoring system.
b, monitoring information display area 1 for displaying monitoring information
5c and the like. FIG. 1 is a schematic flowchart of a video monitoring method according to an embodiment of the present invention, and FIGS. 4 and 5 are detailed flowcharts of steps S10 and S20 in FIG. 1, respectively. Hereinafter, the video monitoring method of FIG. 1 will be described with reference to FIGS. First, when the power supply of the video monitoring system shown in FIG. 2 is turned on, the video signal of the monitoring area captured by the video camera 11 is transmitted to the A / D 12.
Is converted into a digital signal by the
Is stored in On the other hand, data for displaying the operation menu is provided from the CPU 18 to the moving image control unit 16 via the common bus 17, and the menu data is written from the moving image control unit 16 to a predetermined storage area of the moving image frame memory 13. The data written in the moving image frame memory 13 is repeatedly read at a constant cycle by the D / A 14, converted into an analog signal, and output to the display unit 15. As a result, the display section 15 displays the alert area display area 15a and the menu display area 15b as shown in FIG.

Next, the operator uses the mouse of the input unit 19, for example, to display the menu display area 15b of the display unit 15.
Select "Monitoring location setting" in step S
The monitoring line setting process of 10 is started. In step S11 in FIG. 4, the operator inputs the start point 1a of the monitoring line 1 on the alert area display area 15a using a mouse. Subsequently, in step S12, the end point 1b of the monitoring line 1 is input. Accordingly, the process proceeds to step S13, where the coordinates of a plurality of pixels constituting the monitoring line 1 connecting the input start point 1a and end point 1b with a straight line are calculated as monitoring points, and the process proceeds to step S14. In step S14, in order to limit the number of monitoring points to an appropriate number (for example, 30 points), a thinning process of excluding the monitoring points at a fixed rate is performed. This is a measure for not detecting all the pixels on the monitoring line 1 for detecting an abnormality, and a measure for reliably processing each screen information input every 1/30 second. After step S14, the process proceeds to step S15.

In step S15, the operator selects "recording range setting" in the menu display area 15b, and performs an operation for setting a recording range of a monitoring screen to be recorded when an alarm is output upon detection of an abnormality. I do. In this step S15, the operator designates, for example, the upper left point 1c and the lower right point 1d of the recording range with the mouse, so that the area of the rectangle 1X having these two points 1c and 1d as diagonal lines is set. Is set as After step S15, the process proceeds to step S16. Step S
At 16, the operator selects “alarm threshold setting” in the menu display area 15b and performs an operation for setting the amount of change in the screen corresponding to the output of the alarm as the threshold. In step S16, the operator inputs, for example, a first threshold value indicating how much the luminance of a pixel monitored as a monitoring point changes with respect to the average luminance of the monitoring point to detect the pixel as a changed pixel. A numerical value is input from the keyboard of the unit 19. For example, when the luminance of each pixel is represented by 8-bit data, the luminance has 256 levels of gradation. If the first threshold value is set to 10, the average luminance (for example, level 100) becomes 10 levels. When the above change occurs, the monitoring point is determined to be a changed pixel.

After step S16, the process proceeds to step S17, where the operator inputs a second threshold value for determining a change rate. The second threshold is used to output an alarm when the ratio of the number of pixels determined to be changed pixels to the total number of pixels at the monitoring points forming the monitoring line 1 exceeds the second threshold. This is a value used as a criterion. For example, if the second threshold is set to 30%, an alarm is issued when the brightness of the monitoring points of 30% or more on the monitoring line 1 has changed from the average brightness by the first threshold (= 10 levels) or more. Will be output. Upon completion of step S17, the monitoring line setting process ends, and the process proceeds to the monitoring point extraction process of step S20 in FIG. In the monitoring point extraction process, screen information is input in step S21 in FIG. In this step S21, among the data photographed by the video camera 11 and temporarily stored in the moving image frame memory 13, the remaining R, G, B data of the 30 monitoring points thinned out in step S14 are extracted. Is read. After step S21, the process proceeds to step S22,
For each monitoring point, the luminance Y is calculated based on the R, G, and B data, for example, by the following equation. Y = 0.3R + 0.59G + 0.11B After step S22, the process proceeds to step S23.

In step S23, the maximum value and the minimum value of the luminance are updated for each monitoring point in order to investigate the fluctuation range of the luminance. That is, the brightness at each monitoring point of the input screen is compared with the contents of the memory holding the maximum value and the memory holding the minimum value prepared for each monitoring point, and a value larger than the past maximum value is compared. ,
Alternatively, when a value smaller than the past minimum value is input, the maximum value or the minimum value in the memory is updated by the input value. After step S23, step S
Proceeding to S24, it is determined whether or not the processing of steps S21 to S23 has been performed for a predetermined number of screens (for example, 10 screens). Then, the process returns to step S21 and the processes of steps S21 to S23 are repeated until the predetermined number of screens is reached. Upon completion of the processing for the predetermined number of screens, the flow proceeds to step S25. In step S25, the difference between the maximum value and the minimum value is calculated for each monitoring point to calculate the luminance variation width, and the process proceeds to step S26.

In step S26, step S25
The monitoring point whose luminance variation width calculated in Step S1 exceeds the first threshold set in Step S16 is deleted. In other words, points on the contour line of the screen have large fluctuations in luminance due to noise in the video signal and small fluctuations of the subject and the video camera 11 and are not suitable as monitoring points. Therefore, such points are deleted from the monitoring points. It is a measure for. Upon completion of step S26, the monitoring point extraction process ends, a valid monitoring point is extracted, and the process proceeds to the screen information input process in step S30 of FIG. In step S30, the R, G, and B data of the effective monitoring points extracted in step S20 are read from the screen information for each screen temporarily stored in the moving image frame memory 13, and the R, G, and B data are read out. The brightness of each monitoring point is calculated based on the above data. After step S30, step S
Proceed to 40.

In step S40, based on the latest N screen information (for example, 10 screens) input from the video camera 11, the average brightness of each monitoring point is calculated by a temporal moving average. Calculation processing is performed. This average luminance calculation process is a process for preventing, for example, a slow change in the environment in which the position of the shadow gradually changes due to the movement of the sun from being detected as abnormal. After step S40, the process proceeds to the change number counting process of step S50. In step S50, step S3
0 and the brightness of the monitoring point calculated in step S40
The difference from the average value of the brightness of the monitoring points calculated in step (1) is calculated for each monitoring point as the amount of change in brightness. Then, the number of monitoring points where the calculated change in luminance exceeds the first threshold set in step S16 is counted, and the process proceeds to the monitoring information display processing in step S60. In step S60, the number of monitoring points exceeding the first threshold counted in step S60 and the number of monitoring points in step S20
Is calculated as a ratio with the number of valid monitoring points extracted in step (1). The calculated change rate is displayed, for example, in a bar graph as shown in FIG. 3 in the monitoring information display area 15c of the display unit 15, and the process proceeds to step S70.

In step S70, based on the second threshold set in step S17, it is determined whether or not the rate of change calculated in step S60 is greater than the second threshold. If the rate of change is greater than or equal to the second threshold, the process proceeds to step S80, and if the rate of change is less than the second threshold, the process proceeds to step S90. In step S80,
For example, while outputting an alarm such as ringing a bell, the monitor screen information in the recording range specified in step S15 is read out from the moving image frame memory 13, and the date,
An alarm recording process is performed to record in the data file unit 22 together with search information such as time, and the process proceeds to step S90. In step S90, it is checked whether or not an input for stopping monitoring has been made. If no stop input has been made, the process returns to step S30 assuming that monitoring is to be continued. If a stop input has been made, this monitoring process ends.

As described above, the video surveillance method according to the present embodiment has the following advantages (i) to (v). (I) In step S14, a thinning process is performed to leave pixels necessary for abnormality detection, and the minimum necessary number of pixels is monitored on the monitoring line. A reliable monitoring process can be performed every time. (Ii) Since the recording range of the monitor screen is specified in step S15, unnecessary screen information can be deleted and only the necessary portion of the monitor screen can be recorded. It is possible to record time. (Iii) In step S40, a temporal moving average of the luminance of each monitoring point is calculated, so that it is possible to prevent a slow environmental change due to the movement of the sun or the like from being detected as abnormal. (Iv) In step S60, the rate of change is displayed in the alarm information display area 15c of the display 15, so that by comparing with the screen information displayed in the warning area display area 15a of the same display 15, It is possible to determine the validity of this change rate. This makes it possible to take measures such as changing the first and second threshold values if it is determined that the threshold value is not appropriate, and prevent an inappropriate alarm output from occurring. (V) In step S80, when a change exceeding the threshold is detected, an alarm is output and the screen information of the monitoring screen is recorded.
5 does not need to be monitored, and the monitor screen at the time of abnormality can be searched later.

The present invention is not limited to the above embodiment, but can be variously modified. For example, there are the following modifications (a) to (f). (A) The video camera 11 outputs a color video signal of the NTSC system. However, the video camera 11 is not limited to the NTSC system, and may use a color video signal of another system. Further, the present invention can be applied to a monochrome video signal instead of a color video signal. In the case of a single color, the process of calculating the luminance from the data of R, G, and B becomes unnecessary. (B) In step S10, a case has been described in which one monitoring line 1 is set. However, as shown in FIG. 3, a plurality of monitoring lines 1, 2, 3,... Can be set. However, as the number of monitoring lines increases, the number of monitoring points also increases.
It is necessary to use PU18 or the like. (C) In the setting of the monitoring line in step S10, the starting point 1a and the ending point 1b of the monitoring line 1 are set using the mouse of the input unit 19, but the setting is not limited to the mouse, and the cursor key of the keyboard is used. May be.

(D) In step S10, first and second threshold values for abnormality detection are input prior to the monitoring process.
And the second threshold value may be changed. This makes it possible to output an appropriate abnormality alarm. (E) In step S40, the moving average of the brightness is calculated for each monitoring point of the latest N screens. The average luminance may be calculated based on the data. This makes it possible to reduce the processing load for calculating the average luminance. (F) In the video surveillance system of FIG.
Although only one camera 1 is provided, it is also possible to provide a plurality of video cameras 11 and switch the images of the video cameras 11 to monitor. In that case, each video camera 1
What is necessary is just to set the monitoring line 1 etc. for every one.

[0022]

As described in detail above, according to the first invention, the following effects (a) to (d) are obtained. (A) a change number counting process for counting the number of changed pixels in which the difference between the luminance of the pixel at the monitoring point given from the video camera and the average value calculated in the average value calculation process exceeds a first threshold value; Monitoring information display processing for calculating a rate of change based on the ratio of the number of changed pixels to the total number of pixels forming the monitoring point; and outputting an alarm or displaying screen information when the rate of change exceeds a second threshold. And an alarm recording process for recording the information. This eliminates the need for a sensor such as an infrared sensor or the like, and makes it possible to immediately detect the occurrence of an abnormality based on the screen information captured by the video camera. (A) Since a monitoring point extraction process is performed to delete pixels from a monitoring line for which the fluctuation width of the luminance exceeds a certain value during a certain period, there is no possibility that an erroneous alarm is output due to a shaking of the outline of the screen. (C) When an abnormality is detected, the screen information captured by the video camera can be recorded simultaneously with the output of the alarm, so that the observer does not need to constantly monitor the display device. Can be reproduced.

(D) Since the average brightness calculation processing for calculating the average brightness value based on the latest screen information of the N screens is performed for each monitoring point, erroneous changes in the screen such as the movement of the sun are mistaken. There is no possibility of outputting a warning. According to the second invention, in addition to the effects (A) to (C) similar to those of the first invention, the following effect (E) is obtained. (E) Since the average brightness calculation process of calculating the average brightness based on the brightness data of the monitoring points on a plurality of screens is performed in a fixed cycle, the load of the process for calculating the average brightness can be reduced, and the movement of the sun, etc. There is no possibility that an erroneous alarm is output for a gradual screen change. According to the third invention, in addition to the effects (A) to (E) similar to those of the first and second inventions, the following effect (F) is obtained. (F) Since the first and second thresholds can be changed at any time, it is determined whether or not the setting of these thresholds is appropriate based on the monitoring information displayed on the display device. In such a case, these thresholds can be changed. For this reason, it becomes possible to set an appropriate threshold, and it is possible to prevent inappropriate alarm output.

[Brief description of the drawings]

FIG. 1 is a schematic flowchart of a video monitoring method according to an embodiment of the present invention.

FIG. 2 is a configuration diagram illustrating an example of a video monitoring system to which a video monitoring method according to an embodiment of the present invention is applied.

FIG. 3 is a diagram showing a display example on a display unit 15 in FIG. 2;

FIG. 4 is a detailed flowchart of step S10 in FIG.

FIG. 5 is a detailed flowchart of step S20 in FIG. 1;

[Explanation of symbols]

 11 video camera 15 display unit 18 CPU 19 input unit 22 data file unit S10 step of monitoring line setting processing S20 step of monitoring point extraction processing S30 step of input processing S40 step of average luminance calculation processing S50 step of change count processing S60 monitoring Step of information display processing S80 Step of alarm recording processing

Claims (3)

(57) [Claims] 1. A video monitor, comprising: a video camera that captures an image of a monitoring area; and a display device that displays screen information of a video captured by the video camera, and monitors whether there is an abnormality based on a change in the screen information. In the system, based on the first and second points specified on the display screen of the display device, a monitor that sets a plurality of pixels forming a straight line connecting the first and second points as a monitor line Line setting processing, collecting the luminance data of a plurality of pixels constituting the monitoring line in the screen information captured by the video camera for a certain period of time, and selecting a pixel whose fluctuation range of the luminance during the certain period exceeds a certain value. After performing a monitoring point extraction process of deleting a monitoring point composed of only pixels whose variation width does not exceed the fixed value by deleting the monitoring line, the video camera An input process of sequentially inputting the captured screen information; and a luminance of each pixel constituting the monitoring point based on the latest N (where N is a plurality) screen information input from the video camera. Average luminance calculation processing for calculating the average value of the pixels, and for each pixel constituting the monitoring point, the luminance of the pixel of the monitoring point input from the video camera and the luminance calculated in the average luminance calculation processing A change number counting process for counting the number of changed pixels whose difference from the average value exceeds a first threshold; and calculating a change rate based on a ratio of the number of changed pixels to the total number of pixels constituting the monitoring point. Monitoring information display processing for displaying, on the display device, monitoring information corresponding to the change rate together with the screen information; and outputting an alarm or the screen information when the change rate exceeds a second threshold value. A video surveillance method characterized by sequentially and repeatedly performing recording of information or alarm recording processing for outputting the alarm and recording the screen information. 2. A video surveillance system comprising: a video camera for photographing an area to be monitored; and a display device for displaying screen information of a video photographed by the video camera, and monitoring presence or absence of an abnormality based on a change in the screen information. In the system, based on the first and second points specified on the display screen of the display device, a monitor that sets a plurality of pixels forming a straight line connecting the first and second points as a monitor line Line setting processing, collecting the luminance data of a plurality of pixels constituting the monitoring line in the screen information captured by the video camera for a certain period of time, and selecting a pixel whose fluctuation range of the luminance during the certain period exceeds a certain value. After performing a monitoring point extraction process of deleting a monitoring point composed of only pixels whose variation width does not exceed the fixed value by deleting the monitoring line, the video camera An input process of sequentially inputting the captured screen information, and, based on the plurality of pieces of screen information input from the video camera, an average value of luminance of each pixel constituting the monitoring point, at a predetermined period. Average brightness calculation processing to calculate, for each pixel constituting the monitoring point, the brightness of the pixel of the monitoring point given from the video camera, and the average value of the brightness calculated in the average brightness calculation processing A change number counting process for counting the number of changed pixels in which the difference exceeds a first threshold value; and calculating a change rate based on a ratio of the number of changed pixels to the total number of pixels constituting the monitoring point. Monitoring information display processing for displaying, on the display device, monitoring information corresponding to the rate together with the screen information; and outputting an alarm or writing the screen information when the rate of change exceeds a second threshold value. Recording, or an alarm recording process of outputting the alarm and recording the screen information, sequentially and repeatedly. 3. The video surveillance method according to claim 1, wherein the first and second threshold values can be changed at any time.