JP2008288870A - Video image monitoring system and video image monitoring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accelerate the discovery of an abnormal phenomenon by reducing the load of an observer. <P>SOLUTION: This video image monitoring method as one aspect of the invention extracts one or more features representing features of video images from respective video images picked up by a plurality of monitoring cameras, estimates respective abnormal degrees to a monitoring event represented by each of the video images by using a database storing a plurality of examples including the one or more features and the abnormal degrees of the monitoring event, and the features extracted from the respective video images, selectively outputs the video images of the respective monitoring cameras to the prescribed number of monitors, controls video image selection on the basis of the estimated abnormal degrees, acquires information representing the behavior of the observer to the video image displayed on each of the monitors, calculates an abnormal degree to the monitoring event represented by the video image displayed on each of the monitors from the acquired behavior of the observer, and registers each example including the features of the video images displayed on each of the monitors and the calculated abnormal degree in the database. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video monitoring system and a video monitoring method.

Due to security issues in recent years, the installation of surveillance cameras has increased, and the penetration of IP (Internet Protocol) technology has made it easier to transmit video over longer distances. The number is increasing. Along with this, the burden on humans who monitor in the monitoring room has increased, and it has become difficult to keep paying attention to all camera images.
JP 2006-79272 A

  In the conventional monitoring system, the number of monitors installed in the monitoring room is small with respect to the many cameras installed, and the monitoring staff switches the video at any time or monitors the video that changes at regular intervals. It was. For this reason, it is necessary for the monitor to always watch the monitor carefully, and if the number of cameras becomes very large, it will be difficult to overlook important events, or it will be difficult to keep watching concentratedly. Met. Conventionally, a monitoring support system that detects anomalies by image processing and notifies the detected anomalies has been used, but this system can only detect specific events defined in advance.

  The present invention provides a video surveillance system and a video surveillance method that can reduce the burden on a supervisor and accelerate the discovery of abnormal events.

The video surveillance system as one aspect of the present invention includes:
A feature quantity extraction unit that extracts one or more feature quantities representing features of the video from each video captured by a plurality of surveillance cameras;
A database storing a plurality of cases including one or more feature quantities and an abnormality degree of a monitoring event;
Using the feature amount extracted from each video and the database, an abnormality degree estimation unit that estimates the degree of abnormality for the monitoring event represented by each video,
A video switching unit that selectively outputs video of each of the monitoring cameras to a predetermined number of monitors;
A management control unit that controls video selection by the video switching unit based on each estimated degree of abnormality;
An information acquisition unit for acquiring information representing the behavior of the monitor for the video displayed on each of the monitors;
An abnormality degree calculation unit for calculating an abnormality degree with respect to a monitoring event represented by a video displayed on each monitor from the action of the acquired observer;
A database updating unit that registers each case including the feature amount of the video displayed on each monitor and the degree of abnormality calculated by the degree of abnormality calculating unit in the database.

An image monitoring method as one aspect of the present invention includes:
A feature amount extraction step of extracting one or more feature amounts representing features of the video from each video captured by a plurality of surveillance cameras;
Abnormality with respect to the monitoring event represented by each video using a database storing a plurality of cases including one or more feature values and the degree of abnormality of the monitoring event, and feature values extracted from each video A degree of abnormality estimation step for estimating each degree;
A video switching step of selectively outputting video of each of the monitoring cameras to a predetermined number of monitors;
A management control step for controlling video selection by the video switching step based on each estimated abnormality degree;
An information acquisition step of acquiring information representing the behavior of the observer with respect to the video displayed on each of the monitors;
An abnormality degree calculating step for calculating an abnormality degree with respect to the monitoring event represented by the video displayed on each monitor from the acquired action of the observer;
A database update step of registering each case including the feature amount of the video displayed on each monitor and the calculated degree of abnormality in the database.

  According to the present invention, it is possible to reduce the burden on the monitoring staff and speed up the detection of abnormal events.

  FIG. 1 shows the configuration of a video surveillance system as an embodiment of the present invention.

  The plurality of monitoring cameras 1 capture an image of a monitoring target such as a person or an object, and output the captured camera video.

  The video switching device (video switching unit) 2 selectively outputs the camera video captured by each monitoring camera 1 to a predetermined number of monitors 3 for video display under the control of the management control device 7. It is assumed that the number of monitors 3 is smaller than the number of surveillance cameras 1.

  The image processing device (feature amount extraction unit) 4 extracts a feature amount representing the feature of the behavior to be monitored from the camera video imaged by each monitoring camera 1. The extracted feature value data is transmitted to the degree-of-abnormality estimation & calculation unit 5.

  The feature amount to be extracted is, for example, the amount of change in angle that is the sum of the residence time of a person or object, the number of turns, the frequency of sudden acceleration / deceleration, the number of stops, the walking time, the running time, and the angle change at the time of turning There are sums, whether they are prone, whether they are prone, etc.

  In the correlation data storage unit (database) 6, a plurality of correlation data (examples in the past) including an imaging time, a camera ID, a multidimensional feature amount, an abnormality degree of a monitoring event, a supervisor behavior feature amount, and the like are registered. . An example of the contents of the correlation data storage unit 6 is shown in FIG. “Time” is, for example, a representative time of a frame period from which a feature amount is extracted. “A”, “b”, “c”... Are types of feature values. The “abnormality” and the “monitorer action feature” will be described later.

The degree-of-abnormality estimation & calculation unit 5 compares the feature amount data with the past cases (correlation data) in the correlation data storage unit 6 for each camera image, and estimates the degree of abnormality of the event reflected in each camera image. To do. Specifically, a case (correlation data) having a feature amount most similar to the feature amount extracted from the camera video is detected, and the degree of abnormality included in the detected case (correlation data) is acquired as the estimated degree of abnormality. . The degree of abnormality estimation unit may perform the degree of abnormality estimation according to an algorithm that improves the estimation accuracy as time is spent on the calculation and that can acquire the degree of abnormality estimated at that point in time during the calculation. The estimated abnormality degree estimated for each camera image is transmitted to the management control device 7. The abnormality level estimation & calculation unit 5 includes, for example, functions of an abnormality level estimation unit, an abnormality level calculation unit, a list maintenance unit, and a database update unit.

  The management control device (management control unit, information acquisition unit) 7 controls the video switching device 2 so that a video with a high estimated abnormality level is displayed more preferentially according to a display priority order determined according to the estimated abnormality level. To do. In addition, for a video whose estimated abnormality level is higher than a preset threshold value, the operator is notified of the possibility of abnormality through the operation terminal 8.

  The monitor monitors the monitor video displayed on each monitor 3 according to the priority order, and checks whether there is any abnormality. The monitor checks the video displayed on the monitor 3 and inputs a determination result indicating that there is no abnormality when it is not necessary to continue watching. In addition, when the monitoring person is notified of the possibility of abnormality from the management control device 7 via the operation terminal 8, the determination result for this notification, for example, the notification need not be notified in the future (there was no abnormality) The determination result indicating that is input. The monitor can perform operations such as enlargement of the video displayed on the monitor 3 from the operation terminal 8, and the contents of this operation are reflected in the video displayed on the monitor 3 via the management control device 7. . The determination result and the operation content input by the monitor are passed to the abnormality degree estimation & calculation unit 5 via the management control device 7.

  The image processing device (measurement unit, second feature amount extraction unit) 10 detects the movement of the observer's line of sight based on the video obtained by the line-of-sight camera 9 that images the observer, and from the detected line-of-sight information, A feature amount (monitorer action feature amount) representing a feature of the monitor's behavior, such as which monitor the monitor has seen from among the plurality of monitors 3 or how long the monitor has been watched, is extracted.

  The degree-of-abnormality estimation & calculation unit 5 is represented by a video displayed on the monitor based on the determination result and the operation content passed from the management control device 7 and the supervisor behavior feature amount extracted by the image processing device 10. The degree of abnormality of the monitoring event is calculated, and the calculated degree of abnormality is stored in the correlation data storage unit 6 as one piece of correlation data (example) in association with the time, camera ID, feature quantity, supervisor action feature quantity, and the like. By sequentially storing the correlation data in the correlation data storage unit 6, it is expected that the accuracy of estimating the degree of abnormality is improved.

  FIG. 2 is a flowchart showing a flow of processing performed by the video monitoring system of FIG. The processing at each stage may be realized by causing a computer to execute a program including an instruction code describing the processing at each stage shown in the flowchart. The program may be stored in a computer-readable recording medium and read and executed by a computer.

  In FIG. 2, first, a person or object to be monitored is specified by image processing in the image processing device 4 from the video imaged by each monitoring camera 1, and the position, speed, direction, etc. of the specified person or object are calculated. (S1). Then, based on information such as the calculated position, speed, direction, etc., the above-described feature quantities (such as the residence time of the person or object, the number of direction changes, the frequency of sudden acceleration / deceleration, etc.) are calculated (S1).

  Next, the degree of abnormality is estimated in the degree of abnormality estimation & calculation unit 5 (S2). First, the degree of abnormality is estimated by comparing the multi-dimensional feature value obtained by image processing of the camera image with the feature value (see FIG. 3) included in the case accumulated in the past, and the most similar feature value. Search for cases with. Then, the degree of abnormality included in the searched case is obtained as the estimated degree of abnormality. As a method for determining whether or not they are similar, for example, Nearest Neighbor method or the like can be used.

  The degree-of-abnormality estimation & calculation unit 5 estimates the degree of abnormality for all camera images, and creates an abnormality degree list including each estimated degree of abnormality (S3). An example of the degree of abnormality list is shown in FIG. “Time” is, for example, a representative time of a frame period from which a feature amount is extracted. “A”, “b”, “c”... Are types of feature values. “Final display time” is the final time when the video of the monitoring camera having the corresponding camera ID is displayed. The “confirmation” field will be described later.

  The management control device 7 determines display priorities in descending order of the estimated abnormality degree based on the abnormality degree list, and controls switching of video output to each monitor 3 according to the decided priority order (S4). For example, when the number of surveillance cameras 1 is X and the number of monitors 3 is Y (<X), among the X surveillance cameras, the images of the top Y surveillance cameras having a high estimated abnormality degree are displayed as Y monitors. 3 is displayed.

  The management control device 7 displays an abnormality occurrence on the operation terminal 8 as a video including a particularly noticeable event for the video of the monitoring camera whose estimated abnormality is higher than a preset threshold in the abnormality list (Yes in S5). (S6), the monitoring staff is notified of the possibility of occurrence of an abnormality.

  The monitor checks the video displayed on the monitor 3, determines whether there is any problem (whether there is no abnormality), and information on the confirmation result, such as whether there is an abnormality or whether similar notification is necessary in the future. (Determination result) is input from the terminal 8 (S7). Even if there is no notification of the possibility of occurrence of an abnormality, the monitor may confirm the video and input information of the confirmation result from the terminal 8 (see the broken line connecting from S4 to S7). The monitor may take measures such as contacting a related department if necessary to deal with an event included in the video.

  Further, the supervisor is picked up by the line-of-sight camera 9, and the line of sight and the movement of the video of the supervisor picked up by the line-of-sight camera 9 are detected by image processing in the image processing apparatus 10 (S8). The image processing apparatus 10 determines the behavior of the supervisor based on the gaze of the supervisor and the movement thereof, such as how long the supervisor stares at the specific monitor or the frequency at which the specific monitor is repeatedly looked back (gaze movement frequency). A supervisory action feature amount representing the feature of the monitor is calculated (S8).

  In addition, when the monitor performs an operation on the video (the monitor) such as zooming in to see the video displayed on the monitor in more detail, the operation details on the video (monitor) are displayed. The information to be represented is sent to the abnormality degree estimation & calculation unit 5 via the management control device 7 (S9).

  The management control device 7 determines whether the monitor has confirmed the event included in the video displayed on the monitor from the operation information for the monitor acquired in S9 or the information of the line of sight of the monitor acquired in S8. Information to be expressed is generated (S10).

  The management control device 7 confirms whether or not the monitor has confirmed the event included in the video based on the confirmation result information (determination result) input in S7 or the information generated in S10 (S11). . Regarding the video (surveillance camera) including the confirmed event, the abnormality degree estimation & calculation unit 5 adds “completed” to the abnormality degree list (S12) (see the confirmation field in FIG. 4).

In addition to the addition to the abnormality degree list, the abnormality degree estimation & calculation unit 5 uses the at least one of the information obtained in S7 to S10 and uses the abnormality degree for the feature quantity estimated in S2 Is calculated (S13). An example of the calculation of the degree of abnormality is shown in the following formula.
Gaze time x a1 + Eye movement frequency x a2 + Operation history x a3 = Abnormality

  In the above formula, a1, a2, and a3 are the weights of each variable. Further, the line-of-sight time, the line-of-sight movement frequency, and the operation history are the supervisor behavior feature quantities, and the supervisor behavior feature quantities (for example, operation history) other than the supervisor behavior feature quantities extracted by the image processing apparatus 10 are S7 and S9. , Calculated from the information obtained in S10.

  The degree-of-abnormality estimation & calculation unit 5 stores the calculated degree of abnormality in the correlation data storage unit 6 as learning correlation data (example) together with the feature amount and the like (S14).

  The video of the video camera determined to have been confirmed in S11 does not need to be displayed on the monitor 3, and the process returns to S2 to calculate the estimated abnormality degree again. At that time, by using the correlation data accumulated in S14 as learning data, the accuracy of estimation of the degree of abnormality is improved. When the abnormality degree is estimated again, the abnormality degree list is updated with the estimated abnormality degree, and at that time, “completed” indicating confirmation is deleted.

  Here, even when the monitor does not confirm the monitor video in S11 (No in S11), when a certain time has passed (Yes in S15), the video to be monitored most at that time is displayed on the monitor each time. In order to continue to maintain the state, the degree of abnormality is estimated again for the monitor video (S2), and the degree of abnormality list is updated (S3).

  Since the above-described processing alone may cause operational problems such as the display of a specific video camera being biased to a specific monitor, the management control device 7 can prevent the video camera that has an estimated abnormality level above a certain threshold. For video, even if the value drops after a certain period of time (for example, even if the estimated abnormality level of the video camera is excluded from the estimated abnormal level of the video of another video camera), the video is controlled until it is confirmed. You may go.

  In addition, the management control device 7 performs control so that the video of the confirmed video camera is not displayed for a certain period of time even if the estimated abnormal degree continues to be high, so that the video camera with the low estimated abnormal degree is displayed. The video may be displayed appropriately.

  In addition, the management control device 7 controls the video of the video camera that has been confirmed not to be displayed for a certain period of time even if the estimated abnormality degree is high, and the abnormality degree estimation & calculation unit 5 When the video of the video camera is confirmed, the abnormality level estimation may be interrupted even during the calculation, and the estimated abnormality level at that time may be acquired and output to the management control device 7. As a result, it is possible to always display on the monitor monitor with the latest priority order except for the confirmed video, and to prompt the monitor to confirm the event.

  In addition, the management control device 7 may perform control to display once in a fixed time in preparation for a detection failure even if the state where the estimated abnormality degree is low continues.

  As described above, according to the present embodiment, it is possible to reduce the burden on the monitoring staff and speed up the discovery of abnormal events.

  Also, the accuracy of estimation of the degree of abnormality is improved by learning more appropriate degree of abnormality based on the operation of the observer on the monitor video, the action of the observer on the notification of the possibility of abnormality, and the gaze of the observer on the monitor Can be made.

  In addition, since the monitoring staff confirms the presence / absence of an abnormality for an event that is not clearly defined in advance and the result of the confirmation is reflected in the learning of the abnormality level, the accuracy of the abnormality level estimation can be improved.

The block diagram which shows the structure of the video surveillance system as one embodiment of this invention The flowchart which shows the flow of a process of the system of FIG. The figure which shows the example of the contents of the correlation data accumulation section Figure showing an example of the abnormality list

Explanation of symbols

1: surveillance camera 2: video switching device 3: video display monitor 4: image processing device (feature amount extraction unit)
5: Abnormality estimation & calculation part (abnormality estimation part, abnormality degree calculation part, list maintenance part, database update part)
6: Correlation data storage (database)
7: Management control device (management control unit, information acquisition unit)
8: Operation terminal 9: Line-of-sight camera 10: Image processing device (measurement unit, second feature amount extraction unit)

Claims (9)

A feature quantity extraction unit that extracts one or more feature quantities representing features of the video from each video captured by a plurality of surveillance cameras;
A database storing a plurality of cases including one or more feature quantities and an abnormality degree of a monitoring event;
Using the feature amount extracted from each video and the database, an abnormality degree estimation unit that estimates the degree of abnormality for the monitoring event represented by each video,
A video switching unit that selectively outputs video of each of the monitoring cameras to a predetermined number of monitors;
A management control unit that controls video selection by the video switching unit based on each estimated degree of abnormality;
An information acquisition unit for acquiring information representing the behavior of the monitor for the video displayed on each of the monitors;
An abnormality degree calculation unit for calculating an abnormality degree with respect to a monitoring event represented by a video displayed on each monitor from the action of the acquired observer;
A database updating unit for registering each case including the feature amount of the video displayed on each monitor and the degree of abnormality calculated by the degree of abnormality calculating unit in the database;
Video surveillance system with The information acquisition unit
An information output unit that outputs information urging the supervisor to confirm the video with the estimated abnormality degree exceeding the first threshold, to the supervisor's operation terminal;
The video monitoring system according to claim 1, further comprising: an information receiving unit that receives information representing a confirmation result of the video input to the operation terminal by a monitor.   The video monitoring system according to claim 1, wherein the information acquisition unit acquires an operation performed by a monitor on the video displayed on the monitor via the operation terminal. A measuring unit for measuring the movement of the line of sight of the observer;
A second feature amount extraction unit that extracts one or more feature amounts representing features of the movement of the line of sight;
The said information acquisition part acquires the feature-value extracted by the said 2nd feature-value extraction part as information showing the action of the said monitoring person, The Claim 1 thru | or 3 characterized by the above-mentioned. Video surveillance system. A list maintaining unit that generates and updates a list that stores a set of the estimated abnormality degree and the identifier of the video surveillance camera for which the abnormality degree is estimated;
5. The control unit according to claim 1, wherein the management control unit controls the video switching unit based on the list so that the estimated video with a high degree of abnormality is preferentially displayed. 6. The video surveillance system described in 1.   The management control unit has detection means for detecting whether or not confirmation by a monitor is performed on the video displayed on each monitor, and the estimated abnormality degree exceeds the second threshold value. In contrast, the video switching unit is controlled so that the video is continuously displayed until confirmation by a monitor is performed regardless of the estimated value of the degree of abnormality. Item 6. The video surveillance system according to any one of Items 1 to 5.   The management control unit controls the video switching unit so that the video is not displayed for a predetermined time after confirmation by a monitor for the video in which the estimated degree of abnormality exceeds the second threshold. The video surveillance system according to claim 6, wherein the video surveillance system is controlled. The abnormality degree estimation unit
Estimating the degree of abnormality according to an algorithm that can obtain the estimated degree of abnormality at any point in the middle of the calculation, and the estimation accuracy improves as time is spent on the calculation.
If it is detected in the management control unit that confirmation by a monitor has been performed on a video that exceeds the second threshold, the abnormality degree estimation is interrupted, and the estimated abnormality at the time when confirmation is performed Get the degree,
The video management system according to claim 7, wherein the management control unit controls video selection by the video switching unit based on the estimated degree of abnormality at the time when the confirmation is performed. A feature amount extraction step of extracting one or more feature amounts representing features of the video from each video captured by a plurality of surveillance cameras;
Abnormality for the monitoring event represented by each video using a database storing a plurality of cases including one or more feature quantities and the degree of abnormality of the monitoring event, and feature quantities extracted from each video A degree of abnormality estimation step for estimating each degree;
A video switching step of selectively outputting video of each of the monitoring cameras to a predetermined number of monitors;
A management control step for controlling video selection by the video switching step based on each estimated abnormality degree;
An information acquisition step of acquiring information representing the behavior of the observer with respect to the video displayed on each of the monitors;
An abnormality degree calculating step for calculating an abnormality degree with respect to the monitoring event represented by the video displayed on each monitor from the acquired action of the observer;
A database update step of registering each case including the feature amount of the video displayed on each monitor and the calculated degree of abnormality in the database;
A video surveillance method comprising:

Images