WO2017047687A1

WO2017047687A1 - Monitoring system

Info

Publication number: WO2017047687A1
Application number: PCT/JP2016/077236
Authority: WO
Inventors: 佑一郎小宮; 佐々　敦; 山口　宗明; 雅俊近藤
Original assignee: 株式会社日立国際電気
Priority date: 2015-09-17
Filing date: 2016-09-15
Publication date: 2017-03-23
Also published as: JP6584024B2; JPWO2017047687A1

Abstract

A system for detecting a reflected obstruct from among a large number of camera videos with a low processing load is disclosed. A plurality of cameras installed in roadside areas regularly detect the locus of a vehicle traveling down a road or a vehicle speed or a frequency of passage obtained during locus acquisition, and notify a center when there is abnormality. In the center, a camera to be analyzed is narrowed down to the vicinity of a congestion starting point on the basis of the notification, advanced analysis is made on vehicle loci, etc., and an unsteady state is detected from the analyzed data. The camera chosen to be analyzed performs picture quality heightened processing over locus or vehicle speed and other processing preferentially in order to deliver a clearer video to the center.

Description

Monitoring system

The present invention relates to a monitoring system that detects an abnormality occurring on a road based on the movement of a vehicle from a video of a monitoring camera installed on the road.

In recent years, a monitoring system is required to have a function of automatically detecting an abnormality occurring in the field from a video of a monitoring camera and issuing an alarm. Prompt confirmation of abnormalities leads to prompt response after an accident and prevention of serious accidents. Such a function is particularly effective when a large number of surveillance cameras are used, such as in a large-scale facility, and the supervisor cannot check all the images simultaneously.

One of the facilities that require an automatic detection function is an expressway. Abnormalities on the highway can be an emergency stop of the vehicle due to road damage, obstacles, breakdowns, etc., all of which can lead to serious accidents. Therefore, a technique for automatically detecting an abnormal situation from a video of a surveillance camera set on a highway is considered.

As a technique related to the present invention, one that is mounted on a vehicle and detects a movement of a preceding vehicle that avoids an obstacle or the like is known (for example, see Patent Documents 1 and 2).

JP 2005-145153 A JP 2003-276538 A JP 2001-319218 A JP 2000-057474 A JP 2014-157522 A JP-A-4-67299

However, in order to detect anomalies, video analysis from various points is required. In particular, since a correction process is required for a camera installed in the outdoors where the illumination conditions change, a complicated process is required for video analysis. In order to carry out these processes for all surveillance camera videos, a huge amount of calculation is required, and there is a problem that equipment costs increase.

In view of the above problems, an object of the present invention is to provide a monitoring system that performs systematic image processing on a number of surveillance camera images and realizes abnormality detection with limited computational resources.

A monitoring system according to an aspect of the present invention determines a priority order from a monitoring camera having a function of detecting a speed abnormality of a vehicle in a video, speed abnormality information from the monitoring camera, and a positional relationship between the monitoring cameras, A central control unit having a function of analyzing a video of a high-priority surveillance camera video is provided.
The setting of the monitoring camera can be dynamically switched depending on whether or not it is a target of video analysis and whether or not an abnormality is detected.

According to the present invention, an abnormality in the monitoring area can be efficiently detected from the video of the monitoring camera with a small amount of calculation.

1 is a block diagram illustrating an example of a logical configuration of a monitoring system 1 according to a first embodiment. 3 is a schematic state transition diagram of the operation of the monitoring system 1. FIG. The schematic diagram explaining the principle which judges a traffic jam start point. The functional block diagram of the surveillance camera 2. FIG. The schematic diagram which shows the principle which calculates an actual speed. 10 is a flowchart of speed calculation by the speed analysis unit 22. The flowchart of the obstacle alarm alerting which the system control part 7 performs.

The monitoring system according to the embodiment of the present invention is based on the trajectory (route) of a vehicle traveling on the road, or the vehicle speed and the passing frequency obtained in the process of obtaining the trajectory. The presence of the error), and notify the abnormality. In addition, an operation for directly detecting an obstacle causing the abnormality from the video is performed in cooperation with the estimation of the abnormality.

FIG. 1 is a block diagram showing an example of a logical configuration of the monitoring system 1 according to the first embodiment of the present invention.
The monitoring system 1 of this example is a centralized system that is centralized at an arbitrary number N of monitoring camera devices 2-1 to 2-N arranged at predetermined intervals along a highway and a base such as a traffic control center. And a processing system 3.
The central processing system 3 includes a video analysis unit 4, a video display unit 5, a video recording unit 6, and a system control unit 7 that controls them.

The monitoring camera devices 2-1 to 2-N (hereinafter referred to as the monitoring camera 2 as a representative when there is no need to distinguish each) are, for example, a PTZ (panning pan) equipped with an electric head and an electric zoom. In this example, a high-sensitivity color camera is used as the camera body. The pan, tilt, and zoom to be applied are given as camera control information. The surveillance camera 2 encodes and outputs a video signal obtained by the camera body as a surveillance camera video, and measures the speed of the vehicle in the video by a known technique such as Non-Patent Document 1. The measured speed is output as speed abnormality detection information at least when the speed is abnormal. The monitoring camera video can be subjected to processing such as shake correction, gradation correction, and super-resolution before being encoded in order to facilitate processing in the video analysis unit 4.

The video analysis unit 4 analyzes the monitoring camera video received from the monitoring camera 2 and outputs the analysis result. The video analysis unit 4 only needs to process at most M channels (M is an integer smaller than N) designated by the system control unit 7, and simultaneously processes the monitoring camera videos from the N monitoring cameras 2 in real time. There is no need to have the ability. The video analysis unit 4 has a capability of recognizing a vehicle, accumulates or learns a normal vehicle travel locus in advance, generates a vehicle trajectory recognized in a given image, and is not normal. Identify the occurrence of a running track. For example, a trajectory of a vehicle that overtakes a stopped vehicle, a trajectory that avoids falling objects on a road, or a trajectory of a plurality of vehicles that decelerate and accelerate before and after the same point is detected as abnormal. The position information of falling objects and points is output. Further, in response to a request from the system control unit 7, an event (object) itself that causes the abnormal traveling locus is further detected.

The video display unit 5 displays the monitoring camera video received from the monitoring camera 2 according to the display control information. For example, L (L is an integer smaller than N) flat panel displays are arranged vertically and horizontally, an L channel image is input, and the image is output together with surveillance camera information (photographing location name) and the like. The video display unit 5 preferentially displays a video in which an abnormality has been detected or reveals an alarm that has been issued in accordance with a command from the system control unit 7.

The video recording unit 6 records the surveillance camera video received from the surveillance camera 2 on a hard disk or the like according to the recording control information. In general, it is too costly to transmit and record images from N surveillance cameras 2 at the same time at a full rate, and therefore, the images are recorded by being thinned out temporally, spatially, and geographically. The video recording unit 6 may change the video recording method based on the result of abnormality detection. For example, in the case of recording video after compressing it, the video of the surveillance camera in which an abnormality has been detected is saved at a reduced compression rate.

The system control unit 7 has a function of issuing an alarm based on the speed abnormality detection information from the connected N monitoring cameras 2 and the analysis result of the video analysis unit 4 and notifying the video display unit. Similarly to general CCTV, display control information is generated and the recording rate of the video recording unit 5 is set in order to display video on the video display unit 5 while cyclically switching the video of the surveillance camera 2. Generate recording control information to be controlled. The system control unit 7 can be implemented as a part of a process control system (also called a distributed control system (DCS)) that performs traffic control. The traffic control system is mainly managed by traffic counters and traffic measured by ETC 2.0.

FIG. 2 shows a schematic state transition diagram of the operation of the monitoring system 1.
The monitoring system 1 of this example generally has four states: steady state, congestion cause search, obstacle search, and obstacle observation. In a traffic control system managing a long route, this state transition can be performed geographically and locally.
The steady state is a state in which there is no traffic jam and no obstacle, and the N monitoring cameras 2 are used like a traffic counter to monitor whether a speed abnormality has occurred. Further, the video is analyzed while selecting the surveillance camera 2 in a cyclic manner, and an attempt is made to detect the vehicle avoidance (trajectory of travel that avoids the obstacle) or the obstacle itself. When a speed abnormality is detected, the state transits to a congestion cause search state, when an escape is detected, the state transits to an obstacle search state, and when an obstacle is detected, the state transits to an obstacle observation state.

The congestion cause search state is a state in which the start point and cause of the congestion are searched, the start point of the congestion is estimated from the shooting point of the video where the speed abnormality is detected and its change, and detection of the vehicle indicating the avoidance at that point is detected. Try. At the same time, the speed abnormality is continuously monitored in the places other than the starting point as in the steady state. As shown in FIG. 3, if there is a section where speed anomalies (low speed) continue, it is estimated as a traffic jam section, and the end of the traffic jam section on the vehicle traveling direction side (that is, a camera with speed anomalies and no speed anomalies) It can be estimated that there is a traffic jam start point between the cameras). If the speed of the traveling vehicle suddenly decreases with a certain camera or reaches a low speed that can be regarded as a traffic jam, if there is no speed abnormality before and after that, it can be estimated that it is the starting point of a newly generated traffic jam.
If the estimated start point of the traffic jam is a location where natural traffic jams frequently occur, or if other known causes are known, a transition is made to a steady state. Otherwise, a high-magnification image near the starting point is acquired cyclically by PTZ control, attempts to detect a vehicle that shows avoidance, and if detected, transitions to the obstacle search state and issues an alarm (obstacle alarm) To do.

In the obstacle search state, the image of the escape detection point is analyzed to try to detect the obstacle itself. At the same time, monitoring of speed anomalies is continued outside the detection point. When using the background subtraction method to detect obstacles, it is necessary to create an image (background image) when there are no obstacles in advance. And When an obstacle is found, an alarm is issued as appropriate and a transition to the obstacle observation state is made.

In the obstacle observation state, it is a state in which the detected obstacle is continuously monitored, and when it is caught by the vehicle and disappears from the spot, it transits to the congestion cause search state and is dropped again on the road in the driving direction. You will search for traffic jams due to obstacles. Moreover, when the removal of the obstacle is completed by a worker who has visited the site, the state transitions to a steady state.

FIG. 4 shows a functional block diagram of the surveillance camera 2 of the present embodiment. The surveillance camera 2 includes an imaging unit 21, a speed analysis unit 22, an image quality enhancement processing unit 23, and a camera control unit 24.
The monitoring camera 2 of this example calculates the speed of the subject from the captured video, detects a speed abnormality, and notifies the central processing system as speed abnormality detection information.

The imaging unit 21 includes a color imaging element, and outputs a subject image formed on the imaging surface as an imaging unit output video.
The speed analysis unit 22 calculates and outputs the actual speed of the subject (vehicle) in the input video.
The high image quality processing unit 23 performs high image quality processing such as super-resolution and contrast correction on the input video and outputs the result. The speed analysis unit 22 and the image quality improvement processing unit 23 can be realized by software operating on a common processor. Alternatively, it can be realized in such a manner that a multiplier or the like is exclusively used by hardware such as a common FPGA (Field-Programmable Gate Array).

The camera control unit 24 controls operations of the PTZ, the speed analysis unit 22, and the image quality improvement processing unit 23 based on the camera control information from the central processing system 3. When a request for speed abnormality detection information is received from the central processing system 3, the monitoring camera 2 measures whether or not a speed abnormality has occurred within a certain period and responds as speed abnormality detection information. For example, two threshold values TL and TH are set for the speed and its temporal change, and an abnormality occurs when the vehicle speed is equal to or lower than TL or equal to or higher than TH.
The camera control unit 24 controls the speed analysis unit and the image quality improvement processing unit in accordance with a request from the central processing unit. For example, only one function may be operated according to the request. When the high image quality processing unit is operating, the output is output as a monitoring camera video, and when it is not operating, the imaging unit output video is output as a monitoring camera video.

The electric camera platform 25 changes the imaging direction of the imaging unit 21 in accordance with the camera control information received via the camera control unit 24.
The electric zoom lens 26 changes the imaging range (zoom magnification) of the imaging unit 21 according to the received camera control information.
The speed analysis unit 22, the image quality improvement processing unit 23, and the camera control unit 24 may be provided in the case of the imaging unit 21.

Fig. 5 shows the principle of calculating the actual speed in the real world. First, a speed detection range that is a range in which the speed is calculated in the video is determined. The actual distance of the speed detection range can be obtained from the height and angle at which the camera is installed. Alternatively, it may be set using a reference distance in the video such as a white line on the road. The speed of the vehicle can be obtained, for example, by measuring how many frames it takes for the vehicle to pass through the speed detection range set above.

FIG. 6 shows a flow of speed calculation by the speed analysis unit 22.
First, in S11, one frame of the imaging unit output video is captured.
In step S12, the moving object region is detected for the input frame. The moving object region can be obtained by a known method, for example, a difference in pixel value between frames.
If it is determined in S13 that a moving object exists, a feature point is detected in the moving object region in S14 by, for example, Harris corner detection or SUSAN operator.
Next, as S15, tracking of the moving object is performed by associating the feature points of the moving object region in the time domain. Well-known technologies such as mean shift, particle filter, Kalman filter, etc. can be used as a process for associating feature points of each moving object region between frames, and even if the vehicle is temporarily hidden by a roadside pillar, etc. A trajectory can be obtained. Tracking can be done in moving object region units, feature point units, or cluster units of feature points that are close in distance and similar in motion, and can be used in addition to differences in feature point positions to be matched. For example, the difference in the direction and type of corners (edges) may be considered. In addition, when there are feature points that are always detected at the same position, they may be learned to mask the first detection. A train of vehicles that are closely connected in a traffic jam can be detected as a single moving object region in S12, but the feature points can be handled by appropriately grouping them.
Finally, as S15, the number of frames required to pass the speed detection range is measured from the tracked result.

FIG. 7 shows a flow of obstacle alarm issuing performed by the system control unit 7.
First, in step S31, speed abnormality information is collected from each monitoring camera 2. The speed abnormality information is information indicating whether or not the speed of the subject of each monitoring camera is out of the range set within a certain period. Since speed monitoring information cannot be obtained from the surveillance camera 2 that is already subject to video analysis and is operating in the high image quality mode, the trajectory information obtained in the analysis by the video analysis unit 4 should be used instead. Can do. The calculation of speed can be realized by image analysis with a low calculation cost compared to video analysis using recognition, learning, or the like. In the vicinity of the monitoring camera where the speed abnormality is detected, some abnormality may have occurred, such as traffic jam or the vehicle stepping on the brake.

In step S32, the video of the surveillance camera 2 to be analyzed by the video analysis unit 4 is selected.
The system control unit 7 gives priority to the video of the monitoring camera 2 in which the speed abnormality is detected and transfers it to the video analysis unit 4. If the cameras in which the speed abnormality is detected are located adjacent to each other, the priority of only the head (downstream) monitoring camera 2 is increased as shown in FIG. This is intended to prioritize the camera corresponding to the head of the traffic jam, because it is expected that multiple consecutive cameras on the road will detect speed abnormalities depending on the traffic jam section. . When there are more cameras 2 with higher priority than M, analysis is performed with priority given to a long period from the time of the last analysis to the current time. If the number of high-priority cameras is M or less, depending on the remaining computing resources, the images of cameras and other surveillance cameras that are geographically adjacent to the high-priority cameras are analyzed from the time of the last analysis to the present. Analysis may be performed with priority given to those having a long period. The history selected as the video analysis target is retained for a certain period.

When the monitoring camera for video analysis is determined, the system control unit 7 transmits the information to the target monitoring camera 2 in S33. During that time, the monitoring camera 2 to be analyzed stops the operation of the speed abnormality detection function, and performs processing for improving the image quality of the video with the computation resources. Further, when the surveillance camera 2 compresses and outputs video, the bit rate may be increased only during analysis to improve the image quality.

In S <b> 34, the video analysis unit 4 continues to analyze the video from the monitoring camera 2 that is the analysis target, and sequentially returns the analysis results to the system control unit 7.
In S35, the system control unit 7 determines whether or not the received analysis result (the latest one) indicates an abnormality such as avoidance. Since the analysis result may include erroneous detection, the analysis result may be retained for a certain period in the past and judged based on them.

If no abnormality is confirmed, the system control unit 7 searches the history as to whether there is a camera subject to video analysis more than once in the past fixed period as S36.
In S37, based on the search in S36, it is determined whether or not there is a camera that has been frequently analyzed. If there is a corresponding camera, the process proceeds to the same processing (alarm issue) as when it is determined to be abnormal in S35. If there is no corresponding camera, the process returns to S31.

On the other hand, if an abnormality is confirmed in S35 or if there is a corresponding camera in S37, the system control unit 7 issues an alarm in S38.
The alarm is performed by transmitting the video of the camera in which an abnormality is detected and information such as the position and time to the video display device.

In S39, the information is transmitted to the monitoring camera 2 in which the abnormality is detected. Thereby, on the surveillance camera 2 side, if it is being performed, the operation of the speed abnormality detection function can be stopped, and the processing for improving the image quality of the video can be performed with the calculation resources. The system control unit 7 also transmits camera control information that allows the position information of the abnormality output from the video analysis unit 4 to be clearly captured at a higher magnification to the corresponding monitoring camera 2. At the same time, the video analysis unit 4 is instructed to detect the cause of the abnormality itself. At that time, if necessary, the video analysis unit 4 is also provided with information on the shooting direction and range changed by the camera control information. After S39, the process returns to S31.

The scope of the present invention can include, but is not limited to, the configurations of the embodiments described above.
For example, the video analysis unit 4 may perform road damage detection, obstacle recognition (detection), vehicle emergency stop detection, and the like using the above-described recognition / learning technique or the like. In addition, in order to analyze a video with a high resolution on the monitoring camera 2 side, contrast correction, fluctuation correction, fluctuation correction, super-resolution processing, or the like may be performed as preprocessing.
Further, instead of measuring the speed of the vehicle, the number of passing vehicles per predetermined time may be measured, and the traffic jam start point may be searched from the change.
Further, the video analysis unit 4 may detect the boundary line of the traveling lane drawn on the road surface by Hough transform or the like, and track the moving object using the detected line. For example, it is possible to reduce the amount of processing by excluding movements greatly different from the direction of the traveling lane from the candidates, and it is possible to determine the apparent size of the vehicle and the degree of avoidance based on the width of the lane. Also, when an obstacle is detected from the video, an alarm with a different severity can be issued based on the size and the positional relationship with the travel lane. For example, when an obstacle initially detected in the roadside zone moves to the traveling lane, an alarm with a higher severity is issued again.

Further, the manner of sharing image processing between the monitoring camera 2 and the video analysis unit 4 includes various modes other than those described in this example. For example, the speed analysis unit 22 of the monitoring camera 2 may always calculate the trajectory of the moving area (vehicle) and pass the data to the video analysis unit 4 to perform the subsequent processing. In the video analysis unit 4, the trajectory data is converted from the screen coordinates to the global coordinate system, standardized to pass points that do not depend on the speed, and averaged (accumulated), and the avoidance is determined based on the difference from the averaged data can do. A known technique such as Patent Document 3 can be used for coordinate system conversion. The number of trajectory data (passage point data) to be averaged is required to be a certain number (for example, for 10 vehicles), and the video analysis unit 4 holds averaged data for each shooting field angle of each monitoring camera 2 To do. A well-known algorithm (for example, 1 class SVM, Singular spectrum analysis, etc.) for detecting non-stationary data such as avoidance can also be used.

1 monitoring system, 2 monitoring camera device, 3 central processing system (center), 4 video analysis unit, 5 video display unit,
6 video recording unit, 7 system control unit,
21 imaging unit, 22 speed analysis unit, 23 image quality enhancement processing unit, 24 camera control unit, 25 electric pan head, 26 electric zoom lens.

Claims

In a monitoring system that detects an abnormality from images of a plurality of cameras and issues an alarm, a plurality of monitoring cameras having a function of detecting an abnormality in the speed of a subject or the number of passing objects, and the abnormality detected by the monitoring camera And a central processing unit for performing video analysis by prioritizing video of the plurality of surveillance cameras based on
The monitoring system, wherein the central processing unit switches the setting of the plurality of monitoring cameras depending on whether or not the video analysis target or an abnormality is detected by the video analysis.
The monitoring system according to claim 1, wherein each of the plurality of monitoring cameras is
An imaging unit for outputting video;
An electric pan head for changing the imaging direction or range of the imaging unit;
A speed analyzer that receives the video and calculates and outputs the actual speed of the subject in the video;
An image quality improvement processor that performs an image quality improvement process on the video and outputs the image,
A camera controller that controls to operate only one of the speed analysis unit and the image quality improvement processing unit according to a request from the central processing unit;
When the speed from the speed analyzer is abnormal, the camera controller sends speed abnormality detection information indicating whether or not the speed of the subject is out of a set range within a certain period to the central processing unit. And when the fact that the analysis target has been transmitted from the central processing unit and the fact that the abnormality has been detected by the video analysis is transmitted, the image quality improvement processing unit is operated, and otherwise The speed analysis unit is operated at the time of the monitoring system.
3. The monitoring system according to claim 1, wherein the central processing unit is
Received from the surveillance camera, a video analyzer that analyzes the video received from the surveillance camera that is the subject of the video analysis using recognition or learning, a video display that preferentially displays the video in which an abnormality is detected, and the like. A video recorder that records the video, and based on speed abnormality detection information collected from the plurality of monitoring cameras, a predetermined number or less of the plurality of monitoring cameras is selected as the target of the video analysis, A system controller that issues an alarm based on a result of analyzing the selected video by the video analysis unit, and controls display on the video display and recording on the video recorder. A characteristic surveillance system.
3. The monitoring system according to claim 2, wherein the velocity analyzer detects a moving object region based on a difference in pixel values between frames, detects a feature point in the moving object region, and associates the feature point with a time region. The monitoring system is configured to track the subject and measure the actual speed based on the number of frames required for the tracked subject to pass through the speed detection range.
4. The monitoring system according to claim 3, wherein the subject is a vehicle traveling on a road, and the video analyzer has a capability of recognizing the vehicle, and accumulates or learns a normal vehicle traveling locus in advance. The system recognizes the vehicle recognized in the given video and detects the occurrence of an abnormal driving locus, and the system controller gives priority to the video of the monitoring camera that outputs the speed abnormality detection to the video analyzer. When the surveillance camera that has analyzed and output the speed abnormality detection is continuous on the road, the video analyzer analyzes the video by increasing the priority of the surveillance camera at the head in the traveling direction of the vehicle on the road. In addition, when the video analyzer detects the abnormal traveling locus, the direction or range of imaging of the corresponding monitoring camera toward the estimated position of the event causing the abnormal traveling locus Monitoring system and performs switching of such the setting to change.
A road video monitoring method for observing an abnormality detected by the video analysis using the monitoring system according to claim 1.