JPH0676195A - Abnormal event detector - Google Patents

Abstract

PURPOSE:To precisely detect an abnormal event on a road and to detect by prediction even an event outside a measuring range in capable of being monitored by a television camera. CONSTITUTION:Synthetic judgement can be rendered by providing function parts for (traffic flow measurement), (abnormal travelling monitoring), (stopping vehicle detection), (congestion measurement), (inter-camera interpolation processing) so as to respond to various kinds of abnormal events. Abnormality in the visual field of the television camera can be measured in a moment by the function parts 13, 14, 16, and 18 for the (traffic flow measurement), (abnormal running monitoring), (stopping vehicle detection), and (congestion measurement), and the event outside the visual field of the television camera can be detected by applying interpolation processing in spatial and temporal fashion to a result processed by each image processing device 100, and detection probability can be improved by judging them synthetically.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting abnormal events on a road or the like, and more particularly to a device for processing and detecting an image of a television camera by image processing.

[0002]

2. Description of the Related Art The purpose of detecting an abnormal event on a road is to detect "accident", "stopped vehicle", "falling object", etc. early so that a secondary disaster does not occur due to these abnormalities. . In particular, it is important to detect abnormal events in tunnels, and such systems are expected to increase in the future.

However, the current image processing system only measures so-called traffic flow data such as the number of vehicles passing through and the speed, and is not configured to detect various abnormal events. An example of a traffic flow measurement system is "Architecture of traffic flow measurement system using image processing"; IPSJ 37th National Convention Lecture Paper 6T
-6, 1988;

[0004]

Considering abnormal event monitoring in a tunnel, the reality is that television cameras are not installed at intervals so that the entire tunnel can be monitored. However, it is not known where the abnormal event occurs. For this reason, the conventional traffic flow measurement function alone has made it almost impossible to measure abnormal events outside the field of view of the TV camera. In addition, there is a drawback in that it is not possible to understand all abnormal events using only traffic flow data.

An object of the present invention is to accurately detect various abnormal events and also detect abnormalities other than the field of view of a television camera.

[0006]

[Means for solving the problem] When the abnormal events are sorted out,
It is necessary to detect abnormalities within the field of view of the TV camera and abnormalities in other areas. For this reason, "traffic flow measurement", "stop vehicle detection", "abnormal vehicle detection", and "congestion level measurement" are executed within the field of view of the TV camera, and the measurement results are spatially and temporally interpolated. The above object can be achieved by detecting an abnormality other than the above.

Further, in order to make various abnormality determinations accurately, the above-mentioned respective functions are comprehensively determined.

[0008]

[Function] "Traffic flow measurement" measures the speed and number of vehicles in the field of view of the camera, monitors speed abnormalities, etc., "Detects an abnormal vehicle" monitors the running pattern of vehicles in the field of view of the camera,
The "stopped vehicle detection" detects a stopped vehicle, an accidental vehicle, or a fallen object within the field of view of the camera, and the "congestion degree measurement" measures the congestion degree of the vehicle in stages. These are comprehensively judged to be abnormal. In addition, the above function is measured at each point, and these data are spatially interpolated to predict anomalies other than the field of view of the television camera.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows the overall configuration of the present invention. The abnormal event detection device 200 includes a plurality of image processing devices 100 and a central comprehensive determination unit 22. In order to catch various abnormal events, the image processing apparatus 100 processes the input image of the processing target to obtain various information, the traffic flow measuring unit 13, the abnormal traveling monitoring unit 14, the stopped vehicle detecting unit 16, the congestion degree measuring. It is realized by the unit 18 and the local comprehensive determination unit 19. Further, the central comprehensive judgment unit 22 collects the data of each image processing apparatus and interpolates spatially, for example, to make a comprehensive judgment.

The operation of each unit will be briefly described. In the image processing apparatus 100, an image is input (10), a background difference feature is extracted from the image (11), and a vehicle is tracked (1).
2) As a result, traffic flow is measured (number of vehicles, speed) (1
3). Further, the feature of the frame is extracted from the input image (15) and the stopped vehicle is detected (16). In addition, the features of spatial differentiation and time differentiation are extracted from the input image (1
7) Measure the degree of congestion (18). These results are comprehensively judged by the local comprehensive judging section 19 to judge the presence / absence of an abnormal event. Simultaneously with the abnormal event determination in each device, the central comprehensive determination unit 22 collects all the measurement information obtained in each device, for example, the congestion degree and the speed, and spatially interpolates the abnormal event other than the camera view. To detect.

The details of each section will be described below in order.

(1) Traffic flow measurement section Traffic flow measurement data includes "speed", "passing vehicle number", and the like. An example of a method for obtaining these is shown in FIG. The processing procedure is as follows.

The background image 30 stored in advance is removed from the input image f (t) 31 at time t to extract the vehicle.

The end coordinates of the extracted vehicle image 32 (when the vehicle is photographed from behind) are detected (33).

The above processes 35 and 36 are performed on the images f (t + dt) 34 that are sequentially obtained, and the tracking process 37 determines to which position the end coordinates have moved.

"Number of passing vehicles" 3 due to changes in tracked coordinates
8. Measure "speed" 39.

With such processing, the flow of the car in the field of view of the television camera can be measured. By monitoring this data, it is possible to determine that some abnormality has occurred, for example, when the speed is abnormally reduced.

When the tail coordinates of the vehicle are extracted, the existence range of the vehicle, that is, the coordinates of the left end of the vehicle, the coordinates of the right end, and the like are also measured at the same time in order to monitor abnormal running described later.

(2) Abnormal Travel Monitoring Section If there is a small falling object, it will be difficult for the stopped vehicle detection function described later to detect it. However, if there is a fallen object, the object usually travels avoiding the object, which causes an abnormality in the running pattern of the vehicle. This function is a function of monitoring this abnormal traveling pattern. Abnormal driving may include many vehicles traveling across lanes, frequent lane changes, and sudden changes in speed.

In order to detect such an event, the present invention uses the processing shown in FIG. Road image 4
Shown as 0.

A vehicle is extracted from the input image as in the traffic flow measurement.

Various vehicle coordinates are obtained from the extracted vehicle image. For example, the distance ds from the lane boundary to the vehicle left end coordinate, the distance de to the right end coordinate, the center coordinate, and the like.

Each vehicle is tracked at the time of traffic flow measurement, and changes in coordinates are obtained. As a result, the instantaneous speed and moving direction can be obtained.

The above processing is performed, each data is stored as cumulative data as shown in FIG. 4, and an abnormality is judged.

For example, the accumulated data of the left end coordinate and the right end coordinate shows the range in which many vehicles are traveling with respect to the traveling lane as shown in FIG. Therefore, if the frequency outside the normal traveling range exceeds a certain threshold value, it can be determined that there is an abnormal traveling.

Similarly, if the velocity data and the moving direction data are managed as shown in FIGS. 4-2 and 4-3, it is possible to determine whether or not there are many abnormal values.

(3) Stopped Vehicle Detection Function When a stopped vehicle exists within the field of view of the television camera, it is desirable to directly detect the stopped object by indirect measurement such as abnormal traveling monitoring. That is, a stationary object having a size that can be detected by image processing (a stopped vehicle, a large falling object) is directly detected.

FIG. 5 shows a method of detecting a stationary object, and the outline of the processing is as follows.

An image is input at a certain time interval dt f (t-
dt) 42 and f (t) 43, and the background image 41 stored in advance is removed from the input image.

Only the image of the car remains in the images 44 and 45 from which the background is removed, but a moving object is detected using this image. For moving object detection, there is also a method of obtaining velocity like traffic flow measurement, but here the moving object is detected by the feature between the frames of the image. That is, the difference processing is executed on the two images to extract the moving area (4
6).

Of the images from which the background has been removed, the region from which the moving region image 46 has been removed becomes the stationary object candidate image (4
7). It is confirmed that the position of this object does not move, and it is determined that there is a stopped object as a final result (48).

By executing the above processing, it is possible to detect a stopped vehicle or a falling object within the field of view of the television camera.

This method is a part of the method disclosed in Japanese Patent Application No. 4-10746.

(4) Congestion degree measurement function Abnormal events outside the field of view of the television camera cannot be directly detected. Therefore, the phenomenon that an abnormality spreads within the field of view of the television camera, that is, the degree of congestion is measured. When the degree of congestion becomes different from the normal one, it can be determined that some abnormality has occurred in the forward direction of the vehicle.

There are various methods for measuring the degree of congestion,
The method adopted in the present invention does not use traffic flow data, but is based on the measurement such as a large number of cars in the macro, a small number of cars, a high speed, a slow speed as shown in FIG. this is,
This is because when the vehicles are crowded, the images taken by the TV camera will overlap, making it impossible to microscopically measure the number and speed. Especially in the tunnel, because the TV camera is installed at a low position, it is very likely that vehicles will overlap.

The processing outline is as follows.

Differentiating the input image f (t) 50 (extracting contours perpendicular to the traveling direction) (spatial differentiating 5
1), a feature amount related to the amount of vehicles is calculated (52). Here, the differential image is binarized to determine the number of vehicle contours.

Further, a certain time interval dt (for example, 20
The image f (t + dt) 53 is input at an interval of 0 ms, and a differential image for each pixel of f (t) 50 and f (t + dt) 53 is obtained (time differentiation processing 54). If the vehicle is moving, some image will appear in the difference image, but if it is stationary, no information will be obtained. Therefore, the feature amount (width, etc.) of the image obtained from the difference image is obtained as data relating to speed (55).

The above-mentioned characteristic amount relating to the amount and speed of the vehicle is given to the judgment function 56 which has been learned in advance, and the congestion degree is calculated. The determination function here uses a neural network.

Abnormal events can be monitored not only by monitoring the degree of traffic congestion obtained by the above processing, but also by monitoring the amount and speed of the vehicle, which is the required characteristic amount.

This method is the method disclosed in Japanese Patent Application No. 3-197726.

(5) Local comprehensive judgment section In the local comprehensive judgment section, "traffic flow measurement" and "abnormal running monitoring"
Mainly processes for improving the reliability of information obtained from the functions of "stop vehicle detection" and "congestion degree measurement". Abnormalities include local events such as stopped vehicles (accidents) and speed over,
Divide into a wide range of abnormalities such as traffic jams. An alarm should be issued at the time when each image processing device detects a local abnormality, but there is a problem with the measurement accuracy and it is necessary to check the measurement result. In addition, the alarming method changes depending on whether the wide range of abnormalities or congestion levels are only those within the camera's field of view, or whether there is congestion at all measurement points. Therefore, for a wide range of abnormalities, information (congestion degree, traffic flow measurement result) is sent to a higher rank to make a comprehensive judgment.

Information entered into the local comprehensive determination section includes: traffic flow: fast, slow, high traffic volume, low abnormal running: overspeed, low running line abnormalities, low Stopped vehicle: presence / absence congestion level: high, There are information such as middle and lower, but among these results, the local abnormalities are overspeed and whether a stopped vehicle is detected. Among these, regarding the speed, it is considered that the abnormal event of the overspeed does not occur during the traffic jam when the overspeed measured by the abnormal traveling function during the traffic jam occurs, and thus such inconsistent data is canceled. Therefore, the relation between the measurement result of the congestion degree measuring function and the speed information is checked, and the inconsistent data is canceled to prevent an overalarm. Since the information obtained by detecting the stopped vehicle is highly reliable, an alarm may be issued immediately upon detection.

Further, if it is known in advance how much an error occurs in the measured speed data according to the congestion level, it is possible to correct the speed data according to the obtained congestion level. is there. The same can be done for the number of cars.

The local comprehensive judging section checks the data inconsistency as described above and transmits the information to the upper central comprehensive judging section 22, and when a stopped vehicle exists,
The information on the presence of abnormal traveling obtained when the speed can be accurately measured (“empty” state) is controlled so that an alarm is output immediately.

(6) Central Comprehensive Judgment Section When the television cameras are installed at intervals of 200 m, for example, the range that can be actually measured by image processing is about half of the range. Therefore, it is necessary to interpolate (estimate) an abnormal event at a position that cannot be monitored by image processing from the processing results of the surrounding TV cameras.

Therefore, in the present invention, the following interpolation processing is executed.

(I) Spatial Interpolation The spatial interpolation in the central comprehensive judgment section spatially captures the result of processing the image of each television camera and performs interpolation processing. By spatially capturing, it is possible to determine whether a traffic jam is a chronic traffic jam, a traffic jam due to an accident or a stopped vehicle, and to estimate an abnormality in an area not photographed by a camera.
For example, taking the congestion degree as an example, the congestion degree at each point is plotted in a plane as shown in FIG. 7, and some abnormality occurs in the vicinity of the point where the congestion degree is largely different from the surrounding situation. Determine that In the figure, the congestion degree in the left lane is low, and the congestion degree in the opposite lane (overtaking lane) is high. Therefore, it can be estimated that lane changes are abnormally frequent in this section.

Specifically, the degree of congestion obtained from each image processing apparatus is comprehensively judged, and if there is congestion in all sections, it is an area other than the measurement range or a place where chronic congestion occurs. It is determined that it is a chronic traffic jam, and it is judged to be abnormal only when only a part of the traffic is within the measurement range.

For this reason, all the obtained information is data-processed, a part where the degree of congestion is locally changed is searched, and if there is such a part, it is judged to be abnormal, and otherwise it is judged to be a normal situation. To do. As a data analysis method, there is a method of using a general waveform (congestion degree-one-dimensional data of a place) analysis method or a method of inputting the waveform itself to a neural network to find a change point.

As a target of data analysis, not only the congestion degree but also abnormal traveling monitoring data (travel lane data, speed data, etc.) can be analyzed. In this case, it is possible to determine that there is an abnormality when the traveling line is deviated by a certain portion.

In addition, in order to monitor an abnormal event, in general, an observer often observes the monitor screen to monitor the presence or absence of an abnormality, but a spatial graph as shown in FIG. It is possible to make it easier to understand the situation of the entire measurement range by showing the result of judgment to the observer.

The type of abnormality cannot be specified because it is an invisible phenomenon, but it is possible to determine whether or not it has occurred. Although the congestion degree is plotted in FIG. 7, the coordinate distribution of the vehicle, the speed distribution, and the like may be spatially captured.

(Ii) Temporal Interpolation The data measured by each image processing apparatus is collected, and these are managed in time series as shown in FIG. 8 so that the time of abnormality occurrence and the primary abnormality (noise unique to image processing). It is caused by the)), or whether it is a continuing abnormal condition. This determination method is obtained by waveform analysis similarly to the above spatial interpolation method.

As a result, noise-like abnormalities can be removed, and the reliability of the system can be improved.
Also, by displaying it to the observer in the same way as the above spatial interpolation,
It is possible to make it easier to understand the temporal changes.

FIG. 9 shows an example of a hardware configuration for realizing the above abnormal event detection. Multiple TV cameras 6
0 is installed, and the video signal of each camera is input to the image processing apparatus 100. Image processing device 10
In 0, “traffic flow measurement”, “abnormal running monitoring”, “stopped vehicle detection”, and “congestion degree measurement” are executed, and the result is transmitted to the upper central comprehensive determination unit 22.

As shown in FIG. 10, the image processing apparatus 100 stores an A / D converter 70 for converting a video signal of the television camera 60 into a digital signal, an image memory 71 for storing this data, and data in the image memory 71. An image processing processor 72 for processing, a data output section 73 for transmitting data to the central comprehensive determination section 22, and a CPU 74 for controlling these. The image processor 72 can execute calculation between images such as difference and addition of images, spatial product sum calculation such as differentiation, binarization processing, and histogram processing. Further, the local comprehensive judgment is processed by the CPU 74.

As shown in FIG. 11, the central comprehensive judging section 22 has an interface section 74 for transmitting / receiving data to / from each image processing apparatus 100 (not shown), a data storing section 75 for storing the obtained data, and a stored data. Spatially,
It is composed of a data analysis unit 76 that performs temporal interpolation processing.

[0060]

According to the present invention, by providing the functions of "traffic flow measurement", "abnormal running detection", "stopped vehicle detection", "congestion degree measurement", and "interpolation between cameras", various abnormal events can be accurately detected. It becomes detectable.

In the above description, the monitoring in the tunnel has been described, but it goes without saying that the monitoring of the general road can be similarly performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of an abnormal event detection device according to the present invention.

FIG. 2 is a flowchart illustrating an outline of processing of traffic flow measurement.

FIG. 3 is an explanatory diagram showing an outline of processing of abnormal traveling monitoring.

FIG. 4 is an explanatory diagram showing an example of accumulating data obtained for abnormal traveling monitoring.

FIG. 5 is a flowchart illustrating an outline of processing for detecting a stopped vehicle.

FIG. 6 is an explanatory diagram showing an outline of processing of congestion degree measurement.

FIG. 7 is an explanatory diagram showing an example of an interpolation principle between cameras by spatial interpolation.

FIG. 8 is an explanatory diagram showing an example of an interpolation principle between cameras by temporal interpolation.

FIG. 9 is a block diagram showing an example of a hardware configuration of an abnormal event detection device according to the present invention.

10 is a block diagram showing a specific configuration of the image processing apparatus in FIG.

11 is a block diagram showing a specific configuration of a central comprehensive determination unit in FIG.

[Explanation of symbols]

 13 Traffic Flow Measuring Unit 14 Abnormal Running Monitoring Unit 16 Stopped Vehicle Detection Unit 18 Congestion Level Measuring Unit 19 Local Comprehensive Judgment Unit 22 Central Comprehensive Judgment Unit 71 Image Memory 72 Image Processor 73 Data Output Unit 74 CPU 75 Data Storage Unit 76 Data Analysis Department

Front page continuation (72) Inventor Haruki Inoue 3-2-1, Sachimachi, Hitachi, Ibaraki Hitachi Engineering Co., Ltd. (72) Inventor Takeshi Horie 4-6 Kanda Surugadai, Chiyoda-ku, Tokyo Hitachi, Ltd. In-house (72) Inventor Katsuyuki Yamamoto 4-6 Kanda Surugadai, Chiyoda-ku, Tokyo Inside Hitachi Ltd. (72) Inventor Kazuka Asada 5-2-1 Omika-cho, Hitachi-shi, Ibaraki Hitachi Mitsuka Ota in the factory

Claims (6)

[Claims] 1. An abnormal event detection device for detecting an abnormal event by photographing a road with a television camera and image-processing the image of the television camera, in a speed, number measurement, abnormal traveling monitoring function, detection of a stopped object, traffic jam An abnormal event detection device having a degree measurement function, and having a local comprehensive judgment unit that corrects data related to speed and number obtained by a speed / number of vehicles measurement and abnormal travel monitoring function according to the degree of congestion. 2. An abnormal event by installing a plurality of TV cameras at a predetermined distance on a road, photographing the road by each of the plurality of TV cameras, and image-processing the images of the plurality of TV cameras. In the abnormal event detection device for detecting the, image processing of the video of the plurality of television cameras, speed, number measurement, abnormal traveling monitoring function, stationary object detection,
Congestion degree measurement function, image processing device having a local comprehensive judgment unit that corrects data related to speed and number obtained by speed, number measurement and abnormal running monitoring function according to congestion degree, and measurement of multiple image processing devices An abnormal event detection device having a central comprehensive judgment unit for making a comprehensive judgment by interpolating results spatially and temporally. 3. The abnormal traveling monitoring function according to claim 1, wherein the vehicle is extracted from the input image, the left end coordinate, the right end coordinate, the end coordinate, etc. of the vehicle are obtained from the extracted vehicle image, and the obtained coordinate and the next coordinate The amount of movement of the vehicle is calculated based on the image input at the time, and the coordinate values obtained as a result, the speed, and the moving direction are accumulated for each of the number of vehicles, and the coordinate is traveling at a position other than the predetermined position. If the frequency and the frequency of vehicles other than the predetermined speed and the frequency of vehicles other than the predetermined direction of movement exceed the thresholds, it is judged that abnormal running has occurred. An abnormal event detection device having an abnormal travel monitoring function. 4. The abnormality according to claim 1, wherein the local comprehensive judgment unit cancels the speed data when the speed data obtained when the degree of congestion is high is above a certain threshold value. Event detector. 5. The abnormal event detection device according to claim 1, wherein the local comprehensive determination unit corrects speed data, vehicle number data, and the like obtained according to the degree of congestion. 6. The central comprehensive judgment unit according to claim 2, wherein a plurality of types of information are respectively collected from a plurality of image processing apparatuses, these pieces of information are spatially captured, and locally changed with each piece of information. If there is an abnormal event, the spatial interpolation means determines that an abnormal event has occurred, and multiple pieces of information are captured in time series to determine the time at which an abnormality has occurred and whether it is a momentary abnormality. An abnormal event detection device characterized by comprising temporal interpolation means.