RU2534827C2 - Method for video surveillance of open space with fire hazard monitoring - Google Patents

Abstract

FIELD: physics, video.

SUBSTANCE: invention relates to video surveillance, primarily of open spaces, with fire hazard monitoring and can be used to monitor forest areas in regions with underdeveloped infrastructure. The method comprises video monitoring of a protected area; selecting mobile regions on the obtained image; comparing said regions with reference images from a library of images and making a decision on burning based on the similarity of the obtained images and the available images; wherein mobile objects are selected on the image by breaking down the whole image into rectangular blocks; the shifting of blocks over time is diagnosed from the shift of the centre of a block; separate mobile blocks are merged by spatial clusterisation; time clusterisation of mobile objects is carried out based on a criterion of the intersection of trajectories of separate blocks with each other, wherein the shape of a mobile object is reconstructed by cutting off short edges, joining adjacent boundary points and smoothing the obtained boundaries, wherein the internal region of the object is reconstructed on the obtained boundary, and clusterisation of the selected mobile object is carried out on the normal component of optical flux.

EFFECT: high probability of recognising signs of fire on an open area owing to early detection of fire from smoke traces and air currents.

3 cl, 1 dwg

Description

The invention relates to the field of video surveillance, mainly in open spaces, with fire hazard control and can be used to track forests in regions with poorly developed infrastructure.

A known method of detecting an event for a video surveillance system, comprising a training phase in which training images of the controlled area are obtained at different times in the absence of any events to be detected, and a working phase of detection in which current images of the mentioned zone are obtained, while the event is detected by comparing images with an image corresponding to a linear combination of a plurality of reference images approximating the corresponding training images or matching with them, according to the application No. 2009106852/08, publ. 09/10/2010

The known method is effective for situations in which the absence of events to be observed is deliberately ensured, which is difficult to guarantee when controlling large open spaces.

A known method of video surveillance, which allows you to control the premises and territory, to carry out video recording of disturbing events and continuous recording of video information, according to the patent of the Russian Federation No. 2381533, publ. 02/10/2010

The method has not enough high information content when working in real time.

A known method of video surveillance, including the steps of receiving a video signal of a controlled area of at least one video camera, transmitting it and analyzing it in at least one computing device with memory, in which according to the invention the signal received by the video camera is analyzed using a computing device using information with descriptions of alarm situations , pre-recorded in the form of a database in the memory of the computing device, according to the results of the analysis, data on the target is obtained, the control drove it, transmit it to a mobile video camera, adapted for aiming at a target in accordance with a control signal, using a mobile video camera obtain a target image suitable for further analysis, and transfer it to a database for storage.

Preferably, in the method according to the invention, according to the results of the analysis, an alarm is additionally generated and transmitted to the operator. The analysis is carried out with the ability to calculate the velocity vector, the current coordinate of finding the target, and additionally generate and transmit to the mobile video camera the corresponding control signal to ensure that the moving video camera is aimed at the target. The control signal is transmitted to the mobile video camera via a serial communication channel. I analyze the signal from the video camera taking into account the target parameters selected from the group including brightness, size, duration of stay in the controlled area, speed and direction of movement. (RF patent No. 2268497, published on January 20, 2006, adopted as a prototype of the invention).

There is a method of video surveillance and fire hazard control, according to which the protected area is video-monitored, moving areas are selected on the resulting image, these areas are compared with model images from the image library and a decision is made on fire based on the similarity of the received and available images.

The technical problem that the claimed invention solves is to increase the likelihood of recognizing signs of a fire in an open area due to the early diagnosis of a fire by smoke traces and air flows.

The problem is solved in that in the method of video surveillance of open space with fire hazard control, according to which the protected area is video-monitored, moving areas are selected on the received image, these areas are compared with model images from the image library and a decision is made on fire based on the similarity of the received and available images ,

According to the invention, the allocation in the image of moving objects is done by dividing the entire image into square blocks, the displacement of the blocks with time is diagnosed by the displacement of the center of the block, individual moving blocks are combined by spatial clustering, temporary clustering of moving blocks is carried out according to the criterion for the intersection of the trajectories of the individual blocks with each other, and the shape a moving object is restored by cutting off short edges, connecting adjacent edge points and smoothing along scientists boundaries, the inner region of the object is reduced by the resulting boundary, and classification of the selected moving object produced by the normal component of the optical flow. In addition, a library of images is formed with the categories “vehicles”, “people”, “animals”, “swaying branches of trees”, “smoke”, “thermal footprint”, “open fire”.

Кроме того, в каждой из категорий библиотеки образов формируют подкатегории «траектория», «скорость», «направление движения» и привязывают полученные образы к характеристическим особенностям охраняемой площади. $$$$

In addition, in each of the categories of image libraries, subcategories “trajectory”, “speed”, “direction of movement” are formed and the resulting images are tied to the characteristic features of the protected area.

The invention is illustrated in figure 1, which presents a block diagram of the implementation of the proposed method.

An example implementation of the method.

1. The selection in the image of moving blocks

The whole image (position 1 in figure 1) is divided into blocks B _{i of} size n × n pixels. The value of n depends on the size of the objects and the resolution of the image and is selected from a range from 1 to 100. Suppose that the center of the block B _i is at the point (x, y) at time t and at the point (x + Δx, y + Δy) in time t + 1. Then the values of the correspondence of the blocks and their displacements for two consecutive moments of time are expressed as follows:

$$D\left(B_i\right)=\frac{\mathrm{one}}{{\mathrm{<figure-callout\; id="2"\; label="n"\; filenames="00000021.png"\; state="\{\{state\}\}">n</figure-callout>}}^2}{{\displaystyle \sum _{\left(i,j\right)\in B_i}|\; |\; |I_{t+\mathrm{one}}\left(x,y\right)-I_t\left(x+\Delta x+i,y+\Delta y+j\right)|\; |\; |}}^2,\left(\mathrm{one}\right)$$

$$\Delta \left(B_i\right)=\sqrt{{\left(\Delta x\right)}^2+{\left(\Delta y\right)}^2}.\left(2\right)$$

To eliminate errors of the second kind (false moving blocks) arising due to noise, it is necessary that the value of D exceed the threshold value δ _α , which can be determined using the significance test. For a pixel (x, y), where no movement occurs, the quantity d (x, y) = I _t (x, y) -I _{t + 1} (x, y) is the camera noise. It is known that it can be described by a Gaussian distribution with a zero mean and a dispersion equal to twice the dispersion of the camera noise. The significance level can be considered as the probability of detecting a block as a moving block, although in reality it is motionless (an error of the second kind). If you select the block size n = 16 and put δ _α = 1, then this will correspond to a probability equal to 0.9 (position 2 in figure 1).

2. Spatial clustering of moving blocks

At this stage, using clustering methods, separate similar moving blocks, for which the value of D exceeds the threshold value δ _α , are combined into moving regions. We will consider two blocks adjacent if they have a common side (or part of it) or a vertex, i.e. they are connected by an 8-adjacency relation. As characteristic features of the blocks, you can use the connectivity C (B _i , B _j ), the normalized difference in the correspondence of the blocks D (B _i , B _j ) and the normalized offset of the blocks Δ (B _i , B _j ):

$$C\left(B_i,B_j\right)=\{\begin{array}{cc}\mathrm{one,}& B_i,B_j-\mathrm{from}\mathrm{about}\mathrm{from}edn\mathrm{and}e\\ \infty ,& \mathrm{and}n\mathrm{but}he\end{array},\left(3\right)$$

$$D\left(B_i,B_j\right)=\frac{|\; |\; |D(B_i)-D(B_j)|\; |\; |}{|\; |\; |D(B_i)+D(B_j)|\; |\; |},\left(\mathrm{four}\right)$$

$$\Delta \left(B_i,B_j\right)=\frac{|\; |\; |\Delta (B_i)-\Delta (B_j)|\; |\; |}{|\; |\; |\Delta (B_i)+\Delta (B_j)|\; |\; |}.\left(5\right)$$

The distance between the moving blocks will be determined as

$$D_c\left(B_i,B_j\right)=C\left(B_i,B_j\right)\cdot \lfloor \theta \cdot D\left(B_i,B_j\right)+\left(\mathrm{one}-\theta \right)\cdot \Delta \left(B_i,B_j\right)\rfloor$$

where the parameter θ is 0.3. Blocks B _i and B _j are combined in one area if this value is less than the threshold δ _c equal to 0.5 (position 3 in FIG. 1).

3. Temporary clustering of moving blocks

The problem with the segmentation of gaseous objects such as smoke is their discontinuity - at different points in time, initially a single object can consist of several separate areas. In order to identify them as one and the same moving object, we will accompany the moving areas from frame to frame. Then, although the areas that make up one object may not touch each other in any frame, the trajectories of such objects will intersect (position 4 in figure 1). Thus, individual objects can be combined in the time domain.

4. Determining the shape of objects

The clusters obtained in the previous step consist of several blocks. To establish the exact shape of objects, the procedure for selecting edge points is used, for example, according to the Canny method. The resulting boundary points for gaseous objects (smoke) are usually not connected with each other. To construct a closed border from such edges, it is necessary to apply morphological operations: remove very short edges, since they mainly represent the influence of noise, then use the dilatation operation to connect the adjacent edge points, and then smooth the resulting borders to give them a more natural look .

Let X be a binary image of the detected edge points. A pixel value of 1 means that this pixel is an edge point. The combination of morphological operations for constructing the boundary of an object from disparate boundary points is as follows:

$$\left(\left(X\varnothing V_c\right)\oplus V_d\right)\otimes V_b,\left(6\right)$$

where ⊕ is the dilatation operation,

$$\left(X\varnothing V_c\right)=\left\{x:|\; |\; |(X\backslash V_c)\cap V_c|\; |\; |>3\right\},$$

, $$X\otimes V_b=\left\{x:|\; |\; |X\cap V_b|\; |\; |\ge 3\right\}$$

,

, $$V_d=\left(\begin{array}{ccc}0& 1& 0\\ 1& 1& 1\\ 0& 1& 0\end{array}\right)$$

, $$V_b=\left(\begin{array}{ccc}1& 0& 1\\ 0& 1& 0\\ 1& 0& 1\end{array}\right)$$

. $$V_c=\left(\begin{array}{ccc}0& 0& 0\\ 0& \mathrm{one}& 0\\ 0& 0& 0\end{array}\right)$$

, $$V_d=\left(\begin{array}{ccc}0& \mathrm{one}& 0\\ \mathrm{one}& \mathrm{one}& \mathrm{one}\\ 0& \mathrm{one}& 0\end{array}\right)$$

, $$V_b=\left(\begin{array}{ccc}\mathrm{one}& 0& \mathrm{one}\\ 0& \mathrm{one}& 0\\ \mathrm{one}& 0& \mathrm{one}\end{array}\right)$$

.

The next step is to restore the internal area of the object from the resulting border. If areas of low intensity are very long, then even with the help of morphological operations it is impossible to restore the border. This means that the detected boundary will be open and may not fully include the internal region of the object. Therefore, it is necessary to distinguish between areas not covered by the resulting border, and independent individual areas. To do this, find the intersection with the border in horizontal and vertical directions. Only if the horizontal and vertical lines passing through a given pixel intersect the same border, we will consider such a pixel to belong to the internal region of the object (position 5 in figure 1).

5. Classification of selected objects

Classification of selected objects for the determination of smoke and fire is carried out using dynamic textures. One of the main classification methods using dynamic textures is the use of an optical stream, the calculation of which is a rather resource-intensive problem. We will not determine the total optical flux, but only its normal component. This is quite sufficient for the purpose of recognizing the type of an object, and at the same time it will allow avoiding a large amount of computation. Normal flow i.e. the flow in the direction of the intensity gradient ∇I (x, y) is written as follows:

$${\overrightarrow{\upsilon }}_N(x,y)=-\frac{dI/dt}{|\; |\; ||\; |\; |\nabla I|\; |\; ||\; |\; |}\overrightarrow{n},\left(7\right)$$

- единичный вектор в направлении ∇I(x,y). Нормальный поток содержит в себе как временную, так и пространственную информацию о динамических текстурах: временная информация связана с движением краевых точек, а пространственная - с их градиентами.Where $$\overrightarrow{n}$$

is the unit vector in the direction ∇I (x, y). A normal flow contains both temporal and spatial information about dynamic textures: temporal information is associated with the movement of boundary points, and spatial information is associated with their gradients.

In addition to the normal flow, we will also use the maximum value of the directional regularity of textures R (i) as characteristic features for classifying objects, which will allow us to achieve affine invariance of the results. This approach is based on the fact that regular (periodic) textures are perceived as such in a wide range of viewing angles. Therefore, an adequately defined degree of regularity of the objects under consideration can serve as an invariant, physically substantiated characteristic feature. We will use the polar coordinates on the grid α _i = Δα · i, d _j = Δd · j.

. Величина M_R лежит в пределах 0≤M_R≤1, где M_R=0 соответствует текстурам случайного вида, a M_R=1 отвечает текстурам с хорошо выраженной регулярностью; она вычисляется для набора перекрывающихся окон, полностью содержащих область объекта, полученную на предыдущем этапе, и выбирается максимальное значение $$P=\underset{W}{\max }{M}_R\left(W\right)$$

. Здесь через W обозначены окна, размер которых определяется таким образом, чтобы включать в себя как минимум два периода функции контрастности. Так как описанная процедура повторяется для нескольких последовательных кадров, то таким образом мы получаем временную зависимость P(t).We define the maximum value of directional regularity R (i) as $$M_R=\underset{{\alpha }_i}{\max }R\left({\alpha }_i\right)$$

. The value of M _R lies in the range 0≤M _R ≤1, where M _R = 0 corresponds to random textures, and M _R = 1 corresponds to textures with well-defined regularity; it is calculated for a set of overlapping windows that completely contain the area of the object obtained in the previous step, and the maximum value is selected $$P=\underset{W}{\max }{M}_R\left(W\right)$$

. Here, through W, windows are indicated whose size is determined in such a way as to include at least two periods of the contrast function. Since the described procedure is repeated for several consecutive frames, in this way we obtain the temporal dependence P (t).

As the characteristic features of textures, we will use the following values averaged over all frames of the video sequence:

и $$rot{\overrightarrow{\upsilon }}_N\left(x,y\right)$$

.- Average values $$div{\overrightarrow{\upsilon }}_N\left(x,y\right)$$

and $$rot{\overrightarrow{\upsilon }}_N\left(x,y\right)$$

.

- The ratio of the average of the normal flow and its standard deviation.

,- The degree of uniformity of the orientation of the normal flow $$\varphi =\frac{|\; |\; ||\; |\; |{\displaystyle \sum {}_{\left(x,y\right)\in \Omega }{\overrightarrow{\upsilon }}_N\left(x,y\right)}|\; |\; ||\; |\; |}{{\displaystyle \sum {}_{\left(x,y\right)\in \Omega }|\; |\; ||\; |\; |{\overrightarrow{\upsilon }}_N\left(x,y\right)|\; |\; ||\; |\; |}}$$

,

where Ω is the set of pixels for which the normal flux value is nonzero.

- The average value of P (t).

- Dispersion P (t).

All indicated quantities are invariant to displacement and rotation. The belonging of the object in question to the class of objects corresponding to smoke in the image can be determined using some classification method, for example, the method of reference vectors or neural networks (position 6 in figure 1).

Claims (3)

1. The method of video surveillance of open space with fire hazard control, according to which the protected area is video-monitored, moving areas are selected on the received image, these areas are compared with model images from the image library and a decision is made on fire based on the similarity of the received and available images, characterized in that the selection in the image of moving objects is done by dividing the entire image into rectangular blocks, the displacement of the blocks with time they are predicted by the displacement of the center of the block, individual moving blocks are combined by spatial clustering, temporary clustering of moving blocks is carried out according to the criterion for the intersection of the trajectories of the individual blocks with each other, and the shape of the moving object is restored by cutting off short edges, connecting adjacent edge points and smoothing the obtained boundaries, while the inner region of the object is restored according to the obtained boundary, and the classification of the selected moving object is carried out according to normal delivering optical flow. 2. The method according to claim 1, characterized in that the library of images is formed with the allocation of the categories of "vehicles", "people", "animals", "swaying tree branches", "smoke", "thermal footprint", "open fire" . 3. The method according to claims 1 and 2, characterized in that in each of the categories of image libraries form subcategories "trajectory", "speed", "direction of movement" and tie the resulting images to the characteristic features of the protected area.

Images