EP1762995A1 - Detection of smoke with a video camera - Google Patents

Abstract

The method involves examination of the moving area of at least one video image by determining the direction and the size of the moving area at the probable existence of the smoke. After positive inspection result, a part of the moving area is evaluated, depending on the information characterized for smoke regarding the existence of smoke. An independent claim is also included for the smoke detection device.

Description

The invention relates to a method and apparatus for detecting smoke by analyzing at least one video image taken by a video camera monitoring an area.

Efforts have recently been made to use the video systems for the detection of smoke which are already present for security monitoring in buildings, tunnels, etc. Since the video images for a viewer are often uninteresting and also caused by smoke only very small changes in the video image, monitoring by the staff at the screens is out of the question. If anything, the monitoring can only be done by an automatic evaluation of the video images. In a known method for the automatic examination of video images for the occurrence of smoke, the intensity values of the individual pixels of successive images are compared with one another. When measuring intensity values representative of a lighter image caused by the presence of smoke, the presence of smoke is signaled and an alarm is sounded.

One of the problems with this method is that smoke is not detected against a light background and even fire that produces little smoke is not detected. In addition, brightness changes, such as those caused by persons moving through the field of view of the camera, can trigger a false alarm. This problem has been solved by the fact that in addition to the actual surveillance area still one outer area and, in case of changes in this outer area, interrupt the observation of the surveillance area. This method has the disadvantage that a fire may not be detected until after a certain delay and that sources of smoke are not detected in the outer area provided in addition to the monitoring area.

The object of the present invention is to propose an efficient way of detecting smoke by means of at least one video image taken by a field-monitoring video camera.

The object is achieved in each case by the subject matters of the independent claims. Further developments of the invention are specified in the subclaims.

A gist of the invention is that smoke is detected by analyzing at least one video image captured by a video camera monitoring an area. An area can be a room, a tunnel (section), a parking lot, a street or a street section etc. In principle, in a first step, by determining the direction and the size in a moving region of the at least one video image, a probable presence of smoke in the moving region is checked. If a moving area has a positive test result, there is a certain probability of the presence of smoke. Thereafter, at least a part of the moving area is evaluated depending on at least one smoke characteristic information regarding the presence of smoke. According to the invention, the smoke-characteristic information is the speed of the smoke, the number of pixels in the video image that describe this movement, the luminance change (brightness change) of the at least one video image with respect to the background, the change in the color of the moving smoke and the movement of the smoke are considered.

An advantage of the method according to the invention or the device according to the invention can be seen in the fact that smoke can be detected efficiently. In particular, this is achieved by the two-part evaluation and by the appropriate selection of information characteristic of smoke.

Figur 1

ein erfindungsgemässes Blockschema zur Detektion von Rauch,

Figur 2

eine vereinfachte Darstellung eines Videobildes,

Figur 3

ein Entscheidungsdiagramm für die Detektion von Rauch,

Figur 4

eine erfindungsgemässe Vorrichtung.

The invention will be explained in more detail with reference to an embodiment shown in a figure. Show

FIG. 1

an inventive block diagram for the detection of smoke,

FIG. 2

a simplified representation of a video image,

FIG. 3

a decision chart for the detection of smoke,

FIG. 4

an inventive device.

FIG. 1 shows a block diagram according to the invention for the detection of smoke. From at least one video image, which was generated with a certain frequency, at least one intensity image [X _ij (t)] is obtained. The video image can have, for example, a size of 352x288 pixels. The next step is preprocessing. The preprocessing has the goal that the areas of interest for the detection of smoke are filtered out of the video image. For this purpose, a background accumulation matrix [B _ij (t)] is created first. The background accumulation matrix [B _ij (t)] is obtained from the weighting factor weighted intensity images [X _ij (t)], where the weighting factor α indicates how strongly the intensity _images flow into the accumulation matrix [B _ij (t)].
The accumulation matrix is determined as follows: $$\mathrm{bij}\left(\mathrm{t}\right)\mathrm{=}\mathbf{\alpha }\begin{array}{}\end{array}\mathrm{bij}\left(\mathrm{t}\mathrm{-}\mathrm{1}\right)\mathrm{+}\left(\mathrm{1}\mathrm{-}\mathrm{\alpha }\right)\mathrm{xij}\left(\mathrm{t}\right)\mathrm{.}\mathbf{\alpha }\mathrm{=}\mathrm{weighting\; factor}$$

Next, a subtraction matrix D _ij ( t ) = / B _ij ( t ) - X _ij ( t ) / is calculated for at least one moving area. The color weighting of the subtraction matrix D _ij ( t ) finally yields the color-weighted subtraction matrix [S _ij (t)].

, wobei Luma{Dij} der Helligkeitsteil von Dij, ChromaU(Dij) der U-Farbanteil von Dij and ChromaU(Dij) der V-Farbanteil von Dij ist.This subtraction matrix [S _ij (t)] is calculated from $${\mathrm{S}}_{\mathrm{ij}}\left(\mathrm{t}\right)\mathrm{=}\mathrm{Luma}\left\{{\mathrm{D}}_{\mathrm{ij}}\left(\mathrm{t}\right)\right\}\mathrm{\times }\left\{\mathrm{1}\mathrm{-}\left|\mathrm{ChromaU}\mathrm{\left\{}{\mathrm{D}}_{\mathrm{ij}}\left(\mathrm{t}\right)\mathrm{\right\}}\mathrm{-}\mathrm{ChromaV}\mathrm{\left\{}{\mathrm{D}}_{\mathrm{ij}}\left(\mathrm{t}\right)\mathrm{\right\}}\right|\right\}$$

where Luma {Dij} is the luminance part of Dij, ChromaU (Dij) is the U color part of Dij and ChromaU (Dij) is the V color part of Dij.

The likely presence of smoke at location (i, j) is finally determined, for example, by the projection of the color-weighted subtraction matrix [S _ij (t)] on the x / y axis of a Cartesian coordinate system.

$$\mathrm{x}-\mathrm{Projektion\; von\; Sij}\left(\mathrm{t}\right):{\mathrm{p}}_{\mathrm{xi}}\left(\mathrm{t}\right)={\mathrm{S}}_{\mathrm{i}0}\left(\mathrm{t}\right)+{\mathrm{S}}_{\mathrm{i}1}\left(\mathrm{t}\right)+{\mathrm{S}}_{\mathrm{i}2}\left(\mathrm{t}\right)+\mathrm{...}+{\mathrm{S}}_{\mathrm{iV}}\left(\mathrm{t}\right)$$

$$\mathrm{y}-\mathrm{Projektion\; von\; Sij}\left(\mathrm{t}\right):{\mathrm{p}}_{\mathrm{yi}}\left(\mathrm{t}\right)={\mathrm{S}}_{0\mathrm{j}}\left(\mathrm{t}\right)+{\mathrm{S}}_{1\mathrm{j}}\left(\mathrm{t}\right)+{\mathrm{S}}_{2\mathrm{j}}\left(\mathrm{t}\right)+\mathrm{...}+{\mathrm{S}}_{\mathrm{Hj}}\left(\mathrm{t}\right)$$

The projection on a Cartesian coordinate system looks like this: $$\begin{array}{cc}\mathrm{[}{\mathrm{i}}_{\mathrm{m}}\mathrm{.}{\mathrm{j}}_{\mathrm{m}}\mathrm{]}\left(\mathrm{t}\right)\mathrm{=}\mathrm{\left\{}\left(\mathrm{i},,\mathrm{j}\right)\mathrm{\right|}& \mathrm{i}\mathrm{=}\mathrm{Max}\left\{\mathrm{x}\mathrm{-}\mathrm{projection\; of}{\mathrm{S}}_{\mathrm{ij}}\left(\mathrm{t}\right)\right\}\mathrm{.}\\ & \mathrm{j}\mathrm{=}\mathrm{Max}\left\{\mathrm{y}\mathrm{-}\mathrm{projection\; of}{\mathrm{S}}_{\mathrm{ij}}\left(\mathrm{t}\right)\mathrm{\}}\right\}\end{array}$$

$$\mathrm{x}-\mathrm{Projection\; by\; Sij}\left(\mathrm{t}\right):{\mathrm{p}}_{\mathrm{xi}}\left(\mathrm{t}\right)={\mathrm{S}}_{\mathrm{i}0}\left(\mathrm{t}\right)+{\mathrm{S}}_{\mathrm{i}1}\left(\mathrm{t}\right)+{\mathrm{S}}_{\mathrm{i}2}\left(\mathrm{t}\right)+\mathrm{...}+{\mathrm{S}}_{\mathrm{iV}}\left(\mathrm{t}\right)$$

$$\mathrm{y}-\mathrm{Projection\; by\; Sij}\left(\mathrm{t}\right):{\mathrm{p}}_{\mathrm{yi}}\left(\mathrm{t}\right)={\mathrm{S}}_{0\mathrm{j}}\left(\mathrm{t}\right)+{\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="imgf0001.png"\; state="\{\{state\}\}">S</figure-callout>}}_{1\mathrm{j}}\left(\mathrm{t}\right)+{\mathrm{S}}_{2\mathrm{j}}\left(\mathrm{t}\right)+\mathrm{...}+{\mathrm{S}}_{\mathrm{hj}}\left(\mathrm{t}\right)$$

Sij in this example has the magnitude HxV (H = smoke velocity x smoke movement = V). Of course, the choice of the coordinate system is arbitrary. For example, spherical coordinates, cylindrical coordinates, etc. could also be used.

With the aid of the color-weighted subtraction matrix [S _ij (t)], a probable presence of smoke in a moving area of the video image can then be checked. In the case of a likely presence of smoke, a reduced video image area of interest (ROI = Region of Interest) compared to the original image is defined. Of course, multiple ROI areas can also be defined in one video image or in multiple channels. By reducing the data to about 1: 100, for example, the size of the ROI can be 8x128 pixels, the processor load for the actual analysis or evaluation is considerably reduced. Whether smoke is present in a moving area of the recorded video image is clarified on the basis of at least one information characteristic of smoke. In the present example, the following information is used to increase the detection security.

As information characteristic of smoke, the speed of the smoke (movement of the smoke), the number of pixels (active pixels) describing this movement, the luminance change (brightness change) of the at least one video image with respect to the background, the change of the color ( Color change) of the moving smoke and the movement of the smoke (y-position in the histogram) viewed.

• die Rauchbewegung von SROI(t): v(t) = Zeitkorrelation der γ-Projektion von SROI(t), zum Beispiel pyj(t),
• die Varianz von BROI(t) und XROI(t), die zur Bestimmung der Helligkeitsveränderung relativ zum (normalen) Hintergrund: 1(t) = 1 - var{B_ROI(t)}/var{B_ROI(t)},
• aktive Pixel von S_ROI(t): a(t) = Anzahl der Pixel von S_ROI(t) mit einem Wert grösser als 0,
• Farbwechsel: c(t) = Anzahl der Pixel mit {1-|ChromaU(D_ROI(t)) - ChromaV(D_ROI(t)|} < Schwellwert,
• y-Position im Histogramm: h(t) = Werte der y-Projektion von S_ROI(t), zum Beispiel p_yj(t) wird genutzt um ein Histogramm mit 64 Kanälen zu erstellen

For each ROI range, the following information characteristic of smoke is calculated:

• the smoke movement of SROI (t): v (t) = time correlation of the γ-projection of SROI (t), for example, pyj (t),
• the variance of BROI (t) and XROI (t) for determining the change in brightness relative to the (normal) Background: 1 (t) = 1 - var {B _ROI (t)} / var {B _ROI (t)},
• active pixels of S _ROI (t): a (t) = number of pixels of S _ROI (t) with a value greater than 0,
• Color change: c (t) = number of pixels with {1- | ChromaU (D _ROI (t)) - ChromaV (D _ROI (t) |} <threshold,
• y-position in the histogram: h (t) = values of the y-projection of S _ROI (t), for example p _yj (t) is used to create a histogram with 64 channels

Thereafter, the smoke-characteristic information v (t), l (t), a (t), c (t) and h (t) are integrated over a certain time and thus over several images. The function looks like this, for example: $$F_{\mathit{X}}=\underset{{\mathrm{t}}_{\mathrm{0}}\mathrm{<}\mathrm{t}\mathrm{<}{\mathrm{t}}_{\mathrm{0}}}{X}=\mathbf{\Sigma }\begin{array}{}\end{array}\mathrm{x}\left(\mathrm{t}\right)\mathrm{with\; X}=\mathrm{V}\mathrm{.}\mathrm{L}\mathrm{.}\mathrm{A}\mathrm{.}\mathrm{C}\mathrm{.}\mathrm{H}$$

Mittelwert Rauchbewegung

F_V = V

Mittelwert Helligkeitswechsel

F_L = L

Mittelwert aktive Pixel

F_A = A

Mittelwert Farbwechsel

F_C = C

Mittelwert y-Position im Histogramm

F_H = H

From the information integrated over time, the respective mean value is determined.

Mean smoke movement

F _V = V

Mean value brightness change

F _L = L

Mean active pixels

F _A = A

Mean color change

F _C = C

Mean y position in the histogram

F _H = H

Thereafter, the probability of the presence of smoke is calculated for each of these averages. this happens about the pattern recognition. For each average, a discriminator value Ψ is determined. For example, a threshold δ (or even a probability function) may define the discriminator in the following manner:

oder 0 ≤ Γ(F _L ) ≤ 1, mit Γ(x) als WahrscheinlichkeitsfunktionFor the brightness change $${\mathbf{\Psi }}_L=\mathbf{\Gamma }\left(F_L\right)=\{\begin{array}{c}{\mathrm{F}}_L>{\mathbf{\delta }}_L.\mathrm{then}{\mathbf{\Psi }}_L=1\\ {\mathrm{F}}_L>{\mathbf{\delta }}_L.\mathrm{then}{\mathbf{\Psi }}_L=0\end{array}$$

or 0 ≤ Γ (F _L ) ≤ 1, with Γ (x) as the probability function

oder als Mittelwert aller Diskriminatoren $$\begin{array}{cc}\mathrm{\aleph }\left(\mathrm{t}\right)=1/{\mathrm{N}}_{\mathrm{F}}& \Sigma {\mathbf{\Psi }}_{\mathrm{i}}=\left\{{\mathbf{\Psi }}_V+{\mathbf{\Psi }}_L+{\mathbf{\Psi }}_A+{\mathbf{\Psi }}_C+{\mathbf{\Psi }}_H\right\}/{\mathrm{N}}_{\mathrm{F}}\hfill \\ & \mathrm{i}\mathrm{=}\mathrm{Information}\hfill \end{array}$$

, wobei N_F = 5 die Anzahl der Informationen ist.The smoke pattern is defined by the product of all discriminators $$\begin{array}{cc}\mathrm{\aleph }\left(\mathrm{t}\right)=& \mathbf{\Pi }{\mathbf{\Psi }}_i=\left\{{\mathbf{\Psi }}_V\cdot {\mathbf{\Psi }}_L\cdot {\mathbf{\Psi }}_A\cdot {\mathbf{\Psi }}_C\cdot {\mathbf{\Psi }}_H\right\}\hfill \\ & \mathrm{i}\mathrm{=}\mathrm{information}\hfill \end{array}$$

or as the mean of all discriminators $$\begin{array}{cc}\mathrm{\aleph }\left(\mathrm{t}\right)=1/{\mathrm{N}}_{\mathrm{F}}& \Sigma {\mathbf{\Psi }}_{\mathrm{i}}=\left\{{\mathbf{\Psi }}_V+{\mathbf{\Psi }}_L+{\mathbf{\Psi }}_A+{\mathbf{\Psi }}_C+{\mathbf{\Psi }}_H\right\}/{\mathrm{N}}_{\mathrm{F}}\hfill \\ & \mathrm{i}\mathrm{=}\mathrm{information}\hfill \end{array}$$

where N _F = 5 is the number of pieces of information.

falls ℵ(t) = 1

dann wird I(t) = I(t-1) + σ₊
(hinzugefügt zu S₊ falls I (t) > S₊)
sonst I(t) = I(t-1) - σ_-
(hinzugefügt zu S_- (üblicherweise 0) falls
I(t) < S_-)

, wobei σ+,σ- üblicherweise den Wert +1 annimmtFinally, the decision is made as to whether the moving area of the video image is the image of smoke. For this purpose, an integrator I (t), which increases or decreases by a value σ, is determined $$\mathrm{I}\left(\mathrm{t}=0\right)=0;$$

if ℵ (t) = 1

then I (t) = I (t-1) + σ ₊
(added to S ₊ if I (t)> S ₊ )
otherwise I (t) = I (t-1) - σ _-
(added to S _- (usually 0) if
I (t) <S _- )

, where σ +, σ- usually takes the value +1

Falls I(t) > κ

dann Rauch
sonst kein Rauch

Smoke is detected and, for example, an alarm is triggered if I (t) exceeds a critical value κ:

If I (t)> κ

then smoke
otherwise no smoke

FIG. 2 shows a simplified representation of a video image VB. The image contains a moving area that is supposed to be smoke. Furthermore, the video image VB shows an ROI range, which was determined according to the description of FIG.

Figure 3 shows a decision diagram for the detection of smoke, as described under Figure 1. If I (t) exceeds a certain threshold κ, alarm is triggered and smoke was detected with high probability. So that I (t) does not rise to infinity and thus unnecessarily reduces the reaction time for smoke detection, a maximum value I _{T is} defined. The critical time is the time until the alarm is triggered. This time should be as short as possible.

FIG. 4 shows a device VR according to the invention with a receiving unit E and a transmitting unit S for communicating, for example, with other units, such as sensors, central units, etc., and a processing unit V for carrying out the method according to FIG. 1. The apparatus can be used in a video camera, a Central unit etc. be integrated or represent a separate unit.

Claims (15)

A method of detecting smoke by analyzing at least one video image taken by a video camera monitoring an area,
characterized,
that at least one moving area of the at least one video image is checked for the likely presence of smoke by determining the direction and size of the moving area, and
that, depending upon a positive test result, at least a portion of at least a moving range at least one is evaluated for smoke characteristic information as to the presence of smoke. Method according to claim 1,
characterized,
that as at least one smoke characteristic information, the speed of the smoke, the number of pixels describing this movement, the luminance change of the at least one video image with respect to the background, the change of the color of the moving smoke and the movement of the smoke are used. Method according to one of the preceding claims,
characterized,
that generates at least one video image with a specific frequency and from this at least one intensity image [X _ij (t)] is obtained. Method according to claim 3,
characterized,
that a background accumulation matrix [B _ij (t)], which is obtained from the weighting factor-weighted intensity images [X _ij (t)], wherein the weighting factor indicates how much the intensity images in the accumulation matrix [B _ij (t)] to be used. Method according to claim 4,
characterized,
in that at least one moving region is determined with the aid of a subtraction matrix D _ij ( t ) = / B _ij ( t ) _{-X ij} ( t ) /. Method according to claim 5,
characterized,
that a color-weighted subtraction matrix [S _ij (t)] is determined from the subtraction matrix [D _ij (t)]. Method according to claim 6,
characterized,
that with the aid of the color-weighted subtraction matrix [S _ij (t)] a probable presence of smoke in a moving area of the video image is checked and in the presence of a positive test result a reduced video image area of interest (ROI) compared to the original image is defined. Method according to claim 7,
characterized,
in that the video image area of interest (ROI) represents at least a part of the moving area of the video image. Method according to claim 7,
characterized,
that the video image area of interest (ROI) is represented as a rectangle with the length / width ratio 16: 1. Method according to claim 6,
characterized,
that the probable presence of smoke at the location (i, j) is determined by the projection of the color-weighted subtraction matrix [S _ij (t)] on the x- / y-axis of a Cartesian coordinate system. Method according to one of the preceding claims,
characterized,
that the at least one characteristic for smoke is evaluated in the information of interest video image area. Method according to claim 11,
characterized,
that the at least one named information characteristic of smoke is integrated over a certain time and thus over several images and its mean value is determined and that for each of these mean values the probability of the presence of smoke is calculated. Method according to claim 12,
characterized,
that the probability of whether smoke is present δ by the comparison with a threshold value and / or by a probability function Γ (x) is determined. Method according to claims 12 and 13,
characterized,
that an overall probability for the presence of smoke is computed in the relevant video image area from the probabilities of the average values, that this overall probability is integrated over a plurality of images, and that when exceeding a threshold (κ) is triggered by the integrated value alarm. Device (VR) for detecting smoke by analyzing at least one video image taken by a field-monitoring video camera, with a receiving unit (E) and a transmitting unit (S) for carrying out the communication with further units, - With a processing unit (V) for checking the probable presence of smoke by determining the direction and the size of a moving portion of the at least one video image and in a positive test result for evaluating at least a portion of the at least one moving region depending on at least one of smoke characteristic information regarding the presence of smoke.

Images