EP1496483B1 - Method and apparatus for the detection of flames - Google Patents

Method and apparatus for the detection of flames Download PDF

Info

Publication number
EP1496483B1
EP1496483B1 EP03015846A EP03015846A EP1496483B1 EP 1496483 B1 EP1496483 B1 EP 1496483B1 EP 03015846 A EP03015846 A EP 03015846A EP 03015846 A EP03015846 A EP 03015846A EP 1496483 B1 EP1496483 B1 EP 1496483B1
Authority
EP
European Patent Office
Prior art keywords
images
image
roi
flame
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP03015846A
Other languages
German (de)
French (fr)
Other versions
EP1496483A1 (en
Inventor
Giuseppe Dr. Marbach
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens Schweiz AG
Original Assignee
Siemens Schweiz AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to ES03015846T priority Critical patent/ES2282550T3/en
Application filed by Siemens Schweiz AG filed Critical Siemens Schweiz AG
Priority to AT03015846T priority patent/ATE357714T1/en
Priority to EP03015846A priority patent/EP1496483B1/en
Priority to DE50306852T priority patent/DE50306852D1/en
Priority to AU2004202851A priority patent/AU2004202851B2/en
Priority to KR1020040050227A priority patent/KR20050009135A/en
Priority to PL04369016A priority patent/PL369016A1/en
Priority to NO20042948A priority patent/NO330182B1/en
Priority to CN200410063587A priority patent/CN100595583C/en
Publication of EP1496483A1 publication Critical patent/EP1496483A1/en
Application granted granted Critical
Publication of EP1496483B1 publication Critical patent/EP1496483B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00Fire alarms; Alarms responsive to explosion
    • G08B17/12Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00Fire alarms; Alarms responsive to explosion
    • G08B17/12Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
    • G08B17/125Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke

Definitions

  • the present invention relates to a method and a device for detecting flames in a monitored area referred to below as a monitoring space by analyzing at least one parameter of a radiation occurring in the interstitial space.
  • WO 02/093525 A1 discloses an apparatus and a method for the simultaneous processing of first and second image processing, especially for the detection of flames on the images.
  • the apparatus includes an image sensor for creating video images, a frame grabber for receiving a first and a second image (frame), a processor for processing the captured images, and an output unit.
  • Previously known devices as described for example in US Pat. Nos. 4,866,420 and 4,280,058, contain at least one sensor which evaluates the flicker frequency spectrum of the radiation, whereby signals located outside a specific frequency band are evaluated as interference signals.
  • the typical flickering of the flames in a very low-frequency oscillation range as a feature for distinguishing between the radiation emitted by a flame and interfering radiation.
  • the determination of the frequency band takes place in the simplest case by the sensor upstream filter or by this downstream frequency-selective amplifier, in both cases, a certain passband of, for example, 5 to 25 Hz is obtained.
  • the invention will now be given a method for the detection of flames, which is characterized by a high noise immunity at low cost and.
  • the search for regions of high light intensity and local flickering movement takes place with the aid of an accumulation matrix which is obtained from the weighting factor-weighted difference images of successive intensity images, wherein the weighting factor indicates how much the accumulation matrix is accumulated on the difference images.
  • a first preferred embodiment of the method according to the invention is characterized in that the video images are generated at a specific frequency and intensity images are obtained therefrom.
  • a second preferred embodiment of the method according to the invention characterized in that the coordinates of the brightest pixels are searched with the aid of the accumulation matrix.
  • a third preferred embodiment is characterized in that an interesting image area containing the brightest or the brightest pixels and reduced in relation to the original image area is defined and analyzed for the presence of a flame.
  • the reference numeral 1 denotes a video camera which has video output via an output to an evaluation stage 2 supplies, wherein the evaluation stage 2 can be integrated into the camera 1 or connected to this.
  • the evaluation level 2 can be provided at the site of the camera 1 or in the immediate vicinity of this or it can also be spatially distant from the camera 1, wherein in the latter case between camera 1 and evaluation level 2 is a communication link.
  • the evaluation stage 2 contains a processor (not shown), which has an algorithm for the localization of flames found in the images of the camera 1 and the subsequent analysis of the corresponding image detail.
  • a processor not shown
  • the extraction of intensity and / or chrominance images X ij (t) (hereinafter referred to as intensity images) from the video sequences supplied by the camera 1 takes place in a first process of the algorithm designated image acquisition; i and j are the coordinates of each pixel.
  • the frequency of these pictures is at least 15 frames per second, the picture size for example 352 times 288 pixels. Intensity images are obtained because it can be assumed that a flame is a place of high light intensity and also has a characteristic hue.
  • a difference image is formed by comparing successive images to find out such a movement; since dark objects fall out due to the definition of the weighting factor w ij (t), moving, dark objects that can not be flames are filtered out when forming the difference image. So you are looking for areas of high light intensity and local flickering and that means that, for example, a stationary light source that does not flicker would not be interpreted as a flame and also not a transversely across the interstitial space moving lamp.
  • a ij (t) ⁇ ⁇ A ij ⁇ t - 1 + 1 - ⁇ ⁇ Q ij t
  • is a constant between 0 and 1, which indicates how much the difference image Q ij (t) flows into the accumulation matrix A ij (t) .
  • the accumulation matrix is formed primarily to obtain a smoothed image with no noise and short-term changes.
  • This determination thus provides the coordinates of the pixels with local flickering and maximum brightness. It will usually be a single pixel, but of course also several brightest pixels can be determined, what you can use a multi-channel selection and preferably specifies minimum distances between the individual pixels.
  • a significant data reduction first takes place in that the analysis does not take place in the entire original image of 352 by 288 pixels but in the image region ROI of interest having a reduced size of, for example, 32 by 32 pixels. This results in a reduction to one hundredth. Of course, this reduction can also be lower, for example, to a fiftieth, or even much stronger.
  • the mean value of the frequency is obtained, for example, by counting the pixels of the average brightness L (t) .
  • the frequency caused by the characteristic flickering of a flame is an important quantity for the detection of a flame because it is within a defined narrow range of usually between 1 Hz and 10 Hz.
  • the probability that a flame is present is calculated from the properties obtained during extraction 6. In this case, for example, it is examined for each of the above properties whether the mean value is above or below a threshold value and the probability is set equal to one or equal to zero. Then, from the probabilities of all n properties, a total probability is formed.
  • N F is the sum of w i over all i.
  • the described device has the advantage that can be used in many applications on already installed video cameras and installation of special flame sensors is not required, which undoubtedly means a cost reduction.
  • a further cost reduction results from the restriction of the evaluation to the possibly containing a flame image sections, which allows a significant reduction of computer power. It can also be assumed that the evaluation of these image sections is sufficiently robust to interference.

Abstract

A flame detection procedure (2) analyses a video (1) image (3) for bright spots and moving areas using a weighted accumulation matrix (4) constructed from the differences between successive images with low intensity and so dark moving objects filtered (5) out to leave small areas of interest for integration (6) and flame presence probability (7) estimation (8). Includes INDEPENDENT CLAIMs for equipment using the procedure.

Description

Die vorliegende Erfindung betrifft ein Verfahren und eine Einrichtung zur Detektion von Flammen in einem nachfolgend als Überwachungsraum bezeichneten überwachten Gebiet durch Analyse mindestens eines Parameters einer im Überwachungsraum auftretenden Strahlung.The present invention relates to a method and a device for detecting flames in a monitored area referred to below as a monitoring space by analyzing at least one parameter of a radiation occurring in the interstitial space.

Die WO 02/093525 A1 offenbart eine Vorrichtung und ein Verfahren zum gleichzeitigen Verarbeiten einer ersten und zweiten Bildverarbeitung, speziell zur Detektion von Flammen auf den Bildern. Die Vorrichtung beinhaltet einen Bildsensor zum Erstellen von Videobildern, eine Bildfangschaltung zum Empfangen von einem ersten und einem zweiten Bild (Frame), einen Prozessor zum Verarbeiten der aufgenommenen Bilder und eine Ausgabeeinheit.WO 02/093525 A1 discloses an apparatus and a method for the simultaneous processing of first and second image processing, especially for the detection of flames on the images. The apparatus includes an image sensor for creating video images, a frame grabber for receiving a first and a second image (frame), a processor for processing the captured images, and an output unit.

Bisher bekannte Einrichtungen, wie sie beispielsweise in der US-Patenten Nr. 4 866 420 und Nr. 4 280 058 beschrieben sind, enthalten mindestens einen Sensor, welcher das Flackerfrequenzspektrum der Strahlung auswertet, wobei ausserhalb eines bestimmten Frequenzbandes liegende Signale als Störsignale bewertet werden. Man benutzt also das typische Flackern der Flammen in einem sehr niederfrequenten Schwingungsbereich als Merkmal zur Unterscheidung zwischen der von einer Flamme ausgesandten Strahlung und Störstrahlung. Die Festlegung des Frequenzbandes erfolgt im einfachsten Fall durch dem Sensor vorgeschaltete Filter oder durch diesem nach-geschaltete frequenzselektive Verstärker, wobei in beiden Fällen ein bestimmter Durchlassbereich von beispielsweise 5 bis 25 Hz erhalten wird.Previously known devices, as described for example in US Pat. Nos. 4,866,420 and 4,280,058, contain at least one sensor which evaluates the flicker frequency spectrum of the radiation, whereby signals located outside a specific frequency band are evaluated as interference signals. Thus one uses the typical flickering of the flames in a very low-frequency oscillation range as a feature for distinguishing between the radiation emitted by a flame and interfering radiation. The determination of the frequency band takes place in the simplest case by the sensor upstream filter or by this downstream frequency-selective amplifier, in both cases, a certain passband of, for example, 5 to 25 Hz is obtained.

Diese bekannten Flammenmelder haben sich durchaus bewährt, sie stellen aber in einer Brandmeldeanlage einen nicht unbeträchtlichen Kostenfaktor dar. Abgesehen davon, können auch bei optimaler Abstimmung des Frequenzbandes auf das Flackern von Flammen Störungen und Fehlanzeigen nicht ausgeschlossen werden, weil es immer wieder vorkommt, dass zufällige Intensitätsänderungen der Umgebungsstrahlung im Durchlassbereich liegen. Solche Intensitätsänderungen können beispielsweise durch Abschattungen oder Reflexe von vibrierenden oder sich langsam bewegenden Gegenständen, durch Reflexe des Sonnenlichts an Wasseroberflächen oder durch flackernde oder schwankende Lichtquellen verursacht sein.Apart from that, even with optimal tuning of the frequency band to the flickering of flames malfunctions and false positives can not be ruled out because it happens again and again that accidental Intensity changes of the ambient radiation are in the passband. Such changes in intensity may be caused, for example, by shadows or reflections from vibrating or slowly moving objects, reflections of the sunlight on water surfaces, or flickering or fluctuating light sources.

Durch die Erfindung soll nun ein Verfahren zur Detektion von Flammen angegeben werden, welches sich durch eine hohe Störsicherheit bei geringen Kosten und auszeichnet.The invention will now be given a method for the detection of flames, which is characterized by a high noise immunity at low cost and.

Diese Aufgabe wird gemäss der Ansprüche 1 und 10 gelöst.This object is achieved according to claims 1 and 10.

Nach dem erfindungsgemässen Verfahren erfolgt die Suche nach Gebieten von hoher Lichtintensität und lokaler Flackerbewegung mit Hilfe einer Akkumulationsmatrix, welche aus den mit einem Gewichtungsfaktor gewichteten Differenzbildern aufeinanderfolgender Intensitätsbilder gewonnen wird, wobei der Gewichtungsfaktor angibt, wie stark die Akkumulationsmatrix an die Differenzbilder akkumuliert wird.According to the method according to the invention, the search for regions of high light intensity and local flickering movement takes place with the aid of an accumulation matrix which is obtained from the weighting factor-weighted difference images of successive intensity images, wherein the weighting factor indicates how much the accumulation matrix is accumulated on the difference images.

Eine erste bevorzugte Ausführungsform des erfindungsgemässen Verfahrens ist dadurch gekennzeichnet, dass die Videobilder mit einer bestimmten Frequenz erzeugt und daraus Intensitätsbilder gewonnen werden.A first preferred embodiment of the method according to the invention is characterized in that the video images are generated at a specific frequency and intensity images are obtained therefrom.

Eine zweite bevorzugte Ausführungsform des erfindungsgemässen Verfahrens dadurch gekennzeichnet, dass mit Hilfe der Akkumulationsmatrix die Koordinaten der hellsten Pixel gesucht werden.A second preferred embodiment of the method according to the invention, characterized in that the coordinates of the brightest pixels are searched with the aid of the accumulation matrix.

Eine dritte bevorzugte Ausführungsform ist dadurch gekennzeichnet, dass ein das hellste oder die hellsten Pixel enthaltender und gegenüber dem ursprünglichen Bildbereich reduzierter, interessierender Bildbereich definiert und auf das Vorhandensein einer Flamme analysiert wird.A third preferred embodiment is characterized in that an interesting image area containing the brightest or the brightest pixels and reduced in relation to the original image area is defined and analyzed for the presence of a flame.

Weitere bevorzugte Ausführungsformen des erfindungsgemässen Verfahrens ergeben sich aus den abhängigen Ansprüchen 7 bis 9.Further preferred embodiments of the method according to the invention emerge from the dependent claims 7 to 9.

Bevorzugte Ausführungsformen der erfindungsgemässen Einrichtung sind in den abhängigen Ansprüchen 11 bis 15 beansprucht.Preferred embodiments of the device according to the invention are claimed in the dependent claims 11 to 15.

Mit der immer stärkeren Verbreitung von CCTV-Systemen und -Anlagen kann man davon ausgehen, dass in vielen Fällen in einem Überwachungsraum eine Videokamera vorhanden sein wird, so für die Flammendetektion nicht ein eigener Sensor installiert werden muss, was zweifellos eine Kostenreduktion bedeutet. Eine weitere Kostenreduktion ergibt sich durch die Beschränkung der Auswertung auf die eventuell eine Flamme enthaltenden Bildausschnitte, was eine deutliche Reduktion der Rechnerleistung ermöglicht. Man kann auch davon ausgehen, dass die Auswertung dieser Bildausschnitte gegenüber Störungen ausreichend robust ist.With the proliferation of CCTV systems and systems, it can be assumed that in many cases a video camera will be present in a surveillance room, so that a separate sensor does not have to be installed for flame detection, which undoubtedly means a cost reduction. A further cost reduction results from the restriction of the evaluation to the possibly containing a flame image sections, which allows a significant reduction of computer power. It can also be assumed that the evaluation of these image sections is sufficiently robust to interference.

Im Folgenden wird die Erfindung anhand eines eine erfindungsgemässe Einrichtung zur Detektion von Flammen zeigenden Blockschemas beispielsweise näher erläutert. Mit dem Bezugszeichen 1 ist eine Videokamera bezeichnet, welche über einen Ausgang Videosequenzen an eine Auswertestufe 2 liefert, wobei die Auswertestufe 2 in die Kamera 1 integriert oder mit dieser verbunden sein kann. Die Auswertestufe 2 kann am Aufstellungsort der Kamera 1 oder in unmittelbarer Nähe zu diesem oder sie kann auch räumlich entfernt von der Kamera 1 vorgesehen sein, wobei im letzteren Fall zwischen Kamera 1 und Auswertestufe 2 eine Kommunikationsverbindung besteht.In the following, the invention is explained in more detail by way of example with reference to a block diagram showing a device according to the invention for detecting flames. The reference numeral 1 denotes a video camera which has video output via an output to an evaluation stage 2 supplies, wherein the evaluation stage 2 can be integrated into the camera 1 or connected to this. The evaluation level 2 can be provided at the site of the camera 1 or in the immediate vicinity of this or it can also be spatially distant from the camera 1, wherein in the latter case between camera 1 and evaluation level 2 is a communication link.

Die Auswertestufe 2 enthält einen Prozessor (nicht dargestellt), welcher einen Algorithmus für die Lokalisierung von in den Bildern der Kamera 1 gefundenen Flammen und die nachfolgende Analyse der entsprechenden Bildausschnitte aufweist. Darstellungsgemäss erfolgt in einem mit Bildgewinnung bezeichneten ersten Prozess des Algorithmus die Gewinnung von Intensitäts- und/oder Chrominanzbildern Xij(t) (nachfolgend als Intensitätsbilder bezeichnet) aus den von der Kamera 1 gelieferten Videosequenzen; i und j sind die Koordinaten der einzelnen Pixel. Die Frequenz dieser Bilder beträgt mindestens 15 Bilder pro Sekunde, die Bildgrösse beispielsweise 352 mal 288 Pixel. Intensitätsbilder werden deswegen gewonnen, weil man davon ausgehen kann, dass eine Flamme einen Ort hoher Lichtintensität darstellt und ausserdem eine charakteristische Farbtönung aufweist.The evaluation stage 2 contains a processor (not shown), which has an algorithm for the localization of flames found in the images of the camera 1 and the subsequent analysis of the corresponding image detail. According to the representation, the extraction of intensity and / or chrominance images X ij (t) (hereinafter referred to as intensity images) from the video sequences supplied by the camera 1 takes place in a first process of the algorithm designated image acquisition; i and j are the coordinates of each pixel. The frequency of these pictures is at least 15 frames per second, the picture size for example 352 times 288 pixels. Intensity images are obtained because it can be assumed that a flame is a place of high light intensity and also has a characteristic hue.

In einem mit Vorverarbeitung 4 bezeichneten nächsten Prozess wird nach Flammen in den Intensitätsbildern Xij(t) gesucht und es erfolgt eine Lokalisierung der gefundenen Flammen in entsprechenden Bildausschnitten. Diese Lokalisierung erfolgt mit Hilfe einer so genannten Akkumulationsmatrix, die auf folgende Weise gebildet wird:

  • In einem ersten Schritt erfolgt die Bestimmung des Maximalwerts max [Xij(t)] und des Mittelwerts mean [Xij(t)] der Intensität und daraus wird eine Helligkeitschwelle q(t) bestimmt, wobei gilt: q t + 1 = λ 1 max X ij t , wenn mean [ X ij t ] < λ 1 max X ij t ,
    Figure imgb0001
    und q t + 1 = λ 2 max X ij t - mean [ X ij t ] + mean X ij t ,
    Figure imgb0002
    in allen anderen Fällen. λ1 und λ2 sind Konstante, die zwischen 0 und 1 liegen, wobei beispielsweise λ1 gleich 0.68 und λ2 gleich 0.05 ist.
  • Mit Hilfe dieser beiden Bedingungen wird ein die Flammeneigenschaften berücksichtigender Gewichtungsfaktor wij(t) bestimmt: w ij t = X ij t , wenn X ij t > max [ X ij t - q t ,
    Figure imgb0003
    und w ij t = 0
    Figure imgb0004
    in allen anderen Fällen.
In a next process designated preprocessing 4, flames in the intensity images X ij (t) are searched for and the localized flames are localized in corresponding image sections. This localization takes place by means of a so-called accumulation matrix, which is formed in the following way:
  • In a first step, the determination of the maximum value max [X ij (t)] and the mean value mean [X ij (t)] of the intensity and from this a brightness threshold q (t) is determined, where: q t + 1 = λ 1 Max X ij t . if mean [ X ij t ] < λ 1 Max X ij t .
    Figure imgb0001
    and q t + 1 = λ 2 Max X ij t - mean [ X ij t ] + mean X ij t .
    Figure imgb0002
    in all other cases. λ 1 and λ 2 are constants that lie between 0 and 1, where, for example, λ 1 is 0.68 and λ 2 is 0.05.
  • With the aid of these two conditions, a weighting factor w ij (t) which takes into account the flame properties is determined: w ij t = X ij t . if X ij t > Max [ X ij t - q t .
    Figure imgb0003
    and w ij t = 0
    Figure imgb0004
    in all other cases.

Das bedeutet, dass alle Pixel mit einer Intensität unterhalb des Werts max [Xij(t)] - q(t), also dunkle Objekte, heraus gefiltert und nicht weiter berücksichtigt werden. Wie sich gleich zeigen wird, sind es dunkle, bewegte Objekte, die heraus gefiltert werden.This means that all pixels with an intensity below the value max [X ij (t)] - q (t) , ie dark objects, are filtered out and ignored. As will be seen shortly, they are dark, moving objects that are filtered out.

Da man davon ausgehen kann, dass eine Flamme als Bewegung von hoher Lichtintensität erkennbar ist, bildet man durch Vergleich aufeinander folgender Bilder ein Differenzbild, um eine solche Bewegung herauszufinden; da dunkle Objekte aufgrund der Definition des Gewichtungsfaktors wij(t) heraus fallen, werden also bei der Bildung des Differenzbildes bewegte, dunkle Objekte, die keine Flammen sein können, heraus gefiltert. Man sucht also nach Gebieten hoher Lichtintensität und lokaler Flackerbewegung und das bedeutet, dass beispielsweise eine stationäre Lichtquelle, die nicht flackert, nicht als Flamme interpretiert würde und ebenso auch nicht eine quer durch den Überwachungsraum bewegte Lampe.Since one can assume that a flame is recognizable as a movement of high light intensity, a difference image is formed by comparing successive images to find out such a movement; since dark objects fall out due to the definition of the weighting factor w ij (t), moving, dark objects that can not be flames are filtered out when forming the difference image. So you are looking for areas of high light intensity and local flickering and that means that, for example, a stationary light source that does not flicker would not be interpreted as a flame and also not a transversely across the interstitial space moving lamp.

Für das Differenzbild Qij(t) gilt: Q ij t = X ij t - X ij t - 1 w ij t

Figure imgb0005
For the difference image Q ij (t) : Q ij t = X ij t - X ij t - 1 w ij t
Figure imgb0005

Dann wird aus dem Differenzbild Qij(t) die Akkumulationsmatrix Aij(t) bestimmt: A ij t = α A ij t - 1 + 1 - α Q ij t

Figure imgb0006
Then the accumulation matrix A ij (t) is determined from the difference image Q ij (t) : A ij t = α A ij t - 1 + 1 - α Q ij t
Figure imgb0006

α ist eine Konstante zwischen 0 und 1, die angibt, wie stark das Differenzbild Qij(t) in die Akkumulationsmatrix Aij(t) einfliesst. Für α = 0 wird die Akkumulationsmatrix gleich dem Differenzbild und für α = 1 hat das Differenzbild keinen Einfluss mehr, weil Aij(t) gleich Aij(t-1) ist. Die Akkumulationsmatrix wird in erster Linie deswegen gebildet, um ein geglättetes Bild ohne Rauschen und kurzfristige Änderungen zu erhalten.α is a constant between 0 and 1, which indicates how much the difference image Q ij (t) flows into the accumulation matrix A ij (t) . For α = 0, the accumulation matrix becomes equal to the difference image, and for α = 1, the difference image no longer has any influence because A ij (t) equals A ij (t-1) . The accumulation matrix is formed primarily to obtain a smoothed image with no noise and short-term changes.

Als letzter Schritt der Vorverarbeitung 4 wird mit Hilfe der Akkumulationsmatrix das Pixel oder die Pixel [im,jm](t) mit dem höchsten Wert gesucht und es wird ein dieses oder diese Pixel enthaltender, so genannter interessierender Bildbereich ROI definiert, in dem sich eine Flamme befinden könnte: i m j m t = i j | max A ij t

Figure imgb0007
As the last step of preprocessing 4, the pixel or pixels [i m , j m ] (t) having the highest value are searched by means of the accumulation matrix and a so-called interesting image area ROI containing this or these pixels is defined, in which there could be a flame: i m j m t = i j | Max A ij t
Figure imgb0007

Diese Bestimmung liefert also die Koordinaten der Pixel mit lokaler Flackerbewegung und maximaler Helligkeit. Es wird sich in der Regel um eine einziges Pixel handeln, wobei aber selbstverständlich auch mehrere hellste Pixel bestimmt werden können, wozu man sich einer Mehrkanal-Selektion bedienen kann und vorzugsweise Mindestabstände zwischen den einzelnen Pixeln festlegt.This determination thus provides the coordinates of the pixels with local flickering and maximum brightness. It will usually be a single pixel, but of course also several brightest pixels can be determined, what you can use a multi-channel selection and preferably specifies minimum distances between the individual pixels.

In dem mit Analyse 5 bezeichneten nächsten Prozess erfolgt zunächst eine deutliche Datenreduktion, indem die Analyse nicht im gesamten ursprünglichen Bild von 352 mal 288 Pixel erfolgt, sondern im interessierenden Bildbereich ROI einer reduzierten Grösse von beispielsweise 32 mal 32 Pixel. Das ergibt eine Reduktion auf ein Hunderstel. Diese Reduktion kann selbstverständlich auch geringer sein, beispielsweise auf ein Fünfzigstel, oder auch wesentlich stärker.In the next process designated by analysis 5, a significant data reduction first takes place in that the analysis does not take place in the entire original image of 352 by 288 pixels but in the image region ROI of interest having a reduced size of, for example, 32 by 32 pixels. This results in a reduction to one hundredth. Of course, this reduction can also be lower, for example, to a fiftieth, or even much stronger.

Dann werden für jeden interessierenden Bildbereich die folgenden Bildinformationen bestimmt:

  • Mittlere Helligkeit L(t) des interessierenden Bildbereichs X ROI (t): L t = mean von X ij t | ROI
    Figure imgb0008
  • Chrominanz C(t) des interessierenden Bildbereichs X ROI (t):
    • C(t) = [Anzahl der Cij(t)|ROI] ∈ "Feuer-Chroma-Sektor"/R(t), wobei Cij(t) das Chrominanzpaar (Vij, Uij) des Bildes Xij(t) zur Zeit t bezeichnet. YUV ist eine bekannte Darstellung des Farbraums, mit zwei Farbkomponenten U und V auf der x- bzw. der y-Achse und der Intensität Y auf der z-Achse, wobei die Länge des Vektors vom Nullpunkt zu einem Pixel in der UV-Ebene die Farbsättigung dieses Pixels angibt. Der Feuer-Chroma-Sektor R(t) ist ein Sektor des Farbraums in der UV-Ebene, in dem für eine Flamme typischen Farbbereich, der insbesondere die Farbe rot enthält.
  • Anzahl der aktiven Pixel R(t) der Akkumulationsmatrix A ROI(t): R t = Anzahl der A ij t | ROI > η 1 ; 1 η 1 < Z Z = Gesamtanzahl Pixel des interessierenden Bildbereichs ROI , beispielsweise ist η 1 = 30
    Figure imgb0009
  • Sättigungsgrad S(t) des interessierenden Bildbereichs X ROI (t): S t = Anzahl der X ij t | ROI > η 2 . 1 η 2 < Z beispielsweise ist η 2 = 5
    Figure imgb0010
Then, for each image area of interest, the following image information is determined:
  • Average brightness L (t) of the image area of interest X ROI (t) : L t = mean of X ij t | ROI
    Figure imgb0008
  • Chrominance C (t) of the image area of interest X ROI (t) :
    • C (t) = [number of C ij (t) | ROI ] ∈ "Fire Chroma Sector" / R (t) , where C ij (t) denotes the chrominance pair (V ij , U ij ) of the image X ij (t) at time t. YUV is a well-known representation of the color space, with two color components U and V on the x and the y-axis and the intensity Y on the z-axis, where the length of the vector from the zero point to a pixel in the UV plane Indicates the color saturation of this pixel. The fire chroma sector R (t) is a sector of the color space in the UV plane, in the color region typical for a flame, which in particular contains the color red.
  • Number of active pixels R (t) of the accumulation matrix A ROI ( t ): R t = number of A ij t | ROI > η 1 ; 1 η 1 < Z Z = Total number of pixels of the image area ROI of interest . for example η 1 = 30
    Figure imgb0009
  • Saturation degree S (t) of the image area of interest X ROI (t) : S t = number of X ij t | ROI > η 2 , 1 η 2 < Z for example η 2 = 5
    Figure imgb0010

Damit das Ergebnis stabil bleibt, erfolgt anschliessend in einem mit Extraktion 6 bezeichneten  Prozess eine Zeitintegration der bei der Analyse 5 bestimmten Bildinformationen. Wenn die Integration beispielsweise über 1 Sekunde durchgeführt wird, erstreckt sie sich im PAL-Format über 25 Bilder. Man integriert also die mittlere Helligkeit, die Chrominanz, die aktiven Pixel und den Sättigungsgrad über die Zeit t von t0 bis tn und erhält die folgenden Eigenschaften:

  • Mittelwert Helligkeit:     FL = L
  • Mittelwert Frequenz:    FF = F
  • Mittelwert Amplitude:     FM = M
  • Mittelwert Feuer-Chroma-Pixel:     FC = C
  • Mittelwert aktive Pixel:     FR = R
  • Mittelwert Sättigung:     FS = S
In order that the result remains stable, a time integration of the image information determined in the analysis 5 is then carried out in a process designated by extraction 6. For example, if the integration is done over 1 second, it will span 25 frames in PAL format. Thus, the average brightness, the chrominance, the active pixels and the degree of saturation are integrated over the time t from t 0 to t n and are given the following properties:
  • Average brightness: F L = L
  • Mean frequency: F F = F
  • Mean value amplitude: F M = M
  • Average Fire Chroma Pixel: F C = C
  • Mean active pixels: F R = R
  • Mean saturation: F S = S

Den Mittelwert der Frequenz erhält man beispielsweise durch Zählen der Pixel der mittleren Helligkeit L(t). Die Frequenz, die durch das charakteristische Flackern einer Flamme verursacht wird, ist eine für die Detektion einer Flamme wichtige Grösse, weil sie in einem definierten engen Bereich von in der Regel zwischen 1 Hz und 10 Hz liegt.The mean value of the frequency is obtained, for example, by counting the pixels of the average brightness L (t) . The frequency caused by the characteristic flickering of a flame is an important quantity for the detection of a flame because it is within a defined narrow range of usually between 1 Hz and 10 Hz.

Im anschliessenden mit Mustererkennung 7 bezeichneten Prozess wird aus den bei der Extraktion 6 gewonnenen Eigenschaften die Wahrscheinlichkeit berechnet, dass eine Flamme vorliegt. Dabei wird beispielsweise für jede der obigen Eigenschaften untersucht, ob der Mittelwert oberhalb oder unterhalb eines Schwellwertes liegt und die Wahrscheinlichkeit entsprechend gleich eins bzw. gleich null gesetzt. Dann wird aus den Wahrscheinlichkeiten aller n Eigenschaften eine Gesamtwahrscheinlichkeit gebildet. Ψ L = Γ F L = 1 , wenn F L > δ L

Figure imgb0011
Ψ L = Γ F L = 0 , wenn F L < δ L
Figure imgb0012
und so weiter für die anderen Eigenschaften.In the subsequent process, designated by pattern recognition 7, the probability that a flame is present is calculated from the properties obtained during extraction 6. In this case, for example, it is examined for each of the above properties whether the mean value is above or below a threshold value and the probability is set equal to one or equal to zero. Then, from the probabilities of all n properties, a total probability is formed. Ψ L = Γ F L = 1 . if F L > δ L
Figure imgb0011
Ψ L = Γ F L = 0 . if F L < δ L
Figure imgb0012
and so on for the other properties.

Gesamtwahrscheinlichkeit Π(t): Π t : = 1 / N F Σ Ψ n = Ψ L . w L + Ψ F . w F + Ψ M . w M + Ψ C . w C + Ψ R . w R + Ψ S . w S / N F

Figure imgb0013
Π t : = 1 N F n Ψ n = Ψ L w L + Ψ F w F + Ψ M w M + Ψ C w C + Ψ R w R + Ψ S w S / N F
Figure imgb0014
für wi gilt 0 ≤ wi 1, wobei die Werte wi empirisch bestimmt werden. NF ist die Summe der wi über alle i. Total probability Π (t): Π t : = 1 / N F Σ Ψ n = Ψ L , w L + Ψ F , w F + Ψ M , w M + Ψ C , w C + Ψ R , w R + Ψ S , w S / N F
Figure imgb0013
Π t : = 1 N F Σ n Ψ n = Ψ L w L + Ψ F w F + Ψ M w M + Ψ C w C + Ψ R w R + Ψ S w S / N F
Figure imgb0014
for w i , 0 ≦ w i 1, where the values w i are determined empirically. N F is the sum of w i over all i.

In dem mit Entscheidung 8 bezeichneten Prozess erfolgt anschliessend die Entscheidung, ob Alarm ausgelöst wird. Dieser Prozess enthält eine Integration, bei der die Gesamtwahrscheinlichkeit Π(t) über aufeinanderfolgende Bilder aufintegriert wird. Die Integration beginnt bei null und zählt für jedes Π(t) > κ (κ ist eine Schwelle) ein Inkrement dazu und zieht für jedes Π(t) < κ ein Inkrement ab. Wenn I(t) den Wert des Integrals bezeichnet, gilt: I t = 0 = 0

Figure imgb0015

wenn Π(t) > κ
dann I(t) = I(t-1) + σ+ (gesättigt zu S+, wenn I(t) > S+ in allen anderen Fällen gilt I(t) = I(t-1) - σ- (gesättigt zu S- (üblicherweise 0), wenn I(t) < S.
σ+ und σ- sind in der Regel gleich +1.In the process designated by decision 8, the decision is then made as to whether an alarm is triggered. This process includes an integration in which the total probability Π (t) is integrated over successive images. The integration starts at zero and counts an increment for each Π (t) > κ (κ is a threshold) and subtracts an increment for each Π (t) <κ. If I (t) denotes the value of the integral, then: I t = 0 = 0
Figure imgb0015
if Π (t) > κ
then I (t) = I (t-1) + σ + (saturated to S + , if I (t) > S + in all other cases I (t) = I (t-1) - σ - (saturated to S - (usually 0) if I (t) <S.
σ + and σ - are usually equal to +1.

Mit Hilfe des Integrals I(t) erfolgt nun die Entscheidung, ob ein Alarm ausgelöst wird:

  • Wenn I(t) > β (β ist eine Schwelle), wird Alarm ausgelöst, in allen anderen Fällen nicht.
With the help of the integral I (t) , the decision is now made as to whether an alarm is triggered:
  • If I (t) > β (β is a threshold), the alarm is triggered, but not in all other cases.

Die beschriebene Einrichtung hat den Vorteil, dass in vielen Anwendungsfällen auf bereits installierte Videokameras zurückgegriffen werden kann und eine Installation spezieller Flammensensoren nicht erforderlich ist, was zweifellos eine Kostenreduktion bedeutet. Eine weitere Kostenreduktion ergibt sich durch die Beschränkung der Auswertung auf die eventuell eine Flamme enthaltenden Bildausschnitte, was eine deutliche Reduktion der Rechnerleistung ermöglicht. Man kann auch davon ausgehen, dass die Auswertung dieser Bildausschnitte gegenüber Störungen ausreichend robust ist.The described device has the advantage that can be used in many applications on already installed video cameras and installation of special flame sensors is not required, which undoubtedly means a cost reduction. A further cost reduction results from the restriction of the evaluation to the possibly containing a flame image sections, which allows a significant reduction of computer power. It can also be assumed that the evaluation of these image sections is sufficiently robust to interference.

Claims (15)

  1. Method for detecting flames in a monitored zone, referred to hereinafter as monitoring space, by analysis of at least one parameter of a radiation that occurs in the monitoring space, in which case a video image of the monitoring space is generated and zones having high light intensity and local flicker motion are sought in said video image, in which case, in a first step, said zones are localized and the relevant image excerpts are subsequently analyzed with regard to the presence of a flame, characterized in that the search for zones having high light intensity and local flicker motion is effected with the aid of an accumulation matrix [Aij(t)], which is obtained from the difference images of successive intensity images [Xij(t)], said difference images being weighted with a weighting factor, the weighting factor specifying the extent to which the difference images influence the accumulation matrix [Aij(t)].
  2. Method according to claim 1, characterized in that the video images are generated with a specific frequency and intensity images [Xij(t)] are obtained therefrom.
  3. Method according to claim 1, characterized in that all pixels having a brightness that lies below a predetermined threshold and thus all moving, dark objects are filtered out during the formation of the difference images and thus of the accumulation matrix [Aij(t)].
  4. Method according to claim 3, characterized in that the coordinates of the brightest pixels are sought with the aid of the accumulation matrix [Aij(t)].
  5. Method according to claim 4, characterized in that an image region of interest [ROI] which contains the brightest pixel or pixels and is reduced with respect to the original image is defined and analyzed with regard to the presence of a flame.
  6. Method according to claim 5, characterized in that the size of the image region of interest (ROI) amounts at most to one fiftieth of the size of the original image.
  7. Method according to claim 6, characterized in that the image information items of brightness [L(t)], chrominance [C(t)], number of active pixels [R(t)] above a specific intensity threshold, and the saturation [S(t)], are determined in the image region of interest [ROI].
  8. Method according to claim 7, characterized in that said image information items are integrated over a specific time and thus over a plurality of images and their mean value is determined in that the mean value of the frequency [F] and the mean value of the amplitude [M] are determined as additional parameters during the integration, and in that the probability for the presence of a flame is calculated for each of said mean values.
  9. Method according to claim 8, characterized in that the probabilities of the mean values are used to calculate an overall probability for the presence of a flame in the reduced image region [ROI], in that said overall probability is integrated over a plurality of images, and in that an alarm is triggered when a threshold is exceeded by the integrated value.
  10. Device for detecting flames in a monitored zone, referred to hereinafter as monitoring space, by analysis of at least one parameter of a radiation that occurs in the monitoring space, characterized by a video camera [1] with an evaluation stage [2] for the images supplied by the camera [1], the evaluation stage [2] having a processor with an algorithm for the localization of regions having high light intensity and local flicker motion in the images of the camera [1] and the subsequent analysis of the corresponding image excerpts with regard to the presence of a flame, and the algorithm containing a process, referred to below as preprocessing [4], during which an accumulation matrix [Aij(t)] is determined for the search for zones having high light intensity and local flicker motion, said accumulation matrix being obtained from the difference images of successive intensity images [Xij(t)], said difference images being weighted with a weighting factor, the weighting factor specifying the extent to which the difference images influence the accumulation matrix [Aij(t)].
  11. Device according to claim 10, characterized in that the algorithm contains a process, referred to below as image obtaining [3], during which intensity images [Xij(t)] are obtained from the video images generated with a specific frequency.
  12. Device according to claim 10, characterized in that during the preprocessing [4], with the aid of the accumulation matrix [Aij(t)], the coordinates of the brightest pixels are determined and an image region of interest (ROI) which contains the brightest pixel or pixels and is reduced with respect to the original image is defined.
  13. Device according to claim 12, characterized in that, the algorithm contains a process, referred to below as analysis [5], for the analysis of the image region of interest [ROI] during which analysis the image information items of brightness [L(t)], chrominance [C(t)], number of active pixels [R(t)] above a specific intensity threshold, and the saturation [S(t)] are determined.
  14. Device according to claim 13, characterized in that the algorithm contains a process, referred to below as extraction [6], during which said image information items are integrated over a specific time and thus over a plurality of images and the mean values of the image information items are determined, in that the mean value of the frequency [F] and the mean value of the amplitude [M] are determined as additional parameters during the integration, and in that the probability for the presence of a flame is calculated for each of said mean values.
  15. Device according to claim 14, characterized in that the algorithm contains a process of pattern recognition [7] and a process of decision [8], during which the probabilities of the mean values are used to calculate an overall probability for the presence of a flame in the reduced image region [ROI] and said overall probability is integrated over a plurality of images and an alarm is triggered in the event of a threshold being exceeded by the integrated value.
EP03015846A 2003-07-11 2003-07-11 Method and apparatus for the detection of flames Expired - Lifetime EP1496483B1 (en)

Priority Applications (9)

Application Number Priority Date Filing Date Title
AT03015846T ATE357714T1 (en) 2003-07-11 2003-07-11 METHOD AND DEVICE FOR DETECTING FLAMES
EP03015846A EP1496483B1 (en) 2003-07-11 2003-07-11 Method and apparatus for the detection of flames
DE50306852T DE50306852D1 (en) 2003-07-11 2003-07-11 Method and device for detecting flames
ES03015846T ES2282550T3 (en) 2003-07-11 2003-07-11 PROCEDURE AND DEVICE FOR THE DETECTION OF FLAMES.
AU2004202851A AU2004202851B2 (en) 2003-07-11 2004-06-25 Method and Device for Detecting Flames
KR1020040050227A KR20050009135A (en) 2003-07-11 2004-06-30 Method and device for detecting flames
PL04369016A PL369016A1 (en) 2003-07-11 2004-07-09 Method and device designed for flame detection
NO20042948A NO330182B1 (en) 2003-07-11 2004-07-09 Flame detection method and apparatus
CN200410063587A CN100595583C (en) 2003-07-11 2004-07-12 Method and apparatus for the detection of flames

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP03015846A EP1496483B1 (en) 2003-07-11 2003-07-11 Method and apparatus for the detection of flames

Publications (2)

Publication Number Publication Date
EP1496483A1 EP1496483A1 (en) 2005-01-12
EP1496483B1 true EP1496483B1 (en) 2007-03-21

Family

ID=33442787

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03015846A Expired - Lifetime EP1496483B1 (en) 2003-07-11 2003-07-11 Method and apparatus for the detection of flames

Country Status (9)

Country Link
EP (1) EP1496483B1 (en)
KR (1) KR20050009135A (en)
CN (1) CN100595583C (en)
AT (1) ATE357714T1 (en)
AU (1) AU2004202851B2 (en)
DE (1) DE50306852D1 (en)
ES (1) ES2282550T3 (en)
NO (1) NO330182B1 (en)
PL (1) PL369016A1 (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060041555A (en) * 2004-11-09 2006-05-12 한국서부발전 주식회사 System and method for detecting and alarming a fire of thermal power plants
US7289032B2 (en) * 2005-02-24 2007-10-30 Alstom Technology Ltd Intelligent flame scanner
KR100680114B1 (en) * 2005-05-12 2007-02-07 (주)에이치엠씨 Device, method and recording medium of robust fire detecting using color of image
US7868772B2 (en) 2006-12-12 2011-01-11 Industrial Technology Research Institute Flame detecting method and device
CN101316371B (en) * 2007-05-31 2012-11-28 财团法人工业技术研究院 Flame detecting method and device
EP2000998B1 (en) * 2007-05-31 2013-01-02 Industrial Technology Research Institute Flame detecting method and device
DE112009003247A5 (en) * 2008-11-03 2012-05-03 IQ Wireless Entwicklungsges. für Systeme und Technologien der Telekommunikation mbH METHOD AND DEVICE FOR THE NOMINANT DETECTION OF FIRE AND DISTINCTION OF ARTIFICIAL LIGHT SOURCES
CN101441712B (en) * 2008-12-25 2013-03-27 北京中星微电子有限公司 Flame video recognition method and fire hazard monitoring method and system
CN101847304B (en) * 2009-12-04 2012-05-23 四川川大智胜软件股份有限公司 Image-based method of finding flames with large-space intelligent fire-fighting system
CN102645246B (en) * 2011-02-17 2014-04-23 中国石油化工股份有限公司 Method for real-time measurement of torch flow rate of torch discharge system based on torch videos
CN103776056B (en) * 2012-10-25 2016-04-20 宁波立诚电子制造有限公司 A kind of lighter detection method and device thereof
CN107064113B (en) * 2017-06-13 2020-02-11 华电青岛发电有限公司 System and method for detecting pulverized coal combustion quality of burner by using optical fiber
CN111141504B (en) * 2019-12-25 2022-04-15 Oppo(重庆)智能科技有限公司 Fire-break detection method and device and computer readable storage medium
CN111583610B (en) * 2020-04-30 2021-07-16 深圳市前海用电物联网科技有限公司 Fire-fighting linkage control method and system of causal model
CN113436406B (en) * 2021-08-25 2021-11-12 广州乐盈信息科技股份有限公司 Sound-light alarm system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9216811D0 (en) * 1992-08-07 1992-09-23 Graviner Ltd Kidde Flame detection methods and apparatus
EP1256105B1 (en) * 2000-02-07 2006-09-20 VSD Limited Smoke and flame detection
WO2002054364A2 (en) * 2000-12-28 2002-07-11 Siemens Building Technologies Ag Video smoke detection system
RU2003133287A (en) * 2001-05-11 2005-05-27 Детектор Электроникс Корпорэйшн (Us) METHOD AND DEVICE FOR FLAME DETECTION BY FORMING FLAME IMAGES

Also Published As

Publication number Publication date
NO330182B1 (en) 2011-02-28
DE50306852D1 (en) 2007-05-03
KR20050009135A (en) 2005-01-24
CN100595583C (en) 2010-03-24
NO20042948L (en) 2005-01-12
ES2282550T3 (en) 2007-10-16
AU2004202851A1 (en) 2005-01-27
AU2004202851B2 (en) 2008-08-28
NO20042948D0 (en) 2004-07-09
EP1496483A1 (en) 2005-01-12
PL369016A1 (en) 2005-01-24
ATE357714T1 (en) 2007-04-15
CN1576839A (en) 2005-02-09

Similar Documents

Publication Publication Date Title
EP1496483B1 (en) Method and apparatus for the detection of flames
DE60105006T2 (en) PROCESS AND SYSTEM FOR FIRE FIGHTER IDENTIFICATION
EP1628260B1 (en) Method and system for automatic forest fire recognition
DE19934171B4 (en) Filter system and method
DE102013017395B3 (en) Method and device for automated early forest fire detection by means of optical detection of clouds of smoke
EP0939387B1 (en) Room supervision device
EP1346330B1 (en) Video smoke detection system
DE102012206078A1 (en) Determining a number of objects in an IR image
EP2677460B1 (en) Method for determining the length of a queue
DE112017007246T5 (en) IMAGE ANALYSIS DEVICE, IMAGE ANALYSIS METHOD AND IMAGE ANALYSIS PROGRAM
EP3453000B1 (en) Smoke detection device, method for detecting smoke from a fire, and computer program
DE60123214T2 (en) SMOKE AND FLAME DETECTION
DE102019103963A1 (en) ILDGEBUNGSVORRICHTUNG
DE4332753C2 (en) Process for the detection of moving objects
DE112007000738B4 (en) Filter for disturbing movements
EP1239433A1 (en) Method and apparatus for the detection of smoke and / or fire in spaces
DE19883010B4 (en) Method and device for recognizing a moving object in a sequence of color frames
DE102007016922A1 (en) Method for detecting defects on the backside of a semiconductor wafer
EP0834845A1 (en) Method for frequency analysis of a signal
EP0951776A1 (en) Brightness control on video cameras
DE3214254A1 (en) Method for detecting movements in video camera images
DE102012109390A1 (en) Monitoring device for safety-critical unit, particularly conveyer of conveyer system, has first video sensor unit, which is adapted to generate first image as representation formed from line by first monitoring area of safety-critical unit
DE4105516A1 (en) METHOD AND DEVICE FOR FILTERING SIGNALS
EP1762995A1 (en) Detection of smoke with a video camera
DE4002829A1 (en) METHOD FOR DETECTING METAL OBJECTS

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

17P Request for examination filed

Effective date: 20050718

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SIEMENS SCHWEIZ AG

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REF Corresponds to:

Ref document number: 50306852

Country of ref document: DE

Date of ref document: 20070503

Kind code of ref document: P

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20070522

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070821

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2282550

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

26N No opposition filed

Effective date: 20071227

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070731

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070622

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070711

REG Reference to a national code

Ref country code: CH

Ref legal event code: PUE

Owner name: SIEMENS AKTIENGESELLSCHAFT

Free format text: SIEMENS SCHWEIZ AG#ALBISRIEDERSTRASSE 245#8047 ZUERICH (CH) -TRANSFER TO- SIEMENS AKTIENGESELLSCHAFT#WITTELSBACHERPLATZ 2#80333 MUENCHEN (DE)

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20080715

Year of fee payment: 6

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20090514 AND 20090520

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070711

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070321

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070922

BERE Be: lapsed

Owner name: SIEMENS SCHWEIZ A.G.

Effective date: 20090731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090731

REG Reference to a national code

Ref country code: NL

Ref legal event code: SD

Effective date: 20110318

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20120726

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20120703

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20120830

Year of fee payment: 10

REG Reference to a national code

Ref country code: NL

Ref legal event code: V1

Effective date: 20140201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130711

REG Reference to a national code

Ref country code: CH

Ref legal event code: PUE

Owner name: SIEMENS SCHWEIZ AG, CH

Free format text: FORMER OWNER: SIEMENS AKTIENGESELLSCHAFT, DE

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20150220 AND 20150225

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 50306852

Country of ref document: DE

Owner name: SIEMENS SCHWEIZ AG, CH

Free format text: FORMER OWNER: SIEMENS AKTIENGESELLSCHAFT, 80333 MUENCHEN, DE

Effective date: 20150407

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20150709

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130712

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: SIEMENS SCHWEIZ AG, CH

Effective date: 20160202

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20160920

Year of fee payment: 14

Ref country code: GB

Payment date: 20160713

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20160725

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20161012

Year of fee payment: 14

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50306852

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170711

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20180330

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170731

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170731

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180201

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170731