EP1496483B1 - Method and apparatus for the detection of flames - Google Patents
Method and apparatus for the detection of flames Download PDFInfo
- Publication number
- EP1496483B1 EP1496483B1 EP03015846A EP03015846A EP1496483B1 EP 1496483 B1 EP1496483 B1 EP 1496483B1 EP 03015846 A EP03015846 A EP 03015846A EP 03015846 A EP03015846 A EP 03015846A EP 1496483 B1 EP1496483 B1 EP 1496483B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- images
- image
- roi
- flame
- pixels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000001514 detection method Methods 0.000 title abstract description 6
- 238000009825 accumulation Methods 0.000 claims abstract description 19
- 239000011159 matrix material Substances 0.000 claims abstract description 19
- 238000004458 analytical method Methods 0.000 claims abstract description 11
- 230000010354 integration Effects 0.000 claims abstract description 7
- 238000011156 evaluation Methods 0.000 claims description 11
- 230000005855 radiation Effects 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000004807 localization Effects 0.000 claims description 3
- 238000003909 pattern recognition Methods 0.000 claims description 2
- 230000015572 biosynthetic process Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 2
- 229920006395 saturated elastomer Polymers 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000007257 malfunction Effects 0.000 description 1
- VFBILHPIHUPBPZ-UHFFFAOYSA-N n-[[2-[4-(difluoromethoxy)-3-propan-2-yloxyphenyl]-1,3-oxazol-4-yl]methyl]-2-ethoxybenzamide Chemical compound CCOC1=CC=CC=C1C(=O)NCC1=COC(C=2C=C(OC(C)C)C(OC(F)F)=CC=2)=N1 VFBILHPIHUPBPZ-UHFFFAOYSA-N 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
- G08B17/125—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke
Definitions
- the present invention relates to a method and a device for detecting flames in a monitored area referred to below as a monitoring space by analyzing at least one parameter of a radiation occurring in the interstitial space.
- WO 02/093525 A1 discloses an apparatus and a method for the simultaneous processing of first and second image processing, especially for the detection of flames on the images.
- the apparatus includes an image sensor for creating video images, a frame grabber for receiving a first and a second image (frame), a processor for processing the captured images, and an output unit.
- Previously known devices as described for example in US Pat. Nos. 4,866,420 and 4,280,058, contain at least one sensor which evaluates the flicker frequency spectrum of the radiation, whereby signals located outside a specific frequency band are evaluated as interference signals.
- the typical flickering of the flames in a very low-frequency oscillation range as a feature for distinguishing between the radiation emitted by a flame and interfering radiation.
- the determination of the frequency band takes place in the simplest case by the sensor upstream filter or by this downstream frequency-selective amplifier, in both cases, a certain passband of, for example, 5 to 25 Hz is obtained.
- the invention will now be given a method for the detection of flames, which is characterized by a high noise immunity at low cost and.
- the search for regions of high light intensity and local flickering movement takes place with the aid of an accumulation matrix which is obtained from the weighting factor-weighted difference images of successive intensity images, wherein the weighting factor indicates how much the accumulation matrix is accumulated on the difference images.
- a first preferred embodiment of the method according to the invention is characterized in that the video images are generated at a specific frequency and intensity images are obtained therefrom.
- a second preferred embodiment of the method according to the invention characterized in that the coordinates of the brightest pixels are searched with the aid of the accumulation matrix.
- a third preferred embodiment is characterized in that an interesting image area containing the brightest or the brightest pixels and reduced in relation to the original image area is defined and analyzed for the presence of a flame.
- the reference numeral 1 denotes a video camera which has video output via an output to an evaluation stage 2 supplies, wherein the evaluation stage 2 can be integrated into the camera 1 or connected to this.
- the evaluation level 2 can be provided at the site of the camera 1 or in the immediate vicinity of this or it can also be spatially distant from the camera 1, wherein in the latter case between camera 1 and evaluation level 2 is a communication link.
- the evaluation stage 2 contains a processor (not shown), which has an algorithm for the localization of flames found in the images of the camera 1 and the subsequent analysis of the corresponding image detail.
- a processor not shown
- the extraction of intensity and / or chrominance images X ij (t) (hereinafter referred to as intensity images) from the video sequences supplied by the camera 1 takes place in a first process of the algorithm designated image acquisition; i and j are the coordinates of each pixel.
- the frequency of these pictures is at least 15 frames per second, the picture size for example 352 times 288 pixels. Intensity images are obtained because it can be assumed that a flame is a place of high light intensity and also has a characteristic hue.
- a difference image is formed by comparing successive images to find out such a movement; since dark objects fall out due to the definition of the weighting factor w ij (t), moving, dark objects that can not be flames are filtered out when forming the difference image. So you are looking for areas of high light intensity and local flickering and that means that, for example, a stationary light source that does not flicker would not be interpreted as a flame and also not a transversely across the interstitial space moving lamp.
- a ij (t) ⁇ ⁇ A ij ⁇ t - 1 + 1 - ⁇ ⁇ Q ij t
- ⁇ is a constant between 0 and 1, which indicates how much the difference image Q ij (t) flows into the accumulation matrix A ij (t) .
- the accumulation matrix is formed primarily to obtain a smoothed image with no noise and short-term changes.
- This determination thus provides the coordinates of the pixels with local flickering and maximum brightness. It will usually be a single pixel, but of course also several brightest pixels can be determined, what you can use a multi-channel selection and preferably specifies minimum distances between the individual pixels.
- a significant data reduction first takes place in that the analysis does not take place in the entire original image of 352 by 288 pixels but in the image region ROI of interest having a reduced size of, for example, 32 by 32 pixels. This results in a reduction to one hundredth. Of course, this reduction can also be lower, for example, to a fiftieth, or even much stronger.
- the mean value of the frequency is obtained, for example, by counting the pixels of the average brightness L (t) .
- the frequency caused by the characteristic flickering of a flame is an important quantity for the detection of a flame because it is within a defined narrow range of usually between 1 Hz and 10 Hz.
- the probability that a flame is present is calculated from the properties obtained during extraction 6. In this case, for example, it is examined for each of the above properties whether the mean value is above or below a threshold value and the probability is set equal to one or equal to zero. Then, from the probabilities of all n properties, a total probability is formed.
- N F is the sum of w i over all i.
- the described device has the advantage that can be used in many applications on already installed video cameras and installation of special flame sensors is not required, which undoubtedly means a cost reduction.
- a further cost reduction results from the restriction of the evaluation to the possibly containing a flame image sections, which allows a significant reduction of computer power. It can also be assumed that the evaluation of these image sections is sufficiently robust to interference.
Abstract
Description
Die vorliegende Erfindung betrifft ein Verfahren und eine Einrichtung zur Detektion von Flammen in einem nachfolgend als Überwachungsraum bezeichneten überwachten Gebiet durch Analyse mindestens eines Parameters einer im Überwachungsraum auftretenden Strahlung.The present invention relates to a method and a device for detecting flames in a monitored area referred to below as a monitoring space by analyzing at least one parameter of a radiation occurring in the interstitial space.
Die WO 02/093525 A1 offenbart eine Vorrichtung und ein Verfahren zum gleichzeitigen Verarbeiten einer ersten und zweiten Bildverarbeitung, speziell zur Detektion von Flammen auf den Bildern. Die Vorrichtung beinhaltet einen Bildsensor zum Erstellen von Videobildern, eine Bildfangschaltung zum Empfangen von einem ersten und einem zweiten Bild (Frame), einen Prozessor zum Verarbeiten der aufgenommenen Bilder und eine Ausgabeeinheit.WO 02/093525 A1 discloses an apparatus and a method for the simultaneous processing of first and second image processing, especially for the detection of flames on the images. The apparatus includes an image sensor for creating video images, a frame grabber for receiving a first and a second image (frame), a processor for processing the captured images, and an output unit.
Bisher bekannte Einrichtungen, wie sie beispielsweise in der US-Patenten Nr. 4 866 420 und Nr. 4 280 058 beschrieben sind, enthalten mindestens einen Sensor, welcher das Flackerfrequenzspektrum der Strahlung auswertet, wobei ausserhalb eines bestimmten Frequenzbandes liegende Signale als Störsignale bewertet werden. Man benutzt also das typische Flackern der Flammen in einem sehr niederfrequenten Schwingungsbereich als Merkmal zur Unterscheidung zwischen der von einer Flamme ausgesandten Strahlung und Störstrahlung. Die Festlegung des Frequenzbandes erfolgt im einfachsten Fall durch dem Sensor vorgeschaltete Filter oder durch diesem nach-geschaltete frequenzselektive Verstärker, wobei in beiden Fällen ein bestimmter Durchlassbereich von beispielsweise 5 bis 25 Hz erhalten wird.Previously known devices, as described for example in US Pat. Nos. 4,866,420 and 4,280,058, contain at least one sensor which evaluates the flicker frequency spectrum of the radiation, whereby signals located outside a specific frequency band are evaluated as interference signals. Thus one uses the typical flickering of the flames in a very low-frequency oscillation range as a feature for distinguishing between the radiation emitted by a flame and interfering radiation. The determination of the frequency band takes place in the simplest case by the sensor upstream filter or by this downstream frequency-selective amplifier, in both cases, a certain passband of, for example, 5 to 25 Hz is obtained.
Diese bekannten Flammenmelder haben sich durchaus bewährt, sie stellen aber in einer Brandmeldeanlage einen nicht unbeträchtlichen Kostenfaktor dar. Abgesehen davon, können auch bei optimaler Abstimmung des Frequenzbandes auf das Flackern von Flammen Störungen und Fehlanzeigen nicht ausgeschlossen werden, weil es immer wieder vorkommt, dass zufällige Intensitätsänderungen der Umgebungsstrahlung im Durchlassbereich liegen. Solche Intensitätsänderungen können beispielsweise durch Abschattungen oder Reflexe von vibrierenden oder sich langsam bewegenden Gegenständen, durch Reflexe des Sonnenlichts an Wasseroberflächen oder durch flackernde oder schwankende Lichtquellen verursacht sein.Apart from that, even with optimal tuning of the frequency band to the flickering of flames malfunctions and false positives can not be ruled out because it happens again and again that accidental Intensity changes of the ambient radiation are in the passband. Such changes in intensity may be caused, for example, by shadows or reflections from vibrating or slowly moving objects, reflections of the sunlight on water surfaces, or flickering or fluctuating light sources.
Durch die Erfindung soll nun ein Verfahren zur Detektion von Flammen angegeben werden, welches sich durch eine hohe Störsicherheit bei geringen Kosten und auszeichnet.The invention will now be given a method for the detection of flames, which is characterized by a high noise immunity at low cost and.
Diese Aufgabe wird gemäss der Ansprüche 1 und 10 gelöst.This object is achieved according to
Nach dem erfindungsgemässen Verfahren erfolgt die Suche nach Gebieten von hoher Lichtintensität und lokaler Flackerbewegung mit Hilfe einer Akkumulationsmatrix, welche aus den mit einem Gewichtungsfaktor gewichteten Differenzbildern aufeinanderfolgender Intensitätsbilder gewonnen wird, wobei der Gewichtungsfaktor angibt, wie stark die Akkumulationsmatrix an die Differenzbilder akkumuliert wird.According to the method according to the invention, the search for regions of high light intensity and local flickering movement takes place with the aid of an accumulation matrix which is obtained from the weighting factor-weighted difference images of successive intensity images, wherein the weighting factor indicates how much the accumulation matrix is accumulated on the difference images.
Eine erste bevorzugte Ausführungsform des erfindungsgemässen Verfahrens ist dadurch gekennzeichnet, dass die Videobilder mit einer bestimmten Frequenz erzeugt und daraus Intensitätsbilder gewonnen werden.A first preferred embodiment of the method according to the invention is characterized in that the video images are generated at a specific frequency and intensity images are obtained therefrom.
Eine zweite bevorzugte Ausführungsform des erfindungsgemässen Verfahrens dadurch gekennzeichnet, dass mit Hilfe der Akkumulationsmatrix die Koordinaten der hellsten Pixel gesucht werden.A second preferred embodiment of the method according to the invention, characterized in that the coordinates of the brightest pixels are searched with the aid of the accumulation matrix.
Eine dritte bevorzugte Ausführungsform ist dadurch gekennzeichnet, dass ein das hellste oder die hellsten Pixel enthaltender und gegenüber dem ursprünglichen Bildbereich reduzierter, interessierender Bildbereich definiert und auf das Vorhandensein einer Flamme analysiert wird.A third preferred embodiment is characterized in that an interesting image area containing the brightest or the brightest pixels and reduced in relation to the original image area is defined and analyzed for the presence of a flame.
Weitere bevorzugte Ausführungsformen des erfindungsgemässen Verfahrens ergeben sich aus den abhängigen Ansprüchen 7 bis 9.Further preferred embodiments of the method according to the invention emerge from the
Bevorzugte Ausführungsformen der erfindungsgemässen Einrichtung sind in den abhängigen Ansprüchen 11 bis 15 beansprucht.Preferred embodiments of the device according to the invention are claimed in the dependent claims 11 to 15.
Mit der immer stärkeren Verbreitung von CCTV-Systemen und -Anlagen kann man davon ausgehen, dass in vielen Fällen in einem Überwachungsraum eine Videokamera vorhanden sein wird, so für die Flammendetektion nicht ein eigener Sensor installiert werden muss, was zweifellos eine Kostenreduktion bedeutet. Eine weitere Kostenreduktion ergibt sich durch die Beschränkung der Auswertung auf die eventuell eine Flamme enthaltenden Bildausschnitte, was eine deutliche Reduktion der Rechnerleistung ermöglicht. Man kann auch davon ausgehen, dass die Auswertung dieser Bildausschnitte gegenüber Störungen ausreichend robust ist.With the proliferation of CCTV systems and systems, it can be assumed that in many cases a video camera will be present in a surveillance room, so that a separate sensor does not have to be installed for flame detection, which undoubtedly means a cost reduction. A further cost reduction results from the restriction of the evaluation to the possibly containing a flame image sections, which allows a significant reduction of computer power. It can also be assumed that the evaluation of these image sections is sufficiently robust to interference.
Im Folgenden wird die Erfindung anhand eines eine erfindungsgemässe Einrichtung zur Detektion von Flammen zeigenden Blockschemas beispielsweise näher erläutert. Mit dem Bezugszeichen 1 ist eine Videokamera bezeichnet, welche über einen Ausgang Videosequenzen an eine Auswertestufe 2 liefert, wobei die Auswertestufe 2 in die Kamera 1 integriert oder mit dieser verbunden sein kann. Die Auswertestufe 2 kann am Aufstellungsort der Kamera 1 oder in unmittelbarer Nähe zu diesem oder sie kann auch räumlich entfernt von der Kamera 1 vorgesehen sein, wobei im letzteren Fall zwischen Kamera 1 und Auswertestufe 2 eine Kommunikationsverbindung besteht.In the following, the invention is explained in more detail by way of example with reference to a block diagram showing a device according to the invention for detecting flames. The
Die Auswertestufe 2 enthält einen Prozessor (nicht dargestellt), welcher einen Algorithmus für die Lokalisierung von in den Bildern der Kamera 1 gefundenen Flammen und die nachfolgende Analyse der entsprechenden Bildausschnitte aufweist. Darstellungsgemäss erfolgt in einem mit Bildgewinnung bezeichneten ersten Prozess des Algorithmus die Gewinnung von Intensitäts- und/oder Chrominanzbildern Xij(t) (nachfolgend als Intensitätsbilder bezeichnet) aus den von der Kamera 1 gelieferten Videosequenzen; i und j sind die Koordinaten der einzelnen Pixel. Die Frequenz dieser Bilder beträgt mindestens 15 Bilder pro Sekunde, die Bildgrösse beispielsweise 352 mal 288 Pixel. Intensitätsbilder werden deswegen gewonnen, weil man davon ausgehen kann, dass eine Flamme einen Ort hoher Lichtintensität darstellt und ausserdem eine charakteristische Farbtönung aufweist.The evaluation stage 2 contains a processor (not shown), which has an algorithm for the localization of flames found in the images of the
In einem mit Vorverarbeitung 4 bezeichneten nächsten Prozess wird nach Flammen in den Intensitätsbildern Xij(t) gesucht und es erfolgt eine Lokalisierung der gefundenen Flammen in entsprechenden Bildausschnitten. Diese Lokalisierung erfolgt mit Hilfe einer so genannten Akkumulationsmatrix, die auf folgende Weise gebildet wird:
- In einem ersten Schritt erfolgt die Bestimmung des Maximalwerts max [Xij(t)] und des Mittelwerts mean [Xij(t)] der Intensität und daraus wird eine Helligkeitschwelle q(t) bestimmt, wobei gilt:
- Mit Hilfe dieser beiden Bedingungen wird ein die Flammeneigenschaften berücksichtigender Gewichtungsfaktor wij(t) bestimmt:
- In a first step, the determination of the maximum value max [X ij (t)] and the mean value mean [X ij (t)] of the intensity and from this a brightness threshold q (t) is determined, where:
- With the aid of these two conditions, a weighting factor w ij (t) which takes into account the flame properties is determined:
Das bedeutet, dass alle Pixel mit einer Intensität unterhalb des Werts max [Xij(t)] - q(t), also dunkle Objekte, heraus gefiltert und nicht weiter berücksichtigt werden. Wie sich gleich zeigen wird, sind es dunkle, bewegte Objekte, die heraus gefiltert werden.This means that all pixels with an intensity below the value max [X ij (t)] - q (t) , ie dark objects, are filtered out and ignored. As will be seen shortly, they are dark, moving objects that are filtered out.
Da man davon ausgehen kann, dass eine Flamme als Bewegung von hoher Lichtintensität erkennbar ist, bildet man durch Vergleich aufeinander folgender Bilder ein Differenzbild, um eine solche Bewegung herauszufinden; da dunkle Objekte aufgrund der Definition des Gewichtungsfaktors wij(t) heraus fallen, werden also bei der Bildung des Differenzbildes bewegte, dunkle Objekte, die keine Flammen sein können, heraus gefiltert. Man sucht also nach Gebieten hoher Lichtintensität und lokaler Flackerbewegung und das bedeutet, dass beispielsweise eine stationäre Lichtquelle, die nicht flackert, nicht als Flamme interpretiert würde und ebenso auch nicht eine quer durch den Überwachungsraum bewegte Lampe.Since one can assume that a flame is recognizable as a movement of high light intensity, a difference image is formed by comparing successive images to find out such a movement; since dark objects fall out due to the definition of the weighting factor w ij (t), moving, dark objects that can not be flames are filtered out when forming the difference image. So you are looking for areas of high light intensity and local flickering and that means that, for example, a stationary light source that does not flicker would not be interpreted as a flame and also not a transversely across the interstitial space moving lamp.
Für das Differenzbild Qij(t) gilt:
Dann wird aus dem Differenzbild Qij(t) die Akkumulationsmatrix Aij(t) bestimmt:
α ist eine Konstante zwischen 0 und 1, die angibt, wie stark das Differenzbild Qij(t) in die Akkumulationsmatrix Aij(t) einfliesst. Für α = 0 wird die Akkumulationsmatrix gleich dem Differenzbild und für α = 1 hat das Differenzbild keinen Einfluss mehr, weil Aij(t) gleich Aij(t-1) ist. Die Akkumulationsmatrix wird in erster Linie deswegen gebildet, um ein geglättetes Bild ohne Rauschen und kurzfristige Änderungen zu erhalten.α is a constant between 0 and 1, which indicates how much the difference image Q ij (t) flows into the accumulation matrix A ij (t) . For α = 0, the accumulation matrix becomes equal to the difference image, and for α = 1, the difference image no longer has any influence because A ij (t) equals A ij (t-1) . The accumulation matrix is formed primarily to obtain a smoothed image with no noise and short-term changes.
Als letzter Schritt der Vorverarbeitung 4 wird mit Hilfe der Akkumulationsmatrix das Pixel oder die Pixel [im,jm](t) mit dem höchsten Wert gesucht und es wird ein dieses oder diese Pixel enthaltender, so genannter interessierender Bildbereich ROI definiert, in dem sich eine Flamme befinden könnte:
Diese Bestimmung liefert also die Koordinaten der Pixel mit lokaler Flackerbewegung und maximaler Helligkeit. Es wird sich in der Regel um eine einziges Pixel handeln, wobei aber selbstverständlich auch mehrere hellste Pixel bestimmt werden können, wozu man sich einer Mehrkanal-Selektion bedienen kann und vorzugsweise Mindestabstände zwischen den einzelnen Pixeln festlegt.This determination thus provides the coordinates of the pixels with local flickering and maximum brightness. It will usually be a single pixel, but of course also several brightest pixels can be determined, what you can use a multi-channel selection and preferably specifies minimum distances between the individual pixels.
In dem mit Analyse 5 bezeichneten nächsten Prozess erfolgt zunächst eine deutliche Datenreduktion, indem die Analyse nicht im gesamten ursprünglichen Bild von 352 mal 288 Pixel erfolgt, sondern im interessierenden Bildbereich ROI einer reduzierten Grösse von beispielsweise 32 mal 32 Pixel. Das ergibt eine Reduktion auf ein Hunderstel. Diese Reduktion kann selbstverständlich auch geringer sein, beispielsweise auf ein Fünfzigstel, oder auch wesentlich stärker.In the next process designated by
Dann werden für jeden interessierenden Bildbereich die folgenden Bildinformationen bestimmt:
- Mittlere Helligkeit L(t) des interessierenden Bildbereichs X ROI (t):
- Chrominanz C(t) des interessierenden Bildbereichs X ROI (t):
- C(t) = [Anzahl der Cij(t)|ROI] ∈ "Feuer-Chroma-Sektor"/R(t), wobei Cij(t) das Chrominanzpaar (Vij, Uij) des Bildes Xij(t) zur Zeit t bezeichnet. YUV ist eine bekannte Darstellung des Farbraums, mit zwei Farbkomponenten U und V auf der x- bzw. der y-Achse und der Intensität Y auf der z-Achse, wobei die Länge des Vektors vom Nullpunkt zu einem Pixel in der UV-Ebene die Farbsättigung dieses Pixels angibt. Der Feuer-Chroma-Sektor R(t) ist ein Sektor des Farbraums in der UV-Ebene, in dem für eine Flamme typischen Farbbereich, der insbesondere die Farbe rot enthält.
- Anzahl der aktiven Pixel R(t) der Akkumulationsmatrix A ROI(t):
- Sättigungsgrad S(t) des interessierenden Bildbereichs X ROI (t):
- Average brightness L (t) of the image area of interest X ROI (t) :
- Chrominance C (t) of the image area of interest X ROI (t) :
- C (t) = [number of C ij (t) | ROI ] ∈ "Fire Chroma Sector" / R (t) , where C ij (t) denotes the chrominance pair (V ij , U ij ) of the image X ij (t) at time t. YUV is a well-known representation of the color space, with two color components U and V on the x and the y-axis and the intensity Y on the z-axis, where the length of the vector from the zero point to a pixel in the UV plane Indicates the color saturation of this pixel. The fire chroma sector R (t) is a sector of the color space in the UV plane, in the color region typical for a flame, which in particular contains the color red.
- Number of active pixels R (t) of the accumulation matrix A ROI ( t ):
- Saturation degree S (t) of the image area of interest X ROI (t) :
Damit das Ergebnis stabil bleibt, erfolgt anschliessend in einem mit Extraktion 6 bezeichneten Prozess eine Zeitintegration der bei der Analyse 5 bestimmten Bildinformationen. Wenn die Integration beispielsweise über 1 Sekunde durchgeführt wird, erstreckt sie sich im PAL-Format über 25 Bilder. Man integriert also die mittlere Helligkeit, die Chrominanz, die aktiven Pixel und den Sättigungsgrad über die Zeit t von t0 bis tn und erhält die folgenden Eigenschaften:
- Mittelwert Helligkeit: FL = L
- Mittelwert Frequenz: FF = F
- Mittelwert Amplitude: FM = M
- Mittelwert Feuer-Chroma-Pixel: FC = C
- Mittelwert aktive Pixel: FR = R
- Mittelwert Sättigung: FS = S
- Average brightness: F L = L
- Mean frequency: F F = F
- Mean value amplitude: F M = M
- Average Fire Chroma Pixel: F C = C
- Mean active pixels: F R = R
- Mean saturation: F S = S
Den Mittelwert der Frequenz erhält man beispielsweise durch Zählen der Pixel der mittleren Helligkeit L(t). Die Frequenz, die durch das charakteristische Flackern einer Flamme verursacht wird, ist eine für die Detektion einer Flamme wichtige Grösse, weil sie in einem definierten engen Bereich von in der Regel zwischen 1 Hz und 10 Hz liegt.The mean value of the frequency is obtained, for example, by counting the pixels of the average brightness L (t) . The frequency caused by the characteristic flickering of a flame is an important quantity for the detection of a flame because it is within a defined narrow range of usually between 1 Hz and 10 Hz.
Im anschliessenden mit Mustererkennung 7 bezeichneten Prozess wird aus den bei der Extraktion 6 gewonnenen Eigenschaften die Wahrscheinlichkeit berechnet, dass eine Flamme vorliegt. Dabei wird beispielsweise für jede der obigen Eigenschaften untersucht, ob der Mittelwert oberhalb oder unterhalb eines Schwellwertes liegt und die Wahrscheinlichkeit entsprechend gleich eins bzw. gleich null gesetzt. Dann wird aus den Wahrscheinlichkeiten aller n Eigenschaften eine Gesamtwahrscheinlichkeit gebildet.
Gesamtwahrscheinlichkeit Π(t):
In dem mit Entscheidung 8 bezeichneten Prozess erfolgt anschliessend die Entscheidung, ob Alarm ausgelöst wird. Dieser Prozess enthält eine Integration, bei der die Gesamtwahrscheinlichkeit Π(t) über aufeinanderfolgende Bilder aufintegriert wird. Die Integration beginnt bei null und zählt für jedes Π(t) > κ (κ ist eine Schwelle) ein Inkrement dazu und zieht für jedes Π(t) < κ ein Inkrement ab. Wenn I(t) den Wert des Integrals bezeichnet, gilt:
- wenn Π(t) > κ
- dann I(t) = I(t-1) + σ+ (gesättigt zu S+, wenn I(t) > S+ in allen anderen Fällen gilt I(t) = I(t-1) - σ- (gesättigt zu S- (üblicherweise 0), wenn I(t) < S.
- if Π (t) > κ
- then I (t) = I (t-1) + σ + (saturated to S + , if I (t) > S + in all other cases I (t) = I (t-1) - σ - (saturated to S - (usually 0) if I (t) <S.
Mit Hilfe des Integrals I(t) erfolgt nun die Entscheidung, ob ein Alarm ausgelöst wird:
- Wenn I(t) > β (β ist eine Schwelle), wird Alarm ausgelöst, in allen anderen Fällen nicht.
- If I (t) > β (β is a threshold), the alarm is triggered, but not in all other cases.
Die beschriebene Einrichtung hat den Vorteil, dass in vielen Anwendungsfällen auf bereits installierte Videokameras zurückgegriffen werden kann und eine Installation spezieller Flammensensoren nicht erforderlich ist, was zweifellos eine Kostenreduktion bedeutet. Eine weitere Kostenreduktion ergibt sich durch die Beschränkung der Auswertung auf die eventuell eine Flamme enthaltenden Bildausschnitte, was eine deutliche Reduktion der Rechnerleistung ermöglicht. Man kann auch davon ausgehen, dass die Auswertung dieser Bildausschnitte gegenüber Störungen ausreichend robust ist.The described device has the advantage that can be used in many applications on already installed video cameras and installation of special flame sensors is not required, which undoubtedly means a cost reduction. A further cost reduction results from the restriction of the evaluation to the possibly containing a flame image sections, which allows a significant reduction of computer power. It can also be assumed that the evaluation of these image sections is sufficiently robust to interference.
Claims (15)
- Method for detecting flames in a monitored zone, referred to hereinafter as monitoring space, by analysis of at least one parameter of a radiation that occurs in the monitoring space, in which case a video image of the monitoring space is generated and zones having high light intensity and local flicker motion are sought in said video image, in which case, in a first step, said zones are localized and the relevant image excerpts are subsequently analyzed with regard to the presence of a flame, characterized in that the search for zones having high light intensity and local flicker motion is effected with the aid of an accumulation matrix [Aij(t)], which is obtained from the difference images of successive intensity images [Xij(t)], said difference images being weighted with a weighting factor, the weighting factor specifying the extent to which the difference images influence the accumulation matrix [Aij(t)].
- Method according to claim 1, characterized in that the video images are generated with a specific frequency and intensity images [Xij(t)] are obtained therefrom.
- Method according to claim 1, characterized in that all pixels having a brightness that lies below a predetermined threshold and thus all moving, dark objects are filtered out during the formation of the difference images and thus of the accumulation matrix [Aij(t)].
- Method according to claim 3, characterized in that the coordinates of the brightest pixels are sought with the aid of the accumulation matrix [Aij(t)].
- Method according to claim 4, characterized in that an image region of interest [ROI] which contains the brightest pixel or pixels and is reduced with respect to the original image is defined and analyzed with regard to the presence of a flame.
- Method according to claim 5, characterized in that the size of the image region of interest (ROI) amounts at most to one fiftieth of the size of the original image.
- Method according to claim 6, characterized in that the image information items of brightness [L(t)], chrominance [C(t)], number of active pixels [R(t)] above a specific intensity threshold, and the saturation [S(t)], are determined in the image region of interest [ROI].
- Method according to claim 7, characterized in that said image information items are integrated over a specific time and thus over a plurality of images and their mean value is determined in that the mean value of the frequency [F] and the mean value of the amplitude [M] are determined as additional parameters during the integration, and in that the probability for the presence of a flame is calculated for each of said mean values.
- Method according to claim 8, characterized in that the probabilities of the mean values are used to calculate an overall probability for the presence of a flame in the reduced image region [ROI], in that said overall probability is integrated over a plurality of images, and in that an alarm is triggered when a threshold is exceeded by the integrated value.
- Device for detecting flames in a monitored zone, referred to hereinafter as monitoring space, by analysis of at least one parameter of a radiation that occurs in the monitoring space, characterized by a video camera [1] with an evaluation stage [2] for the images supplied by the camera [1], the evaluation stage [2] having a processor with an algorithm for the localization of regions having high light intensity and local flicker motion in the images of the camera [1] and the subsequent analysis of the corresponding image excerpts with regard to the presence of a flame, and the algorithm containing a process, referred to below as preprocessing [4], during which an accumulation matrix [Aij(t)] is determined for the search for zones having high light intensity and local flicker motion, said accumulation matrix being obtained from the difference images of successive intensity images [Xij(t)], said difference images being weighted with a weighting factor, the weighting factor specifying the extent to which the difference images influence the accumulation matrix [Aij(t)].
- Device according to claim 10, characterized in that the algorithm contains a process, referred to below as image obtaining [3], during which intensity images [Xij(t)] are obtained from the video images generated with a specific frequency.
- Device according to claim 10, characterized in that during the preprocessing [4], with the aid of the accumulation matrix [Aij(t)], the coordinates of the brightest pixels are determined and an image region of interest (ROI) which contains the brightest pixel or pixels and is reduced with respect to the original image is defined.
- Device according to claim 12, characterized in that, the algorithm contains a process, referred to below as analysis [5], for the analysis of the image region of interest [ROI] during which analysis the image information items of brightness [L(t)], chrominance [C(t)], number of active pixels [R(t)] above a specific intensity threshold, and the saturation [S(t)] are determined.
- Device according to claim 13, characterized in that the algorithm contains a process, referred to below as extraction [6], during which said image information items are integrated over a specific time and thus over a plurality of images and the mean values of the image information items are determined, in that the mean value of the frequency [F] and the mean value of the amplitude [M] are determined as additional parameters during the integration, and in that the probability for the presence of a flame is calculated for each of said mean values.
- Device according to claim 14, characterized in that the algorithm contains a process of pattern recognition [7] and a process of decision [8], during which the probabilities of the mean values are used to calculate an overall probability for the presence of a flame in the reduced image region [ROI] and said overall probability is integrated over a plurality of images and an alarm is triggered in the event of a threshold being exceeded by the integrated value.
Priority Applications (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AT03015846T ATE357714T1 (en) | 2003-07-11 | 2003-07-11 | METHOD AND DEVICE FOR DETECTING FLAMES |
EP03015846A EP1496483B1 (en) | 2003-07-11 | 2003-07-11 | Method and apparatus for the detection of flames |
DE50306852T DE50306852D1 (en) | 2003-07-11 | 2003-07-11 | Method and device for detecting flames |
ES03015846T ES2282550T3 (en) | 2003-07-11 | 2003-07-11 | PROCEDURE AND DEVICE FOR THE DETECTION OF FLAMES. |
AU2004202851A AU2004202851B2 (en) | 2003-07-11 | 2004-06-25 | Method and Device for Detecting Flames |
KR1020040050227A KR20050009135A (en) | 2003-07-11 | 2004-06-30 | Method and device for detecting flames |
PL04369016A PL369016A1 (en) | 2003-07-11 | 2004-07-09 | Method and device designed for flame detection |
NO20042948A NO330182B1 (en) | 2003-07-11 | 2004-07-09 | Flame detection method and apparatus |
CN200410063587A CN100595583C (en) | 2003-07-11 | 2004-07-12 | Method and apparatus for the detection of flames |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP03015846A EP1496483B1 (en) | 2003-07-11 | 2003-07-11 | Method and apparatus for the detection of flames |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1496483A1 EP1496483A1 (en) | 2005-01-12 |
EP1496483B1 true EP1496483B1 (en) | 2007-03-21 |
Family
ID=33442787
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03015846A Expired - Lifetime EP1496483B1 (en) | 2003-07-11 | 2003-07-11 | Method and apparatus for the detection of flames |
Country Status (9)
Country | Link |
---|---|
EP (1) | EP1496483B1 (en) |
KR (1) | KR20050009135A (en) |
CN (1) | CN100595583C (en) |
AT (1) | ATE357714T1 (en) |
AU (1) | AU2004202851B2 (en) |
DE (1) | DE50306852D1 (en) |
ES (1) | ES2282550T3 (en) |
NO (1) | NO330182B1 (en) |
PL (1) | PL369016A1 (en) |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20060041555A (en) * | 2004-11-09 | 2006-05-12 | 한국서부발전 주식회사 | System and method for detecting and alarming a fire of thermal power plants |
US7289032B2 (en) * | 2005-02-24 | 2007-10-30 | Alstom Technology Ltd | Intelligent flame scanner |
KR100680114B1 (en) * | 2005-05-12 | 2007-02-07 | (주)에이치엠씨 | Device, method and recording medium of robust fire detecting using color of image |
US7868772B2 (en) | 2006-12-12 | 2011-01-11 | Industrial Technology Research Institute | Flame detecting method and device |
CN101316371B (en) * | 2007-05-31 | 2012-11-28 | 财团法人工业技术研究院 | Flame detecting method and device |
EP2000998B1 (en) * | 2007-05-31 | 2013-01-02 | Industrial Technology Research Institute | Flame detecting method and device |
DE112009003247A5 (en) * | 2008-11-03 | 2012-05-03 | IQ Wireless Entwicklungsges. für Systeme und Technologien der Telekommunikation mbH | METHOD AND DEVICE FOR THE NOMINANT DETECTION OF FIRE AND DISTINCTION OF ARTIFICIAL LIGHT SOURCES |
CN101441712B (en) * | 2008-12-25 | 2013-03-27 | 北京中星微电子有限公司 | Flame video recognition method and fire hazard monitoring method and system |
CN101847304B (en) * | 2009-12-04 | 2012-05-23 | 四川川大智胜软件股份有限公司 | Image-based method of finding flames with large-space intelligent fire-fighting system |
CN102645246B (en) * | 2011-02-17 | 2014-04-23 | 中国石油化工股份有限公司 | Method for real-time measurement of torch flow rate of torch discharge system based on torch videos |
CN103776056B (en) * | 2012-10-25 | 2016-04-20 | 宁波立诚电子制造有限公司 | A kind of lighter detection method and device thereof |
CN107064113B (en) * | 2017-06-13 | 2020-02-11 | 华电青岛发电有限公司 | System and method for detecting pulverized coal combustion quality of burner by using optical fiber |
CN111141504B (en) * | 2019-12-25 | 2022-04-15 | Oppo(重庆)智能科技有限公司 | Fire-break detection method and device and computer readable storage medium |
CN111583610B (en) * | 2020-04-30 | 2021-07-16 | 深圳市前海用电物联网科技有限公司 | Fire-fighting linkage control method and system of causal model |
CN113436406B (en) * | 2021-08-25 | 2021-11-12 | 广州乐盈信息科技股份有限公司 | Sound-light alarm system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB9216811D0 (en) * | 1992-08-07 | 1992-09-23 | Graviner Ltd Kidde | Flame detection methods and apparatus |
EP1256105B1 (en) * | 2000-02-07 | 2006-09-20 | VSD Limited | Smoke and flame detection |
WO2002054364A2 (en) * | 2000-12-28 | 2002-07-11 | Siemens Building Technologies Ag | Video smoke detection system |
RU2003133287A (en) * | 2001-05-11 | 2005-05-27 | Детектор Электроникс Корпорэйшн (Us) | METHOD AND DEVICE FOR FLAME DETECTION BY FORMING FLAME IMAGES |
-
2003
- 2003-07-11 DE DE50306852T patent/DE50306852D1/en not_active Expired - Lifetime
- 2003-07-11 ES ES03015846T patent/ES2282550T3/en not_active Expired - Lifetime
- 2003-07-11 AT AT03015846T patent/ATE357714T1/en not_active IP Right Cessation
- 2003-07-11 EP EP03015846A patent/EP1496483B1/en not_active Expired - Lifetime
-
2004
- 2004-06-25 AU AU2004202851A patent/AU2004202851B2/en not_active Ceased
- 2004-06-30 KR KR1020040050227A patent/KR20050009135A/en not_active Application Discontinuation
- 2004-07-09 PL PL04369016A patent/PL369016A1/en not_active Application Discontinuation
- 2004-07-09 NO NO20042948A patent/NO330182B1/en not_active IP Right Cessation
- 2004-07-12 CN CN200410063587A patent/CN100595583C/en not_active Expired - Fee Related
Also Published As
Publication number | Publication date |
---|---|
NO330182B1 (en) | 2011-02-28 |
DE50306852D1 (en) | 2007-05-03 |
KR20050009135A (en) | 2005-01-24 |
CN100595583C (en) | 2010-03-24 |
NO20042948L (en) | 2005-01-12 |
ES2282550T3 (en) | 2007-10-16 |
AU2004202851A1 (en) | 2005-01-27 |
AU2004202851B2 (en) | 2008-08-28 |
NO20042948D0 (en) | 2004-07-09 |
EP1496483A1 (en) | 2005-01-12 |
PL369016A1 (en) | 2005-01-24 |
ATE357714T1 (en) | 2007-04-15 |
CN1576839A (en) | 2005-02-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1496483B1 (en) | Method and apparatus for the detection of flames | |
DE60105006T2 (en) | PROCESS AND SYSTEM FOR FIRE FIGHTER IDENTIFICATION | |
EP1628260B1 (en) | Method and system for automatic forest fire recognition | |
DE19934171B4 (en) | Filter system and method | |
DE102013017395B3 (en) | Method and device for automated early forest fire detection by means of optical detection of clouds of smoke | |
EP0939387B1 (en) | Room supervision device | |
EP1346330B1 (en) | Video smoke detection system | |
DE102012206078A1 (en) | Determining a number of objects in an IR image | |
EP2677460B1 (en) | Method for determining the length of a queue | |
DE112017007246T5 (en) | IMAGE ANALYSIS DEVICE, IMAGE ANALYSIS METHOD AND IMAGE ANALYSIS PROGRAM | |
EP3453000B1 (en) | Smoke detection device, method for detecting smoke from a fire, and computer program | |
DE60123214T2 (en) | SMOKE AND FLAME DETECTION | |
DE102019103963A1 (en) | ILDGEBUNGSVORRICHTUNG | |
DE4332753C2 (en) | Process for the detection of moving objects | |
DE112007000738B4 (en) | Filter for disturbing movements | |
EP1239433A1 (en) | Method and apparatus for the detection of smoke and / or fire in spaces | |
DE19883010B4 (en) | Method and device for recognizing a moving object in a sequence of color frames | |
DE102007016922A1 (en) | Method for detecting defects on the backside of a semiconductor wafer | |
EP0834845A1 (en) | Method for frequency analysis of a signal | |
EP0951776A1 (en) | Brightness control on video cameras | |
DE3214254A1 (en) | Method for detecting movements in video camera images | |
DE102012109390A1 (en) | Monitoring device for safety-critical unit, particularly conveyer of conveyer system, has first video sensor unit, which is adapted to generate first image as representation formed from line by first monitoring area of safety-critical unit | |
DE4105516A1 (en) | METHOD AND DEVICE FOR FILTERING SIGNALS | |
EP1762995A1 (en) | Detection of smoke with a video camera | |
DE4002829A1 (en) | METHOD FOR DETECTING METAL OBJECTS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
17P | Request for examination filed |
Effective date: 20050718 |
|
AKX | Designation fees paid |
Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: SIEMENS SCHWEIZ AG |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 Ref country code: IE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REF | Corresponds to: |
Ref document number: 50306852 Country of ref document: DE Date of ref document: 20070503 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
GBT | Gb: translation of ep patent filed (gb section 77(6)(a)/1977) |
Effective date: 20070522 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070821 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2282550 Country of ref document: ES Kind code of ref document: T3 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FD4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 |
|
26N | No opposition filed |
Effective date: 20071227 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20070731 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070622 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: TP |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20070711 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PUE Owner name: SIEMENS AKTIENGESELLSCHAFT Free format text: SIEMENS SCHWEIZ AG#ALBISRIEDERSTRASSE 245#8047 ZUERICH (CH) -TRANSFER TO- SIEMENS AKTIENGESELLSCHAFT#WITTELSBACHERPLATZ 2#80333 MUENCHEN (DE) |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: BE Payment date: 20080715 Year of fee payment: 6 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20090514 AND 20090520 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20070711 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070321 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20070922 |
|
BERE | Be: lapsed |
Owner name: SIEMENS SCHWEIZ A.G. Effective date: 20090731 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20090731 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: SD Effective date: 20110318 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20120726 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20120703 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20120830 Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: V1 Effective date: 20140201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130711 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PUE Owner name: SIEMENS SCHWEIZ AG, CH Free format text: FORMER OWNER: SIEMENS AKTIENGESELLSCHAFT, DE |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20150220 AND 20150225 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 50306852 Country of ref document: DE Owner name: SIEMENS SCHWEIZ AG, CH Free format text: FORMER OWNER: SIEMENS AKTIENGESELLSCHAFT, 80333 MUENCHEN, DE Effective date: 20150407 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FD2A Effective date: 20150709 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 13 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130712 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: TP Owner name: SIEMENS SCHWEIZ AG, CH Effective date: 20160202 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 14 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20160920 Year of fee payment: 14 Ref country code: GB Payment date: 20160713 Year of fee payment: 14 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20160725 Year of fee payment: 14 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20161012 Year of fee payment: 14 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 50306852 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20170711 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20180330 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170731 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170731 Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180201 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170711 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170731 |