EP2194353A1 - Method and system for detecting improvised explosive devices or similar - Google Patents

Method and system for detecting improvised explosive devices or similar Download PDF

Info

Publication number
EP2194353A1
EP2194353A1 EP09177072A EP09177072A EP2194353A1 EP 2194353 A1 EP2194353 A1 EP 2194353A1 EP 09177072 A EP09177072 A EP 09177072A EP 09177072 A EP09177072 A EP 09177072A EP 2194353 A1 EP2194353 A1 EP 2194353A1
Authority
EP
European Patent Office
Prior art keywords
images
current
route
database
previous
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP09177072A
Other languages
German (de)
French (fr)
Other versions
EP2194353B1 (en
Inventor
Christophe Hubert-Habart
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institut Franco Allemand de Recherches de Saint Louis ISL
MBDA France SAS
Original Assignee
MBDA France SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MBDA France SAS filed Critical MBDA France SAS
Publication of EP2194353A1 publication Critical patent/EP2194353A1/en
Application granted granted Critical
Publication of EP2194353B1 publication Critical patent/EP2194353B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F42—AMMUNITION; BLASTING
    • F42D—BLASTING
    • F42D5/00—Safety arrangements
    • F42D5/02—Locating undetonated charges
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F41—WEAPONS
    • F41H—ARMOUR; ARMOURED TURRETS; ARMOURED OR ARMED VEHICLES; MEANS OF ATTACK OR DEFENCE, e.g. CAMOUFLAGE, IN GENERAL
    • F41H11/00—Defence installations; Defence devices
    • F41H11/12—Means for clearing land minefields; Systems specially adapted for detection of landmines

Definitions

  • the present invention relates to a method and system for detecting improvised explosive devices (IEDs) or the like.
  • IEDs improvised explosive devices
  • Improvised explosive devices are known to be increasingly used to trap a route, such as a road or road and their aisles, from a conflict zone, because they are easily concealed by reason of their small size and that they can be designed and manufactured relatively easily and artisanally from components available commercially and / or mounted on equipment and appliances public by being diverted from their initial application. Thus, they pose a significant threat to vehicles and individuals, civilian or military, who use these trapped routes.
  • the present invention aims to remedy these drawbacks and relates to a method and a system for detecting improvised explosive devices, which can be used on a moving vehicle without being constrained vis-à-vis the people in the vehicle or close to it.
  • the image correlation detection method is totally passive since it emits no dangerous radiation and is not subject to regulatory constraints as a result of use, so that can be put technically into application very easily.
  • Such an image analysis method makes it possible to notice a change in the current image compared to a previous image, which change may correspond to the presence of an explosive device, and therefore operates in real time with a moving vehicle.
  • the method consists in replacing one current image on two by an earlier image and successively displaying said current and previous images thus alternated.
  • Alternate image groups greater than two images
  • Any change between images (or groups of images) representative of the road and its surroundings is thus highlighted step by step along the route, which allows to secure it at best.
  • the method further comprises, before displaying said current and previous images, to readjust the geographical position of said current and previous successive images, so that the fixed points of said successive images coincide.
  • the invention also relates to a system for detecting improvised explosive devices or the like, intended to implement the method defined above.
  • Such a system easily integrates into a vehicle because of its reduced size and consumes, moreover, little energy for its operation, since its source of shooting is a simple camera and its display device a monitor.
  • the detection range of the system can be important since it depends on the resolution of the sensor and the characteristics of the camera lens. A range of a few tens to a few hundred meters is thus attainable depending on the size of the objects to be detected.
  • said processing device replaces a current two-image taken by said source by an earlier image of said database, so as to display on said display device an alternating succession of said current and previous images.
  • the system further includes means for re-aligning the geographical position of two successive current and previous images respectively, so that the fixed points of the images coincide.
  • said database of previous images contains images provided by the same source of shooting, conferring on the system an important reliability.
  • said current images provided by said source are sent into said database, so that it is updated regularly every time the vehicle is on the route.
  • the figure 1 schematically represents the improvised explosive device detection system applying the method according to the invention.
  • the detection system 1 is mounted on board a vehicle 2 generally of the military type, symbolized by a mixed rectangular line and traveling on a route (road or track) likely to include improvised explosive devices that can be totally or partially buried and / or concealed in the road or its aisles.
  • the system comprises, as will be seen in more detail below, a current image shooting source 3, a database 4 of reference images IA of said route, a processing device 5, an image registration means 6 and a display device 7.
  • the shooting source 3 is a camera 3A embedded in the vehicle 2 and making it possible to take the current images I of the route and thus to display on the display device 7, the road and the aisles of the route on which the vehicle travels, so as to deliver in real time and continuously, the current images of the route.
  • the database 4 contains previous IA images of the same route but previously filmed and contained on any appropriate computer storage medium, integrated into the vehicle. These images may have been taken by the same camera during a previous pass on the same route.
  • the image processing device 5 is connected to the camera 3A and to the database 4 and thus receives the information (images I and IA) sent by the camera 3A and the database 4.
  • the role of the processing device 5 is to replace current images I delivered by the camera by previous images IA provided by the database 4 and located in the same place.
  • a current image I out of two is replaced by an earlier image IA, so that the alternative display of the current images I of the camera and the previous images IA of the
  • the database allows you to immediately see the changes that have occurred along the current route compared to the previous one.
  • the display device 7 is for example a visualization monitor 7A.
  • the two images provided, respectively current I and previous IA, at a given moment in the same place may not be strictly identical as regards their spatial positioning, geographical.
  • the vehicles do not travel exactly the same way along the route and / or the cameras are not placed in the same place on the vehicles. It is thus seen that the route R of the IT route with its two trees AR of the current image I delivered by the camera 3A is slightly shifted to the right with respect to the same route R of the IT route with the same two AR trees of the previous image IA issued by the database 4, which road is in the center of the previous image IA.
  • the system 1 of the invention provides this and its resetting means 6 can consist of several interventions, for example: either to reset the current image I filmed by the camera 3A with that IA of the database 4 from the position of the vehicle or from the position of the vehicle and the orientation of the camera, so as to superimpose said images I and IA, either to use equipment specific to the detection system 1 or to use specific equipment of the vehicle such as that a satellite positioning system, inertial center, if it has one, which equipment allows to know where were taken the images I and IA, thus facilitating their registration.
  • Such a registration of the current images I and earlier IA is also necessary to ensure a good visual comfort of the operator in charge of monitoring the images scrolling alternately on the display monitor 7A.
  • the system 1 can identify, by image correlation, any change occurring between successive images scrolling on the monitor 7A.
  • any change occurring between successive images scrolling on the monitor 7A For example, it can be seen that on the current image I represented and delivered by the camera 3A, as well as on the imaged IR image of monitor 7A, there is a black spot T, on the left-hand side of the road R, while it does not appear on the previous image IA.
  • the operator sees a flashing on the monitor 7A representative of the change between the successive images and corresponding in the example to the spot T, which can be an improvised explosive device. If this is the case, the latter thus detected is then neutralized.
  • the images taken by the camera during the displacement are used to feed, via a link 8, the database 4 which is thus updated.
  • the replacement of old images with recent images makes it possible to overcome slow natural changes in the landscape, such as those of vegetation.
  • the image processing device 5 can be performed automatically to detect and highlight the elements that have changed in the landscape.
  • the image processing device 5 can also act on the images so as to reduce the effect of brightness variations or shadows worn between the two images that have been taken at different times.
  • the image database referenced 4 can of course be shared between several vehicles that can use for example the same portion of the route.

Abstract

The method involves continuously delivering current images (I) representing a followed route by correlation of the images during detection. The current images are replaced by previously recorded anterior images (IA) captured at the same route. The current images and the anterior images are successively displaced to allow detection of an explosive device by correlation of the current and anterior images. A geographical position of the current and anterior images is readjusted before displaying the current and anterior images. An independent claim is also included for a system for detecting an explosive device dissimulated partially or completely on a route or near the route, from a vehicle circulating on the route.

Description

La présente invention concerne un procédé et un système de détection d'engins explosifs improvisés (EEI) ou analogues.The present invention relates to a method and system for detecting improvised explosive devices (IEDs) or the like.

On sait que les engins explosifs improvisés sont de plus en plus employés pour piéger un itinéraire, tel qu'une route ou une piste et leurs bas-côtés, d'une zone de conflit, du fait qu'ils sont facilement dissimulables en raison de leur dimension réduite et qu'ils peuvent être conçus et fabriqués relativement facilement et artisanalement à partir de composants disponibles commercialement et/ou montés sur des matériels et appareils grands publics en étant détournés de leur application initiale. Ainsi, ils constituent une menace importante pour les véhicules et les individus, civils ou militaires, qui empruntent ces itinéraires piégés.Improvised explosive devices are known to be increasingly used to trap a route, such as a road or road and their aisles, from a conflict zone, because they are easily concealed by reason of their small size and that they can be designed and manufactured relatively easily and artisanally from components available commercially and / or mounted on equipment and appliances public by being diverted from their initial application. Thus, they pose a significant threat to vehicles and individuals, civilian or military, who use these trapped routes.

Pour sécuriser une telle route ou piste, on peut par exemple utiliser des systèmes destructifs attelés à des véhicules motorisés et à base de mécanismes de leurrage ou neutralisation qui déclenchent, lorsqu'ils passent sur les engins, l'explosion de leur charge entraînant par ailleurs l'endommagement d'une partie des mécanismes, ou, des systèmes non destructifs auxquels l'invention appartient et qui permettent de détecter des engins explosifs et, donc, de localiser lesdits engins à partir entre autres de rayons X ou analyses neutroniques.To secure such a road or track, one can for example use destructive systems coupled to motorized vehicles and based on decoy or neutralization mechanisms that trigger, when they pass on the gear, the explosion of their load causing otherwise damage to a part of the mechanisms, or non-destructive systems to which the invention belongs and which can detect explosive devices and, therefore, to locate said machines from among others X-rays or neutron analyzes.

Cependant, ces derniers systèmes sont difficilement applicables à un véhicule en mouvement destiné à la détection d'engins explosifs en raison notamment :

  • du délai des mesures trop important nuisant à la progression du véhicule,
  • de la faible portée des systèmes inférieure à quelques mètres, et
  • de la législation sur l'utilisation de rayons X ou neutroniques sur les personnes.
However, the latter systems are difficult to apply to a moving vehicle intended for the detection of explosive devices, in particular because:
  • excessively long measurement delays affecting the vehicle's progress,
  • the short range of systems less than a few meters, and
  • legislation on the use of X-rays or neutrons on people.

La présente invention a pour but de remédier à ces inconvénients et concerne un procédé et un système de détection d'engins explosifs improvisés, qui peuvent être utilisables sur un véhicule en mouvement en étant sans contrainte vis-à-vis des personnes situées dans le véhicule ou proches de celui-ci.The present invention aims to remedy these drawbacks and relates to a method and a system for detecting improvised explosive devices, which can be used on a moving vehicle without being constrained vis-à-vis the people in the vehicle or close to it.

A cet effet, le procédé de détection d'engins explosifs improvisés ou analogues dissimulés partiellement ou totalement sur un itinéraire ou à proximité de celui-ci, à partir d'un véhicule circulant sur ledit itinéraire, est remarquable, selon l'invention, en ce qu'il est du type par corrélation d'images et consiste, lors d'une mission de détection :

  • à délivrer en continu des images actuelles représentatives dudit itinéraire suivi ;
  • à remplacer au moins certaines desdites images actuelles par des images antérieures précédemment enregistrées, prises au même endroit ; et
  • à afficher successivement lesdites images actuelles et lesdites images antérieures de remplacement pour permettre, par la corrélation des images actuelles et antérieures, la détection desdits engins explosifs.
For this purpose, the method of detecting improvised explosive devices or the like partially or totally concealed on a route or near it, from a vehicle traveling on said route, is remarkable, according to the invention, in what is the type of correlation of images and consists, during a detection mission:
  • continuously delivering current images representative of said tracked route;
  • replacing at least some of said current images with previously recorded prior images taken at the same location; and
  • successively displaying said current images and said replacement prior images to allow, by the correlation of the current and previous images, the detection of said explosive devices.

Ainsi, grâce à l'invention, le procédé de détection par corrélation d'images est totalement passif puisqu'il n'émet aucune radiation dangereuse et ne fait pas l'objet en conséquence de contraintes réglementaires d'utilisation, de sorte qu'il peut être mis techniquement en application très facilement. Un tel procédé par analyse d'images permet de remarquer un changement dans l'image actuelle comparativement à une image antérieure, lequel changement peut correspondre à la présence d'un engin explosif, et fonctionne par conséquent en temps réel avec un véhicule en mouvement.Thus, thanks to the invention, the image correlation detection method is totally passive since it emits no dangerous radiation and is not subject to regulatory constraints as a result of use, so that can be put technically into application very easily. Such an image analysis method makes it possible to notice a change in the current image compared to a previous image, which change may correspond to the presence of an explosive device, and therefore operates in real time with a moving vehicle.

De préférence, pour obtenir une analyse optimale de l'itinéraire emprunté par le véhicule, le procédé consiste à remplacer une image actuelle sur deux par une image antérieure et à afficher successivement lesdites images actuelles et antérieures ainsi alternées. On peut aussi afficher des groupes d'images alternés (supérieurs à deux images). Tout changement entre les images (ou groupes d'images) représentatives de la route et de ses abords est ainsi mis en évidence pas à pas le long de l'itinéraire, ce qui permet de le sécuriser au mieux.Preferably, to obtain an optimal analysis of the route taken by the vehicle, the method consists in replacing one current image on two by an earlier image and successively displaying said current and previous images thus alternated. Alternate image groups (greater than two images) can also be displayed. Any change between images (or groups of images) representative of the road and its surroundings is thus highlighted step by step along the route, which allows to secure it at best.

Avantageusement, le procédé consiste, de plus, avant d'afficher lesdites images actuelles et antérieures, à recaler la position géographique desdites images actuelles et antérieures successives, de manière que les points fixes desdites images successives coïncident.Advantageously, the method further comprises, before displaying said current and previous images, to readjust the geographical position of said current and previous successive images, so that the fixed points of said successive images coincide.

L'invention concerne également un système de détection d'engins explosifs improvisés ou analogues, destiné à mettre en oeuvre le procédé défini précédemment.The invention also relates to a system for detecting improvised explosive devices or the like, intended to implement the method defined above.

A cet effet, le système est remarquable en ce qu'il comprend :

  • au moins une source de prise de vues montée sur ledit véhicule circulant sur ledit itinéraire et délivrant lesdites images actuelles de celui-ci ;
  • une base de données desdites images antérieures précédemment enregistrées dudit itinéraire ;
  • un dispositif de traitement d'images en temps réel auquel sont reliées ladite source de prise de vues et ladite base de données et qui remplace au moins certaines desdites images actuelles par des images antérieures de ladite base de données, prises au même endroit dudit itinéraire ; et
  • un dispositif d'affichage desdites images actuelles et antérieures ainsi combinées par ledit dispositif de traitement.
For this purpose, the system is remarkable in that it comprises:
  • at least one shooting source mounted on said vehicle traveling on said route and delivering said current images thereof;
  • a database of said previously recorded prior images of said route;
  • a real-time image processing device which is connected to said source of photography and said database and which replaces at least some of said current images with previous images of said database, taken at the same place of said route; and
  • a display device of said current and previous images thus combined by said processing device.

Ainsi, un tel système s'intègre aisément dans un véhicule de par sa dimension réduite et consomme, par ailleurs, peu d'énergie pour son fonctionnement, puisque sa source de prise de vues est une simple caméra et son dispositif d'affichage un moniteur. La portée de détection du système peut être importante puisqu'elle dépend de la résolution du capteur et des caractéristiques de l'objectif de la caméra. Une portée de quelques dizaines à quelques centaines de mètres est ainsi atteignable en fonction de la taille des objets à détecter.Thus, such a system easily integrates into a vehicle because of its reduced size and consumes, moreover, little energy for its operation, since its source of shooting is a simple camera and its display device a monitor. The detection range of the system can be important since it depends on the resolution of the sensor and the characteristics of the camera lens. A range of a few tens to a few hundred meters is thus attainable depending on the size of the objects to be detected.

Avantageusement, ledit dispositif de traitement remplace une image actuelle sur deux prise par ladite source par une image antérieure de ladite base de données, de sorte à visualiser sur ledit dispositif d'affichage une succession alternée desdites images actuelles et antérieures.Advantageously, said processing device replaces a current two-image taken by said source by an earlier image of said database, so as to display on said display device an alternating succession of said current and previous images.

Selon une autre caractéristique, le système inclut, de plus, un moyen pour recaler la position géographique de deux images respectivement actuelle et antérieure successives, de manière que les points fixes des images coïncident.According to another characteristic, the system further includes means for re-aligning the geographical position of two successive current and previous images respectively, so that the fixed points of the images coincide.

De préférence, ladite base de données d'images antérieures contient des images fournies par la même source de prise de vues, conférant au système une fiabilité importante.Preferably, said database of previous images contains images provided by the same source of shooting, conferring on the system an important reliability.

Par ailleurs, lesdites images actuelles fournies par ladite source sont envoyées dans ladite base de données, de sorte que celle-ci est mise à jour régulièrement à chaque passage du véhicule sur l'itinéraire.Furthermore, said current images provided by said source are sent into said database, so that it is updated regularly every time the vehicle is on the route.

La figure du dessin annexé fera bien comprendre comment l'invention peut être réalisée.The figure of the appended drawing will make it clear how the invention can be realized.

La figure 1 représente schématiquement le système de détection d'engins explosifs improvisés appliquant le procédé selon l'invention.The figure 1 schematically represents the improvised explosive device detection system applying the method according to the invention.

Le système de détection 1 selon l'invention est monté à bord d'un véhicule 2 généralement du type militaire, symbolisé par un trait mixte rectangulaire et circulant sur un itinéraire (route ou piste) susceptible de comporter des engins explosifs improvisés qui peuvent être totalement ou partiellement enfouis et/ou dissimulés dans la route ou ses bas-côtés.The detection system 1 according to the invention is mounted on board a vehicle 2 generally of the military type, symbolized by a mixed rectangular line and traveling on a route (road or track) likely to include improvised explosive devices that can be totally or partially buried and / or concealed in the road or its aisles.

Pour cela, le système comprend, comme on le verra plus en détail ci-après, une source de prise de vues 3 d'images actuelles I, une base de données 4 d'images antérieures de référence IA dudit itinéraire, un dispositif de traitement d'images 5, un moyen de recalage d'images 6 et un dispositif d'affichage 7.For this, the system comprises, as will be seen in more detail below, a current image shooting source 3, a database 4 of reference images IA of said route, a processing device 5, an image registration means 6 and a display device 7.

En particulier, la source de prise de vues 3 est une caméra 3A embarquée dans le véhicule 2 et permettant de prendre les images actuelles I de l'itinéraire et de visualiser ainsi sur le dispositif d'affichage 7, la route et les bas-côtés de l'itinéraire sur lequel circule le véhicule, de sorte à délivrer en temps réel et en continu, les images actuelles de l'itinéraire.In particular, the shooting source 3 is a camera 3A embedded in the vehicle 2 and making it possible to take the current images I of the route and thus to display on the display device 7, the road and the aisles of the route on which the vehicle travels, so as to deliver in real time and continuously, the current images of the route.

La base de données 4 contient des images antérieures IA du même itinéraire mais filmées précédemment et contenues sur tout support informatique approprié de stockage, intégré au véhicule. Ces images peuvent avoir été prises par la même caméra lors d'un passage précédent sur le même itinéraire.The database 4 contains previous IA images of the same route but previously filmed and contained on any appropriate computer storage medium, integrated into the vehicle. These images may have been taken by the same camera during a previous pass on the same route.

Le dispositif de traitement d'images 5 est relié à la caméra 3A et à la base de données 4 et reçoit ainsi les informations (images I et IA) envoyées par la caméra 3A et la base de données 4. Le rôle du dispositif de traitement 5 est de remplacer des images actuelles I délivrées par la caméra par des images antérieures IA fournies par la base de données 4 et situées au même endroit. Pour optimiser la détection d'engins explosifs le long de l'itinéraire, une image actuelle I sur deux est remplacée par une image antérieure IA, de sorte que l'affichage alternatif des images actuelles I de la caméra et des images antérieures IA de la base de données permet de voir immédiatement les changements survenus tout le long de l'itinéraire actuel comparativement à celui précédemment enregistré. Pour visualiser cela, le dispositif d'affichage 7 est par exemple un moniteur de visualisation 7A.The image processing device 5 is connected to the camera 3A and to the database 4 and thus receives the information (images I and IA) sent by the camera 3A and the database 4. The role of the processing device 5 is to replace current images I delivered by the camera by previous images IA provided by the database 4 and located in the same place. To optimize the detection of explosive devices along the route, a current image I out of two is replaced by an earlier image IA, so that the alternative display of the current images I of the camera and the previous images IA of the The database allows you to immediately see the changes that have occurred along the current route compared to the previous one. To visualize this, the display device 7 is for example a visualization monitor 7A.

Ainsi, comme le montre la figure 1, les deux images fournies, respectivement actuelle I et antérieure IA, à un instant donné en un même endroit, peuvent ne pas être rigoureusement identique en ce qui concerne leur positionnement spatial, géographique. Notamment du fait que les véhicules n'empruntent pas exactement le même chemin le long de l'itinéraire et/ou que les caméras ne sont pas placées au même endroit sur les véhicules. On voit ainsi que la route R de l'itinéraire IT avec ses deux arbres AR de l'image actuelle I délivrée par la caméra 3A est légèrement décalée vers la droite par rapport à la même route R de l'itinéraire IT avec les deux mêmes arbres AR de l'image antérieure IA délivrée par la base de données 4, laquelle route se trouve au centre de l'image antérieure IA.So, as shown in figure 1 , the two images provided, respectively current I and previous IA, at a given moment in the same place, may not be strictly identical as regards their spatial positioning, geographical. Notably because the vehicles do not travel exactly the same way along the route and / or the cameras are not placed in the same place on the vehicles. It is thus seen that the route R of the IT route with its two trees AR of the current image I delivered by the camera 3A is slightly shifted to the right with respect to the same route R of the IT route with the same two AR trees of the previous image IA issued by the database 4, which road is in the center of the previous image IA.

Le système 1 de l'invention prévoit cela et son moyen de recalage 6 peut consister en plusieurs interventions, par exemple : soit à recaler l'image actuelle I filmée par la caméra 3A avec celle IA de la base de données 4 à partir de la position du véhicule ou à partir de la position du véhicule et de l'orientation de la caméra, de manière à superposer lesdites images I et IA, soit à utiliser un équipement propre au système de détection 1 ou à utiliser un équipement spécifique du véhicule tel qu'un système de positionnement par satellites, centrale inertielle, s'il en possède un, lequel équipement permet de savoir où ont été prises les images I et IA, en facilitant ainsi leur recalage.The system 1 of the invention provides this and its resetting means 6 can consist of several interventions, for example: either to reset the current image I filmed by the camera 3A with that IA of the database 4 from the position of the vehicle or from the position of the vehicle and the orientation of the camera, so as to superimpose said images I and IA, either to use equipment specific to the detection system 1 or to use specific equipment of the vehicle such as that a satellite positioning system, inertial center, if it has one, which equipment allows to know where were taken the images I and IA, thus facilitating their registration.

Un tel recalage des images actuelles I et antérieures IA est nécessaire aussi pour assurer un bon confort visuel de l'opérateur en charge de surveiller les images défilant alternativement sur le moniteur de visualisation 7A.Such a registration of the current images I and earlier IA is also necessary to ensure a good visual comfort of the operator in charge of monitoring the images scrolling alternately on the display monitor 7A.

Le système 1 ainsi décrit peut repérer, par corrélation d'images, tout changement survenant entre des images successives défilant sur le moniteur 7A. Par exemple, on remarque que sur l'image actuelle I représentée et délivrée par la caméra 3A, ainsi que sur l'image recalée IR du moniteur 7A, se trouve une tache noire T, sur le bas-côté gauche de la route R, alors qu'elle n'apparaît pas sur l'image antérieure IA. Ainsi, grâce à l'affichage alternatif des images actuelles et antérieures de la base de données, l'opérateur voit un clignotement sur le moniteur 7A représentatif du changement entre les images successives et correspondant dans l'exemple à la tache T, laquelle peut être un engin explosif improvisé. Si c'est le cas, ce dernier ainsi détecté est alors neutralisé.The system 1 thus described can identify, by image correlation, any change occurring between successive images scrolling on the monitor 7A. For example, it can be seen that on the current image I represented and delivered by the camera 3A, as well as on the imaged IR image of monitor 7A, there is a black spot T, on the left-hand side of the road R, while it does not appear on the previous image IA. Thus, thanks to the alternative display of the current and previous images of the database, the operator sees a flashing on the monitor 7A representative of the change between the successive images and corresponding in the example to the spot T, which can be an improvised explosive device. If this is the case, the latter thus detected is then neutralized.

Pendant ou à l'issue de la mission du véhicule, les images prises par la caméra durant le déplacement servent à alimenter, par une liaison 8, la base de données 4 laquelle est ainsi mise à jour. Le remplacement des images anciennes par des images récentes permet de s'affranchir des changements naturels lents dans le paysage comme, par exemple, ceux de la végétation.During or at the end of the mission of the vehicle, the images taken by the camera during the displacement are used to feed, via a link 8, the database 4 which is thus updated. The replacement of old images with recent images makes it possible to overcome slow natural changes in the landscape, such as those of vegetation.

Pour faciliter l'interprétation par l'opérateur, le dispositif de traitement d'images 5 peut s'effectuer automatiquement pour détecter et mettre en évidence les éléments ayant changés dans le paysage.To facilitate interpretation by the operator, the image processing device 5 can be performed automatically to detect and highlight the elements that have changed in the landscape.

Par ailleurs, le dispositif de traitement d'images 5 peut également agir sur les images de manière à réduire l'effet des variations de luminosité ou des ombres portées entre les deux images qui ont été prises à des instants différents.Furthermore, the image processing device 5 can also act on the images so as to reduce the effect of brightness variations or shadows worn between the two images that have been taken at different times.

La base de données d'images référencées 4 peut bien entendu être partagée entre plusieurs véhicules pouvant utiliser par exemple une même portion d'itinéraire.The image database referenced 4 can of course be shared between several vehicles that can use for example the same portion of the route.

Claims (9)

Procédé de détection d'engins explosifs improvisés ou analogues dissimulés partiellement ou totalement sur un itinéraire ou à proximité de celui-ci, à partir d'un véhicule circulant sur ledit itinéraire, du type par corrélation d'images et consistant, lors d'une mission de détection : - à délivrer en continu des images actuelles (I) représentatives dudit itinéraire suivi ; - à remplacer au moins certaines desdites images actuelles (I) par des images antérieures (IA) précédemment enregistrées, prises au même endroit ; et - à afficher successivement lesdites images actuelles et lesdites images antérieures de remplacement pour permettre, par la corrélation des images actuelles et antérieures, la détection desdits engins explosifs, caractérisé en ce qu'il consiste, de plus, à mettre à jour lesdites images antérieures (IA) précédemment enregistrées par lesdites images actuelles (I) délivrées en continu et représentatives dudit itinéraire suivi.A method of detecting improvised explosive devices or the like partially or totally concealed on or in the vicinity of a route from a vehicle traveling on said route, of the image correlation type and consisting, at a time of detection mission: - continuously delivering current images (I) representative of said route followed; replacing at least some of said current images (I) with previously recorded prior images (IA) taken in the same place; and to successively display said current images and said replacement prior images to allow, by the correlation of the current and previous images, the detection of said explosive devices, characterized in that it furthermore consists in updating said previous images (IA) previously recorded by said current images (I) delivered continuously and representative of said followed route. Procédé selon la revendication 1,
caractérisé en ce qu'il consiste à remplacer une image actuelle (I) sur deux par une image antérieure (IA) et à afficher successivement lesdites images actuelles et antérieures ainsi alternées.Process according to claim 1,
characterized in that it consists in replacing one current image (I) out of two by an earlier image (IA) and successively displaying said current and anterior images thus alternated. Procédé selon l'une des revendications 1 et 2,
caractérisé en ce qu'il consiste, avant d'afficher lesdites images actuelles et antérieures, à recaler la position géographique desdites images actuelles (I) et antérieures (IA) successives.Method according to one of claims 1 and 2,
characterized in that it consists, before displaying said current and previous images, to readjust the geographical position of said current (I) and previous (IA) successive images. Système de détection d'engins explosifs improvisés ou analogues, destiné à mettre en oeuvre le procédé tel que défini selon les revendications 1 à 3, et comprenant : - au moins une source de prise de vues (3) montée sur ledit véhicule (2) circulant sur ledit itinéraire et délivrant lesdites images actuelles (I) de celui-ci ; - une base de données (4) desdites images antérieures (IA) précédemment enregistrées dudit itinéraire ; - un dispositif de traitement d'images (5) en temps réel, auquel sont reliées ladite source de prise de vues et ladite base de données et qui remplace au moins certaines desdites images actuelles par des images antérieures de ladite base de données, prises au même endroit dudit itinéraire ; et - un dispositif d'affichage (7) desdites images actuelles et antérieures ainsi combinées par ledit dispositif de traitement (5), caractérisé en ce que lesdites images actuelles (I) prises par ladite source (3) sont envoyées dans ladite base de données (4) par une liaison (8) de manière à mettre à jour lesdites images antérieures (IA) de ladite base de données (4) par les images actuelles (I) prises par ladite source (3).Improvised explosive device detection system or the like, intended to implement the method as defined in claims 1 to 3, and comprising: at least one shooting source (3) mounted on said vehicle (2) traveling on said route and delivering said current images (I) thereof; a database (4) of said previously recorded prior images (IA) of said route; a real-time image processing device (5), to which said source of photography and said database are connected and which replaces at least some of said current images with previous images of said database, taken at the same place of the itinerary; and a display device (7) of said current and previous images thus combined by said processing device (5), characterized in that said current images (I) taken by said source (3) are sent in said database (4) by a link (8) so as to update said previous images (IA) of said database (4) by the current images (I) taken by said source (3). Système selon la revendication précédente,
caractérisé en ce que ledit dispositif de traitement (5) remplace une image actuelle sur deux prise par ladite source par une image antérieure de ladite base de données, de sorte à visualiser sur ledit dispositif d'affichage (7) une succession alternée desdites images actuelles (I) et antérieures (IA).System according to the preceding claim,
characterized in that said processing device (5) replaces a current two-image taken by said source by an earlier image of said database, so as to display on said display device (7) an alternating succession of said current images (I) and earlier (IA). Système selon l'une des revendications 4 et 5,
caractérisé en ce qu'il inclut, de plus, un moyen pour recaler la position géographique de deux images respectivement actuelle et antérieure successives.System according to one of claims 4 and 5,
characterized in that it further includes means for resetting the geographical position of two successive current and previous images respectively. Système selon l'une quelconque des revendications 4 à 6,
caractérisé en ce que ladite source de prise de vues (3) est une caméra (3A) et ledit dispositif d'affichage (7) est un moniteur (7A).System according to one of Claims 4 to 6,
characterized in that said shooting source (3) is a camera (3A) and said display device (7) is a monitor (7A). Système selon l'une quelconque des revendications 4 à 7,
caractérisé en ce que ladite base de données (4) d'images antérieures contient des images fournies par la même source de prise de vues (3).System according to one of Claims 4 to 7,
characterized in that said database (4) of prior images contains images provided by the same source of shooting (3). Système selon l'une quelconque des revendications 4 à 8,
caractérisé en ce que ladite base de données (4) est partagée entre plusieurs véhicules.System according to one of Claims 4 to 8,
characterized in that said database (4) is shared between several vehicles.
EP09177072.7A 2008-12-02 2009-11-25 Method and system for detecting improvised explosive devices or similar Active EP2194353B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
FR0806758A FR2939188B1 (en) 2008-12-02 2008-12-02 METHOD AND SYSTEM FOR DETECTING IMPROVED OR SIMILAR EXPLOSIVE DEVICES

Publications (2)

Publication Number Publication Date
EP2194353A1 true EP2194353A1 (en) 2010-06-09
EP2194353B1 EP2194353B1 (en) 2013-10-23

Family

ID=40888383

Family Applications (1)

Application Number Title Priority Date Filing Date
EP09177072.7A Active EP2194353B1 (en) 2008-12-02 2009-11-25 Method and system for detecting improvised explosive devices or similar

Country Status (7)

Country Link
US (1) US9696127B2 (en)
EP (1) EP2194353B1 (en)
CA (1) CA2744182C (en)
ES (1) ES2438602T3 (en)
FR (1) FR2939188B1 (en)
WO (1) WO2010063920A1 (en)
ZA (1) ZA201104009B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2103901A (en) 1981-07-30 1983-02-23 Secr Defence Television surveillance systems
GB2243266A (en) 1990-04-09 1991-10-23 Marconi Gec Ltd Image comparison
ZA9510950B (en) * 1994-12-27 1996-07-08 Gavin Hough Detection of movement
JPH10307110A (en) * 1997-05-06 1998-11-17 Omron Corp Image processing device
GB2439627A (en) 2006-05-17 2008-01-02 Boeing Co The detection of the insertion, removal, and change of objects within a scene through the use of captured images

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006074298A2 (en) * 2005-01-06 2006-07-13 Alan Shulman Navigation and inspection system
GB2432071A (en) * 2005-11-04 2007-05-09 Autoliv Dev Determining pixel values for an enhanced image dependent on earlier processed pixels but independent of pixels below the pixel in question

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2103901A (en) 1981-07-30 1983-02-23 Secr Defence Television surveillance systems
GB2243266A (en) 1990-04-09 1991-10-23 Marconi Gec Ltd Image comparison
ZA9510950B (en) * 1994-12-27 1996-07-08 Gavin Hough Detection of movement
JPH10307110A (en) * 1997-05-06 1998-11-17 Omron Corp Image processing device
GB2439627A (en) 2006-05-17 2008-01-02 Boeing Co The detection of the insertion, removal, and change of objects within a scene through the use of captured images

Also Published As

Publication number Publication date
ZA201104009B (en) 2012-02-29
FR2939188A1 (en) 2010-06-04
WO2010063920A1 (en) 2010-06-10
EP2194353B1 (en) 2013-10-23
US9696127B2 (en) 2017-07-04
ES2438602T3 (en) 2014-01-17
CA2744182A1 (en) 2010-06-10
FR2939188B1 (en) 2012-12-28
US20110228090A1 (en) 2011-09-22
CA2744182C (en) 2017-01-03

Similar Documents

Publication Publication Date Title
EP2506073A1 (en) Device for monitoring the condition of an engineering structure
EP2495531B1 (en) Method for measuring the stability of a line of sight and corresponding star sensor
EP0920677A1 (en) Method and device for air-ground recognition for optoelectronic equipment
EP2689403A1 (en) Monitoring system
FR3087985A1 (en) FIRE CHARACTERIZATION DEVICE AND ASSOCIATED METHOD FOR DETERMINING RADIATIVE FLOWS
EP2194353B1 (en) Method and system for detecting improvised explosive devices or similar
EP1168003B1 (en) Device for measuring space pollution
FR2985070A1 (en) System for detecting fall of e.g. person in retirement home, has programmed analysis unit to acquire images taken by sensors in synchronous manner, and to detect fall of person in scene from time sequence of pair of images of scene
EP3817868A1 (en) Method for sorting parcels on a conveyor with movable light indicators
EP3926529B1 (en) Systems and methods for determining a number of persons in a vehicle
EP0902301B1 (en) Optoelectronic observation apparatus improved to detect laser shots
EP2877979B1 (en) Monocamera method of determining a direction of a solid
EP1584079B1 (en) Device for detecting an incident or the like on a traffic lane portion
EP1791078A1 (en) Obstacle detection process implemented in an automobile vehicle
EP4115327B1 (en) Method for assisting in the detection of elements, and associated device and platform
EP4185532A1 (en) Method and system for detecting one or more targets in space based on images taken from a spacecraft such as a visual inspection satellite, and satellite equipped with such a system
FR2942062A1 (en) System for detecting or video monitoring presence and displacement of e.g. human, has scanning module oriented with respect to imager such that ray source and monitoring device are pointed in direction corresponding to part of image
EP0540395B1 (en) Passive artillary fire or analogue detector
WO2024018164A1 (en) Method and system for daytime infrared space surveillance
EP4022506A1 (en) Method for assisting in the detection of elements, and associated device and platform
EP4022495A1 (en) Method for assisting in the detection of elements, associated device and platform
EP3797510A1 (en) Panoramic observation system for platform
FR2785673A1 (en) Detecting displacement of point within given reference frame, especially for tracking potential displacement of points in soil by attaching illuminated object to point and observing from two video cameras
FR3040848A1 (en) METHOD AND SYSTEM FOR BISTATIC IMAGING
FR3004818A1 (en) LOCALIZED IMAGING AND LASER PATH PROFILOMETRY SYSTEM

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

AX Request for extension of the european patent

Extension state: AL BA RS

17P Request for examination filed

Effective date: 20100910

17Q First examination report despatched

Effective date: 20101029

RIN1 Information on inventor provided before grant (corrected)

Inventor name: HUBERT-HABART, CHRISTOPHE

Inventor name: MONNIN, DAVID

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: INSTITUT FRANCO-ALLEMAND DE RECHERCHES DE SAINT-LO

Owner name: MBDA FRANCE

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20130517

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

RIN1 Information on inventor provided before grant (corrected)

Inventor name: MONNIN, DAVID

Inventor name: HUBERT-HABART, CHRISTOPHE

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 637831

Country of ref document: AT

Kind code of ref document: T

Effective date: 20131115

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602009019586

Country of ref document: DE

Effective date: 20131219

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2438602

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20140117

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 637831

Country of ref document: AT

Kind code of ref document: T

Effective date: 20131023

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140123

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140223

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

BERE Be: lapsed

Owner name: MBDA FRANCE

Effective date: 20131130

Owner name: INSTITUT FRANCO-ALLEMAND DE RECHERCHES DE SAINT-L

Effective date: 20131130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140224

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602009019586

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131130

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131130

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131130

26N No opposition filed

Effective date: 20140724

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131125

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602009019586

Country of ref document: DE

Effective date: 20140724

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131125

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20091125

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131023

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131023

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140124

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20231023

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231120

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231218

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20231117

Year of fee payment: 15

Ref country code: IT

Payment date: 20231109

Year of fee payment: 15

Ref country code: FR

Payment date: 20231124

Year of fee payment: 15

Ref country code: DE

Payment date: 20231107

Year of fee payment: 15