EP4169005A1

EP4169005A1 - Method and system for geometric characterisation of fires

Info

Publication number: EP4169005A1
Application number: EP21733124.8A
Authority: EP
Inventors: Vito CIULLO; Lucile ROSSI TISON; Elie EL RIF; Antoine Marie Padoue PIERI
Original assignee: Universite De Corse Pasquale Paoli Ucpp; Centre National de la Recherche Scientifique CNRS
Current assignee: Universite De Corse Pasquale Paoli Ucpp; Centre National de la Recherche Scientifique CNRS
Priority date: 2020-06-19
Filing date: 2021-06-18
Publication date: 2023-04-26
Also published as: FR3111723B1; WO2021255214A1; FR3111723A1

Abstract

The embodiments of the invention provide a method for characterising fires occurring in a geographic area (110) covered by an imaging device (101), the imaging device being mobile, geolocated in a global reference frame (210) and providing multimodal stereoscopic images, the method being characterised in that it comprises the following steps: detecting fire pixels; generating a first three-dimensional (3D) point cloud; transforming the first cloud into a second cloud geolocated in the global reference frame (210); estimating, from the second 3D point cloud, geometric data of a local plane (230) on which the fire (111) occurs; transposing the second cloud into a local measurement reference frame (250) in order to provide a third 3D point cloud; estimating, from the third 3D point cloud, a first plurality of local geometric characteristics of the fire.

Description

DESCRIPTION

Method and system for geometric characterization of fires

Prior art

The present invention relates generally to fires and more particularly to a system and a method for geometric characterization of fires occurring in an open environment.

Many solutions have been considered to deal with fires which have experienced a significant upsurge in recent years with an increasingly significant scale, including considerable human and material damage.

The solutions considered are generally based on studies carried out aimed at understanding these phenomena and identifying the factors on which they depend. Existing solutions conventionally use mathematical models capable of modeling the evolution of fires over time and predicting future risks. From such models, it is possible to determine a fire fighting strategy as well as appropriate means of action.

The development of such mathematical models requires experimental measurements relating to the geometry of propagating fires and their adjacent environments in order to improve and / or validate the current propagation model. There is also a need to assimilate the geometry information of lights in the models over time in order to increase their efficiency.

Many imaging techniques have been proposed to determine the geometric characteristics of fire. Some of these techniques use one or more cameras operating in the visible range to capture images of the fire. From 2D information extracted from the acquired images, geometric characteristics can be obtained. Such a characterization uses a step of detecting the fire pixels in the images. The performance of this stage depends on fire characteristics, such as its color, texture, presence of fumes and the brightness of the environment. In order to achieve such characterization, it is known to use, in addition to cameras operating in the visible range, one or more cameras operating in the infrared range in order to more easily detect fire zones.

The implementation of such imaging techniques can be carried out statically by fixing the various cameras used around the fire to the ground, the cameras generally operating in a synchronized manner. The acquired images are processed in two dimensions (2D) to extract information which is then concatenated in order to determine the geometric characteristics of the fire. For example, the imaging technique proposed by Martinez-De Dios et al. [J. R. Martinez-de Dios, BC Arrue, A. Ollero, L. Merino, F. Gômez-Rodriguez, “Computer vision techniques for forest fire perception”, Image and Vision Computing, Volume 26, Issue 4, 2008, Pages 550- 562, ISSN 0262-8856] uses two cameras operating in the visible range and one camera operating in the infrared range, the visible range cameras being positioned frontally and laterally with respect to the direction of fire propagation. Efficient operation of such a solution requires the implementation on the ground of several geo-referenced beacons in order to calculate the relationship between the image plane of a camera and the terrain plane where the fire is propagating, and therefore to be able to project the pixels of the images on the ground plan. The use of fixed cameras on the ground according to such approaches nevertheless has many limitations mainly related to the rapid spread of fire. Such characteristics make it difficult to find an optimal position for a camera throughout the duration of the fire. During the propagation of the fire, a fixed camera on the ground may find itself too far from the fire, allowing the entire geographical area where the fire is spreading to be covered, to the detriment of degraded accuracy of the images provided. Conversely, a camera can be found close to the fire to provide images of better precision but by covering the geographical area where the fire is only spreading locally. In addition, securing cameras close to the fire is a risky task for the operator and for the equipment given the unpredictable behavior of the fire, the direction of fire propagation being able to change depending on the wind, for example.

It is also known to use a stereoscopic imaging system using two cameras operating in the visible range and two cameras operating in the visible range. infrared domain to simultaneously provide four images of the same scene. Such a solution makes it possible to obtain three-dimensional information from stereoscopic images and thus to overcome the need to deploy georeferenced beacons. The implementation of such a stereoscopic imaging system is carried out statically by positioning them on the ground. This requires anticipating the direction of fire propagation in order to optimally position the stereoscopic imaging system. Furthermore, as the theoretical accuracy of the depth measurement by stereovision of the images provided depends on the distance between the camera and the scene, the accuracy of the estimation of the images provided by such an imaging system deteriorates when the fire s 'away from the stereoscopic imaging system.

There is therefore a need for an improved system and method for effectively characterizing the geometric parameters of a fire. General definition of the invention

The invention improves the situation by proposing a method of characterizing fires occurring in a geographical area partially covered by the fields of view of a stereoscopic imaging device, the imaging device being mobile, geo-located in a global mark and providing multimodal stereoscopic images acquired by means of two imaging units, each of the multimodal images comprising an image of the visible domain and an image of the infrared domain acquired simultaneously, the method of characterization of lights being characterized in that it comprises the following steps, performed in response to receiving a pair of multimodal stereoscopic images:

Detect, in each of the two multimodal stereoscopic images, fire pixels representing a fire occurring in the geographical area;

Generate from the detected fire pixels a first three-dimensional (3D) point cloud representing the fire, the first 3D point cloud being represented in a local coordinate system linked to the position of the imaging device stereoscopic at the time of acquisition of the pair of multimodal stereoscopic images;

Transform the first 3D point cloud into a second geo-localized 3D point cloud in the global coordinate system;

Estimate from the second 3D point cloud geometric data of a local plane on which the fire occurs in the geographical area, the geometric data comprising a longitudinal angle and a lateral angle defined with respect to a reference plane relatively to two perpendicular directions;

Transpose the second 3D point cloud into a local measurement frame associated with the local plane to provide a third 3D point cloud;

Estimate from the third 3D point cloud a first plurality of local geometric characteristics of the fire, the first plurality of local geometric characteristics of the fire comprising a main direction of fire propagation.

In one embodiment, the local measurement coordinate system associated with the local plane can be provided with an axis system comprising: a first axis collinear with the direction defined by the lateral angle; a second axis collinear with the direction defined by the longitudinal angle; and a third axis perpendicular to the room plane.

In another embodiment, the global frame of reference can comprise an origin defined by an initial position of the stereoscopic imaging device, the global frame of reference can also be provided with a system of axes comprising: a first axis oriented parallel to the width of the geographic area; a second axis oriented parallel to the length of the geographic area; a third axis oriented so that the axis system forms a direct rectangle trihedron.

Advantageously, the method can further comprise a step of registration of multimodal images consisting in making the image of the visible domain transposable. and the image of the corresponding infrared domain so that a given pixel of the two images corresponds to the same point.

As a variant, the method can further comprise a step of estimating the main direction of propagation of a fire detected from: a first barycenter of a set of lowest 3D points and extracted from the multimodal stereoscopic images received at the current moment; and a second barycenter of a set of lowest 3D points extracted from multimodal stereoscopic images received at an earlier time; the main direction of fire propagation being directed towards the first barycenter.

In one embodiment, the method may further comprise a step of rotating the third 3D point cloud around the third axis of the local measurement frame associated with the local plane, which provides a transformed fourth 3D point cloud associated with a direction main fire propagation collinear with the second axis of the local measurement frame.

In another embodiment, the method may further comprise a step of estimating a second plurality of local geometric characteristics of the fire from the transformed fourth 3D point cloud.

Advantageously, the second plurality of local geometric characteristics of the fire can comprise at least one characteristic from among the length of the light, the width of the light, the height of the light, the inclination of the light, the surface of the fire, the front of the fire and the volume of fire.

According to certain embodiments, the step of detecting fire pixels in a multimodal image can comprise, for each of the pixels: the determination of a first vector of data associated with the pixel considered in the visible domain and of a second vector data associated with the pixel considered in the infrared domain; calculating a first probability result for the pixel to belong to a "fire" class from the first data vector and a second probability result for the pixel to belong to a "fire" class from the second data vector; the fusion of the obtained probability results; and the classification of the pixel considered according to whether it represents “fire” or “the environment”.

According to one embodiment, the step of generating a first 3D point cloud representing a fire can comprise the steps of:

Detect in each of the multimodal stereoscopic images one or more pixels of interest in the areas of fire pixels;

Match pixels of interest between multimodal stereoscopic images;

Calculate the three-dimensional coordinates of the paired pixels of interest in the local coordinate system related to the position of the stereoscopic imaging device using a triangulation technique.

According to another embodiment, the geometric data of the local plane can be estimated from the lowest 3D points.

Advantageously, a plurality of pairs of multimodal stereoscopic images can be provided by the stereoscopic imaging device and can be processed sequentially over time.

As a variant, the method may further comprise a step of estimating a plurality of global geometric characteristics of the fire from the plurality of pairs of multimodal stereoscopic images, the plurality of global geometric characteristics comprising the speed of propagation of the fire. .

There is also proposed a system for characterizing fires occurring in a geographical area partially covered by the fields of view of a stereoscopic imaging device, the imaging device being mobile, geo-located in a global frame of reference and providing multimodal stereoscopic images acquired by means of two imaging units, each of the multimodal images comprising an image of the visible domain and an image of the infrared domain acquired simultaneously, the light characterization system being characterized in that it further comprises a A characterization device comprising: a detection unit configured to detect, in each of the two multimodal stereoscopic images, fire pixels representing a fire occurring in the geographical area; a 3D point cloud generation unit configured to generate from the detected fire pixels a first three-dimensional (3D) point cloud representing the fire, the first 3D point cloud being represented in a local frame of reference linked to the position of the device stereoscopic imaging at the time of acquisition of the pair of multimodal stereoscopic images; a transformation unit configured to transform the first 3D point cloud into a second geo-located 3D point cloud in the global frame of reference; an estimation unit configured to estimate from the second 3D point cloud geometric data of a local plane on which the fire occurs in the geographical area, the geometric data comprising a longitudinal angle and a lateral angle defined with respect to a reference plane relative to two perpendicular directions; a transposition unit configured to transpose the second 3D point cloud into a measurement frame associated with the local plane to provide a third 3D point cloud; an estimation unit configured to estimate from the third 3D point cloud a first plurality of local geometric characteristics of the fire, the first plurality of local geometric characteristics of the fire comprising a main direction of fire propagation.

According to one embodiment, the stereoscopic imaging device can be airborne by means of an aircraft.

According to another embodiment, the stereoscopic imaging device can be mobile by means of an unmanned ground vehicle.

Brief description of the figures

Other characteristics and advantages of the invention will become apparent from the following description and from the figures in which:

FIG. 1 represents a system for characterizing fires, according to one embodiment of the invention; FIG. 2 represents a local propagation plan in which a method for characterizing a fire, according to embodiments of the invention can be implemented;

FIG. 3 is a flowchart showing the method of characterizing fires, according to one embodiment of the invention;

Figure 4 illustrates examples of geometric parameters of a fire that can be estimated by means of a fire characterization method according to embodiments of the invention;

FIG. 5 is a flowchart showing the steps taken to detect fire pixels in a multimodal image according to embodiments of the invention;

FIG. 6 is a flowchart showing steps taken to generate a first 3D point cloud according to embodiments of the invention; and

Figure 7 shows a processing device according to embodiments of the invention.

detailed description

Figure 1 shows a geometric characterization system of lights 100, according to embodiments of the invention. The fire characterization system 100 can be used, for example, in an outdoor environment where fires are likely to occur. Examples of outdoor environments include, without limitation, forests and agricultural fields.

The fire characterization system 100 may include a stereoscopic imaging device 101 movable relative to the ground. The stereoscopic imaging device 101 can be geo-located in a global frame of reference 210.

The fire characterization system 100 can be configured to characterize fires occurring in a geographic area of the external environment, the geographical area being partially covered by the fields of view of the stereoscopic imaging device 101.

The stereoscopic imaging device 101 advantageously comprises two imaging units 1011, 1012. Each of the two imaging units 1011, 1012 may include a first sensor 101a, 101c and a second sensor 101b, 101d. The first sensors 101a, 101c can be configured to capture and provide images in the visible domain, and the second sensors 101b, 101d can be configured to capture and provide images in the infrared domain. Each pair of two sensors of the same imaging unit can be configured to capture a first elementary image in the visible domain and a second elementary image in the infrared domain according to the same elementary image capture instant, the two elementary images. captured forming a multimodal image.

The two imaging units 1011, 1012 can also be synchronized by means of a synchronization signal so as to simultaneously capture two multimodal images according to the same multimodal image capture instant, the two captured multimodal images forming a pair of two multimodal stereoscopic images and comprising a total of four elementary images.

The sensors forming each imaging unit 1011, 1012 can have the same opto-geometric characteristics, such as the same field of view, and can be oriented so that the scenes captured by the two sensors relative to the same multimodal image are aligned. cover substantially completely. Alternatively, the sensors forming each imaging unit 1011, 1012 can have different opto-geometric characteristics (such as for example field of view, resolution, focal length, etc.), and their orientations can be adjusted so that the scenes captured relative to the same multimodal image partially overlap.

The imaging units 1011, 1012 can be arranged next to each other by securing them to a rigid support 1013, which can be for example a metal bar, a carbon fiber bar, or any other rigid support. adapted. In one embodiment, the two imaging units 1011, 1012 may be spaced apart from each other by a chosen spacing distance, for example equal to about one meter. The sensors providing images in the visible domain and equipping the two imaging units can be identical or different in terms of opto-geometric characteristics and orientation. Similarly, the sensors providing images in the infrared range and equipping the two imaging units may be identical or different in terms of opto-geometric characteristics and orientation. In general, the sensors fitted to each of the two imaging units 1011, 1012 can be oriented so that the two multimodal stereoscopic images at least partially overlap.

In embodiments of the invention, the imaging units 1011, 1012 of the stereoscopic imaging device 101 may be calibrated before being operated to acquire multimodal stereoscopic images. Such a calibration makes it possible, for example, to extract intrinsic and extrinsic parameters of the imaging device 101 which can then be used to obtain three-dimensional (3D) geometric information, such as 3D coordinates of a set of identified points, from of a pair of multimodal stereoscopic images. In one embodiment, the calibration can be performed using a grid comprising several side boxes, with alternating black and white boxes, the white boxes being coated with aluminum to provide useful information in the visible range and / or. in the infrared domain.

The stereoscopic imaging device 101 may further include a sync and save unit 1014 configured to provide the sync signal to the imaging units 1011, 1012. The sync and save unit 1014 may further be configured to receive and save the multimodal images captured by the two imaging units 1011, 1012. The images of the visible domain and the images of the infrared domain can be saved in the same image format which can be for example JPEG, BMP, PNG, Or other. The resolutions in number of pixels of such images may be the same or different.

In one embodiment of the invention, the stereoscopic imaging device

101 can be airborne by means of an aircraft 102. Advantageously, the aircraft

102 can be of the drone type. The aircraft 102 can be remotely controlled in real time or be programmed to follow a predefined trajectory. In another embodiment of the invention, the stereoscopic imaging device 101 can be mobile with respect to the ground while being transported by a land vehicle. Advantageously, the land vehicle can be of the unmanned ground vehicle (UGV) type, also known under the name of land robot. The ground robot can be configured not to enter the geographic area 110 on which the fire 111 occurs.

The stereoscopic imaging device 101 may further include several position sensors for locating the stereoscopic imaging device 101 for each multimodal image captured by such a device. Such position sensors can comprise a geolocation unit 103 of the GPS (Global Positioning System) or GNSS (Global Navigation Satellite System) type and an inertial unit 104, configured respectively to determine the position and the attitude (roll angles, pitch and heading) of the stereoscopic imaging device 101 in a global frame of reference 210. The position sensors may further include an orientation device, such as a compass, capable of providing information relating to the orientation of the device. stereoscopic imaging 101 in a global frame 210.

In embodiments of the invention, the aircraft 102 carrying the imaging device can be configured not to overfly the geographic area 110 over which the fire 111 is occurring. Thus, a multimodal image acquired by such an imaging device can correspond to a front, side or rear view of the light.

In embodiments of the invention, the stereoscopic imaging device 101 may comprise an electrical energy supply unit capable of continuously supplying sufficient electrical energy over time to the various elements forming the stereoscopic imaging device. 101.

In embodiments of the invention, the number of stereoscopic imaging devices used in the light characterization system 100 may be greater than or equal to two. Such embodiments make it possible to supply simultaneously several pairs of multimodal stereoscopic images obtained in different spectral bands. In addition, the use of several stereoscopic imaging devices makes it possible, in the event of malfunction of an imaging device, to continue to supply pairs of multimodal stereoscopic images. The fire characterization system 100 can further include a characterization device 120 configured to process the multimodal stereoscopic images provided by the processing and saving unit 1014 of the stereoscopic imaging device 101. The characterization device 120 can be airborne. by means of the same aircraft 102 carrying the stereoscopic imaging device 101. Alternatively, the characterization device 120 can be located on the ground by being fixed or mobile.

In embodiments of the invention, the sensors 101b, 101d providing images in the infrared domain can be configured to operate, in the wavelength band between 8 micrometers and 14 micrometers, such a length band. wave being also known by the English acronym LWIR (Long Wave Infrared).

Figure 2 shows a local propagation plan in which the fire characterization method according to embodiments of the invention can be implemented.

In one embodiment of the invention, the instantaneous position of the stereoscopic imaging device 101, as provided by the geolocation unit 103, can be defined in a global frame of reference 210. Such a global frame of reference 210 may have as its origin an initial position of the imaging device obtained, for example, before the takeoff of the aircraft 102, or the putting into circulation of a terrestrial robot, transporting the imaging device. The global reference 210 can be provided with an orthonormal base formed of three axes 211, 212, 213 forming a direct rectangle trihedron and comprising a first axis 213 parallel to the supposed direction of fire propagation, and two other axes 211, 212 completing the direct rectangle trihedron (and therefore orthogonal to the first axis). The directions of the length and width of the geographic area can be defined according to the assumed direction of fire spread by choosing the direction of the length collinear with that of the fire spread.

In embodiments of the invention, a local landmark 220 associated with the instantaneous position of the stereoscopic imaging device 101 can be defined. Such a local coordinate system 220 can admit as its origin the instantaneous position of one of the imaging units 1011, 1012 and can be provided with an orthonormal base formed of three of axis 221, 222, 223 forming a direct rectangle trihedron. A first axis 221 forming the direct trihedron may be oriented from the origin towards the other imaging unit, a second axis 223 may be oriented parallel to the optical axis of one of the imaging units 1011, 1012 and a third axis 222 completing the direct rectangle trihedron.

According to embodiments of the invention, the geographic area 110 on which the fire 111 occurs, associated with a multimodal image, can be roughly modeled by a local two-dimensional (2D) plane 230. A local plane 230 can be defined with respect to a reference plane 240 by means of two geometric data which can be two angles, comprising a lateral angle (Q) 262 and a longitudinal angle (cp) 261, defining the orientation of the local plane 230 relative to the reference plane 240 relative to two perpendicular directions. The two perpendicular directions can for example correspond to the first axis 211 and to the second axis 213 of the global reference frame 210. The reference plane 240 can for example correspond to the level of the sea or to the plane defined by the first axis 211 and the second axis 213. of the global benchmark 210.

In another embodiment of the invention, a local measurement benchmark 250 can be defined for each local plane 230 modeled. Such a local measurement benchmark 250 can be determined from the associated local plane 230 and from the global benchmark 210 as defined above. The local measurement coordinate system 250 can have the same origin as the global coordinate system 210. Its system of axes 251, 252, 253 can be determined so that the first axis 251 and the second axis 253 are parallel to the direction defined by l 'lateral angle (Q) 262 and the direction defined by the longitudinal angle (cp) 261, respectively. The third axis 252 of the local measurement frame 250 can be chosen perpendicular to the associated local plane 230. Thus, the system of axes of a local measurement frame 250 can form a rectangle trihedron, in the sense that its axes 251, 252, 253 are perpendicular, two by two.

FIG. 3 represents a method of characterizing fires which can be implemented in the characterization device 120. The steps of the method can be executed in a sequential manner after the reception of a pair of multimodal stereoscopic images supplied by a device of stereoscopic imaging 101.

In step 301 of the method, the two multimodal stereoscopic images are received in order to detect pixels which represent a fire in the images. elementary elements forming the pair of multimodal stereoscopic images received. The detection of fire pixels relative to a multimodal image can be carried out by processing one of the associated elementary images, that is to say by detecting fire pixels in the image of the visible domain or by detecting pixels. of fire in the infrared domain image. This makes it possible, for example, to reduce the complexity of the calculation.

Advantageously, the detection of fire pixels relative to a multimodal image can be carried out using multimodal information, that is to say information obtained from the visible domain and information obtained from the infrared domain. The use of multimodal information makes it possible to overcome the difficulties linked, for example, to the non-homogeneous color of the flames and the possibility of the presence of smoke.

The detection of fire pixels in an image of the visible domain can be carried out using a detection method implementing one or more color systems. Generally, in a given color system, each color can be synthesized from the basic elements of the considered color system, and intrinsic characteristics of the fire pixels can be defined. The color systems known to those skilled in the art include without limitation the RGB, YCbCr, TSI, YUV, L ^* a ^* b systems. In an RGB type color system, for example, each color can be synthesized from the three basic colors: red, green and blue. Intrinsic characteristics of the fire pixels usable in such a color system can include, in the absence of obstacles such as smoke, the dominance of the red component in the fire pixels. In general, each color system implements several rules that a given pixel must meet in order for it to represent a fire.

In one embodiment, the method of detecting fire pixels in an image of the visible domain can implement several color systems so as to use the rules of these color systems to detect fire pixels. Advantageously, the detection method can use learning techniques from previously acquired fire images to define new rules and / or to modify the current rules. Further, the fire pixel detection method can use pixel texture rules, that is, rules using the distribution of fire pixel colors in space. In one embodiment, the same method of detecting fire pixels can be used to process a series of pairs of multimodal stereoscopic images supplied by the same imaging device 101 relative to the same fire.

In another embodiment, the fire pixel detection method can change between a first series of pairs of multimodal stereoscopic images acquired by a first imaging system 101 and a second series of pairs of multimodal stereoscopic images acquired by a second imaging system 101. Such a change may for example relate to the color systems used.

On the other hand, the detection of fire pixels in an infrared domain image can make use of the fact that the fire 111 has a higher intensity than that of its adjacent environment. Thus, an intensity threshold can be defined in order to classify each pixel according to whether it represents fire 111 or whether it represents the environment adjacent to the fire. Such an intensity threshold used to classify the pixels can be determined by a thresholding method suitable for the detection of fire pixels in the infrared range.

In one embodiment of the invention, the detection of fire pixels can be performed using multimodal information from the visible domain image and the infrared domain image forming the multimodal image under consideration. Such an embodiment requires prior processing of the elementary images forming the multimodal image so as to make them superimposable, that is to say that each pixel of two images taken simultaneously in the visible domain and in the infrared domain relates to in the same place of the captured scene. The detection of fire pixels according to such embodiments can be performed sequentially starting with the infrared domain image using the approach described above, for example. The visible domain image can then be processed by considering only the pixels identified as representing a fire after processing the infrared domain image. The detection of fire pixels in the image of the visible domain among the previously identified pixels can be performed according to one of the approaches described above.

In step 302, a first three-dimensional (3D) point cloud representing a fire can be generated from the fire pixels detected in the two images. Multimodal stereoscopic images received so that each 3D point relates to a fire pixel detected in each of the multimodal stereoscopic images. A 3D point belonging to the first 3D point cloud can represent the three-dimensional coordinates of the location to which the associated fire pixel relates. Such a 3D point cloud can be defined in a local frame of reference 220 associated with the position of the stereoscopic imaging device 101 at the instant of acquisition of the processed multimodal stereoscopic images.

In step 303, the first 3D point cloud expressed in a local frame of reference 220 associated with the position of the stereoscopic imaging device 101 can be transformed to be expressed in a global frame of reference 210. Such a transformation provides a second point cloud. 3D and can be performed using the position and orientation of the stereoscopic imaging device 101 relative to the global coordinate system 210.

According to one embodiment of the invention, the transformation of the first 3D point cloud can be carried out by using a passage matrix M ₂₂₀ 2io from the local reference frame 220 to the global reference frame 210. Such a transition matrix can be defined from the following elements: the angles of roll a, of pitch b and of heading g provided by the inertial unit 104 and representing the attitude of the stereoscopic imaging device 101 in the global frame of reference 210 relative to the instant of acquisition of the images multimodal stereoscopic; angles of roll a ₀ , of pitch b ₀ and of heading g ₀ supplied by the inertial unit 104 and representing the attitude of the stereoscopic imaging device 101 in the global frame of reference 210 relative to an initial instant; rotation matrices R _a , and R _Y of the local coordinate system 220 around the respective axes 211, 212 and 213 of the global coordinate system 210; a translation matrix T defined from the coordinates N, E and U of the imaging device 101 in the local coordinate system 220 as supplied by the de

/ I 0 0 E \ geolocation 103, according to the following expression: T = u ^{; and}

\ 0 0 0 1 /

- an exchange matrix S ₂ - | _2®2 13 between axes 212 and 213 of global reference 210. The passage matrix M ₂ 2o 2io can be expressed according to the following relation:

M220 210 = Ry (-Uq) ^* Bb (bq) ^* Ba (Cto) ^* S212 213 * Ba (-0 ⁽ ) ^* Bb (b) ^* Bg (U) ^* G ^* S212 213 *

Ra (180).

In step 304, the geometry of the local 2D plane 230 on which the fire 111 occurs can be determined by estimating the lateral angle (Q) 262 and the longitudinal angle (cp) 261 quantifying the inclination of such 2D local plane 230 with respect to a reference plane 240 relative to two perpendicular directions, the geometry of the reference plane 240 in the global frame of reference 210 being previously known with precision. The estimation of the geometry of the 2D local plane 230 can be carried out by using the 3D point cloud expressed in the global coordinate system 210. It is advantageous for the 3D points to be obtained from a rear view of the light which makes it possible to have information based on the flame touching the earth while a frontal view may have part of the flame obscured by vegetation. For example, the 3D points, associated with a minimum altitude with respect to the adjacent 3D points and selected over a large area of the area on which the fire 111 occurs, can be chosen to estimate the geometry of the local 2D plane 230. Advantageously, the method of least squares can be used to determine the geometry of the local 2D 230 plane from the 3D points chosen so as not to lose information on the real inclinations of the 2D 230 local plane. Thus, the reconstruction of the geometry of the plane local 2D 230 can be carried out without requiring prior information on the digital model of the land on which the fire occurs. The development of the digital terrain model, generally complex to implement given the extent of the geographic areas concerned and the difficulty of accessing them, is thus simplified. In addition, the inaccuracy of the digital terrain model data is incompatible with the fire characterization method according to the invention.

According to another embodiment of the invention, the geometry of the 2D local plane 230 can be estimated from a current 2D local plane obtained according to the approach described above and from at least one other previous 2D local plane obtained from a pair of multimodal stereoscopic images associated with an earlier instant. In such an embodiment, the 3D points belonging to the current 2D local plane and the 3D points surrounding at least one other preceding 2D local plane are used to determine the geometry of the 2D local plane. In step 305, a local measurement frame 250 associated with the determined local 2D plane can be defined from the lateral angle (Q) 262 and the longitudinal angle (cp) 261 defining the orientations of the local 2D plane by relative to the reference plane 240. As described above, the local measurement frame 250 can have the same origin as the global frame 210 and its axis system 251, 252, 253 can be determined so that its first axis 251 and its second axis 253 are parallel to the direction defined by the lateral angle (Q) 262 and to the direction defined by the longitudinal angle (cp) 261, respectively, the third axis 252 of the local measurement frame 250 being able to be chosen perpendicular to the local 230 associated plane. Step 305 of the method may also consist in transposing the second 3D point cloud expressed in a global frame of reference 210 into a third 3D point cloud expressed in a local measurement frame of reference 250 associated with the local 2D plane.

According to one embodiment of the invention, the transposition of the second 3D point cloud can be carried out using a matrix of passage M ₂₂₀ 25o from the global frame of reference 220 to the local measurement frame 250 associated with the local 2D plane. Such a passage matrix can be defined from the following elements: a first homogeneous rotation matrix R ₀ defined from the lateral angle (Q) and from the axis of rotation 213 of the global reference frame 210; and a second homogeneous rotation matrix IT _f defined from the longitudinal angle (cp) and the axis of rotation 211 of the global reference frame 210.

The passage matrix M ₂₂₀ 25o can be expressed according to the following relation: M ₂₂₀

250 R (p * Re-

In step 306, a first plurality of local geometric characteristics of the fire can be determined from the third 3D point cloud, as expressed in the local measurement benchmark 250. The first plurality of geometric characteristics of the fire can comprise the main direction of fire spread.

In embodiments of the invention, the main direction of fire propagation can be determined from a first barycenter of a set of lowest 3D fire points, in terms of vertical elevation from a point of view. given reference level, and extracts from the third 3D point cloud resulting from a pair of current multimodal stereoscopic images, and from a second barycenter of a set of lowest 3D points of fire and extracted from a third 3D point cloud resulting from a pair of multimodal stereoscopic images associated with a previous instant. The main direction of fire propagation can be oriented towards the first determined barycenter.

The fire characterization method can further include a step of transforming the third 3D point cloud when the main direction of fire propagation is not collinear with the second axis 253 of the local measurement frame 250. Such a transformation can consist of in performing a rotation of the third 3D point cloud around the third axis 252 of the local measurement frame 250 so that the main direction of fire propagation is collinear with the second axis 253 of the local measurement frame 250. Such a transformation can provide a fourth transformed 3D point cloud.

In embodiments of the invention, a second plurality of local geometric characteristics of the fire can be determined from the transformed fourth 3D point cloud. The second plurality of local geometric characteristics of the light can include, by way of nonlimiting example, at least one characteristic among: the inclination of the light, the length of the light, the width of the light, the height of the light, the position of the light , the surface of the fire and the volume of the fire.

Figure 4 illustrates examples of geometric characteristics of fire that can be estimated from a transformed fourth 3D point cloud. For example, the length of the fire can be determined by measuring the distance from the top of the fire to the base of the fire front that delimits the burned area. The height of the fire can be determined by measuring the distance between the top of the fire and its orthogonal projection on the ground. In addition, the angle of inclination of the fire can be defined by the perpendicular to the ground passing through the base of the fire front and the line passing through the top of the flame and the base of the fire front. The surface of the fire can be calculated from the triangules obtained by connecting the outer 3D points of the fourth transformed 3D point cloud.

FIG. 5 represents the steps implemented to detect pixels of lights in a multimodal image, according to one embodiment of the invention. The multimodal image considered can comprise two superimposable images, that is to say that a pixel of two images relates to the same place of the captured scene. The steps in Figure 5 can be performed for each pixel in the image multimodal in order to classify it according to whether it represents a fire or that it represents the adjacent environment. The steps of FIG. 5 will be described by considering a single pixel of the multimodal image, by way of simplification. Those skilled in the art will easily understand that the steps of FIG. 5 apply to a plurality of pixels of the multimodal image comprising a plurality of pixels, by iterating these steps.

In step 3011, a data vector associated with the pixel considered can be defined for each of the elementary images forming the multimodal image. Such data may, for example, include the light intensity, the dominant color and / or the temperature of the pixel.

In step 3012, a result quantifying the probability that the considered pixel belongs to a "fire" class can be calculated for each of the data vectors. Such a result may correspond for the image of the visible domain to a "low probability" or a "high probability". For the infrared domain image, the returned result may for example correspond to a pixel intensity level, generally uncorrelated to temperature due to absorption by fumes.

In step 3013, the results from different images can be merged according to a given merging method to determine the class of the pixel considered. The pixel class can be either the class corresponding to the fire ("fire 111") or the class corresponding to the adjacent environment ("adjacent environment"). The results fusion method can be, for example, a statistical method, a method based on Dempster-Shafer belief theory or a method based on neural networks.

FIG. 6 represents the steps implemented to generate a first 3D point cloud from the pixels representing a fire in two multimodal stereoscopic images according to one embodiment of the invention.

In the first step 3021, the positions of the fire pixels in each of the two multimodal stereoscopic images, as provided at the output of step 301, are received. Step 3021 may consist in selecting from the two multimodal images received one or more light pixels of interest. The selection of pixels of interest in a multimodal image can be done by processing only the image of the associated visible domain. The selected light pixels of interest in a multimodal image may belong to different areas of the image. The selection of the pixels of interest can be done manually by an operator or can be done by means of one or more selection algorithms executed sequentially. For example, the selection of pixels of interest can be carried out by applying the Harris method which calculates the matrix of second order moments for each pixel of the image considered, from the matrix of second order moments, a function said force being then calculated. Harris' method defines the pixels of interest in an image as being those exhibiting a force function greater than that of neighboring pixels.

In addition, step 3021 may consist of calculating a description parameter, also called a "descriptor", for each pixel of interest identified in the multimodal stereoscopic images. Usually, a descriptor is used to describe the pixels surrounding the pixel of interest. Examples of methods known to those skilled in the art making it possible to calculate such a description parameter include, by way of nonlimiting example, a method of descriptors based on the filters or a method of histogram of the gradients of orientations and d. 'site.

In the second step 3022, a matching operation of the pixels of interest can be performed between the two multimodal stereoscopic images. Such a matching operation consists in seeking for each pixel of interest of the multimodal reference image the corresponding pixel of interest, that is to say the pixel which relates to the same location of the captured scene, in the other multimodal picture. The search for a corresponding pixel of interest can be done automatically using a similarity measurement method between descriptors. For example, the so-called "sum of normalized squared distances to zero mean" method can be used to find the pixel of interest corresponding to a pixel of interest identified in the multimodal reference image. Advantageously, the pairing of the pixels of interest can be carried out taking into account the intrinsic and extrinsic parameters of the stereoscopic imaging device 101.

In embodiments of the invention, only the paired pixels of interest are used to generate the first 3D point cloud. Thus, the unpaired pixels of interest at the end of step 3022 can be ignored. In the third step 3023, the coordinates of the paired pixels can be calculated in a local coordinate system associated with the instantaneous position of the stereoscopic imaging device 101. The calculation of the coordinates of the paired pixels can be carried out using a triangulation technique.

In embodiments of the invention, step 3023 can also consist in eliminating the paired 3D points which are separated from the volume where the other 3D points are concentrated. For example, if the distance 'd' separating a 3D point from its nearest neighbor satisfies the following relation | d - m _ά \ ³ 3a _d then the considered 3D point can be eliminated, m _ά and a _d respectively representing the mean and the standard deviation of the distance separating the point under consideration from these four nearest neighbors.

In one embodiment of the invention, several pairs of multimodal stereoscopic images successively acquired by one or more stereoscopic imaging devices 101 relative to different positions can be received and processed sequentially according to one of the characterization methods. fire described above. The fire characterization method can, in this case, comprise an additional step consisting in representing in the same measurement frame the fourth transformed 3D point clouds extracted from at least two pairs of multimodal stereoscopic images to provide a point cloud Global 3D.

A plurality of global geometric characteristics of the fire can be estimated from a global 3D point cloud. Examples of global geometric characteristics can include the speed of fire propagation, which can be measured from the position of the fire front in every fourth transformed 3D point cloud, and the time interval between the acquisition of the two. pairs of multimodal stereoscopic images used.

According to another embodiment of the invention, several multimodal stereoscopic images provided simultaneously by several stereoscopic imaging devices 101 can be received and processed, sequentially or in parallel, according to one of the fire characterization methods described. above. In particular, the fire characterization method can, in this case, include an additional step consisting in representing in the same measurement mark the fourth transformed 3D point clouds extracted from at least two pairs of multimodal stereoscopic images to provide a fifth 3D point cloud from which the second plurality of geometric characteristics of the fire can be extracted with better precision than that extracted from a transformed fourth 3D point cloud. In one embodiment, at least two pairs of multimodal stereoscopic images are acquired by means of a first airborne imaging device 101 and of a second imaging device 101 transported by a terrestrial robot.

FIG. 7 shows a fire characterization device 120 configured to implement the steps of a fire characterization method according to embodiments of the invention. The characterization device 120 can receive multimodal stereoscopic images provided by the processing and storage unit 1014. The transmission of the multimodal images can be done wirelessly by means of radio frequency signals, or wired. The characterization device 120 may include a detection unit 1201 configured to detect fire pixels in received multimodal images, and a 3D point cloud generation unit 1202 configured to generate a 3D point cloud from the fire pixels. detected. The characterization device 120 may further include a 3D point cloud transforming unit 1203 and a geometric data estimation unit 1204. The 3D point cloud transforming unit may be configured to apply a transformation to a first one. 3D point cloud received to provide, for example, a second 3D point cloud expressed in a global coordinate system. The characterization device may further include a transposition unit 1205 and a geometric characteristic estimation unit 1206.

According to embodiments of the invention, additional characteristics of the fire can be determined from the multimodal stereoscopic images provided by a stereoscopic imaging device 101. Such additional characteristics can be thermal characteristics such as the maximum temperature of the fire. and the radiative flux emitted by the fire.

The embodiments of the invention thus make it possible to characterize in a precise and efficient manner a fire which propagates over a large geographical area, without requiring prior information on the geometry of the area considered. They make it possible to determine the geometric data of the zone on which the fire propagates from multimodal stereoscopic images provided by a stereoscopic imaging device 101.

The embodiments of the invention also allow adaptation to the evolution of fire unlike state of the art solutions which use fixed ground imaging devices.

Those skilled in the art will understand that the method of characterizing fire according to the embodiments can be implemented in various ways by hardware, software, or a combination of hardware and software, in particular in the form of hardware. program code which may be distributed as a program product, in various forms. In particular, the program code may be distributed using computer readable media, which may include computer readable storage media and communication media. The methods described in the present description can in particular be implemented in the form of computer program instructions executable by one or more processors in a computer computing device. These computer program instructions may also be stored in computer readable media.

The invention is not limited to the embodiments described above by way of non-limiting example. It encompasses all the variant embodiments which may be envisaged by a person skilled in the art.

Claims

1. Method of characterizing fires occurring in a geographical area (110) partially covered by the fields of view of a stereoscopic imaging device (101), said imaging device being mobile, geo-located in a global frame of reference. (210) and providing multimodal stereoscopic images acquired by means of two imaging units (1011, 1012), each of said multimodal images comprising an image of the visible domain and an image of the infrared domain acquired simultaneously, said method of characterizing lights being characterized in that it comprises the following steps, executed in response to the reception of a pair of multimodal stereoscopic images:

Detecting, in each of the two multimodal stereoscopic images, fire pixels representing a fire occurring in the geographic area (110);

Generate from the detected fire pixels a first three-dimensional (3D) point cloud representing fire (111), said first 3D point cloud being represented in a local frame of reference (220) linked to the position of the stereoscopic imaging device ( 101) at the time of acquisition of said pair of multimodal stereoscopic images;

Transforming said first 3D point cloud into a second geo-located 3D point cloud in said global coordinate system (210);

Estimate from said second 3D point cloud geometric data of a local plane (230) on which fire (111) occurs in the geographical area (110), said geometric data comprising a longitudinal angle (261) and an angle lateral (262) defined with respect to a reference plane (240) relative to two perpendicular directions;

Transpose said second 3D point cloud into a local measurement frame (250) associated with said local plane to provide a third 3D point cloud;

Estimate from said third 3D point cloud a first plurality of local geometric characteristics of the fire.

2. Method according to claim 1, characterized in that said local measurement coordinate system (250) associated with said local plane is provided with a system of axes comprising: a first axis (251) collinear with the direction defined by said lateral angle

(262); a second axis (253) collinear with the direction defined by said longitudinal angle (261); and a third axis (252) perpendicular to said room plane (230).

3. Method according to claim 2, characterized in that said global frame (210) comprises an origin defined by an initial position of the stereoscopic imaging device (101), said global frame (210) being provided with a system of axes comprising: a first axis (211) oriented parallel to the width of said geographic area (110); a second axis (213) oriented parallel to the length of said geographic area (110); a third axis (212) oriented such that said axis system forms a direct rectangle trihedron; the width and the length of said geographical area (110) being defined according to an assumed direction of fire propagation.

4. Method according to one of the preceding claims, characterized in that it further comprises a step of registering multimodal images consisting in making transposable the image of the visible domain and the image of the corresponding infrared domain so that a given pixel of the two images corresponds to the same point.

5. Method according to one of the preceding claims, characterized in that said first plurality of local geometric characteristics of the fire comprises a main direction of fire propagation, the main direction of fire propagation being determined from: a first barycenter of a set of lowest 3D points, in terms of vertical elevation, in the third 3D point cloud extracted from the pair of multimodal stereoscopic images received at the current instant; and a second barycenter of a set of lowest 3D points, in terms of vertical elevation, in the third 3D point cloud extracted from a pair of multimodal stereoscopic images received at an earlier instant; the main direction of fire propagation being directed towards the first barycenter.

6. Method according to claim 5, characterized in that it further comprises a step of rotating said third 3D point cloud around said third axis (252) of the local measurement frame (250) associated with said local plane, which provides a fourth transformed 3D point cloud associated with a main direction of fire propagation collinear with the second axis (253) of said local measurement frame (250).

7. Method according to claim 6, characterized in that it further comprises a step of estimating a second plurality of local geometric characteristics of the fire from said fourth transformed 3D point cloud.

8. Method according to claim 7, characterized in that said second plurality of local geometric characteristics of the fire comprises at least one characteristic from the length of the fire, the width of the fire, the height of the fire, the inclination of the fire, the surface. fire, fire front and fire volume.

9. Method according to one of the preceding claims, characterized in that the step of detecting fire pixels in a multimodal image comprises, for each of the pixels: determining a first vector of data associated with the pixel considered in the image. visible domain and a second vector of data associated with the pixel considered in the infrared domain; calculating a first probability result for said pixel to belong to a "fire" class from said first data vector and a second probability result for said pixel to belong to a "fire" class from said second data vector; the fusion of the obtained probability results; and the classification of the pixel considered according to whether it represents “fire” or “the environment”.

10. Method according to one of the preceding claims, characterized in that the step of generating a first 3D point cloud representing a fire comprises the steps of:

Detect in each of said multimodal stereoscopic images one or more pixels of interest in the areas of fire pixels;

Matching pixels of interest between said multimodal stereoscopic images;

Calculate the three-dimensional coordinates of the pixels of interest paired in said local coordinate system (220) related to the position of the stereoscopic imaging device (101) using a triangulation technique.

11. Method according to one of the preceding claims, characterized in that said geometric data of said local plane 230 are estimated from the lowest 3D points associated with a minimum altitude with respect to the adjacent 3D points.

12. Method according to one of the preceding claims, characterized in that a plurality of pairs of multimodal stereoscopic images is provided by said stereoscopic imaging device (101) and is processed sequentially over time.

13. The method of claim 12, characterized in that it further comprises a step of estimating a plurality of global geometric characteristics of the fire from said plurality of pairs of multimodal stereoscopic images, said plurality of geometric characteristics. global including the speed of fire propagation.

14. System for characterizing fires occurring in a geographical area (110) partially covered by the fields of view of a stereoscopic imaging device (101), said imaging device being mobile, geo-located in a global frame of reference. (210) and providing multimodal stereoscopic images acquired by means of two imaging units (1011, 1012), each of said multimodal images comprising an image of the visible domain and an image of the infrared domain acquired simultaneously, said light characterization system ( 100) being characterized in that it further comprises a characterization device comprising: a detection unit (1201) configured to detect, in each of the two multimodal stereoscopic images, fire pixels representing a fire occurring in the geographic area (110); a 3D point cloud generation unit (1202) configured to generate from the detected fire pixels a first three-dimensional (3D) point cloud representing the fire (111), said first 3D point cloud being represented in a local frame of reference (220) linked to the position of the stereoscopic imaging device (101) at the time of acquisition of said pair of multimodal stereoscopic images; a transformation unit (1203) configured to transform said first 3D point cloud into a second 3D point cloud geo-located in said global frame of reference (210); an estimation unit (1204) configured to estimate from said second 3D point cloud geometric data of a local plane (230) on which the fire (111) occurs in the geographical area (110), said geometric data comprising a longitudinal angle (261) and a lateral angle (262) defined with respect to a reference plane (240) relative to two perpendicular directions; a transposition unit (1205) configured to transpose said second 3D point cloud into a measurement frame associated with said local plane (250) to provide a third 3D point cloud; an estimation unit (1206) configured to estimate from said third 3D point cloud a first plurality of local geometric characteristics of the fire, said first plurality of local geometric characteristics of the fire comprising a main direction of fire propagation.

15. A fire characterization system according to claim 14, characterized in that said stereoscopic imaging device (101) is airborne by means of an aircraft.

16. A fire characterization system according to claim 14, characterized in that said stereoscopic imaging device (101) is mobile by means of an unmanned land vehicle.