FI74827B - FOERFARANDE FOER TREDIMENSIONELL OEVERVAKNING AV ETT FOEREMAOLS TEMPERATUR. - Google Patents

Description

74827

METHOD FOR THREE-DIMENSIONAL MONITORING OF OBJECT TEMPERATURE

The invention relates to a method for three-dimensional monitoring of the temperature of an object and / or a target space, which object 5 is detected by at least two infrared imaging devices, such as a thermal imager or a similar arrangement, in the field of view of which the target is included and at a fixed angle target areas sufficiently different from each other, the imaging devices receiving the images of the space are digitized, are located in the image planes of the imaging devices, and the image coordinates and predetermined constants of these target areas are used to calculate the coordinates of the target area in the three-dimensional target space.

Monitoring temperature changes in a particular object or space requires both continuous three-dimensional monitoring of the temperature distribution and the ability to influence the temperature distribution and its changes by external measures. An example is a firefighting mission. Early localization of critical temperature differences can substantially reduce the magnitude and extent of potential damage compared to current similar systems.

Your fire alarm system is a status monitoring system in which a message received with point heat or smoke detectors is first transmitted to a central control station and then to a local 30 control room or area alarm center. In some system options, instead of point detectors, flame detectors can be used, a kind of externally monitoring mesh that monitors either the ultravio- ···. as indicators of bolt or infrared radiation.

35 2 74827 The disadvantage of heat and smoke detectors is that they monitor exactly where they are. Especially in high spaces, a slowly developing fire may continue long before the effect of the fire reaches per detector. In addition, in order to save on wiring costs, the detectors are connected in groups covering a maximum space of up to 1,600 m 2, from which the incoming alarm cannot be located more precisely. Also, the excited radiation source of the flame detector cannot be located other than that the radiation source is in some field of view of the detector.

Various methods for three-dimensional monitoring of the target space by means of imaging devices are already known, e.g. British application GB 2 099 255 A and German application DE-A 2 402 204.

In the British publication, the position of a moving object in the target space is determined by means of at least two cameras. The location of the object is identified from the image plane of each camera, and 20 the image coordinates of the object are used to calculate the position of the object in the target mode. The calculation method is based on the lens imaging equation. The focal lengths of the optics of both cameras are known. Prior to the description, a target coordinate system with respect to which the angles 25 of the main axis of the camera optics and the distances of the cameras from the origin of the target coordinate system have been determined.

In the German publication, the position of an object in the target coordinate system is determined by means of three cameras placed in the same plane at a 90 * angle to each other. The distances of the cameras from the origin of the target coordinate system are known. *. The position of the object in the image plane of each camera is indicated and the deviation of the object, i.e. the angle with respect to the main axis (axes of the target coordinate system) of each camera, is determined. The coordinates of .. 35 are calculated from the specified geometric equations **; in which the above angles and Constants are placed.

3,74827

A disadvantage of the methods and devices disclosed in the above-mentioned publications is e.g. their rigidity; they are placed to monitor a specific space and cannot be moved thereafter. In particular, when several cameras 5 are used for the actual three-dimensional measurement, the cameras are placed at certain angles (45 *, 90 °) with respect to each other. This avoids cumbersome calculation in coordinate changes. In particular, when the cameras are in a position deviating from a right angle to each other, the possibility of error in locating the object is greatly increased; the angles of the main axis of the cameras with respect to the axes of the target coordinate system should be determined very precisely as well as the distances of the cameras from the origin of the target coordinate system. The accuracy of the installation and orientation of the cameras limits the accuracy of the whole system to 15 rather modest if this is not done very carefully. Additional errors are caused by deficiencies in camera optics. High-quality optics and precise installations mean high costs.

20 Neither of the above methods can take into account changes in the temperature of the body or other changes in the state of the body. The most significant shortcomings are that they do not allow detailed monitoring of temperature changes at the site, that temperature changes cannot be located in detail, and that there is no immediate feedback between temperature changes and control measures.

By means of the method according to the invention e.g. the above 30 problems can be solved in several cases. The invention is characterized by what is stated in the appended claims.

35 4 74827

The measurement method according to the invention is real-time and the feedback is obtained at the time of observation. The method can be used to detect the surface temperature distribution of an object or one or more objects in the target space in three dimensions and to reveal the location of the radiation source in the target coordinate system in three dimensions. The target or target space to be measured must be large and there is no need to limit the number of measuring points. The method does not require physical contact with the object, nor the activation or display of measuring points. The measuring area is the entire freely visible area. The measuring system implementing the method can be moved, easily re-tuned if necessary and is very highly automated. The measuring system is suitable for connection to any other target space for detailed target-15 operations.

The method according to the invention further has some indirectly useful properties. First of all, fire fighting! early detection of a fire can prevent damage, usually total, from both the fire itself and the extinguishing operations, and limit the damage to a narrow area. Secondly, by locating the fire area and continuously monitoring the spread, people leaving the space can be directed to the most advantageous exit routes and, if necessary, the instructions can be revised. This is especially true when the measurement system is related to comprehensive status control. Thirdly, the system according to the invention can also be used for non-fire protection tasks, such as the control of automated industrial processes, where the control can be controlled on the basis of three-dimensional information on the temperature distribution of the object and its location. In addition, the method can be applied to security of premises.

35 5 74827

In the following, the invention and its background will be explained in detail with the aid of the accompanying drawings, in which Figure 1 shows the geometric model on which the method according to the invention is based and Figure 2 illustrates the location of a target point on the basis of at least two planar views; Figure 3 is a block diagram of a measuring system applying the method according to the invention;

Figure 4 shows an image processing unit in block diagram form; Figure 5 illustrates the use of fulcrum points to determine direction factors.

The method according to the invention for three-dimensional monitoring of the temperature of an object and / or a target space is based on the use of projective, two-dimensional planar imaging. When a detail is detected and located with at least two images, its location can be determined in the target space as three-dimensional coordinates. Consider Figure 1. There, the target point P is mapped to the image plane as a point P '. The connection between the point-20 point P and the pixel P 'is determined by the project ivi through its description in the so-called as a back cut.

The general imaging equation in back section can be represented by: (x-xq) __ anCX-Xp) + β2χ (Υ-Υρ) + ajj ^ Z-Zp) (z-zq) a23 (X-Xo) + β23 (Υ-Υο ) + a33 ^ _z0 ^ (1) <30 (y-yp) ^ 812 (χ-χθ) + a22 ^ Y_Yo) + a32 (Z-Zg) (z-zo) a ^ 3 (X-Xq) + a23 (Y “Yo) + a33 (z ~ zo) 35 74827 where x, y, z = pixel image coordinates, x0> YO» z0 = camera projection center image coordinates, X, Y, Z = target point target coordinates, 5 Xo »Yo» Zo = the target coordinates of the camera's projection center it, all *** a33 = the elements of the orthogonal rotation matrix of the coordinate system transformation between the camera and the target coordinate systems, ie the direction coefficients.

10

Denoting z-zq = c, that is, the absolute value of the distance between the projection center of the camera and the image plane ("focal length") and placing the expression in equations (1), we get: 15 x = xQ + c, 8h (X-Xq) + a2l (V-Vo) + ^ Ι ^ Ζ-Ζρ) a13 (X-Xo) + θ23 (γ_γθ) + a33 (Z-Zq) (2) y = y + c al2 (X-xo) + a22 ^ Y ~ Y0) + β32 (Ζ -Ζ0) c ° a13 (X-Xq) + a23 (γ_γ0) + 033 (Z-Zg) 20

Direction factors an · --033 containing unknowns Cp and Cv, which are the directional angles between the target and camera coordinate systems. In this case, the solution of the unknowns in each image is to determine at least the following unknowns: # * y »« v (3) xo, yo »z0 = c ^ x0 * Y0» zo 30

There are a total of 9 unknowns. Since two observation equations (2) are obtained for each predetermined reference point, at least five reference points are known for solving unknowns in one image, with X, Y and 7. 35 It should also be noted that the reference points must be 74827 7 independent of each other so that they do not not at the same level to make the solution unambiguous.

The imaging rays are never completely straight, but they or-5 rot in the medium (atmosphere, lens, water, etc.). These errors can be taken into account by extending the mathematical model to the so-called additional parameters. If additional parameters can be treated as systematic sources of error, they can be resolved on an image-by-image basis. The most commonly used models for the additional parameters 10 correct lens drawing errors and image coordinate system errors.

The use of an extended model should always be considered on a case-by-case basis. Practice has shown that when the effect of the additional para-15 meter is at least 1/5 of the measurement accuracy of the image coordinate, the additional parameter should be used. The use of additional parameters also requires corresponding measurement accuracy from the target coordinates of the support points. Each additional parameter requires new reference points and observation equations 20 (2).

The projective description in reverse, Fig. 2, i.e. from image to object, is not unambiguous with respect to the target point P. At least two level-25 descriptions are used to locate the target point. Locating is performed from projective imaging by means of reconstructed imaging lines O ^ P ^ (i = 1, 2, 3 ...). eteenpäinleikkauksena.

Forward cutting uses the inverse forms of the imaging equations (1) 30. Since it must be possible to determine three coordinate values in each case by defining the waypoints, the waypoint must always be detected with at least two images i and j · 35 8 74827

The general mapping equation can be represented as: X-Xp __ an (x-x0) + a22 (y-yp) + β13 (ζ-ζρ) Z-Zo a31 (x-x0) + B32 (y-yo) + a33 (z -z0) 5 (4) «Y-Yp _ a21 (x ~ XQ) + Q22 (y-yQ ^ + a23 (z ~ z0) w Z-Z0 a3l (x“ xo) + a32 (y-yo) + a33 (z-zo)> where 10 x and y are the detected camera coordinates of the new point in Figures i and j, and X, Y, Z are the calculated coordinates of the new point.

The other quantities, i.e. the direction coefficients βη ··· ®33, have been solved ku-15 horizontally in connection with the back section.

By fitting the observations and solved unknowns to equations (4), we obtain: X-X0i = (Z-Z0i) * lii Y-Y0i = (Z-Z0i) - Ii2 (5) 'x ~ xoj = (z-zoj) · Ijl Y-Y0j = (Z-Z0J) - Ij2 25 Equation the right side of equations (4) is denoted by figures for the constants: In, Ii2 * Ijl and I j 2 * After T'am®, the target coordinates X, Y, Z can be solved from equations (5) step by step, for example as follows: a) X = X0i + (Z-Zoi) · In = Xoj + (Z-Zqj) * Ijl b) x0i + z · in - z0i »in = xoj + z · iji - z0j · iji c) z = X0j ~ Xoi ~ Zpj · Ijl + Zoi 'lii lii - 1jl 35 where Ζ is solved, and we continue for example 9 74827 d) X = (Z-Z0i) · Iϋ - X0i and e) Y = (Z-Z0i) * Ij2 - Yoi »where solved also X and Y.

5 If an extended model with additional parameters is used, before solving the target coordinates X, Y and Z for the image observations yj_; xj and yj correspond to corrections as in the case of backward cutting.

In the method according to the invention, imaging devices, such as thermal cameras, are placed at an angle to each other to monitor the desired object, in particular the target space; support points are provided in this target state, the target coordinates of which are X ^, Υ ^, Ζ ^; k = 1, 2, 3, ... are measured and the image-specific image coordinates x ^, Yki »xkj> Ykj are determined from the respective imaging devices i, j, after which the reference points can be removed from the target state; the image and target coordinate coordinates of the support points are used to calculate the directional coefficients of the projected back section 8 ^ ... 833, after which the unknown target coordinates X, Y, Z of the sufficiently changed temperature points of the ha-20 can be solved in real time using the image coordinates x, taking advantage of projective forward surgery.

23

It should be noted that in the method according to the invention, it is not necessary to determine the imaging devices or their location before performing the imaging, nor the directional angles between the target and the camera coordinate system, nor the focal length of the cameras. In addition, the used support points 30 are generally removed from the target space as soon as their location has been determined and / or the direction factors have been calculated, so that they do not interfere in any way with the monitoring of the target space. Once the direction factors of the imaging devices have been determined, any object of thermal radiation that is sufficiently altered or otherwise sufficiently distinguished from the background in the common field of view of the imaging devices, i.e., the target space, can be located.

10 74827

In the method according to the invention, the direction coefficients βΐΐ ··· β33, once determined, are continuously used to solve the unknown target coordinates X, Y, Z of the target points by means of the image coordinates x, y observed from the imaging device 5 as long as the imaging devices are at a fixed angle to each other. Doing so greatly speeds up the determination of the coordinates of the waypoints; calculating the direction factors takes the most time in the configuration procedure.

10

A preferred embodiment of the method according to the invention is one in which the imaging devices are connected at a fixed distance to each other and at a fixed angle to each other, whereby their common field of view, i.e. the target space, is determined and this space can be monitored continuously. This means that the imaging devices together, i.e. the objects il ansa, form a closed system. It is not dependent on external factors. Thus, the measuring system can be built on a movable platform (car, 20 train carriages, ship, etc.) and can monitor its environment outside this movable platform within the common field of view of the imaging devices. The direction factors can be determined in advance under the desired conditions, after which the measurement system can be used in the field.

25

The general imaging equations (1) in the back section and (4) in the forward section can be generalized as a transformation matrix: 30 X X0 an a12 a13 x

(6) Y = Y0 + a2i a22 a23 · Y

2 ij ^ oij a31 a32 a33 ij c ij 35 74827 where

Xi, Yi, Zi s target coordinates, ie coordinates of target points

Xj, Yj, Zj dyninates in the XYZ coordinate system of the target state; X0i, ^ oi »Z0i = Constants representing the projection point 0 ^ 0j of each imaging device X0j, Yoj> Z0j; all *** a33 = projection direction factors of the images; c ^, cj = imaging device constants; Χχ, yi = pixel coordinates for each shooting

Xj, yj in the image plane of device i, j; 10 i, j = imaging equipment! and j.

The transformation matrix can be used to solve all the necessary quantities as shown above in connection with Equations (1) and (4).

13

Figure 3 shows a measuring system in which the method according to the invention is applied. The observable target state l is located in the three-dimensional XYZ coordinate system. The object consists of observable points or surface elements P (X, Y, 20 Z). The measurement is made by imaging the target space with spaced imaging devices 2 and 3 such as a thermal imager. The imaging devices are connected to a data processing system 4. The data processing system 4 consists, for example, of an imaging part 3, an image processing part 6, a logic-23 part 7 and an operation part 8. The imaging part 3 monitors the imaging, possible display of measurement points and It includes, for example, the necessary timers and A / D converters. In the image processing section 6, the images are interpreted: the changes in the temperatures of the target surface elements and the image coordinates P '(x, y) are searched for from each image and the target coordinates of the changed surface elements P (X, Y, Z) are calculated, intermediate results are stored, filtered relevant information and may be followed up over time between 35 results. The final result is entered in the logic part1 le 7, 12 74827 where decisions are made about the measures. The operation part 8 takes care of the necessary measures for the target space 5 11 such as suppressing the fire extinguisher of the measures 12 for the shooting space, such as orienting the imaging device and other measures such as removing and clearing .

10

The imaging devices used in the above method shall be such that they can express the hypothetical, infrared or other similar radiation caused by and / or predicting temperature changes reflected or produced by the object by a two-dimensional image. The imaging devices can be thermal cameras (wavelength range eg 1 ... 13 un), or video cameras sensitive in the infrared range. A conventional camcorder can also be used in conjunction with a thermal imager or the like.

20

Figure 4 shows a solution as an image processing unit 6. A digitized image signal from each imaging device 2, 3 is input to the first image memory 15a, 15b via the input connection A, B of the unit. The first and second image memories 15a, 15b and 16a, 16b are connected to a sensor 17a, 17b for image areas that have changed in temperature, such as pixels or image areas formed therefrom, to which a temperature threshold is also set. The sensor 17a, 17b is connected to the circuit 18a, 18b for determining the image coordinates x, y of the change point. If necessary, the contents of the second image memory 16a, 16b can be renewed from the imaging device 2, 3 via the second image memory supplement circuit 19a, 19b.

The digitized image signal is read into the first image memory 35 15a, 15b and further into the image area identifier 17a, 17b, to which the image information or the corresponding information already transmitted from the second image memory 16a, 13 74827 16b is also read. When a certain image area is detected as changed in the sensor 17a, 17b, it is checked whether the changed image information, such as the intensity of the image area corresponding to a certain temperature, exceeds a set threshold value, and if it exceeds, the change point coordinates x, y are calculated. a, 18b.

When the change point 10 x, y is determined in the image plane of each imaging device 2, 3, these image coordinates are input to the computing unit 20 or the corresponding data processing unit. Equation (4) is solved with respect to the target coordinates, after which the calculated coordinates X, Y, Z of the target point are fed via the output interface C to the logic section 8.

15

Before the actual measurement, the direction factors must be calculated, as stated above. This is done by determining the support points, which operation is shown in block 3 in block 21. The direction factors are calculated in the calculation unit 20 20 e.g. by means of equation (1) or (2) or matrix (6).

The measurement of the fulcrums is illustrated in Figure 5. The imaging devices and their imaging directions (or major axes) are shown by arrows 22, 23. The target coordinate system XYZ has 25 monitored objects 24, which and their surroundings are included in the field of view of each imaging device. In this case, the support points 25 are clearly marked from the background on different sides of the target space, most preferably so that the object to be monitored is included in the space bounded by the support points. Nine points of support 25 are marked in Fig. 5. The image coordinates x ^, y ^ of the points are measured from the images using the system's own measurement system. Target coordinates of support points Χ ^, Υ ^, Ζ ^; k = 1, 2, ... 9 are measured, for example, geodetically by means of an electronic tachometer and the values of the coordinate-35 path are entered, for example, by means of a keyboard into the calculation unit 20. Then the direction factors 8 ^ are calculated by means of a calculation unit using the transformation matrix (6). ... 833. The target coordinates of the support points can also be entered directly into the computing unit from a total station-5 sufficient to be connected to the equipment.

As reference points, measuring marks can be used, as shown in Fig. 5, special light sources or points that erase the object and / or the target space. In place of the thermal cameras 10, it is then possible to use, for example, conventional video cameras operating at wavelengths of visible light. If, on the other hand, thermal cameras are used, the measuring marks must differ from their surroundings at least in terms of their temperature, but also preferably be clearly perceptible, so that locating with other aids is easy.

Once the direction factors have been calculated with the aid of the support points, the target state is defined and the support points, i.e. their markings or traces thereof, can be deleted. Thereafter, they do not in any way limit the measurement of the object in the target space. It is clear that the measurement of the support points described above can be performed in an empty target space, i.e. without any actual object to be measured. Once the hardware has "learned" these support points, it will be able to determine all other points 25 in the target mode that both cameras will see.

In Figures 3 and 4 above, the structure of the measuring system applying the method according to the invention may differ considerably in different applications and environments. The structure 30 is also affected by the other data processing applied by the user, the degree of automation required of the equipment and the nature of the measures 11, 12 and 14.

35 15 74827

The imaging devices indicate the temperature distribution of the object, each seen from its own perspective. Combining the perspective descriptions of the surface elements solves the location of these elements in the target space as stated above.

5 In order to solve the temperature distribution, it is essential that the temperature is observed at least per surface element from one description, and the location of the element from at least two descriptions with different perspectives. In the latter case, the second imaging device can be replaced by the target state interface guarantee if the interface remains unchanged. Correspondingly, there may be more than two imaging devices detecting the target space, if this is necessary to control the entire target space. If the temperature changes in the target space are slow, the minimum requirements for perspective shots can also be met by moving one imaging device to more perspective shots.

20

Claims (6)

16 74827 A method for three-dimensionally monitoring the temperature of an object and / or a target space, the object being detected by at least two infrared imaging devices, such as a thermal imager, or a similar arrangement, the imaging devices including a target and the imaging devices spaced apart and at a fixed angle to each other the target areas sufficiently deviating in temperature from the imaging devices 10 are digitized in the image planes of the imaging devices and the coordinates of the target regions and predetermined constants are used to calculate the coordinates of the target area in the three-dimensional target space, characterized by arranging the target and / or the target coordinates (X | <, Y | <, Z (<; k = .1, 2, 3 ...) are measured and the image-specific image coordinates (x | <i * Yki »xkj» Ykj) are determined from the corresponding imaging devices, after which the reference points are poi-20 statavis sa target space; the image and target coordinate coordinates of the support points are used to calculate the directional coefficients of the projective back section (··· a33) · after which the unknown target-25 coordinates (X, Y, Z) of the target points with sufficiently changed temperature in the target and / or target space can be solved in real time using image coordinates (x, y) utilizing projective forward cutting. A method according to claim 1, characterized in that the once determined direction coefficients (a ^ *** a33) are continuously used to solve the unknown target coordinates (X, Y, Z) of the target points with the aid of the image coordinates (x, y) detected from the imaging device as long as the imaging devices are at a fixed angle to the others and monitor the desired target state. 74827 Method according to Claim 1 or 2, characterized in that the coordinates of the object are determined by means of a transformation matrix: Method according to claim 1, 2 or 3, characterized in that the support points are selected so as to cover the three-dimensional space controlled by the cameras. Method according to Claim 3, characterized in that the number of support points is more than 5, i.e. more than the minimum required number for calculating the direction factors. 5. X0 an a12 a13 x (6) Y = Y0 + a2i a22 a23 · y 1 ij zo ij a31 a32 a33 ij c ij where 10 Χΐ, Υ ^, Ζι = the coordinates of the target points, ie the coordinates of the target points Xj, Yj, Zj XYZ-coordinate system; X0i, Yoi> Zoi = Constants representing the projection points 0 ^, 0j of each imaging device Xoj * Y0j * zoj; all ,,, a33 = projective direction factors of images; 15 en cj = recording equipment constants; Xi, yi = the coordinates of the pixels in the image plane of each imaging device xj, yj i, j; i, j = imaging equipment i and j. Method according to one of the preceding claims, characterized in that visible measuring marks or the like are used as reference points, and imaging devices operating in the visible wavelength range, such as video cameras, are used as imaging devices for determining their imaging-specific coordinates. 18 74827