JP2021171257A - Fire extinguishing system - Google Patents

Abstract

To cause discharged water to fall at a desired position accurately.SOLUTION: A pixel extraction unit 20 determines whether or not a water discharge part is represented for each pixel from a plurality of thermal images having pixel values according to the temperature, which has captured fire extinguishing activities in a time series manner. A contour extraction unit 22 extracts a pixel group indicating a contour part for each of two contour parts in a water discharge region from a pixel group consisting of the pixels determined to represent the water discharge part. The contour extraction unit 22 obtains an approximate curve corresponding to the pixel group indicating the contour part for each of the two contour parts in the water discharge region, and outputs an image indicating the approximate curve obtained for each of the two contour parts in the water discharge region.SELECTED DRAWING: Figure 3

Description

The present invention relates to a fire extinguishing system.

Conventionally, a method of estimating the trajectory of bubble radiation based on the brightness of a visible image has been known (Patent Document 1).

Further, a mathematical model for estimating the deviation of the drop point of the fire extinguishing water due to the wind from the value of the wind direction and the wind speed and controlling the water cannon to drop the fire extinguishing water to the target position is known (Patent Document 2).

Further, a method for setting fire extinguishing and discharging conditions for a large space fire and an automatic fire extinguishing facility for a large space using the method are known (Patent Document 3).

Further, there is known a disaster prevention system in which a plurality of flame detectors are used to determine the position where a fire occurs and a water discharge nozzle is used to extinguish the fire (Patent Document 4).

Further, an algorithm for extracting only a fire area by omitting an artificial light source other than the monitoring target is known (Patent Document 5).

Japanese Unexamined Patent Publication No. 2014-036756 Japanese Unexamined Patent Publication No. 2000-042128 Japanese Unexamined Patent Publication No. 08-141103 Japanese Unexamined Patent Publication No. 2002-153572 Japanese Unexamined Patent Publication No. 10-188169

In the prior art, there was a method of estimating the landing point based on a mathematical model from the measured value of the wind direction and wind speed and controlling the angle of the water cannon, but the wind direction and wind speed differed greatly depending on the measurement location, and the accuracy was poor. Moreover, it was unrealistic to measure the wind direction and speed near the actual water discharge.

Further, in the prior art, a device combining an infrared camera and a water cannon is widely used in a fixed fire extinguishing system in a large-scale space (for example, a garbage pit, a sports stadium, etc.). In this equipment, the protection range (coordinate system to protect) is determined in advance, and in the event of a fire, the coordinates of the fire occurrence point are specified by an infrared camera, and the fire source is generated based on the predetermined water discharge trajectory formula. A method has been proposed in which water cannons that can optimally discharge water to a point, a method of determining the depression / elevation angle, and automatically discharging water and extinguishing a fire.

On the other hand, for automatic fire extinguishing robots, fire engines, large-capacity foam water cannons, and mobile / portable disaster prevention equipment such as 3-piece sets owned by the self-defense disaster prevention organization of a specific business establishment, water cannons can be used in response to a fire. Since the installation position and the target water discharge position change, the protection range has not been determined, and it was difficult to take the above method.

Therefore, in the above-mentioned materials and equipment, a method in which a human visually monitors the water discharge trajectory and the drop point in the vicinity of the water discharge and manually controls the water cannon according to the person's sense is the mainstream. In addition, especially in an environment where it is difficult to make a visual judgment such as backlight, cloudy weather, or at night, a method of making it easier to observe the water discharge trajectory and the water discharge center point with a thermal image camera has been used conventionally, but it is only visual. It was only used as an image (not data) as an auxiliary application, and was not used for controlling water cannons.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a fire extinguishing system capable of accurately dropping discharged water to a desired position.

In order to achieve the above object, the fire extinguishing system according to one aspect of the present invention includes a water cannon, a thermal image camera for photographing the water discharged by the water cannon, and a water discharge trajectory and water discharge from the photographed thermal image. It includes a water discharge locus estimation device that analyzes the center position or the landing point and controls the water discharge angle of the water cannon according to the result of the analysis.

According to the fire extinguishing system according to one aspect of the present invention, the locus of water discharge and the water discharge center position or water landing point are analyzed from the captured thermal image, and the water discharge of the water cannon is performed according to the result of the analysis. By controlling the angle, it is possible to obtain the effect that the discharged water can be accurately dropped to a desired position.

It is a block diagram which shows the structure of the fire extinguishing system which concerns on embodiment of this invention. It is a schematic block diagram of an example of a computer functioning as a firefighting robot control device according to the embodiment of the present invention. It is a block diagram which shows the structure of the fire-fighting robot control device which concerns on embodiment of this invention. It is a flowchart which shows the water discharge locus estimation processing routine in the firefighting robot control apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the 1st method of noise removal. It is a figure which shows an example of the binarization data obtained by the 1st noise removal. It is a figure for demonstrating the 2nd noise removal method. It is a figure which shows an example of the binarization data obtained by the 2nd noise removal. It is a figure for demonstrating the method of determining a scan origin pixel. It is a figure for demonstrating the scanning method of a pixel. It is a figure for demonstrating the scanning method of a pixel. It is an image diagram of the point cloud data showing the outline of the upper part of the water discharge area. It is a figure for demonstrating the method of calculating an approximate function. It is a figure for demonstrating the scanning method of a pixel. It is a figure which shows an example of the binarization data obtained by noise removal. It is a figure for demonstrating the method of determining a pixel extraction range by an approximation function. It is an image diagram of the point cloud data showing the outline of the lower part of the water discharge area. It is an image figure which drew the approximate curve of the point cloud data which shows the outline of the upper part of a water discharge area on a thermal image. It is an image figure which drew the approximate curve of the point cloud data which shows the outline of the lower part of a water discharge area on a thermal image. It is an image figure which drew an approximate curve representing two contours of a water discharge area on a thermal image. It is a figure for demonstrating the method of setting a horizontal line. It is an image diagram which drew the center point of water discharge on the horizon. It is a figure for demonstrating the method of setting the water discharge target position on the horizon. It is a figure which shows an example of the thermal image and the binarization data at the time of the back-view photography. It is a figure which shows an example of the binarization data obtained by the 1st noise removal. It is a figure for demonstrating the 2nd noise removal method. It is a figure which shows an example of the binarization data obtained by the 2nd noise removal. It is a figure which shows an example of the binarization data obtained by reversing the image coordinates of each pixel. It is an image figure which drew the approximate curve of the point cloud data which shows the outline of a water discharge area on the binarized data. It is an image figure which drew an approximate curve representing two contours of a water discharge area on a thermal image. It is an image of the discharge trajectory for each discharge turning angle. It is an image figure which drew an approximate curve representing two contours of a water discharge area on a thermal image. It is an image figure which drawn the approximate curve which shows the two contours of a water discharge area and the center point of a water discharge on a thermal image. It is an image figure which drawn the approximate curve which shows the two contours of a water discharge area and the center point of a water discharge on a thermal image. It is an image figure which drawn the approximate curve, the vertical line, and the center point of a water discharge which represent two contours of a water discharge area on a thermal image.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of Embodiment of the present invention>
In the embodiment of the present invention, in a fire-fighting robot system that automatically and autonomously extinguishes a large-scale fire such as an oil complex or a special fire, the trajectory of water or foam fire extinguishing agent emitted from a water cannon (hereinafter referred to as Explains a fire extinguishing system that captures the water discharge locus) and the water discharge center point with a thermal image, identifies and visualizes the locus and the water discharge center point or the water landing point by image processing, and controls the water cannon based on the identification result. do.

Specifically, a plurality of water discharge loci are photographed with a thermal image camera, the water discharge locus and the water discharge center point or the water landing point are identified by image processing, and the water discharge center point or the water landing point is visualized on the GUI. , The turning angle or depression / elevation angle of the water cannon is automatically corrected according to the deviation from the target water discharge center point or the target water landing point set by the operator in the GUI.

As described above, in a fixed fire extinguishing facility in a large-scale space, a protection range (coordinate system to protect) is predetermined, and in the event of a fire, the coordinates of the fire occurrence point are specified by an infrared camera and determined in advance. Based on the water discharge trajectory formula, the water cannon that can optimally discharge water to the fire source generation point, the turning and depression / elevation angles are determined, and water is discharged and the fire is extinguished automatically (Patent Document 4).

On the other hand, for fire engines, large-capacity foam water cannons, mobile portable disaster prevention equipment such as 3-piece sets owned by the self-defense disaster prevention organization of a specific business establishment, etc. The protection range is not fixed because the target position of the water discharge changes. Therefore, it is difficult to take the fire extinguishing method, and a method in which a human visually identifies the water discharge trajectory and the water discharge center point and manually controls the water cannon is still the mainstream.

Therefore, in the embodiment of the present invention, based on a plurality of thermal image data obtained by photographing the water discharge locus, it is unique to determine where on the thermal image (image coordinate system) the water discharge locus, the water discharge center point, or the water landing point exists. Identify and visualize by image processing. With this visualization, it is possible to easily grasp the water discharge status (the difference between the water discharge target position arbitrarily set by the operator and the current landing position, etc.), and further, the amount of the deviation on the image is calculated. This made it possible to automatically correct the turning angle or depression / elevation angle of the water cannon. According to the embodiment of the present invention, in the event of a fire in a large-scale space, it is not necessary for a human being to be stationed in the vicinity of the water cannon to operate the water cannon, and safety is improved.

Further, in a large-scale or special disaster, various automatic fire extinguishing robots have been proposed in consideration of the safety of firefighters, and the embodiment of the present invention can be applied to these robots as well.

Next, the principle of estimating the water discharge trajectory will be described.

Comparing the thermal images showing the discharge locus over time, it is possible to confirm a relatively high temperature dispersion around the locus and in the place where the water discharge falls. Therefore, for each pixel of a plurality of thermal images, the water discharge locus (pixel group that seems to correspond to the water discharge locus) can be identified by dividing (binarizing) the standard deviation of the temperature change with a preset threshold value. .. However, since there is also a pixel group (noise) that does not actually correspond to the water discharge locus in the pixel group, a unique noise process is provided in the image processing process to accurately extract only the water discharge locus. The thermal image acquisition conditions (frame rate, number of acquired images) are also important for the above identification, and there are optimum values for analysis.

By the above method, an object (rain cloud, fire hose, puddle, etc.) that emits the same amount of radiation as the discharged water can be deleted as a noise component, and only the discharge trajectory can be accurately extracted. After the water discharge locus is identified on the thermal image, the water discharge center point is calculated using an approximate curve, and if there is a deviation from the water discharge target position set by the operator with GUI etc., the amount of deviation from the camera performance or shooting position Is calculated, and the turning angle or depression / elevation angle of the water cannon is automatically corrected.

By repeating the process from acquiring the infrared thermal image to correcting the water cannon rotation and depression / elevation angle with respect to the water discharge target position arbitrarily set by the operator, even if the environment such as wind direction and speed changes during water discharge, each time , The turning angle or depression / elevation angle of the water cannon is corrected, and the water can be continuously discharged to the water discharge target position desired by the operator.

As described above, the embodiment of the present invention is characterized in that the water discharge locus and the water discharge center point are identified and controlled from a plurality of thermal image data.

In addition, by drawing the water discharge locus and the water discharge center point on the GUI and making it possible to set the water discharge target position on the GUI, the rotation angle or depression / elevation angle of the water cannon can be corrected and controlled visually. It is characterized by being able to do it.

<System configuration>
Hereinafter, the fire extinguishing system according to the embodiment of the present invention will be described.

As shown in FIG. 1, the fire extinguishing system 100 according to the embodiment of the present invention includes a fire fighting robot control device 10, a flying reconnaissance robot 60, a traveling reconnaissance robot 70, and a water cannon robot 80. There is. The flight-type reconnaissance robot 60 and the traveling-type reconnaissance robot 70 are connected to the fire-fighting robot control device 10 by wireless communication. The water cannon robot 80 is connected to the fire fighting robot control device 10 by wire or wireless communication.

The flight-type reconnaissance robot 60 flies in response to a command from the fire-fighting robot control device 10, and controls the fire-fighting robot by controlling the thermal image taken by the mounted thermal image camera 60A and the position information of its own device measured by GPS or the like. It is transmitted to the device 10.

The traveling reconnaissance robot 70 travels on the ground in response to a command from the firefighting robot control device 10, and fire-fights the thermal image taken by the mounted thermal image camera 70A and the position information of its own device measured by GPS or the like. It is transmitted to the robot control device 10.

The water cannon robot 80 includes a water cannon 80A that radiates water or foam fire extinguishing agent sent from a pump truck. The water cannon robot 80 controls water discharge by the water cannon 80A in accordance with a command from the fire fighting robot control device 10. Specifically, according to a command from the fire-fighting robot control device 10, the depression / elevation angle and the turning angle of the water cannon 80A are controlled, and the amount of water discharged from the water cannon 80A is controlled. Further, the water cannon robot 80 uses the thermal image taken by the mounted thermal image camera 80B, the sensor information detected by the mounted wind sensor (not shown), and the position information of the own device measured by GPS or the like as the fire fighting robot. It is transmitted to the control device 10.

As the thermal image cameras 60A, 70A, 80B, for example, an infrared camera may be used.

<Configuration of firefighting robot control device>
As shown in FIG. 2, the fire robot control device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and communication. It has an interface (I / F) 17. The configurations are connected to each other via a bus 19 so as to be communicable with each other.

The CPU 11 is a central arithmetic processing unit that executes various programs and controls each part. That is, the CPU 11 reads the program from the ROM 12 or the storage 14, and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above configurations and performs various arithmetic processes according to the program stored in the ROM 12 or the storage 14. In the present embodiment, the ROM 12 or the storage 14 stores a water discharge locus estimation program for estimating the water discharge locus. The discharge trajectory estimation program may be one program, or may be a program group composed of a plurality of programs or modules.

The ROM 12 stores various programs and various data. The RAM 13 temporarily stores a program or data as a work area. The storage 14 is composed of an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used for performing various inputs.

The display unit 16 is, for example, a liquid crystal display and displays various types of information. The display unit 16 may adopt a touch panel method and function as an input unit 15.

The communication interface 17 is an interface for communicating with other devices, and for example, standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark) are used.

Next, the functional configuration of the fire-fighting robot control device 10 will be described. FIG. 3 is a block diagram showing an example of the functional configuration of the fire-fighting robot control device 10.

Functionally, as shown in FIG. 3, the fire-fighting robot control device 10 includes a pixel extraction unit 20, a contour extraction unit 22, a water discharge center point calculation unit 24, a water discharge target position setting unit 26, and a water discharge control unit 28. ing. The pixel extraction unit 20 is an example of a pixel determination unit.

The pixel extraction unit 20 represents water or foam fire extinguishing agent (hereinafter, water discharge portion) radiated for each pixel from a plurality of thermal images in which fire extinguishing activities are photographed in time series by a thermal image camera 60A or 70A or 80B. Judge whether or not.

Specifically, an arbitrary number of thermal images obtained by photographing the fire extinguishing activity at an arbitrary time interval using a thermal image camera 60A or 70A or 80B are acquired. However, the acquisition time interval and the number of acquisitions are important parameters for identifying whether or not the water discharge portion is represented. Although appropriate values for these parameters vary depending on the shape and flow rate of the water cannon nozzle and the radiation environment (wind direction and speed, etc.), the constant flow nozzle with a radiation flow rate of about 400 to 6000 [L / min] can be identified by the following values. It was clarified experimentally that it would be possible. This nozzle is a nozzle that is generally used in fire extinguishing / fire prevention equipment and disaster prevention equipment. Further, it is possible to identify nozzles having a flow rate other than the relevant value or nozzles having different shapes by adjusting the following values.

Thermal image acquisition interval: 0.1 [fps] to 9 [fps]
Number of extractions: 5 to 15

Then, from the plurality of thermal images, it is determined for each pixel whether or not it corresponds to the water discharge portion, and a pixel group representing the water discharge portion is extracted. If the thermal images showing the actually obtained water discharge portion are compared over time, a peculiar temperature dispersion can be confirmed around the water discharge portion. Therefore, by analyzing the standard deviation of the temperature change (elapsed time) for each pixel, setting a threshold value for the standard deviation value, and binarizing each pixel, a pixel group representing the water discharge portion can be extracted.

Further, the above threshold value is an important parameter in the binarization of pixels, and although an appropriate value varies depending on the environmental conditions as in the thermal image acquisition condition, the following values are used in the thermal image acquisition condition. It was found that the identifiability is high. An air temperature sensor or a sensor for measuring the temperature of water discharged to the water cannon may be provided, and the above threshold value may be changed according to the value (for example, when the difference between the water temperature and the air temperature is large). , Raise the threshold, etc.). This enables more accurate identification.

Standard deviation lower threshold: 0.5
Standard deviation upper threshold: 4.0

Therefore, by determining the pixels having a standard deviation of 0.5 or more and 4.0 or less as the pixels represented by the water discharge portion, the pixel group representing the water discharge portion is extracted.

Furthermore, since the temperature fluctuation caused by the flame is large in an actual fire site, etc., by setting the temperature threshold value for each pixel of the thermal image as follows, only the pixel group representing the water discharge portion can be accurately extracted. do.

Lower temperature threshold: 0 [° C]
Temperature upper threshold: 100 [° C]

Therefore, a pixel having a temperature of 0 [° C.] or more and 100 [° C.] or less and a standard deviation of 0.5 or more and 4.0 or less is determined as a pixel represented by the water discharge portion, thereby determining the water discharge portion. The pixel group to be represented is extracted.

By this method, objects (rain clouds, fire hoses, puddles, etc.) that emit the same amount of radiation as the discharged water are deleted as noise components, and only the discharge trajectory can be accurately extracted.

The contour extraction unit 22 extracts a pixel group representing the contour portion for each of the two contour portions of the water discharge region representing the water discharge portion from a pixel group consisting of pixels determined to represent the water discharge portion. Then, for each of the two contour portions of the water discharge region, an approximate curve corresponding to the pixel group representing the contour portion is obtained, and an image representing the obtained approximate curve for each of the two contour portions of the water discharge region is output. do.

Specifically, the binarized image (expressed in white and black for convenience) obtained by the pixel extraction unit 20 includes pixels that are noise components in addition to the pixels that correspond to the water discharge portion. Therefore, the obtained binarized image is subjected to image processing such as noise removal, and only the pixel group corresponding to the water discharge portion is accurately extracted. Further, regression analysis is performed on the extracted pixel group, and an approximate curve corresponding to the contour portion of the water discharge region is calculated. However, the analysis method of the image recognition algorithm differs depending on the shooting direction of the water discharge portion as follows.

Shooting direction 1: When shooting the water discharge part from the side (in a thermal image, water moves from the right to the left of the screen or from the left to the right of the screen)
Shooting direction 2: When shooting the water discharge part from the front or back (in a thermal image, water moves from the front to the back or from the back to the front)

The water discharge center point calculation unit 24 calculates the water discharge center point based on the intersection of the horizontal line designated on the output image and the approximate curve, and displays the water discharge center point on the image.

The water discharge target position setting unit 26 receives the water discharge target position on the horizon designated on the output image.

The water discharge control unit 28 controls a vertical angle (hereinafter, depression / elevation angle) and a rotation angle (hereinafter, turning angle) as the water discharge angle by the water cannon 80A based on the difference between the water discharge target position and the water discharge center point. Therefore, an angle signal is output to the water cannon robot 80. As a result, the water cannon robot 80 controls the angle of the water cannon 80A in response to the angle signal. It should be noted that the angle type controlled by the shooting direction of the water discharge part is different, and the depression / elevation angle is controlled in the thermal image of the water discharge part taken from the side, and the turning angle is controlled in the thermal image of the water discharge part taken from the back. do.

<Operation of fire extinguishing system>
Next, the operation of the fire extinguishing system 100 according to the embodiment of the present invention will be described. First, when a fire breaks out, the thermal image camera 60A of the flying reconnaissance robot 60, the thermal image camera 70A of the traveling reconnaissance robot 70, or the thermal image camera 80B of the water cannon robot is used to remove the water discharged portion by the water cannon 80A. The thermal images represented are taken in chronological order and transmitted to the fire robot control device 10. At this time, the fire-fighting robot control device 10 executes the water discharge trajectory estimation processing routine shown in FIG. It should be noted that whether the shooting direction of the water discharge portion is side shooting or back shooting is automatically identified. For example, the shooting direction of the water discharge portion can be automatically identified from the position coordinates (water discharge direction) of the water discharge target and the water cannon robot, the position coordinates of the reconnaissance robot (thermal image camera), and the orientation of the thermal image camera. ..

In step S100, the pixel extraction unit 20 acquires an arbitrary number of thermal images obtained by photographing the fire extinguishing activity at an arbitrary time interval using the thermal image camera 60A or 70A or 80B.

In step S102, the pixel extraction unit 20 calculates the standard deviation of the temperature represented by the pixel for each pixel from the plurality of thermal images. In step S104, the pixel extraction unit 20 determines whether or not the pixel corresponds to the water discharge portion by using the threshold value of the standard deviation of the temperature and the threshold value of the temperature for each pixel, and the pixel representing the water discharge portion. The group is extracted and the image is binarized.

In step S106, it is determined whether the shooting direction of the designated water discharge portion is side shooting or back shooting. If the shooting direction of the designated water discharge portion is side shooting, the process proceeds to step S108, while if the shooting direction of the designated water discharge portion is rear shooting, the process proceeds to step S124.

In step S108, the contour extraction unit 22 performs image processing such as noise removal on the binarized image obtained in step S104, and extracts only the pixels corresponding to the contour of the water discharge region. Specifically, the first noise removal and the second noise removal described below are performed.

In the first noise removal, pixels are scanned for each row of the binarized image (see FIG. 5). Pixels are scanned to the left with the rightmost pixel as the starting point in each row, and when five or more black pixels (pixels with a pixel value = 1) are continuously present, they are considered as white pixels (pixels with a pixel value = 0). Only adjacent black pixels are extracted. For the binarized image in the figure, the place where the numerical value is "0" is "white" and the place where the numerical value is "1" is "black" so that the binarization can be visually understood. The same expression method is used for the figures described below.

Further, pixel scanning is also performed to the right side starting from the leftmost pixel, and black pixels are extracted in the same manner. In the case of the example of FIG. 5, according to this rule, the pixels marked with “x” are extracted. However, if the conditions of Rule A or B below are met, the rules shall be followed.

Rule A: When the rightmost (leftmost) pixel is a black pixel and five or more black pixels including the pixel are continuous, only the rightmost (leftmost) pixel is extracted.

Rule B: When all the lines to be scanned are black pixels, only the rightmost (leftmost) pixel is extracted.

Based on the above conditions, all lines of the binarized image are scanned to extract pixels. Here, an image of binarized data representing the pixels extracted from the binarized image is shown in FIG. The black pixels in the figure are the pixels extracted by the first noise removal.

In the second noise removal, the binarized data obtained through the first noise removal is scanned again for each row of pixels. In this scanning, pixel scanning is performed on the left side starting from the rightmost pixel, and the first black pixel found is extracted. Further, pixel scanning is also performed to the right side starting from the leftmost pixel, and black pixels are extracted in the same manner. In the case of the example of FIG. 7, according to this rule, the pixels marked with “x” are extracted. However, if the conditions of the following rules are met, the rules shall be followed.

Rule: If the rightmost (leftmost) pixel is a black pixel, the rightmost (leftmost) pixel is extracted.

Based on the above conditions, all rows of binarized data are scanned to extract pixels. Here, an image of the binarized data extracted from the binarized data shown in FIG. 6 is shown in FIG. In the figure, the black pixels are the pixels extracted from the rightmost edge of the screen, and the gray pixels (the pixels of the dot portion in FIG. 8; the same applies to the subsequent drawings) are the pixels extracted by scanning from the rightmost edge of the screen.

Both color pixels are distinguished because they are used in the contour extraction of the upper part of the water discharge area, which will be described later.

In step S110, the contour extraction unit 22 performs contour extraction of the upper part of the water discharge region and contour extraction of the lower part of the water discharge region.

Specifically, as the contour extraction of the upper part of the water discharge region, the contour of the upper part of the water discharge locus is extracted based on the binarized data obtained in the above step S108.

In this extraction, the pixel existing in the highest row of the gray and black pixels is set as the origin, and the pixels of the same color existing in the surroundings are scanned to extract the pixels corresponding to the contour. As an example, in the dotted line portion of the binarized data in FIG. 8, the origin is set as shown in FIG. However, if the conditions of Rule A or B below are met, the rules shall be followed.

Rule A: If the gray and black pixels have the same origin, scanning is performed from each pixel.
Rule B: If the origin of a gray and black pixel is in the top row of the binarized data, the pixel is removed from the binarized data and the pixel is in the top row and in the middle column of the gray and black pixels. Is rewritten as the origin pixel, and scanning is performed from this pixel.

After determining the origin, scanning is performed on the gray and black pixels, respectively.

In the case of gray pixels, as shown in FIG. 10, refer to row Y-1 one row below the origin pixels (X, Y), and within the range of columns 0 to X + 40 from column 0 (left end of the image) of the row. Search for pixels of the same color. If the corresponding pixel exists, the pixel is reset as the origin pixel (X', Y), and the same search as above is repeatedly performed in the lower row based on the present coordinates. On the contrary, if it does not exist, the Y-2 row two rows below the origin pixel (X, Y) is referred to, and it is searched whether the same color pixel exists in the range from 0 column to X + 40 column of the row. Further, when the corresponding pixel does not exist, the Y-3 line three lines below is searched, and the Y-50 line three lines below is searched in order at the maximum. Here, the parts of the "40" column and the "50" row are parameter values, which are variables in the program (can be changed in the setting file). The process is terminated when the search is repeated and any of the following cases A to C is obtained.

Case A: When the corresponding pixel does not exist even if the search is performed up to the Y-50 line.
Case B: When the origin pixel (X', Y) reaches the leftmost column (X'= 0) of the binarized data.
Case C: When the origin pixel (X', Y) reaches the bottom row (Y'= 0) of the binarized data.

In the case of black pixels, scanning is performed in the same manner as in gray pixels, but as shown in FIG. 11, the scanning range in each row is different, and columns X-40 to 639 (range in the case of VGA) are defined as the scanning range. do. Here, the parts of the "40" column and the "639" column are parameter values and are parts that become variables in the program (can be changed in the setting file). The process is terminated when the scanning is repeated and any of the following cases A to C is obtained.

Case A: When the corresponding pixel does not exist even after scanning up to the Y-50 line.
Case B: When the origin pixel (X', Y') reaches the rightmost column of the binarized data.
Case C: When the origin pixel (X', Y') reaches the bottom row of the binarized data.

A point cloud image of the upper contour of the water discharge region extracted from the binarized data by the above process is shown in FIG.

Further, in the contour extraction of the lower part of the water discharge area, first, in the point cloud image (FIG. 12) of the upper contour of the water discharge area extracted by the upper contour extraction, a quadratic regression analysis is performed only on the black pixel portion. Is performed, and the approximation function f ₁ (x) is calculated as shown in the following equation (see FIG. 13). Here, this function is a reference function for searching the pixel corresponding to the contour of the lower part of the water discharge area. Each S in the equation is a covariance for the subscript.

Further, returning to the binarized data (see FIG. 6 above) obtained through the first noise removal, the pixels are scanned for each column to create the binarized data. In this scanning, as shown in FIG. 14, pixel scanning is performed upward from the pixel in the bottom row as a starting point, and the black pixel first found in each column is extracted. However, when the lowest pixel is a black pixel, the pixel is extracted. Here, an image of the binarized data extracted from the binarized data shown in FIG. 6 is shown in FIG.

Then, from the binarized data shown in FIG. 15, pixels satisfying any of the following conditions A and B are extracted. In this operation, as shown in the image shown in FIG. 16, black pixels existing in the range determined by _{the calculated approximation function f 1 (x) are extracted.}

Condition A: The black pixel exists in the column (within the range of x) of the _{approximation function f 1 (x).}

Condition B: For each column, based on _{the pixels existing in the value f 1} (X) (rounded to an integer) obtained by substituting an arbitrary X for the _{approximation function f 1 (x), f 1} It is a black pixel existing in a range satisfying (X) -220 <Y <f _{1 (X) -20.}

However, the parts "220" and "20" above are parameter values and are variable parts in the program (can be changed in the setting file).

A point cloud image of the lower contour of the water discharge region extracted from the binarized data (FIG. 15) by the above process is shown in FIG.

In step S112, the contour extraction unit 22 obtains the contour of the water discharge region based on the point cloud images (FIGS. 12 and 17) of the upper contour and the lower contour of the water discharge region obtained in step S110. Draw the approximate curve to be represented.

Here, regarding the contour of the water discharge region, both the upper part and the lower part are subjected to a third-order regression analysis based on the point cloud image, and an approximate curve is calculated and drawn as follows. However, it is assumed that the calculation of this approximate curve can be performed up to a 2nd to 5th order function, and can be arbitrarily changed by GUI.

Specifically, when drawing an approximate curve representing the upper contour of the water discharge area, the point group image (Fig. 12) of the upper contour is subjected to regression analysis by combining both gray pixels and black pixels to approximate it. formula

Is calculated. After that, x is substituted into the calculated approximate expression f ₂ (x) at equal intervals (Δx = 5, 0 ≦ x ≦ 639 _{) to calculate y = f 2} (x), and the approximate curve is obtained by heat. Draw on the image (see FIG. 18). However, the approximate curve is not drawn for the parts corresponding to the following conditions A to C.

Condition A: The _{part where f 2} (x) <0.
Condition B: When the water discharge direction is from right to left on the screen, it is the part on the right side of the rightmost column of the point cloud image (FIG. 12) of the outline of the upper part of the water discharge area.
Condition C: When the water discharge direction is from left to right on the screen, it is the part on the left side of the leftmost column of the point cloud image (FIG. 12) of the outline of the upper part of the water discharge area.

In addition, when drawing an approximate curve representing the contour of the lower part of the water discharge area, regression analysis is performed on the point cloud image (Fig. 12) of the lower contour, and the approximate expression is used.

Is calculated. Then, x is substituted into the calculated approximate expression f ₂ (x) at equal intervals (Δx = 5, 0 ≦ x ≦ 639 _{) to calculate y = f 3} (x), and the approximate curve is obtained as a thermal image. Draw above (see Figure 19). However, the approximate curve is not drawn for the portion corresponding to the following conditions A to D.

Condition A: The _{part where f 3} (x) <0.
Condition B: When the water discharge direction is from right to left on the screen, it is the part on the right side of the rightmost column of the point cloud image (FIG. 12) of the outline of the upper part of the water discharge area.
Condition C: When the water discharge direction is from left to right on the screen, it is the part on the left side of the leftmost column of the point cloud image (FIG. 12) of the outline of the upper part of the water discharge area.
Condition D: When the approximate curve related to the contour of the lower part of the water discharge area penetrates the approximate curve related to the contour of the upper part of the water discharge area from below and intersects, it is the part after the intersection (the point where it penetrates).

Finally, the approximate curve (Fig. 18) showing the contour of the upper part of the water discharge area and the approximate curve (Fig. 19) showing the contour of the lower part of the water discharge area are made into two contours of the water discharge area, and one sheet is put together. Drawing is performed on the thermal image of (Fig. 20). Further, although this result is displayed on the GUI, when the approximate curve calculated from the analysis is only the approximate curve of the upper contour, only the curve is displayed on the thermal image. On the contrary, when the calculated approximate curve is only the approximate curve of the lower contour, or when none of them is calculated, the fact that the analysis cannot be performed is displayed on the thermal image.

In step S114, the water discharge center point calculation unit 24 accepts the setting of the horizontal line on the output thermal image.

Specifically, first, as shown in FIG. 21, the horizontal line is set by the operator's operation on the composite image (FIG. 20) of the approximate curve representing the two contours of the water discharge region and the thermal image shown in the GUI. Accept. By setting the horizon, an intersection occurs between the approximate curve representing the lower contour of the water discharge region and the approximate curve representing the upper contour and the horizontal line, and the distance between the intersections becomes the width between the contours of the water discharge region on the horizon. Here, assuming that the x-coordinate of the intersection of the approximate curve representing the lower contour and the horizontal line is X _max and the x-coordinate of the intersection of the approximate curve representing the upper contour and the horizontal line is X _min , the widths of both are (X _max −X _min). ). However, if the following rules apply, the rules shall be followed.

Rule A: If the approximate curve representing the contour of the lower part is not drawn, the right edge of the screen of each line is set to X _min .
Rule B: If the intersection of the approximate curve representing the upper contour and the horizontal line cannot be calculated, the left edge of the screen of the line is set to _Xmax, and the intersection of the approximate curve representing the lower contour and the horizontal line cannot be calculated. _{Is X min} at the right end of the screen of the line.

In step S116, the water discharge center point calculation unit 24 calculates the water discharge center point based on the intersection of the horizontal line designated on the output image and the approximate curve, and displays the water discharge center point on the image.

Specifically, the center point is drawn according to the setting location (setting line) of the horizontal line. For example, it is assumed that the water discharge center point is located at a fixed ratio of the _{width (X max} −X _{min ) calculated above in any row, and is the product α (X max} −X _min ) of the width and the ratio α. Be expressed. Therefore, the coordinates of the intersection and the above width change according to the set position of the horizon, and the position of the water discharge center point also changes. However, α is a part that becomes a variable in the program as a parameter value, and can be changed according to the shape and type of the water cannon nozzle. In the semi-aspirate nozzle mounted on the water cannon robot this time, α was set to 0.73 as an actually measured value. Finally, the X coordinate X _M of the water discharge center point is calculated by any of the following formulas according to the water discharge direction.

When the water discharge direction is from right to left on the screen, the X coordinate X _M of the water discharge center point is calculated by the following formula.

When the water discharge direction is from left to right on the screen, the X coordinate X _M of the water discharge center point is calculated by the following formula.

The water discharge center point is calculated from the above, and the water discharge center point is shown along with the horizontal line arbitrarily set by the operator on the composite image (FIG. 20) of the approximate curve representing the contour of the water discharge area and the thermal image (FIG. 22). ..

In step S118, the water discharge target position setting unit 26 accepts the setting of the water discharge target position on the horizontal line designated on the output thermal image.

Specifically, the setting of the vertical line as shown in FIG. 23 is accepted on the display image (FIG. 22) of the approximate curve representing the outline of the discharge region and the discharge center point shown in the GUI. The main line can be moved arbitrarily in the left and right column directions, and the water discharge target position is set as the intersection with the horizon. However, the water discharge center position and the water discharge target position are set to the same line on the horizon, and both points move according to the (up and down) movement of the line.

Here, the water discharge target position is the water discharge position that the operator wants to change (target) with respect to the current water discharge center position (indicated by the dotted line "x" on the image), and the main target position and the water discharge center point. The control direction (plus, minus) of the depression / elevation angle of the water cannon 80A is determined based on the positional relationship. Therefore, the control direction is the correction direction (plus, minus) of the water cannon depression / elevation angle for discharging water to the water discharge target position with reference to the depression / elevation angle of the current water cannon 80A (at the time when the thermal image is acquired).

In step S120, the water discharge control unit 28 acquires the current depression / elevation angle of the water cannon 80A of the water cannon robot 80.

Then, in step S122, the water discharge control unit 28 controls the depression / elevation angle as the water discharge angle by the water cannon 80A based on the difference between the water discharge target position and the water discharge center point and the current depression / elevation angle. A signal is output to the water cannon robot 80.

Specifically, the control direction (plus, minus) of the depression / elevation angle of the water cannon 80A is the water discharge direction (right to left, left to right) on the above screen, and the setting location of the water discharge target position with respect to the water discharge center position. It is also determined according to the depression / elevation angle θ of the acquired water cannon 80A. For example, as shown in Table 1, the control direction of the depression / elevation angle of the water cannon 80A is determined.

For example, in the case of FIG. 23, assuming that the water discharge direction is "right to left", the water discharge target position is set to the "right side" with respect to the water discharge center position, and the current water cannon depression / elevation angle θ is 35 °, it is "plus". Control direction is determined. However, as described in the table, under some conditions where the depression / elevation angle θ is 30 ° or more and 33 ° or less, the control direction is not output and the depression / elevation angle cannot be corrected because it is out of the water discharge range. ..

Then, after the control direction is determined as described above, the output value is output as the initial value of the correction angle of 2 °. As an example, when the correction direction is "minus", the value (θ-2 °) obtained by adding "-2 °" to the current water cannon depression / elevation angle θ is output. However, the part with "2 °" is a parameter value and is a part that becomes a variable in the program (it can be changed in the setting file). Then, by repeating steps S100 to S122 at regular intervals while memorizing the water discharge target position set in step 118, the water discharge gun depression / elevation angle is controlled so that the water discharge center position approaches the water discharge target position. The case where the specified angle is repeatedly corrected has been described as an example, but the present invention is not limited to this, and is based on the distance between the water discharge locus and the thermal image camera, the angle of view of the thermal image camera, and the resolution. , The correction angle may be calculated as an absolute value.

On the other hand, in step S124, the current turning angle of the water cannon 80A of the water cannon robot 80 is acquired.

In step S126, the contour extraction unit 22 performs image processing such as noise removal on the binarized image obtained in step S104, and extracts only the pixels corresponding to the contour of the water discharge region. An example of the thermal image and binarized data used in the analysis is shown in FIG. 24A.

In the binarized data shown on the right side of FIG. 24A, in addition to the pixels corresponding to the contour of the water discharge region, the pixels serving as noise components are included. Therefore, noise is removed from the obtained binarized image, and only the pixels corresponding to the contour are extracted. The underlined part is the parameter value, which is a variable part in the program (it can be changed in the setting file).

Specifically, the third noise removal and the fourth noise removal described below are performed.

The third noise removal is carried out according to the same rules as the first noise removal in step S108. Here, an image of the binarized data extracted from the binarized data shown on the right side of FIG. 24A is shown in FIG. 24B. The black pixels in the figure are the pixels extracted by the third noise removal.

In the fourth noise removal, the binarized data obtained through the third noise removal is scanned again for each row of pixels. In this scanning, pixel scanning is performed on the left side starting from the rightmost pixel, and the second black pixel found is extracted. Further, pixel scanning is also performed to the right side starting from the leftmost pixel, and black pixels are extracted in the same manner. In the case of the example of FIG. 25, according to this rule, the pixels marked with "x" are extracted. However, if the conditions of the following rules are met, the rules shall be followed.

Rule: When the rightmost (leftmost) pixel is a black pixel, the rightmost (leftmost) pixel is the black pixel found first.

The pixels are extracted by scanning the entire line of the binarized data as described above. Here, an image of the binarized data extracted from the binarized data shown in FIG. 24B is shown in FIG. 26. In the figure, black pixels are pixels extracted from the rightmost edge of the screen, and gray pixels are pixels extracted from the rightmost edge of the screen. Both color pixels are distinguished because they are used in drawing the outline of the water discharge area.

In step S128, the contour extraction unit 22 extracts two contours of the water discharge region, and in step S130, draws an approximate curve representing the two contours of the water discharge region. Specifically, based on the gray and black point group images obtained by removing noise from the binarized image in step S126, an approximate curve is extracted as two contours of the water discharge region, and the water discharge region Draw an approximate curve that represents the two contours. Here, when drawing the approximate curve, there is a case where the contour of the water discharge region cannot be expressed by a higher-order function related to x (when two or more solutions occur in x), and binarization is performed so as to deal with such a case. Binarized data is created by reversing only the coordinates of each pixel of the data (see FIG. 27).

The contour extraction unit 22 performs a quadratic regression analysis on each of the gray pixel and black pixel point clouds with respect to the obtained binarized data, and performs an approximation formula.

Is calculated. The regression analysis and the calculation of the approximate expression are performed only on the data whose y-coordinate is included in the designated range. This is to reduce the influence of the vicinity of the ground (lower part of the screen) where noise is likely to occur. Further, the specified range of the y coordinate is a part that becomes a variable in the program, and the initial value is 120 ≦ y ≦ 479 (it can be changed in the setting file). After that, x is substituted into the approximate expression f ₄ (x) calculated above at equal intervals (Δx = 5, 0 ≦ x ≦ 479 _{) to calculate y = f 4} (x), and the approximate curve is obtained. Draw (see Figure 28). However, since the pixel coordinates are reversed, the substitution range of x is 0 to 479. It should be noted that the calculation of this approximate curve can be performed up to a quadratic function, and can be arbitrarily changed by GUI.

When the approximate curve of black pixels and the approximate curve of gray pixels intersect within the range of 320 <x <479 columns, the curves from the intersection (X, Y) to the right column and thereafter (X <x <479) Do not draw.

Finally, using the approximate curve (FIG. 28) obtained above as the contour of the water discharge region, data is created by reversing _{each pixel coordinate of the binarized data and the contour y = f 4 (x) of the water discharge region.} Then, drawing is performed on one thermal image (FIG. 29). This result is displayed on the GUI, but when there is only one approximate curve calculated from the analysis, only the approximate curve is displayed on the thermal image. Further, if the approximate curve is not calculated, the fact that the analysis cannot be performed is displayed on the thermal image.

In step S132, the water discharge center point calculation unit 24 receives the setting of the horizontal line on the output thermal image as in step S114.

In step S134, the water discharge center point calculation unit 24 calculates the water discharge center point based on the intersection of the horizontal line designated on the output image and the approximate curve, and displays the water discharge center point on the image.

Specifically, first, of the two approximate curves drawn in step S130, one corresponds to the water discharge trajectory from the water cannon 80A to the maximum firing height (hereinafter, the water cannon side), and the other corresponds to the maximum firing. It corresponds to the water discharge trajectory (hereinafter referred to as the water landing side) from the high to the water discharge center point. Here, in the analysis of the water discharge center point, it is necessary to identify which of the two corresponds.

As shown in FIG. 30, the thermal image camera 80B that captures the water discharge from the back is linked with the water cannon 80A, and the water discharge locus closest to the water cannon 80A is the position corresponding to the rotation angle (turning angle) of the water cannon in the thermal image. Exists in. Taking advantage of this feature, the reference location (coordinates) corresponding to the numerical value of the water cannon rotation angle (turning angle) is set in advance in a table file or the like, and the water discharge direction is identified based on the angle information. Specifically, of the two approximate curves, the one with the shorter pixel distance in the reference coordinates (X, Y) and the Y row is identified as the "water cannon side", and the farther one is identified as the "water landing side". As an example, assuming that the reference coordinates are the positions shown in FIG. 31, among the pixels corresponding to the two approximate curves, the pixels close to the reference coordinates are black pixels, and the approximate curve including the pixels is set as "water cannon side". .. If the distances to both are the same, the same judgment shall be made on the Y + 1 line, which is one line above the reference coordinates. If they are further equal, the same judgment is made on Y + 2 lines, and judgments are made in order up to Y + 50 lines, which are 50 lines above.

Then, the water discharge center point is analyzed based on the approximate curve representing the contour of the water discharge region obtained in step S130. Here, on the composite image of the water discharge locus and the thermal image shown in the GUI, when the horizontal line is set as shown in FIG. 32, the center point is drawn according to the setting location (setting line) of the main line. Further, as shown in the figure, regarding the water discharge trajectory on the landing side, the portion descending from the horizontal line is represented by a dotted line according to the set position of the horizontal line.

Here, depending on the setting of the horizon, an intersection occurs between the "water discharge gun side" discharge locus and the "landing side" water discharge locus and the line, and the point where the x-coordinate is large is X _max and the point where the x-coordinate is small is X. _Assuming that it is _min , the X coordinate X M of the water discharge center point is calculated by one of the following two equations. However, the water discharge locus width L _h is a value set separately for the water cannon rotation angle (turning angle), and is determined based on the referenced angle information.

When X _max is the intersection of the “landing side” water discharge locus and the horizon (see FIG. 32 above), the X coordinate X _M of the water discharge center point is calculated by the following formula.

When X _max is the intersection of the “water cannon side” water discharge locus and the horizon (see FIG. 33), the X coordinate X _M of the water discharge center point is calculated by the following equation.

The water discharge center point is calculated from the above, and the water discharge center point is shown along with a horizontal line arbitrarily set by the operator on the composite image of the approximate curve representing the contour of the water discharge region and the thermal image (FIGS. 32 and 33).

In step S136, the water discharge target position setting unit 26 receives the setting of the water discharge target position on the horizontal line designated on the output thermal image.

Specifically, the setting of the vertical line as shown in FIG. 34 is accepted on the display image of the approximate curve representing the outline of the water discharge area and the water discharge center point shown in the GUI. The main line can be moved arbitrarily in the left and right column directions, and the water discharge target position is set as the intersection with the horizon. However, the water discharge center position and the water discharge target position are set to the same line on the horizon, and both points move according to the (up and down) movement of the line.

Here, the water discharge target position is the water discharge position that the operator wants to change (target) with respect to the current water discharge center position (indicated by gray letters "x" in the image), and the main target position and the water discharge center point. The control direction (plus, minus) of the turning angle of the water cannon 80A is determined based on the positional relationship. Therefore, the control direction is the correction direction (plus, minus) of the water cannon turning angle for discharging water to the water discharge target position with reference to the current turning angle of the water cannon 80A (at the time when the thermal image is acquired).

Then, in step S138, the water discharge control unit 28 controls the turning angle as the water discharge angle by the water cannon 80A based on the difference between the water discharge target position and the water discharge center point and the current turning angle. A signal is output to the water cannon robot 80.

Specifically, the control direction (plus, minus) of the turning angle of the water cannon 80A is determined according to the setting location of the water discharge target position with respect to the water discharge center position. For example, as shown in Table 2, the control direction of the turning angle of the water cannon 80A is determined.

最終的に上記で制御方向が決定された後、修正角度の初期値２°として、制御値を出力する。例として修正方向が「プラス」であった場合、「＋２°」を現在の放水砲旋回角θに加えた値（θ＋２°）を出力する。ただし、「２°」との部分はパラメータ値であり、プログラム上の変数となる部分である。そして、ステップ１３６にて設定した放水目標位置を記憶したままステップＳ１００からステップＳ１０６、およびステップＳ１２４からステップＳ１３８までを一定時間毎に繰り返すことにより、放水中心位置が放水目標位置へ近づくよう放水砲旋回角が制御される。なお、定められた角度の修正を行う場合を例に説明したが、これに限定されるものではなく、放水軌跡と熱画像カメラ間の距離や、熱画像カメラの画角、解像度を基に、修正角度を絶対値として算出してもよい。

After the control direction is finally determined in the above, the control value is output as the initial value of the correction angle of 2 °. As an example, when the correction direction is "plus", the value (θ + 2 °) obtained by adding "+ 2 °" to the current water cannon turning angle θ is output. However, the part of "2 °" is a parameter value and is a part that becomes a variable in the program. Then, by repeating steps S100 to S106 and steps S124 to S138 at regular intervals while memorizing the water discharge target position set in step 136, the water cannon turns so that the water discharge center position approaches the water discharge target position. The angle is controlled. The case where the specified angle is corrected has been described as an example, but the present invention is not limited to this, and is based on the distance between the water discharge locus and the thermal image camera, the angle of view of the thermal image camera, and the resolution. The correction angle may be calculated as an absolute value.

As described above, according to the fire extinguishing system according to the embodiment of the present invention, from a pixel group consisting of pixels determined to represent a water discharge portion based on a plurality of thermal images taken in time series, For each of the two contour portions of the water discharge region, a pixel group representing the contour portion is extracted, and an approximate curve corresponding to the pixel group representing the contour portion is obtained to accurately estimate the water discharge trajectory on the image. be able to.

In addition, even if the environment such as wind direction and speed changes during water discharge by repeating the process from acquiring the thermal image to correcting the water cannon turning angle and depression / elevation angle with respect to the water discharge target position arbitrarily set by the operator. Each time, the turning angle and depression / elevation angle of the water cannon are corrected, and the water can be continuously discharged to the water discharge target position desired by the operator.

<Modification example>
The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

For example, in the above embodiment, the case where the water cannon is controlled with respect to the water discharge target position arbitrarily set by the operator has been described as an example, but the present invention is not limited to this. For example, when a flame exists within the monitoring range of the thermal image camera, the position of the flame may be identified from the thermal image and set as the water discharge target position. As a result, when a flame occurs within the monitoring range of the thermal image camera, the water discharge target position is automatically set, and the water can be accurately discharged to the flame by controlling the turning angle and depression / elevation angle of the water cannon.

Furthermore, when extinguishing a fire in a storage tank by radiating a foam fire extinguishing agent, the center of the flame changes over time as the flame shrinks due to the coating on the combustion oil level, so the position is measured with an infrared camera. Efficient fire extinguishing is possible by identifying and correcting the water discharge center position each time and controlling the angle of the water cannon accordingly.

Further, the case where the thermal image is acquired from any one of the thermal image cameras 60A, 70A, and 80B to control the water cannon has been described as an example, but the present invention is not limited to this. For example, a thermal image is acquired from the thermal image camera 60A of the flying reconnaissance robot 60, and a thermal image is acquired from the thermal image camera 70A of the ground reconnaissance robot 70, and both thermal images are used for controlling the water cannon. You may try to do it. Specifically, a thermal image of the side surface of the water discharge is acquired by the thermal image camera 70A of the ground reconnaissance robot 70, and a thermal image of the back surface of the water discharge is acquired by the thermal image camera 60A of the flight reconnaissance robot 60. Therefore, the turning angle and the elevation angle of the water discharge gun may be controlled at the same time. In addition, a thermal image is also acquired from the thermal image camera 80B of the water cannon robot 80, and the operator arbitrarily selects two thermal images from the thermal images of the thermal image cameras 60A, 70A, and 80B of the water cannon. Control may be performed. As a result, even if one thermal image camera cannot capture the water discharge trajectory well due to an obstacle or the like, the water cannon can be stably controlled by using another thermal image camera.

Further, from a pixel group consisting of pixels determined to represent a water discharge portion based on a plurality of thermal images taken in time series, one contour portion of the water discharge region is represented by the one contour portion. A pixel group may be extracted to obtain an approximate curve corresponding to the pixel group representing the one contour portion. For example, when the water discharge is photographed from a close distance or only the upper side of the water discharge is photographed, only one contour portion of the water discharge area is projected (only the upper trajectory). A pixel group representing the portion may be extracted, and an approximate curve corresponding to the pixel group representing the one contour portion may be obtained.

In addition, the water landing point may be estimated instead of the water discharge center point.

10 Fire-fighting robot control device 15 Input unit 16 Display unit 20 Pixel extraction unit 22 Contour extraction unit 24 Water discharge center point calculation unit 26 Water discharge target position setting unit 28 Water discharge control unit 60 Flying reconnaissance robot 60A Thermal image camera 70 Traveling reconnaissance robot 70A Thermal image camera 80 Water cannon robot 80A Water cannon 80B Thermal image camera 100 Fire extinguishing system

Claims (5)

With a water cannon,
A thermal image camera that captures the water discharged by the water cannon,
From the photographed thermal image, the trajectory of the water discharge and the position of the center of the water discharge or the landing point are analyzed.
A water discharge trajectory estimation device that controls the water discharge angle of the water cannon according to the result of the analysis, and
Fire extinguishing system including. The water discharge trajectory estimation device is
A pixel determination unit that determines whether or not a water discharge portion is represented for each pixel from a plurality of thermal images having pixel values according to temperature obtained by the thermal image camera.
From the pixel group consisting of pixels determined to represent the water discharge portion, a pixel group representing the contour portion is extracted for each of one or two contour portions of the water discharge region.
For each of the contour portions of the water discharge region, an approximate curve corresponding to a pixel group representing the contour portion was obtained.
A contour extraction unit that estimates the trajectory of water discharge on the thermal image from the approximate curve obtained for each of the contour portions of the water discharge region.
The fire extinguishing system according to claim 1. The water discharge trajectory estimation device is
A display unit that displays a display image in which the locus of water discharge on the estimated thermal image is superimposed on the thermal image, and a display unit.
A water discharge center point calculation unit that calculates a water discharge center point based on the intersection of the horizontal line designated on the display image and the approximate curve and displays it on the display image, and a water discharge center point calculation unit.
A water discharge target position setting unit that accepts the water discharge target position on the display image,
The fire extinguishing system according to claim 2, further comprising. The water discharge trajectory estimation device is
The fire extinguishing system according to claim 3, further comprising a water discharge control unit that controls the angle of water discharge by the water cannon based on the difference between the water discharge target position and the water discharge center point. Claims 2 to claim that the pixel determination unit determines whether or not a water discharge portion is represented for each pixel based on the temperature of the pixels or the dispersion of the temperatures in the plurality of thermal images taken in time series. Item 4. The fire extinguishing system according to any one of items 4.

Images